Archived - Evaluation of the Government of Canada Program for International Polar Year

Immediate outcomes:

IPY has led to higher levels of engagement of Aboriginal people and Northerners in northern scientific research - for the first time in many instances. Almost all of the scientific projects engaged Northerners and northern communities to some extent. IPY projects engaged communities, as research assistants and guides, in logistics, in leading to employment and economic and educational benefits, and in helping to incorporate traditional knowledge in many IPY projects. Communications and outreach activities involved approximately 2,000 northern and Aboriginal adults, Elders and especially youth in the classroom and in community activities such as Polar Days, films and presentations. Although many IPY projects were characterized as a "gold standard" in northern engagement, and there was widespread recognition that researchers have a much better understanding of the importance of collaboration, many key informants, especially Northerners, said that engagement and collaboration remain a challenge. IPY program activities have led to improved research infrastructure and employment for Northerners.

The pool of science students and potential next generation researchers expanded greatly through the practical learning experiences for youth provided through IPY. IPY research has produced a huge body of valuable work that is only beginning to have an impact on policy, on the actions of governments and others, and on future research. Canada has been a leader in the management and dissemination of data and research results produced through IPY.

Intermediate outcomes:

IPY-funded research has significantly increased the amount and availability of high quality data and scientific publications on the Canadian North. IPY has also led to higher levels of broad-based public awareness and knowledge about climate change and the health and well-being of Northerners through a wide range of project-specific and more general IPY communications activities. Through IPY, research partners and organizations generally have a better understanding of conducting scientific research in the North. It was widely acknowledged that inadequate feedback to communities about the results of research has sometimes been a problem. The projects that were the most effective at public communications were ones involving community partnerships.

Long-term outcomes:

It is widely agreed that IPY has increased Canada's international profile as a leader in northern scientific research, particularly in the areas of climate change and the health and well-being of Northerners. Research results generated through the Canadian IPY Program have contributed to other international research programs. Improving contacts and organizational networks among Canadian and international scientists are major successes of IPY. As a legacy of the IPY Program, the Canadian High Arctic Research Station (CHARS) is a signature deliverable of Canada’s Northern Strategy and was announced in 2008. In addition, many key informants believe that, even at this relatively early stage, IPY results are influencing government policy and decision making. Some others believe that it is too early to know.

1.2.1 Background and Description

1.2.2 Program Objectives/Activities and Expected Outcomes

A key objective for the Government of Canada Program for IPY has been to support a targeted science and research program focused on two of Canada’s most important challenges for its northern regions – climate change impacts and adaptation, and the health and well-being of northern communities. This is directly linked to AANDC’s 2010-2011 Program Activity Architecture Strategic Outcome - The North and more specifically the Program Activity – Northern Land and Resources and the Sub-Activity – Northern Science. One of the Government’s aims with the Government of Canada Program for IPY is to leave a positive, lasting legacy for Northerners and Aboriginal communities. Canada’s prominent role in IPY activities is to position the country as an international leader in Arctic Science. Canada’s IPY initiatives aim to increase the capacity of Canadians, and in particular Northerners, to engage in, direct, undertake and use science to build a strong economy, to contribute in the demonstration of Canadian sovereignty, to conserve and manage the environment and natural resources, and to ensure their health and the health of their communities. Part of the Government’s investment went towards engaging northern communities in research activities, capacity building and training opportunities.

The Government of Canada Program for IPY is composed of six components.

1. Canadian Science:
   
   The objective of Canada's IPY science program was to generate data and knowledge on polar regions—especially information on Arctic environments and communities—and to support science that is policy-relevant and useful at all levels of government and society, both nationally and internationally. The Canadian science and research community (including federal departments and agencies and Canadian universities and non-governmental organizations) were invited to submit funding proposals that met the international program requirements for an IPY project and addressed Canada’s northern policy priorities pertaining to one or both major policy and science themes of the Government of Canada Program for IPY science and research themes (i.e., science for climate change impacts and adaptation, and health and well-being of northern communities).
2. Logistics for health and safety and emergency preparedness
   
   The objective of this component was to ensure the health and safety of researchers and northern communities during IPY and to prepare northern research infrastructure. Federal funds were intended for training and planning emergency preparedness and response for land and marine-based search and rescue, and security to handle the anticipated influx of Canadian and international scientists and observers to Canada’s North. Also, enhancements to facilities were provided for specific and special services, such as enhanced ice and weather information, helicopter and fixed wing aircraft, ship time, scientific equipment, fuel caches, communications, training, and necessary support staff.
3. Northern science capacity building
   
   The objective of this component was to develop the next generation of northern scientists and enhance capacity in the North to carry out polar science. The intent was to focus on providing capacity building, education and on-the-ground training of Northerners and Aboriginal people so that they could contribute to the IPY Federal Program and carry out strong northern research activities in the decades to follow. The intent was also to develop IPY Northern Coordination Offices in key northern locations, who would be positioned as points of contact on IPY matters for northern communities and researchers alike and help build northern capacity.
4. Communications and outreach
   
   The objective of this program component was to share information and results of IPY research, promote polar science, and foster a greater awareness of Canada's Arctic, its peoples and the importance of polar regions in a global context. The northern science communications and outreach community was invited to submit proposals for outreach and communications activities for IPY.
5. Data management and archiving initiatives
   
   The objective of this component was to ensure that IPY data and information is archived and made accessible, and to support data management through capacity building. This program component was intended to complement the data management elements of the Canadian Science program component.
6. Meeting Procedural Requirements (research licensing)
   
   The objective of this component was to provide support to governments, boards, communities and other organizations that review and issue research licenses and permits, and which require enhanced capacity to effectively handle the increased demands for regulatory approvals of IPY research activity between 2006-2007 and 2009-2010. In addition, the intention was to enable work to be carried out by research licensing and permitting bodies, in partnership with the IPY Federal Program Office and other stakeholders, to clarify, streamline and/or harmonize research licensing processes in the North, making them more efficient and effective for the benefit of communities and researchers alike, as part of the legacy of IPY.

1.2.3 Expected Outcomes

The Government of Canada Program for IPY was expected to generate significant economic, environmental and social benefits.

Some anticipated results included:

• New knowledge and information to support programs, decision making and development in the North;
• Leverage in research funding nationally and internationally, with the Government of Canada investment leveraging three to four times that amount of new research in Canada;
• High value, innovative research results, products and data and information that support Government of Canada policy initiatives;
• Direct immediate economic benefits in the North, and indirect economic benefits, such as through increased tourism;
• Greater participation by northern communities in the planning and delivery of research;
• Active participation of Canadian scientists in international research in the northern circumpolar regions;
• Development of a new generation of Canadian scientists, technicians, and other trained individuals, both Aboriginal and non-Aboriginal, with northern experience and polar science expertise;
• Enhancement of science and research capacity in the North; and
• Enhancement of Canada’s reputation as a leading polar nation.

1.2.4 Logic Model

A logic model (outcome map) was developed for the Government of Canada Program for IPY as part of the Treasury Board Submission. The logic model includes outputs, direct outcomes, intermediate outcomes and ultimate outcomes reflecting the program objectives. This logic model is presented in Annex A.

1.3.1 Program Management

The management responsibility was shared by AANDC with the five lead departments and exercised through the direction of the ADM Committee on IPY. The governance structure is described below.

Canadian IPY National Committee- Worked with the international community to develop the IPY international scientific themes and played a pivotal role in making the human dimension a focal point of IPY. In Canada, the members of the National Committee were key to establishing support for Canada’s participation in IPY and for making this initiative known across Canada and throughout the North.

Canadian IPY Secretariat - Based at the University of Alberta in Edmonton and established in September 2004 to support the National IPY Committee and Canada’s participation in IPY while linking the Canadian program to the international IPY community. It played a coordinating role for the many organizations and individuals involved in IPY in Canada and supported collaboration and communications among IPY researchers, Northerners and the international community. It acted as the liaison between IPY in Canada, National IPY Committees in other countries and the IPY International Program Office. It also provided a point of contact in Canada for information about IPY and initiatives to engage partner contributions.

Assistant Deputy Minister Committee on IPY - The ADM Committee for IPY was established in 2005 to provide strategic guidance and to act as the decision-making body for the Government of Canada Program for IPY. The committee was made up of ADMs from AANDC (Chair), EC, DFO, HC, NRCan, and the Vice-President of the NSERC (on behalf of Industry Canada). After the IPY Program was announced, the Committee was expanded to include the Chair of the National IPY Committee.

The ADM Committee provided direction for the program design and implementation, focusing heavily on two key policy-driven science priorities (climate change impacts and adaptation; and the health and well-being of northern communities) and working to ensure that an appropriate balance between the two priorities was achieved. The committee was responsible for final review and approval of decisions on granting and distribution of IPY Federal Program funds, ensuring that IPY investments met program objectives and appropriate accountability for the IPY Program.

IPY Federal Program Office -The IPY Federal Program Office was established within AANDC to administer IPY funding and coordinate the program on behalf of all participating federal departments and agencies. The IPY Federal Program Office also developed the program elements with input from other government department’s (federal) (OGDs) through the IPY Working Group.

In order to meet the overall objectives of the Government of Canada Program for IPY, the Federal Program Office developed and implemented competitive processes for IPY project selection. The office developed policies and processes to evaluate, select, fund and assess the results of IPY Federal Program proposals in all areas of the program, including science, data management, training and capacity building, communications and logistics for health and safety. It also worked in close collaboration and partnership with a range of stakeholders on the planning, governance and administration of the program.

IPY Program Subcommittees- IPY subcommittees and review panels were established to provide advice and recommendations to the ADM Committee on IPY. Program areas with subcommittee and/or review panel support included science; data management; logistics, infrastructure and emergency preparedness; research licensing; and training, communications and outreach.

Northern Coordination Offices - Canada’s IPY Northern Coordination Offices (NCO) were designed to play an important role in engaging the interest and involvement of Northerners in IPY and establishing dialogue and strengthening links between northern communities and the broader Arctic research community. Four IPY NCOs were established on an interim basis in February 2006 within established research-oriented organizations in each of the four regions across the North. The interim NCOs facilitated the involvement of Northerners in IPY projects and proposals, developed an IPY network within their region, and coordinated the social and cultural review of science funding applications pertaining to their region. Beginning in the summer of 2006, a series of consultation meetings with northern regional and national organizations and stakeholders began to gather input and recommendations for establishing IPY NCOs on a longer-term basis. Based on recommendations obtained through the consultation process, the same four northern organizations that served on an interim basis were confirmed as the longer-term hosts of NCOs for the duration of IPY: the Council of Yukon First Nations in Whitehorse, Yukon; the Aurora Research Institute in Inuvik, Northwest Territories (NWT); Nunavut Research Institute in Iqaluit, Nunavut; and the Nunavik Research Centre in Kuujjuaq, Quebec.

AANDC and Other Department Responsibilities - The management responsibilities for the Government of Canada Program for IPY were shared among the departments already identified. AANDC was responsible for the administration and coordination of the Government of Canada Program for IPY for each of its components and the management of the funds for northern organizations, and participated in the process of allocating funds to OGDs. AANDC also ensured that appropriate consultation occurred with OGDs with key roles in the Arctic and internationally to ensure the IPY Program was managed in a manner consistent with Government of Canada priorities.

While AANDC retained the overall accountability for the Government of Canada Program for IPY, each of the six lead departments were responsible to complete reports on their respective activities under the IPY Program through their own annual Departmental Performance Reports (DPR) and science activities through annual progress reporting to the IPY Federal Program Office.

In addition to the six lead IPY departments, collaboration within the Government of Canada involved maintaining close working relations among other departments involved in IPY. In total, 21 federal departments and agencies have been involved in the development and implementation of IPY. Some important contributors include the Canadian Polar Commission, the Canadian Coast Guard, the Polar Continental Shelf Program (NRCan), both the Canadian Ice Service and the Meteorological Service of Canada (EC), the Canadian Museum of Nature, the Canadian Museum of Civilization and Parks Canada.

1.3.2 Key Stakeholders and Beneficiaries

Organizations, individuals, federal government departments and agencies, other levels of government and other non-federal entities, public or private have an interest in social and scientific development in the North, including:

• Federal departments (AANDC, DFO, EC, NRCan, HC and Industry Canada (represented by NSERC)
• Canadian Institutes of Health Research
• Social Sciences and Humanities Research Council
• Public Health Agency of Canada
• The territorial governments
• Provincial governments
• Aboriginal organizations
• Northern communities
• The science community
• Universities
• Academic institutions
• Industry

2.3.1 Phase 1 - Pre-assessment

Prior to the evaluation, a pre-assessment of the Government of Canada Program for IPY has been conducted by EPMRB to better understand the program and its environment.

The objective of the pre-assessment was to increase evaluation efficiency by identifying information and data sources that could be used to measure or assess impacts, assessing the quality of performance data collected and measuring to what extent other departments and stakeholders reported their activities as stated in their role and responsibilities, and whether or not this information could be used for the evaluation.

The objective of the pre-assessment was also to review the scope, methodological approach and collaborative approach of the evaluation by assessing whether or not other departments should have a more participative role and whether the evaluation could best be launched horizontally.

To this end, EPMRB reviewed files and documentation provided by stakeholders and conducted preliminary consultation with representatives at AANDC, other departments and organizations (that received a high percentage of the allocated funds) as follows:

• preliminary interviews with eight AANDC program representatives (IPY office);
• preliminary interviews with three northern coordinators;
• preliminary interviews with representatives of five departments and agencies (DFO, Canadian Coast Guard, EC, NRCan, NSERC); and
• a preliminary interview with one representative of the Canadian IPY Secretariat.

2.3.2 Evaluation - Data Sources

The evaluation's findings and conclusions are based on the analysis and triangulation of the following multiple lines of evidence:

• Literature Review/ environmental scan:
  
  The purpose of the literature review was to explore key issues related to the relevance and needs, achievement of results, lessons learned and best practices and to assess the economy and efficiency of the IPY.
  
  The evaluation team used various sources of information for the reviews, including academic articles and papers, government reports and other publications. The methodology employed to carry out the review of relevant background materials included the identification and provision of documents by the program for review by the evaluation team and a review of available documentation at the international level. The literature review included an environmental scan portion that looked at the governance approaches used by other participant countries, as well as national and international organizations to inform best practices and assess the economy and efficiency issue.
• Document, file and governance reviews:
  
  A document and file review has been undertaken with the objective to develop a thorough understanding of IPY and address a number of the evaluation issues (e.g. procedural, governance, success, design and delivery aspects of the program). EPMRB identified and reviewed relevant files and documents during the pre-assessment phase of the evaluation and Goss Gilroy Inc. supplemented them during the data collection phase. Key documentation related to the Government of Canada Program for IPY and projects have been reviewed, including Treasury Board Submissions, IPY highlights and achievements reports, proposals, committees' review reports, frameworks, guidelines, annual reports, terms of reference, and strategic plans. As per November 2011, end of the evaluation's document collection, 29 project final reports were submitted to the IPY office regarding science research projects. The analysis of these reports looked at results and performance measurement indicators. The main sources of documents for this review were the IPY office, IPY partner departments, and information available in the public domain.
  
  A governance review was based on interviews within the Department (IPY office and committees members) and with other departmental government (managers of other science programs), intended to supplement the document review component in the evaluations of the Government of Canada Program for the International Polar Year and the Northern Contaminants Program. It represents a comparative review of governance for science programs.
  
  The document and file review provided a background briefing for the evaluation team prior to the field work phase and informed the findings for a number of the evaluation issues and questions.
• Data Analysis:
  
  The purpose of the administrative data review was to assess information on program performance in terms of success, efficiency and economy. A database for analysis drawing from science proposals was developed and the evaluation team reviewed the available financial data (budgetary and expenditures information) from the IPY office. As well, performance measurement information was reviewed.
• Key informant interviews:
  
  Key informant interviews helped in gaining a better understanding of the perceptions and opinions of individuals who have had a significant role in or experience with IPY, or who had a key stake in it. Interviews were conducted by two different consulting firms: Capra Inc: completed the first set of interviews while Alderson-Gill and Associates completed the second set of interviews during the case studies.

First set of interviews:

The initial list of key informants was populated by EPMRB totaling approximately 50 names from which 46 key informants were successfully interviewed by Capra inc. Due to non-responses, eight alternate names were used in addition to the initial list. These interviews were in addition to the interviews that were conducted during the case studies.

Face-to-face interviews were conducted with interviewees when possible. When an individual was not available for a face-to-face interview, an interview was conducted by telephone. An interview protocol was e-mailed to the individuals prior to the interviews. When key informants did not respond to the initial contact, several attempts (up to 10 attempts) were made via phone, email, and fax. This element was conducted at the same time as the Northern Contaminants Program (NCP) and Northern Scientific Training Program (NSTP) evaluation for leverage and efficiency.

A total of 46 key informant interviews were conducted during the period of November 2011 – February 2012. Key informants included representatives of the following roles:

• AANDC IPY Headquarters Directors, Policy Directorate, and IPY Office Coordinators (n=10)
• Other Federal Government Headquarters Departmental IPY Working Group Members and the Canadian Coast Guard (n=7);
• IPY Sub-Committee Members, as well as International and National Committee members (n=8); and
• Professors, and Assistant Professors, Stakeholder Organizations, IPY Science Project Funding Beneficiaries (n=21).

Second set of interviews:

Another set of key informant interviews was undertaken by Alderson-Gill and Associates during the field work and case studies. The objective was to meet people from the Territories in person and benefit from decreased data collection costs by coordinating with the case study site visits. In total, 28 interviews were conducted (on-site / telephone):

Key informant interviews- Outside the National Capital Regions - 23

• AANDC – 3
• Territorial government – 7
• Researchers – 4
• Northern participants – 9

Key informant interviews-National Capital Region - 5

• AANDC – 3
• OGDs - 1
• Territorial government – 0
• Researchers – 1

Analysis

Given that the number of interviews is fewer than 100, preference for maintaining the wholeness of the message within each interview was adopted by Capra Inc. Upon completion of the interviews, data were stored in individual Microsoft Word files. No quantitative interview data were collected. Where appropriate, the repetition of a theme was quantified; however, these were not reported as percentages for two main reasons: the total number of respondents was fewer than 100 and while respondents were asked similar questions, the questions were open-ended and often resulted in responses that emphasized varying aspects of the initiative and perspectives that were not possible to group with other responses in a useful manner.

The qualitative data, stored in Microsoft Word files, were colour coded in a systematic manner according to the evaluation matrix. The generated themes, relevant to the evaluation objectives, were then shared among the team of evaluation analysts to ensure agreement among those who facilitated the interviews.

Based on the key informant interview technical report produced by Capra Inc, Alderson-Gill and Associates completed the analysis with the additional interviews.

For the qualitative data, as well as describing the limited quantitative results, certain terms were used to indicate the proportion of respondents or types of respondents to which the finding refers. These terms are roughly equivalent to the percentages shown in the last column of the table below (Table 4).

• Case Studies:

The case studies addressed a number of the evaluation issues and questions. Case studies covering all territories and different streams (science, logistics, communication and outreach) ^{Footnote 13} were conducted. Selection of the case studies was based on criteria such as: type of project, geographic coverage, budgetary considerations and the beneficiaries associated with the projects. Goss Gilroy Inc. and EPMRB developed the case study background based on project files (annual reports, proposals) and additional information from the web. Preliminary findings on the design, implementation, results, impacts, lessons learned and challenges were extracted from those files. Findings from interviews conducted by Alderson-Gill & Associates during the site visits complemented the background and the range of evaluation issues. A total of seven case studies were conducted, which included:

• Science: three
• Logistics: two
• Communications and Outreach (one including document review only): two

2.3.3 IPY 2012 Conference Evaluation Survey

The IPY 2012 Conference was the final event of International Polar Year 2007–2008. The conference was held on April 22-27, 2012, in Montréal. Entitled From Knowledge to Action, the conference brought together internationally renowned scientists, key policy makers, stakeholders and northern residents.

This conference was expected to provide an opportunity to apply and disseminate the knowledge and scientific results from IPY from around the world and focus on next steps. It was intended that participants will consider ways to translate those new understandings into policy that will guide activities in and enhance stewardship of the polar regions. As this conference was an important activity and cost for AANDC's IPY Program, and an accumulation of the efforts undertaken during the six year duration of the program, this activity was evaluated by itself in terms of results achieved.

To assess the conference, Harris/Decima, in partnership with Paradigm Consulting Group, conducted an on-site study with attendees of the IPY 2012 Conference in Montreal. For this assignment, participants were selected at random using a floating sample technique. Interviewers began by selecting the first eligible respondent they encountered; once they secured a participant to complete the survey, they approached the next available person. In total, 397 surveys were completed during core conference hours, generally between 9am and 5pm. Interviews were conducted over a four day period between April 23 and 26, 2012. Upon completion of the data collection, Harris/Decima produced a cleaned, fully labelled, and coded SPSS data file and an analysis plan was drafted in consultation with the AANDC Project Authority.

According to the evaluation work plan, the survey was conducted after the evaluation data collection. While some relevant information from the results of the survey was used at the end of the analysis to inform this evaluation, most of the results are presented in its own report as it looked at particular elements pertaining to the conference. The survey was a separate line of evidence and was not part of the triangulation of findings.

2.3.4 Considerations, Strengths and Limitations

The reader is encouraged to take the following limitations into account when reviewing the findings presented in this report.

Working group

As part of the evaluation the Government of Canada Program for IPY, the EPMRB formed an evaluation working group in order to draw on the knowledge and expertise in the IPY Federal Office and other departments (DFO, NRCan, NSERC, EC and HC). The broad mandate of the working group was to provide ongoing advice to EPMRB, particularly with respect to:

• The quality and relevance of the evaluation's approach and research instruments as outlined in the methodology report and data collection instruments (for example, file review guides and key informant interview guides);
• Proposing key data sources and stakeholders whose perspectives are vital to an analysis of the impact and, where possible, facilitating access to such resources; and
• The quality and relevance of draft deliverables and research findings to ensure results are thorough, balanced and useful.

Participation with the working group resulted in three meetings where the methodology report and preliminary findings were discussed, and through emails to review the draft final report.

Internal Peer Review

Internal peer reviews were conducted by EPMRB evaluators who are not directly involved in the evaluation project. The reviewers have sufficient knowledge and experience to assess the quality of methodology and final reports and ensure they comply with relevant Treasury Board, AANDC and branch policy, criteria, and standards. Internal peer reviewers also examined the degree to which final reports correspond with the evaluation's Terms of Reference and methodology reports. Peer reviewers are not expected to validate factual information about programs or policies or edit reports. These tasks are the responsibility of the Project Team. Internal peer review work is guided by the EPMRB'S Peer Review Guides. These guides include questions, which reflect Treasury Board standards for evaluation quality and Treasury Board guidelines for final reports.

Peer reviews are typically conducted at two critical points in the evaluation project cycle: methodology report and draft final report.

Knowledge gaps and ongoing research needs

Almost all key informants agreed that there were some gaps in the research conducted through the IPY Program. It was recognized by many that these were future research needs and, given IPY objectives, did not necessarily reflect gaps in IPY-funded research. Research needs identified included research on infrastructure such as highways, ports, sewers, water treatment facilities, considering what is known about the potential impacts of climate change. Some northern-based key informants believe that IPY investments in physical and natural sciences were excellent, but that investments in health and social sciences were not sufficient as unfortunately few proposals were submitted. Other key informants stated that IPY has provided a baseline of information and provides guidance for the type of research to be conducted in a future wave of research.

Many key informants, particularly with northern-based organizations, believe there is a continuing and compelling need for more applied science, particularly in areas related to human health, including mental health, and on the impacts of resource developments. Some would have liked to see a better balance between the physical, natural and social sciences in IPY, even though IPY produced "great science overall", and think that future initiatives should include more human, health and social research projects. These were described as "unfinished pieces of IPY".

Territorial governments and other northern organizations would like to have better mechanisms to incorporate their concerns and research priorities into plans for the direction of future research initiatives. IPY highlighted the need to identify and articulate their scientific research priorities because of what they saw as a gap in this regard. Efforts are well under way in Yukon and the NWT to develop territorial research strategies.

Some northern-based key informants, as well as some scientists and government representatives, commented that there were limits to the geographical focus of IPY: specifically, that IPY focused on the Arctic to the exclusion of the Antarctic, and that the IPY focus concentrated on coastal areas to the detriment of land-based or off-shore areas. However, IPY projects had good coverage across the North and one fifth of the budget went to projects specifically identified land-based.

A few key informants commented that a lack of collaboration with the private sector was a gap in the IPY Program. Specific areas where the private sector could have been involved were not specified.

A review of the literature confirmed the wide range of issues of continuing concern in the Arctic, especially in the areas of climate sciences, biodiversity, and health sciences.

Immediate Outcomes:

Northerners, Aboriginal people, youth and northern communities are engaged in IPY activities

Given the high goals of IPY for engagement of Northerners, Aboriginal people, youth, and northern communities in IPY activities, it can be concluded that engagement was partially achieved. Considering the broader context and barriers, including northern organizational capacity challenges, limited previous experience with northern scientific research, and low levels of educational attainment in the northern population at large, the levels of engagement associated with IPY activities were a positive achievement. Overall, a small majority of key informants believe that the right level of engagement of Northerners, Aboriginal people, youth, and northern communities in IPY activities was achieved. The rest were relatively evenly divided between thinking that engagement goals were partially achieved and thinking that engagement did not occur to the intended or desired extent.

Engagement in scientific research - Many key informants reported that Aboriginal people and Northerners were involved in research, that IPY "brought people into the equation" for the first time. The project file review found that almost all science projects (96 percent) engaged Northerners and northern communities to some extent. Many key informants reported that many IPY projects engaged community groups in logistics (e.g., field support) leading to employment and economic benefits, in education, and in science projects, including linkages with traditional knowledge (e.g., through the Inuit Health Survey and the Voice of the Caribou People project). Of the 45 projects funded at AANDC by the Government of Canada's IPY Program, 33 included the integration of traditional knowledge as part of the research. An IPY highlights report indicates that more than 1,800 Northerners were involved in IPY research projects, in positions ranging from researchers to field guides for on-site research teams.

Communications and outreach - IPY involved northern and Aboriginal youth, adults and elders through communication and outreach activities such Polar Days, researchers in classrooms, and films (e.g., as sponsored by the Ontario Science Centre). Approximately 2,000 Northerners participated directly in various ways through communication and outreach activities. The project file review found that most IPY projects undertook communication and outreach activities and those that did not were technical projects (e.g., chemical sampling and analysis) or projects conducted in remote locations where these activities were not applicable. Northerners and community members almost always benefited from communication and outreach activities.

Almost all researchers (96 percent) made some type of presentations to Northerners and northern communities when visiting communities before, during or after the fieldwork, often during activities organized by Northern Coordinators. Final project reports indicate that these presentations often provided a good opportunity for exchanges between the researchers, community members and northern organizations to discuss the relevance and implications of the research and results for communities.

Student engagement - Student activities were a strong focus of IPY and many students participated in IPY activities such as Students on Ice and the Alianait Arts Festival. Key informants representing northern organizations reported that southern scientists often wanted to use northern research assistants - who were often students - wherever possible because this reduced costs and northerners brought their knowledge of northern logistics and safety to the project. According to the IPY Program, a total of 226 northern students were engaged in IPY research projects.

Traditional knowledge integration - Approximately half of the projects examined in the project review had a traditional knowledge component. Knowledge on different topics such as nutrition (country food), sea ice and climate change and their impact on communities was shared with researchers. Some researchers reported that this traditional knowledge was an important factor in the success of their studies by making the research process more effective and by providing knowledge for future studies. Traditional Knowledge (TK) was collected in many cases during exchanges with elders, hunters and trappers, teachers, nurses and other community representatives. To collect this information, local researchers, translators and interpreters in the community were hired on many projects. In one project, a Traditional Knowledge Steering Committee that involved community and regional representatives was put in place to guide and coordinate traditional knowledge collection and to develop the project from the community level.

The consensus among northern-based key informants was that, while some IPY projects presented a "gold standard" in engagement, true engagement and collaboration remain a challenge. Despite the IPY design, which incorporated significant engagement requirements, northern research license permit requirements, and considerable effort by all stakeholders, key informants reported that local engagement was missing in many instances. Key informants attributed this lack of local engagement to the absence of ongoing, meaningful consultation with territorial governments except in the early planning stage, the high cost of travel to northern communities, and researchers' lack of experience and knowledge about how to engage Northerners in research. They also noted that the degree of success in engagement, even though partial, is an indicator of success for the program given the challenges of conducting research in the North.

Almost all researchers recognized the importance of engaging Northerners and northern communities in their project. Many researchers, as well as other key informants, observed that the participatory processes of IPY research set a new precedent for how research is conducted in the Canadian Arctic. The views of an Aboriginal organization, which reported that, "for the first time, people were involved in science", were typical. The experiences of researchers varied widely, however, with both successes and challenges in engaging Aboriginal people and Northerners in IPY activities.

For successes, the visibility of IPY was high and there was a lot of engagement by the public in Arctic science. Many projects involved Aboriginal organizations, community groups and other northern organizations in the planning, implementation, and dissemination of results. Some researchers reported that their projects engaged these organizations and the wider public. For example, researchers engaged NCOs/community groups to hire local people for logistics, as guides, translators and for field support, which led to employment and economic benefits. The level of cooperation between the Northern Coordinators and northern organizations was reported to be very good and contributed to engagement. Northern Coordinators contributed to forums for community engagement were set up through IPY.

For challenges, northern-based key informants stated that, overall, there were deficiencies in communications with communities.^{Footnote 14} Key informants reported that communications and outreach projects were successful at engaging Northerners but that science projects were less successful at involving communities. Some researchers did not appreciate community interests and the mechanisms to bridge researchers and communities were not always adequate, despite the efforts of Northern Coordinators. Several key informants reported that small community organizations could be overwhelmed by the number of inquiries from researchers and by the expectation to review research proposals and reports.

Increased capacity by northerners for participation in northern scientific research / Increased capacity of Northerners to carry out northern scientific research

Almost all key informants agreed that IPY initiatives have increased northerners' capacity to conduct northern scientific research. Some notable projects, such as the Inuit Health Survey, were particularly strong at developing research capacity through research techniques learning and training of Inuit surveyors. The Logistics for Health and Safety component was identified by key informants as IPY successes for increasing capacity through new health and safety equipment, enhanced search and rescue infrastructure, and personnel training, including training of youth as research participants.

Research infrastructure - IPY contributed to an enhanced scientific research infrastructure across the North, both through research and academic institutions (e.g., Nunavut Research Institute, Aurora Research Institute, Nunavut Arctic College, Aurora College)^{Footnote 15} and the engagement of Aboriginal organizations in all northern regions and other key organizations. Strong networks and relationships were established among scientific organizations, scientists and northern organizations for monitoring the Arctic (and the Antarctic among the scientific community, for example, the Polar Earth Observing Network). IPY funding criteria also required researchers to do multidisciplinary work, which broadened networks both within the scientific community and between the scientific community and northern organizations.

Employment of Northerners - A review of 29 project reports (of 45 projects in total) found that a total of 566 Northerners were hired on these projects during the IPY years. While in a few cases, their contribution was participating on projects through interviews, some others assisted directly with the project by doing wildlife monitoring (e.g. bears), guiding, tagging, translating, interpreting and providing support in areas such as transportation, accommodation, hunting, wildlife protection and logistics. This review and cases studies found that several of the Northerners who have been involved in IPY projects have become valuable local scientists who have collaborated on other research projects or with local government departments.

Youth and student engagement - An important success of capacity development from IPY was the learning and practical research experience provided to young people. IPY research projects exposed northern students to on-site research, which provided an excellent training experience and motivation to follow a science path in their education and future careers. Many projects hired northern students or involved them as participants. For example, 32 of 75 students participating in the most recent Students on Ice project were northern and Aboriginal students. Northern students were hired by researchers during the IPY years for many different roles, including data archivists, community liaisons, organization of the key informant interviews and focus groups, recruitment of participants and as field assistants during field work activities (for example, training in sampling and excavation methods, and measurements on the sea ice).

Other projects (e.g., communications and outreach) had student mentorship and training components for technical and administrative positions. A Northern Coordinator called IPY "a good experience [for youth], especially for capacity building." Some schools involved students and teachers (e.g., studying the cryosphere). Some key informants stated that getting youth involved and emphasizing the importance of staying in school was a very positive impact of IPY.

It was widely agreed that the involvement of students in IPY projects will help to build the next generation of northern and Canadian polar scientists, technicians and trained individuals (see long-term outcomes; enhanced northern research capacity). IPY generated increased interest in the North and northern science among students from both the South and North. The review of IPY project files found that many of the northern students that worked on IPY activities are continuing in the field they were part of either by working in their community or by continuing their studies or research with academics, all of which is helping to build a foundation for science and technology in the North.

Pool of researchers - Most IPY projects were led by well-established northern researchers, and the pool of senior researchers (Principal Investigators) did not expand. The IPY project file review found that, of 45 projects, eight projects were led by Northerners or Aboriginal organization (based on Project Lead affiliation). As one key informant said: "there were not a lot of rookie researchers... most Principal Investigators had done work in the North." Key informants emphasized that the selection of IPY projects and scientists was only a two-year event and that, although the pool of senior scientists did not expand, the pool of next generation and science students expanded greatly.

Building a foundation for Science and Technology (S&T) expertise in the North - non-Northerners - The project file review found that, in the 29 project final reports submitted as per November 2011, a total of 536 non-northern students (mostly in post-graduate studies) were involved with IPY activities. These students learned research techniques, and quantitative and qualitative analysis methods, and gained experience and insights in Aboriginal life and culture. One report stated: "These students form the basis of a new generation of polar researchers, with increased capacity in the areas of community-based field work and integrating northern and southern knowledge systems." Another researcher noted, however: "The downside is that we are still searching for the mechanisms for maintaining a sustained effort in polar research. We are losing these people that we trained – they are moving into other fields and to other countries because there has been no connection between the enhanced effort on IPY and any ongoing program."

While agreeing with the positive conclusions about the involvement of students and the next generation, some northern-based key informants also provided some cautions. For Northerners, IPY represented a positive ambition but achieved only moderate capacity gains. Some thought there was not enough involvement of Northerners and that project specific training resulted in little capacity development. Others thought more could have been done in education and that there were missed educational opportunities. Some said that the best results in capacity building were achieved by northern-led research projects and projects led by researchers associated with universities with experience in the North (e.g., University of Montreal, McGill, Carleton, University of Calgary). Key informants from Nunavut cautioned that the Government of Nunavut has limited capacity as it has only two scientists.

Some key informants said that the post-IPY environment raises concerns about the sustainability of improvements in northern capacity achieved through IPY. Concerns were expressed that the momentum and capacity built through IPY will dissipate and that the end of IPY funding represents the loss of opportunity for this new generation of scientists and technological experts.

Improved measurement and monitoring systems

It refers to IPY projects that can scientifically measure and put in place some monitoring systems to collect data in order to get better understanding of a situation and its impact, such as freshwater systems; measuring freshwater biodiversity, developing predictive models and establishing community-based monitoring programs. As per the IPY Highlights and Achievements document and Annual Progress Reports, it is known that the program contributed to achieve this outcome through science projects and training and capacity building projects.

New data, information and knowledge related to impacts and adaptation to a changing climate and health and to well-being in the North

The file review found that a total of 513 publications were produced and 1,579 presentations were made related to the 29 projects looked at out of 45 IPY projects. Science projects together produced a large amount of data that will benefit different stakeholders in ways, including:

• improved day-to-day forecasting with a much improved representation of Arctic weather and environmental trends;
• understanding of the atmospheric processes of climate and climate change;
• communities that can use the results to help them adapt to any projected changes in their landscape;
• scientific information that policy makers and decision makers can use as a basis for chemical management strategies, domestic and international policy, regulatory options related to the protection of air quality and the environment; and
• information to provide a basis for new municipal wastewater effluent standards for communities across the North.

Many researchers mentioned the potential impact of their results on policy (e.g., changes in the DFO Management Framework on Beluga). At the time of the evaluation, some researchers are still at the point where data analysis and interpretation need to be completed and validated through peer review of publications or presentations. Nevertheless, there already are indications that IPY research results are informing policy:

• Contaminants - Information collected on contaminants (e.g., mercury) is being used by government: the global mercury models will be the foundation for scientific information that will be used for the United Nations Environment Program – Global Mercury Negotiations; the information gathered through this research will be included in the forthcoming Canadian Mercury Science Assessment, which will provide the Canadian and international governments the state of science in Canada in regards to mercury; results on contaminants will be integrated in the upcoming third Canadian Arctic Contaminants Assessment Report (CACAR III) due for release in 2012; and the research findings of this program will be included in future reports to the Northern Contaminants Program (AANDC) and the Arctic Monitoring and Assessment Programme for use by the Arctic Council on policy issues related to the Arctic.
• Arctic Wildlife - Data obtained from the different sub-projects are being used by agencies such as Environment Canada (Canadian Wildlife Service), Parks Canada, Yukon Territorial Parks, Churchill Northern Studies Centre, Churchill Health Authority and by local communities. IPY-generated data are being used to prepare ecosystem or species conservation plans (e.g., studies at Alert have been highly influential in interpreting and understanding reasons for the steep decline in some red knot (northern shorebird) populations in the Western Hemisphere and has contributed to the report prepared for the Committee on the Status of Endangered Wildlife in Canada, which led to the designation of different populations as Endangered, Threatened, and of Special Concern).
• Traditional Food - The results of IPY research are directly affecting nutritional policy at the public health level in Nunavik. Public Health management is integrating knowledge from the research into efforts and programs to reduce trans-fats consumption and to promote traditional foods consumption.
• Community Adaptation - information on community landscape hazards in Nunavut were requested by the community research committee in Clyde River, and the Government of Nunavut and the Canadian Institute of Planners are helping communities to incorporate climate change and other impacts into revisions of their community plans. IPY mapping and the data integration protocol are being investigated for use in other Nunavut communities (e.g., Arviat, Cambridge Bay, Kugluktuk).
• Wetlands - IPY data is contributing to the development of new standards for effluent in municipal wastewater for communities across the Canadian North. Communities have been using this data to meet requirements for their current water licence permits, as well as assisting Community Government Services in the Kivalliq Region in developing their records and capacity building projects.

Key informants reported that, from a scientific perspective, IPY created high value, innovative research results, products, data and information. IPY represented a "huge step forward" in available high quality data and publications on the North. IPY research has led to increased levels of knowledge and awareness among the public concerning climate change and the health and well-being of northern residents.

Proper archiving and management of IPY data

The IPY Program had ambitious goals for data sharing, the achievement of which would have required a major change in the culture of academic research. The IPY Governance Review determined that the IPY Program implemented sufficient internal project monitoring systems (e.g., annual progress report and end of project report template). In addition, research results are published in scientific journals and are available through online repositories.

A Framework for Data Management produced in January 2008 stated the following guiding principles for the IPY Data Management components:

• Implement and assist researchers in implementing the Canadian IPY Data Policy;
• Promote and build upon existing best practices, resources, programs and processes; and
• Build a legacy of improved data management through increased awareness and understanding of good data management standards and procedures.

There were four main expected results of the data management component:

• a metadata database for Canadian IPY data;
• capacity development and training of Northerners and researchers in data management techniques;
• new IPY data sets archived as per IPY requirements made available to the public in a timely manner; and
• guidelines, standards and best practices for data management.

The Framework for Data Management also identified some significant challenges to achieving these data management objectives. These challenges, both anticipated in the framework and confirmed by key informants as relevant during implementation, included:

• Awareness among researchers and policy makers of the importance of data management and responsible stewardship of data - this "will require a shift from the conventional thinking of data ownership and control to a culture of data sharing".
• The short duration of the program funding versus the long-term stewardship requirements of data, which requires the development and implementation of innovative approaches and partnerships.
• Legal issues surrounding ownership, access and control of data - particularly for multi-institutional projects. The intellectual property policies established by institutions may conflict with those of other institutions or with Government of Canada policies.
• The capacity within the research community and northern communities to manage their data - capacity for data management needed to be developed among researchers, existing data centers and nascent data centers.
• Stewardship of TK - proper stewardship of TK/Inuit Knowledge, including addressing issues of intellectual property, appropriate use of accreditation and citations as well as ensuring long-term preservation of the TK, needed to be developed for the IPY Program.
• The need for standards in data formats and procedures, issues of privacy and confidential data - at the onset of IPY, there were no universally accepted standards for data formats. The program aimed to streamline the processes for archiving IPY data and making access to the data more widely available.
• Issues of privacy and confidentiality - these issues needed to be addressed for research projects dealing with northern communities and human subjects, which presented a challenge in the context of the IPY principles of openness and accessibility of data and information.
• The wide varieties of data types - IPY generated many different types of data from physical samples to oral tradition, each with its own challenges for data management.

Within the context of these challenges, it can be concluded that Canada was a leader in data management through IPY. The percentage of the IPY budget allocated to data management was approximately five percent - $7.0 million - which was much higher than the proportion allocated by other countries. Of the total funding for data management, $2.4 million was allocated to data assembly and archiving, $2.4 million for ensuring access to IPY data, and $1.2 million to supporting data management. Data management was part of the program's annual progress reporting. Data-targeted funding allowed for the creation of new data archiving systems, and the resulting infrastructure has become an operational model including the likely development of a national multi-sector data base.

Although there has been much progress towards achieving the program goals, data management is as of yet incomplete and the success of data management and archiving has been mixed. One problem was that the late start of the data management component meant that IPY activities started without an established data management infrastructure. It will take several years for all datasets to be in place, accessed, and analyzed, and for all connections to be made. On the other hand, few datasets were made available by the end of 2011, with the main problem being that researchers did not submit their information and data as per their agreements. The project file review found that 35 percent of projects had not put their metadata in the Polar Data Catalogue by the end of 2011. ^{Footnote 16} To some extent, delays were anticipated, as it was expected that researchers would have the time to publish their results before submitting the data to the IPY and that most of the IPY funding allocated to data management would be spent near the end of the program.

Key informants suggested that specific terms and conditions for data management should be incorporated into the agreements with researchers.

Timely emergency and search and rescue response, and increased security of Canadian and all IPY participants in Canada's North

Specific IPY projects were designed to address research infrastructure in the North and to increase security and emergency search and rescue for researchers. For example, the Qaujisarvik Research Network project in Nunavik received IPY funding to support the purchase of electronic devices, such as GPS receivers, locator beacons and satellite phones, as well as other terrestrial and marine safety equipment, to increase field safety and communications. This project was completed by the Nunavik Research Centre (operated by Makivik Corporation), the Centre d'études nordiques (coordinated by Université Laval), Anniturvik Landholding Corporation (owners of the former abattoir at Umiujaq), and the Kativik Regional Government, through a formalized research relationship. The Qaujisarvik Research Network and the partnership supporting it were presented in the Fall/Winter 2007 edition of Meridian, a publication of the Canadian Polar Commission.

In the NWT, the Royal Canadian Mounted Police purchased safety and search and rescue equipment for communities in the Western Arctic, and provided a program of search and rescue training for community volunteers that according to participants was a great success in terms of increasing search and rescue capability and by leaving a legacy of much better equipment, that is now available for future use in the area.

In all, 12 organizations received support from IPY to train more than 200 northern search and rescue volunteers. Coordination between federal, territorial and community emergency preparedness groups in the North was improved through conferences, workshops and meetings convened in preparation for IPY. Search and rescue capabilities were expanded in the North through the testing of amphibious all-terrain vehicles, the development of remote site command posts, the establishment of emergency fuel caches in the high Arctic, and the enhancement of northern ice and weather forecasting.

Improved governmental regulation and improved timeliness of licenses processing

IPY was the only federal government research program that made investments to increase capacity for the review of science project applications and licensing. The Canadian Arctic Research Licensing Initiative (CARLI) was developed to lead to more efficient approval of scientific and research activities in the North (see below the section on more efficient approval of scientific and research activities in the North, under intermediate outcomes). The IPY funding for CARLI provided an opportunity for northern research licensing regulators to examine and improve the licensing processes for which there are responsible.

A national advisory committee with broad representation was established to provide advice and information on the issues and challenges in northern research licensing. The objectives of CARLI were to:

• build relationships, share best practices and improve communications and expectations about science licensing amongst northern regulators, Aboriginal organizations and researchers/industry;
• explore the possibilities for coordination in the various northern research-related application, approval and reporting processes within the current regulatory legislation; and based on this,
• develop projects and information to improve the scientific licensing application and approval process that will benefit northern regulatory bodies, Aboriginal organizations, northern communities, and future northern science programs and activities.

The IPY file review identified 20 projects that requested a total of 305 licenses for research in northern territories and regions (including Nunavik and Nunatsiavut). Usually, acquiring research licenses was straightforward. As one researcher explained:" The experience of obtaining research licences was generally streamlined after years one and two in the project, once the steps for obtaining the licence were learned by initial researchers, their experiences were passed on to subsequent researchers in each region." All researchers mentioned that community consultations required to acquire their licence went well.

Some researchers (25 percent of the sample) identified problems related to delays in processing applications and in licence renewals (which could take several months). Some researchers explained that this situation was due to the high number of applications for licenses being processed by the northern authorities, combined with limited resources to process them and staffing changes at the licensing authorities. One researcher mentioned that "the long processing time" caused delays in their field work and created difficulties in the already complicated logistics of operating in the North. Some also mentioned that licensing needs to be streamlined, especially for small projects and ongoing projects for which the impacts already are known. A few researchers identified issues with longer forms and redundant questions. All forms asked for the same information, but in slightly different ways so it took a fair amount of time to make sure each application answered the questions specifically." Another researcher suggested streamlining the process so that only one application is required for each project, with activities segregated into subprojects so that reporting for different researchers can be done separately. However, as per the program, this was looked into during CARLI and the regulators in the territories unanimously agreed that this would not be possible.

All researchers mentioned that, despite challenges, they received excellent assistance when completing the research licenses and their renewals, and that licensing authorities guided them through the process and answered their questions promptly.

Intermediate Outcomes:

Raised awareness of northern issues and Government of Canada activities in the North, and new generation of northern and polar Canadian scientists, technicians and other trained individuals, both Aboriginal and non-Aboriginal, building a foundation for S&T expertise in the North

All key informants either fully or partially agreed that IPY has resulted in increased levels of knowledge and awareness among the public concerning climate change and the health and well-being of Arctic residents. Federal government representatives said that data generated through IPY research will be useful in a wide range of fields, including climate change and public health, for which IPY-generated data is leading to a new public health nutrition campaign in Nunavut. Researchers identified projects that brought climate change issues to a large public audience, for example, the Students on Ice project that was widely publicized with presentations and films shown in many communities.

Representatives of northern organizations agreed that some partners and organizations now have a better understanding of scientific research because of IPY. They also stated, however, that sometimes there was a lack of feedback to communities about the results of research. In their view, the most effective projects were ones involving community partnerships. Factors contributing to the success of projects included: 1) planning and building rapport with community representatives; 2) spending time in the community; and 3) having money to pay local people (which respondents understood is often not possible within project budgets). One noted that IPY funded the research phase, but not the post-project analysis or knowledge transfer phase, which will limit public awareness and knowledge of the issues. As per the program, it was possible to receive funds for travel to communities, honoraria and communicating results if researchers requested them as part of their project proposal.

Government, business and other organizations make operational and management changes that respond to opportunities and threats in the North

There were no evaluation findings regarding this outcome.

High value, innovative research results, products, data and information with increased depth, breadth and accuracy and improved consistency, comparability and reliability

As stated previously, IPY has created high value, innovative research results, products, data and information. IPY-funded research has led to a major increase in the availability of high quality data and publications on the North. IPY has also increased the levels of knowledge and awareness among the public concerning climate change and the health and well-being of Arctic residents.

Non-confidential data easily and widely accessible to researchers, the public and northern decision makers

For the public and northern decision makers, this issue is about communications and the accessibility of research results. Many key informants believe that the IPY program made positive progress in providing results to local stakeholders:

• IPY encouraged researchers to provide feedback to Northerners on their research results.
• Project findings were readily available through multiple sources: e.g., program website, publications database, meetings and presentations.
• The accessibility of the results to scientists was especially valuable.

Methods and mechanisms for communicating research results and for making data more accessible included the publications database, the IPY website, TV vignettes, film festival, newsletters, various meetings including the IPY 2012 Conference, presentations, and radio interviews.

Many communications activities were project specific. There also were more general communications about the program undertaken by the IPY Federal Program Office, Northern Coordination Office, and others.

Although much progress was made, there were significant challenges in translating research findings into publicly-accessible formats. The electronic-digital focus of most communications about IPY, including the research results, was not suitable for some northern communities. A representative of a northern organization stated that they do not yet have effective communications and reporting relationships with scientists. They said that their organization has to actively pursue researchers to get copies of their results. The IPY Program published regionally focused newsletters for the Yukon, Northwest Territories, Nunavut and Nunavik highlighting IPY project results at end of March 2012.

Canada recognised as a credible host to the international community

All key informants agreed that IPY has strengthened Canada's profile internationally as a leader in climate change research and research on the health and well-being of Northerners. Many noted that Canada invested far more on research, including on human studies and health research, than other countries. Researchers agreed that IPY was an excellent platform for their project to become known internationally, that they have formed strong partnerships with international organizations, and that their project has enhanced Canada's reputation internationally.

AANDC representatives emphasized that international partnerships are a success of IPY. Representatives of northern organizations agreed that connecting Arctic researchers internationally was a major success of IPY and that IPY opened doors and increased interest from international partners. For example, the United Kingdom now has a Memorandum of Agreement with Canada for Arctic research.

The literature review conducted for this evaluation examined how the Government of Canada Program for the IPY contributed to the broader international initiative. Two relevant examples include the following:

• Fisheries and Oceans scientists and staff published a paper^{Footnote 17} describing how the "Canada's Three Oceans" (C3O) project constitutes a Canadian contribution to the International Polar Year. The C3O project explored marine Canada from the surface to the seabed, from the smallest (virus) to the largest (whales) organisms, and from the Pacific to the Arctic to the Atlantic. C3O also showed the inter-connectedness of Arctic and sub-arctic domains and how such domain boundaries may be affected by a changing climate.
• Environment Canada published a document detailing EC's contribution to the IPY^{Footnote 18}. According to the document, for IPY 2007-2008, EC's research contributions reflects its science mandate, from weather, water, snow and ice, to the transport and fate of contaminants in the northern environment, to Arctic wildlife and ecosystems. The document indicates that Canada's participation in the fourth IPY involved unprecedented cooperation nationally and internationally, bringing together more than 1,750^{Footnote 19} researchers from government agencies, universities and northern communities and over 240 international collaborators from 23 countries.
• A survey conducted for the evaluation at the recent IPY 2012 Conference in Montreal found that the majority of conference attendees consider Canada to be a leader in IPY, with many citing the Government of Canada IPY Program's research and support as the main contribution to the overall IPY.

The Canadian IPY Program has contributed to other international IPY research programs and Canadian researchers have collaborated with international IPY research teams. For example, the Canadian IPY funded project, Pan-Arctic Tagging of Beluga Whales, collaborated with other international beluga tagging programs. Similarly, the DFO-led IPY project, Climate Variability and Change Effects on CHARS in the Arctic, proposed that the project would help to establish an international network of Char researchers. Furthermore, the NRCan-led IPY project, Environmental Change in the High Arctic from Snow and Ice Cores, also collaborated with researchers in Greenland to collect ice core samples from the last interglacial period, to provide further data on climate and contaminant change.^{Footnote 20}

More efficient approval of scientific and research activities in the North

It was noted in a previous section (see improved government regulation and improved timeliness of licenses processing) that the CARLI was developed to lead to more efficient approval of scientific and research activities in the North. Following an assessment in 2009 of northern research licensing application approval and reporting processes, a series of seven CARLI workshops were conducted in 2010 in the NWT, Nunavut and Ottawa. During these workshops northern regulators, Aboriginal/community organizations and researchers identified gaps, prioritized recommendations and discussed possible initiatives that could be implemented by 2012, and helped to define the objectives and eligibility criteria for CARLI-funded projects. Territorial research licensing regulators were given the opportunity to submit proposals for the development of territorial/regional research licensing retrospectives, web-based tools, guidelines and training material, and for workshops and meetings to discuss and deliver these initiatives.

Key informants reported that CARLI built some relationships among regulators, but that there was limited success at improving coordination and approval processes. Northern-based key informants said that while the intentions underlying CARLI were good, the process was flawed and the expectations were too optimistic, that the legal implications of changing processes controlled by regulations and the resources required were not adequately considered.

Despite the limited success of CARLI, key informants reported that the capacity of some research licensing authorities (e.g., Nunavut Research Institute, Aurora Research Institute) was increased. Some key informants described licensing of IPY research activities as "smoother" than in the past. Some key informants expressed significant concerns about capacity of Aboriginal organizations to participate in research licensing reviews, both in terms of people and expertise.

Long-term outcomes:

Increased understanding (impacts of a changing climate and health and well-being of northern communities) that informs policy and decision making, and contributes to recognition of Canada as an expert on the Canadian North

Experts consulted for the evaluation identified the connections among Arctic researchers internationally and the partnerships between Canada and other circumpolar nations as a major success of IPY. Almost all key informants believe that IPY has strengthened Canada's profile internationally as a leader in research on climate change and health and well-being of Northerners. It is noteworthy that Canada was selected to host the final, wrap-up IPY From Knowledge to Action Conference held in Montreal in April 2012.

Some evaluation findings show that IPY research results have influenced policy and operational decision making. Many key informants agreed that there have been instances at this relatively early stage of IPY results influencing policy or operational decision making. CHARS is a signature deliverable of Canada's Northern Strategy and was announced in 2008 as a legacy of the IPY Program. Other examples include the Government of Nunavut using unpublished data on dietary needs to inform health care delivery. Although, some key informants stated that it is too early to know the influence of IPY research on policy and decision making. An AANDC representative stated that the program should have done more to consider the knowledge-to-action component in the early stages as, in their view, it is not clear how the results of IPY projects have been passed on to policy makers. A few territorial government representatives said that IPY research has, as yet, had little or no influence on policy. A representative of a northern organization stated that policy relevance of research was part of the IPY guidelines but that it was an oversight in the planning for IPY not to include measures to evaluate this. However, we should note the challenges associated with determining the uptake of research results into government policy. Another AANDC representative said that the IPY conference will play an important role in examining the implications of IPY research for policy.

A review of Internet sources conducted as part of the literature review found that key IPY studies were published in international journals. For example, Egeland et al.^{Footnote 21} used information from the Inuit Health Survey for their article on food insecurity and the nutrition transition in Arctic communities in the Journal of Nutrition (United States). Results of the Inuit Health Survey also are quoted in the Arctic Monitoring and Assessment Programme publications, such as the one devoted to mercury^{Footnote 22}.

According to the IPY Highlights and Achievements Report (Government of Canada, 2010), the Canadian IPY Program demonstrated its "expertise and leadership on the international stage as eight Canadians led international IPY projects", as follows:

• Gilles Gauthier (Université Laval) - Impact of Climate Change on Tundra Wildlife;
• Don Russell (Yukon College) - Monitoring the Impacts of Global Change on Caribou and their Link to Human Communities;
• Manon Simard (Makivik Corporation) - Engaging Communities in the Monitoring of Country Food Safety;
• Greg Henry (University of British Columbia) - Climate Change Impacts on Canadian Arctic Tundra;
• Fred Wrona (Environment Canada) - Arctic Freshwater Systems;
• James Reist (Fisheries and Oceans Canada) - Climate Variability and Change Effects on CHARS in the Arctic
• Hayley Hung (Environment Canada) - Intercontinental Atmospheric Transport of Pollutants to the Arctic; and
• Sheila Greer (Champagne and Aishihik First Nations) - Kwaday Dan Ts'inchi Discovery - Expanding our Understanding of the "Long Ago Person Found".

The Canadian IPY Program was also a champion for the inclusion of a research theme on the human dimension of polar science. With the Human Dimension identified internationally as an IPY theme, key knowledge gaps on the vulnerability, resilience, adaptability and sustainable development of northern communities were filled, helping Canada to address the specific needs of the populations living in Canada's North.

Enhanced northern research capacity through newly trained scientists, knowledge and skills transfer to Northerners, and greater participation in planning and delivery of research by Northerners

As per the Government of Canada Program for IPY Highlights and Achievements, the science projects provided valuable research opportunities and training to nearly 1,200 students and early career scientists, including 215 students from Canada's North. A total of 18 northern research stations received IPY support to improve equipment and facilities, and provide training to Northerners through the logistic projects. These projects trained more than 80 Northerners as base camp managers, medical safety officers, technicians, field assistants and polar bear guides. The training and capacity projects brought the participation of approximately 85 students and new researchers, including 50 northern students.

Unexpected Impacts/Successes

Q 11. Are there any unexpected impacts of the IPY Federal Program, either positive or negative?

Impacts that were not expected in the IPY design were reviewed in the evaluation. Five such impacts were noted:

1. IPY elevated the status of AANDC in terms of its reputation and capacity for coordinating northern Science.
2. Because of IPY highlighting the community engagement aspect of research, and due to some successful projects in this area, northern communities will likely be more assured and insistent in the future on being able to participate in research planning and the research itself where possible, and on having research results shared. IPY has raised expectations in the North and among northern researchers for future science programs.
3. The Association of Polar Early Career Scientists (APECS) is an organization to mobilize student and early career polar science researchers that came out of IPY at the international level. APECS is an international and interdisciplinary organization for undergraduate and graduate students, postdoctoral researchers, early faculty members, educators and others with interests in polar regions and the wider cryosphere. The International Directorate Office is currently located in Norway. According to observers, the momentum generated through the Canadian IPY Program led to the fact that there were 10 Canadian representatives on the 2011-12 APECS Council (of the 36 members). APECS' aims are to stimulate interdisciplinary and international research collaborations, and develop effective future leaders in polar research, education and outreach. APECS achieves these aims by:
   • facilitating international and interdisciplinary networking to share ideas and experiences and to develop new research directions and collaborations;
   • providing opportunities for professional career development; and
   • promoting education and outreach as an integral component of polar research and to stimulate future generations of polar researchers.
4. The Polar Data Catalogue is a database of metadata and data that describes, indexes, and provides access to data sets generated by Arctic and Antarctic researchers. Data entries follow a standard format (Federal Geographic Data Committee) to facilitate exchange with other data centres. The search tool can be used by both researchers and the public. Since April 2011, the Polar Data Catalogue has an IPY data upload capability. Researchers can contact their IPY Data Assembly Centre Network coordinator for assistance to upload their IPY database. Furthermore, it brings together the metadata for other Arctic science with other programs, including NCP and ArcticNet.
5. The Canadian International Polar Year Publications Database describes publications from IPY 2007-2008 and the three previous IPYs. Funding to support the Publications Database was provided by the Government of Canada Program for IPY and Encana Corporation. The database includes publications from Canadian IPY projects, as well as publications from foreign IPY projects that studied northern Canada. The database currently describes 4000 publications. It includes a detailed search function and a link to the Polar Data Catalogue. The Publications Database has been adopted internationally.

Funding mechanisms:

The funding mechanism created some inefficiency. These challenges were inherent in the program design and were not related to program delivery. The movement of funds between different stakeholders (e.g., federal government departments and research funding agencies) were difficult because of different policies, rules and departmental procedures. Also, federal government departments were not able to use lapsing funding of academic partners due to financial administration rules, so some funds lapsed that might otherwise have been used to good effect. If funds could have been more easily carried over, there would have been more work done. Treasury Board funds went to NSERC, and the recipients of grants through NSERC could carry-forward the funding. However, NSERC could not fund government researchers.

As a one-time program without a track record, the financial management of the program was a challenge because managers had difficulties to re-profile funds to deal with any unexpected delays or changes. The program was too optimistic about the speed with which operations would be up and running and the money distributed. With the delays associated with the heavy workload to start the program and establish the IPY office, which had to be done before funding could begin, some research project funding was lost because the financial structure would not allow for money not spent to be re-profiled or rolled over the next year. Program managers believe that this type of program needs to allow for some creativity and flexibility in the distribution of funding.

Timing of resources:

The annual funding cycle is problematic for work in the North, which has a limited research season and timing constraints related to logistics (e.g., shipping supplies). Many key informants, including almost all funding recipients, did not agree that the timing of the funding was effective for achieving the intended outcomes. Some projects reviewed in the case studies had to arrange bridge financing from their sponsoring organization to conduct their work as planned. The March 31 deadline associated with project reporting and completion of contracts was not efficient for coordinating sea-lift transportation.

Stakeholder Perceptions about the Efficient Use of Program Resources:

Information provided by program managers and comments by some other stakeholders suggested that the governance processes were lean in comparison to the allocated resources. An objective of program managers was to keep administrative costs below five percent of total spending. IPY project approval processes were generally efficient and effective. This included the operation of the committees and sub-committees and the multi-phased process for IPY proposal evaluation, including scientific and social-cultural reviews. While having many positive aspects, the social-cultural review process was cited as problematic. The resources devoted to this process were adequate but, as been stated previously in this report, from the perspective of committee members, often there was not enough time to prepare for and to conduct the reviews due to delays in the distribution of materials to committee members. Some key informants, both within and outside the Department, stated that the IPY governance process was under-resourced in terms of administration, the social-cultural review process, and the reporting of results.

Many key informants from outside the Department also agreed that budgeted resources allocated to IPY activities were seen to be used efficiently in that the Government of Canada successfully leveraged resources from other sources. Examples of the application of IPY resources resulting in leveraging include academic institutional commitments for data management over the long term; other bodies or territorial governments using IPY-generated systems.

A typical comment was made by a representative of a northern organization who stated that "we got a lot out of IPY for the money." In terms of the adequacy and sufficiency of IPY funds, many agreed that IPY funds were adequate and sufficient.

External stakeholders, who generally agreed that the money was well spent overall, identified the following problems related to the efficient use of program resources.

• The Northern Coordinator position would have been more effective if stable, multi-year funding agreement had been provided to the NCOs for this position. Year-by-year funding led to high turnover in the Northern Coordinator position. Although, the Terms of Reference for the NCOs state that their term was from April 1, 2007, to March 31, 2011, and high turnover was only in Nunavut.
• It was noted that problems with conducting good community consultations is not unique to IPY, as this also is a challenge for NCP and ArcticNet.

Program design

Q 4. To what extent has the design effectively responded to the needs of First Nations, Northerners, Inuit, scientific community and government community?

A majority of key informants agreed that, overall, IPY was designed to be responsive to the needs of Aboriginal people, Northerners, the scientific community and Government. The remaining respondents were unsure, or felt that the IPY design responded to these needs only in part. Only a few thought that the IPY Program did not respond to the needs of these different stakeholders.

Aspects of IPY governance was modelled on the Northern Contaminants Program, which was structured to provide a strong voice for Northerners. Federal government representatives and researchers serving on IPY committees reported that an Aboriginal and northern focus was important from the start of IPY, that projects were selected both on the basis of science and northern relevance and, that there were deliberate efforts to put a structure in place to include northern voices. They noted that Canada was the only country participating in the IPY to take this collaborative approach. Most project leaders and northern-based key informants also agreed that much of the research met northern needs, especially research projects conducted by Northerners or involving them directly (e.g., putting Inuktitut language on GPS, sea ice research, contaminants in animals). Some important projects such as the Inuit Health Survey responded directly to Northerners' needs.

Key informants who thought that the IPY Program did not respond to Aboriginal and northern needs or responded only partially expressed concerns that not enough of the IPY research was useful or practical for the North in terms of addressing the impacts of climate change on infrastructure and food security. It was also noted that, although IPY was designed to be a northern and Aboriginal-based program, there was not sufficient appreciation of the limitations in the capacity of Aboriginal and northern groups to respond to the call for proposals. The Training and Capacity Building and Communications and Outreach components helped to address this need. This component was described as more "northern friendly."

Q 20. To what extent does the planning process ensure that appropriate decision making is on line with program objectives?

Program managers and many other stakeholders believe that the program parameters and project criteria were comprehensive and that the proposal evaluation criteria were flexible enough to ensure that all projects could meet them, either as originally submitted or with some modification. Managers stated that the need for proposal modifications, particularly for northern-led projects, was anticipated by the IPY Program and that mechanisms were put in place to deal with this (e.g., adequate time to respond, opportunities to provide supplementary information to proposals).

A majority of key informants agreed that IPY attracted high quality projects and proposals. They also agreed that IPY was accessible to eligible applicants. It was noted that IPY was a departure from standard procedure, and had provided the opportunity for academic and government researchers to compete on equal ground.

Some stakeholders perceived that the proposal and evaluation criteria limited access to northern-led projects. The following reasons were cited by most of these stakeholders for having this perception:

• the requirement for international partners (a criterion that was included for the projects to be recognized internationally);
• insufficient process to build in northern priorities;
• perceptions that the IPY Program was very academic, with research design criteria that were too onerous for most prospective northern applicants;
• perceptions that, depending on the respondent, there was too great a focus on the Eastern Arctic, or on coastal areas;
• implementation happened so quickly – the proposal process, administration mechanisms, etc. -- that some projects to address community needs could not be considered; and,
• communications with Northerners was not adequate at the beginning, which resulted in few proposals from Northerners.

Q 21. To what extent are the roles, responsibilities and accountabilities of key players clear and well understood?

Consultations conducted for the evaluation found widespread agreement that IPY roles and responsibilities for AANDC and the horizontal partners were clear and understood by stakeholders. Some key informants said that the roles and responsibilities of communication could have been clearer as it imposed inefficiencies and led to disagreements over communications priorities.

Q 19 and 25. Did the governance mechanisms function as intended? Was the governance structure program design the most effective approach for achieving the program objectives?

Overall, the IPY governance structure was recognized by most key informants as effective and with well-qualified representation on the various committees. It is widely recognized that IPY was a large-scale and complex undertaking that was delivered in a short duration, with limited administrative capacity. The program modeled aspects of its governance structure after the NCP, which was recognized as a proven, effective governance model. For the most part, the IPY governance mechanisms functioned as intended. The main identified shortfall was in the limited capacity for First Nations, Inuit and Northerners to provide input through the Social/Cultural Review process due to time constraints. This was largely a result of the timelines under which the Canadian program was being delivered, as it had to move ahead very quickly to meet the March1, 2007, start date of the official observing years of IPY, which constrained preparation and review time for the Social/Cultural Review.

The horizontal ADM Committee for IPY, set up as the decision-making body for the IPY Program, was responsible for ensuring accountability. The ADM Committee also provided a high-level reporting structure for the program. Program managers stated that horizontal structures are not common in government and that the success of this committee is noteworthy. On the other hand, it sets IPY apart from other scientific research programs due to the extensive involvement of the ADMs in setting direction.

A few key informants stated that this role might not be appropriate as ADMs are not experts in northern research, and that more direction should be taken from scientific and social/cultural reviews. However, the IPY Program Subcommittees and Review Panels provided advice and recommendations to the ADM Committee in all areas, including: science; data management; logistics, infrastructure and emergency preparedness; research licensing; and training, communications and outreach.

A few key informants noted that most committees in the governance structure included the appropriate representatives. For example, a few stated that the review committee for Climate Change Impacts and Adaptation had wide representation of members who were knowledgeable and objective. In addition, a few key informants noted that the scientific peer review was rigorous and followed appropriate protocols to avoid conflict of interest. This is a particular concern within northern science research programs as there is a relatively small pool of researchers qualified to complete peer reviews.

Key informants stated that the main factor that would have improved governance effectiveness would have been to allocate more time and human resources to administration and proposal review. It should be noted that key informants recognized that timing was not entirely under the control of the IPY Program.

Another suggestion by key informants for improving the effectiveness of the IPY governance process was to clarify the proposal detail requirements. In some circumstances, project proposal instructions could have been more precise to ensure consistent applications. For example, applications contained varying levels of detail about the expected cost of logistics. Some applicants provided clearly itemized expected logistics costs, while others were not precise. Some reviewers reported that this was a limiting factor in validating the overall value of the proposed projects.

As previously mentioned, the timing of the IPY implementation was rushed and additional preparation time would have improved the effectiveness of the process. One of the consequences of the timing was that it compressed the application time for potential applicants, making it challenging for some to secure international collaborators for projects. However, key informants noted that, considering the timeframes, the program did a good job in meeting the needs of committees with supporting information (e.g., scientific review committee). A few key informants questioned whether the compressed timeframe limited an objective review of all project submissions, compromising a process that was designed to ensure that the best possible projects received support. Committee members reported that projects were generally aligned with the program objectives, although a few were overlapping both program objectives (e.g., some contaminants projects).

The design of the NCOs was appropriate and they provide a good model to follow in the future as, according to northern-based respondents, the regions have a better understanding than southern managers of what is needed and of how to accomplish objectives. Also, according to personnel working in the NCOs, the NCOs component could have been better structured. The NCOs were dependent on year-to-year funding agreement and working through AANDC slowed things down and created some tension with Headquarters.

Program delivery:

Q 23. Was the Government of Canada program for IPY implemented as intended?

Q 10. To what extent have external and internal factors influenced the IPY Federal Program?

Key informants stated that IPY and the approach of scientists responsible for projects evolved through delivery to give greater consideration to northern and Aboriginal needs.

The program timelines and time management were a challenge that had an impact on the delivery of IPY activities. The Government of Canada Program for the IPY had to start very quickly to meet the March 1, 2007, start date of the international program. Putting the resources in place to meet this start-date was a challenge since the funding was not officially in place until authorization was received from the Treasury Board.

The program relied on the different federal government departments to support the effort and the departments paid the salaries of individuals assigned to the program for the first year. Once Treasury Board approval for the program was received, the salaries of IPY personnel were paid by AANDC.

The Communications and Outreach and Data Management components were challenged by a delay in flow of funds.

It is important to note that Canada was ahead of other countries operating under the same time constraints in program implementation. Program managers noted that some program activities were phased and that delays were by design. Managers also identified some obstacles that help explain the delays, including the timelines imposed by the international program, and a large number of proposals received that created a large workload with insufficient time to recruit resources to accomplish the work.

Almost all key informants thought that the Northern Coordinators made useful contributions to the delivery of the program in the North. Northern Coordinators fulfilled a very important role in IPY by reaching out to communities and by generating interest in research, activities which differentiated IPY from other research programs. Northern Coordinators facilitated communications between researchers and communities. Researchers appreciated that they could access the coordinators directly. Situating Northern Coordinators within host organizations, outside of AANDC and regional office programs, was identified as a good feature, as effective, and as contributing to community engagement. The arms-length, Northern Coordination Offices were seen as independent of the federal government, which increased their effectiveness with respect to community engagement.

Most key informants indicated that IPY attracted high quality proposals and that the process was accessible to eligible applicants. Program managers reported that proponents typically were given four months to submit their proposals, following a year of notice that the Call for Proposals would be coming. Nevertheless, some Aboriginal organizations and territorial governments disagreed that the application process was readily accessible, particularly in relation to the amount of time available to respond to the proposal call. A few key informants said that there was insufficient time to address northern barriers adequately and that the North was not ready to engage and realize the full benefits of the program.

Program managers reported that they recognized the limited capacity of some Aboriginal and northern organizations to respond, and that measures were put in place to improve their opportunities to succeed. Deliberate efforts were made by the program, where needed, to strengthen northern-led projects. Support was provided to northern-led and other projects, where needed, to improve key elements such as the rigour, community input, communications and outreach. Some project approvals were also delayed to ensure compliance with requirements for northern participation.

Northern-based key informants tended to agree only partially that the program was implemented as they expected. Following are some of the reasons they provided for their judgements about implementation:

• It was very difficult for communities to get involved in the first Call for Proposals.
• The timelines were always too tight to have meaningful input (e.g., members of a social-cultural review committee received paper copies of large proposals only a few days before the review, which made it impossible to conduct a thorough review).
• Some IPY projects did not go through a social-cultural review. Often, project proposals sent to Aboriginal organizations for review were returned with little comment. The proposals tended to be complex and technical in nature and the timing of requests were viewed as unrealistic by a substantial minority of northern respondents. "More thought needs to be given about what is realistic for getting feedback from northern peoples."
• The social-cultural review component was excellent - Northerners got to sit at the table and express their views - but often the process was not followed (for reasons noted above).
• More people from the North could have been hired for various IPY functions.
• Program expectations about the CARLI were unrealistic. Licensing boards were not going to change their processes and the streamlining of scientific research licensing processes was never going to succeed in the context of an initiative such as IPY.

All researchers were required to undertake public engagement activities to communicate with local communities and members of the public on their research and key results. Key informants commented on the lack of clear guidelines given to researchers to assist them with these communication activities, leading to varying levels of effort and success. In addition, a number of film projects were funded through IPY and represented a major resource for communicating about IPY and Arctic science to the public. If it had been possible to secure the rights for the Government of Canada to show these films for non-commercial purposes as part of the initial contribution agreements, the films could have been used more extensively as part of federal outreach efforts.

Program committees
Q 24. Is the project selection process effective?

Some committees (i.e., Training, Communications and Outreach, Data Management) were not implemented as planned, and this was viewed by several project leaders as reducing the thoroughness of the review process. However, the evaluation did not obtain specific information about the impacts of the reduced number of committees on overall project quality.

Aboriginal and non-Aboriginal Northerners were engaged through the Social-Cultural Review Committees. However, the rapid implementation of the IPY meant that the committees were not established in time to facilitate in depth engagement in identifying research needs and reviewing proposed research projects.

Inputs from the Social-Cultural Review Committees were factored into the science review process. Comments from the regional reviews were included as part of the packages for the Science Review Subcommittee, which reviewed the IPY science proposals and provided recommendations to the ADM Committee on IPY. Representatives from the Social-Cultural Review Committees were also included as members of the Science Review Subcommittee. Nevertheless, the social-cultural review process was viewed by many key informants in the North as not being influential. They reported that projects not supported by northern committees often emerged as successful, and that there was a tendency not to hear back about how northern concerns were addressed. Territorial government representatives said that the social-cultural review was well intended, but the good intentions were not realized. Program managers noted that feedback from the social-cultural review and Science Review Subcommittee was available to applicants upon request.

The proposals that were received were quite diverse. Some were community-led, others did not consider community involvement. A few key informants stated that the selection criteria for the lead investigator was stringent and disqualified most northern community members from that role, creating a barrier for community-led projects. A few key informants also reported that science-oriented projects had the priority. The Social Cultural Committee looked for projects with added-value to the communities, with training and involvement/partnerships with community members, and/or with positive long-term impacts for the communities. Ultimately, key informants stated that there were good projects selected, however, very few IPY projects worked with communities through the research process and in the reporting of results.

Administrative Requirements
Q 22. Are the administrative requirement (applications and reporting) proportional to the level of funding provided?

A majority of key informants agreed that IPY administrative requirements such as the application process, project selection and reporting were appropriate relative to the level of funding that was provided and the kinds of activities being undertaken. Researchers generally agreed that the administrative processes were straightforward, that the transfer of funds was convenient, that the peer review approach was standard procedure for research and that it was "a fair process by and large." The only common problem was the timing of the release of money, which was late for many proponents, requiring some to arrange bridge financing, without which they could not have done the work.

Performance measurement - accountability:
Q 26. To what extent have performance measures been established, gathered and used for the IPY Federal Program?
Q 27. Are there monitoring and reporting measures in place and appropriate? Do they provide timely and useful information?

A Performance Measurement Framework was put in place at the beginning of the program. In November and December 2007, representatives from the six lead government departments developed and refined a suite of performance measurement indicators for the IPY Program. Some departments did not provide reporting on certain performance measurement indicators. However, the vast majority were obtained through the annual progress reports and the horizontal DPR. Using the information collected through the annual progress reports, Highlights and Achievements from the Government of Canada Program for IPY was published in 2010.

Many key informants did not think that appropriate mechanisms were in place to monitor the performance of IPY. For example, a representative of a northern organization involved with IPY said that no clear criteria were established at the beginning to measure if the objectives were achieved. Northern Coordinators noted that they were asked to review project reports from a community engagement perspective, but they were not able to communicate with communities due to privacy agreements with researchers. Consequently, they had no way to substantiate the contents of the reports.

The Governance Review conducted for this evaluation found that internal monitoring processes appeared to be appropriate (e.g., with interim/annual progress report and final project reports) and that appropriate project monitoring and reporting processes generally were in place.

Key informants, including project proponents and managers, reported that the administrative requirements were as expected by most stakeholders and that they were proportional to the level of funding provided.

Although monitoring systems were robust, some respondents felt that the reporting requirements were onerous. It was noted that the across-the-board reporting requirements were onerous for small projects. For example, the reporting requirements did not vary by size of project, therefore, small projects (e.g. $20,000) had the same requirements as large projects ($1 million dollars), which meant that smaller projects spent a disproportionate amount of time/resources on reporting. It was suggested that a tiered-approach to monitoring would have addressed this problem without compromising accountability.

Key informants suggested that additional effort is required to translate research findings and to transfer them back to communities. In 2012, IPY updates for four northern regions (Yukon, Northwest Territories, Nunavut and Nunavik) were put on the IPY website, distributed in each region and at the IPY 2012 Conference. In addition, the International Polar Year Canadian Science Report: Highlights was published by the Canadian Polar Commission and the IPY Science Report will be published in fall 2012.

Lessons Learned

Three factors were identified as important lessons through the IPY Program that could be applied to achieve success in future programs: collaboration and partnership, good coordination, and the funding approach:

Collaboration and partnership - Important aspects of IPY research success depended on meaningful partnerships between scientists, northern and Aboriginal peoples and organizations, and research sponsors/funders. For example, the Inuit Health Survey was cited a number of times as a very successful example of a large-scale study that was based on identified needs, that engaged Aboriginal and non-Aboriginal Northerners, that built capacity for research in northern communities, and results of which have already influenced Nunavut government policy with respect to health care delivery. Having stakeholders involved over the long term is also key for success in the North. Because IPY implementation had to be accomplished within a very compressed time period and there was a risk of losing the first season in the field, some projects moved too quickly to have time to establish adequate partnerships with Northerners. Key informants emphasized that there is a need to develop relationships well in advance of the next 'pulse" of research funding. These relationships should involve:

• collaborative approaches to establish an appropriate research agenda;
• mechanisms and funding for establishing collaborative research teams that include academics, northern institutions, northern scientists and research assistants and community liaison; and
• Aboriginal consultations on projects to facilitate licensing processes.

Based partly on the experiences of IPY, the success of CHARS will likely depend in large measure on how it resonates with Aboriginal and non-Aboriginal northern peoples. In this respect, two of the principles for CHARS include promoting partnerships and collaboration among the private, Aboriginal, academic, and public sectors both domestically and internationally, and working with Aboriginal peoples of Canada's Arctic and recognizing the importance of traditional knowledge in advancing Arctic research.

Coordination and Partnership - The synergies between AANDC and NSERC in the IPY Program contributed to its success. NSERC adopted the IPY priorities and there was an alignment of reporting requirements, and the experience with this kind of alignment among partners is viewed as something worth repeating in future initiatives of this kind. Furthermore, IPY used NSERC mechanism to facilitate academic funding.

In terms of coordinated planning of science initiatives such as IPY, the territorial governments and some northern communities found that they were not adequately prepared to participate in the early stages of IPY because they had not developed their own research agendas and priorities. The experience with IPY highlighted the lack of an ongoing science intergovernmental coordinating committee such as exists in other substantive areas.

Funding approach - IPY represented a peak period of scientific activity – a 'scientific rush' – to be followed by a period of much lower levels of activity. Some key informants advised that more consistent levels of funding for scientific research - even if at lower levels - are more likely to create and sustain the next generation of scientists and more likely to maintain and help augment northern science capacity. As well, northern participants in IPY have pointed to difficulties in developing northern-based projects that could meet the high scientific and technical standards of IPY. In order to increase the involvement of northern communities and organizations and encourage northern-based project development, future research funding may have to designate funding streams to address a broad range of priority areas, possibly with variable proposal and project evaluation criteria in the different streams.

Other lessons learned included the following:

• Funding mechanisms - The added complexities related to conducting scientific research in the North, particularly when multiple stakeholders and funding agencies are involved, mean that the funding mechanisms and annual funding cycle are not as effective as they should be. The IPY Program experience revealed that a new, more flexible funding model for multi-stakeholder projects conducted in the North is required.

Best Practices

Governance structures - The governance structure for the IPY Program, including the horizontal ADM Committee, was instrumental in avoiding or addressing problems related to the involvement of multiple stakeholders and funders (e.g., maintaining accountability for departmental funds allocated to other program stakeholders/funders). Given the complexities and potential pitfalls in managing a horizontal initiative such as IPY over a tight time frame, a high-level committee of this type, accountable for program effectiveness, appears to offer a model that could be transferable to other horizontal initiatives.

Northern Coordination Offices - The NCOs contributed in tangible ways to effective program delivery. The independence of Northern Coordinators, situated outside of AANDC and regional office programs, fostered greater community engagement and generated northern public interest in research. The high levels of cooperation between the Northern Coordinators and northern organizations also contributed to community engagement and collaboration between communities and researchers.

Complementary scientific and social-cultural project reviews - The social-cultural review process, which complemented the scientific review of projects, contributed to the success of the IPY funding process. Although the social-cultural reviews did not always unfold as planned - late timing and receipt of materials by committee members sometimes limited effectiveness and there were problems due to the limited capacity of northern communities and committee members to review highly technical material - they provided a valuable mechanism for incorporating the views of Northerners and for considering the broader impacts of research in the North.

Data Management - Canada has emerged from IPY and other federal scientific initiatives such as the Northern Contaminants Program as a leader in data management. The percentage of the IPY budget allocated to data management was approximately five percent - $7.0 million - which was much higher than the proportion allocated by other countries. Of the total funding for data management, $2.4 million was allocated to data assembly and archiving, $2.4 million for ensuring access to IPY data, and $1.2 million to supporting data management. Data management was part of the program's annual progress reporting. Data-targeted funding allowed for the creation of new data archiving systems, and the resulting infrastructure has become an operational model including the likely development of a national multi-sector data base.