Archived - Evaluation of the Northern Contaminants Program and Northern Scientific Training Program

Relevance

The evaluation examined the continued need for the NCP and NSTP. For NCP, continued need would be based on the ongoing presence of northern contaminants, including their extent, severity of contamination, and bioaccumulation, as well as the continued existence of contaminant-related health issues and the need for research and ongoing monitoring to address these issues. For the NSTP, continued need for the program is based on the availability of other options for funding student research in the North, evidence of continued need for research support for students, and the perceptions of stakeholders and recipients about the importance of fostering interest and capacity among Canadian science students related to northern research.

There is a strong consensus among all stakeholders that there is a continued need for the NCP. NCP-funded research has demonstrated that levels of some contaminants have decreased while others have increased over the last two decades. The emergence of new types of contaminants and new domestic and international sources of contaminants also make the situation dynamic. Information from NCP research, which was identified as the only continuous source of funding for contaminant research in the North, was identified as critical for understanding the nature, extent and location of contaminants, and for studying the linkages between contaminants and human health.

Similarly, all stakeholders believe there is a continued need for the NSTP to support student researchers in the North, both to ensure that needed research is conducted and to develop future northern research specialists.

The evaluation examined the extent to which the NCP and NSTP are aligned with federal government and AANDC priorities and, for the NCP, the alignment with the priorities of other federal departmental partners. This component of the evaluation was conducted through a review of Government of Canada documents and policy statements and the perceptions of stakeholders.

The NCP is found to be consistent with the Government of Canada's four-point northern strategy, including "protecting the northern environment", as outlined in the 2011 Speech from the Throne. The program is also consistent with Canada's Arctic Foreign Policy, which states that strong environmental protection is an essential component of sustainable development. The NSTP supports AANDC's sustainable development strategy, contributes to economic development within the North, and supports investigation into strategic priorities, including the impacts of climate change.

The evaluation reviewed the extent to which the NCP and NSTP are aligned with AANDC's statutory mandate (and with the mandates of other departmental partners for NCP) and the extent to which federal government efforts duplicate or complement other research initiatives. Stakeholder perceptions regarding program alignment with federal roles and responsibilities were also examined.

There was a consensus among stakeholders that the NCP is consistent with federal government responsibilities to track long-range, trans-boundary airborne pollutants and that human health studies are valuable and appropriate complements to the environmental studies. Similarly, the NSTP is widely viewed as being consistent with Government of Canada responsibilities. Both NCP and NSTP initiatives are considered to be complementary to initiatives funded through other agencies and there is no evidence of overlap or duplication. AANDC is considered to be the most appropriate department to host the NCP because of the Department's focus on the North and responsibilities to northern communities.

Performance-NCP

The evaluation addressed the issue of the extent to which NCP activities and outputs have contributed to the expected outcomes. NCP outcomes include northern engagement in program activities, creation of new knowledge, more informed decision making by Northerners based on greater awareness of contaminant impacts on human health, research results made accessible to Canadians and internationally, increased capacity of northern scientists and northern organizations to conduct research, and improved health and reduced risk to northern communities and ecosystems.

Immediate outcomes: The NCP-funded initiatives have increased public awareness of contaminant issues in the North, although project follow-up and communication of results have been less consistent and successful despite considerable program effort. NCP has led to increased northern research capacity by requiring engagement of Northerners as a condition of funding, requiring a capacity-building component in NCP project proposals, involving and providing financial support to Aboriginal organizations to participate in program governance, providing support through Inuit Research Advisors and other mechanisms, involving Northerners and incorporating traditional knowledge into the research process, and various other measures. However, despite NCP efforts to foster input through Regional Contaminants Committees, there is a widely-perceived need among Northerners for more meaningful northern input on priority setting, and new mechanism to support the ability of communities to develop proposals that meet scientific standards. There is a consensus among stakeholders that the NCP has made a substantial contribution to the development of new contaminants data, information and knowledge since 2003-2004, including information and knowledge related to the impacts of contaminants on human health and ecosystems. NCP research results have been the basis of numerous Canadian and international publications.

Intermediate outcomes: There have been improvements in the engagement of Northerners and northern communities in contaminants research and in decision making informed by this research. Several program mechanisms have proven to be effective in involving stakeholders in program planning and decision making, including Regional Contaminant Committees, the blueprint process, and consensus-based decision making. Based on program experience with reaching target audiences, the program approach has evolved to include front-line personnel who provide community members with information on contaminants. NCP funding has been an important factor in decisions of researchers to conduct northern research. The NCP has made important and ongoing contributions to Canadian and international regulations and agreements, including the United Nations Economic Commission for Europe Convention on Long-range Transboundary Air Pollution and the Stockholm Convention on Persistent Organic Pollutants. The NCP continues to be Canada's main contributor on contaminants to the Arctic Council's Arctic Monitoring and Assessment Programme. Joint work is conducted regularly between NCP and Arctic Monitoring and Assessment Programme, including contaminant assessments and engagement with international bodies such as the United Nations Environment Programme. The results of NCP-funded research are widely and readily available, both nationally and internationally.

Long-term outcomes: Health risk reduction benefits resulting from NCP-funded research have included identification of contaminant levels (some have increased and some have decreased), development of Canadian and international protocols on the use of toxic substances, and health advisories based on findings of NCP research. NCP-funded research has led to increased capacity of some organizations to conduct and support research and to review contaminant related information. Most stakeholders reported that the NCP has had an impact on the development and implementation of domestic and international policy and regulation.

Performance-NSTP

The evaluation examined the extent to which NSTP activities and outputs have contributed to the expected outcomes. NSTP outcomes include: increased opportunities for students to experience and learn about the North; an increased number of northern specialists committed to northern studies; a strengthened commitment to northern studies; a stronger scientific community involved in the North; and training of the next generation of scientists.

Results show that program funding has increased the opportunities for students to experience and learn about the North. From 2003 to 2011, 2,891 students received NSTP funding and approximately 92 percent of the students surveyed said NSTP influenced them to carry out research in the North. Furthermore, all students surveyed will have their research published. During the period from 2003 to 2011, 35 percent of the students who received funds were doing research in physical sciences, 44 percent in life sciences, 19 percent in human sciences and two percent in health sciences. Without NSTP funding, many students would not have gone to the North to participate in research projects. Some estimates from former participants are that a majority of experienced researchers engaged in northern research have been NSTP recipients, although the evaluation could not confirm those numbers.

Introduction

Effective engagement with Northerners has been a priority for the NCP since the program began in 1991. As indicated in this evaluation, NCP has set the standard for northern engagement in Arctic science programs and activities, and significantly influenced the community engagement element of programs such as International Polar Year and ArcticNet.

To ensure the priorities and needs of Northerners are addressed in all program decisions and funded activities, numerous initiatives are currently in place. For example, the involvement of territorial and other northern governments, Aboriginal organizations and Regional Contaminants Committee representatives on the Management Committee, as well as the membership of the Regional Contaminants Committees themselves in five northern regions, bring a broad range of Northerner perspectives into the program’s decision-making processes. Also, NCP-supported Inuit Research Advisors in four Inuit regions and AANDC staff in Whitehorse, Yellowknife and Iqaluit further assist with engaging Northerners and communicating NCP findings. A separate funding envelope, the Community-Based Monitoring and Knowledge Integration fund, was established specifically to address community-based issues. As well, the Communications, Outreach and Capacity-building funding envelope is led by Northerners / community representatives.

NCP Reporting Requirements

All NCP-funded projects are required to report their progress on an annual basis in the format of both a report (published each September in the Synopsis of Research report) and a poster/presentation at the annual NCP Results Workshop. The reports must include information on the project’s engagement of Northerners through training and capacity building initiatives, communications and outreach activities, and incorporation of traditional knowledge. As part of the reporting requirements, project leaders also provide the following statistics in their annual reports: number of Northerners engaged; and the number of meetings/workshops in the North.

As outlined in this evaluation, the NCP conducts major initiatives with a goal to foster information sharing and collaboration. For example, the NCP holds an annual Results Workshop, publishes an annual summary report of key results that are made public and utilizes the web to inform partners of the Call for Proposals process.

NCP Project Proposal Criteria

Project proposals are evaluated against the following criteria as part of a Social/Cultural review by the Regional Contaminants Committees to ensure effective engagement with northern communities:

1. An effective communications plan covering the period prior, during and after the project.
2. Relevance to northern priorities.
3. Opportunities for training and capacity building.
4. Appropriate use of traditional knowledge.

The Social/Cultural review entails a review of progress to date and plans for the upcoming year and also provides an opportunity for Regional Contaminants Committees to provide feedback to applicants for improvement. The Social/Cultural review of proposals contributes to ensuring proper engagement of Northerners – in the review process, as well as in projects that ultimately get funded.

The NCP's requirements for engagement of Northerners in research and monitoring projects and related communications activities, is backed up with guidelines and appropriate checks and balances. Program expectations are laid out in the NCP Guidelines for Responsible Research and other program documents, such as the Call for Proposals; the required "Approval of Consultation" forms provide confirmation from community organizations that consultations are proceeding satisfactorily, and provide another opportunity for feedback to the program and its funding recipients.

1.2.1 Background, context and program rationale

NCP was established in 1991 in response to concerns about human exposure to elevated levels of environmental contaminants, namely persistent organic pollutants (POPs) and metals such as lead, cadmium and mercury. Early studies found a wide variety of substances, many of which had no Arctic or Canadian sources, but which were, nevertheless, reaching unexpectedly high levels in the Arctic ecosystem. Though far removed from the industrial centres that generate pollution, the Arctic forms a sink for a class of pollutants that arrive by long-range transportation through the atmosphere, oceans and rivers. Arctic biological systems then biomagnify contaminants resulting in high levels of dietary exposure for upper trophic level wildlife and people that rely on these animals for subsistence and cultural, economic and spiritual purposes. Research and monitoring shows that exposure to certain pollutants at elevated levels presents a greater risk for a range of human health problems, including developmental disabilities in infants and children.

The first phase of the NCP (1991-1997) was intended to gather the data required to determine the levels, geographic extent, and source of contaminants in the northern atmosphere, environment and people. Data collected during this stage improved the understanding of the spatial patterns and the trends of contaminants in the North. Further, data confirmed that the major sources of contaminants were other countries. As a result, transboundary pollution became an important element in the assessment of human health risks resulting from contaminants in traditional foods.

The second phase of the program (1998-99 to 2002-03), continued research and monitoring on the health benefits and risks of consuming traditional/country foods; developed effective community communication; and continued work on supporting international agreements to control contaminants. The incorporation of traditional knowledge of northern Aboriginal peoples into northern contaminants research became increasingly important during Phase II. Northern Aboriginal organizations played an important role in building awareness and an understanding of contaminants.

Based on information gathered through the Arctic Monitoring and Assessment Programme – the circumpolar environmental monitoring program of the Arctic Council of which the NCP is a key contributor – three international protocols have been implemented to regulate the production, use and release of POPs and heavy metals. These include the United Nations Economic Commission for Europe Convention on Long-range Transboundary Air Pollution Protocols on POPs and Heavy Metals (1998) and the global United Nations Environment Programme Stockholm Convention on POPs (2001). These conventions recognize that the presence of a given chemical in the Arctic environment is evidence of long-range transportation. In addition, a global monitoring plan on POPs was produced under the Stockholm Convention to partly satisfy the requirement of Article 16 of the Stockholm Convention to evaluate the effectiveness of the overall convention. The NCP continues to play an instrumental role in meeting Canada's obligations under the Stockholm Convention through contributions to the Global Monitoring Plan and Article 16, in addition to providing critical new information which is used to add new POPs to the overall convention.

The next phase of the NCP is now under way and aims to build on the findings of phases I and II. The current focus of the program is to address high priority issues in communities in the three territories, Nunavik and Nunatsiavut where people are being exposed to contaminant levels that are of concern to health authorities. The NCP aims to address these issues through: conducting research and monitoring to enable the provision of sound dietary advice; meeting Canada's monitoring obligations under international agreements (Long-range Transboundary Air Pollution Protocols on POPs and Heavy Metals, as well as the Stockholm Convention); contributing data and information for the addition of new POPs to existing international agreements, and the development of a new, legally binding global agreement on mercury by 2013 under United Nations Environment Programme; and undertaking education and communications efforts in these high priority areas.

Funding for the NCP's annual budget currently comes from AANDC ($4 million per year in addition to in-kind support through salaries and operations and maintenance) and Health Canada ($800,000 per year). In addition, AANDC, Health Canada, Environment Canada and the Department of Fisheries and Oceans provide in-kind support based on their mandates, which collectively cover improving human health and the environment and conducting health, environmental and oceanographic science. The supporting AANDC authority is Contribution for promoting the safe use, development, conservation and protection of the North's natural resources.

1.2.2 Program objectives and expected outcomes

The key objective of the NCP is to: "work towards reducing and, where possible, eliminating contaminants in traditional/country foods, while providing information that assists individuals and communities in making informed decisions about their food use."^{Footnote 1} This key objective contributes to the North strategic outcome of well-being for the people and communities of the North.

The key immediate, intermediate and long-term expected outcomes flowing from this objective are listed below. The relationship between these outcomes and other activities and outputs is illustrated in the program logic model (Annex A). Note that several of the outcomes outlined in the logic model have been combined below to streamline analysis:

Immediate outcomes

• Northerners and northern communities are engaged in NCP activities and develop an increased capacity to participate in contaminants research
• Creation of new data, information and knowledge related to impacts of pollutants on human health and ecosystems in the North through culturally-sensitive research

Intermediate outcomes

• Through greater awareness of nutrition and contaminant issues, Northerners and northern communities make informed decisions related to their food use
• World-class, innovative research results and information are made accessible nationally and internationally
• Contribution to the development and implementation of domestic and global regulations and agreements to reduce and/or eliminate the production, use, and release of contaminating substances into the environment

Long-term outcomes

• Increased capacity of northern scientists and northern organizations to conduct culturally-sensitive research and address environmental health issues
• Improved health and reduced risk to northern communities and ecosystems, as a result of reduced contaminants levels
• Increased understanding of the impacts of long-range contaminants on the health of people and ecosystems in the North

1.2.3 Program activities

In support of its objectives and expected outcomes, the NCP funds research through five subprograms in order to collect data on the presence of contaminants and determine the effects that these contaminants have on the health and safety of consuming traditionally harvested foods:

• Human Health Research, in the areas of exposure assessment, epidemiology, toxicology and benefit/risk management, to better assess, understand and manage the health risks in northern Canada related to the long-range transport of contaminants.
• Environmental Monitoring and Research, to monitor contaminant levels and trends in Arctic air and biota, provide early warning support to the assessment of human health, and support reviews of the effectiveness and sufficiency of international agreements.
• Communications, Capacity and Outreach to ensure that individuals and communities in the North receive the information needed to assist informed decision making in their food use.
• National/Regional/International Coordination and Aboriginal Partnerships, to ensure the coordination of program activities and that appropriate communications and participation occur with respect to northern communities.
• Community-based Monitoring and Research, established in 2010, to increase community-based capacity to conduct research and monitoring on contaminants that is both community-led and driven by community interests and priorities.

The NCP Secretariat manages the operation of NCP. Activities associated with program management include: leading and participating in domestic and international northern scientific assessments and other science initiatives; issuing an annual Call for Proposals and coordinating a review process; allocating funds for northern research, monitoring and related activities to academic individuals and organizations, other federal and territorial government departments and Aboriginal organizations; contributing data and expertise to international agreements; developing ongoing publications and annual reports; communication, consultation and outreach activities; and actively engaging Northerners and northern communities in the delivery of the program.

1.2.4 Program management and governance

The issue of Arctic food chain contaminants is one that is multi-jurisdictional, being of concern to federal, territorial and Aboriginal governments; and, it is also one that can only be addressed through cooperation from the international community on a global scale. Addressing northern contaminants issues and the concerns requires information that meets internationally acceptable scientific standards, but research must also respond to the needs expressed at the community level, and at the level of individual consumers of traditional/country foods.

The multi-disciplinary nature of the NCP has allowed it to develop structures and strategies to address such scientifically and politically complex issues. A partnership approach forms the basis of NCP's management process. This encompasses representatives who speak to the key areas of Arctic contaminants research based on an ecosystem approach; northern community concerns, needs and priorities; and the international and domestic agendas for the control of toxic substances.

The NCP is governed on a partnership model that brings together federal departments, Aboriginal and territorial stakeholders, and experts in northern research in a series of inter-related committees, as described below.

NCP also works closely with other Government of Canada, regional, national and international northern science programs to promote coordination and cooperation in arctic science. These include ArcticNet, International Polar Year, Arctic Monitoring and Assessment Program, the Canadian High Arctic Research Station, and others. Input from other programs is often sought to facilitate coordination with other northern science programs.

The NCP Management Committee

The NCP is directed by a Management Committee that is chaired by AANDC, and which includes representatives from four northern Aboriginal organizations (Council of Yukon First Nations, Dene Nation, Inuit Tapiriit Kanatami and Inuit Circumpolar Council – Canada); the Yukon, Northwest Territories and Nunavut Territorial governments; the Nunavik Nutrition and Health Committee; the Nunatsiavut Government; and four federal departments (Environment Canada, Fisheries and Oceans, Health Canada, and AANDC). ArcticNet became a formal member of the Management Committee in 2011.

The Management Committee establishes NCP policy and research priorities; facilitates the implementation of the NCP through managing partnerships; makes final decisions on the allocation of funds; and reviews the annual progress of approved projects. This Committee gives final approval to funding decisions based on recommendations from NCP's four technical review committees/teams, Regional Contaminants Committees, the Nunavik Nutrition and Health Committee, and the Nunatsiavut Health and Environment Research Committee.

AANDC's lead role on the Management Committee is consistent with its responsibility for the health and safety of Northerners and its statutory obligations under the Department of Indian Affairs and Northern Development Act.

Sub-committees

Three Regional Contaminants Committees representing the Territories – the Yukon Contaminants Committee;theNorthwest Territories Regional Contaminants Committee; and the Niqiit Avatittinni Committee – are responsible for reviewing all proposals for work proposed in their respective regions from a socio-cultural perspective, as well as addressing contaminants issues and coordinating initiatives in regionally specific manner. For instance, the Yukon Contaminants Committeedevelops and coordinates a comprehensive regional contaminants research program and establishes priorities for the Yukon for consideration by the NCP Management Committee; promotes research on contaminants in the Yukon; communicates the latest results of research to the Yukon public; and provides a venue for the public to seek answers regarding contaminants in the Yukon. In Nunavut, the Niqiit Avatittinni Committeeuses a cooperative and collaborative approach to ensure that contaminants research is conducted on topics that are important to Nunavut residents; works to identify community priorities and information gaps for future research activities; and keeps communities informed and involved in research projects.

In addition to the three territorial based Regional Contaminants Committees, Nunavik and Nunatsiavut are also represented by their own Regional Contaminants Committees. The Nunavik Nutrition and Health Committeewas formed to represent the interests of Northern Quebec and the Nunatsiavut Health and Environment Committee fulfills this role in northern Labrador. Much like the territorial sub-committees, these committees seek to apply NCP strategies to the unique context in Nunavik and Nunatsiavut and coordinate efforts on the ground.

Proposal review teams and ad hoc committees

The program has formed proposal review teams and ad hoc committees reporting to the NCP Management Committee to facilitate the scientific/technical review of project proposals and assist regional committees in proposal review and support Management Committee decision making, while ensuring that impartiality is maintained throughout the proposal review process. A set of Blueprints for each NCP sub-program guides all stages of the proposal development, review and funding allocation decision process. These documents are reviewed annually and updated when necessary.

The Human Health Review Team, currently co-chaired by Health Canada – Healthy Environments and Consumer Safety Branch and AANDC, facilitates the technical review of all Human Health proposals seeking NCP funding through the use of external scientific peer reviewers. Members of the team include representatives of Inuit Tapiriit Kanatami, Health Canada – First Nations and Inuit Health Branch, Health Canada – Health Products and Food Branch, Government of the Northwest Territories - Health and Social Services, Government of Nunavut - Health and Social Services, and the Nunavik Regional Health Board.

The Environmental Monitoring and Research Sub-Committee, chaired by AANDC and comprised of Aboriginal partners, other government departments and external peer reviewers, oversees the annual review and revisions of the blueprint, reviews and oversees the revision of proposals, ensures the coordination of projects, and makes project funding recommendations to the Management Committee.

The Communication, Capacity and Outreach Independent Review Team was established to conduct the technical review of proposals submitted under the Communications, Capacity and Outreach category. It is chaired by AANDC and comprised of at least three other members, including social science researchers, Aboriginal research centres (e.g. Nasivvik), and other government health and research programs (e.g. National Aboriginal Health Organization, Health Canada Climate Change and Health Adaptation Program).

The Community Based Monitoring and Knowledge Integration Review Team, chaired by AANDC and comprised of Management Committee members, Aboriginal Partners and other government health and research programs, was established in 2010. This review team oversees the annual review and revisions of the blueprint, directs the solicitation of proposals, reviews and oversees the revision of proposals related to community based monitoring, and knowledge integration, ensures the coordination of projects, and makes recommendations to the Management Committee.

1.2.5 Key stakeholders and beneficiaries

As discussed above, the NCP has incorporated a variety of stakeholder groups into its management and governance structure. In addition to those mentioned above, research organizations and individuals act as partners and beneficiaries. The aims of the program to increase community-level awareness and capacity, and to reduce contaminant levels in the North, are intended to benefit northern communities and individuals who are exposed to these harmful substances, making them stakeholders as well.

Within AANDC, NCP works closely with the Cumulative Impact Monitoring Program and the Nunavut General Monitoring Plan to deliver coordinated research and monitoring activities in the Northwest Territories and Nunavut. The NCP has also been in discussions with the Beaufort Regional Environmental Assessment to ensure coordination of activities in the Beaufort region. The NCP worked closely with the International Polar Year Federal Program Office, and is contributing to the development of northern science and technology plans in support of the Canadian High Arctic Research Station. The NCP also works closely with the ArcticNet Network Centre of Excellence to coordinate joint science activities across the Canadian Arctic.

The NCP represents Canada's primary contribution to monitoring and research on contaminants under the Arctic Council's Arctic Monitoring and Assessment Programme. Since the Arctic Monitoring and Assessment Programme was created at the same time as NCP in 1991, the NCP representatives have represented Canada on the Arctic Monitoring and Assessment Programme working group and its affiliate expert groups. The AANDC Chair of the NCP Management Committee is also the current international Chair of Arctic Monitoring and Assessment Programme. Through participation and leadership in the Arctic Monitoring and Assessment Programme, the NCP contributes Canadian Arctic science data and information on contaminants and climate change to the Arctic Council. The assessments and other products generated by the Arctic Monitoring and Assessment Programme in turn deliver important information on the Arctic environment to international regulatory bodies and to organizations such as the United Nations Environment Programme, the Inter-governmental Panel on Climate Change and the United Nations Framework Convention on Climate Change.

The ultimate means to achieve the NCP's objective of reducing Arctic pollution is through the implementation of international agreements that limit emissions. To this end, the NCP works closely with representatives from other government departments on Canadian international engagement on pollutants. This includes compiling scientific information on chemical contaminants being considered for addition to the Stockholm Convention on POPs, or similarly providing scientific information on mercury in the Arctic for the United Nations Environment Programme intergovernmental negotiations on mercury. NCP/AANDC representatives are often asked to be part of the Canadian Delegation and attend international meetings to provide advice on Arctic science.

1.2.6 Program resources

Table 1 below shows budgeted spending of the Northern Contaminants Program between 2003-04 and 2010-11 as $36,400,000. Of this total, the program was allocated $3.5 million for Operation and Maintenance (O&M), $28.1 million for grants and contributions and an additional $4.8 million through interdepartmental transfers from Health Canada.

1.3.1 Background, context and program rationale

The long-running NSTP takes its roots in the Northern Scientific Training Grants Program, established in 1961. Initially, the rationale for the program was to provide opportunities for student research in the North, as few universities were active in this area, which in turn had contributed to a shortage of experienced northern scientists and other specialists with northern expertise. The NSTP was created to reduce barriers that university students continue to face when conducting research projects on-site in the North, since the exceedingly high costs of northern travel and accommodation present a challenge for student researchers. To this end, the program supports advanced and graduate university-level research in the North through supplementary grants that offset the high costs of transportation, accommodation and other incidental expenses, thereby enabling university students to develop valuable fieldwork research experience.

The purpose of the program is to foster science and technology in the North by contributing to the development of a body of northern experts. The NSTP supports student research from all disciplines and covering all northern topics, including human sciences, health sciences, life sciences and physical sciences. In total, the program provides funding to approximately 350 students annually at over 30 universities and northern colleges across Canada.

Since 2005, an additional set of NSTP awards, specifically for students from the North, have been funded annually: the Northern Resident Awards and the Northern Resident Scholarships are awarded to approximately 12 recipients each year. These awards/scholarships are administered by the Association of Canadian Universities for Northern Studies on behalf of the NSTP, as part of the Canadian Northern Studies Trust.

1.3.2 Program objectives and expected outcomes

The objective of the NSTP is to "increase the number of graduate and other advanced students within Canadian universities who have specialized in some aspects of northern studies and who have northern research experience."^{Footnote 2} As part of this objective, the program aims to increase young researchers' interest in and commitment to northern science and "contribute to the advancement of traditional and scientific knowledge of the North."^{Footnote 3}

The terms and conditions of the authority, grant for the advancement of scientific knowledge of the North include the following results:

• Strengthened capacity of Canada's scientific community involved in northern studies;
• Strengthened commitment of individuals or groups to northern studies through national recognition of their work;
• Increased capacity of Association of Canadian Universities for Northern Studies to support Canadian scholarship and institutions involved in northern research;
• An increased number of advanced students in Canadian universities and northern colleges, including Northerners, specialized in northern scientific studies; and
• Increased scientific information and knowledge about northern issues.

One additional expected outcome identified in the Northern Affairs Organization Business Plan, 2010-11 is the creation of a strong research network across Canada's North.

1.3.3 Program activities

The NSTP works toward its objective through partnering with universities and Association of Canadian Universities for Northern Studies; promoting the program through university information sessions; conducting an annual call for proposals; providing funding support to Canadian students for transportation costs, living expenses, freight costs and interpreter fees associated with carrying out research fieldwork; and producing information manuals and reports, including an Annual Report and a Bi-Annual Bibliography.

1.3.4 Program management and governance

The NSTP Management Committee, consisting of between 13 and 16 members from federal departments and agencies that have mandates encompassing northern scientific activities, makes program funding and allocation decisions. The Deputy Minister of AANDC appoints Committee members for three-year terms. The NSTP Management Committee votes in an existing Committee member as Chair, currently Environment Canada, for a three-year period. The committee includes additional representatives from the Government of Yukon, Aurora Research Institute and Nunavut Research Institute. The Association of Canadian Universities for Northern Studies is represented as a non-voting member and provides a linkage between universities and the NSTP. Individual universities are responsible for distributing funds to students and are given the opportunity to provide input into program design and implementation. The NSTP Secretariat of AANDC's Northern Science and Contaminants Research Directorate, provides administrative support to the committee and fulfills ongoing management responsibilities associated with program delivery.

1.3.5 Key stakeholders and beneficiaries

The key stakeholders of the NSTP are 36 universities across Canada, northern colleges and research organizations, and Association of Canadian Universities for Northern Studies. Direct beneficiaries are students that receive grants through the program. Since 2005-06, the program has partnered with Association of Canadian Universities for Northern Studies to encourage northern recipients through the Northern Resident Scholarship and the Northern Resident Award.

Through contributing to the body of northern scientific literature and data, as well as advancing the careers of young scientists, the NSTP enables a greater understanding of issues facing Northerners and builds scientific capacity, thereby benefiting northern communities.

1.3.6 Program resources

Table 2 below shows budgeted spending through NSTP over the period covered by the evaluation. Of the total $7,905,600 spent, the program was allocated $269,600 for O&M and $7,636,000 for grants. Other departments contributed additional funding in 2008-09, 2009-10 and 2010-11 to provide additional NSTP support during the period of elevated northern research activity resulting from Canada's involvement in the International Polar Year.

2.3.1 Evaluation phases

The course of the evaluation followed three broad phases:

Phase 1: Planning. The planning phase of the evaluation began in the spring of 2011 with a review of literature and program related documents to help inform the development of the evaluation Terms of Reference and subsequent methodology report.

Phase 2: Research and analysis. The core evaluation issues and the evaluation questions identified in the preceding phase guide all stages of analysis and reporting with the end goal of responding to each question with as much evidence from as many lines of inquiry as possible through a process known as triangulation.

Two additional lenses of analysis – gender-based analysis and sustainable development analysis – were applied throughout the evaluation. Both the NCP and NSTP touch on several of AANDC’s Sustainable Development Strategy guiding principles, for instance, the engagement of interested local communities and organizations during program implementation and contributing to decision making based on the best available scientific, traditional and local knowledge. The manner in which the program addresses gender-specific considerations is assessed through gender-based analysis. One example is NCP’s emphasis on funding projects that collect data on a specific group in need, for instance, pregnant women and children. Gender-based analysis is also applied to NSTP if possible, for instance, to determine the proportion of male and female students that have received funding through the program.

Phase 3: Reporting. A number of documents analyzing individual lines of inquiry were developed for internal use over the course of the evaluation. The final product is an evaluation report that will be made available to the public following approval by AANDC’s Evaluation, Performance Measurement and Review Committee. The evaluation report includes findings, conclusions and recommendations related to both the NCP and NSTP separately or aggregated based on the findings. Analysis and reporting are sensitive to the differences between the two programs (e.g. program profiles and expected outcomes), but allow for cross-cutting analysis when applicable.

2.3.2 Data Sources

The evaluation’s findings and conclusions are based on the analysis and triangulation of the following multiple lines of evidence:

2.3.3 Considerations, Strengths and Limitations

This section discusses several strengths of the evaluation as well as possible challenges that the evaluation faces, including mitigating strategies.

• Availability of data. There was a lack of program performance monitoring data available for the NCP and NSTP in order to properly assess the performance of these programs. Although NCP developed a logic model and performance indicators and started to collect some of the indicator data in 2009-10; some indicators were not collected and some data collected was not reliable as it includes contribution from other programs, such as International Polar Year. NSTP collects a lot of data and uses some of them as indicators. However, more information could be provided using data collected such as the percentage of NSTP contribution to total research. File reviews, interview and case studies were used to mitigate this risk and collect information to report on outcomes, efficiency and economy.
• Coordination with the Evaluation of International Polar Year. Data collection for the present evaluation was done in parallel with the Evaluation of the Government of Canada Program for International Polar Year. Though these remain two separate evaluations, this coordination facilitated logistics and decreased data collection costs as these programs serve some of the same stakeholders and recipient groups. Concurrent research and analysis also brought to light ways in which the two research initiatives complement one another, as well as lessons learned for future program design and delivery. One example is the assessment of the governance structures of International Polar Year, NCP and NSTP. Finally, sharing contract costs between the two projects resulted in cost savings.
• Evaluation scope. The evaluation’s focus on both the NCP and NSTP, presents both strengths and challenges. The simultaneous evaluation of the two programs, and the coordination of the evaluation with the International Polar Year evaluation, increased cost savings related to data collection. Further, this juxtaposition enables the evaluation to examine whether there were shared design and delivery processes in place that contribute to greater efficiency (e.g. reporting efficiencies, sharing of best practices, networking, cross-program referrals, operational costs, etc). Conversely, as is the case with all AANDC evaluations that combine one or more programs, there is a risk that the performance of each program could be conflated in analysis, or that the organization or style of the report loses coherency. In order to mitigate these risks, the evaluators were careful to organize findings around a set of evaluation questions and performance indicators that are particular to each program.
• NSTP Survey. Basic account with Survey Monkey is limited to 10 questions per survey and provides only one type of quantitative analysis and report. In addition, the response rate was limited by the fact that 74 percent of all addresses in the student database were linked to a university e-mail address that could be no longer valid. However, the response rate was still 44 percent, which provides sufficient information to be indicative of student perspectives, especially given the limited variance in responses.
• Evaluation work conducted by various groups. Three consultant firms were contracted to carry out this evaluation in order to complete it on a timely basis and with the maximum use of internal resources to perform some of the evaluation tasks, such as methodology report and questionnaires, literature review, document and file review, field interviews, case studies, and some report writing. To have so many people involved in the evaluation increased the coordination required and could have had an impact on the quality of the final product. It meant, for example, that the consultants who were responsible for drawing together the findings from various lines of evidence and produce the final report were not involved in the project from the beginning, and therefore, did not have the level of background knowledge and the time to develop a comprehensive understanding of the initiative, that they would have had if they had conducted the entire study. Similarly, those conducting individual components of the evaluation did not have the opportunity to learn from the totality of information and revise their initial findings. The evaluation team and the consultants drawing the findings together were aware of these challenges and mitigated the risk by incorporating new findings into the lines of evidence, and by drawing on the expertise of the evaluation Working Group. The evaluation team is confident that the findings and conclusions are based on sound evidence.

2.4.1 Project management and governance

The Project Authority for this evaluation is the Director of AANDC’s EPMRB. A Senior Evaluation Manager responsible for AANDC’s northern programs is the project manager. An evaluation team working under the direction of this senior manager and a lead evaluator/project leader undertook tasks associated with the evaluation; methodology report and questionnaires document and file review, selection of key informants and case studies, participated in most interviews, field work and reporting. In addition, the Senior Evaluation Manager and project leader managed contracts through which additional evaluation tasks were completed by Capra International Inc. (contributed to the methodology report and questionnaires, key informant interviews working document), Goss Gilroy Inc (literature review, governance review, administrative data review, and background case studies, and Alderson-Gill & Associates (key informant interviews working document and field work, finalized case studies, prepared and presented key preliminary findings deck and wrote the draft final report.

As discussed in greater detail below in relation to methodology, research deliverables were produced and reviewed internally to ensure the evaluation was on track at all times and that the quality of research meets AANDC’s standards.

2.4.2 Quality assurance

Formal review, validation and approval processes were put in place at all stages of the project to ensure the scope and focus of the evaluation reflect the perspective of AANDC program officials, Northerners and other experts. Quality was ensured by an appropriate mix of decision making, internal and external review and the application of quality control tools. The objective is to produce evaluation products, which are reliable, useful and defendable to both internal and external stakeholders. A description of each governance and quality assurance measure follows:

• Evaluation working group. The evaluation working group is comprised of AANDC program managers (Headquarters) representing both NCP and NSTP; and includes stakeholders such as representatives from other federal departments and agencies, program committees mentioned above, and one northern Aboriginal organization. This group was invited to review, validate and provide input into the methodology report, preliminary findings deck and draft report. The broad mandate of the group includes the following:
  • The quality and relevance of the evaluation’s approach as outlined in the methodology report
  • Proposing key data sources and stakeholders whose perspectives are vital to an analysis of the impact of the programs under evaluation and, where possible, facilitating access to such resources; and
  • The quality and relevance of the draft report and research findings to ensure results are thorough, balanced and useful.
• Evaluation, Performance Measurement and Review Committee. This committee, chaired by the Deputy Minister of AANDC and comprised of the Chief Financial Officer, senior assistant deputy ministers and external experts, is responsible for approving the evaluation Terms of Reference, draft evaluation report and management response and action plan.
• AANDC evaluation peer review process. A structured internal peer review and response process was undertaken on the evaluation methodology, including research instruments and the draft evaluation report. Evaluators with no involvement in the project conducted the review. Reviewers assessed the quality of methodology and final reports and ensured they comply with relevant Treasury Board, AANDC and Branch policy, criteria, and standards. Reviewers also examined the degree to which final report corresponds with the evaluation’s terms of reference and methodology report and the degree to which recommendations are supported by evidence-based findings.

3.1.1 NCP: Is there a continued need for the NCP?

All key informants consulted on this issue, including researchers, northern stakeholders, and federal government representatives, emphatically stated that there is a continued need for the NCP. They indicated that the issues related to environmental contaminants are dynamic and require a continuity of attention. Contaminants have an impact on human health and NCP-funded research has demonstrated that levels of some contaminants have decreased while others have increased.

Data and information generated through the NCP are seen to be critical for:

• supporting Northerners and their communities to live safely and be healthy in a rapidly changing environment;
• raising awareness among Northerners of the effects of contaminants;
• supporting communities that want to be engaged in the research and made aware of the results;
• monitoring climate change and the impacts of increasing development in the North;
• supporting international and domestic decision making;
• evaluating the impact of international agreements and domestic policy; and
• following up on rich baseline data that has been developed through the NCP.

Researchers, including those affiliated with government, academic institutions and northern organizations, reported that NCP is the only continuous source of funding for contaminant research in the North. This is consistent with the finding of the 2002 evaluation that the NCP was the only program in the North that addresses health and safety issues related to contaminants. A finding of the 2002 NCP evaluation, reconfirmed by this evaluation, is that there is a continuing need for the program to meet Canada's commitments under international contaminants agreements.

All key informants identified gaps in knowledge of contaminants and their effects that still exist. Important knowledge gaps include contaminants in drinking water, air quality, how contaminants are transferred from the air or water into the food chain, the effects of contaminants on Arctic species, Arctic-wide monitoring, the links between contaminants and health, the effects of climate change on the status of contaminants, and the effects of new chemicals.

A majority of key informants indicated that a significant gap and ongoing challenge for the program is the effective and appropriate communication of research results to communities that may be affected by the findings. The NCP has some mechanisms to ensure that knowledge translation occurs. However, the communication of results was frequently described as inaccessible to lay audiences because of its technical nature and the way that it was presented. There remains an apparent need for the communication of contaminant-related information to northern people, which is an ongoing objective of the NCP.

Almost all project leaders indicated that the priorities and objectives of the NCP were consistent with or supportive of the goals and priorities of their organization (and the others indicated that the priorities and objectives of the NCP were partially consistent with the objectives of their organization).

3.1.2 NSTP: Is there a continued need for the NSTP?

All stakeholders consulted believe there is a continued need for the NSTP for the following reasons:

• NSTP enables student researchers to conduct research in the North by providing funding to cover additional costs associated with northern research that is not provided through base funding offered by other programs, including the Natural Sciences and Engineering Research Council, the Social Sciences and Humanities Research Council, ArcticNet, and the Canadian Institutes of Health Research.
• The North is experiencing rapid social and environmental change.
• There is a growing interest in northern research among students.
• There is a need for consistent support to Aboriginal students.
• Discontinuing the NSTP could lead to fewer northern research specialists and less northern research capacity as current research specialists retire.

3.2.1 NCP

Is the NCP aligned with federal government priorities?

Overall, the NCP aligns with Government of Canada priorities and AANDC Strategic Outcomes. The 2011 Speech from the Throne stated that the federal government has made the North a cornerstone of its agenda. In this regard, the speech stated that the Government will "continue to exercise leadership in the stewardship of northern lands and waters".

The 2007 Speech from the Throne announced a Northern Strategy that would protect Canada's environmental heritage while promoting economic and social development. The same speech specifically identified children and seniors as groups that need to be targeted with action due to their vulnerability to threats from the environment. Finally, the speech makes a broad link between health and the environment with the statement that "environmental protection is not just about protecting nature. It is about the health of Canadians".

Canada's Statement on Canada's Arctic Foreign Policy claims that "strong environmental protection, an essential component of sustainable development, starts at home and is another important way in which Canada exercises its sovereignty in the North" (p17). The same document cites pursuing and strengthening international standards as one of the four ways that the Government cooperates internationally. The document speaks directly to NCP when it states: "Canada will continue to address the problems arising from … [contaminants], including waste management practices in the North, and will engage actively in global negotiations to reduce mercury emissions".

3.2.2 NSTP

The NSTP supports AANDC's sustainable development strategy by providing annual funding to northern residents to study in areas that are relevant to their communities. From 2003 to 2011, approximately 90 percent of the NSTP funding went to Canadian students doing study in the Canadian North; 28 percent were undergraduate, 41 percent masters and 31 percent doctoral level. The program contributes to economic development within the North and supports investigation into strategic priorities, including the impacts of climate change.

3.3.1 NCP: Is this initiative aligned with federal roles and responsibilities?

Departmental managers consulted for the evaluation believe that the role of the federal government in the NCP is legitimate and necessary. Because contamination is a trans-boundary issue covering all of northern Canada, the federal government is seen as the logical host for the NCP or any similar program. Key informants could not identify another organization or governance structure whose mandate fits the NCP. They also pointed out that other stakeholders and jurisdictional bodies are included in the NCP process through the program's Management Committee and Regional Contaminants Committees.

Departmental representatives stated that AANDC is the most appropriate department to host the NCP, not only because of the Department's focus on the North and responsibilities to northern communities, but also because the Department has experience coordinating research in the North.

All key informants identified other organizations that are involved in research in the North, including ArcticNet, the Natural Sciences and Engineering Research Council, and the Cumulative Impact Monitoring Program in the Northwest Territories and the Nunavut General Monitoring Program. However, they also indicated that the NCP acts as a complement to existing programs and does not duplicate other program activities. Some of the ways in which the NCP is reported to complement existing programs include the NCP Management Committee sharing members with ArcticNet's management committee, NCP working closely with the Cumulative Impact Monitoring Program (which provides a level of informal oversight and shared knowledge to reduce duplication between programs), and some researchers receiving funding from NCP and other programs for specific aspects of a large research project.

Other federal government departments, most notably Health Canada and Environment Canada, provide some funding for Arctic research projects. Key informants view the NCP as unique and distinct from other existing programs for the following reasons:

• The long standing nature and continuity of NCP research activities are unique to the program and a factor in the success of the NCP.
• Alternative programs are not viewed as continuous or stable sources of funding - factors which are seen as critical to the success of NCP projects. For example, ArticNet's funding is scheduled to last for another six-seven years. Funding from Health Canada and Environment Canada is also reported to be unstable/non-continuous, and the International Polar Year has ended already.
• Important relationships between researchers, research organizations and Northerners have developed over time through the continuity of the program.
• The NCP differs from other programs by targeting contaminants specifically in the North. Other funders have a broader focus or, in the case of Cumulative Impact Monitoring Program, a narrower focus.

Key informants representing international organizations indicated that the NCP complements and feeds into international research efforts, including Arctic Monitoring and Assessment Programme, ArcRisk, and the U.S. National Science Foundation Polar programs. The international respondents also stated that the NCP is viewed as an international model for funding contaminants research.

3.3.2 NSTP

All stakeholders believe that the NSTP is consistent with Government of Canada responsibilities. NSTP funding is considered to be complementary to funding provided through other agencies such as the Natural Sciences and Engineering Research Council, the Social Sciences and Humanities Research Council and the National Research Council. There is no evidence of overlap or duplication. Most stakeholders believe that no other organization has the capacity to adequately administer the NSTP, with the possible exception of Natural Sciences and Engineering Research Council, which they believe would present some challenges in terms of their existing funding procedures, and would not offer appreciable benefits.

Northern engagement in NCP activities:

From a program management perspective, the Regional Contaminants Committee and the blueprint process are effective mechanisms for involving stakeholders in program planning and decision making. The reliance on consensus-based decision making is also viewed as an important mechanism to ensure active engagement of stakeholders. The Communications, Capacity and Outreach subprogram as well as other efforts of the NCP to provide relevant communications to communities, were provided as evidence of how local and northern priorities were incorporated into the NCP, although many key informants believe that this is an area where the NCP can continue to improve.

Many key informants stated that the design and implementation of the NCP has led to increased northern capacity to engage in the program (e.g. by identifying research priorities, preparing proposals, and participating in research and monitoring activities). Specific features of the program that are viewed as having increased northern capacity include the following:

• The Inuit Research Advisors positions have allowed more meaningful engagement of Inuit in the NCP. The Inuit Research Advisors provide information to communities in Inuktitut, act as a liaison between the researchers and the community, and assist communities in understanding the design and implementation of research. One issue noted by three key informants is that it is difficult to keep people in the Inuit Research Advisors positions because they quickly move to higher-level jobs.
• Proposal requirements of most funders, including the NCP, require engagement of local communities as a condition of funding. The NCP has helped to set this standard.
• The provision of funding to Northerners to attend the NCP's Annual Results Workshop.
• The involvement of Aboriginal organizations in the governance aspects of the program has reportedly increased their profile on the international stage.
• The distribution of NCP synopsis and synthesis reports to northern communities.

All key informants agreed that value is derived from involving local people and incorporating traditional knowledge into the research process. AANDC representatives stated that this approach in NCP projects has given Northerners a stake in the process – that they are not just passive recipients of research. Territorial government representatives reported that most projects have traditional knowledge/ Inuit Qaujimajatuqangit or Inuit Knowledge component. In Nunavut, Government of Nunavut representatives reported that all NCP projects incorporated Inuit Qaujimajatuqangit. About half of the project leaders interviewed for the evaluation reported that their project had integrated traditional knowledge and practices. Fewer than one-third indicated that their project did not integrate traditional knowledge and this was not applicable to other project leaders as their project did not have a field work component (e.g., modeling projects).

Research project leaders reported that community members were most likely to be engaged as experts in the local geography and logistics and as recipients of information from scientists. Northerners and northern communities were engaged in their research projects in the following ways: providing traditional and land-based knowledge to support research efforts (e.g., tracking animals, travelling on the land); communication of research results to communities; participating as partners in the design of the research; and, working as researchers and assistants.

Other key informants noted that some progress has been made at increasing engagement of communities as research partners (e.g., through the Community-based Monitoring and Knowledge Integration sub-program). There was also broad agreement that there has been an increase in public awareness of science in North partly due to the NCP.

Although the overall levels of awareness and engagement have improved, NCP project follow-up and communication of results to communities have been less consistent and successful than the awareness, despite considerable program effort. With several notable exceptions, relationship-building between researchers and communities did not meet northern expectations. The shortcomings in relationship-building occurred despite the requirements of the NCP Terms of Reference and apparently genuine efforts to advance such collaborations.

The challenges of community engagement, without which the capacity of people to make informed decisions can be limited, were well recognized by all stakeholders. The challenges include:

• the high cost of travel with tight research budgets;
• the linkages between research findings and community concerns are not always obvious;
• the availability of academic researchers is restricted due to short field research season; and
• the required communications skills are not widespread among scientists.

With respect to limitations, a northern-based organization reported that Northerners involved in a data collection role as sample-takers usually are not asked to provide information on the context (e.g., the condition of the animal), which could be useful in other areas beyond the particular NCP project. An AANDC regional office representative reported that researchers usually do not contact the Department's contaminants specialists and that they are "not seen as part of the team."

A territorial government representative said that the NCP encourages research using community members to communicate the results. However, another territorial government health official advised that more could be done on public communications, stating that: "considering the wealth of information collected through NCP, there could be more done to communicate the overall impacts to Inuit."

Research capacity:

There is a broad consensus that NCP-funded research and monitoring has increased the overall capacity of Northerners and northern communities, particularly through the development of ongoing working relationships between territorial governments, Aboriginal organizations, and researchers.

Approximately half of the project leaders interviewed agreed that the NCP has increased northern capacity to engage in the program (e.g. identifying research priorities, preparing proposals, and participating in research and monitoring activities). Other project leaders said that this issue was not relevant to their project (e.g., modeling work) or that it is difficult to assess whether the NCP has built capacity among Northerners (e.g., because of a lack of quantitative evidence of improved capacity).

Through involvement with NCP, partner Aboriginal organizations have increased their capacity to work on contaminants issues at the national and international levels. For instance, they have played an active role in the Arctic Council as well as influencing the Executive Body of the United Nations Economic Commission for Europe Convention on the Long-range Transboundary Air Pollution, leading to the protocol for POPs. Similarly, northern Aboriginal organizations formed the Canadian Arctic Indigenous Peoples against POPs with support from NCP. This organization has influenced global negotiations and agreements on POPs. Northern Aboriginal organization representatives have been part of Canada's Delegation to international agreements such as the Stockholm Convention and also international negotiating meetings such as the United Nations Environment Programme Global Mercury Negotiations. It is reportedly unique in the world to have non-governmental reps from Aboriginal organizations as part of a national delegation, and Canada has been commended by many nations for doing this. Program managers say that this international involvement has been a result of the direct involvement of the northern Aboriginal organization representatives in all aspects of the NCP.

NCP has built capacity among northern Aboriginal organizations and encouraged them to become involved in both the Arctic Monitoring and Assessment Programme and Sustaining Arctic Observing Networks. Similarly, NCP supported a northern Aboriginal representative to join the Canadian delegation to the POPs convention and mercury negotiations held May 2007, 2009, and 2011 (POPs) as well as June 2010, January 2011 and November 2011 (mercury).

Territorial government representatives reported that there is a capacity-building component in every NCP proposal. When reviewing proposals for NCP projects, they review "what locals will do" and the extent to which the project will hire and train locals and Northerners as researchers.

Many Northerners have been trained through NCP projects, mostly for work as field assistants (e.g., collecting samples), field coordinators, and lab technicians. Through this training, NCP projects have developed local data collection skills and a sustainable data collection capacity. A small number of Northerners have gone on to more advanced post-secondary training, something that is strongly encouraged and supported whenever possible by all key organizations involved in the NCP.

Territorial government representatives stated that a lack of capacity, not money, makes it difficult for their governments to incorporate research into policy. They have problems finding qualified people and there is high staff turnover. Consequently, it is difficult for them to be proactive in policy development, health planning, and public communications related to contaminants.

One territorial government representative stated that territorial governments and Aboriginal organizations are "bombarded" with NCP and other research proposals to review. They have limited capacity and limited time to engage in this process and to undertake the background and capacity-building work required to be prepared for it. They advised a more strategic approach by the federal government to engagement by, for example, conducting shorter meetings, providing opportunities to participate at a distance, and providing summaries of key documents. It should be noted that unlike most government programs that engage stakeholders in helping to select projects to be funded, the NCP financially compensates organizations for their time reviewing proposals and taking part in review meetings.

Other remaining challenges to NCP's efforts to increase northern capacity include the following:

• There is a need for better mechanisms to strengthen community partnerships, which must be fostered during the program design and which require ongoing attention throughout the research.
• There is widespread perception across the North that research is driven by southern interests and priorities. There is a perceived need for more meaningful northern input on priority setting.
• Communities have concerns about contaminants, but it is very difficult for them to develop proposals to meet scientific standards. There is a need to establish research teams with local leads or co-leads and to foster North-South partnerships.

b. Creation of new data, information and knowledge related to impacts of pollutants on human health and ecosystems in the North through culturally-sensitive research

The NCP incorporates several components designed to create and disseminate new knowledge gained through research funded by the program:

• NCP science projects are designed specifically to generate new data on contaminants and their impacts.
• NCP project leaders are required to develop a plain language summary of their findings to ensure that project results can be communicated to Northerners in a manner that is understandable and useful.
• NCP assesses the performance of laboratories carrying out contaminant analyses to ensure inter-comparability of data. There is a requirement that any laboratory receiving funding from the NCP to conduct analyses must participate in the NCP's quality assurance quality control program to ensure high quality data and inter-comparability.
• NCP holds an annual Results Workshop to share the results of funded projects. Project leaders are allocated up to $2,000 for costs associated with travel for one individual per project to take part in the workshop.
• NCP publishes an annual synopsis of research report outlining project results.

All key informants, including project leaders, project managers and international respondents agreed that the NCP has contributed to the development of new contaminants data, information and knowledge since 2003-2004. These stakeholders reported that the NCP has made "significant" and "extensive" contributions to the development of new contaminants data, information and knowledge related to impacts on human health and ecosystems. The contributions include important research on contaminants in marine and mammal health – information that Inuit are reportedly asking for - and research on the nutrient values and the health protective factors associated with country foods – research that is needed to provide people with the information to balance the risks and benefits of eating a traditional diet.

A majority of project leaders reported that their projects integrated traditional knowledge and practices (for applicable projects).

NCP-generated contaminants data are the basis for numerous key Arctic research publications and have been included in publications by the Arctic Monitoring and Assessment Programme, in the Canadian Arctic Contaminants Assessment Report and the Canadian Arctic Contaminants and Health Assessment Report produced by NCP in 2003 and 2009 respectively.

Key informants stated that the contaminants component of the Inuit Health Survey "definitely succeeded", with results showing higher than expected lead and methyl mercury levels in some people. A Government of Nunavut representative agreed that the Inuit Health Survey provided an opportunity to get more detailed data on contaminants and the linkages to diet for Nunavut. Key informants also stated that future iterations of the survey will allow health officials to look at trends in contaminants and contaminant-related health issues.

A territorial government representative agreed that the NCP has advanced knowledge of contaminants. In their opinion, the next step is to move beyond looking at contaminants in isolation and to consider other lifestyle and environmental factors such as smoking, diet, indoor air quality. A representative of a northern-based organization stated that, while there are no obvious gaps in the contaminant sources covered by the program, the absence of Canadian guidelines for acceptable levels of contaminants is an issue. Other northern-based key informants noted that the NCP focus on long-range contaminants and not on what they described as issues of greatest concern to local communities (locally produced contaminants) can make NCP seem less relevant to these communities and limit their interest and engagement.

An AANDC official emphasized the international impacts of NCP research, stating that the research provides scientific evidence about POPs and mercury in the environment that Northerners and Canadians can use in international forums, which have led to international agreements/regulations on the production, use and release to the environment of several contaminants.

Intermediate outcomes

c. Through greater awareness of nutrition and contaminant issues, Northerners and northern communities make informed decisions related to their food use

In addition to funding studies that contribute to an understanding of the nature and extent of contaminants in the North, NCP and its partner organizations take direct action on contaminants data that may present serious health consequences to the northern population. The program has in place well-defined processes that validate data collected through the program, initiate health risk assessments when deemed necessary and disseminate information to communities regarding risks of consuming country food in a particular area.

The NCP has been communicating findings related to long-range contaminants for over 20 years. A great deal has been learned over this time about the presence, trends, and health effects of contaminants in the North, as well as about how to communicate complex information about contaminants. Target audiences have included community members, front-line workers, hunters, youth, mothers, pregnant women and women of childbearing age. Many efforts to reach these target audiences have been made over the years using a variety of methods such as posters, newsletters, development of school curricula, community tours, radio call-in shows, workshops, frontline training courses, and Elder-Scientist retreats.

The NCP emphasizes the need to place communications efforts within the broader context of research in the North, and where possible, in collaboration with other programs. The methods used to reach target audiences have evolved over the years. Recently, instead of targeting community members directly, the program now emphasizes the need to target front-line personnel who community members turn to for information on contaminants. These personnel include Regional Contaminants Committees, nurses, doctors, community leaders, nutritionists, wildlife and fisheries officers, land and environmental protection offices, adult educators, school teachers and Inuit Research Advisors. This approach is expected to help ensure that information is communicated within an appropriate regional context.

NCP contaminants research has led directly to public health advisories being issued in the Northwest Territories, a long-standing consumption advisory regarding toxaphene in fish in Yukon lakes being lifted based on long-term monitoring data from NCP; and a public health advisory and messaging being released in Nunavik as a result of long-term NCP-funded health effects study.

There is a consensus among key informants that NCP-funded initiatives have led to an increase in broad public awareness of contaminant issues in the North. Although awareness has increased, northern key informants stated that a lack of trust can still be a big issue in some communities, and that northern communities usually prefer direct participation in research to simply being informed after the fact. Northern-based key informants emphasized that cooperation from local people and communities improves when they understand the rationale for research. They also reported that direct communications about research results at the community level has proven successful, with one citing an example of people in communities who initially were concerned about food contamination being less concerned following a series of community events involving presentations by the researchers and territorial health officials.

The inherent complexity of contaminants research findings has also made communications about their meaning difficult. Key informants reported that NCP-generated knowledge has had both positive and negative impacts on food choices. In the past, information about contaminants led some consumers to (temporarily at least) stop eating country foods. More recently, researchers, Regional Contaminants Committees, northern governments and organizations are being careful to present a balanced perspective, one that includes the benefits of eating country food and that is more specific about risks, when communicating research results.

Most northern-based key informants stated that project results sometimes are too complicated for most Regional Contaminants Committee members to understand given how the results are presented. Consequently, Regional Contaminants Committee members often do not bring the results back to their constituents. The Results Workshops are viewed as valuable for networking among scientists but not as the best mechanism for conveying information to northern participants.

d. World-class innovative research results and information are made accessible nationally and internationally

There was broad agreement among stakeholders that the results of NCP-sponsored research are widely and readily accessible, both nationally and internationally. It is fair to say that as a result of the NCP, Canada has become a world leader in POPs research and monitoring. NCP project-level synopsis reports are produced and made available annually. In addition, project leaders must now enter metadata into the Polar Data Catalogue, which has strengthened data sharing. Interest in meta-analysis of NCP research findings is one of the outcomes of improved data access and sharing.

At the regional level, health-related NCP information is relayed to communities through the local health authorities, and synopsis reports are distributed to all jurisdictions. The annual NCP Results Workshop also serves to communicate NCP results to other scientists and stakeholders.

At the national level, high-level NCP results are incorporated into the Canadian Arctic Contaminants Assessment Report, which has been published periodically by AANDC beginning in 1997. The second Canadian Arctic Contaminants Assessment Report was published in 2003, and a third set of reports is expected in 2012-2013. Key informants reported that NCP-generated data has informed both Canada's national regulatory framework and the international regulatory framework. They also reported that NCP has helped to foster excellent networks among Canadian and international scientists, with good cross-linkages among senior researchers.

Other ways in which NCP results have had an impact internationally include the establishment of an NCP publications database, and the use of NCP data and high-level results in the international Arctic Monitoring and Assessment Programme reports. Individual scientists and their teams regularly publish NCP-funded research in international science journals. The Inuit Circumpolar Council Canada has received support through NCP to engage in work related to the United Nations Environment Programme and Stockholm Convention. NCP results have been used to identify priorities under the Convention on Long-range Trans-boundary Air Pollution. The fact that NCP data has influenced international agreements was cited as evidence by key informants that NCP-generated data is accessible to decisions makers.

Key informants identified two challenges to the accessibility of results. These challenges are the perceived inaccessibility of workshops and project reports for community audiences that make it more difficult for researchers to present their work both internationally and at the community level.

e. Contribution to the development and implementation of domestic and global regulations, agreements to reduce and/or eliminate the production, use and release of contaminating substances into the environment

NCP contributions to domestic and global regulations and agreements have been longstanding. Chapter Six of the 1999 Commissioner of the Environment and Sustainable Development Report, Making International Environmental Agreements Work: The Canadian Arctic Experience, reports that research developed through NCP helped to establish that Arctic pollutants are derived from external sources and "provided much of the scientific evidence to underpin Canada's push for international controls on certain pollutants", for example, the United Nations Economic Commission for Europe Convention on Long-range Transboundary Air Pollution, and similar work on mercury that is expected to influence international approaches for control.

The NCP plays a leadership role in Arctic Monitoring and Assessment Programme, a body that coordinates international Arctic science activities under the Arctic Council. The NCP is also playing a role in the development of the Sustaining Arctic Observing Networks, an initiative that aims to achieve "long-term Arctic-wide observing activities that provide free, open, and timely access to high-quality data that will realize pan-Arctic and global value-added services and provide societal benefits".

Internationally, key informants reported that the NCP is contributing to international agreements by providing an evidence base to support negotiations of new conventions and agreements, and by providing the data that enables monitoring of the impacts of international agreements. NCP data has been used by international bodies to help shape international policies and in several important processes leading to regulation and/or agreements on contaminants. For example, through negotiations for the Stockholm Convention, nine new chemicals were added to the list, making a total of 21 restricted or banned chemicals. With input from Canada based on NCP-funded research, the United Nations Environment Programme Governing Council agreed to negotiations for legally binding agreement on mercury. The Inuit Circumpolar Council Canada, with financial support from the NCP, supported Arctic Council activities such as work on the Mercury Assessment of the Arctic Monitoring and Assessment Programme.

NCP data has supported policy development and influenced Canadian positions in several areas beyond those mentioned above. These include a northern ban on lead shot, better methods for cleaning up contaminated sites, and lifting of the 20-year advisory on consumption of certain fish in Lac Labarge, related to levels of toxaphene.

Long-term outcomes:

f. Creation of a new generation of northern scientists, and increased capacity of northern scientists and northern organizations to conduct research and address environmental and health issues

Earlier sections have discussed the contributions of the NCP to northern capacity, and the remaining challenges. However, it is worth noting here that NCP-funded projects have led to increased capacity of some organizations to undertake work such as:

• Improved contaminants research approaches and methodologies, as well as enhanced capacity to support other organizations conducting contaminants research through the provision of expertise and equipment related to health and safety, sample storage, and data management (e.g., Makivik Corporation in Nunavik); and
• Reviewing contaminant related information and communicating with communities and with authorities such as Members of the Regional Contaminants Committees.

A majority of research project leaders reported that the NCP funding had an impact on their career and on their decision to continue in northern research. Most importantly, almost all of the project leaders reported that they were introduced to northern research through NCP-funded projects and that the funding available from the program has allowed them to continue to conduct research in the North. Other reported impacts that the program has had on the careers of funded researchers include helping to determine research directions, inspiring further education, and increasing research networks. The NCP was also credited by researchers with increasing awareness of the importance of Aboriginal involvement in research, as exemplified by the program's requirements on community engagement, its support for Inuit Research Advisors, and its inclusion of northern community and organizational representatives on Regional Contaminants Committees.

Key informants emphasized the importance of the development of good networks to collect data (e.g., tissue samples) that have been established through NCP-funded research. Hunters, trappers and others have been trained in proper data collection procedures, which is extremely valuable to high quality research into contaminants. Despite some success at training personnel to support and assist with research, key informants identified human resources at this level as a major issue in the North. They reported that finding students to be research assistants and staff and technicians to work in research stations is difficult. The seasonal work and lack of available full-time positions are deterrents to potential candidates.

g. Improved health and reduced risk to northern communities and ecosystems as a result of reduced contaminants levels

Key informants identified three main ways in which NCP-funded research has contributed to benefits related to the mitigation of health risks: identification of pollutant levels, Canadian and international protocols on the use of toxic substances, and health advisories resulting from the findings of NCP research.

NCP-funded research has determined that some contaminants have increased and some decreased in the Arctic, and this report has documented how NCP information and activities have contributed to Canadian and international action on contaminants.

h. Improved Canadian and international regulations

Earlier sections have demonstrated the considerable contribution that the NCP has made in influencing Canadian and international regulations related to contaminants. A large majority of key informants reported that their work specifically, or the collective body of work of the NCP more generally, have had an impact on the development and implementation of both domestic and international policy and regulation, as described earlier. None of the key informants disagreed with this proposition. Others either indicated that an influence on domestic or global agreements or regulation was not applicable to their project or that it is too early to determine whether their specific project would have an impact on policy or regulation.

4.3.1 NCP

Is the NCP designed to respond to northern research needs?

Most key informants believe that the NCP is aligned with northern research needs because it has involved Northerners in setting priorities through the Management Committee, the Regional Contaminants Committees, and the annual blueprint process. A northern-based organization stated that the presence of Inuit Research Advisors has allowed for more meaningful engagement of Inuit across the North. There is a generally-held view that the program has evolved in a positive way, and that several years of high quality NCP research has produced a body of data that has made the more recent program emphasis on human health possible.

Key informants, including representatives of governments, non-government organizations, and researchers generally support the NCP research focus on long-range contaminants as being important for meeting the environmental and health needs of the North and Northerners. Some representatives of northern-based organizations also believe that there is an emerging need to focus on more localized sources of contaminants as the number of resource developments and their potential impacts increase.

Research project leaders and management representatives indicated that there is room to improve the processes designed to ensure that the program is relevant to Northerners and northern research needs. They identified three main challenges to improving these processes:

• Communities are represented on some of the Regional Contaminants Committees but it can be difficult for community members to be meaningfully engaged because of the highly technical nature of the proposals.
• More needs to be done to communicate NCP results and implications to communities.
• It can be difficult for new researchers or community-driven research to secure funding through the program. The NCP is perceived by some researchers and communities to be a "closed shop", with funding tending to go to those researchers who have a long history with the program. However, starting in 2010, a Community-based Monitoring and Research sub-program was established to increase community-based capacity and this sub-program is community-led and driven by community interest and priorities.

AANDC regional office personnel indicated that the NCP blueprint is explicit about research priorities so that applicants are in a position to understand at the outset what kinds of research proposals are being sought. In terms of meeting northern needs, the program incorporates measures requiring community engagement, but there is a lack of accountability from researchers to demonstrate that they are meeting northern research needs (e.g., by incorporating Inuit Knowledge, consulting with communities). Although, "Approval of Consultation" forms for all projects before funding is awarded and requires reporting on capacity building/communication/outreach/traditional knowledge aspects of all projects.

Is the NCP being delivered in a way that will achieve outcomes?

For program administration, the NCP Secretariat was characterized by researchers as being competent, supportive and approachable.

Some researchers and northern-based organizations said that the NCP proposal review process has not been transparent enough, especially for the non-core component. Applicants reported that they do not know what other proposals have been approved, and the budgets of other projects. The concern of researchers was that this lack of information could potentially result in missed opportunities for researchers to collaborate on projects.

A majority of researchers and research organizations consulted for the evaluation reported that the timing of project selection and funding transfers are a significant problem for NCP funding recipients. The limited season and transportation logistics make timing critical in the North. A few respondents reported that late receipt of funding impinged on the completion of some projects. Others noted that large organizations could afford to bridge-finance their research projects, but that this option was not available to all funding recipients.

Coordination of NCP activities among the various participants has been important to program success. Researchers and northern-based organizations reported that NCP regional staff at AANDC in Whitehorse, Yellowknife and Iqaluit, as well as Inuit Research Advisors have been important for securing collaboration from northern communities. As noted previously, the Inuit Research Advisors facilitate engagement of Inuit in NCP research. Significant efforts have been made, particularly in the Northwest Territories, to link local NCP activities with other federal initiatives such as Cumulative Impact Monitoring Program and Environment Canada and Fisheries and Oceans programs.

Key informants observed that northern communities and researchers often had different perspectives and expectations about the conduct of research. One major project, the Inuit Health Survey, made it a high priority to address these differences and the result was considered to be an important success of the program. Subsequent to this project, an Inuit organization has developed guidelines for researchers as a result of their experience with resolving community-researcher differences.

Variations were noted in the overall effectiveness of Regional Contaminants Committees. One Regional Contaminants Committee is closely linked with the public health authority, has a protocol for analyzing the health implications of research, and a process for communications planning. Conversely, another Regional Contaminants Committee does not have a public health representative and does not communicate contaminant information. One only meets once a year to preview proposals and does not have a terms of reference or strategic planning or priority setting processes.

4.3.2 NSTP

With respect to program design and delivery, there was near unanimity among key informants that NSTP is a real success? — "great value for a small investment". They reported that the program succeeds in making the students the primary beneficiaries of the funding rather than project supervisors.

Members of the northern science community reported that science students increasingly are lacking practical and technical knowledge from their university programs. Because of this deficiency, they believe that NSTP-funded on-site experience is very important for aspiring student-scientists.

One concern identified by key informants is that the average grant per student recipient is about $2,800 in 2009-10 and 2010-11, which does not cover the full costs of northern travel. They also noted that limiting funding to Canadian students restricts the opportunities there might be to attract foreign students to northern science in Canada.

Another issue identified by key informants was that, with a decentralized model, NSTP funding is not targeted and there are no formal mechanisms to ensure that the program supports specific types of projects (as is the case with NCP blueprint process).

No major issues with program management and administration were identified. Key informants reported that the NSTP is effective at incorporating stakeholder interests. The broad composition of the NSTP Management Committee is a useful mechanism for ensuring stakeholder involvement. Key informants also reported that the Management Committee is an effective body, with northern representatives providing a good reality check to the overall process. This committee is especially effective at promoting the need for projects to go back to the local communities to discuss research results.

With respect to performance measurement, program activities and output statistics currently are being tracked. The NSTP has not yet reported on program impacts.

Key informants reported that NSTP has influenced stakeholder priorities and interests. For example, they reported that universities have broadened the range of subject areas covered by northern studies programs as result of interaction with the NSTP. Some universities have included social scientists on their Northern Studies Committees.

NCP

Recommendations

• It is recommended that to be effective in meeting Northern Science needs, NCP project proposal and reporting criteria demonstrate effective engagement with communities.
• It is recommended that the NCP project proposal process be enhanced, in order to foster information sharing and collaboration on approved projects.

NSTP

There were no recommendations for the NSTP emerging from the evaluation. However, it is suggested that program managers:

• Consider widening eligibility criteria for funding, in order to expand the potential pool of young northern researchers; and
• Consider using data collected for performance measurement and program management purposes to develop a greater understanding of student clients.

Efficiency

Successful coordination of NCP activities among federal departments and agencies has represented an important efficiency of the NCP, with expertise and resources drawn from appropriate sources as required. The use of northern structures and key individuals such as AANDC regional officers and Inuit Research Advisors has also led to efficiencies. As noted previously, Inuit Research Advisors facilitate engagement of Inuit in NCP research.

The most significant program efficiency issue for funding recipients was the annual funding cycle, which, according to their reports, has been characterized by the lengthy project selection process and significant delays in receipt of funding. These delays resulted in some projects not being completed as planned, a greater administrative workload for some project leaders, and the necessity of bridge financing from other sources. Most key informants stated the annual funding cycle was not sensitive to the short summer season and the limited amount of time available to conduct field work in the North. Two key informants indicated that their problems with the timing of funding could possibly discourage them from submitting future project applications. These key informants recommended that the program decrease the time gap between approval and release of funds, either by releasing funding before the summer or by providing approval agreements before the summer to make it easier for researchers to secure bridge funding from their organizations. Some organizations also recommended that the program make provisions for multi-year funding where appropriate.

From the program perspective, the project selection process is not lengthy but it is comprehensive and inclusive, and that a good balance between efficiency and effectiveness was found. Proposals for the new fiscal year are due in mid-Jan of the previous year, and funding decisions are made in early April (11-12 weeks). In the meantime, there are external peer reviews, social/cultural reviews by five committees, and technical reviews by review teams, followed by a final review and decision-making. As far as the delays in issuing project funds, the program is constrained by the federal government’s annual funding cycle and the administrative regime associated with grants and contributions, and program managers see little opportunity for improvement under the existing administrative requirements.

Most stakeholders agreed that the NCP has always been very community-oriented about communicating results. However, for a majority of stakeholders, the primary challenge of the NCP is communications, particularly with respect to communications with communities, which they believe could be improved in ways discussed earlier in the report. Communications between projects and with international audiences also were mentioned as being challenges, with one territorial government representative stating that the value of the program's input and contribution to global knowledge is not communicated to Inuit.

Variations in the approaches of Regional Contaminants Committees, described earlier in the report, may have led to some inefficiency in the overall approach to communicating contaminant information and linking sufficiently with local public health authorities. This inconsistency may have led to variations in the kinds of public messages emerging from NCP research. Consequently, more consistent messaging is a focus of the Government of Nunavut and Nunavut Tunngavik Inc. communications strategy for Nunavut in 2012, and is also a focus of a new NCP Risk Communications Subcommittee.

Economy

The evaluation did not examine program expenditures to compare costs of individual inputs with potentially less expensive alternatives. From a management perspective, the NCP is continually working to ensure that the program achieves the highest impact using the fewest resources.

The NCP achieves economies through its complementarity with other AANDC programs. The NCP has worked with ArcticNet and International Polar Year to promote coordination and cooperation in Canadian Arctic science. The NCP has played an important role in the development of these programs by providing a model for success in the areas of Aboriginal involvement, management structure, and the conduct of dual scientific and socio-cultural reviews. The NCP has also influenced the Canadian High Arctic Research Station as well, which is looking at the NCP’s blueprint process and management framework as possible models.

Most key informants indicated that the funding for the NCP has been sufficient to meet the basic needs of the researchers. Three areas that were identified as being limited by the availability of funding were field work, in-person interactions with northern communities, and participation in international conferences to communicate project results internationally.

6.1.1 NCP

What are the best practices and lessons learned in program design and implementation?

6.1.2 NSTP

What are the best practices and lessons learned in program design and implementation?

6.2.1 NCP

The evaluation reviewed documents for alternative delivery models and asked interview respondents if they had suggestions for alternative approaches for the NCP. There were no broad, overall alternative models recommended, but there were several suggestions for specific aspects of the NCP:

• Representatives of northern-based organizations recommended that, for future human health research, some NCP work should look at contaminants in a more holistic context, considering other health and lifestyle factors, to gain a better understanding of the health impacts of contaminants.
• Representatives of northern-based organizations also recommended that it may be better to hold Results Workshops every two years and use the resources saved to enable more scientists to spend time in communities.
• Health Canada's National First Nations Environmental Contaminants Program was identified as a useful model for how to attract projects from Aboriginal communities. The key features of this Health Canada program include third party administration by an Aboriginal organization, project applications from Aboriginal communities, local identification of needs and concerns about contaminants, local ownership of results with resulting community-led action, and linkages between northern communities and the contracted scientists.^{Footnote 4}
• The NCP could establish a clearinghouse or inventory of contaminants-related research projects conducted in Canada and internationally to increase overall awareness of the state of contaminants research and to help researchers to identify collaboration opportunities.

6.2.2 NSTP

No alternative delivery models were identified for the NSTP, but several participants in the evaluation noted that allowing non-Canadian students to obtain NSTP funding would expand the number of students conducting research in the North, and may help produce more future northern scientists.