Crown-Indigenous Relations and Northern Affairs Canada -
Five-Year Evaluation Plan 2022-2023 to 2026-2027

2.1 Roles and Responsibilities

The role of the Evaluation Branch is to provide quality and timely evaluations that contribute to the Department's comprehensive and reliable base of evidence to support policy and program improvement, expenditure management, senior management and Cabinet decision-making and public reporting. The Branch's evaluators scope and plan new evaluation projects as per the Policy on Results in order to contribute to helping address questions related to the delivery of results for Canadians.

The implementation and monitoring of the Plan falls to the Evaluation Branch of the Audit and Evaluation Sector (AES).

2.2  Purpose of the Evaluation Plan

The purpose of the Evaluation Plan is to meet accountability requirements and to help the Deputy Minister ensure that credible, timely and neutral information on the ongoing relevance and performance of program spending is available to support evidence‑based decision making. The Plan also:

• Provides an opportunity to align evaluations to ensure that the information needs of the Department and other evaluation users (e.g., other government departments) are being met;
• Helps ensure that evaluations supporting program redesign are planned and completed in advance of program renewal;
• Allows departmental units responsible for the development of the Departmental Plan and the Departmental Results Report, as well as other groups engaged in strategic planning and reporting activities, to identify when evaluations will be available to inform their work;
• Initiates regular communication and consensus-building on evaluation needs and priorities across the Department; and
• Provides central agencies with advanced notice of when evaluations will be available to inform their work (e.g., in support of Memoranda to Cabinet, Treasury Board Submissions, etc.).

Moreover, the Plan serves as an important tool for the Department's Head of Evaluation and the Evaluation Branch to manage project workflow and plan the activities of its human resources.

2.3 Policy Framework and Guidance

CIRNAC is subject to the Treasury Board Policy on Results (2016) and Directive on Results (2016)^{Footnote 1}. The Policy on Results allows departments to establish a results framework that identifies core responsibilities, associated results and appropriate indicators. It also limits requirements so that departments will collect performance information while remitting only some of it to Treasury Board. It allows departments to prioritize evaluation coverage and issues, while supporting the professionalization of the evaluation function.

The Policy requires departments to report on both planned program spending and ongoing programs of Grants and Contributions (Gs&Cs) evaluation coverage, but permits greater flexibility by allowing for risk-based planning for program spending coverage, and the option to defer evaluations beyond five years where average yearly expenditures are less than $5 million (over the preceding five-year period). The Policy on Results further recognizes that various types of evaluation are possible and essential to meeting the information needs of decision-makers.

The Directive on Results requires evaluations to focus on relevance, effectiveness and efficiency. It also clarifies many of the requirements in support of the Policy on Results, including roles and responsibilities of individuals and the Performance Measurement and Evaluation Committee (PMEC) as well as the expectations around planning departmental evaluation coverage.

2.4 Evaluation Planning Methodology

CIRNAC revised its Departmental Results Framework – which sets out its Core Responsibilities, Departmental Results and Departmental Results Indicators – and program inventory for 2022‑2023. While there were some modifications to departmental results (indicators), there were no additions or deletions to the program inventory^{Footnote 2}. The Evaluation Branch has taken the approach to plan for evaluations based on the core responsibilities and program inventory. This has resulted in the amalgamation of some previously distinct programs into larger programs, reflecting the streamlining efforts that have been undertaken in response to the Policy on Results. Occasionally, an evaluation is added to the plan because it has been identified as a departmental priority by senior management. The most recent example of this was the Evaluation of the Survey on Indigenous Peoples conducted on behalf of Indigenous Services Canada (ISC). Appendix A presents the Program Inventory for CIRNAC by Core Responsibility.

In addition, there is the requirement under the Financial Administration Act (Section 42.1) to evaluate all ongoing programs of Gs&Cs every five years. Credit for evaluation coverage is granted upon deputy head approval of an evaluation report, under the premise that approval allows for the sharing of information in support of management action and the sharing of results with the public. Evaluation planning is conducted with those requirements in mind, reviewing the last fiscal year in which a program was evaluated, and scheduling completion and approval of the program evaluation five years later.  

The planning timeframe for the Department's evaluations has typically been 12 months (e.g., the date of approval for the Terms of Reference signifies the official start of an evaluation and is generally 12 months before the planned approval date).

This Evaluation Plan is also developed in consultation with the Department's Internal Audit team, and with consideration for the Risk-Based Audit Plan. Consideration has been given to timing audits and evaluations in a way that does not overburden programs. When developing the Evaluation Plan, planned evaluation projects, scheduling and resources are discussed with senior management and then recommended for approval by the Deputy Minister. 

3.1 Departmental Core Responsibilitie

3.2 Program Structures

Departmental activities are largely structured by funding arrangements or formal agreements with Indigenous partners and/or provincial or territorial governments. Most departmental programs, representing a majority of spending, are delivered through partnerships with Indigenous organizations, Indigenous governments or Indigenous communities, and through federal‑provincial or federal‑territorial agreements.

Currently, CIRNAC has 25 active grant and contribution authorities (Gs&Cs) to support the 18 Programs in its program inventory and the multiple sub-programs, initiatives and activities that support those 18 Programs. Therefore, a one-to-one relationship does not exist in all cases between the authorities listed in the Main Estimates and the ongoing programs of Gs&Cs identified by  the Department. A number of Programs draw on multiple Gs&Cs; and conversely, there are Gs&Cs that contribute to multiple Programs. This plan has been developed using the Treasury Board's Interim Guide on Results, which was updated most recently in August 2018, where there is the recognition that a 'program' of Gs&Cs can cut across two or more Programs in the Program Inventory and the relationship is not always one‑to-one.  

3.3 Exemptions from Section 42.1 of the Financial Administration Act

According to Section 42.1 of the Financial Administration Act, government departments and agencies must review, at least once every five years, the relevance and effectiveness of each ongoing Gs&Cs program. Note that programs that are not funded through voted Gs&Cs are not subject to the Financial Administration Act requirements. However, as per the Policy on Results, programs with five-year average actual expenditures of less than $5 million per year can be exempted from Section 42.1 and do not need to be evaluated every five years. Guidance from Treasury Board Secretariat states the five-year average is to be calculated on three years of actuals (Public Accounts) and two years of planned spending (Main Estimates). Based on these parameters, the following programs are not scheduled for evaluation:

Program expenditures are examined every year during the development of the Plan to assess which programs fall below the threshold. There have been instances where a program has moved above the $5 million threshold. This most recently occurred with Northern Strategic and Science Policy, which will now be the subject of an evaluation in 2025-2026. An evaluation of Northern Regulatory and Legislative Framework, which falls below the $5 million threshold, will also be conducted in 2024-2025 as there is the capacity to conduct this evaluation in that fiscal year and evaluations of all spending should be conducted at some point to inform program improvements.

3.4 Planned Expenditures and Transfer Payments

According to the Main Estimates, total planned spending for 2022-2023 is outlined in Table 2 by core responsibility.

4.1 Status of Evaluation Completion in 2021-22

The CIRNAC evaluation function continues to report through CIRNAC's Chief Audit and Evaluation Executive. Looking back on the previous year, three evaluation reports addressing CIRNAC program spending were completed and approved:

• Evaluation of Engagement and Capacity Support (Federal Interlocutor's Contribution Program, Basic Organizational Capacity, and Consultation and Policy Development)
• Evaluation of the Exceptional Contracting Limits Authority 2015-2016 to 2019-2020
• Evaluation of the Northern Contaminated Sites Program

Also completed was the Evaluation of the Survey on Indigenous Peoples, conducted on behalf of Indigenous Services Canada, the department that takes the lead for the survey.

In addition, the following evaluations were substantially completed in 2021-22 and are in the final approval stage, which is expected in Q1 2022-2023:

• Evaluation of Specific Claims
• Evaluation of the Management and Implementation of Agreements and Treaties
• Evaluation of Climate Change Adaptation and Clean Energy

4.2  Implementing the Plan

Evaluations are used to inform decision-making and reporting (such as within the Departmental Plan and the Departmental Results Report), to assess progress made by programs in achieving expected results and to incorporate lessons learned in order to improve program design and delivery. The Branch posts approved evaluation reports on its website, as is required, after sharing reports with Treasury Board.

The Evaluation Branch's evaluators work collaboratively with partners and stakeholders to tailor evaluation projects to meet the needs of decision-makers, as well as perhaps experimenting with new and innovative methodologies and approaches for use in the next generation of federal evaluation. The Policy on Results provides for more flexibility in the various types of evaluations and the identification of issues, allowing the Branch more flexibility in scoping and calibrating its work.

The Department is working to build capacity in collaboration with Indigenous partners to improve the design and implementation of programming. In support of that objective, evaluators work closely with the program managers to assess the availability of Indigenous representatives to participate in evaluation working groups.

4.3 Planned Evaluation Coverage

Table 3, Planned Coverage of Gs&Cs, enumerates the planned evaluations that will cover 100 per cent of CIRNAC's voted Gs&Cs amounts.

Table 4, Planned Coverage Departmental Planned Spending, enumerates CIRNAC's 17 planned evaluations covering 91 per cent of planned departmental spending from 2022‑2023 to 2026-2027. These coverage figures include grants and contribution funding as well as salaries, operations, maintenance, statutory payments, loans, capital and the cost of employee benefits plans. 

Areas that are not subject to evaluation activities include: Internal Services ($124,253,346 – 2.1%); Loans ($25,903,000 – 0.5%), and programs and other spending that are not funded through grants and contributions. The bulk of the non-Gs&Cs planned spending is for the Other Claims Program ($410,000,000 – 7%).

4.4 Other Evaluation Branch Activities

In addition to conducting evaluations, the Evaluation Branch undertakes a wide range of activities to support and strengthen evaluation and performance measurement work. These include providing advice and support on performance measurement, corporate planning and reporting at the departmental level, supporting capacity building and outreach activities, and developing research and special studies on evaluation issues.

4.5 Challenges