Crown-Indigenous Relations and Northern Affairs Canada - Five-Year Evaluation Plan 2023-2024 to 2027-2028

2.1 Roles and Responsibilities

The role of the Evaluation Branch is to provide quality and timely evaluations that contribute to the Department's comprehensive and reliable base of evidence to support policy and program improvement, expenditure management, senior management and Cabinet decision-making and public reporting. The Branch's evaluators scope and plan new evaluation projects as per the Policy on Results in order to contribute to helping address questions related to the delivery of results for Canadians.

The implementation and monitoring of the Plan falls to the Evaluation Branch of the Audit and Evaluation Sector (AES).

2.2 Purpose of the Evaluation Plan

The purpose of the Evaluation Plan is to meet accountability requirements and to help the Deputy Minister ensure that credible, timely and neutral information on the ongoing relevance and performance of program spending is available to support evidence based decision making. The Plan also:

• Provides an opportunity to align evaluations to ensure that the information needs of the Department and other evaluation users (e.g., other government departments) are being met;
• Helps ensure that evaluations supporting program redesign are planned and completed in advance of program renewal;
• Allows departmental units responsible for the development of the Departmental Plan and the Departmental Results Report, as well as other groups engaged in strategic planning and reporting activities, to identify when evaluations will be available to inform their work;
• Initiates regular communication and consensus-building on evaluation needs and priorities across the Department; and
• Provides central agencies with advanced notice of when evaluations will be available to inform their work (e.g., in support of Memoranda to Cabinet, Treasury Board Submissions, etc.).

Moreover, the Plan serves as an important tool for the Department's Head of Evaluation and the Evaluation Branch to manage project workflow and plan the activities of its human resources.

2.3 Policy Framework and Guidance

CIRNAC is subject to the Treasury Board Policy on Results (2016) and Directive on Results (2016)^{Footnote 1} . The Policy requires departments to report on both planned program spending and ongoing programs of Grants and Contributions (Gs&Cs) evaluation coverage, but permits greater flexibility by allowing for risk-based planning for program spending coverage, and the option to defer evaluations beyond five years where average yearly expenditures are less than $5 million (over the preceding five-year period).

The Directive on Results requires evaluations to focus on relevance, effectiveness and efficiency. It clarifies many of the requirements in support of the Policy on Results, including roles and responsibilities of individuals and the Performance Measurement and Evaluation Committee (PMEC) as well as the expectations around planning departmental evaluation coverage.

2.4 Evaluation Planning Methodology

The Evaluation Branch plans for evaluations based on the Core Responsibilities and Program Inventory as set out in the Departmental Results Framework (DRF). The 2023-24 CIRNAC DRF was updated with the goal of being able to improve the Crown-Indigenous Relations results story and improve the comprehension and measurement of certain Departmental Results and Departmental Results Indicators. CIRNAC has revised its DRF for 2023 2024 (Appendix A) and it is expected to remain stable for the next three years ^{Footnote 2}. While the Core Responsibilities remain unchanged, there have been several changes to the Program Inventory noted below (Section 3.1). These changes have been taken into account in the development of the Plan.

Occasionally, an evaluation is added to the Plan because it has been identified as a departmental priority by senior management. However, for the 2023-24 to 2027-28 Plan, there are no evaluations that have been added to the Plan as a departmental priority. As the Plan is updated every year, priority projects are added when identified and when resources are available to carry them out.

The Financial Administration Act (Section 42.1) requires departments to evaluate all ongoing programs of Gs&Cs every five years and evaluation planning is conducted in accordance with that requirement. During the evaluation planning process, all evaluations that have been approved in the last fiscal year, in this case 2022-2023, are rolled into the final year of the new Plan, and scheduled for completion and approval by the end of the fifth fiscal year.

The timelines used for completing the Department's planned evaluations have typically been 12 months (e.g., the date of approval for the Terms of Reference signifies the official start of an evaluation and is generally 12 months before the planned approval date). This is based on the Evaluation Branch's experiences of the time taken to conduct past evaluations. The 12-month timeline is used to identify a date to present an evaluation's Terms of Reference to the PMEC for approval and then identifying a planned approval date for the evaluation report.

This Evaluation Plan was also developed in consultation with the Department's Internal Audit team, and with consideration for the Risk-Based Audit Plan. Consideration has been given to timing audits and evaluations in a way that does not overburden programs.

When developing the Evaluation Plan, planned evaluation projects, scheduling and resources are discussed with departmental management and consultations are conducted to obtain input from Sectors. The Plan is then presented to the Department's Performance Measurement and Evaluation Committee (PMEC) for review. Final approval is then obtained from the Deputy Minister.

3.1 Departmental Core Responsibilities

CIRNAC continues to renew the nation-to-nation, Inuit-Crown, and government-to-government relationship between Canada and First Nations, Inuit and Métis. The Department's activities focus on modernizing Government of Canada structures to enable Indigenous peoples to build capacity and support their vision of self determination. CIRNAC also leads the Government of Canada's work in the North.

The 2023-2024 Core Responsibilities for CIRNAC are:

• Crown-Indigenous Relations — with 10 supporting programs; and
• Northern Affairs — with 8 supporting programs.

The Program Inventory has changed from previous years to reflect changes made to CIRNAC programming and allow for better financial and reporting results. Specifically, the:

• merging of three programs (Consultation and Policy Development, the Federal Interlocutor's Contribution Program; and Basic Organizational Capacity) into a single Program called Indigenous Engagement and Capacity Support; and
• creation of a new Program: Indigenous-Led Services.

New to the Program Inventory is the Residential Schools Legacy Program, which was created in 2022-23 and added to the 2023-24 DRF. A horizontal initiative Implementing the Federal framework to address the legacy of residential schools^{Footnote 3}, exists within, but distinct from, the Residential Schools Legacy Program. CIRNAC is the lead department and the initiative involves Indigenous Services Canada, Library and Archives Canada, Canadian Heritage, Parks Canada Agency, the Royal Canadian Mounted Police, and the Department of Justice.

Finally, there was a modification to the name of the Residential Schools Resolution Program, which is now called the Indian Residential Schools Settlement Agreement.

3.2 Program Structures

Departmental activities are largely structured by funding arrangements or formal agreements with Indigenous partners and/or provincial or territorial governments. Most departmental programs, representing a majority of spending, are delivered through partnerships with Indigenous organizations, Indigenous governments or Indigenous communities, and through federal provincial or federal territorial agreements.

Currently, CIRNAC has 25 active grant and contribution authorities (Gs&Cs) to support the 18 Programs in its Program Inventory and the multiple sub-programs, initiatives and activities that support those 18 Programs. A one-to-one relationship does not exist in all cases between the authorities listed in the Main Estimates and the ongoing programs of Gs&Cs identified by the Department. A number of Programs draw on multiple Gs& and conversely, there are Gs&Cs that contribute to multiple Programs. This plan has been developed using the Treasury Board's Interim Guide on Results, which was updated most recently in August 2018, where there is the recognition that a "program" of Gs&Cs can cut across two or more Programs in the Program Inventory and the relationship is not always one to-one.

3.3 Exemptions from Section 42.1 of the Financial Administration Act

As noted, the Financial Administration Act (Section 42.1), requires government departments to review, at least once every five years, the relevance and effectiveness of each ongoing Gs&C program. However, as per the Policy on Results, programs with five-year average actual expenditures of less than $5 million per year can be exempted from Section 42.1 and do not need to be evaluated every five years. Guidance from Treasury Board Secretariat states the five-year average is to be calculated on three years of actuals (Public Accounts) and two years of planned spending (Main Estimates). TBS provides coverage tables (see Section 4.3) to departments for their use in calculating these averages. Based on these parameters, the following programs are not scheduled for evaluation:

Program expenditures are examined every year during the development of the Plan to assess which programs fall below the threshold. There are two Programs that fall under the $5 million threshold:

• Northern Strategic and Science Policy. This program has been on the Plan for an evaluation in 2025-2026 as it has previously been above the threshold; the capacity exists in that fiscal year to conduct the evaluation.
• Northern Regulatory and Legislative Frameworks. This program is consistently below the $5 million threshold but an evaluation is planned in 2024-2025 as there continues to be the capacity to conduct the evaluation in that fiscal year and evaluations of all spending should be conducted at some point to inform program improvements.

3.4 Planned Expenditures and Transfer Payments

As per the Main Estimates, total planned spending for 2023-2024 is outlined in Table 2.

4.1 Status of Evaluation Completion in 2022-23

Looking back on the previous year, five evaluation reports addressing CIRNAC program spending were completed and approved:

• Evaluation of the Management and Implementation of Agreements and Treaties;
• Evaluation of Specific Claims;
• Evaluation of Climate Change Adaptation and Clean Energy;
• Evaluation of the Impact Assessment and Regulatory Processes Horizontal Initiative; and
• Horizontal Evaluation of the Federal Leadership Towards Zero Plastic Waste in Canada Initiative.

4.2 Implementing the Plan

Evaluations are used to inform decision-making and reporting (such as within the Departmental Plan and the Departmental Results Report), to assess progress made by programs in achieving expected results and to incorporate lessons learned in order to improve program design and delivery. The Evaluation Branch posts approved evaluation reports on its website after sharing reports with Treasury Board.

The Evaluation Branch's evaluators work collaboratively with partners and stakeholders to tailor evaluation projects to meet the needs of decision-makers, as well as perhaps experimenting with new and innovative methodologies and approaches for use in the next generation of federal evaluation. The Policy on Results provides for more flexibility in the various types of evaluations and the identification of issues, allowing the Branch more flexibility in scoping and calibrating its work.

The Department is working to build capacity in collaboration with Indigenous partners to improve the design and implementation of programming. In support of that objective, the Evaluation Branch emphasizes the composition of Evaluation Working Groups (EWGs) to help ensure that EWGs are representative of the entity being evaluated and that Indigenous partners can participate in the governance of the evaluation. EWG members provide advice and guidance that helps with planning evaluations, interpreting evaluation findings, and developing recommendations. The evaluation teams are strongly committed to being culturally responsive and, in addition to personal practices of reconciliation, are guided by the departmental mandate to renew the nation-to-nation relationships and support self-determination. As such, there have been events in recent years, such as the discovery of unmarked graves, that have necessitated placing some evaluations on pause to demonstrate respect and care for the impact on key stakeholders and partners involved in the evaluations. Overall, in undertaking evaluations, CIRNAC is focused on advancing new relationships with Indigenous Peoples. At times, this means that evaluations will take more time than planned and that, to be respectful and inclusive, delays may be experienced.

In planning evaluation projects, the Evaluation Branch scopes each project to avoid overlap. For example, the Northern Contaminated Sites Program receives some of its funding from the Federal Contaminated Sites Action Plan (FCSAP) — a horizontal initiative that spans multiple departments, led by Environment and Climate Change Canada. FCSAP conducts the evaluation of the horizontal initiative as per its own requirements. On that basis, Evaluation Branch would scope its own evaluation project to look at areas or activities not evaluated by the horizontal initiative.

Additional planning for individual evaluation projects includes the application of GBA Plus, which is central throughout all phases of CIRNAC evaluations. From inception through to report writing, GBA Plus is a key consideration built into all phases, including in development of the key evaluation questions and the evaluation matrix. For example, in early discussions with the program, the evaluation teams ask for information on how the program intersects with GBA Plus issues and whether there are any data collected.

4.3 Planned Evaluation Coverage

The Policy on Results requires departmental evaluation plans to present planned evaluation coverage of both the department's ongoing programs of Gs&Cs, as listed in the Program Inventory, as well as for the total planned departmental spending ^{Footnote 4}. The goal is for a department to demonstrate that it is meeting its obligation to evaluate all departmental and Program spending, and if 100 per cent coverage is not planned, to explain exceptions. There are two 'coverage tables' annexed to this Plan, the templates (spreadsheets) of which are produced by Treasury Board Secretariat. These templates are used in the planning process to calculate coverage, which means the level (in dollars) and percentage of spending that planned evaluations are expected to address. The results of the calculations produced by the coverage tables are used to analyze the extent to which the department is meeting its evaluation obligations.

Table 3, Planned Coverage of Departmental Gs&Cs Spending, summarizes the results of that analysis and lists the number of planned evaluations that will cover 100 per cent of CIRNAC's voted Gs&Cs amounts.

Table 4, Planned Coverage of Departmental Planned Spending, summarizes the results of the coverage table analysis and lists the number of CIRNAC's planned evaluations, by fiscal year, showing that coverage is at 60% per cent of planned departmental spending from 2023–24 to 2027-28. These spending figures include grants and contribution funding as well as salaries, operations, maintenance, statutory payments, capital and the cost of employee benefits plans.

Areas not subject to evaluation activities include: Internal Services ^{Footnote 5} ($151,520,293 – 1.7%); and programs and other spending that are not funded through grants and contributions. The bulk of the non-Gs&Cs planned spending is for the Other Claims Program ($3,469,744,942 – 38%).

4.4 Evaluation Branch's Other Activities

In addition to conducting evaluations, the Evaluation Branch undertakes activities to support and strengthen evaluation and performance measurement work increasing transparency and accountability. These activities include providing advice and support on performance measurement, corporate planning and reporting at the departmental level, supporting capacity building and outreach activities, and developing research and special studies on evaluation issues. Some of these activities are described below.

4.5 Challenges