Five-Year Evaluation Plan 2018-19 to 2022-23

Transformation of INAC into CIRNAC and ISC

In August 2017, the Prime Minister announced the dissolution of Indigenous and Northern Affairs Canada (INAC), and the creation of two new departments designed to better meet the needs of the people they serve, to accelerate self-determination and the closing of socio-economic gaps and to advance reconciliation.

The work that CIRNAC is doing is transforming how we relate to, and work in partnership with, Indigenous peoples, so as to accelerate the renewal of our relationship with First Nations, Inuit and Métis. The Department's activities focus on modernizing institutional structures and governance so that First Nations, Inuit, and Métis peoples can build capacity that supports the implementation of their vision of self-determination. CIRNAC also leads the Government of Canada's work in the North, supporting northern programming, governing institutions, and scientific initiatives.

ISC, a service organization, was created to ensure a consistent, distinctions-based approach to the delivery of services to Indigenous peoples. The Department's activities focus on improving access to high-quality services for First Nations, Inuit and Métis, supporting and empowering Indigenous peoples to control the delivery of those services, and improving the socio-economic conditions and quality of life in their communities. In addition, the First Nations and Inuit Health Branch (FNIHB) was transferred from Health Canada to ISC in 2017, as the fourteen health programs complement the Department's responsibilities around income assistance, education and infrastructure. 

Because the process of creating two new departments will take place over the next fiscal year, including finalizing the organizational structure for programs and internal services, EPMRB has developed an evaluation plan that takes into account the necessary transition period and the increased volume of activity associated with the addition of fourteen FNIHB programs. 

2.1 Roles and Responsibilities

Across the Government of Canada, evaluation is understood as the systematic and neutral collection and analysis of evidence to judge merit, worth or value. This is done to provide an assessment of, and allow for the judgement of, the relevance, performance, and efficiency with which resources are utilized. It helps to identify possible alternative ways to contribute to or achieve the same results in a more effective and efficient manner. The information evaluation provides is used to inform decision-making and make improvements.

The purpose of the evaluation function is to ensure that CIRNAC and ISC have a comprehensive, timely and reliable base of evidence to support policy and program improvement, expenditure management exercises, senior management and Cabinet decision making and public reporting. Evaluations are playing an increasingly important role in decision making, improvements, innovation and accountability across government. They are intended to provide transparent, clear and useful information on the results that departments have achieved and the resources used to do so, in order to manage and improve programs, policies and services as well as to allocate resources based on performance to optimize results.

2.2 Purpose of the Evaluation Plan

The primary purpose of the evaluation plan is to help the deputy heads ensure that credible, timely and neutral information on the ongoing relevance and performance of direct program spending and ongoing programs of grants and contributions is available to support evidence-based decision making. The plan also:

• Provides an opportunity to align evaluations to ensure that the information needs of the departments and other evaluation users (e.g., other government departments) are being met;
• Helps ensure that evaluations supporting program redesign are planned and completed in advance of program renewal;
• Allows departmental units responsible for the development of the Departmental Plan and the Departmental Results Reports, as well as other groups engaged in strategic planning and reporting activities, to identify when evaluations will be available to inform their work;
• Initiates regular communication and consensus-building on evaluation needs and priorities across the Departments; and
• Provides central agencies with advanced notice of when evaluations will be available to inform their work (e.g., in support of Memoranda to Cabinet, Treasury Board submissions, etc.).

Moreover, the plan serves as an important tool for the Head of Evaluation and EPMRB to manage project workflow and plan the activities of its human resources. In previous years, status reports on the plan were regularly presented at meetings of INAC's Evaluation, Performance Measurement and Review Committee to provide assurance of timely implementation, and alert the Committee to any required changes or additions. During this period of Transformation, evaluation plan review and approval will be done according to the approval processes as determined by each department. Although for this transition year, the plan has been prepared as a joint plan, in future years, separate plans for each department may be considered.

2.3 Policy Framework and Guidance

CIRNAC and ISC are subject to the Treasury Board Policy on Results (2016) and Directive on Results (2016). The Policy on Results allows departments to establish a results framework that identifies core responsibilities, associated results and appropriate indicators. It also limits requirements so that departments will collect performance information while remitting only some of it to Treasury Board. It allows departments to prioritize evaluation coverage and issues, while professionalizing the evaluation function.

It will continue to ask departments to report on both direct program spending and ongoing programs of grants and contributions evaluation coverage, but permits greater flexibility by allowing for risk-based planning for direct program spending coverage, and the option to defer evaluations beyond five years where average yearly expenditures are less than $5 million (over the preceding five year period). The Policy on Results further recognizes that various types of evaluation are possible and essential to meeting the information needs of decision makers.

The Directive on Results requires evaluations to focus on relevance, effectiveness and efficiency. It also clarifies many of the requirements in support of the Policy on Results, including roles and responsibilities of individuals and the evaluation committee as well as the expectations around planning departmental evaluation coverage.

2.4 Evaluation Planning Methodology

Under the Treasury Board Policy on Results, the Department had until November 2017 to replace its Management, Resource and Results Structures (i.e., Strategic Outcome and Program Alignment Architecture) with a Departmental Results Framework – used to report its high level, core responsibilities and outcomes – and a Program Inventory. A transitional 2018-2019 Departmental Results Framework and Program Inventories were approved. The first step in preparing the evaluation plan was transitioning to the new Departmental Results Frameworks and Program Inventories for CIRNAC and ISC. Appendix A presents the program inventory for CIRNAC and ISC by core responsibility.

EPMRB has taken the approach to plan for evaluations based on these Transitional Core Responsibilities and Program Inventories for 2018-19. This has resulted in the amalgamation of some previously distinct programs into larger programs, reflecting the streamlining efforts that have been undertaken in response to the Policy on Results.

In addition, there is the requirement under the Financial Administration Act (Section 42.1) to evaluate all ongoing programs of grants and contributions every five years. Credit for evaluation coverage is granted upon deputy head approval of an evaluation report, under the premise that approval allows for the sharing of information in support of management action and the sharing of results with the public. Evaluation planning is conducted with those requirements in mind, reviewing that 'last fiscal year' in which a program was evaluated, and scheduling for the completion and approval of the program evaluation five years later.

The planning timeframe for the Department's evaluations has typically been 12 months (e.g. the date of approval for the Terms of Reference – the start of the evaluation – is generally 12 months before the planned approval date), while the FNIHB evaluations recently transferred from Health Canada/Public Health Agency of Canada have generally been planned on a 14-18 month timeframe.

This evaluation plan was also aligned with the Multi-Year Risk-Based Audit Plan prepared by the Audit and Assurance Services Branch (AASB). Both the AASB and EPMRB are part of the AES, serving both CIRNAC and ISC. Consideration has been given to timing audits and evaluations of program in a way that does not over burden programs.

Consultations with CIRNAC and ISC leadership/senior management and with Treasury Board officials have also influenced which programs will be evaluated during which fiscal year.

Each evaluation on the plan has been further assessed as to the type of evaluation required (e.g., impact, formative) based on the review of Financia Administration Act or Treasury Board requirements and stakeholder consultations, as well as noting whether it is a horizontal or non-horizontal evaluation.

3.1 Departmental Strategic Outcomes

3.2 Program Structures

Departmental activities are largely structured by funding arrangements or formal agreements with First Nations and/or provincial or territorial governments. Most departmental programs, representing a majority of spending, are delivered through partnerships with Indigenous communities and federal-provincial or federal-territorial agreements.

Currently, CIRNAC has 27 authorities (voted grants and contributions) to support the 23 programs in its Program Inventory. ISC has 20 authorities to support the 26 programs in its Program Inventory. A one-to-one relationship between the authorities listed in the Main Estimates and the Program Inventories of ongoing programs of grants and contributions deemed a 'program' by the departments exists in the minority of instances.

A number of programs draw on multiple grants and contributions; and conversely, there are grants and contributions that contribute to multiple programs. This plan has been developed using the Treasury Board's Interim Guide on Results, which was updated most recently in November 2017, where there is the recognition that a 'program' of grants and contributions can cut across two or more programs in the Program Inventory and the relationship is not always one-to-one.  

3.3 Exemptions from Section 42.1 of the Financial Administration Act

According to Section 42.1 of the Financial Administration Act, government departments and agencies must review, at least once every five years, the relevance and effectiveness of each ongoing grant and contribution program. However, as per the Policy on Results, programs that average below the $5 million over five years can be exempted from Section 42.1 and so need not be evaluated every five years.

Previously, this calculation was based on the average of five years of actuals in the preceding years of the current plan year; latest guidance from Treasury Board Secretariat is three years of actuals and two years of Main Estimates, which has resulted in some changes to the five year evaluation schedule. Based on those parameters, the following programs are not scheduled for evaluation:

CIRNAC (by 2018-2019 Program Inventory Number):

1. Statutory, Legislative and Policy Support to First Nations Governance
5. Consultation and Accommodation
9. Other Claims
13. Residential Schools Resolution
18. Northern Strategic and Science Policy
19. Northern Regulatory and Legislative Frameworks
20. Northern and Arctic Environmental Sustainability
22. Canadian High Arctic Research Station

ISC does not have any programs that fall into this category.

3.4 Planned Expenditures and Transfer Payments

According to the 2018–2019 Main Estimates, total forecasted spending for 2018-19 is outlined in the table below according to each Department's core responsibilities.

Source: 2018-19 Main Estimates. Figures include the following expenditures: salary; operations and maintenance; capital; statutory and grants and contributions funding; and, the costs of employee benefit plans.

Source: 2018-19 Main Estimates. Figures include the following expenditures: salary; operations and maintenance; capital; statutory and grants and contributions funding; and, the costs of employee benefit plans.

4.1 Status of Evaluation Completion in 2017-18

Looking back on the previous year, six evaluation reports addressing direct program spending were approved. These evaluations were:

• Evaluation of Implementation and Enforcement Supports for the Family Homes on Reserves and Matrimonial Interests or Rights Act (Matrimonial Real Property)
• Evaluation of Land and Water Management
• Evaluation of Family Violence Prevention
• Horizontal Evaluation of the National Action Plan to Combat Human Trafficking
• Evaluation of the Horizontal Coordination Function of the Family Violence Initiative
• Evaluation of Climate Change Adaptation and Clean Energy

Looking forward, the following evaluations, which were on the plan for completion in 2017-18, are now expected to be completed and approved in 2018-19:

• Evaluation of Specific Claims
• Evaluation of Income Assistance
• Evaluation of Assisted Living
• INAC Evaluation of the Strategic Partnerships Initiative Component of the West Coast Energy Initiative
• Evaluation of Nutrition North Canada
• Evaluation of Clinical and Client Care (First Nations and Inuit Health)

There was one internal evaluation on the 2017-18 plan, the Evaluation of Annual Agreement Management. The Evaluation, Performance Measurement and Review Committee agreed with a recommendation that it be discontinued and that other work, specifically the Department's engagement activities, speak to this topic instead.

AES staff participated as note-takers across the country in the engagement sessions around the new approach for mutual transparency and accountability between First Nations and the Government of Canada. As the Evaluation of Annual Agreement Management was an evaluation of internal services and there were no grants and contributions attached, there is no obligation to evaluate and it is not planned to be rescheduled.

4.2 Implementing the Plan

Evaluations conducted by EPMRB are used to inform decision making and reporting (such as the Departmental Plan, Departmental Results Reports), to assess progress made by programs in achieving expected results and to incorporate lessons learned in order to improve program design and delivery.

During this transformation year, EPMRB will look to working collaboratively with partners and stakeholders to tailor evaluation projects to meet the needs of decision makers, as well as perhaps experimenting with new and novel methodologies and approaches for use in the next generation of federal evaluation. The Policy on Results provides for more flexibility in the various types of evaluations and the identification of issues, allowing EPMRB more flexibility in scoping its work.

EPMRB plans to move towards developing and implementing protocols for engagement with Indigenous representative organizations so that First Nations, Inuit and Métis peoples can contribute their knowledge and experience throughout the evaluation process.

Table 3 provides an overview of evaluation expenditures and human resources, both planned and actual over a three-year period. Note that this does not reflect cost recovery amounts.

Source: Audit and Evaluation Sector, Business Management Unit
¹ Actuals reflect net expenditures on contracted services less cost recoveries received from programs. For 2018-19, amount reflects required funds to be transferred from Health Canada/Public Health Agency of Canada for increased workload resulting from FNIHB evaluations.
² As of the date of this plan, full-time equivalents for 2018-19 are not yet known due to ongoing discussions with Treasury Board in regard to departmental resources.

EPMRB will work to address vacancies on an ongoing basis as it has seldom been at full capacity given the general movement of staff to other opportunities and the timelines needed to fill positions. EPMRB also leverages the skills and talents of undergraduate and graduate students throughout the year.

4.3 Planned Evaluation Coverage

CIRNAC plans for 18 evaluations covering 86.50 percent of direct program spending from 2018-19 to 2022-23. Programs that average less than $5 million per year over the preceding five years, and are thus exempt from Section 42.1 of the Financial Administration Act, represent 5.2 percent of total organizational spending. The remaining 8.3 percent total organizational spending is Internal Services ($204,678,749) and Loans ($56,303,000). Neither Internal Services nor the Loans are required to be evaluated under the Policy on Results.

ISC plans for 24 evaluations covering 99.3 percent of direct program spending from 2018-19 to 2022-23. The remaining 0.7 percent represents Internal Services ($64,986,485). There are no programs below the $5 million threshold.

4.4 EPMRB Performance Metrics

EPMRB has established performance measures to ensure progress towards coverage requirements. The Audit and Evaluation Sector Business Plan includes targets for EPMRB regarding the initiation, progression and completion of scheduled evaluations.

4.5 Other EPMRB Activities

In addition to conducting evaluations, EPMRB undertakes a wide range of activities to support and strengthen evaluation and performance measurement work. These include providing advice and support on performance measurement, corporate planning and reporting at the departmental level, supporting capacity building and outreach activities, and developing research and special studies on evaluation and performance measurement issues.