Archived - Evaluation of the Intervention Policy

Please note the expression "First Nations under Intervention Policy" or "First Nations under a specific level of intervention" applies to intervention in relation to the funding received by First Nations and organizations from INAC.

Introduction

Indian and Northern Affairs Canada's (INAC) current Intervention Policy was adopted in 2006 and is designed to ensure the ongoing delivery of programs and services, and to maintain accountability while defaults under funding agreements are addressed by recipients (i.e. First Nations, Aboriginal Organizations). Over the long term, application of the policy should lead to improved performance by recipients and a reduction in the number of First Nations that have seen their funding put under intervention policy[Note 1] and in the duration of such intervention.

There are four triggers for intervention: the terms and conditions of the funding agreement are not met by the recipient; the recipient's auditor gives a denial of opinion or an adverse opinion; the recipient has incurred a cumulative operating deficit equivalent to eight percent or more of its annual operating revenues; and the health, safety or welfare of First Nation members is being compromised. There are three main levels of intervention: recipient-managed, co-managed and third party managed.

The objectives of this evaluation are to assess the relevance, success, cost-effectiveness, design and delivery of the Intervention Policy, and to explore improvements to the policy and alternatives to it. At the time of the evaluation, INAC was working on a new Intervention Policy (tentatively named the Default Prevention and Management Policy). This new policy is seeking to be more aligned with INAC's new orientation and tools regarding capacity development with a focus on prevention and ongoing sustainability. This evaluation is also expected to inform the development of the new policy.

Methodology

Six lines of evidence informed the evaluation:

1. A literature review.
   
   
2. A document and file review.
   
   
3. A data analysis of INAC's performance and financial data.
   
   
4. Key informant interviews with INAC regional and Headquarter officials, other key federal departments and key people from other organizations; as well as a select number of First Nations and other Indian-administered organizations, third party managers, co-managers and funding services officers for the case studies. A total of 85 interviews were conducted with individuals or groups of individuals by phone, teleconference or during site visits for the case studies.
   
   
5. Case studies of 24 First Nations and three other Indian-administered organizations. The case studies were designed to be broadly representative of the statistics on intervention as well as other variables, such as regions, remoteness, size and community well-being. They included successful and less successful recipients.
   
   
6. Subject expert panel to review the evaluation approach and preliminary findings and conclusions.

The evaluation was conducted from January to June 2010 and included site visits to five regions, video conferences with two regions, and telephone interviews with two regions. The evaluation report was revised by the Evaluation, Performance Measurement and Review Branch (EPMRB) based on the findings of research conducted by the consulting firm Institute On Governance and EPMRB.

Evaluation Findings

The Intervention Policy Continues to be Relevant:
The evaluation demonstrates that the Intervention Policy remains relevant and there is a continued need to protect funding for programs and services, to ensure the delivery of essential services, and to ensure accountability for how federal monies are spent. There is also a continued need to build the capacity of recipients to manage and administer funds. The Intervention Policy is consistent with federal government and INAC plans and priorities, and the federal government's roles and responsibilities. INAC's Intervention Policy has been used as the basis for the intervention policies of Health Canada (HC) and the First Nations Financial Management Board (FNFMB); and INAC, HC and FNFMB are working together to coordinate changes to the policy.

The Intervention Policy Has Had Mixed Success in Achieving its Objectives:
There has been a modest reduction in the number of recipients under intervention, particularly in the last few years. There has also been a reduction in the number of recipients with a cumulative operating deficit of eight percent or higher. The Intervention Policy has been successful in ensuring the ongoing delivery of essential services, maintaining the health and safety of band members, and ensuring the external accountability of recipients to INAC.

On the other hand, the level of intervention has not decreased – in fact, an increasing proportion of First Nations are under co-management and a decreasing proportion of First Nations are under recipient managed intervention over the past ten years. Also, a number of First Nations that have been under some form of intervention policy for a long period of time – for example, 42 percent of First Nations under some form of intervention as of March 31, 2010, had been under intervention for ten or more years.

Any changes in the incidence, level and duration of intervention cannot necessarily be attributed solely to the implementation of the policy since other factors, such as access to own source revenue, economic performance in certain sectors, or the supply of credit can also have an impact. The major deficiency under the policy identified by the evaluation is capacity building.

A Number of Factors Affect the Success of the Intervention Policy:
There are a number of underlying factors related to governance, management, community engagement and funding practices that affect the success of the Intervention Policy. There are also a number of contextual factors related to community endowments (natural resources, geographic location, economic opportunities and access to own source revenue, the strength of the private sector, population size and growth); community member endowments (education, health, cultural identity); and social cohesion and security that could affect the success of the policy. Many of these factors are outside of the control of the Intervention Policy.

The key success factors identified by the evaluation are: willing and committed leadership in recipients, stable and competent management, community engagement, positive external relationships, and access to significant own source revenue.

The Intervention Policy is not Cost-Effective:
Considerable time and effort is required from INAC and recipients to implement the Intervention Policy. The cost of co-managers and third party managers affects the availability of band support funding for governance and administration in recipients. Third party managers are not able to use surpluses to pay off debt. Any improvements in the results of intervention as well as other interventions outside of the policy to address willingness, capacity and low own source revenue should increase cost-effectiveness.

The Intervention Policy is not Well Designed to Achieve all of its Objectives:
The major gaps in the design of the Intervention Policy identified by the evaluation are: a capacity development framework, prevention strategies, and common assessment tools. Some key informants suggested that INAC provide additional funding for co-managers and third party managers in order to increase the willingness of recipients, enhance recipients' capacity, and improve INAC's ability to monitor. Some key informants also suggested that alternative debt retirement tools are needed for recipients with little access to own source revenue.

INAC's Intervention Policy is Similar to Other Intervention Approaches:
INAC's Intervention Policy is similar to provincial government approaches to their subnational entities (municipalities, school boards and hospitals) and to other countries' approach to Aboriginal recipients in terms of objectives, triggers, levels of intervention and remedies, and oversight and monitoring. The key differences are that provincial governments are better able to control the debt of their subnational entities under a legislative framework, and that some of the other approaches provide more capacity building support through a broader range of capacity building mechanisms.

The Intervention Policy is Being Implemented as Planned, With Some Gaps in Monitoring and Support:
The Intervention Policy provides a common framework for the objectives, triggers, assessment, and levels of intervention, and INAC's regional offices are adhering to the required procedures. There are, however, variations within and between regions, primarily in terms of the capacity of recipients and INAC funding service officers. Key informants suggested that capacity be strengthened with training and other tools related to the Intervention Policy. Other suggested improvements are: increased monitoring by INAC of the performance of co-managers and third party managers; changes to the information collected in First Nations and Inuit Transfer Payment (FNITP); and more formalized processes for communication and sharing of information with other government departments.

Suggested Improvements and Alternatives

The main suggested improvements to the Intervention Policy are:

• Improved prevention and early detection linked to the general assessment of recipients that will be conducted annually;
  
  
• Revision to the trigger on debt to better reflect different aspects of the financial health of recipients;
  
   
• A broader range of tools and mechanisms for assessing and addressing the capacity building requirements of recipients, and additional dedicated funding for this purpose; and
  
  
• Simplifying the recipient-managed level of intervention; strengthening the co-managed level; and supplementing third party management with other capacity building initiatives, separately contracted and funded.

Other alternatives were suggested to deal with issues that are outside of the influence of the Intervention Policy but have an effect on its success. These issues include: increased leadership stability and community engagement, improved funding practices or a modified funding regime, and alternative debt management approaches.

 It is recommended that INAC:

1. Implement prevention and early detection strategies to prevent First Nations going into intervention status or escalating to a more serious level of intervention. Activities should include:
   
   
   1. Better identification of financial and governance capacity gaps and needs (linked to the general assessment);
      
      
   2. Better and broader identification of triggers for third party management and co-management;
      
      
   3. Identification of incentives for third party managers to build First Nations capacity, to be written into agreements with third party managers;
      
      
   4. Development of questions to assess properly key success factors to analyze trends in relation to escalation and de-escalation (consider undertaking community surveys to better assess community capacity factors for success); and
      
      
   5. Improve communication and coordination at the national and regional level with other federal government who deliver services and programs to First Nations.
      
      
2. It is recommended that:
   
   
   1. Third party managers be prequalified and assessed against performance criteria;
      
      
   2. INAC audit co-management and third party management arrangement on a risk basis; and
      
      
   3. Revise the third party management agreement to request participation of third party managers in evaluations as well as in audits.
      
      
3. The proposed new Default Prevention and Management Policy should address the design and delivery gaps of the current Intervention policy by:
   
   
   1. Implementing national tools and formalized processes in the assessment of First Nations;
      
      
   2. Clarifying FSO roles and responsibilities, developing job descriptions that identify competencies and knowledge needed, and identifying training to meet requirements; and
      
      
   3. Clearly communicate the new policy and assessment processes to stakeholders and First Nations.
      
      
4. Develop and implement better monitoring and reporting systems that involve:
   
   
   1. A performance measurement strategy to allow for meaningful reporting through quarterly progress reports and the Departmental Performance Report. Indicators such as duration, incidence and level of intervention, and level of implementation of Remedial Management Plan could be considered;
      
      
   2. A cost tracking system at the Headquarters and regional level to capture cost data in order to measure cost effectiveness and inform future decisions; and
      
      
   3. Re-design the intervention policy module in FNITP that will involve a revision of the inputs, processing and output (reports) of the system at the regional and national level in order to make it more user-friendly, reliable and to inform performance measurement strategy to be developed in line with the new policy.

1.1 Overview

The purpose of the evaluation is to assess the relevance, design, delivery, success and cost-effectiveness of Indian and Northern Affairs Canada's (INAC) Intervention Policy and to look at best practices, improvements and possible alternatives. The evaluation report provides information on the policy, the methodology that was used in the evaluation, the findings and conclusions for each of the evaluation questions, and overall conclusions and recommendations.

1.2 Policy Profile

1.2.1 Background

INAC's Intervention Policy is an internal financial management policy that has evolved over time to deal with various issues such as indebtedness, remedial management plans, and third party management. The current policy was developed partly in response to a 2003 audit by the Auditor General of Canada of the implementation of third party management; and partly, as a result of a comprehensive legal review to reduce Crown exposure to liabilities.

The policy was introduced in December 2006 and came into effect on April 1, 2007. It is issued under the authority of INAC's Chief Financial Officer. It consolidates all previously existing intervention practices under a single policy.

At the same time as the Intervention Policy was adopted, a Companion Initiative to develop the capacity of recipients under Intervention Policy was introduced. The objective of the Companion Initiative was "to support short term, high impact initiatives that will make a measurable difference."[Note 2] It was funded for 2006/07 and 2007/08 under the authority of the Indian and Inuit Management Development transfer payment authority after which time it was to be included in a renewed authority.[Note 3]

At the time of the evaluation, INAC was working on a new Intervention Policy (tentatively named the Default Prevention and Management Policy) as some elements were considered not addressed by the 2006 Intervention Policy. This new policy is seeking to be more aligned with INAC's new orientation and tools regarding capacity development with a focus on prevention and ongoing sustainability. To assist the Department in the development of this new policy, the Department seeking for the advice of experts in financial management, including advice from the First Nations Financial Management Board, the Aboriginal Financial Officers Association and the Assembly of First Nations through consultation and engagement session. Engagement sessions with stakeholders and regional staff will be held over the summer 2010 with the goal of having a new policy in place for the beginning of fiscal year 2011-2012. This evaluation is expected to inform the development of the new policy.

1.2.2 Policy Objectives and Expected Outcomes

The purpose of the Intervention Policy is to permit the delivery of programs and services and maintain accountability while problem situations are being addressed; and to put the onus on the recipient to correct the problem situations. The policy is also designed to support timely intervention and consistency in regional operations; to facilitate ongoing monitoring of intervention; and to improve the effectiveness of intervention. The aim of the policy is to encourage the recipient in default to enhance their capacity to provide programs and services, and to provide for an exit strategy where a lesser or no form of intervention is required.[Note 5]

1.2.3 Policy Framework

The Intervention Policy sets out a framework for intervention by the Minister in the event that a recipient defaults under the terms and conditions of a funding arrangement. It applies to all funding arrangements except legislated self-government agreements. It includes the triggers for intervention, the levels of intervention, the process, roles and responsibilities, performance review, accountability framework (for third party managers), and exit strategy.

There are four triggers for intervention for use with First Nations and tribal councils:

• the terms and conditions of the funding agreement are not met by the recipient;
  
  
• the recipient's auditor gives a denial of opinion or adverse opinion with respect to the recipient's financial statements;
  
  
• the recipient has incurred a cumulative operating deficit equivalent to eight percent or more of its total annual operating revenues; and
  
  
• the health, safety or welfare of First Nation members is being compromised.

The Intervention Policy does not apply to election disputes and allegations and complaints except where an election dispute or concern raised gives rise to a default. If the recipient is a corporation, bankruptcy, insolvency or liquidation can also lead to intervention.

According to the policy, in the event of a default, INAC notifies the recipient, undertakes an assessment of the recipient's capacity and willingness to address the default, and determines the level of intervention required. There are three main levels of intervention:

1. Recipient-managed level – a low level intervention when the recipient is willing and has the capacity to address the default based on a Remedial Management Plan (RMP).
   
   
2. Co-managed level – a moderate level intervention when the recipient is willing but lacks the capacity to address the default. The Council of the First Nation or Tribal Council is required to enter into a co-management agreement with a co-manager in addition to developing or amending and implementing a RMP.
   
   
3. Third party managed level – a high level intervention when INAC determines that there is a high risk to the funding provided under a funding arrangement or to the provision of programs and services, or that the council is unwilling to address the default. INAC then appoints a third party manager to administer its funding. Third party management is also used in a situation where no funding arrangement has been signed by the council.

INAC may also withhold funds, terminate the funding agreement, take any other reasonable action, or require any other reasonable action to be taken by the recipient.

According to the policy, RMPs are developed and implemented by the recipient. They are approved by the Council of the First Nation or tribal council (or the appropriate authority for other entities) and by the Regional Director of Funding Services, and are attached to the funding arrangement. RMPs should include the following elements:

• The purpose of the RMP;
  
  
• The effective date;
  
  
• A description of the causes that resulted in intervention;
  
  
• The corrective action to be undertaken within specific time frames to address the causes;
  
  
• Performance indicators that will be used to measure the effectiveness of the corrective action;
  
  
• The roles and responsibilities of the parties to the RMP;
  
  
• In cases where financial difficulties have resulted in intervention, a financial projection of the estimated consolidated revenue and expenditures and a debt management plan;
  
  
• Capacity building or training to be undertaken to strengthen the recipient's capacity to implement corrective action;
  
  
• A reporting and monitoring clause;
  
  
• A provision for amendments to the plan; and
  
  
• A provision for an intervention exit strategy.

At the co-management level, the council selects and contracts a co-manager. The role of a co-manager is to assist the council in remedying the default, developing sound management practices, restoring financial health, and building capacity. The co-manager is to be given financial signing authority for all accounts containing INAC funding. The council pays for the co-manager's services, usually from its Band Support Grant.

At the third party management level, INAC selects the third party managers from a pre-qualified list of individuals or firms and negotiates a third party agreement (see Section 7.1). All INAC funding, or funding for a particular program or service, is redirected to the third party manager to administer. INAC may also require that the third party manager assist the council in enhancing capacity to administer funding in order to remediate the default that gave rise to the intervention. The third party manager is required to maintain a system of accountability to the community in terms of the programs and services that are funded under the third party management agreement. The third party manager's services are normally paid from the INAC's Band Support Grant.

Councils are responsible for their credit transactions and any debt they may have incurred. The Crown is not obligated to pay or accept responsibility for any current or future debt. Employees remain the responsibility of the council but the third party manager determines which of the council's operational and administrative staff are necessary for the continuation of programs and services, and pays the salaries and benefits of those staff from INAC funding.

According to the policy, INAC monitors the progress of the intervention at least quarterly. Where possible, the council is included in the review of the performance of the third party manager. Based on the monitoring, INAC may decide to escalate or de-escalate the level of intervention or terminate it. In the event that co-management or third party management is being de-escalated or terminated, the council is required to develop an administrative transition plan for the transfer of full authority back to the council.

1.2.4 Policy Management

The Chief Financial Officer Sector manages the Intervention Policy and the regions implement the policy. According to the policy, INAC Headquarters is responsible for:

• ensuring consistency across the regions;
  
  
• providing direction and training on the application of the policy;
  
  
• strengthening management processes in support of the policy;
  
  
• ensuring that plans are monitored and performance reviews are undertaken;
  
  
• ensuring that the transfer of responsibility back to the funding recipient is done as soon as practical;
  
  
• providing for the periodic review of regional compliance and implementation of the policy; and
  
  
• periodically reviewing and revising the policy with input from recipients.

INAC regional offices are responsible for:

• identifying potential intervention situations;
  
  
• implementing the intervention process;
  
  
• approving the level of intervention;
  
  
• notifying other federal government departments if a third party manager is appointed;
  
  
• assessing and approving RMPs:
  
  
• assessing capacity deficiencies;
  
  
• appointing third party managers from the approved list;
  
  
• operating systems and procedures for implementation;
  
  
• maintaining proper documentation and files; and
  
  
• updating the intervention module in the First Nations and Inuit Transfer Payment System (FNITP).

1.2.5 Policy Resources

The Intervention Policy is issued under the authority of the Chief Financial Officer (CFO) and no specific funds are currently targeted to this policy. Funding in support of the development of the policy is included in the CFO's A-base budget. Funding for regional implementation of the policy is included in regional A-base budgets. All fees related to services provided by a co-manager or a third party manager are the responsibility of the council and are normally drawn from the Band Support Grant provided to the council by INAC.

The Companion Initiative on Capacity Development was funded in fiscal years 2006-2007 (starting January 2007) and 2007-2008 using Budget 2006 funds authorized by the Treasury Board. Treasury Board approved $10.6 million. Seventy-five percent of the funding was allocated to the regions on the basis of the average regional distribution of required interventions over the past three years, and the remaining 25 percent of funding was allocated on the basis of regional population. Other funds (about $500,000) were available for tribal councils and other institutions, such as regional Aboriginal Financial Official Officers Association (AFOA) Chapters to provide support.

2.1 Evaluation Objectives

In accordance with the Treasury Board Secretariat (TBS) Evaluation Policy, the evaluation examines issues related to relevance, success and performance. Design and delivery issues are also assessed to guide and improve the policy and performance measurement efforts. Best practices, improvements and alternatives are also explored. Terms of Reference were approved by INAC's Evaluation, Performance Measurement and Review Committee on September 24, 2009.

2.2 Evaluation Issues and Questions

The following broad evaluation issues guide the evaluation:

• Relevance

Evaluation of relevance looks at the extent to which the Intervention Policy addresses existing needs and priorities; is consistent with federal priorities, roles and responsibilities; and does not duplicate or overlap with the policies of other stakeholders.

• Success

Evaluation of success looks at whether the Intervention Policy objectives are being achieved and what factors have facilitated or hindered the achievement of the policy objectives. It also looks at the underlying causes and key success factors, as well as the unintended, positive or negative, impacts of the policy.

• Cost-Effectiveness

Measuring cost-effectiveness of the Intervention Policy involves identifying the cost of intervention in relation to the outcomes and comparing it to the cost of other intervention approaches (of other First Nation and tribal council funders in Canada, of provincial governments, and of other countries).

• Design

Evaluation of the design of the Intervention Policy looks at whether the policy is well designed to achieve its objectives and how it compares to the design of other intervention approaches.

• Delivery

Evaluation of the delivery of the Intervention Policy complements the work already done by the Audit of the Application of the Intervention Policy (2009) by looking at whether the policy is being implemented as planned, whether good guidance and monitoring is in place, what constraints impede implementation, and what are the best practices.

• Alternatives

Finally, the evaluation looks at the alternatives to the Intervention Policy to achieve the same results or to improve the results. In particular, the evaluation examines other models of intervention used by other funders of First Nations and tribal councils in Canada, by provincial governments, or by other countries.

The evaluation questions related to each of these issues can be found in Appendix B, Revised Evaluation Matrix.

2.3 Evaluation Scope and Timing

The evaluation examines Intervention Policy activities undertaken since its adoption in December 2006 and up to March 31, 2010. Because the Intervention Policy has never been evaluated, however, historical information over the past ten years or more have been looked, drawing from INAC's financial information systems, previous audit data and key informants. This helps us to place recent activities into an historical context and identify any overall trends.

INAC is in the process of reviewing and revising the 2006 Intervention Policy. While the evaluation team received documentation related to the revised Intervention Policy, it has not been included in the analysis because it has not been finalized. Our focus is therefore on the current Intervention Policy.

The relevance, design, delivery, success and cost-effectiveness of the Intervention Policy were assessed through qualitative and quantitative data; and triangulate the information across a number of sources. In order to assess what may have affected the achievement of the outcomes of the policy and make recommendations for changes or alternatives, possible underlying causes, preventive measures, and exit strategies were explored.

The evaluation takes into consideration previous audits, other evaluations, reports on the Intervention Policy and the Companion Initiative, and other relevant documents. It also takes into account other related INAC policies on community development, transfer payments, risk assessment, capacity building, and funding arrangements.

The perspectives of various INAC officials as well as the recipients in the evaluation were solicited. Information from the Chief Financial Officer Sector and the Regional Operations Sector, as well as from all INAC regions, except Nunavut, were collected.[Note 6] Information from a range of First Nations, tribal councils and other Aboriginal organizations was also collected. Finally, the intervention approaches of other key federal departments, other jurisdictions, and other countries, as well as the perspectives of subject matter experts who have a knowledge and interest in the evaluation issues were examined.

The evaluation was conducted from January to June 2010. It was preceded by a pre-consultation phase conducted by the Evaluation, Performance Measurement and Review Branch (EPMRB), which led to the development of a draft Evaluation Methodology Report and draft Logic Model. From January to March 2010, the Institute On Governance (IOG) finalized the Evaluation Methodology Report and developed the data collection instruments; drafted the Document Review; and conducted the Literature Review. From April to June 2010, IOG made site visits with the assistance of EPMRB to five of the regions (Atlantic, Quebec, Ontario, Manitoba and Saskatchewan) and conducted the case studies and interviews. EPMRB also compiled and analyzed data during this period. Two subject expert panel meetings were held by IOG with EPMRB participation – one at the end of the planning phase and one at the end of the data collection phase. Preliminary findings and conclusions were presented by IOG to INAC senior management at the end of May 2010. EPMRB developed the recommendation based on the report submitted.

2.4 Evaluation Methodology

2.4.1 Development of a Framework Logic Model

EPMRB developed a draft Framework Logic Model for the Intervention Policy based on the purpose, aims and objectives of the policy; pre-consultations with key informants in INAC; and INAC's Strategic Outcomes (Appendix C, Draft Intervention Policy Framework Logic Model). The draft Logic Model was used by EPMRB to help clarify the evaluation issues and the short, medium and long-term outcomes.

According to the draft Logic Model, INAC carries out a number of activities in order to ensure that services are delivered and accountability is maintained, remedial measures are implemented, and recipient capacity is strengthened. In the long term, this should lead to improved performance and ultimately the de-escalation of intervention and earlier exit from intervention; a reduction in the number of First Nations that have seen under intervention and the duration of such intervention; and effective and responsibly managed and operated programs and services funded by INAC. The evaluation questions and the evaluation methodology were designed to measure the achievement of these outcomes.

2.4.2 Data Sources

The evaluation methodology incorporates six lines of evidence: a literature review, a document and file review, data analysis, key informant interviews, case studies, and a subject expert panel. The subsequent sections provide a summary of each of these lines of evidence. The Revised Evaluation Matrix (Appendix B) indicates which line of evidence was used to answer each of the evaluation questions.

1. Literature Review

The purpose of the literature review was to define and explore the key issues related to intervention by one level of government of another level of government, or by a funder to a recipient organization. The literature review has informed the findings and conclusions in this evaluation report, particularly related to design, delivery and effectiveness, as well as potential alternatives.

The literature review includes a brief survey of public policy and administration and social science, academic journal databases using key words like intervention, default, debt management, remedial management, third party management, supervision, trusteeship ("mettre en tutelle" in French) and remediation. As a result of the survey, three key lines of inquiry emerged as being most relevant to INAC's Intervention Policy – systemic intervention, intervention in distressed states and communities, and intervention in subnational entities.

Eight Canadian and international intervention examples were initially selected for further investigation, and another example of an alternative approach was identified during the data collection phase. The examples were used to illustrate the various approaches that have been taken to intervention and include:

IOG gathered information on the other intervention approaches from government publications and other reports, and interviewed key informants for six of the examples.

The literature review concludes with a summary of the issues, lessons learned and potential alternatives.

2. Document and File Review

EPMRB identified relevant files and documents during the preliminary consultation phase of the evaluation and IOG supplemented them during the data collection phase. These documents included audits, evaluations, and policy documents on the Intervention Policy and related policies. IOG also conducted a Google search of media reporting on INAC's intervention over the past five years.

EPMRB asked INAC's regional offices to provide samples of RMPs for five recipients under remedial management and five recipients under co-management or third party management and were reviewed by IOG. All regions except the Northwest Territories (NWT) region provided a sample that covered a total of 63 First Nations, two Indian-administered organizations, two co-managers, and two third party managers. A summary of the sample of RMPs is provided in Appendix D.

The document and file review provided a background briefing for the evaluation team prior to the data collection phase and informed the findings for a number of the evaluation issues and questions.

3. Data Analysis

In collaboration with EPMRB, IOG conducted an analysis of INAC's performance and financial data related to the Intervention Policy from 1999/2000 to 2009/2010. EPMRB compiled a database for analysis, drawing from existing INAC management systems, financial systems and other relevant databases. EPMRB also cleaned the data and validated the data on intervention over the past 11 years with INAC's regional offices. PASW Statistics 18[Note 7] software was used by EPMRB for the analysis and EPMRB worked with INAC's Research and Analysis Directorate to integrate and analyze data on the Community Well-being (CWB) Index.

The databases and sources of information used in the analysis were:

The purpose of the data analysis was to provide a description of the application of the intervention policy and to assess quantitative information on its underlying causes or impacts. This includes indicators and trends at a national and regional level related to: the incidence of intervention, the level of intervention, the duration, and the triggers. It also includes variables that may be related to underlying causes including: size, population, remoteness and the CWB Index. Where possible, EPMRB correlated the data on intervention with the data on underlying causes and interpreted the results.

Because the total number of Indian-administered organizations under intervention in any year is very small in comparison to the total number of First Nations under intervention, the data analysis for these organizations was limited to the incidence and level of intervention.

Appendix E – Summary of Data Analysis provides a summary of what was originally planned and what was actually achieved in terms of the data analysis. Appendix F provides the type of analysis that was conducted. We have incorporated relevant results into this Evaluation Report.

4. Key Informant Interviews

IOG conducted a number of interviews with INAC regional office and Headquarter managers, other key federal departments, and key informants from other organizations. We also interviewed a select number of First Nations and other Indian-administered organizations, third party managers, co-managers and funding service officers as part of the case studies. A total of 122 interviews with individuals or groups of individuals were planned, and a total of 82 interviews or 67 percent were actually conducted (Appendix G - Summary of Key Interviews). The interviews were conducted from April to June 2010.

The interviews were based on interview guides linked to the evaluation issues and questions. They were conducted in person, by phone or by teleconference. The results were written up and provided in the Key Informant Interviews Technical Report and incorporated into this evaluation report.

5. Case Studies

IOG conducted a total of 27 case studies covering all regions except the North. The cases include 24 First Nations and three other Indian-administered organizations. IOG and EPMRB selected the case studies to be broadly representative of the statistics on intervention over the past ten years, as well as other variables such as remoteness, size and community well-being. We consulted with INAC's regional offices about the cases selected and made a few changes due to limitations in accessing recipients or to INAC's funding.

The First Nation case studies cover:

The other Indian administered organizations include a tribal council, a child and family services agency, and an education authority. A breakdown of the cases by region is provided in Appendix H - Summary of Case Study Sample.

The case studies addressed a number of the evaluation issues and questions, particularly underlying causes, delivery, and success. In particular, the evaluation team used the First Nation success and other success cases to analyze key success factors; the First Nation chronic recipient-managed and chronic co-managed and third party managed cases to analyze constraints to implementation; and the other Indian-administered organizations cases to analyze how the situation in a First Nation affects its service delivery organization or vice versa.

The case studies were conducted through a documentation review; a site visit where time and resources permitted; and interviews with the recipient, the relevant INAC funding service officer, and co-managers or third party managers, if applicable and where possible. EPMRB sent out an email or letter to the Chief and council, band manager, and co-manager or third party manager for each of the First Nations, and to the Executive Director for each of the other Indian-administered organizations, inviting them to participate in the evaluation. The IOG followed up through email and telephone with all of the contact persons in order to set up a site visit or a telephone interview. The contact person identified other relevant people to include in the interviews. As indicated in Section 2.4.3 Limitations, the evaluation team was only able to conduct interviews with about half of the recipients, three co-managers and one third party manager despite repeated follow-up. Appendix H indicates which cases included interviews with recipients.

6. Subject Expert Panel

IOG convened a subject expert panel of four members and facilitated two half-day workshops. Panel members were individuals with senior experience in INAC and Health Canada, in financial management, governance, and capacity building. IOG also invited members of the advisory committee to attend the workshops.

IOG held one workshop at the outset of the evaluation to review the key lines of inquiry and possible alternatives; and one workshop at the end of the data collection phase to review preliminary findings and conclusions. EPMRB participated to both workshops.

2.4.3 Limitations and Constraints

During the course of the evaluation, the evaluation team faced a number of constraints that limit the reliability of the data and the interpretation of the results. These limitations are presented below in terms of what the issue was, what efforts were taken to mitigate the impact, and how the issue ultimately affected the evaluation. Overall, the evaluation team attempted to address the limitations by using information from all of the data sources; complementing quantitative data with qualitative data; and incorporating different perspectives from the range of key informants and the subject expert panel. We have also qualified the findings where there are issues of data reliability, potential bias, and attribution of results to the Intervention Policy.

Limitation #1: No validated logic model for the Intervention Policy

As indicated previously, there was no logic model developed for the Intervention Policy at the time that it was adopted that outlined the link between the activities, outputs and outcomes.

Mitigation:

EPMRB developed a draft framework logic model based on components of the policy, pre-consultations with key INAC informants, and INAC's Program Activity Architecture. The draft logic model was used to detail the evaluation issues, to guide the data collected, and to assess the success and cost-effectiveness of the Intervention Policy. This logic model has, however, not been finalized.

Impact on Evaluation:

The emphasis in the evaluation and the interpretation of the results could change if there are changes in the logic model, particularly in terms of the long-term outcomes.

Limitation #2: Availability and reliability of data and data analysis

• Some bands had incomplete information due to the fact that data came from a multitude of sources. This affected the analysis because the total population differed from one analysis to another. Data on the CWB and financial information are the variables with the most missing data. Between 25 and 28 percent of the data was missing in the case of CWB because Statistics Canada did not collect census data in all First Nation communities. Between five and seven percent of the data in the case of financial information was missing because bands were under a self-governance agreement (34 bands), bands did not receive funding for essential programs and services directly from INAC but rather through their tribal councils (30 bands), bands did not have a funding arrangement with INAC (three bands), bands had split into more than one band, or the band received less than $30K.
  
  
• INAC changed its financial information system from TPMs to FNITP in 2006/07 and the configuration of FNITP differs from TPMs. Financial information, therefore, had to be extracted manually for each band and re-entered onto the database for analysis. Only data for fiscal year (FY) 2004/05 to 2008/09 was therefore extracted.
  
  
• A comprehensive picture of the total duration of intervention could not be compiled because of the time required to validate data and changes in the financial system. The analysis of duration was therefore limited to the timeframe from 1999/2000 to 2009/2010.
  
   
• There was no statistical competency to derive a composite intervention index that incorporated both the level of intervention and the duration of intervention. Therefore, the correlations between this composite index and other variables were not analyzed. Given the limitations of the underlying data, however, it is questionable whether this composite index would have contributed any additional insights.
  
  
• No quantitative data was available on the costs of intervention for INAC. No comprehensive and comparable data was available on the costs of intervention for recipients, except for the cost of TPMs.
  
   
• No comprehensive and comparable performance data is available on the quality of services provided by recipients, either under intervention or not under intervention.
  
  
• Health Canada does not collect data on recipient-managed intervention, and does not compile data at the national level on co-managed intervention.

Mitigation:

• Intervention data was validated with INAC's regional offices to increase its reliability; the coding of band numbers in different databases was cleaned based on the First Nation Profile Website; and some bands were removed to address the limitations in the financial data noted above. Where data was determined to be incomplete or unreliable – e.g. dependence on INAC funding – it was not used in the data analysis.
  
  
• The scope of the data analysis was reduced in some instances – i.e. financial information from 2004/05 to 2008/09 rather than 1999/2000 to 2009/2010; duration over an 11-year timeframe rather than the entire history of intervention; limited comparison with Health Canada intervention data.
  
  
• Different statistical tools were used to conduct the analysis depending on the nature of the underlying data – i.e. averages, frequencies, correlations, Anova and chi squares.
  
  
• Caution was exercised in the interpretation of the results where a significant number of cases were missing – i.e. for the CWB, or for the cumulative deficit.
  
  
• Qualitative information was used from the interviews to supplement the quantitative data that was missing – i.e. on the costs of intervention and the impact on the quality of services.

Impact on Evaluation:

While the evaluation includes an accurate profile of intervention and a statistical analysis of some of the contextual factors that may affect the level or duration of intervention, other statistical analyses of underlying causes were not possible and have not been included. The analysis of cost-effectiveness is also limited by the lack of comprehensive quantitative information on costs.

Limitation #3: Potential Bias in Case Studies

In terms of the key informant interviews and case studies, about 60 percent of the recipients were interviewed despite repeated follow up by telephone and email with all of the contact persons. In terms of success cases – First Nations that have never been under any form of intervention – and in terms of co-managers and third party managers, it has been the least successful (33 percent of the sample). Interviews with First Nations that have been under co-management and/or third party management for more than five years were the most successful (77 percent of the sample). Three Indian-administered organizations were also interviewed.

In some cases, interviews were scheduled but the interviewees were no longer available at the designated time. There was not sufficient time to select alternate cases to replace those cases where recipient interviews could not be conducted. Funding Services Officers were, however, interviewed for all of the cases. There is, therefore, a potential for bias in the case studies towards INAC's perspective rather than recipients' perspective.

Mitigation:

The recipient interviews have been supplemented by the perspectives of national Aboriginal organizations, by the document review, and by the literature review.

Impact on Evaluation:

The evaluation may not reflect the full range of perspectives among recipients and among co-managers and third party managers and may be biased towards INAC's perspective. Where recipient views are referenced, they may emphasize views about co-management and third party management more than views about recipient-managed intervention.

Limitation #4: Availability of historical information

Another limitation in the case studies was the availability of historical information. Since many of the cases covered a decade or more of intervention, there was very little documentation from the earlier stages of implementation. In addition, remedial management plans for the cases were often not up to date (refer to Section 7, Evaluation Findings - Delivery). In a few cases, the recipient contact persons and the Funding Service Officer were relatively new and did not have a historical perspective.

 Mitigation:

The case studies were written up on the basis of the information available or received. For a few of the interviews, previous chiefs or councillors or funding service officers who would have a historical perspective were tentatively identified.

Impact on Evaluation:

The case studies vary in terms of the depth of the historical analysis.

Limitation #5: Availability of documentation

The only constraint in terms of the document review was that the NWT did not provide any remedial management plans for the RMP analysis.

 Mitigation:

An interview was conducted with regional officials in the NWT to obtain information on the implementation of the intervention policy in that region.

 Impact on Evaluation:

The NWT region is unique within INAC because only three of the 26 First Nation bands in the territory have a land base; essential services are funded by INAC through the territorial government and not directly to bands; and the regional office does not report to Regional Operations Sector at INAC HQ but rather the Northern Affairs Sector. The lack of RMPs for the NWT is therefore not seen as a major constraint in terms of the evaluation findings on RMPs.

Limitation #5: Analysis of impact and attribution to the Intervention Policy

As indicated in the findings on Success (Section 4), any positive changes in the incidence, level and duration of intervention and the cumulative deficit of First Nations cannot necessarily be attributed to the Intervention Policy. Evidence from interviews, case studies and the data analysis would indicate that other factors, such as access to own source revenue, economic risks in particular economic sectors, the degree of geographic isolation, very small size, lack of a land base, or the unwillingness of creditors to lend money can also have an impact on the incidence, level or duration of intervention. Variations across the regions in terms of these other factors also mean that it is not possible to attribute regional variations in results to differences in the ways that the regional offices are implementing the Intervention Policy.

Mitigation:

Limitations on attribution were noted in the findings and analysis, as well as the underlying causes and contextual factors that could affect the success of the policy.

 Impact on Evaluation:

The evaluation is not able to attribute success in the achievement of long-term outcomes or variations across regions to the implementation of the Intervention Policy. This is due to a number of other factors or intervening variables. The evaluation does, however, include suggestions for alternatives to the Intervention Policy that could address these other underlying factors.

2.5 Evaluation Governance, Management and Quality Assurance

EPMRB of INAC's Audit and Evaluation Sector directed and managed the evaluation in line with its Engagement Policy and Quality Assurance Strategy. Unless otherwise noted, the Institute On Governance carried out the evaluative work.

EPMRB set up an external Advisory Committee to provide it with ongoing advice on the evaluation. The Advisory Committee included representatives from the Department of Finance and the Treasury Board Secretariat and one First Nation organization, and members were invited to participate in the two workshops of the subject expert panel. EPMRB also set up an internal working group that included representatives of Transfer Payment and Financial Policy Directorate (part of the Chief Financial Office), Regional Operations Sector, and INAC regional offices.

The evaluation methodology report and the draft evaluation report were peer reviewed for quality assurance and compliance with relevant evaluation policies of Treasury Board, INAC and EPMRB.

2.6 Presentation of Findings and Analysis

The next section provides a summary of the findings and conclusions in relation to the evaluation questions, drawing from the different lines of evidence. For the qualitative data, certain terms were used to indicate the proportion of respondents, regions, or case studies to which the finding refers; or other terms to indicate the frequency. These terms are roughly equivalent to the following percentages:

Findings may not apply to certain respondents, regions, or case studies because no response was provided or no response could be inferred from other comments; or because a different response was provided. Where possible, alternative views, as well as the views of the majority, are presented.

3.1. Continuing needs and priorities

Is the Intervention Policy responding to actual needs?

According to almost all INAC officials, the Intervention Policy meets the needs of First Nation communities and other recipients by assisting them to manage within the resources available, to ensure the ongoing delivery of essential services, and to be accountable to INAC. Many of the recipients agreed that the Intervention Policy was necessary, especially at the recipient-managed and co-managed levels. Almost all of the recipients under third party management, however, did not agree that the Intervention Policy met their needs.

An analysis of the triggers for the First Nations under some form of intervention as of February 4, 2010, indicates that the cumulative operating deficit is overwhelmingly cited as a reason for intervention (i.e. in 82.12 percent of all First Nations). This is particularly true for co-managed intervention (88 percent of the First Nations) and recipient-managed intervention (85 percent of the First Nations). For third party management, the reasons are more varied, but the cumulative operating deficit as well as the risk of insolvency are still important considerations (72.22 percent of the First Nations).

The case studies indicate that operating deficits and debt can affect the delivery of essential services to First Nations by:

• Increasing cash flow problems leading to delayed social assistance payments, delayed salary or benefit payments, delayed payments to provincial school for students, and to other suppliers, halts on construction projects, etc.;
  
  
• Increasing the cost of doing business and delivering services by increasing the cost of capital;
  
   
• Reducing the flexibility to enhance services, to maintain capital infrastructure, to build capacity, or to promote economic development;
  
  
• Restricting access to new funds from INAC or creditors for improved housing, infrastructure, economic development, etc.; and
  
  
• In extreme cases, risking the seizure of all program funds by creditors, or increase creditor control.

The next most prevalent reason for intervention is a denial or adverse audit opinion (10.6 percent of all First Nations). A denial, adverse or qualified audit opinion may be an indication that financial management and accounting procedures are deficient or that the First Nation has refused to provide information for a consolidated audit. An accurate picture of the financial health of the recipient is therefore not available. The recipient then runs the risk of increasing deficits beyond their carrying capacity, of not being able to recover certain expenditures because they have not been properly accounted for, or of losing the confidence of funders and creditors. Unqualified audits, on the other hand, provide accurate reporting to the council or other governing body, the members, funders, creditors and other stakeholders.

In terms of third party management, some of the other reasons recorded on FNITP for intervention are:

• A serious breach of the terms and conditions of the funding arrangement (27.78 percent of First Nations under third party management);
  
  
• The health, safety or welfare of First Nation members is being compromised (22.22 percent of First Nations under TPM); or
  
  
• Other reasons (44.44 percent of First Nations under TPM).

Our interviews with INAC regional officials and the case studies indicate that serious breaches or other reasons could include the refusal to sign a funding agreement or to provide a consolidated audit. A less common reason is a serious election dispute and no legitimate leadership in place. Even if no funding agreement can be concluded, essential services still need to be delivered to First Nation members.

Many of the recipients and other key informants noted that First Nations also needed access to other sources of funding, increased funding from INAC, or increased flexibility in order to address their needs and build capacity.

3.2 Consistency with federal government and INAC priorities and policies

To what extent does the Intervention Policy align with a) federal government's priorities; and b) with the departmental strategic outcomes?

Over the past 30 years, program delivery has been transformed from direct delivery by INAC to delivery by First Nations through transfer payments. The Intervention Policy is consistent with federal government priorities and departmental strategic outcomes by supporting adequate oversight and stewardship of these resources; ensuring that First Nations receive and provide essential services; and ensuring that health and safety are protected; while also helping to build the administrative capacity of First Nation governments and organizations.

The Intervention Policy is most closely aligned with the Governance and Institutions of Government Program Activity of INAC's Program Activity Architecture. This Program Activity seeks to foster "stronger governance and institutions of government through supporting legislative initiatives, programs and policies, and administrative mechanisms that foster stable, legitimate and effective First Nations and Inuit governments that are culturally relevant, provide efficient delivery of services and are accountable to their citizens."[Note 8]

3.3 Duplication or overlap

Does the program duplicate or overlap with other programs, policies, or initiatives delivered by other stakeholders? (including other federal departments)

Research demonstrated that the INAC's Intervention Policy is one of its kinds and there were no duplication with other initiatives delivered by other stakeholders. There is little guidance on intervention in the Treasury Board Secretariat's Policy on Transfer Payments as the purpose was to provide flexibility to departments. According the Policy, in the event of a default under a funding agreement, the repayment of the grant or contribution may be required,[Note 9] the funding agreement may be terminated, or "other remedies and procedures" (unspecified) may be provided for.[Note 10]

Health Canada has used INAC's Intervention Policy as the basis for the design of its own intervention policy; and participates with INAC in the development of the third party management framework and the pre-qualification of third party managers. INAC and Health Canada HQ officials indicated that they let each other know if TPM is required and INAC advises Health Canada if a Canada-First Nation Funding Arrangement is going to be terminated due to intervention. Health Canada looks to INAC to support the development of general management and administrative capacity within First Nations and tribal councils and focuses on building the capacity of health related entities.

The First Nations Financial Management Board (FNFMB) has also used INAC's Intervention Policy as the basis for its own intervention policy. FNFMB is working closely with INAC on revisions to the policy and exploring how information can be better shared between the two for mutual benefit.

3.4 Federal government role and responsibility

Does the Intervention Policy align with the federal government role and responsibility?

The Intervention Policy is an operational policy that supports the federal government's roles and responsibilities as it helps to maintain the continuity of essential services to First Nations. However, the federal government is not responsible for First Nations' debt,[Note 11] but debt can have an impact on the delivery of essential services.

3.5 Conclusions related to relevance

• There is a continued need to protect funding for programs and services, to support the delivery of essential services, and to ensure accountability for how federal monies are spent. There is also a continued need to build the capacity of recipients to manage and administer funds.
  
  
• The Intervention Policy is consistent with federal government and INAC plans and priorities and the federal government's roles and responsibilities.
  
  
• INAC's Intervention Policy has been used as the basis for the intervention policies of Health Canada and FNFMB; and INAC, Health Canada and FNFMB are working together to coordinate changes to the policy.

4.1 Intervention trends

What are the intervention trends?

Indicators

A review of documentation demonstrated that no performance measurement has been developed for the Intervention Policy. But since the beginning of the implementation of this policy, internal services measured some indicators, such as the number of communities under intervention, their level of intervention, the deficit ratio and the reasons for being under intervention. However, in the recent Report on Plan and Priorities information on the percentage of First Nations communities under financial intervention has been added[Note 12]. This information depends on different factors, such as the consistency of the end-user to input the information in the information system some reliability issued rose during the collection of the data. The information below on the incidence and level of intervention, as well as the duration were collected through regional offices rather than the FNITP system to ensure the reliability of the information. In this case, there may be inconsistencies with the information used by the Transfer Payment Directorate (i.e. departmental performance report) as the process to collect the information differs. Indicators based on a review of the documentation on Intervention Policy since the `90 as well as based on the observations of the evaluators during this process are listed in Appendix K.

Incidence and Level of Intervention

One measure of the success of the Intervention Policy over the long-term is the incidence and level of intervention – i.e. the more successful the policy, the lower the number of recipients under any form of intervention (incidence) and the lower the level of intervention. As Chart 1 below and Table 1 (Appendix I – Data Tables) indicate, the total number of First Nations under some form of intervention actually increased for the last 11 years (1999/2000 to 2009/2010). However, the total number has however declined from its peak in 2003/04 and has been on a downward trend since 2007/08 when the Intervention Policy was introduced. As of March 31, 2010, there were 168 First Nations under some form of intervention, or 29 percent of the total number of First Nations to which the Intervention Policy applies.[Note 13]

Source: FNITP and Regional Offices

Chart 1: This figure shows a time series chart and legend, which represent the evolution of the total number of First Nations under intervention and by level of intervention, since fiscal year 1999/2000 up to 2009/2010.

Fiscal years (1999/2000 up to 2009/2010) are segmented along the horizontal x axis and the numbers of First Nations under intervention are indicated along the vertical y axis.

Four time series are represented in the chart, each represented by a color: Remedial Management Plan (blue), Co- Management (red), Third Party Management (green) and total of the three levels (purple).

Remedial management plan – In this time series, the number of First Nations under Remedial Management Plan fluctuated in a range where the minimum is 81 First Nations in fiscal year 2009/2010 and the maximum is 103 First Nations in 2001/2002

Co-Management – In this time series, the number of First Nations under Co-Management fluctuated in a range where the minimum is 36 First Nations in fiscal year 1999/2000 and the maximum is 66 First Nations in 2007/2008

Third Party Management – In this time series, the number of First Nations under Third Party Management fluctuated in a range where the minimum is 15 First Nations in fiscal year 1999/2000 and the maximum is 37 First Nations in 2003/2004.

Table 1 in Appendix I presents the detailed numbers for each level of intervention and total for every fiscal year.

From 1999/2000 to 2009/2010, the number of First Nations under recipient-managed intervention (RMP) declined, the number of First Nations under co-management (CM) substantially increased, and the number of First Nations under third party management also increased. From 2007/08 to 2009/2010, however, the number of First Nations under co-managed or third party-managed intervention has remained fairly stable. As of March 31, 2010, 81 First Nations were under recipient-managed intervention; 64 First Nations were under co-managed intervention; and 23 First Nations were under third party managed intervention.

Chart 2 and Table 2 (Appendix I) provides information on the level of intervention in terms of the proportional share of each level of intervention. From 1999/2000 to 2009/2010, the proportion of recipient-managed interventions has declined from 64 percent to 48 percent while the proportion of co-managed interventions has increased from 25 percent to 38 percent. The proportion of third party-managed interventions has fluctuated over the same period. From 2007/08 to 2009/2010, however, the proportion of each level of intervention has been relatively stable. As of March 31, 2010, 48 percent of the First Nations under some form of intervention were under recipient-managed intervention; 38 percent were under co-managed intervention; and 14 percent were under third party-managed intervention.

Source: FNITP and Regional Offices

Chart 2: This figure shows a chart and legend, which represent the level of intervention as a proportion of the total number of First Nations under intervention since fiscal year 1999/2000 up to 2009/2010.

Fiscal years (1999/2000 up to 2009/2010) are segmented along the horizontal x axis and the percentage of First Nations under intervention at a particular level are indicated along the vertical y axis.

Three levels are represented in the chart, each represented by a color: Remedial Management Plan (blue), Co- Management (red), and Third Party Management (green).

Remedial management plan – In this time series, the proportion of First Nations under Remedial Management Plan fluctuated in a range where the minimum is 48% of First Nations in fiscal year 2006/2007, 2008/2009, and 2009/2010 and the maximum is 64% of First Nations in 1999/2000.

Co-Management – In this time series, the proportion of First Nations under Co-Management fluctuated in a range where the minimum is 25% of First Nations in fiscal year 1999/2000 and the maximum is 38% of First Nations in 2009/2010.

Third Party Management – In this time series, the proportion of First Nations under Third Party Management fluctuated in a range where the minimum is 10% of First Nations in fiscal year 1999/2000 and the maximum is 20% of First Nations in 2003/2004.

Table 2 in Appendix I presents the detailed proportions of First Nations for each level of intervention and total for every fiscal year.

Ontario, Manitoba and Saskatchewan regions have the highest number of First Nations under some form of intervention; followed by the Atlantic, British Columbia and Quebec regions (Table 3, Appendix I). When the number of First Nations in each region is taken into account, Manitoba, NWT, Saskatchewan and Atlantic regions have the highest proportion of First Nations under some form of intervention; followed by Quebec, Ontario and Alberta regions. British Columbia and Yukon regions have a low proportion of First Nations under intervention, but 11 Yukon First Nations are self-governing and the Intervention Policy does not apply to them.

INAC regional officials noted some key variations across the regions that could lead to a higher incidence and level of intervention:

• economic risks in particular sectors, such as fisheries, forestry, or oil and gas;
  
  
• the number of communities without a land base (NWT) and, therefore, fewer opportunities for economic development;
  
  
• the number of fly-in communities (Manitoba, Ontario North, Yukon and NWT) or very small communities (British Columbia) and their difficulty accessing capacity; and
  
  
• limited availability of trust funds or claims settlements that could be used to pay off debt.

Some regions also noted that they had an active early warning system that they thought kept the incidence and duration lower. Officials in one region noted that they may have moved First Nations into multi-year flexible funding agreements too soon, leading to a higher incidence of intervention later on.

As indicated in Section 2.4.3, Limitations and Constraints, any changes in the incidence and level of intervention cannot necessarily be attributed to the Intervention Policy – for example, evidence from interviews and case studies would indicate that increased access to own source revenue also has a positive influence.

The incidence of intervention in other Indian-administered organizations increased after the introduction of the 2006 Intervention Policy, but declined in the following two years (Chart 3 below and Table 4, Appendix I). All of these organizations were under recipient-managed intervention except three cases in the past where co-management or third party management was used. As of March 31, 2010, there were seven organizations under recipient-managed intervention.

Source: FNITP and Regional Offices

Chart 3: This figure shows a time series graph indicating the total of organizations under intervention policy over the fiscal years 1999/2000 up to 2009/2010. The total numbers are indicated along the y axis and the fiscal years are indicated along the x axis. In this time series, the number of organizations under a level of intervention fluctuated in a range where the minimum is 3 organizations in fiscal year 1999/2000 and the maximum is 10 First Nations in 2007/2008. Table 4 in Appendix I presents the total number of organizations under intervention for every fiscal year since 1999/2000.

In comparison to INAC, Health Canada has a lower incidence of intervention, although they do not track recipient-managed interventions and information on co-managed intervention is not compiled at the national level. For example, there are currently only nine First Nations under third party management by Health Canada compared to 23 First Nations under third party management by INAC. Health Canada regional officials also indicated that their co-management and third party management interventions are of shorter duration.

Health Canada and INAC officials' explanation for the differences between the two departments were: Health Canada funding may be provided to a separate health authority and not the band council; a separate health authority can be set up if the band council is under third party management by INAC; there has been less flexibility in Health Canada funding; and capital funding is managed separately from program funding. Health Canada also revised its approach to its funding arrangements in 2008 to invest more time and effort up front in risk assessment and capacity building, and less time during implementation on compliance monitoring and intervention.[Note 14] According to Health Canada officials, however, it is too early to tell whether this change has had the desired impact on the incidence and level of intervention.

Duration of Intervention

One objective of the policy is to encourage the recipient in default to enhance their capacity to effectively provide programs and services so that intervention measures are no longer required. As mentioned in the 2003 Report of the Office of the Auditor General of Canada[Note 15] "intervention is intended to be temporary" and the 2005 Office of the Auditor General Report[Note 16] added that there was a lack of strategy for building the capacity of First Nations management to end third party management. A measure of success is the duration of intervention. In the past eleven years (1999/2000 to 2009/2010), a total of 349 First Nations have been under intervention at some time or another. About half of those First Nations are no longer under intervention. There are, however, First Nations that have been under some form of intervention for a long time. For example, of the 168 First Nations under some form of intervention as of March 31, 2010, 42 percent had been under intervention for 10 or more of the past eleven years; 29 percent had been under intervention for six to nine years; and 29 percent had been under intervention for one to five years.

Table 5 - Duration of Intervention as of March 31, 2010

Source: FNITP and Regional Offices

Saskatchewan, Manitoba and the Atlantic regions account for most of the First Nations that have been under some form of intervention for a long time.

Table 6 - Long-Term Duration by Region as of March 31, 2010

Source: FNITP and Regional Offices

Cumulative Deficit

Since a key trigger for intervention is the cumulative operating deficit, another measure of success is the trend in the cumulative deficit – i.e. the more successful is the implementation of RMP under Intervention Policy over the long-term, the fewer the number of First Nations with a cumulative operating deficit greater than eight percent of total operating revenues. In fact, the total number of such First Nations has declined by 32 percent from 2004/05 to 2009/10 (see note). British Columbia and Saskatchewan regions have seen the most substantial improvements – i.e. a reduction of over 50 percent in the number of First Nations with a cumulative operating deficit over eight percent. Quebec, Manitoba and Ontario have seen moderate improvements – i.e. a reduction of over 35 percent. The number in the Atlantic region has marginally increased from nine to ten First Nations.

Table 7 - Number of First Nations with a Cumulative Operating Deficit >8%

Source: FNITP financial data

Note: This report reflects information entered into the FNITP Statement of Revenue and Expenditure where the Audit Reporting Requirement "Approved" has been completed. First Nations with qualified audits or a denial of audit opinion are therefore excluded.

Based on this sample, as it would be expected with a decline in the number of First Nations, the total size of the cumulative deficit of those First Nations has also declined (Table 7, Appendix I). The average size of the deficit for each First Nation has however increased. Manitoba First Nations account for about 30 percent of the total cumulative deficit of First Nations with a cumulative operating deficit greater than eight percent.

The cumulative operating deficit is only one of a number of indicators that can be used to measure the indebtedness or financial health of a First Nation. It is not a totally accurate reflection of the extent of First Nation debt because it does not cover those First Nations with qualified audits or a denial of audit opinion. It will also no longer be applicable with the adoption of Generally Accepted Accounting Principles for First Nations in 2010/2011.

Changes in the cumulative deficit are also not necessarily attributable to the Intervention Policy – for example, an increase in own source revenue or a decrease in the willingness of creditors to lend money could also reduce the cumulative deficit.

In counterpart, a decline in certain key economic sectors (fisheries, forestry or oil and gas) and funding arrangements could be factors that increase the cumulative deficit. Funds are transferred to First Nations for programs and services on reserve through different types of arrangements. Since the implementation of the evaluation policy of 2006, when a third party manager is appointed to a First Nation community, the community should fall under the Comprehensive Funding Arrangement (CFA) which inhibit the third party manager ability to use surpluses to pay off debt as according a clause of this arrangement, any surplus need to be sent back to INAC. Furthermore, some respondents believe that the implementation of the revised Policy on Transfer Payments could bring the same impact certain First Nations that are assessed to be medium to high risk.

4.2 Underlying causes

What are the underlying causes of the intervention trends?

Based on the review of the sample RMPs, case studies, and interviews, the following list of the immediate causes that can lead to financial difficulties have been compiled (refer to Appendix J – Causal Factors Framework)[Note 18]:

• Economic development (e.g. failed businesses);
  
  
• Housing (overruns and new pressures from new home construction; no or poor rent collection; construction delays, mould in housing);
  
  
• Other capital projects (unfunded, cost overruns, delays);
  
  
• Unfunded or non-recoverable expenditures on social assistance or other programs;
  
  
• Overruns in elementary and secondary education; decline in enrolment;
  
  
• Uncollected loans or advances to band members & employees;
  
  
• Overruns on honoraria and travel costs of chief and council;
  
  
• Unpaid HST/GST and tax remittances;
  
  
• Travel costs for relatives of band members to attend funerals on reserve;
  
  
• Financing court processes;
  
  
• Easy access to credit;
  
  
• Natural disasters – floods, fire, early closure or late opening of ice roads;
  
  
• Over-reliance on fluctuating own source revenue;
  
  
• Denial or adverse audit report; and
  
  
• Fraud, corruption, nepotism.

As mentioned in Section 3.1, the immediate causes that lead to third party management are somewhat different and more varied. They may include the threat of creditors seizing funds, but also include other non financial reasons, such as the refusal to provide consolidated financial statements, the refusal to sign a funding agreement, or less commonly, a serious election dispute.

A list of potential underlying causes has been developed from the literature review, and supplemented this list through the review of sample RMPs, interviews and the case studies (Appendix I). The underlying causes are summarized into four key categories: poor governance, poor management, community factors and issues with funders and applies to First Nations and/or INAC. In the list below, highlighted underlying causes are addressed in whole or in part by the Intervention Policy, with its focus on financial management and some aspects of governance and funding arrangements. The other underlying causes that are not highlighted may, however, affect the success of the intervention – and would require a different approach to address.

Poor governance

• Disputed elections (legitimacy issues)
  
  
• Leadership (lack of skills, knowledge, experience, health issues)
  
  
• Leadership instability (no shared community vision)
  
  
• Leadership – family v. whole of community orientation
  
  
• Leadership – intimidation by gangs
  
  
• Inappropriate political interference (e.g. portfolio system)
  
  
• Poor accountability to members, funders
  
  
• Few laws or policies
  
  
• Ethical issues (conflict of interest etc.)
  
  
• Unproductive relationships with external bodies (funders, business etc.)
  
  
• Poor dispute resolution, redress
  
  
• Low engagement with community
  
  
• Other

Poor management

• Lack of skills, experience
  
  
• Instability (high turnover)
  
  
• Lack of tools
  
  
• Lack of policies, plans, budgets, expenditure control
  
  
• Lack of basic understanding of money management
  
  
• Poor information to Council

Community disengagement

• Apathetic, low expectations of government
  
  
• Lack of knowledge about government
  
  
• Fear

Poor funding practices

• Inappropriate demands, reporting requirements, late payments
  
  
• Poor risk analysis
  
  
• Poor monitoring
  
  
• Underfunding (e.g. isolated communities)
  
  
• ‘Dumping' money at year end leading to cost overruns
  
  
• Poor co-ordination among the federal family
  
  
• Poor ex ante controls over borrowing

Various contextual factors shape these underlying causes (Appendix I) and are related to community endowments, community member endowments and social cohesion and security. These contextual factors were also initially developed from the literature review and supplemented during our interviews and case studies.

Community endowments

• Few natural resources
  
  
• Geographic isolation
  
  
• Lack of economic opportunities, low own source revenue
  
  
• Dominance of public sector, small or non-existent private sector
  
  
• Rapid population growth (especially if generated by teen pregnancies)
  
  
• Small size – negative effects on capacity
  
  

Community member endowments

• Low education levels
  
  
• Significant health issues (TB, HIV/AIDS, FASD, mental health)
  
  
• Lost cultural identity

Social cohesion and security

• Fierce competition among families, groups (low social cohesion)
  
   
• Psychology of victimization
  
  
• Insularity (few external relationships)
  
  
• High crime rates (gangs, gender violence, vandalism)

Again, the Intervention Policy cannot address these factors but the factors can influence the success of the Intervention Policy.

The relationship between a number of these contextual factors and the level and duration of intervention in our statistical analysis have been tested. The key variables that were examined were: the CWB Index[Note 19] and its components (education, labour, income and housing); remoteness; registered population on reserve; and the type of electoral system (Appendix E - Summary of the Statistical Analysis). In brief, the analysis shows:

• There is a significant relationship between the CWB in 2006 and each of its four components, and whether or not a First Nation was under intervention in 2005/06. In other words, the lower the CWB, the more likely a First Nation would be under some form of intervention. This makes intuitive sense because the lower the CWB, the higher the needs and the lower the capacity.
  
  
• There is, however, no clear relationship between the CWB and its components and the level of intervention. This may be because of the complexity of reasons that lead to third party management.
  
  
• There is some relationship between the CWB and the number of years under some form of intervention – i.e. the lower the CWB, the higher the number of years under intervention from 1999/2000 to 2009/2010. Similarly, there was some relationship between each of the CWB components and the number of years under intervention, except for the education component.
  
  
• There is also a significant relationship between the remoteness of a band and the number of years that it will be under intervention. On average, a band in an urban area will be under intervention for 2.95 years; a band in a rural area (50+ kms from the nearest service centre) will be under intervention for 4.10 years; and a band in an isolated area (no year round road access) will be under intervention for 4.91 years. This also makes intuitive sense because of fewer economic development opportunities in isolated areas and the higher costs for all goods and services.
  
  
• There was no correlation between the size of a community and the number of years under intervention. This may be because much of INAC's funding is based on population size or number of beneficiaries.
  
  
• There was also no correlation between the type of electoral system (Indian Act or custom election code) and the level of intervention. Under the Indian Act, elections are required every two years; under custom election codes the frequency of elections varies, but they tend to occur every three to four years. The type of electoral system was therefore used as a proxy for the frequency of elections. A better measure would be the stability of leadership over time.

4.3 Achievement of objectives

Is the Intervention Policy successful in achieving its objectives and it intended results? Does this vary regionally? a) to what extent does the Intervention Policy reinforce the recipient's accountability? b) does the Intervention Policy contribute to building local capacity in order to deliver higher quality services? c) does the Intervention Policy contribute to the continuity of service delivery, appropriate use of funds?

Section 4.1 above looked at the achievement of the longer-term objectives of reducing the level, incidence and duration of intervention, and Section 4.2 looked at the underlying causes and contextual factors that affect the achievement of the long-term objectives, many of which the Intervention Policy has no influence over. This section will look at the achievement of the short to medium term objectives of ensuring the delivery of essential services, maintaining accountability, and building capacity.

Delivery of essential services

One of the objectives of the Intervention Policy is to ensure that essential services continue to be delivered. Essential services are education, social assistance and some community infrastructure. Non-essential services are governance support, economic development, post-secondary education and some minor capital. Regional officials and recipients indicated that INAC funding for both essential and non-essential services has continued to flow regardless of the level of intervention. First Nations under some form of intervention are also eligible to access targeted funding and many do. Under third party management, however, no new Ministerial Loan Guarantees (MLGs) can be approved by INAC for CMHC-funded social housing. For other levels of intervention, the RMP has to have been working for six months in order for a MLG to be approved.

The impact of intervention on the quality of the programs and services that are being delivered was not assessed because of the lack of program performance data. Twelve of our case studies[Note 20] indicate that implementing remedial measures and reducing debt had numerous benefits for First Nations and other Indian-administered organizations, including:

• Increased efficiency;
  
  
• Improved financial management;
  
  
• Tighter expenditure control;
  
  
• Better program planning and budgeting;
  
  
• Improved relationships with creditors and increased access to credit; and
  
  
• Increased access to funding from a number of sources and the potential to qualify for more flexible funding from INAC.

Ensuring accountability

INAC officials were of the view that the Intervention Policy maintains the external accountability of the recipient to INAC for funding. Although, INAC officials did not think that the Intervention Policy necessarily increased the internal accountability of the recipient to its members.

According to the Intervention Policy, third party managers are expected to account to the community in terms of the funding that they are administering. A few regional officials noted that some third party managers were accounting to community members and this had the benefit of increasing community expectations of their leadership. Regional officials and recipients under third party management, however, noted that some third party managers do not visit communities. The one third party manager interviewed indicated that a lack of guaranteed security while on reserve was a major reason for not visiting the community. A couple of the recipients under third party management also indicated that INAC needed to monitor the performance of third party managers more closely.

Capacity building

According to the Intervention Policy, the onus is on the recipient to enhance their capacity but INAC recognizes the need to support capacity development and commits to use the limited resources available for assisting those recipients most in need.[Note 21] At the time that the 2006 Intervention Policy was adopted, additional funds for capacity building were provided through the Companion Initiative. However, those funds were for short-term projects and were only available for two years.

A total of $10.6 million was allocated to the Companion Initiative in 2006/07 and 2007/08. 115 projects were funded from the 2006/07 funding allocation, or an average of $29,304 per project;[Note 22] and 147 projects were funded from the 2007/08 funding allocation, or an average of $36,000 per project.[Note 23] The projects covered three broad categories: financial development, management development, and intervention system, including the development of RMPs and RMP implementation projects. The funding was provided for tools, training and resources to increase capacity. In addition, the Aboriginal Financial Officers Association (AFOA) developed three new seminar-based and on-line courses for band office staff, including "Developing an Effective Remedial Management Plan."[Note 24]

Professional and Institutional Development (P&ID) funding[Note 25] is available on an ongoing basis for governance and management development, but the amount of funding is relatively small compared to INAC's other programs. For example, the P&ID program transferred $13.9 million to First Nations, Aboriginal organizations and tribal councils in 2007/08.[Note 26] P&ID funding is proposal driven and the demand outstrips the funds available.[Note 27] P&ID also has other priorities, although four regions mentioned that they have managed to set aside funds from P&ID for First Nations under intervention or at risk.

An internal audit of INAC's capacity development programs and authorities, including P&ID, found that INAC does not have a coordinated approach to the delivery of capacity development programs; there are no specific resources to support the design and implementation of effective capacity development programs; capacity development programs often address common issues, such as governance design and implementation, community engagement or skills development; and appropriate performance measurement data to demonstrate program effectiveness is not being gathered.[Note 28] The audit also found that underlying issues at the recipient level impede capacity development including: lack of basic education, difficulty in attracting and retaining qualified staff, periodic changes in leadership, and lack of a comprehensive plan.[Note 29]

The most significant and sustainable capacity building in our case studies occurred through the recruitment by First Nations or other Indian-administered organizations of qualified managers and financial directors or the contracting of expertise, including co-managers. For very small First Nations, as well as isolated First Nations, attracting and retaining qualified staff was an issue and raised questions about the sustainability of any progress.

According to regional officials and recipients, co-managers vary in terms of their contribution to capacity building. Some co-managers set up systems and procedures, control expenditures, train staff and the leadership, and improve financial results. Other co-managers lack the required skills and experience, are overstretched, or do not contribute adequately to the capacity building of band staff. Some regional officials attributed the poor performance of co-managers to inadequate controls by INAC over their selection and performance. Co-managers themselves said that their success was dependent to a large extent on the quality of their relationship with chief and council.

The Intervention Policy requires that INAC review the Co-Management Agreement between a co-manager and the council before it is executed in order to ensure that minimum requirements have been met and that the co-manager will be effective in remedying the default identified. [Note 30] The policy also provides examples of criteria for the selection of a co-manager and of the various roles and responsibilities that could be defined in the Co-Man[Note 31] Some regions have used this provision in the policy to ensure that co-managers have the required level of expertise and experience that they have been given co-signing authority at least for INAC funding, that they develop an exit strategy, and that their performance is reviewed regularly by the council and they are replaced if necessary. Other regions have exercised less control over the selection and management of co-managers by councils. Officials in those regions thought that they should exert more control and that more control might be more tenable to First Nations if INAC also provided funding for the co-managers. One region raised the issue of INAC's potential liability for the performance of co-managers if more control is assumed for their selection and performance.

The consensus among INAC HQ and regional officials and recipients was that most third party managers do not enhance capacity. They are too expensive and getting more expensive; they may not have the required expertise if the underlying cause is non-financial; they meet with resistance from the First Nation; and they usually only deal with INAC's funds and not all sources of revenue.

INAC's Funding Services Officers (FSOs) can provide limited support but according to INAC HQ and regional officials, they are not necessarily qualified and are overstretched. There is also a potential conflict between the FSOs' supportive role and their monitoring role. Many FSOs themselves indicated that they needed capacity building on remedial management.

Tribal councils are another potential source of capacity. INAC has devolved the provision of advisory services for economic development, financial management, community planning, technical services, and band governance to tribal councils or larger unaffiliated bands. Tribal councils are required to provide all five advisory services to their members, but an evaluation in 2009 found that not all were, particularly, financial management advisory services. Communities may resist sharing financial information with their tribal council, and remote and isolated communities have difficulties accessing their tribal council.[Note 32] These findings were echoed in our interviews and case studies - some tribal councils are very supportive and some are not, some communities resist getting support from their tribal council, and some tribal councils lack the skills and resources themselves.

AFOA was most often mentioned by key informants as a potential source for capacity building – training, certification, and advisory services. However, AFOA itself recognizes that it has variable capacity – some of its regional chapters are strong and some are weak. First Nations councils also have to pay for AFOA's services and it is often the bands that are most in need that use the services the least.

4.4 Unintended impacts

What are the unintended impacts of this policy? (positive or negative)

Unintended potential positive impacts that were identified in the interviews and the case studies are:

• intervention at the recipient-managed and co-managed levels provides justification for political leaders to take tough political decisions like laying off staff or reducing payments to band members; and
  
  
• there is a greater awareness within the community or the council about financial problems and the required solutions, and an increase in support for those solutions.

A number of negative impacts were mentioned in our interviews and case studies, but often these negative impacts were related more to the debt situation than to the intervention itself - such as, a reduction in the availability of flexible funding, a reduction in access to MLGs, a reduction in access to credit, staff lay-offs, shortages of staff, loss of benefits for staff, declining staff morale, and band members losing access to loans and advances.

Other potential negative impacts that were noted in the interviews are:

• The additional social costs to a community if some employees are laid off – increased poverty, an increase in crime and vandalism, a loss of self-esteem;
  
  
• Negative public, media, and political attention, particularly in terms of third party management;
  
   
• Increased litigation related to third party management;
  
  
• Absolving the council of responsibility for tough decisions under third party management;
  
  
• Loss of community confidence and credibility in the leadership;
  
  
• The promotion of two sets of books under third party management, an outcome, which is bad public accounting practice; and
  
  
• A deterioration in the relationship with INAC.

One major impact that can affect the success of the Intervention Policy itself was the reduction in the availability of band support funding because of payments to co-managers and third party managers. INAC provides band support funding as a grant so there is a high degree of flexibility in terms of how it is used. However, the evaluation of Indian Government Support Programs conducted in 2009 found that band support funding has not kept pace with the realities of modern governance and administration.[Note 33] The cost of third party management in particular can take most or, in a few cases, all of the band support funding, leaving the band to find other sources to employ a band manager and administrative staff.

There are also perverse incentives for co-managers and third party managers to remain in place rather than to work themselves out of a job. There is no incentive for First Nations to save under contributions or for third party managers to realize savings since they cannot be used to pay off debt.

4.5 Key Success Factors

What are the key success factors?

Success and other success cases were used to explore the reasons why some First Nations have never been under intervention despite being disadvantaged, and why some First Nations were able to get out of intervention after a long period of time under intervention. The reasons for success were also explored in the interviews and the document review. Based on these sources, the key success factors identified are:

• The sustained commitment and attitude of the leadership to take tough decisions, cut costs, adhere to budgets and plans, improve financial systems and practices, and reduce or replace staff.
  
  
• Stability and capacity in leadership and management, or a change in leadership and management if they are part of the problem.
  
  
• Effective management of the political-administrative interface and a clear line of authority between chief and council and staff.
  
  
• Consistent and meaningful community engagement and strong accountability to the community.
  
  
• Access to significant own source revenue to pay off debt.
  
  
• Positive external relationships with INAC, other federal government departments, provincial governments, business investors, other First Nations or their tribal councils.

As with the underlying causes and the contextual factors, the Intervention Policy has little influence over the majority of these key success factors. Our case studies and interviews also noted that improvements in the financial situation or capacity of a First Nation can be fragile – for example, a band-owned business can start to lose money or a key staff person can retire.

4.6 Conclusions related to Success

• In terms of longer term objectives, the incidence of intervention has declined but not the level or duration of intervention. The number of First Nations with a cumulative deficit greater than eight percent has also declined. Any success or failure cannot necessarily be attributed to the Intervention Policy, however, since other factors such as own source revenue, economic performance in certain sectors, or the supply of credit can also have an impact.
  
  
• There are four underlying causes for intervention – poor governance, poor management, community disengagement and poor funding practices. The Intervention Policy only addresses some of the underlying causes. There are also contextual factors that are related to the incidence and duration of intervention, most notably community well-being and isolation, and the Intervention Policy does not address these.
  
  
• In terms of shorter term objectives, the Intervention Policy has been successful in protecting INAC's funding, ensuring accountability from recipients to INAC, and ensuring that essential services continue to be delivered. It has been less successful in ensuring accountability to community members or building the capacity of recipients. Co-managers are the most important capacity-building tool under the policy, but their performance is variable. Third party managers do not generally contribute to capacity building. The most effective and sustainable capacity building comes from recipients recruiting and retaining qualified staff or contracting the appropriate expertise.
  
  
• There are a few unintended impacts of the policy – most notably the negative impact on the availability of band support funding for governance and administration, and the negative consequences of third party management.
  
  
• The key success factors are: willing and committed leadership, stable and competent management, community engagement, and positive external relationships – as well as access to significant own source revenue.

5.1 Costs of intervention

What are the costs of intervention for the recipient and INAC?

The direct cost of the Intervention Policy for INAC cannot be quantified since it is part of the transfer payment process and, therefore, part of INAC's ongoing operations. HQ officials reported that the cost of policy development, monitoring and compliance for the Intervention Policy is not significant. INAC regional officials mentioned that the cost of implementing the policy can be significant, especially in the time and travel of FSOs. Some regional offices also have a specific person dedicated to supporting intervention.

The direct cost of the Intervention Policy for First Nations can be significant, depending on the amount of funding they receive and what level of intervention they are under. Recipients have to pay for a co-manager directly, and payment of the third party manager is taken from the recipient's band support grant. Travel costs for co-managers and third party managers can also be high for remote and isolated communities.

The costs of co-managers have not been quantified due to the lack of comprehensive information for all recipients. Regional officials or recipients mentioned total costs ranging from $100,000 to $300,000 per year for co-managers. These costs would need to be analyzed in relation to the total amount of INAC funding provided in order to gauge the significance for the First Nation.

Cost of Third Party Managers

Since fiscal year 2006/2007, the portion of Band Support Funding allocated to third party managers is recorded as a unique account in the First Nations Information Transfer Payment System (Band support funding for third party management services). Prior to that, there was no distinction in the funds allocated from the Band Support Funding. Even if only two regions were recording the information in this specific account in fiscal year 2006/2007 and the following years, there were a lack of inconsistency in the data inputted in the system (e.g. First Nations under TPM missing, First Nations under co-management listed into this account), in fiscal year 2009-2010, almost all regions recorded the proper information in a consistent manner.

The cost of third party managers varies depending on the fee rates of the service provider, the type of services provided, their level of effort required and the travel and living expenses. The TPM life cycle is also a factor as regional officials also told us that there is a higher level of effort required from TPMs at the beginning of any contract and whenever a TPM is changed.

As shown in Table 8, the amount paid to TPM varies depending on the selected firm or type of resources. Examination of a sample of TPMs agreement for fiscal year 2009-2010 showed that, according resources selected, fees per day, per resources could vary within a range of $557 to $2300 for an administrator, within a range of $575 to $1950 for an advisor and within a range of $275 to $1200 for an administrative support. The latest tender for TPMs (2009) required that third party administrators have an accounting designation (Chartered Accountant, Certified Management Accountant or Certified General Accountant).[Note 34] Some regions noted that this mandatory requirement had resulted in a doubling or tripling of daily fee rates and a smaller pool of potential service providers. There were concerns that the cost of TPM could start to exceed the band support grant of smaller communities and those in the North.

Table 8 - TPM remuneration for fiscal year 2009-2010

Source: Third Party Management Agreements

Data analysis of the fees for TPMs since 2006/07 demonstrated that the cost of TPMs has been stable. While in fiscal year 2006/07, the fees ranged from $29,842 to $768,200, in fiscal year 2008/10 the fees ranged from $43,825 to $339,197. Percentage of the Band Support Funding remained almost the same. In fiscal year 2006/07, the fees ranged from 8.5 percent to 125.8 percent and in fiscal year 2009/10, it ranged from 23.2 percent to 100 percent. It should be noted that in this sample, only one community used all its band support funding in fees for TPMs. Our analysis of TPM professional fees also indicates that smaller bands pay a higher proportion of their INAC's funding and Band Support Funding for TPMs - it could be due to the fact that there are certain fixed costs, the complexity and the amount of INAC program funding that are not linked to the amount of funding that the First Nation receives from INAC's Band Support Funding.

Table 9 - TPM fees and % of total Band Support Funding and total INAC's funding

Source: FNITP (FY 2007- 2008 to 2009-2010) and Q51 for fiscal year 2006/07

5.2 Comparison to cost of other intervention approaches

How does this cost compare to the cost of other intervention approaches?

The costs for Health Canada and its recipients are similar to those of INAC, except that the incidence and duration of intervention is lower. FNFMB has not yet implemented its intervention policy but will have few staff for all of its functions and will be contracting out for a lot of the expertise required. FNFMB has also not decided whether or not to assume the cost of co-managers and third party managers.

The intervention approaches of other countries and provincial governments are discussed in more detail under Section 8, Improvements and Alternatives. Like INAC, the cost for the funder cannot be separated from other costs related to managing transfer payments. In most cases, however, the funder assumes the cost of any special assistance, financial controllers, or third party administrators. The incidence and duration of intervention is also much lower in these other jurisdictions than in INAC's case, with one exception – Queensland Indigenous Councils.

5.3 Cost-effectiveness/value for money

Do recipients and INAC believe that they have obtained value for money?

• Are the costs of remedying the default and building capacity minimized?
  
  
• Could the same results be achieved with less money?
  
  
• Are the most appropriate and efficient means being used to achieve the Intervention Policy's outcomes?
  
  
• Is the process in place economic, efficient and effective?

The Intervention Policy is not cost-effective because it has not been successful in achieving most of its long-term and some of its short-term objectives. The high incidence and duration of intervention drives the costs up for both INAC and the recipient. Unwillingness, lack of capacity and low revenues cannot, however, be addressed by the Intervention Policy alone. They require other "interventions" from other players.

A lot of time and effort is being invested by INAC and recipients in remedial management plans, but they are of questionable value if they are not adhered to, and INAC has limited means to monitor and enforce compliance. It is difficult to think of an alternative, however, other than simplifying them – for example, an action plan to respond to the audit observations, and a brief description of how surpluses are going to be generated. Monitoring could still be carried out quarterly.

Co-managers can perform a useful treasury or controller function and build capacity and their cost appears to be reasonable in most circumstances. The alternative would be a full-time financial controller but that might be more than a recipient needed given their other staff. Some recipients have retained their co-managers on contract after co-management is no longer required by INAC, or contracted co-managers when they are under recipient management. One recipient suggested that INAC pay for the position of a financial director in every First Nation, to manage all revenues, not just INAC funding.

Third party managers are not cost-effective, but they are intended to be used as a last resort when no other option is available. As one region noted, a third party manager who draws thousands of dollars a year from a band budget, has no authority over own source revenue, and leaves nothing behind in terms of infrastructure (or often less than when he/she started), is not a good investment of public money. One recipient noted that the cost of its third party managers for 14 years would have been enough to pay off their debt.

The costs of not intervening are very high as well – loss of accountability, the risk of INAC funds being seized, the risk that essential services will not be delivered, and the risk to the health and safety of band members.

5.4 Conclusions related to Efficiency - Economy

• The Intervention Policy is not cost-effective given the limited success. Considerable time and effort is required from FSOs. The cost of co-managers and third party managers is high for smaller and more remote communities. The main cost driver is the incidence and duration of intervention.
  
   
• Any improvements or alternatives that would address willingness, capacity and low revenue should increase the cost effectiveness. Improvements to the Intervention Policy could include reducing the amount of time invested in remedial management plans, improving the performance of co-managers, and resorting to third party management only as a last resort.
  
  
• Other jurisdictions usually pick up the cost of co-managers or third party managers as well as capacity building. If INAC provided additional funds for the cost of co-managers and monitored their selection and performance more closely, then cost-effectiveness might increase for this intervention level.

6.1 Clarity and understanding of objectives

To what extent are the objectives of the Intervention Policy clearly articulated and understood?

According to HQ and regional officials, the objectives of the Intervention Policy are clear – to ensure funding is used for intended purpose, essential programs and services are delivered, accountability is achieved, and to address the problems that led to intervention. INAC officials were of the view that recipients understood these objectives as well. Recipients themselves did not express any views on the clarity of the policy.

INAC and recipients both agreed that there needs to be a parallel piece on capacity building or other actions to address the situation. Capacity building is not really a focus of the Intervention Policy and there is little direction in the policy about how capacity building is to be delivered.

The financial institution interviewed thought that there should be greater transparency about the Intervention Policy so that lenders know the rules. For example, the Intervention Policy is not currently accessible on INAC's website.

6.2 Clarity and understanding of roles, responsibilities and accountabilities

 To what extent are the roles, responsibilities and accountability of key players clear and well understood? Is there a clear understanding of terminology used?

According to HQ officials and other interviewees, the roles, responsibilities and accountabilities of INAC and First Nations are generally clear and well understood. The onus is on the recipient to address the default and build capacity. Recipients are also responsible for their debt. The terminology is also clear.

Co-managers and third party managers were also of the view that their roles, responsibilities and accountabilities were well-defined but observed that often, they were not able to work in collaboration with chief and council and this impeded their effectiveness. Prior to be appointed with a Third Party Manager, some First Nations were not clear about their responsibilities or were not willing to accept them – e.g. signing the funding arrangement, arranging for a consolidated audit, and hiring staff.

A Google search of media coverage regarding Intervention Policy over the past year indicates that the media and the public do not clearly understand the Intervention Policy and there is a tendency to focus on third party management. INAC HQ officials and other interviewees suggested that there needs to be more effective communication and a clearer distinction among the levels of intervention.

6.3 Consistency of delivery and results nationally

To what extent does the design of the Intervention Policy lead to consistent national delivery/results?

The Intervention Policy provides triggers, assessment criteria, intervention processes, levels of intervention, records management, and roles and responsibilities within INAC. These components of the Intervention Policy are consistent across the regions (refer to Section 7, under Evaluation Findings- Delivery).

The implementation of the Intervention Policy requires judgement to be applied, however, the context and circumstances are different from one recipient to another, within regions, and across regions. First Nations vary by size, degree of isolation, access to capacity or access to own source revenue. There are also variations in the way that INAC's regional offices are structured, the availability of funding services and program staff, and their knowledge and abilities. All of these variations may have an impact on the decisions that are taken. According to the Intervention Policy, however, these decisions should be documented, reviewed, communicated to the recipient, and the recipient given an opportunity to respond (refer to Section 7, Evaluation Findings- Delivery). Some differences among INAC's regional offices have also been mentioned in the Audit of the Application of the Intervention Policy[Note 35].

6.4 Comparison to other intervention approaches

To what extent does the Intervention Policy design compare with other intervention approaches (e.g. other government departments, provincial governments, other jurisdictions, financial institutions, etc.)?

In the literature review, eight other intervention approaches have been looked at, including Health Canada and the FNFMB. As mentioned previously, Health Canada and FNFMB's approach are based on INAC's approach. The other examples were:

• three subnational bodies in the provinces – municipalities in British Columbia, school boards in Ontario and public hospitals in Ontario; and
  
  
• three Aboriginal examples from other countries -  American Indian Tribal Governments, Indigenous Corporations in the Commonwealth of Australia, and Indigenous Councils in Queensland, Australia.

Following is a comparison of the key design features of those other approaches in comparison to INAC's approach.

Intervention objectives

The objectives of intervention are similar to INAC's and include:

• to protect public money, trust lands and trust interests;
  
  
• to ensure accountability that public monies are being used for the purpose intended;
  
  
• to ensure the ongoing delivery of public services; and
  
  
• to protect public safety, health or welfare.

It is usually not an option to dissolve a public sector entity in the event that it is insolvent. Funders or higher orders of government, therefore, have an interest in ensuring that the entity can continue to deliver services without jeopardizing public health or safety. Funders or higher orders of government also have an interest in protecting public funds and assets for present and future generations and ensuring accountability for the use of those funds.

Triggers

In the examples, the triggers that have been used for intervention are similar to those that are used by INAC – a breach in the funding agreements, contract or accountability agreements; operating deficits; adverse audit opinions; and risk to health and safety (or trust lands in the Aboriginal examples). In some cases, performance-based triggers are also being used. For example, the Ontario Ministry of Health and Long-Term Care can intervene in public hospitals if there are accessibility or quality issues. The Ontario Ministry of Education can intervene in school boards if education outcomes are not being attained.

Levels of intervention and remedies

The levels of intervention that have been used are similar to those that INAC uses, with variations in the terminology that is used. These levels include the provision of advice and support by the government or other parties; co-management or the appointment of a financial controller; and third party management or supervision. Co-management is less frequently used in favour of special assistance teams that go in to address problems. In some cases, a higher order of government also has the option of dissolving and reconstituting the governing body of a subnational government – an option that is not available to INAC. In the case of American Tribal Governments, the United States federal government has the option of reassuming responsibilities for the delivery of programs and services because delivery has not been fully devolved to tribal entities.

As with INAC, some form of assessment is conducted prior to assuming some measure of control over the affairs of a recipient. This assessment may be conducted by government officials, by independent bodies, or by peers, whereas only INAC officials conduct the assessment. As with INAC, the assessment not only determines whether intervention is required, but also informs the actions that need to be undertaken in order to rectify the problem.

There are also some examples where assessments or examinations are used more than audits. These assessment or examinations tend to be more frequent, more rigorous, more risk-based, and more preventive than audits. This is similar to the proposed general assessment that INAC will be conducting.

Capacity building for prevention and remediation

A lack of capacity is often identified in the examples as underlying the need for intervention. Many governments have provided a range of capacity building support to recipients, particularly related to financial management but also in governance. This support has been delivered through training, through guides and other tools, through peers, through technical assistance teams, and so on.

In some examples, capacity building is linked to the certification of individuals – for example, governance certification for board members on Ontario's school boards and public hospital boards or on Queensland's indigenous councils. In a couple of examples, capacity building is also linked to the accreditation of the organization itself – i.e. FNFMB or Ontario public hospitals.

Oversight and monitoring

There is a lot of emphasis on ex ante controls in the literature on subnational debt management, as well as in the examples involving provincial governments and municipal governments, school boards or hospitals. These controls include the requirement for balanced budgets, no operating deficits on current accounts, limits on borrowing, approval of long-term liabilities, etc.

In some cases, such as the FNFMB or the Registrar of Indigenous Corporations in Australia, the oversight and monitoring of intervention is carried out by an independent party, separate from the funders. This independent third party can address problems in the organization as a whole, and can act on behalf of all funders rather than just one funder or department.

6.5 Well designed to achieve objectives

Is the Intervention Policy well designed to achieve its objectives?

According to INAC officials, the Intervention Policy is not well designed to achieve its objectives. It does not include prevention measures; the categories of default or the triggers are too broad or ill-defined; capacity building is not adequately addressed and there is no framework to assist recipients to exit intervention; there is no performance measurement framework; the debt management rules are restrictive; recipients are not encouraged to take responsibility and build capacity at higher levels of intervention; and there is a lack of funding to assist recipients under intervention without having to resort to band support grants. These deficiencies will be addressed further under the suggested improvements to the policy (Section 8.1).

6.6 Conclusions related to Design

• The Intervention Policy is not well designed to achieve all of its objectives. The key suggestions for changes were: adding a capacity development framework as part of the policy or as a companion piece; incorporating prevention strategies; and promoting common assessment tools linked to the new Policy on Transfer Payments. Key informants also suggested providing additional funds to pay for co-managers and third party managers; and developing alternative debt retirement tools for those recipients with little access to own source revenue.
  
  
• The Intervention Policy is comparable to other intervention approaches, but constrained by the overall legal framework (i.e. that the federal government's authority to intervene is limited to the terms and conditions of funding agreements and not prescribed by legislation).

This section presents the division of roles and responsibilities within INAC, the key activities that are being undertaken (prevention, detection, remediation, exit strategies and monitoring), and the key outputs related to the implementation of the Intervention Policy (refer to Section 1.2.3 Policy Framework and 1.2.4 Policy Management). A more thorough review of delivery is provided in the Audit of the Application of the Intervention Policy (2009).

7.1 Implementation as planned

Is the Intervention Policy implemented as planned?

Although the Intervention Policy does not address prevention, the Atlantic, Quebec and Alberta regions have instituted formal systems to detect potential early warning signs by using more comprehensive assessment grid that might lead to the need for intervention. Other regions have a more informal process of monitoring communities that might be experiencing difficulties more closely. Atlantic and Alberta regions also provide an orientation to new chiefs and councils on their roles and responsibilities under the funding agreements. All regional staff spend significant time explaining INAC programs, the funding available, and the terms and conditions of funding agreements in order to avoid the need for intervention.

According to regional officials, prevention and early detection requires effort by the FSO and a willingness by the recipient to address issues before they get to the point where intervention is required. FSOs don't necessarily have the time or the ability to monitor developments closely. There is, therefore, a tendency to rely on the audit report which comes in several months after the fiscal year end and the situation may have deteriorated further by then.

Interviews and the document review indicate that regional assessments are based on the Performance Review Model in the Intervention Policy. The model looks at the willingness and capacity of the recipient, financial indicators relating to expenditures and revenues, reporting, and audit results. Our case studies indicate that the imposition of remedial management appears to be linked more to the relatively small size of the deficit (e.g. 0 to 16 percent) than to the willingness and capacity of the recipient; co-management to larger deficits (e.g. 17 to 32 percent) rather than a lack of capacity; and third party management to very large deficits (33 to 100 percent) as well as unwillingness.

Other factors that may be considered when making a decision about intervention are complaints and allegations or calls from creditors. Co-managers may also conduct a situational review once appointed, or an interim audit may be commissioned to update the figures on the financial situation of the recipient.

Our review of the sample of remedial management plans indicates that RMPs are variable in quality and format across and within regions. In some of the regions, a standard format has been adopted and used by recipients. In other regions, the RMPs vary by the recipient. RMPs are not always available or if available, they are not always approved by the First Nation, particularly when co-managers or third party managers are appointed.

In terms of the minimum RMP components (refer to Section 1.2.3 Policy Framework), our review of the sample of RMPs indicates:

• The purpose of the RMP was stated in 95 percent of the RMPs and was usually to address a deficit and certain management issues.
  
   
• The effective date was not clearly stated in 28 percent of the RMPs, and it has been noted that it took quite a while to finalize some of the RMPs. The 2009 Audit also observed that in 47 percent of the files tested, RMPs were not obtained in a timely manner and no consequences were imposed on the recipient.[Note 36] Our sample RMPs covered various timeframes – anywhere from three years to as high as fifteen years.
  
  
• Problem identification and corrective action were included in 94 percent and 97 percent of the RMPs, respectively, although the degree of detail varied.
  
  
• Performance indicators were included in about two-thirds of the RMPs. They varied in terms of the specificity of targets and deadlines.
  
  
• Roles and responsibilities were defined in 81 percent of the RMPs, but varied in terms of the level of detail. Some RMPs included the roles and responsibilities of INAC in addition to the chief and council, some broke down the responsibilities within the band council among different managers, and some included the roles and responsibilities of co-managers and third party managers.
  
  
• Financial projections were included in 61 percent of the RMPs where the cumulative deficit was an issue. Sometimes the opening position (current year and current year projections) was not known and was one of the first action steps under the RMP. In some cases, financial projections included a monthly breakdown by program over a three to five year period; in other cases, financial projections were annual rather than monthly. In some of the examples, a summary by program was included that described where and how surpluses would be generated and it has been found useful in gaining an understanding of what was intended.
  
  
• Debt management plans were included in 77 percent of the RMPs where the cumulative deficit was an issue. Sometimes debt management plans were to be developed. Some of the examples included a detailed analysis of accounts payable, terms and payment obligations and sources of funds. A few examples merely planned to pay off the deficit through program surpluses without providing any further detail.
  
   
• Capacity development plans were included in 45 percent of the RMPs. In some cases, capacity development plans were to be developed separately. A few examples included detailed capacity development plans at all levels of the band council, to be implemented through a variety of capacity building mechanisms.
  
   
• Reporting and monitoring was included in 78 percent of the RMPs, with the most common form of monitoring being monthly financial statements and monthly or quarterly meetings with INAC.
  
  
• Exit strategies were included in 86 percent of the RMPs and usually provided for termination once conditions were met. In a few of the examples, a more detailed exit strategy was provided indicating all of the conditions that would need to be in place for termination or de-escalation of the intervention (e.g. unqualified audit, timely reporting, implementation of an improved management control framework) and an outline of the process to be followed.
  
   
• It was not always clear whether the RMPs had been approved by the recipient and by INAC. Only about half of the RMPs received included signatures from both INAC and the chief and council, but there may be other documents that the evaluation team did not receive that include the signatures. Some RMPs were in the form of amendments to funding agreements with signature blocks for INAC and the recipient. Other RMPs were accompanied by band council resolutions indicating agreement by chief and council, or by a signed regional committee decision memorandum indicating approval by INAC.

All regions have a regional review and approval process to recommend the level of intervention and approval of remedial management plans, and to monitor implementation. These processes are linked to the audit review process. Manitoba region invites the First Nation to make a presentation to the regional committee when a decision or recommendation on intervention is pending. Quebec region has an internal committee and an external committee that is involved in each intervention decision. In the Alberta region, chiefs and councillors or members of the band council administration may attend the review committee.

FSOs play a key role in the delivery of the Intervention Policy by maintaining a positive relationship with the First Nations that they are responsible for, providing assistance in the development of RMPs, and monitoring the implementation of RMPs. Some regions have a high turnover or high vacancy rate for FSOs that impedes delivery of the policy. In all regions, the workload and requirements of the position limits the amount of capacity building that an FSO can provide. Some FSOs also commented on their lack of training and orientation to the Intervention Policy.

Co-managers are considered the weakest link by regional officials. As noted in Section 4.3, Achievement of Objectives, some co-managers are not considered qualified or experienced, some are not co-signing for the expenditure of INAC funds, and some are not performing well. Some regions are proactive in approving the selection and monitoring of co-managers by First Nations. Others are reluctant to exercise authority in approving the selection.

Third party managers are appointed from the national list of pre-qualified firms or individuals and funded through a third party management framework agreement. The decision to impose third party management is made by the regional director generals. In term of qualification / pre-qualified firms or individuals, the MERX process requested in 2010 that the third party administrator have an accounting designation. However, some have concerns that the individuals without designation that were delivering third party services be hired by those firms to support third party administrator.

7.2 Constraints or pressures impeding implementation

Are there constraints or pressures that impeded the implementation of Intervention Policy? (Note: As the delivery has been addressed in the audit, the evaluationteam looked at this question regarding the impact on the success)

Case studies of recipients that have been under recipient-managed, co-managed or third party-managed intervention for a long time have been used in order to assess the constraints or pressures impeding implementation of the Intervention Policy. Analysis of those cases demonstrated that the situation and circumstances of each First Nation were different and there was no single answer for each level of intervention.

First Nations under recipient-managed intervention:

• In three of the cases, fairly steady progress has been made in reducing the deficit over a period of ten or more years. In one of those cases, the First Nation has exited intervention. In the other two, the First Nation still has to provide an unqualified consolidated audit or demonstrate that they can retain the required financial management capacity.
  
   
• In another case, progress in reducing the deficit has been more erratic – with initial declines over several years, then a sudden jump, a revised RMP, declines again, and recent indications that the deficit has increased again. In this case, consideration is being given to escalation to co-management and the First Nation's bank has already imposed a voluntary co-manager.
  
  
• In the final case, the First Nation should probably be under co-management but it is a very small First Nation and the cost of co-management would exceed its Band Support Grant. Its biggest challenge is deciding what to do with an unprofitable band-owned business.

First Nations under co-management or third party management:

In the nine cases where First Nations have been under co-management or third party management, the evaluation team looked at whether capacity was being developed by the co-manager or third party manager while expenditures were being controlled.

• Four of the cases have been under co-management for a number of years. Three had managed to reduce their cumulative operating deficits to an acceptable level. In one of those cases, a change in chief and council and in the co-manager appears to have improved performance, together with their ability to use own source revenues to pay off their debt. In the other two cases, the First Nation was working well with their current co-manager and could move to recipient-managed intervention except that they have qualified audits and there is still an ongoing need for capacity development. The fourth case should potentially be under third party management but INAC is reluctant to impose that level of intervention.
  
  
• Five of the cases have been under third party management for a number of years. Two of those cases had just moved to co-management – because the third party manager had been able to improve the finances, negotiate an agreement with creditors, and develop an RMP; or because the FSO had been able to overcome the resistance of chief and council to providing consolidated audits and signing funding agreements. In two other cases, no progress had been achieved in obtaining consolidated audits or having funding agreements signed and the overall financial situation of each of the First Nations was not known. In these cases, a recent change in the chief or increased support from the council or the community may change the situation. In the final case, the First Nation was previously under co-management and placed under third party management because of impending action by creditors. The major challenge appears to have been poor relationships with both the co-managers and the third party managers, as well as frequent changes in the leadership.

The three cases involving other Indian-administered organizations are under recipient-managed intervention and in all three cases the intervention was triggered by an operating deficit. In two of the cases, there were also governance and financial management issues. All three organizations are working off their deficit, and two have made significant governance and management changes. In two of the cases, there was political interference in the organization by the First Nation(s) to which it was linked. In the third case, the band council did not interfere in the operations of the organization, nor did it provide any support.

In summary, therefore, the case studies indicate that progress under intervention is usually not linear. Progress is made for a while, then there are setbacks; First Nations move up and down the various levels of intervention. Many of the problems that have led to intervention cannot be addressed in the short-term, but require long-term solutions. It takes time to reduce deficits, especially if there is limited access to own source revenue; to build up financial policies, systems and procedures and to train staff; and to build up a set of accounts that can be audited. Meanwhile, the context is constantly changing – changes in governance, management and staff, the membership, funding conditions, access to own source revenue, natural disasters, etc.

Some of the explanations as to why it was so difficult for recipients to exit intervention are included in the analysis of underlying causes and contextual factors in Findings and Conclusions on Success. Some other constraints included:

• INAC did not intervene soon enough or did not escalate the intervention soon enough so that the problems were compounded. This has been particularly noticeable with the escalation to TPM because of its impacts on the First Nation. When a First Nation is under TPM, relations between INAC and the First Nation are more difficult and at long term, its capacity building is diminished.
  
  
• There are very few tools available to enforce compliance with remedial management plans or co-management other than escalation. Regional offices are reluctant to impose third party management because it is generally not effective. They may also be reluctant to impose co-management if the cost will exceed the Band Support Grant in very small First Nations; or there are few co-managers available in more remote areas.

FSOs were generally aware of the issues and problems and many offered support to the recipients, but the support was not always accepted.

7.3 Tools for collecting evidence

To what extent do the means and tools exist for: a) obtaining reliable evidence of consistent policy implementation; and b) success/performance (including indicators)?

The FNITP system includes an intervention module for capturing information related to the application of the Intervention Policy. According to the HQ officials responsible for the system and to the regional officials responsible for entering the data, the module is not easy to use. There are a lot of layers for input and for performance reviews.

It is also not easy to report on intervention using the system because the data does not match the policy. The reports produced are not useful for monitoring, diagnosis and problem-solving. The data had not been kept up to date and accurate until recently due to closer monitoring by senior management. HQ FNITP officers have to spend a lot of time validating the information and producing reports on request.

The FNITP intervention module is to be amended based on the new intervention policy and directive once approved.

7.4 Policy guidance and performance monitoring tools and mechanisms

To what extent is good guidance and monitoring in place?

Policy guidance tools and mechanisms

The Financial Policies and Procedures Manual contains various tools and templates to assist with implementation of the policy – a template for the contents of a remedial management plan, suggested criteria and a statement of work for a co-manager, sample letters to be sent to creditors, a third party manager agreement model, and a performance review model for assessing the willingness and capacity of recipients. Other tools are also available outside of the policy such as reporting requirements or complaints and allegations procedures.

At the time that the 2006 Intervention Policy was launched, an information tool kit was developed for FSOs to assist them in responding to questions from First Nations on the revised intervention framework and to provide guidance on the garnishment of First Nations funding, the selection of third party managers, and related technical and procedural matters.[Note 37]

Ongoing guidance is provided by INAC HQ on third party manager framework agreements and the use of the third party manager list.

Some of the regions have also developed their own policy guidance and tools whereas other regions consider the existing policy guidance to be sufficient. Quebec region had already developed its own regional intervention procedures just prior to the new Intervention Policy being adopted;[Note 38] British Columbia region subsequently issued a policy and procedures manual and intervention plan receipt procedure and FNITP guide;[Note 39] Atlantic region adopted its own Intervention Policy Directive and developed a set of criteria and a monitoring system to manage intervention; and Alberta region has developed standardized templates for RMPs and co-management agreements, Management Development Plans, and financial recovery and repayment plans.

Performance monitoring tools and mechanisms

Senior management has only recently been monitoring performance related to intervention. The Intervention Policy is now a standing item at the Financial Management Committee for monthly review. The Operations Committee is also interested in the issue. A Working Group that includes representatives from the regions and the Chief Financial Officer Sector has been reviewing the Intervention Policy.

INAC's Compliance Unit reviews implementation of the Intervention Policy as part of a review of transfer payments that is conducted on an annual basis across all regions. The Unit reviews files and the supporting documentation for decisions, prepares a report on its review, and follows up in two years time on the implementation of the changes required.

FSOs review the implementation of RMPs monthly or quarterly and regional committees review performance quarterly or annually.

According to the Intervention Policy, the performance of third party managers is supposed to be reviewed monthly, wherever possible with the recipient. Regional offices indicated that they review performance when monthly invoices for payment are submitted by third party managers. The monitoring of our third party management cases and a few of the co-managed cases was however difficult because approved RMPs were not in place, the band council was uncooperative or antagonistic, or the TPMs or CMs were changing frequently. A couple of First Nations also commented that INAC's monitoring of third party managers was poor – i.e. third party managers were not visiting the community, were contracting for services from unqualified contractors, or were allegedly misusing funds.

7.5 Implementationof audit recommendations

To what extent have recommendations from past audits been implemented as planned?

An audit of the management controls associated with the application of the Intervention Policy was conducted in 2008-09.[Note 40] The audit noted that there were significant variations between policy and practice and from region to region in terms of how the intervention policy was being applied. The audit concluded that management controls are not adequate to ensure effective monitoring of the application of the intervention policy and the application of judgement or decision making that is appropriate, deliberate and consistent across the Department.

The audit recommended an assessment of the effectiveness of the intervention policy, clarification of the policy objectives, revision of policy components to ensure consistency with the objectives, and the development of related tools and guidance to ensure the policy is implemented as intended. Policy revisions and related guidance included:[Note 41]

• Roles, responsibilities, accountabilities and expectations regarding Intervention Policy design, development, implementation and monitoring across CFO Sector, Programs and Regional Operations (particularly FSOs).
  
  
• When a recipient's level of intervention should be escalated or de-escalated.
  
  
• Expectations for a risk-based and integrated strategy and approach to evaluate compliance with the Intervention Policy at a national and regional level.
  
  
• Minimum standards of documentation related to ongoing monitoring activities – e.g. trip reports, minutes of meetings, decisions, action items, etc.
  
  
• Practices including monitoring related to the timely receipt and review of RMPs, as well as the enforcement of Intervention Policy requirements in cases when RMPs are not received from recipients.
  
  
• An approach for tracking the status of recipients under intervention, progress against plans at a regional level, and overall effectiveness of Intervention Policy at a national level.
  
  
• Sustainable strategy and approach to performing regional quality assurance reviews.

An Intervention Policy Audit Action Plan has been developed by INAC that includes actions to reassess the policy with a view to clarifying policy objectives, components and related tools and guidance; and actions to identify, recommend and develop tools and practices to support consistent application across regions based on risk and best practices. The actions have been undertaken by a joint Regional Operations Sector-CFO Intervention Policy Working Group. A new intervention policy and directive have been drafted and will be finalized in the fall 2010.[Note 42]

7.6 Coordination within INAC, with OGDs, with provincial governments, with others

What is the extent of coordination of delivery within INAC, with OGDs, with provincial governments, with others?

According to HQ and regional officials, coordination within INAC was generally good – i.e. with governance, education, capital and social programs and between the CFO and Regional Operations Sectors. However, some FSOs mentioned that they would like additional support from program staff when dealing with particular issues related to that program in recipients under intervention.

Communication with other federal government departments, particularly Health Canada, could be improved and formalized. While Health Canada HQ officials considered coordination at the HQ level with INAC to be good, Health Canada officials at the regional level said that they would like to be better informed about INAC's intervention in their recipients, including but not limited to third party management. One regional Health Canada official described a case in which significant capital funds from his department were seized by a creditor, a situation that might not have arisen had INAC officials informed them that the First Nation had been put under TPM.

Other federal governments departments that could be consulted more by INAC regional offices are the Department of Justice if there are election disputes; the Canada Mortgage and Housing Corporation if there are issues with social housing; the Department of Fisheries and Oceans, Agriculture and Agri-Food Canada or Natural Resources Canada if there are issues with economic development activities; the Department of Public Safety if there are policing issues; and Human Resources and Skills Development Canada if there are issues of employment, skills development or community development.

Quebec region has a formalized interdepartmental approach as part of their external review committee that also includes other federal departments, the provincial government, and First Nations. Manitoba and Yukon regions share information on intervention among the federal family in their rederal councils. In other regions, coordination is more informal and there are difficulties sharing information and no common approach. In one of our case studies under third party management, there was no coordination among the different federal government departments outside of INAC and Health Canada, even though those other departments were providing a lot of funding to the First Nation. On the other hand, one of our case studies included an interdepartmental initiative to address underlying causes and contextual factors.

There was almost no coordination with provincial governments outside of exceptional circumstances - for example, intervention in child and family service agencies that are regulated by the province; or joint audits of social welfare services under the INAC-Ontario agreement. Provincial governments are however affected if First Nations are not paying for tuition in provincial schools or for provincial water, electricity or telecommunications services.

Collaboration with financial institutions varied across the regions and by recipient. Financial institutions would like to be informed in a more timely way if third party management is imposed and to have a creditors meeting with the third party manager as soon as possible thereafter. In a few cases, INAC, the First Nation and the financial institution are working together on a common solution.

7.7 Best practices and lessons learned

What are the best practices and lessons learned?

The following best practices or lessons learned were noted by key informants:

• Quebec region's preventive approach requiring an action plan from the recipient when problems start to emerge but before intervention is required;
  
  
• Quebec region's external committee to review and recommend intervention that includes other federal government departments, the provincial government, and First Nations;
  
  
• Manitoba and Alberta regions' advance notice to recipients of the intention to cancel a multi-year agreement or to require intervention;
  
  
• Manitoba and Alberta regions' provision for recipients to attend the regional review committee and present their case;
  
  
• NWT region's support for all First Nations to attend training on preparing an RMP, with INAC Field Officers present;
  
  
• Ontario region's requirement that TPMs prepare an exit strategy;
  
  
• Alberta region's new regional policy on community communication – whereby INAC will have its own meetings with the community even without chief and council, particularly when there is a higher level of intervention; and
  
  
• Atlantic and Alberta regions' orientation of new chiefs and councils.

7.8 Conclusions related to Delivery

• The Intervention Policy is being implemented as planned, with some gaps. Prevention and early detection are difficult and FSOs tend to rely on audit reports. Regional assessment and approval is being conducted according to the Performance Review Model in the policy. RMPs are variable in terms of format, comprehensiveness and quality across and within regions, and there are generally deficiencies in terms of the effective date, performance indicators, debt management plans, capacity development plans, and approval by the recipient. FSOs are in a key position in terms of implementing the Intervention Policy but there are constraints in some regions related to the high turnover among FSOs, vacancies, heavy workloads and insufficient training.
  
  
• The constraints that recipients face in implementing remedial measures vary; many of the problems require long-term solutions; and the context is constantly changing. INAC may also not have intervened early enough or escalated the level of intervention soon enough, and there are few tools available to enforce compliance by recipients.
  
  
• The major tool for monitoring implementation of the policy is FNITP. The intervention module is, however, not user friendly or useful for monitoring.
  
  
• Policy guidance and tools are available within the policy and related documents prepared by INAC HQ or some of the regions. Performance monitoring appears to be done regularly, although it is not necessarily always documented other than through the formal annual audit review process.
  
  
• Action plans were developed to respond to previous audit recommendations and a new intervention policy and directive has been drafted.
  
  
• There are no formalized processes for communication and the sharing of information, more common approaches to recipients and coordination with other federal government departments and other key players.
  
  
• There are a number of best practices in the regions related to different of aspects of the policy.

8.1 Improvements to the Intervention Policy

Are there improvements that could be made to better achieve Intervention Policy objectives?

The main improvements to the Intervention Policy that have been suggested by key informants relate to prevention, revised triggers, capacity development, and strengthening all three levels of intervention. It was also suggested that there needs to be other strategies to address issues, such as governance disputes and program management. These improvements are related to those underlying causes over which the Intervention Policy has some influence.

Prevention and early detection

In terms of prevention and early detection, interviewees suggested that it be linked to the general assessment that will be conducted as part of the new Policy on Transfer Payments (PTP). The PTP requires that transfer payment programs be designed, delivered and managed in a manner that takes account of risk and clearly demonstrates value for money. In addition to grants and contributions, Aboriginal recipients may also be eligible to receive fixed, flexible or block contribution funding. All recipients are to undergo a capacity and risk assessment and the results will influence the duration of funding agreements, the monitoring regimes, the frequency and type of reporting, the frequency and amount of payments, and the audit arrangements. The results will also identify possible capacity strengthening opportunities.[Note 43]

INAC has developed a new tool to conduct recipient risk assessments. Recipients will be categorized as high, medium or low risk based on eight potential risk factors: external factors or extraordinary events; major changes being undertaken by the recipient; the status of remedial plans; governance; planning; entity management (finance and administration); program management; and project management. The general assessments are intended to provide a more formalized process to identify emerging risks and implement risk mitigation strategies before intervention is required. They can also be used to inform the development of capacity building for recipients at risk. For those recipients in the high risk category, a more in-depth assessment would be required to determine capacity building requirements.[Note 44]

General assessments is planned to be conducted in the fall of 2010 for the 2011-2012 funding arrangements and annually, thereafter. They will be completed by the FSO, reviewed by a regional risk management committee, and discussed with the recipients.

Triggers

In terms of the triggers, the default on terms and conditions is considered by some officials to be too broad. They point out that for administrative defaults such as late reports, INAC can hold back the next tranche of funding. Although the evaluation team have focussed in the evaluation on the three levels of intervention (recipient-managed, co-managed and third party managed), the Intervention Policy does provide for other actions to be taken such as withholding funds, terminating the agreement, or requiring or taking any reasonable action. From that perspective, therefore, there needs to be a trigger that is sufficiently broad to cover a variety of circumstances that may arise and provision for a variety of responses that are not related to the operating deficit or adverse audits.

In terms of the eight percent trigger, First Nations are required to use the new Generally Accepted Accounting Principles for local governments in preparing their financial statements from April 1, 2010, moving from cash to accrual accounting. The eight percent cumulative operating deficit will, therefore, no longer exist as it is currently defined. A paper prepared for INAC has recommended the use of three ratios and a review of operating results (annual surplus or deficit) as a means to assess the financial health of a First Nations. The three ratios are: liquidity, net debt to total annual revenue, and debt servicing to total annual revenue. Taken together, the three ratios and the operating results should better indicate whether the First Nation can afford the goods and services provided; has a sufficient level of working capital, has the ability to settle the liabilities it has incurred, and can pay for the debt it has acquired.[Note 45]

Capacity building

INAC officials have suggested that capacity building be linked to a broader assessment of needs and a wider range of tools and mechanisms. A team of people (or special assistance teams) could assist with the assessment, the determination of what support is required, and the delivery of the support. This team of people could include consultants, peers or others such as AFOA.

This type of approach would be consistent with INAC's Community Development Framework, which proposes a differentiated approach to First Nations that recognizes their diversity. Targeted investments and supports are to be provided to build capacity and move communities out of intervention and stabilize fiscal management. The general assessment referred to above will assist in identifying where First Nations are on a continuum of development.[Note 46]

Capacity building could therefore be targeted at a range of potential underlying causes in our Causal Framework (Appendix J), including:

• enhancing leadership knowledge and skills;
  
  
• improving the political-administrative interface;
  
  
• increasing accountability to members as well as funders;
  
  
• developing administrative laws and policies;
  
  
• improving ethics and conduct;
  
  
• improving relationships with external bodies;
  
  
• increasing the management skills and experience of staff;
  
  
• developing management policies, plans, budgets and tools;
  
  
• improving understanding of money management;
  
  
• improving the accountability of management to the council; and
  
  
• increasing community knowledge about government and financial management.

It was also suggested that training on the Intervention Policy be provided to INAC Funding Service Officers to increase consistency and improve monitoring.

Strengthening the Levels of Intervention

INAC regional officials suggested that co-management could be strengthened by INAC approving the selection of the co-manager based on the criteria in the Intervention Policy, as well as the co-management agreement to ensure that the co-manager has the authority to co-sign financial instruments. It was also suggested that INAC fund all or part of the costs of co-managers in order to increase the willingness of recipients under co-management and retain sufficient band staff to be trained.

INAC officials agreed that while third party management was undesirable, it was occasionally unavoidable, and should be reserved for situations where there is unwillingness or a total lack of capacity or where creditors are threatening to garnishee First Nation funds. Third party managers should, however, be supported by other capacity building or other measures that are separately contracted, with separate program funding. It was also suggested that INAC fund all or part of the costs of third party managers in order to retain sufficient band staff to be trained.

Communication and coordination within the federal family could also be improved and formalized as part of a more common approach to recipients.

Overall, regional officials made a strong case for continuing to provide regions with enough flexibility to deal with different circumstances in the most appropriate way, under a consistent policy framework.

8.2 lternatives to the Intervention Policy

Alternatives to the Intervention Policy

A number of other alternatives were suggested in the interviews or the literature review. These alternatives address other underlying causes or contextual factors than those mentioned in the preceding section. They relate to other policies or programs of INAC and are beyond the scope of the Intervention Policy, but affect the success of the policy.

In terms of improving governance and community engagement, the alternatives include:

• Mediation and dispute resolution mechanisms for councils or communities.
  
  
• Suspending elections in certain circumstances.
  
  
• Promoting comprehensive community planning.
  
  
• Amending Indian Act election rules to reduce the frequency of elections.
  
  
• Improving custom election codes to reduce instability, enhance legitimacy, and avoid disputes.

In terms of improving INAC's funding practices or the funding regime, the alternatives suggested include:

• Improving the advice and monitoring of program expenditures.
  
  
• Reducing reporting requirements.
  
  
• Reviewing funding formulas (e.g. for isolated communities) and increasing funding.
  
  
• Revising the community infrastructure program and other programs to increase flexibility and improve outcomes.
  
  
• Improving economic development programming.
  
  
• Examining alternative debt management approaches, particularly for First Nations that have difficult reducing debt because of low own source revenue, little flexibility, and the cost of third party management. Some of the options include: debt consolidation, debt forgiveness or retirement, a review of Ministerial Loan Guarantees, a regulatory framework for borrowing, increased funding, or alternative sources of revenue (i.e. economic development, access to capital, revenue sharing, etc.).

Other longer-term alternatives suggested were:

• Encouraging the aggregation of smaller First Nations into larger units for the delivery of some services.
  
  
• Creating institutions for the oversight and accountability of First Nations, outside of INAC.

An Alternative Approach

Our case studies and the literature review provide models or inspiration for a new intervention approach. This alternative approach could be used for the more intractable cases that have been under intervention for a long period of time with little hope of exiting any time soon. These intractable cases face a number of contextual factors that the Intervention Policy cannot address. The alternative approach would be holistic and long term and encompasses community rebuilding.

In one of our case studies in northern Manitoba, the INAC regional office, other federal departments and the province are working with the First Nation Council on a joint committee to address a set of priorities determined by the First Nation. The initial starting point was to deal with a growing number of youth suicides through the redevelopment of a partially completed community arena that will include a youth centre. (This partially completed arena was one of the principal causes of the community's financial difficulties that led to intervention.) Two federal departments in collaboration with the province and the First Nation were able to develop a solution complete the arena and develop the new youth centre. The joint committee is prepared to assist the First Nation with other community priorities, such as the development of a written election code.

An alternative approach that emerged from our literature review – a northern pilot project - provides additional insights. Like the Manitoba case study, it focuses on youth and has a similar underlying structure – a steering committee of officials from federal departments, the territorial government, Inuit organizations and the Town Council. It goes several steps further, however. First, the community has two important structures: i) a youth council on which a member of the Hamlet Council sits as an ex officio member; and ii) an interagency co-ordinating committee composed of front line workers from the hamlet, territorial and federal governments. Both of these structures help set priorities and are critical sources of information and advice for the steering committee. In addition, there is a hired youth worker who knows the community well but is not from the community. His job is to support youth in developing their increasingly important role in the community. Finally, the federal players are committed to developing a single window through which to funnel federal funding to reduce the reporting burden and alleviate cash flow issues.

For the past three years, annual expenditures on the northern pilot project have been in the range of $400k to $650k. Initiatives included the revitalization of a youth centre and its governance by youth, a small engine repair course, recreation activities, suicide prevention programming, cultural pursuits, education initiatives (e.g. a peer helper program for all grades), and youth carpentry programs. The next priority is crime prevention to counter, among other things, the influence of illegal drugs and alcohol.

The three year pilot phase produced mixed results. On the one hand formal and informal evaluations indicated positive impacts on youth (reduced crime, the growing importance of the Youth Council as a major force in the community, and evidence of a greater sense of pride and self-esteem among youth). On the other hand, the project failed to achieve a single window approach to federal programming support, resulting in some considerable hardship for the community (reporting issues, cash flow problems etc.). Furthermore, the project was under-resourced – i.e. a side project of various individuals working in different federal departments.

After some debate, the Hamlet Council agreed to extend the pilot for an additional three years. Changes to the initiative include the following:

• An Memorandum of Understanding to be developed among federal partners to flow all project funds via the Inuit Relations Secretariat (IRS) in INAC;
  
  
• IRS will develop a single funding arrangement with the Hamlet; and
  
  
• A reconstituted Partnership Board to include the following members: Canadian Heritage; Health Canada; Human Resources and Skills Development Canada; IRS; Public Safety; Hamlet of the community; Department of Education, Nunavut; Department of Culture, Language, Elders & Youth, Nunavut; and the Qikiqtani Inuit Association.

In the literature review, examples in the fragile state literature with remarkable parallels to the two Canadian examples cited above, have been found, but on a larger, country-wide scale. Like the Canadian examples, a central feature is a co-ordinating mechanism for all of the aid agencies supplemented by an intervention team with high credibility among all of the players in the country. This co-ordinating mechanism works directly with the Government but draws in other elements in the country as well. Of note is the use of a comprehensive assessment of the conditions in the country shared by all of the members on the co-ordinating committee. As well, there is a significant emphasis on physical security and on training participants throughout the country in conflict resolution techniques. Finally, the literature suggests collecting baseline data on which to base an evaluation of the intervention.

8.3 Conclusions related to Improvements and Alternatives

• Suggested improvements to the Intervention Policy focussed on better prevention and early detection linked to the new general assessment; a revised trigger related to the financial health of recipients; a broader assessment of capacity needs, and a wider range of capacity building tools and mechanisms to address some of the aspects of poor governance, poor management or community disengagement; strengthening of the co-management level of intervention, including paying for all or part of the costs of co-managers; and retaining third party management as a last resort, with separate capacity building initiatives, and provision for paying for all or part of the costs of third party managers.
  
  
• Suggested alternatives to the Intervention Policy address other underlying causes related to governance, community engagement, and INAC's funding practices or the overall funding regime. Alternative debt management approaches were considered to be of particular importance.

9.1 Conclusions

Program Relevance

The Intervention Policy remains relevant and there is a continued need to ensure that essential services are delivered, accountability to INAC is maintained, deficiencies in financial management and administration are addressed, and the capacity of recipient to provide services is enhanced. It is also consistent with federal government and INAC policies and priorities and the departmental strategic outcomes. As for federal roles and responsibilities, the federal government is not responsible for First Nations' debt. However, the Intervention Policy supports the federal government's other roles and responsibilities. INAC's Intervention Policy doesn't overlap or duplicate with other programs, policies or initiatives delivered by other stakeholders. In fact, it has served as a model for others stakeholders that deliver services to First Nations and is similar to the intervention approaches in other jurisdictions.

Program Success/Performance

There has been some recent progress in reducing the incidence of intervention but not the level and duration of intervention. A number of underlying causes related to poor governance, poor management, community disengagement and poor funding practices affect the long-term success of the Intervention Policy, in addition to certain contextual factors. The Intervention Policy is able to address only some of these factors, and progress across the regions varies partly because of differences in these factors.

The Intervention Policy has been successful in ensuring the ongoing delivery of essential services, maintaining the health and safety of band members, and ensuring the accountability of recipients to INAC. It has been less successful in ensuring the accountability of recipients to their members and building the capacity of recipients to exit and remain out of intervention. Third party management has the most negative consequences in terms of recipient accountability and community perception.

The Intervention Policy has positive unintended impacts in providing justification for political leaders to take tough political decisions, increase the awareness within the community or the council about financial problems and the required solutions, and an increase in support for those solutions. However, there are a number of negative unintended impacts, most notably on the availability of band support funding for governance and administration, and the negative consequences of third party management.

Key success factors to exit the intervention policy or remain out are mainly the willingness and a committed leadership, a stable and competent management, community engagement, and a positive external relationships – as well as access to significant own source revenue.

Efficiency and Economy (Cost-effectiveness)

The cost-effectiveness of the Intervention Policy could not be determined quantitatively because of the lack of comprehensive quantitative data. Similarly, the evaluation team could not compare the cost to other approaches because they also do not have comprehensive quantitative data. However, based on the limited success, it could be said that the policy could be more cost-effective. Costs include INAC's time to develop and deliver the policy, and the cost to recipients of co-managers and third party managers. The major cost drivers are the incidence, level and duration of intervention so that improving the results would decrease the costs. There is also a major cost associated with not intervening in a timely way.

Program Design

The Intervention Policy is not well designed to achieve all of its objectives as capacity development is lacking. The objectives, roles, responsibilities, and accountabilities of the policy are fairly well defined and the policy provides a consistent framework for use by the regions. INAC's Intervention Policy is similar to the intervention approaches of other jurisdictions in terms of objectives, triggers, levels of intervention and remedies, and oversight and monitoring. Other jurisdictions may, however, provide more capacity building and cover the costs of co-managers and third party managers. Provincial governments also have more ex-ante controls over the debt of their subnational entities (municipalities, school boards and hospitals).

Program Delivery

The Intervention Policy is being implemented as planned, with some gaps. The delivery processes for the Intervention Policy are fairly consistent across the regions, with some variations due to different contexts and different capacities of recipients and INAC staff. Some regions have a formal process for prevention and early detection. The performance of co-managers and third party managers varies, with co-managers better able to provide capacity building to recipients than third party managers.

The constraints that recipients face in implementing remedial measures vary; many of the problems require long-term solutions; and the context is constantly changing. INAC may also not have intervened early enough or escalated the level of intervention soon enough, and there are few tools available to enforce compliance by recipients.

The system for capturing the results of the Intervention Policy is not user friendly or useful for reporting and monitoring purposes at a national level. Policy guidance and tools exist at a national level and in some regions; and performance monitoring of implementation is taking place. Communication and coordination with other federal government departments and other key players are not optimized.

INAC is addressing previous audit recommendations through action plans and the development of a revised intervention policy and directive and there are a number of best practices in regions related to different aspects of the policy.

Improvements and Alternatives

The major improvements that have been suggested to the Intervention Policy relate to improved prevention; a revised trigger for financial health; a broader assessment of capacity needs and a more varied capacity building response that responds to some of the underlying governance, management and community causes; and a strengthening of the co-management level of intervention. Alternatives that have been suggested address other underlying causes, including INAC's funding practices and are outside of the scope of the Intervention Policy. Another alternative is a more holistic, long-term approach that addresses a number of underlying causes as well as some of the contextual factors.

9.2 Recommendations

It is recommended that INAC:

1. Implement prevention and early detection strategies to prevent First Nations going into intervention status or escalating to a more serious level of intervention. Activities should include:
   
   
   1. Better identification of financial and governance capacity gaps and needs (linked to the general assessment);
      
      
   2. Better and broader identification of triggers for third party management and co-management;
      
      
   3. Identification of incentives for third party managers to build First Nations capacity, to be written into agreements with third party managers;
      
      
   4. Development of questions to assess properly key success factors to analyze trends in relation to escalation and de-escalation (consider undertaking community surveys to better assess community capacity factors for success); and
      
      
   5. Improve communication and coordination at the national and regional level with other federal government who deliver services and programs to First Nations.
      
      
2. It is recommended that:
   
   
   1. Third party managers be prequalified and assessed against performance criteria;
      
      
   2. INAC audit co-management and third party management arrangement on a risk basis; and
      
      
   3. Revise the third party management agreement to request participation of third party managers in evaluations as well as in audits.
      
      
3. The proposed new Default Prevention and Management Policy should address the design and delivery gaps of the current Intervention policy by:
   
   
   1. Implementing national tools and formalized processes in the assessment of First Nations;
      
      
   2. Clarifying FSO roles and responsibilities, developing job descriptions that identify competencies and knowledge needed, and identifying training to meet requirements; and
      
      
   3. Clearly communicate the new policy and assessment processes to stakeholders and First Nations.
      
      
4. Develop and implement better monitoring and reporting systems that involve:
   
   
   1. A performance measurement strategy to allow for meaningful reporting through quarterly progress reports and the Departmental Performance Report. Indicators such as duration, incidence and level of intervention, and level of implementation of Remedial Management Plan could be considered;
      
       
   2. A cost tracking system at the Headquarters and regional level to capture cost data in order to measure cost effectiveness and inform future decisions; and
      
      
   3. Re-design the intervention policy module in FNITP that will involve a revision of the inputs, processing and output (reports) of the system at the regional and national level in order to make it more user-friendly, reliable and to inform performance measurement strategy to be developed in line with the new policy.

Revised Evaluation Matrix [Note 47]

* For other government departments, the response to these two questions relate to their own mechanisms and tools for performance monitoring and guidance.

The charts depicts the draft intervention policy logic model. The model illustrates the links between the inputs, activities, outputs, short-, medium- and long-term outcomes, and ultimate outcomes. Its purpose is to show the expected effects of the Policy's activities and outputs.

Inputs

The inputs include the A-Based/B-Based budget, human resources, funding arrangement agreements (T&C), the Financial Administration Act and Regulations, national policies, compliance guidelines and procedures, and procurement guidelines. All of the activities in the logic model use the same inputs.

Activities

There are three types of activities:

1. Service procurement – identifying and selecting co-manager or third-party manager;
   
   
2. Compliance monitoring – determining whether the recipient should be placed under the Intervention Policy (triggers, T&C), and assessing the level of intervention required (performance review model);
   
   
3. Capacity-building – delivering the Professional and Institutional Development program (capacity development initiative).

Outputs

The activities have four outputs. The outputs and inputs are linked as follows:

• Service procurement produces output 1 – Call-ups.
  
  
• Compliance monitoring produces outputs 2 – Co-management/third-party management agreement, and 3 – Written notification of intervention.
  
  
• Capacity-building produces output 4 – Capacity development project for community under intervention.

Short-term (or immediate) outcomes

Three short-term outcomes are expected:

1. Accountability and program services delivery maintained during problem situation with co-manager or third-party manager reporting;
   
   
2. Remedial management plan developed;
   
   
3. Capacity-building strategy developed to have recipients administer own funding

More specifically:

• Short-term outcome 1 – Accountability and program services delivery maintained during problem situation with co-manager or third-party manager reporting, and outcome 3 - Capacity-building strategy developed to have recipients administer own funding, are directly linked to output 1 – Call-ups, output 2 – Co-management/third-party management agreement, and output 4 – Capacity development project for community under intervention.
  
  
• Short-term outcome 2 – Remedial management plan developed, is directly linked to output 3 – Written notification of intervention. Note that the remedial plan developed is linked to the capacity-building activity.

Medium-term (intermediate) outcomes

The short-term outcomes lead to two medium-term outcomes.

The medium-term outcome Terms and conditions of funding agreement commitment met is the result of achieving short-term outcome 1 - Accountability and program services delivery maintained during problem situation with co-manager or third-party manager reporting.

The medium-term outcome Recipient capacity (skills, abilities and knowledge) to manage finances, to be accountable and to prevent future intervention enhanced (performance improvement) is the result of achieving short-term outcome 2 – Remedial management plan developed, and short-term outcome 3 – Capacity-building strategy developed to have recipients administer own funding.

Long-term outcomes

The medium-term outcomes lead to three long-term outcomes.

The long-term outcome Effective and responsibly managed and operated programs and services funded by INAC is the result of achieving medium-term outcome Terms and conditions of funding agreement commitment met.

The two long-term outcomes Intervention level de-escalated and Number of recipients under intervention reduced are the result of achieving the medium-term outcome Enhanced recipient capacity (skills, abilities and knowledge) and Capacity-building strategy developed to have recipients administer own funding (performance improvement).

Ultimate Outcomes

Achieving the three long-term results leads to the ultimate outcome: Good governance and effective institutions.

Table 1 - Number of First Nations Under Intervention

Source: Intervention data extracted from FNITP as of March 31st in each year, and validated by EPMRB with INAC's Regional Offices

Table 2 - Level of Intervention As A Proportion of the Total Number of First Nations Under Intervention

Note: Because of rounding, percentages may not add up to 100%

Table 3 - Incidence and Level of Intervention by Region, as of March 31, 2010

Source: Intervention data extracted from FNITP as of March 31, 2010.

Table 4 - Number Of Other Organizations Under Intervention

Source: Intervention data extracted from FNITP as of March 31st in each year, and validated by EPMRB with INAC's Regional Offices

Table 8 - Trend in the Number of First Nations with a Cumulative Operating Deficit > 8% and the Size of the Deficit

Source: FNITP financial data.
Note: This report reflects information entered into the FNITP Statement of Revenue and Expenditure where the Audit Reporting Requirement "Approved" has been completed.

As per evaluation and research

General

• Number of recipient with total deficit per region, per year
  
  
• Number of Audit required, received and % reviewed National

Leadership

• Stability of leadership (# years Chief and Band Council in duty)
  
  
• Community plan approved by Band Council and shared with community

Deficit/ Indebtedness/ Financial Ratio

• Number of Recipients, FN, Tribal Council and Organisations with a cumulative deficit over eight percent
  
  
• Number of recipient under IP with total deficit per region, per year
  
  
• Level of deficit of a recipient per year
  
  
• Number of Audit (Opinion) qualified, unqualified, denial and unspecified per year, per region and national

Remedial Management Plans

• Number of RMP implemented each fiscal year
  
   
• Number of RMP per level of intervention per year
  
  
• Number of RMP approved/refused per region, per year
  
  
• % of Recipients requiring a RMP (National picture)
  
  
• Number RMP required and in place (Regional picture)

Intervention Policy

• Number and % of FN, Tribal Council and Organisations placed under intervention per level/ per year
  
  
• Number of Intervention completed successfully (de-escalation) per year, per region
  
  
• Number of Intervention renewed for an other period per region per year
  
  
• Cumulative number of FN, Tribal Council and Organisations under intervention
  
  
• Number of FN (%) remove from intervention (out of Intervention Policy) per year
  
  
• Number of FN, Tribal Council and Organisations with an increase of level of intervention
  
  
• Duration of Intervention per level / per recipient / Organizations

Operations

• Briefing note sent to Deputy Minister per year (Co/TPM)
  
  
• Number of training sessions, workshops, networking for: FSO, managers, band officers per region / per year
  
  
• Number of visits of FNs under intervention by FSO per year