Archived - Evaluation of the Emergency Management Assistance Program

The evaluation of the Emergency Management Assistance Program (EMAP) was required as part of the Transfer Payment Policy and will support renewal of contribution authorities. It provides evidence-based findings and conclusions regarding the relevance and the performance of the program.

The Department of Indian and Northern Affairs Canada (INAC) established EMAP to assist First Nations communities living on reserves in managing emergencies. The program covers all four pillars of emergency management: mitigation, preparedness, response, and recovery. In addition, the program may provide assistance for search and recovery activities related to missing persons. In recent years, the range of activities undertaken as part of EMAP has broadened to include health-related issues and civil unrest.

The methodology used to conduct this evaluation included a review of departmental policy and program documents; a literature review on theories of emergency management and how it is structured, delivered, and success measured in other jurisdictions to identify best practices, alternative approaches to design and delivery and possible funding options. The methodology also used interviews with a wide range of key informants from federal, provincial, and local authorities; case studies in four provinces; and focus groups involving representatives from First Nations communities. A Working Group and an Advisory Committee provided guidance and feedback throughout the evaluation process.

Key findings and conclusions from the evaluation are as follows:

Relevance

This evaluation confirms the need for EMAP. There is an overall trend towards increased frequency and intensity of emergencies throughout Canada and First Nations communities are considered "high risk" when it comes to disasters due to their small size, social vulnerability and remoteness and isolation. Many First Nations do not have updated emergency management plans in place leaving them unprepared when emergency events occur.

EMAP is the central tool available to INAC to ensure that required assistance services are provided to First Nations communities facing emergencies. However, the Program, as it is currently designed and delivered, does not meet the needs of First Nations communities in the areas of mitigation, preparedness and recovery.

It should be noted that program authorities and objectives are largely aligned with government-wide priorities as documented in the 2007 Emergency Management Act (EMA), as all are based on the four-pillar approach to emergency management. However, the current program objectives do not appear to capture all departmental priorities. In recent years, INAC has paid increased attention to civil unrest as part of EMAP. While not strictly defined as an emergency in itself, and events frequently occurring off reserve, civil unrest has the potential to erupt into a situation involving emergency services and First Nations communities. Current program authority does not include these types of activities (other than search and recovery activities).

EMAP objectives also do not reference Departmental responsibilities in emergency management in the territories. The actual responsibility of INAC when it comes to emergency management in the North has yet to be clearly established.

One final area of responsibility that is not currently reflected in EMAP's outcomes or authorities is the Department's involvement in emergency activities that are outside INAC's jurisdiction such as pandemic planning. INAC dedicated significant resources to a Health Canada process to have pandemic plans in First Nations communities.

Performance

At the national level, INAC has established the Emergency and Issue Management Directorate to coordinate the program's activities, and support regional offices and other stakeholders as required. INAC's regional offices are collaborating with provincial and territorial emergency management organizations, as well as with Aboriginal organizations. There are formal agreements in place in approximately half of the jurisdictions, and negotiations are ongoing elsewhere.

• Program delivery structure

EMAP's delivery structure for response and some aspects of recovery is sound as the program essentially supports provincial emergency management organizations that can offer the expertise and resources needed in the area of emergency management. However, the current program delivery mechanisms and structure do not provide the required framework to pursue an all hazards approach to emergency management as required by the Emergency Management Act. There is essentially no structure in place to deal with mitigation-related issues. Various approaches are currently used to support preparedness activities, and while flexibility in this area is required, the current program delivery structure does not provide a clear understanding of the scope of EMAP activities related to preparedness.

• Distribution of roles and responsibilities

This evaluation points to a lack of defined roles and responsibilities. In particular, INAC's roles and responsibilities in delivering an all hazards approach to emergency management, especially in the areas of mitigation, preparedness and recovery have not been clearly documented resulting in inconsistencies in programming across Canada.

At the local level, the distribution of roles and responsibilities becomes more complex. Depending on the community involved and the nature of emergencies occurring, there can be a wide range of stakeholders involved. This evaluation indicates that ambiguities do exist in that regard. In particular, some First Nations communities remain uncertain as to the extent of their responsibility in dealing with emergencies, from declaring the emergency itself to carrying out the required activities under the four pillars of emergency management.

The fact that the Department has extended the scope of emergency management activities to include issues such as civil unrest also adds to the complexity associated with the distribution of roles and responsibilities.

Moreover, the precise role of the Department in an all-hazards approach to emergency management in the three northern territories is not well defined, nor are the department's roles and responsibilities with respect to emergency-related activities that fall within the responsibility of another department or jurisdiction (such as health issues).

It is important to note that despite these ambiguities in three of the four pillars of emergency management, response services have not been delayed. This evaluation indicates that when faced with an emergency, local stakeholders will proceed and provide the required assistance. Any unresolved administrative issue is addressed after the fact.

• Current funding structure

EMAP's current funding structure is problematic. It does not provide the required financial base to pursue all of the program's goals and objectives. It also creates inefficiencies in providing the required financial assistance needed to allow INAC to fulfill its legal obligations.

At the time of the evaluation, it was practically impossible to assemble a complete financial picture of EMAP. The requirement to proceed with a new Treasury Board submission every time significant resources are required has triggered unintended negative impacts. In some cases, the Directorate or regional offices need to reallocate funding from other programs to cover some costs. The same situation may occur with band councils. In turn, the incomplete financial picture creates challenges in measuring performance and appropriately documenting the achievements of the program.

Experiences in other settings or jurisdictions confirm that there are a number of options INAC could pursue to improve EMAP's funding structure. Such changes are needed if the program is to successfully pursue program objectives relating to the four pillars of emergency management.

• Program results

The Emergency and Issue Management Directorate is currently collecting only a few indicators related to the number of agreements in place and the number of emergency management plans in place in communities. These indicators measure only a portion of the work being undertaken and do not provide a very useful measure on their own as there are indications that the plans in place are of poor quality, are out dated and have not been tested. Aside from these few indicators, there was no procedure in place to measure and document the program's results, best practices and lessons learned. The Directorate has established founding blocks, such as the development of a departmental emergency management plan, and processes to work and communicate with regional offices. On that basis, the Directorate expects to develop a performance measurement strategy.

At the time of the evaluation, the program's outcomes were concentrated in the area of response and recovery. Despite the lack of agreements in some regions, the Department has succeeded in coordinating and securing the collaboration of emergency management stakeholders to adequately respond to emergencies affecting First Nations communities. However, there were some comments that recovery is focussed primarily on returning evacuees to their communities and restoring damaged infrastructure. It was felt by some that more could be done to help communities deal with the trauma of emergencies and restoring governance following an event.

The program's outcomes in the area of preparedness are more limited. The Department has provided assistance to some First Nations communities in developing plans and providing training. However, evaluation findings indicate that the need for support in this area far exceeds what the program has offered to date. Also, the evaluation has not documented any program results in the area of mitigation, although infrastructure work continues to be among INAC's priorities.

Recommendation 1:  Roles and responsibilities

It is recommended that INAC clarify the roles and responsibilities of the Department as they relate to emergency management. This process should consider the current environment of emergency management, specifically the implications of the 2007 Emergency Management Act. To do so, the Department must define relationships with all external stakeholders and put in place the appropriate governance structures and agreements to ensure fulfillment of responsibilities related to emergency management. All aspects of emergency management should be considered in this process, with particular emphasis on the following areas:

1. The precise role of the Department in emergency management in the three northern territories.
   
   
2. The precise role of the Department with respect to emergencies that fall within the responsibility of another department or jurisdiction (such as health issues and civil unrest).
   
   
3. The program delivery mechanisms and structure relating to the four pillars of emergency management: mitigation, preparedness, response and recovery activities.
   
   
4. Horizontal engagement of other relevant INAC programs that have a potential to contribute to an all-hazards approach to emergency management, such as capital infrastructure in mitigation projects or land claims in civil unrest issues.
   
   
5. The precise role of First Nations communities in emergency management.

Recommendation 2:  Program funding structure

It is recommended that INAC consider a revised funding structure, to alleviate the impact on regions, other program areas, and communities and provide a secure funding base for the Department's emergency activities. To facilitate this transition, INAC should document existing INAC funding for emergency management programming and develop forecasts for future expenses relating to an all hazards approach to emergency management.

INAC should also identify appropriate resources in alignment with the Department's roles and responsibilities. Specifically, ensuring that the department has the ability to provide preparedness and mitigation services in accordance with Departmental obligations under the EMA.

Recommendation 3:  Performance measurement

It is recommended that INAC develop a Performance Measurement Strategy for emergency management programming in consultation with the Evaluation Performance Measurement and Review Branch and in accordance with the principles of the new Treasury Board Policy and Directive on Evaluation.

The Final Report for the Evaluation of the Emergency Management Assistance Program was approved by the Evaluation, Performance Measurement and Review Committee on February 24, 2010.

1.1 Overview

This document constitutes the final report of the evaluation of the Emergency Management Assistance Program (EMAP). The primary purpose of this program is to allow the federal government to assist First Nations communities living on reserve and, under some circumstances, Canadians living north of the 60th parallel, to cope with emergencies that significantly affect their communities. The Evaluation, Performance Measurement and Review Branch (part of the Audit and Evaluation Sector) initiated this evaluation in June 2009. The Branch contracted the services of PRA Inc. to provide assistance during all stages of the evaluation process.

This evaluation is required as part of the Transfer Payment Policy and is expected to support the renewal of contribution authorities associated with EMAP, which are due to expire at the end of March 2010. The evaluation is expected to provide evidence-based conclusions regarding relevance and performance (efficiency, effectiveness, and alternatives), particularly with respect to the financing, design, and delivery of EMAP.

This report is divided into five sections. This introduction provides an overview of the evaluation process, along with a description of EMAP. Section 2 describes the methodology associated with the study. It includes a description of the scope and timing of the evaluation, a summary of the evaluation issues and questions addressed in this report, along with a description of the various methods used to collect evaluation data and findings. Section 2 also provides an overview of the roles, responsibilities, and quality assurance used to support this study. Section 3 and 4 include the most critical information relating to the evaluation of EMAP, as they summarize all findings that have emerged during the data collection process. Section 3 specifically explores the relevance of EMAP, while Section 4 focuses on the actual performance of the program. Finally, Section 5 provides conclusions and recommendations as applicable.

1.2 Program Profile

1.2.1 Background and Description

An emergency is a circumstantial notion. It typically refers to situations where a community is overwhelmed by unforeseen or extraordinary events that it can no longer manage using its normally available resources and capacity. Public Safety Canada offers this definition of emergencies:

"[A] social phenomenon that results when a hazard intersects with a vulnerable community in a way that exceeds or overwhelms the community's ability to cope and may cause serious harm to the safety, health, welfare, property or environment of people; may be triggered by a naturally occurring phenomenon which has its origins within the geophysical or biological environment or by human action or error, whether malicious or unintentional, including technological failures, accidents and terrorist acts."[Note 1]

The degree to which a community is impacted by an emergency event will depend on the local context. What may be an inconvenience in a large urban centre may well turn out to be an emergency in a small and remote community. It is well established that the size of a community, and its relative isolation, can have a direct impact on its resiliency when faced with an emergency.[Note 2] Since First Nations communities are often small and remote, they are particularly vulnerable when faced with unforeseen events. This is particularly significant in the current global context, where the frequency and severity of emergencies are increasing.[Note 3]

The nature and range of emergencies that may affect a community contribute to the complexity of emergency management. As it relates specifically to First Nations communities living on reserve, the list of emergencies with which they may be confronted includes both naturally occurring and human-induced emergencies:

• Natural emergencies include (but are not limited to) wildfires, floods, major ice jams, avalanches, tornadoes, landslides, periods of intense cold weather, power blackouts, and severe storms;
  
  
• Human-induced emergencies include (but are not limited to) bomb scares, fuel tank accidents, oil spills, gas leaks, train derailments, consequence management supporting pandemic and communicable disease outbreaks (e.g., H1N1), civil unrest, and lost persons cases.

Another factor that contributes to the complexity of emergency management is the range of emergency management partners that need to be involved in the successful management of actual or potential emergencies. The list of these organizations includes planners, responders, recovery and financial personnel. Firefighters, police services, health care providers, social services providers, band councils, mutual aid partners, emergency management organizations, and provincial and federal governments are among the stakeholders that need to efficiently coordinate their actions and decisions so that an emergency can be successfully managed. In any circumstance, this would be a remarkable challenge: in a period of crisis, this is even more testing.

Over the past 20 years, the specific role of INAC in managing emergencies on reserve and north of the 60th parallel has become increasingly structured. The Department has a long-standing involvement, dating back to the 1960s, in dealing, to some extent, with emergencies relating to these communities. The passing of the Emergency Preparedness Act in 1988 provided somewhat clearer parameters for defining INAC's role. The act requires every Minister be accountable to Parliament to identify "civil emergency contingencies that are within or related to the Minister's area of accountability" and to develop a civil emergency plan.

During this period, EMAP has emerged in an incremental fashion. The federal government established the program's first building block in 1988, when it provided INAC with the authority and resources to support fire suppression services when forest fires (or similar incidents) affected First Nations communities living on reserve. It also allowed the Department to provide financial assistance to First Nations for search and recovery activities related to lost persons, based on compassionate grounds after local authority has called off search and rescue for the continuation of search activities.

The federal government established the program's second building block in 2004, when it expanded the 1988 departmental authority to include activities and services relating more broadly to emergency management. Not only is the Department in a position to support fire suppression services, as well as search and recovery activities, but it also gained the authority to support a range of activities related to mitigation, preparedness, response and recovery. While the Department gained that authority, it did not secure incremental funding on a permanent basis (A-base), to support this expanded mandate. Rather, the federal government has been providing funding on an ad hoc basis (supplementary estimates). This funding aspect is further discussed in subsection 1.2.4. The federal government approved the Terms and Conditions that set the parameters for the current EMAP mandate for a five-year period, from 2005–2006 to 2009–2010.

The passing of the federal Emergency Management Act in 2007 has provided further clarifications on the roles and responsibilities of all federal ministers. First, the new act provides a definition of emergency management, which includes the "prevention and mitigation of, preparedness for, response to and recovery from emergencies." These are the well-established four pillars of emergency management, which are further explored in subsection 3.1 of this report. The act requires each minister accountable to Parliament to identify the risks "that are within or related to his or her area of responsibility" and, on that basis, to prepare, maintain, and test emergency plans. Among other things, these plans must include:

• any programs, arrangements, or other measures to assist provincial governments and, through the provincial governments, local authorities;
  
  
• any federal-provincial regional plans;
  
  
• any programs, arrangements, or other measures to provide for the continuity of the operations of the government institution in the event of an emergency.[Note 4]

It is important to note that the current authority associated with EMAP covers activities occurring on First Nations reserves. EMAP's current Terms and Conditions do not technically cover activities north of the 60th parallel, other than those occurring in the two reserves located in the Northwest Territories.

1.2.2 Program Logic

This subsection describes EMAP's program theory. Simply put, the purpose of this subsection is to better understand what the program is expected to do and what it is expected to achieve. Whether these activities have occurred or these results have been achieved is discussed in Section 3 (evaluation findings). Here, the goal is to understand the program as it was initially designed, and lay out the set of assumptions that link its activities with its expected outcomes. A visual summary of the program's logic model is included in this report as Figure 1, on page 7.

This Logic Model was created as part of this evaluation and was shared with EMAP staff participating in key informant interviews for comment. It is important to emphasize at this juncture that the program theory outlined below varies from the actual activities, outputs and outcomes of the EMAP. Section 4.1 outlines the significant gaps in EMAP's coverage of the four pillars.

Program Objectives

The fundamental purpose of EMAP is to protect First Nations communities living on reserve when they face unforeseen emergency events that they can no longer handle using their normally available resources. This includes the protection of both individuals themselves and their overall community infrastructure. More specifically, the program pursues three objectives:

• To protect the health and safety of First Nations members when they face natural disasters and damages or destruction of community infrastructure and houses, by natural disaster or accident;
  
  
• To assist in the remediation of essential infrastructure and houses through timely assessment of emergency needs and the facilitation of an appropriate emergency response from other areas of INAC;
  
  
• To support communities, on a compassionate basis, through the continuation of search and recovery activities associated with lost persons beyond the expected survival period after search and rescue authority has called off search.

Program Activities and Outputs

To pursue these objectives, the Department has authority to undertake a number of activities and provide financial assistance as required. These program activities can be grouped along the four pillars of emergency management.

Mitigation: These activities may provide assistance to First Nations communities to identify systemic vulnerabilities. This assessment process may be undertaken by the community itself, or may be done in collaboration with an external emergency management organization. INAC regional offices may also work with First Nations communities to identify capital projects that could be included in the departmental long-term capital plan. It is important to note that EMAP does not directly fund capital projects. What comes out of mitigation activities may include risk or impact assessments, training, or the inclusion of specific mitigation-related projects in the departmental capital plan.

Preparedness: Under this heading, the program may provide assistance to First Nations communities to undertake a number of activities related to emergency management planning. INAC regional offices may negotiate various types of agreements with emergency management or other organizations to assist First Nations communities in developing, updating, and testing emergency plans. As a result, these activities may lead to the signing of agreements, training tools and resources, and emergency plans.

Response: In the event that an emergency unfolds, EMAP may provide assistance to First Nations communities to protect individuals and community infrastructure. INAC regional offices typically work with emergency management organizations to ensure that any required evacuations, response activities (such as providing alternative sources of energy), or other measures are taken to address the emergency at hand. In some cases, INAC regional offices may provide direct financial assistance to First Nations communities to respond to a specific emergency. To support these activities, INAC may sign agreements with response organizations or assist in the coordination of activities.

Recovery: Depending on the nature of emergencies, recovery activities may include the repatriation of evacuated families and individuals, repairs to damaged infrastructure, and other related measures needed to bring the community back to pre-emergency conditions. Again, these activities may be undertaken by an emergency management organization or by the community itself via capital projects. As a result, agreements may be signed with an emergency management organization and financial payments may be made directly to band councils.

Expected Outcomes

These various activities are expected to enhance the resiliency of First Nations communities and to provide comparable emergency management services to First Nations communities as found in non-Aboriginal communities in similar circumstances. More specifically, activities undertaken through EMAP are expected to contribute to the following immediate and intermediate outcomes:

• First Nations communities undertake mitigation projects that are required to address their systemic vulnerabilities.
  
  
• First Nations communities enhance their capacity to effectively plan for emergencies and to collaborate with other partners.
  
  
• Efficient and effective responses to emergencies affecting First Nations communities are implemented and relative normalcy is restored following the emergency. This, in turn, is expected to minimize the social and economic impacts of emergencies on First Nations communities.

Ultimately, the program is expected to contribute to the broader departmental goal of having First Nations benefit from their lands, resources, and environment on a >sustainable basis.

Logic Model Emergency Management Assistance Program (EMAP)

Figure 1

Figure 1 shows the logic model for the Emergency Management Assistance Program. A logic model is a chain of results connecting activities to their final outcomes and identifying the steps in between that demonstrate the progress towards each goal. In this logic model, the progression from activities to outputs and then outcomes is displayed vertically with a box from each row linked to the one above it using arrows, demonstrating how the program activities indicated in the logic model lead to specific outputs which in turn lead to immediate outcomes, intermediate outcomes, and the final outcome.

In total there are four columns, one for each pillar of program activity, and six rows, with the first row listing the four main pillars of program activities. The second row details the specific activities undertaken for each pillar, the third row lists outputs, and the fourth, fifth and sixth rows state the immediate, intermediate and final outcomes respectively.

At the top of the chart, in the first row, the four main pillars of program activity are each represented by their own box. In order from left to right we have: mitigation, preparedness, response, and recovery. Each of these boxes is connected to a box below, in the second row, which is then connected to the one below it and so on, continuing down to the last row, forming four columns.

In the first column under mitigation, the box in the second row lists the following activities: identify possible emergencies; identify vulnerable peoples and infrastructure; identify capital projects for inclusion in INACs long term capital plan. This then links to the box in the third row below indicating the outputs for these activities: risk and impact assessments; workshops on risk mitigation; mitigation projects in capital plan. In the next row, the box for immediate outcomes associated with mitigation states: mitigation projects under way in communities, followed by the box for intermediate outcomes in the fifth row which states: community vulnerabilities addressed.

In the second column under preparedness, the activities listed in the second row are: identify eligible costs for reimbursement; negotiate agreements with provincial/territorial BMOs; promote community engagement in emergency management; training community level and INAC Emergency Management officials; develop and implement emergency pans and procedures. The outputs which follow in the third row state: agreements with BMOs; information sessions; training tools, materials, sessions; emergency plans (national, regional, and community), policies, and procedures; continuous monitoring systems. The corresponding immediate outcome in the fourth row is: capacity building in Aboriginal communities, and the intermediate outcome in the fifth row is: partners prepared for possible emergency events.

In the third column under response, the stated activities are: engage partner organizations; coordinate response activities; establish chain of command; evacuation; monitor and report on emergency situations; identify source of INAC funds for services rendered. The box detailing outputs in the third row states: emergency coordinating mechanisms; finance emergency response effort, and the immediate outcome below states: efficient and effective response to emergency events. The intermediate outcome listed in the fifth row reads: social and economic impact of emergency events minimized.

The fourth and last column, found on the far right of the page, corresponds to the activity pillar for recovery. In the second row, the box corresponding to this pillar lists the following activities: repatriation of evacuated individuals; identify capital projects for inclusion in INACs long term capital plan; review best practices; prepare TB submissions to recover funds. In the next row, the outputs identified are: communities repatriated; recovery projects in INAC capital plan; incident review (after action report); TB submissions. The immediate outcome, indicated in the fourth row states: return to normalcy following an emergency event. At this point, when we get to the fifth row listing the intermediate outcomes, the columns for response and recovery both connect to the same box. Although both columns have their own separate boxes for each activity, output, and immediate outcome, they now share the intermediate outcome of: social and economic impact of emergency events minimized.

Finally the sixth and last row contains one box which is connected to each of the four columns. At this point all of the four columns share the final outcome for the program which is: First Nations and Inuit benefit from their lands, resources, and environment on a sustainable basis.

1.2.3 Program Management, Key Stakeholders, and Beneficiaries

INAC's Emergency and Issue Management Directorate is responsible for the overall management of EMAP. The Directorate provides both policy and operational support for the ongoing implementation and management of the program.

INAC's regional offices also play a predominant role in the ongoing management of EMAP. These regional offices work directly with emergency management organizations, Aboriginal organizations, and band councils. At the time of this evaluation, all provincial regional offices had at least one position dedicated to emergency management. Individuals in these positions liaise with all key stakeholders involved in emergency management, particularly in the areas of preparedness (emergency management planning), response, and recovery. In the three territories, responsibilities for emergency management are added to existing positions.

INAC's Emergency Management Goverance Structure

Figure 2

Figure 2 depicts a chart showing the governance and decision making infrastructure within INAC and the Government of Canada as a whole in relation to emergency management. At the top are two boxes with the one on the left indicating INAC and the right showing the Government of Canada.

Under Government of Canada, at the bottom right corner, is a box for the Government Operations Centre. This box then has an arrow which points up to another box called ADM Emergency Management Committee which in turn has an arrow up to DM National Security Committee which then has an arrow pointing up to a box titled Cabinet Operations Committee.

Under INAC, at the bottom and to the far left there is a box for Regional Operations Centres. A double ended/reverse arrow then points to a box to the right for HQ Operations Centre. This box then has an arrow pointing up to Director General, Operations and Planning Support which then has an arrow pointing up to Senior ADM, Regional Operations. That box points to another on the left called Operations Committee as well as to the one above it titled Deputy Minister. The Deputy Minister box then has an arrow pointing up to the top box titled Minister.

There are also arrow links between some of the INAC and Government of Canada boxes. Specifically, HQ Operations Centre has an arrow leading to Government Operations Centre; Senior ADM, Regional Operations links to ADM Emergency Management Committee; the Deputy Minister connects with the DM National Security Committee; and, the Minister box has an arrow connecting it with Cabinet Operations Committee.

The work of INAC in emergency management is part of a much broader web of decision-making infrastructure within the Department itself, and the government of Canada as a whole. As illustrated in Figure 2, the Department has established an informal Operations Centre in its headquarters for normal operations that can be escalated to a fully functional emergency operations centre for large emergencies, all of which is directly supported by the Directorate. The Department also has an Operations Committee, where several senior managers coordinate their respective activities in emergency management. Within the government itself, there are a number of decision-making bodies that range from an Operations Centre, up to the Cabinet Operations Committee.

The ultimate beneficiaries of EMAP are First Nations communities and specific individuals and families within these communities that are affected by emergencies. From an administrative point of view, however, the program does not provide direct funding to individuals and families. Instead, the funding is provided directly to those organizations that are providing emergency management services. The list of these organizations may include:

• Emergency management organizations
  
  
• Aboriginal firefighters association (in BC and MB)
  
  
• Provincial governments
  
  
• Band councils

1.2.4 Program Resources

EMAP's funding structure is both unusual and complex. The set of activities undertaken by the program is funded through a variety of sources, some of which are specifically dedicated to EMAP, while others result from internal reallocations. This subsection describes these various sources of funding currently used to support EMAP activities.

The Formal A-base Funding

The federal government provides ongoing funding to EMAP (A-base funding) in the amount of $10.7 million per year (as of fiscal year 2008–2009). This amount includes $9.5 million in transfer payment resources (contributions), which are specifically assigned to fire suppression activities. As indicated in subsection 1.2.1 of this report, these resources were associated with the authority given to INAC in 1988 to support fire suppression activities affecting First Nations communities living on reserve. An additional $1.2 million is assigned to operating expenditures to cover some of the departmental internal costs associated with emergency management.

For any other financial resources needed to support EMAP activities (particularly in the areas of preparedness, response, and recovery), the Department is left with essentially two options. It may decide to reallocate some existing resources assigned to other programs (capital projects, for instance) to fund EMAP activities. It may also decide that reallocating resources is no longer feasible or appropriate and, on that basis, it may seek supplementary funding.

Supplementary Funding

Over the past five years, since costs associated with emergencies affecting First Nations communities have far exceeded the initial $9.5 million available for fire suppression, the Department has had to turn to the Treasury Board to obtain supplementary funding. As indicated in Table 1, the federal government has allocated $113.7 million over a five-year period in additional funding to EMAP. These funding requests are typically event-based, as they cover costs associated with specific emergencies.

In addition to these amounts, the federal government has allocated resources to address emergency related expenditures using the Capital Facilities and Maintenance programs. These expenditures were typically allocated to repair damaged infrastructures or to address rising fuel costs.

In each case, the Directorate must prepare a Treasury Board submission on behalf of INAC's Minister. Because of the requirements associated with Treasury Board submissions, obtaining these additional resources may require a fair amount of time. Meanwhile, not knowing what the Treasury Board decision will be, the Department (regional offices, in particular, or band councils themselves), have to cash manage the expenditures that have already been committed.

Other Funding Contributing to Emergency Management

There are at least two additional sources of funding that support EMAP-related activities. The first of these is A-base funding allocated to the Department's headquarters or regional offices, which is redirected to support emergency management activities. As previously mentioned in this subsection, the Directorate or regional offices may decide that pursuing supplementary funding through Treasury Board submissions is not the most appropriate strategy for covering the costs related to a specific incident. These decisions, in turn, will affect other programs and activities.

The second source of funding is a program administered by Public Safety Canada called the Disaster Financial Assistance Arrangements (DFAA) program. This provides funding to provinces and territories for emergencies on reserve on the rare occasion when an emergency affects a large territory that includes one or more First Nations reserves. Once certain criteria are met (based on the total amount of eligible expenditures incurred to address an emergency), the DFAA program reimburses any response and recovery expenses related to activities on First Nations reserves that meet the program's guidelines. In the absence of the DFAA program, it can be expected that EMAP would need to cover these costs.

2.1 Evaluation Scope and Timing

This evaluation focuses on EMAP activities that occurred during a five-year period, from 2004–2005 to 2008–2009. The Evaluation, Performance Measurement and Review Committee approved the Terms of Reference for this evaluation in June 2009. The evaluation team conducted the field work between August 2009 and January 2010.

2.2 Evaluation Issues and Questions

In accordance with Treasury Board policy on evaluation, the EMAP evaluation addresses a number of evaluation questions relating to the relevance and performance of the program. Table 2 includes all of the evaluation issues and questions addressed in this report.

2.3 Evaluation Methods

The EMAP evaluation rests on evidence-based findings that were collected using a number of research methods. This subsection describes these various methods, along with a discussion on the rationale for these methodological choices, and the challenges that were faced during the study.

2.3.1 Data Sources

Five data sources were used in support of the EMAP evaluation:

Document and Data Review

The document and data review involved a thorough review of program files, background documents, agreements, performance measurement materials, and further documentation regarding the role of INAC and related stakeholders in dealing with emergency management. This review covered issues relating to First Nations communities living on reserve and to federal land north of the 60th parallel. The document and data review formed a significant source of information for this evaluation, as it addressed all evaluation issues and questions.

Literature Review

The literature review focussed on two broad areas. Firstly, it examined current theories of emergency management to inform the relevance and need of EMAP. These findings provide some of the context for assessing the program rationale.

Secondly, the literature review examined how emergency management programs are structured and delivered in other jurisdictions, including:

• Models of emergency management from other countries, especially approaches tailored to Aboriginal populations. Australia, the USA, and New Zealand were identified as possible countries for study.
  
  
• Emergency management in other Canadian government departments and other jurisdictions (i.e., provinces and municipalities).
  
  
• Other emergency management organizations, with a focus on how emergency management is planned and structured in other countries, other departments, and other jurisdictions; objectives and outcomes; how success is measured; and how emergency management is funded.

The second area of the literature review helped to identify best practices relating to program design and delivery and funding structures. Alternative approaches related to management and performance measurement in other departments, jurisdictions, and organizations allowed for a comparative analysis with EMAP and informed program design and delivery, performance measurement, and funding options.

This review relied on primary (government policies, legislation, and acts) and secondary (program descriptive reports, and academic journals and publications) sources of data.

Key Informant Interviews

In-depth key informant interviews were used to investigate each evaluation issue and question. At least six distinct stakeholder groups were identified to be interviewed in order to capture a diverse range of perspectives on evaluation questions and issues.

A total of 32 interviews were conducted with individuals from the following groups:

• INAC senior management (in regions and headquarters) (n=2)
  
  
• EMAP officials (headquarters and regions) (n=11)
  
  
• Provincial and territorial governments' emergency management organizations (n=7)
  
  
• Representatives of other emergency management organizations outside INAC (n=3)
  
  
• Aboriginal organizations (n=6)
  
  
• Experts in the field of emergency management (n=3)

Before scheduling interviews, the Evaluation, Performance Measurement and Review Branch emailed key informants an introductory letter that described the objectives of the evaluation and explained that PRA Inc. would contact them to schedule an interview. In most cases, interviews were conducted by telephone. Interviews were conducted in key informants' preferred official language. Prior to conducting the interview, key informants were provided with an interview guide so that they could offer thoroughly considered responses. Separate interview guides were prepared for each category of key informant. Key informants were assured anonymity in the Final Report.

Case Studies

A total of four case studies were performed in order to review existing EMAP operations within Aboriginal communities. In three cases, site visits were conducted to allow for the close examination of evaluation issues. The focus of these visits was to examine EMAP's role and experience at each site.

In close collaboration with INAC's regional offices, potential communities were identified and a letter from INAC was sent formally inviting them to participate in the process. The selection of sites was based on the following criteria:

• Size of the community (including at least one small, medium, and large community)
  
  
• Must have responded to an emergency event during the 2004/05 to 2008/09 period
  
  
• Variance in the type of emergency event responded to (i.e., flood, fire, health risk, civil issue, etc.)

A case study template was designed to systematically record information for each site. Interviews with relevant stakeholders such as provincial representatives, regional emergency management officials, and community representatives were performed at each site. A short case study report (approximately five to seven pages) was drafted for each of the site visits.

Focus Groups

A total of five focus groups were employed to gather views and insights from community representatives located in various regions of the country. These focus groups investigated evaluation issues related to program relevance, effectiveness, efficiency, and alternatives. A total of 16 individuals from First Nations communities participated in these focus groups.

2.3.2 Considerations, Strengths, and Limitations

The methodology used for the evaluation of EMAP was structured to allow for a thorough review of documented or undocumented facts about the program, and for the gathering of opinions and perceptions of all key stakeholders involved in EMAP, including First Nations communities and organizations, provincial and territorial organizations, and INAC representatives.

Since the evaluation relied heavily on qualitative data, qualitative data analysis software (NVivo) was used to systematically structure the findings and allow for a complete integration of all qualitative lines of evidence. In particular, this approach supported an analysis by regions, which was particularly important considering regional variations in emergency management across INAC and Canada.

One challenge encountered during this evaluation related to the ongoing evolution of the program. Over the time period covered by this evaluation, INAC implemented a number of changes to EMAP, particularly as it relates to its management structure. This report attempts to adequately reflect these changes.

Another challenge faced related to the site visits and focus groups. Finding community representatives to participate in these two activities has proven challenging in some regions. This resulted in some delay in the data collection process, and some modification to the methodological approach.

It should be emphasized that these methodological challenges did not substantially affect the data collection process, nor the validity of the findings presented in this report.

2.4 Roles, Responsibilities, and Quality Control

The Evaluation, Performance Measurement and Review Branch and PRA Inc. worked collaboratively during the design, data collection, and analytical phases of this evaluation study. To this end, they benefited from the support of two committees:

• Working Group: A Working Group was established and made up of INAC employees from headquarters and four regional offices. Its mandate was to provide advice and guidance on the management and delivery of the EMAP program, identify key informants to interview, and validate findings.
  
  
• Advisory Committee: An Advisory Committee was also established and made up of individuals who have extensive experience in emergency management and have an interest in the EMAP program. The mandate of the Advisory Committee was to provide strategic advice to the evaluation during the early stages (to provide advice and guidance on the evaluation questions and proposed methodology) and the late stages (to review evaluation findings, conclusions, and recommendations).

This section explores the relevance of EMAP. In doing so, it provides an assessment of the anticipated future demand for EMAP from a First Nations perspective, reviews departmental responsibilities and the impact on EMAP of other issues falling within the department's mandate.  EMAP's contribution to government-wide priorities is examined through an assessment of the requirements of the 2007 Emergency Management Act and the expected activities associated with the theory of the four pillar approach. This section also looks at issues of duplication and overlap and potential gaps in delivery through an examination of other relevant programs dealing with emergency management.

3.1 The Role of INAC in Emergency Management

3.1.1 The Legal Responsibility of INAC

At a fundamental level, the relevance of EMAP is directly linked to the well-established responsibility of INAC to First Nations communities living on reserve. Section 91.27 of the Constitution Act prescribes the legislative authority of the federal government for "Indians and Lands reserved for Indians." To this end, the Department of Indian Affairs and Northern Development Act states that the "powers, duties and functions of the Minister extend to and include all matters over which Parliament has jurisdiction, not by law assigned to any other department, board or agency of the Government of Canada, relating to Indian affairs." It is on that basis that the Department has historically provided assistance to First Nations facing emergencies, well before the establishment of EMAP.

The Emergency Management Act (2007)

The passing of the Emergency Management Act in 2007 provided further clarifications as to the extent of the Department's responsibility in emergency management. Under the Act, the Minister is responsible for mitigation, preparedness, response and recovery for emergencies on reserve. The INAC Minister must identify "risks that are within or related to his or her area of responsibility" and prepare "emergency management plans in respect of those risks." It is further expected that such plans would be maintained, tested, and implemented, and that exercises and training would be conducted accordingly.

EMAP provides an important means by which the Department may fulfil its legal obligation to First Nations communities living on reserve, so in response to this new legislation, the Senior Assistant Deputy Minister of Socio-economic Policy and Regional Operations initiated an evaluation of EMAP. The evaluation was to inform new policy development and examine the fiscal pressures of emergency management on the Department. The evaluation was completed in July 2007 and resulted in 39 recommendations.

The majority of the recommendations were very specific and operational in nature such as the creation of a new directorate for emergency and issue management with permanent FTEs and A-base funding for HQ and a number of regions. There were three specific recommendations with respect to First Nations Emergency Services Society (FNESS) in British Columbia. The recommendations also touched on the need to define roles and responsibilities of new positions, mandatory training for staff, and business continuation planning in FN emergency management plans.

As a result of the 2007 evaluation, INAC established the Emergency and Issue Management Directorate and made progress on many of the recommendations.  At the time of the current evaluation, the Directorate completed a National Emergency Management Plan and is actively developing a process and guidelines for the negotiation of emergency management agreements and the identification of eligible expenses for reimbursement which were also recommended in the 2007 evaluation.  There are approximately ten recommendations from the 2007 evaluation where the new Directorate has not made significant headway:

• INAC formally endorse and promote, through policy development, an all hazards approach to emergency management which includes mitigation, preparedness, response and recovery (recommendations 2 and 3).
  
  
• Mitigation as a philosophy be developed in all sectors including capital expenditures and land claim negotiations (recommendation 2).
  
  
• Funding be made available for the development, updating and testing of emergency management plans (recommendation 20).
  
  
• Funding be made available for emergency management positions at the community or Tribal Council level where there was a demonstrated need, and for training or information sessions for newly elected Chiefs and Band Councils (recommendations 21 and 22).
  
  
• Establish clear concise measureable goals for monitoring progress on emergency management plans (recommendation 20).
  
  
• INAC move forward with a Memorandum to Cabinet to update financial authorities and obtain sustainable funding to provide an effective emergency management program to First Nations (recommendation 31).
  
  
• Authority be sought from Treasury Board to create an emergency management reserve that can be easily accessed for extraordinary emergencies (recommendation 32).
  
  
• The department initiate a legal review to determine the best manner to legislatively provide authority for a First Nations community to declare an emergency to protect the federal and provincial governments from potential civil litigation (recommendation 35).
  
  
• The department enter into a partnership with Health Canada in the development of emergency planning so that pandemic issues are included in the all hazards approach to emergency management (recommendation 36).
  
  
• The Department formalize a best practices policy designed to ensure all regions benefit from existing best practices. (recommendation 39).

Responsibilities relating to the territories

While the foundation allowing the Department to intervene in an all hazards approach to emergency management does exist, there are other departmental priorities that are not adequately captured in EMAP such as the situation in the three territories.  According to the Department of Indian Affairs and Northern Development Act, the Minister's duties extend to all matters relating to "Yukon, the Northwest Territories and Nunavut and their resources and affairs; and Inuit affairs." As it currently stands, EMAP allows the Department to intervene in the two First Nations reserves located in the Northwest Territories. Beyond that, there is far less certainty. In cases of emergencies affecting self-governing First Nations and land set aside for First Nations located in the Yukon, this evaluation has found no consensus on what the role of the Department should be. In any case, the current program authority associated with EMAP does not cover activities off-reserve.

At the time of the evaluation, the Department was developing a policy statement on the North, which acknowledges the federal government's responsibility to manage Crown land, as well as water and resources, on federal land located in the three territories. These functions include the management of emergencies affecting such land. The statement also recognizes the government's responsibility for contaminated sites in Nunavut and the Northwest Territories. Based on these principles, the question remains as to which department should lead emergency management in these areas. Evaluation findings on this question are inconclusive. Should the answer point to INAC, it should be emphasized again that the current structure of EMAP does not provide the foundation to intervene in these circumstances. The Department would need to either establish another emergency management program, or extend the current EMAP authority to formally include activities in the territories (beyond the two reserves in the Northwest Territories).

Managing other issues

In addition to the North, ambiguities also persist as to the role of INAC in activities falling beyond the strict parameters of the Department's legislative responsibility. The three primary cases that emerged from this evaluation are civil unrest, health-related issues (such as H1N1), as well as search and recovery activities.

There is little doubt that civil unrest incidents involving or relating to Aboriginal communities is of prime interest to INAC. For instance, the Ontario region has, in recent years, witnessed an increasing number of events regarded as civil unrest involving First Nations. The most notable of these is the escalation of the Grand River land settlement protests into highway blockades, prolonged land occupations, and several incidences of violent interactions between First Nations protesters, non-First Nations residents of surrounding areas, and Ontario Provincial Police officers. The prime responsibility for dealing with civil unrest events occurring off-reserve does not rest with INAC. However, the outcome of these events has a direct impact on First Nations and, by extension, on the Department. This explains why the INAC regional office in Ontario now has staff dedicated to monitoring civil unrest events involving First Nations. This, again, falls beyond the current EMAP program authority.

The same logic largely applies to health issues affecting First Nations communities. The recent pandemic events, linked to H1N1, required extensive coordination and monitoring efforts on the part of INAC and Health Canada. Ultimately, it is Health Canada (its First Nations, Inuit and Aboriginal Health Branch) that has legal responsibility for dealing with health-related issues affecting Aboriginal Canadians.[Note 5] Regardless, INAC staff has had to allocate considerable resources to support the work of Health Canada in dealing with this emergency. How such activities relate to EMAP remains unclear.

Whereas emergency management tends to focus on collective needs, search and recovery activities typically focus on one individual, or a few. Strictly based on compassionate grounds, and not as a result of a clearly established legal responsibility, the Department may provide assistance to pursue search and recovery efforts for missing individuals when the first response effort has been unsuccessful. This assistance is typically provided until no hope of recovery remains.

3.1.2 The Four Pillars of Emergency Management

The Emergency Management Act formally incorporates the four-pillar approach to emergency management. The Act specifically defines emergency management as including "the prevention and mitigation of, preparedness for, response to and recovery from emergencies." This approach is widely supported in the literature on emergency management. While traditional emergency management had a focus on response and recovery, mitigation and preparedness are now playing a prominent role in that field. Combining prevention and mitigation into one pillar, as the Emergency Management Act does, also aligns with the current theory on emergency management. This subsection further explores each of these pillars in order to better understand EMAP's relevance in this particular context.

First Pillar: Mitigation (and Prevention)

The aim of mitigation is to reduce the severity of consequences of an emergency by identifying potential emergency situations and vulnerabilities. Unlike the other three pillars, which focus on finding short-term solutions, mitigation aims to establish long-term strategies that reduce risks.

Activities falling under the mitigation pillar are of a distinct nature. While response and recovery activities are largely operational, mitigation involves strategic activities such as planning, political insight, negotiations, and public relations. Because of that, mitigation typically requires the participation of stakeholders who are outside the traditional emergency management circle.

Mitigation activities can be classified into structural and non-structural activities.[Note 6] Structural mitigation includes strengthening buildings and infrastructure to increase resistance to damage that would be caused by disasters. In the context of First Nations communities, raising homes in flood-prone areas would be a typical example of structural mitigation. Non-structural mitigation does not involve infrastructure. Rather, it requires planning within the context of the environment. Building new houses away from a known hazard or maintaining protective features of the natural environment are examples of non-structural mitigation.

Public Safety Canada has developed a list of activities that may be undertaken in the context of mitigation. It includes hazard mapping, adoption and enforcement of land use and zoning practices, implementing and enforcing building codes, flood plain mapping, burying of electrical cables to prevent ice build-up, raising of homes in flood-prone areas, and disaster mitigation public awareness programs.[Note 7]

The Piikani Nation in Alberta offers a good illustration of what mitigation can achieve. Following a major flood event in 1995, the band council passed a resolution prohibiting residential development on flood plains and conducted environmental assessments to secure funding for the installation of larger culverts. It also invested resources to support infrastructure improvements to its wells. These efforts did reduce the severity of subsequent floods.

Not surprisingly, by reducing the magnitude of future disasters and the risks to life and property associated with them, the cost of disaster response and recovery can also be reduced.[Note 8]

Second pillar: preparedness

Preparedness is about effectively anticipating emergencies. Its goal is to predict potential hazards and develop possible solutions. This is done with the aim of saving lives and reducing damages and injuries. Preparedness may be referred to as "anticipatory measures taken to increase response and recovery capabilities,"[Note 9] or more simply, activities to improve the ability of people and systems to manage an emergency when it occurs. As preparedness assumes that a disaster is likely to occur, it differs from the assumption of mitigation that a disaster may be prevented or that its effects may be minimized.

The main activity undertaken in the context of preparedness is the development of emergency management plans. As stated in subsection 3.1.1, the development of emergency plans is a specific requirement of the Emergency Management Act. Meaningful emergency plans require ongoing monitoring, updates, along with the appropriate training, exercising and public education.

By their very nature, preparedness activities require the involvement of all key sectors of the targeted community. Political authority, program managers, community organizations, first responders, as well as individuals and families, must participate in adequately preparing their community to deal with emergencies.

Third Pillar: Response

Response activities aim to effectively manage the immediate impact of an emergency on the community itself or its infrastructure.  Typically, communities begin a response to an emergency using available resources, but when the required response exceeds the community's ability, a state of emergency is declared.  This triggers the involvement of other organizations such as the provincial or territorial emergency management organizations.

As scenarios of emergencies vary significantly, so do the types of response activities that may be required. One of the first activities typically undertaken is the establishment of an operations centre. Once this is in place, the list of other activities that may be undertaken includes:

• Temporary relocation of individuals and families. This includes the provision of shelter, food, clothing, and required social and community services
  
  
• Provision of medical care
  
  
• Provision of essential services and equipment to sustain public infrastructure
• Security measures
  
  
• Provision of telecommunications equipment
  
  
• Provision of counselling services to those affected by the disaster or its response[Note 10]

Response activities are normally undertaken by first responders, such as emergency management staff, firefighters, police officers, or paramedics. The Canadian Red Cross may also be contracted to coordinate emergency responses and provide some of the required services. In the specific case of First Nations communities, two Aboriginal organizations in Canada are directly involved in emergency management, with a particular focus on preparedness and response:

• In Manitoba, the Manitoba Association of Native Fire Fighters provides response services, particularly related to community evacuation. INAC provides financial support to the organization to undertake emergency management activities on INAC's behalf.
  
  
• In British Columbia, the First Nations' Emergency Services Society also offers a range of emergency management services, including response services. Again, INAC provides financial support to this organization.

Fourth Pillar: Recovery

The primary purpose of recovery is "to restore post-disaster condition to an acceptable level."[Note 11] Once the recovery phase is completed, the community should have gained back a certain level of stability. This pillar is closely linked to the response one. In executing recovery activities, a community may also wish to pursue mitigation goals by implementing long-term solutions to address certain vulnerabilities.

The list of activities that may be undertaken during the recovery stage includes:

• Returning individuals and families to the community
  
  
• Trauma counselling
  
  
• Repairs to essential community infrastructure and equipment, such as water and sewage
  
  
• Clearance of various types of debris
  
  
• Costs associated with the rental of the required machinery to conduct recovery activities
  
  
• Essential landscaping (following a flood, for instance)
  
  
• Property cleanup (elimination of mould in houses affected by a water-related emergency)[Note 12]

As with the response pillar, a variety of organizations may be involved during the recovery stage, from first responders and contractors to local, provincial, and federal authorities.

EMAP in Relation to the Four Pillars

At a fundamental level, EMAP is well-aligned with the four pillars of emergency management. The program's Terms and Conditions specifically refer to each of these four pillars, and describe the process program recipients are expected to follow to obtain financial assistance, along with funding criteria. There are, nonetheless, serious gaps in INAC's approach to implementing this mandate, and these are further explored in Section 4 of this report.

3.1.3 The role of other federal departments

It should be noted that INAC is not the only federal department having a direct stake in emergency management on First Nations reserves. For instance, Health Canada is leading a process to adopt new regulations on water and wastewater management on reserves, while Environment Canada is proposing new regulations on fuel tank storage. Both federal departments will be requiring emergency response plans as part of the roll-out of these regulations. This would create an opportunity for efficiencies to be realized if these departments were to approach First Nations together and develop an all-hazards emergency management plan, instead of having each department approaching communities separately.

3.2 EMAP in Relation to Other Emergency Management Programs

The relevance of EMAP is also determined by the extent to which it complements other programs dealing with emergency management. To this end, three areas are particularly relevant, and they are examined in this subsection.

3.2.1 The Disaster Financial Assistance Arrangements Program

In the 1970s, the federal government established the Disaster Financial Assistance Arrangements (DFAA) program. To this day, the specific purpose of this program is to "assist provinces with the costs of dealing with a disaster where those costs would otherwise place a significant burden on the provincial economy and would exceed what they might reasonably be expected to fully bear on their own."[Note 13] It is worth emphasizing that, technically speaking, the DFAA program does not deal with emergencies falling under the responsibility of the federal government. It is the vehicle by which the federal government provides assistance to provinces and territories for emergencies falling directly within their respective jurisdiction. The rationale behind this program is to assist provinces and territories when an emergency has reached a scope where the province or territory, while responsible, would face an unreasonable burden. It is for this reason that only provinces or territories can be beneficiaries of the DFAA program.

The delivery of this program, which is under the responsibility of Public Safety Canada, is relatively straightforward. The province or territory facing a large emergency is expected to take all necessary measures to respond and recover from it. If the expenditures incurred in dealing with this specific emergency are greater than $1 per capita ($12.2 million in Ontario or $133,000 in Prince Edward Island, for example), the province or territory may request the assistance of the federal government through the DFAA program. Should the federal government agree to help the province or territory, an audit process will be conducted and the federal government will assume a portion of the share, based on the following sliding scale:

Using an example in Ontario, if a forest fire in the northern part of the province triggers $15 million in response and recovery costs, the province will pay the first $12.2 million, and the remaining $2.8 million will be shared equally between the province and the federal government.[Note 14] This means that the federal government will have contributed $1.4 million to the costs towards this specific emergency. To this day, the federal government has contributed over $1.8 billion to provinces and territories for post-disaster assistance through the DFAA.

Under specific circumstances, the DFAA program does contribute to the costs of emergencies on First Nations reserves. These would be circumstances where a specific disaster spans both on and off-reserve lands and where the province or territory has covered the response and recovery costs. It is important to note that the instances where costs for emergencies have been covered by the DFAA are rare. It is, in fact, the only circumstance where DFAA is used to cover expenditures relating to an emergency under the responsibility of the federal government. This was largely done for expediency, as it avoids having a provincial or territorial government submit expenditures to two separate federal departments.

It is worth noting that Public Safety Canada has recently reviewed the DFAA guidelines and the program may now support some mitigation activities. Before that, the program was strictly limited to response and recovery costs.

3.2.2 The Joint Emergency Preparedness Program

Another program that Public Safety Canada operates in the field of emergency management is the Joint Emergency Preparedness Program (JEPP). The purpose of this program relates directly to the preparedness pillar, as it supports activities that enhance the ability of individuals and communities to respond to emergencies. All projects funded through JEPP are cost-shared, where the program may cover up to 75% of the project costs. Public Safety Canada sets aside approximately $5 million a year for this program. Since it was established in 1980, JEPP has contributed over $158 million to support preparedness activities.

The delivery of JEPP involves provincial and territorial emergency management organizations. Any community, including First Nations communities, interested in submitting a project under JEPP must first submit it to its respective provincial or territorial emergency management organization, which, in turn, submits it to Public Safety Canada. A committee is responsible for reviewing all proposals and selecting projects. At the time of the evaluation, Public Safety Canada was receiving approximately $8 million worth of proposals. As a result, not all projects are granted funding.

Projects submitted under JEPP may serve to develop emergency plans or to carry out emergency response exercises under simulated conditions.

At the time of this evaluation, it appears that JEPP was one of the very few programs, if not the only one, in Canada that specifically provides funding assistance for preparedness activities. In this context, communities interested in pursuing preparedness activities must often fund them with their existing budget, unless they are successful at securing JEPP funding.

3.2.3 Provincial and Territorial Programs

Provincial and territorial governments play a predominant role in emergency management. To this end, each province and territory has established an emergency management organization to plan and coordinate the work of first responders. They also manage programs offering financial assistance for response and recovery purposes. Beneficiaries of these programs may be individuals and families, corporations, municipalities, or other local or regional authorities. As one may expect, program criteria, particularly as it relates to costs eligible for reimbursement, vary among provinces and territories. It is for this reason, for instance, that the federal DFAA program has its own set of program criteria covering costs eligible for federal assistance. The fact that two sets of funding criteria co-exist in cases where federal funding through DFAA is provided also explains the requirement for an audit process to determine the final contribution of each government.

The presence of these various provincial programs, each with their own funding criteria, presents a particular challenge in the context of EMAP. When an emergency management organization, or first responders, intervenes in a First Nations community to deal with an emergency, or when a provincial or territorial program is used to cover some of the response and recovery costs on a reserve, the federal government must reimburse these expenditures through EMAP or another source of funding, since it clearly falls under its responsibility. One alternative would be to create a federally funded response team, but this would be inefficient, as it would prove very costly and most probably ineffective, as it would risk duplicating other efforts. In this context, having some form of agreements between INAC and provincial governments becomes particularly important. This topic is further explored in the next section of the report.

This section of the report explores the actual delivery of EMAP, including its current structure for dividing roles and responsibilities and for funding activities. Evaluation findings indicate that gaps remain in effectively covering the four pillars of emergency management and that the current funding structure of EMAP greatly contributes to these shortcomings. This, in turn, has a direct impact on the ability of the program to reach all of its expected results.

4.1 Program Delivery Structure

There are variations in the program delivery structure of EMAP, based on the nature of activities undertaken. For this reason, this subsection is structured along the four pillars of emergency management, and includes a section dealing with other types of activities.

4.1.1. For Mitigation Activities

During the period covered by this report, evaluation findings indicate that the program has not delivered activities specifically related to mitigation. As it currently stands, the program's authority partially covers mitigation. EMAP's Terms and Conditions allow the Department to support First Nations communities during the initial stage of mitigation for activities such as site investigation, planning, and architectural and engineering design. However, the program does not cover actual equipment, construction, or repair.

Mitigation is a critical aspect of emergency management that presents opportunities to prevent emergencies from taking place thus reducing the long-term cost of emergency management. Mitigation also presents a means through which the Minister can meet the requirement under the Emergency Management Act to identify "risks that are within or related to his or her area of responsibility". During interviews, focus groups, and site visits conducted as part of this evaluation, all stakeholder groups emphasized the importance of addressing mitigation needs. A  possible approach for consideration suggested by some key informants was that the Directorate work horizontally with relevant INAC program areas such as capital and land claim negotiations to identify communities at risk for certain emergencies in order to work towards mitigation of those risks.  This idea was also introduced in the 2007 evaluation. 

4.1.2 For Preparedness Activities

EMAP's Terms and Conditions allow INAC to support First Nations communities in adequately preparing themselves to deal with emergencies. This includes developing actual emergency management plans, providing the required training, and proceeding with testing through simulation exercises.

During the period covered by this evaluation, INAC has provided some support in the area of preparedness. The structure used to deliver this component of the program varies among the different regions of the country. Some of INAC's regional offices contract a consultant to work with communities to prepare their emergency management plan. In other regions, INAC has signed agreements with emergency management organizations to provide this type of support. In Manitoba and British Columbia, Aboriginal firefighter organizations are contracted by INAC's regional offices to provide assistance in preparing plans.  In Ontario and Quebec, it is the provincial EMO that provides this service to First Nations.

As a result of the limited support provided through EMAP, evaluation findings indicate that there is a significant portion of activities undertaken by First Nations communities in the area of preparedness that is not supported through EMAP. During interviews, focus groups, and site visits, many representatives of First Nations communities indicated that they had received no specific funding to prepare their emergency plans, let alone to provide training or to test their plan. They essentially use their ongoing operating expenditures to cover these costs. This approach raises concerns. Some communities face more pressing and immediate needs and, as a result, will not undertake emergency preparedness activities. Others will undertake some emergency management activities, but at the expense of other areas.

Another flaw associated with the current program delivery structure is that it does not provide any assurance of the quality of preparedness activities. Since many communities are essentially undertaking these activities on their own, it is challenging to monitor the extent to which the program is actually achieving its expected results. For instance, while some statistics exist on the number of First Nations communities that technically have an emergency management plan in place (see Section 4.4 of this report), one can only speculate as to the number of communities that update their plan, provide the required training, and conduct the cyclical testing. In fact, findings gathered as part of this evaluation indicate that many communities do not conduct the required training and testing of their plan. The following are some of the comments gathered from First Nations representatives:

• "We try to have a tabletop exercise every year, but we don't some years."
  
  
• "We've only had one tabletop; this was five years ago."
  
  
• "My problem is financial: it costs money to organize a tabletop. Just to assemble a meeting, I don't have money for that."
  
  
• "In our community, we have an updated emergency manual, but we don't practice it. It just sits on a shelf and picks up dust."
  
  
• "Most communities have plans, but it just sits on a shelf; they don't run any mock trials, or do any type of training."

The failure to adequately prepare emergency plans is logically expected to have a negative impact on activities related to the other three pillars of emergency management.

4.1.3 For Response Activities

While EMAP's current program parameters refer to the response pillar, they remain ambiguous as to what this area is expected to cover. The program's Terms and Conditions include expenditures related to some of the required activities, such as equipment rental, aircraft rental, fuel costs, transportation, and communications, but they specifically exclude capital repairs.

Working within these parameters, the delivery of activities related to response are typically structured through agreements between INAC's regional offices and emergency management organizations. In the provinces of Manitoba and British Columbia, INAC's regional offices also hold agreements with Aboriginal firefighter organizations to assist during the response and recovery stages of emergency management.

To proceed with agreements with emergency management organizations is clearly the most effective strategy INAC can pursue to structure the delivery of response activities. Since the federal government requires the collaboration of provincial and territorial governments and their emergency management organizations to assist First Nations communities during a response process, having a formal agreement avoids ambiguities and speculations during an actual emergency.

One challenge with this part of the program delivery structure relates to the involvement of First Nations communities. These communities are not technically part of these agreements, yet they are the ones that may need to deal with the emergency management organizations once an emergency is declared. Findings gathered as part of this evaluation indicate that First Nations communities (particularly those that are regularly facing emergencies such as floods or forest fires) have built a relationship with their respective provincial emergency management organization. However, it is uncertain that a systematic process currently exists to ensure that First Nations communities are informed of the parameters set in these agreements.

The roles and responsibilities for communities are unclear, at least in part, due to the fact that there is no statutory requirement for FN communities to have an emergency management plan.

4.1.4 For Recovery Activities

Here again, the program authority specifically mentions recovery as a pillar of emergency management, but it largely remains silent as to the types of activities and expenditures that could be covered. Some emergency management agreements specifically cover recovery activities, while, in other circumstances, issues appear to be managed on a case by case basis.

At this point, the recovery area largely focuses on activities allowing community members to return following an evacuation, and on some capital projects to restore infrastructure to its pre-emergency status. The Department may need to broaden this range of activities. During focus groups held as part of this evaluation, some community members were of the opinion that INAC should play a stronger role in helping individuals and families deal with the trauma of an emergency.  Participants noted a need for internal coordination with relevant program areas, as well as outside organizations that can help community members deal with the emotional trauma of losing their home and belongings, or a community member.  This type of assistance may also be required as a result of civil unrest to help community members on both sides of the protests to heal some of the rifts in the community.

4.1.5 For Other Types of Activities

As indicated in Section 3.1.1 of this report, INAC is currently involved in a number of activities that fall beyond the four pillars of emergency management, or beyond the strict emergency management mandate of the Department. Search and recovery, civil unrest, and coordination of health-related activities are among those mentioned by stakeholders consulted as part of this evaluation.

The only one of these other types of activities that is directly covered by the current program parameters is search and recovery. Essentially, these requests are considered on a case by case basis, and INAC's contribution is strictly made on compassionate grounds.

For all other types of issue management, there is no formal program delivery structure in place, which represents an important gap. At this point, some of INAC's regional offices assign significant resources to issue management. INAC's regional office in Ontario has staff dedicated to the monitoring of activities related to Aboriginal protests. INAC's regional office in British Columbia has staff dedicated to the monitoring of Aboriginal protests expected as part of the 2010 Olympic Games.

An important question is to determine whether these other types of activities should be included within the formal parameters of EMAP. If so, a program delivery structure will be needed to clarify how these activities are to be delivered.

4.2 Roles and Responsibilities

Appropriately defining the roles and responsibilities of all stakeholders involved in emergency management is challenging, regardless of the context. The range of activities included in emergency management, combined with jurisdictional considerations, creates a complex environment in which stakeholders must operate. This subsection explores these issues as they relate to EMAP, and is structured by levels of intervention.

4.2.1 At the National Level

At the national level, INAC's Emergency and Issue Management Directorate has led a process to better define roles and responsibilities within the Department. To this end, the Directorate has developed INAC's National Emergency Management Plan. This document describes the roles of senior officials within the Department, and of various coordinating structures involved in managing emergencies (see Figure 2 on page 8). The plan specifically refers to the four pillars of emergency management and to the need for the Department to provide assistance in relation to each of these pillars. The plan includes broad parameters on the financial assistance to be provided in support of emergency management. It was tabled in May 2009, and the Department expects to add a number of annexes to address issues such as emergency management in the North, a national flood plan, guidelines, reporting template, as well as a list of eligible and non-eligible expenses. It should be noted that, in developing this departmental plan, the Directorate took into account a number of recommendations included in the 2007 formative evaluation of EMAP.

Since its roles and responsibilities may overlap with those of other federal departments, such as Health Canada or Environment Canada, INAC may consider strategies to formalize interdepartmental collaboration. This was suggested in the 2007 EMAP evaluation (in relation to Health Canada, more specifically), and it would cover scenarios where INAC may not act as the lead department, but is nonetheless contributing based on its experience or geographical location (in the North, for instance).

4.2.2 At the Regional Level

In accordance with EMAP's program authority, INAC's regional offices play a predominant role in coordinating emergency management activities. Each regional office benefits from a fair amount of autonomy in structuring activities, including the negotiation of agreements with emergency management organizations.

At the time of the evaluation, the Department was funding one full-time position as emergency management coordinator in all regional offices, with the exception of regional offices located in the three territories. Interviews held with departmental representatives indicate that allowing a staff person to focus on emergency management was effective and desirable. The traditional strategy to manage emergencies "from the corner of one's desk" greatly limited the ability of regional offices to be responsive, particularly during negotiations of Memoranda of Understanding (MOU) with other partners.

At the time of the evaluation, some regions had succeeded in signing an MOU to formally establish roles and responsibilities at the regional level. This was seen as the most effective approach to avoid confusion during an emergency. Table 4 outlines the status of formal agreements between INAC regional offices and provincial emergency management organizations.  It indicates the presence of both a general emergency management agreement and an agreement specific to wildfire suppression.  The table does not include formal agreements held between INAC regions and non-provincial stakeholders, such as First Nations organizations.

Where no agreements exist, INAC regional offices have informal working arrangements with their respective emergency management organizations. It should be noted that negotiations towards a regional MOU in the Atlantic region have been ongoing for several years, but have yet to produce an agreement.

At the Local Level

Many factors shape the roles and responsibilities of emergency management stakeholders at the local level. The types of emergencies, the nature of activities undertaken, the characteristics of the communities affected by an emergency, and the policy framework in which a community operates are some of those critical considerations.

Band councils and, in some communities, tribal organizations are the first to assess and respond to an emergency. They are also expected to play a leading role during the preparedness stage, particularly in the development of emergency management plans. As with any local government, band councils often respond to small-scale incidents or issues, without needing any outside help. When a band council is overwhelmed by what has become an emergency, they have the responsibility to coordinate the involvement of other stakeholders.

On this point, interviews, focus groups, and site visits held as part of this evaluation specifically explored how a state of emergency is actually declared. Findings indicate that there is simply not a straightforward procedure followed across First Nations communities. However, the one consistent finding is that no outside organization will intervene in a reserve without a direct request or approval from the band council. Also, evaluation findings indicate that provincial emergency management organizations would normally respond to any request for assistance made by a band council, without the prior approval of other organizations, including INAC.

Once an emergency has been confirmed, several stakeholders will come into play:

• The emergency management organization would normally establish an operations centre to coordinate activities on the ground. Representatives from the band council and from INAC's regional office regularly participate in the coordinating efforts.
  
  
• Depending on the nature of the emergency, firefighter services, police services, and other social services assist in evacuating the community or providing direct assistance to individuals and families. In some regions, the Red Cross is contracted to provide some of these first response services.
  
  
• If the emergency caused damage to personal property or community infrastructures, the required repairs are undertaken to bring them back to their pre-disaster condition. The non-insured part of these expenditures is normally paid by the provincial or territorial government. In the case of First Nations communities, the provincial or territorial government would then request a full reimbursement from INAC.

In the specific case of forest fires, INAC has ongoing agreements (that may or may not be formalized, as illustrated in Table), whereby a set amount is provided yearly to the provincial or territorial government to cover some of the expenditures it incurred to combat these fires.

The fact that many First Nations communities are geographically isolated directly affects how roles and responsibilities are structured. As indicated by some First Nations representatives interviewed, several days may pass before external first responders are able to reach an isolated community. As one representative noted, "the biggest challenge for us is that you can't jump in a car or send a group of vehicles to help, because we're on an island."

INAC's decision to extend its emergency management activities to other issues such as civil unrest also affects the distribution of roles and responsibilities. The primary issue here is that civil unrest normally occurs outside of First Nations reserves. These protests relate to Aboriginal issues, but the bulk of the activities occurs in a territory that falls under provincial or municipal authority. Under that scenario, the primary responder is the police service and INAC's regional office will focus on monitoring and documenting these events.

Evaluation findings indicate that, despite some of the ambiguities associated with the distribution of roles and responsibilities, first responders do not hesitate to provide the requested assistance to a community in need. The organizational culture widely shared across organizations and regions is that any community in need deserves to be assisted, regardless of jurisdictional uncertainties. The latter are to be resolved after the fact, and should never constitute a barrier to assistance.

4.3 Funding Structure

EMAP's current funding structure is contributing to inefficiencies in program delivery. Experiences in other jurisdictions confirm that there are a variety of options available to the federal government, of which all are preferable to the status quo. This subsection further explores these findings.

4.3.1 Issues with EMAP's current funding structure

At the time of the evaluation, EMAP was in the awkward position of having authority to assist First Nations communities that are facing emergencies without being able to access any stable funding to support this mandate, other than the initial allocation dedicated to forest fire. While emergencies are, by their very nature, unpredictable, experience to date confirms that they are also unavoidable. Their nature, scope, and geographical location may vary from year to year, but INAC must always face the fact that emergencies will occur, and that the Department will be required to fulfill its legal obligations. As illustrated in Table 5, regional offices and headquarters have, over the last five years, recorded between $10 million and $48 million a year in assistance to First Nations communities to deal with emergencies and other issues (such as civil unrest).

It is critical to note that numbers included in Table 5 do not provide a comprehensive picture of the financial support INAC provides to First Nations for emergency management purposes:

• Evaluation findings indicate that some of INAC's regional offices use ongoing operational resources to fund activities related to emergency management. The list of these activities includes (but is not limited to) staff positions dedicated to emergency management (beyond the one position funded by headquarters), individuals contracted to provide direct assistance to First Nations communities for planning and training purposes, or direct payments to First Nations communities for limited emergency-related expenditures.
  
  
• Using its infrastructure programs, the Department funds projects that pursue mitigation purposes. Some of these projects strictly pursue mitigation goals (e.g., irrigation system to avoid cyclical floods), while others may include a mitigation component (e.g., a new school with the capacity to act as an emergency coordinating centre). Yet, these projects are not systematically included in EMAP's official financial figures.
  
  
• Some of the work performed by provincial or territorial emergency management organizations, Aboriginal firefighter organizations, and community organizations never makes it into the official EMAP figures. Most of the work undertaken by these organizations are covered by EMAP (as indicated in Table 5), but there are some costs that these organizations perform that are not reimbursed.

Another serious issue with the current funding structure is its inability to support ongoing activities. The Emergency Management Act formally sanctions the four-pillar approach. The first two of these pillars—mitigation and preparedness—requires an ongoing commitment to assess systemic vulnerabilities; to identify risks; and to elaborate, update, and test emergency management plans. INAC's regional offices and band councils are largely left on their own to secure any funding for these purposes.

The incremental nature of the current funding structure also raises serious problems relating to accountability and performance measurement. Attempting to build the financial picture of EMAP is a daunting task that can only produce, at this point, a puzzle with missing pieces. Knowing how much financial resources are invested in pursuit of a set of program goals is an essential requirement for adequately monitoring the program's progress towards achieving them. An incomplete financial picture is bound to produce an incomplete performance story.

4.3.2 Experiences in Other Jurisdictions or Settings

Models currently in place in Canada, as well as in other jurisdictions, provide valuable insights on how to improve EMAP's funding structure.

In managing the DFAA program, Public Safety Canada has had to face many of the same challenges INAC is currently facing with EMAP. Up until the mid-1990s, Public Safety Canada did not have ongoing (A-base) funding to operate the DFAA program. Instead, it had to seek supplementary funding from the Treasury Board after the fact in order to support provinces and territories. In the mid-1990s, as the severity of emergencies in Canada was increasing, the amounts requested by Public Safety Canada were also escalating, and using supplementary funding was no longer a strategy that was consistent with the purpose of this funding mechanism. At that point, the Department succeeded in securing base funding for the DFAA program (in the form of a class-A contribution program). Since then, the program has been providing a yearly forecast to establish its base funding. If, during a specific year, this base funding was to prove insufficient, the Department would proceed with a request for supplementary funding. This has yet to occur. It is worth repeating that this program focuses on response and recovery activities, which are the least predictable of the four emergency management pillars.

Recognizing that floods are bound to be a reality in Manitoba, the provincial government now includes flood control as a regular item in its annual budget.[Note 15] Again, this type of funding is very much centred on the two pillars of response and recovery.

Around the world, countries that have been particularly exposed to emergencies Costa Rica, for instance, the government established an emergency reserve at the federal level known as the National Emergency Fund. Established in 1969, the Fund provides assistance in responding to and recovering from disasters such as earthquakes and volcanoes.[Note 16] Many American states have also adopted the model of establishing reserve funds.

Working more directly with private insurers, governments in Japan, Great Britain, and New Zealand have opted for disaster reinsurance programs for protection against natural and human-induced disasters. Simply put, these programs offer protection to insurance companies against unsustainable levels of claims, which are often associated with large-scale disasters.

One of the key questions to be addressed when establishing an emergency management fund is the formula to be used to set the actual budgetary level. In that regard, three options are currently used:

• Some governments set aside a fixed percentage of their budget for emergency management purposes. As mentioned during interviews conducted as part of this evaluation, it appears that setting aside one percent of a budget for emergency management purposes would be considered a best practice, particularly at the local level.
  
  
• Another approach consists of looking back at the history of the program to determine the average level of funding demands. This is particularly applicable for programs that have been operating for quite some time.
  
  
• The third approach consists of forecasting expected funding requests. This is the approach Public Safety Canada is currently using with its DFAA program. This works particularly well when dealing with large-scale emergencies, where several years may be required to complete the response and recovery stages.

4.3.3 Options for Considerations

There are a number of options available to INAC to establish an ongoing budget for EMAP. These options must take into account the distribution of roles and responsibilities in the delivery of EMAP:

• At the national level, the Emergency and Issue Management Directorate must first access the appropriate resources to sustain its operational work. Secondly, the Directorate could be tasked with managing a response and recovery fund dedicated to emergencies specifically affecting First Nations communities that are not already covered by the DFAA program. The level of resources for this fund could be established using the average amount of response and recovery expenditures that have occurred over the past three or five years. Using a five-year period and assuming that numbers included in Table 5 are adequately reflecting these types of expenditures, a five-year average would represent approximately $20 million. Funding requests would be managed through INAC's regional offices, using funding eligibility criteria that ensure comparability of services between First Nations communities and their surrounding non-Aboriginal communities.
  
  
• At the regional level, EMAP could formalize the current practice of funding one full-time position in each of the regional offices (except in the territories) to act as emergency management coordinators. In addition, regional offices could manage a fund dedicated to supporting preparedness activities. This would allow regional offices to work directly with first Nations communities that may require assistance in adequately planning for emergencies.
  
  
• Funding local-level activities is more challenging. With over 600 First Nations communities across Canada, it becomes particularly challenging to provide funding to each of them. Yet, the buy-in and collaboration of First Nations leaders is absolutely key to developing meaningful emergency management plans. The Department could either provide a direct allocation to each community, ask them to set aside a percentage of their operating budget for emergency management purposes (1 percent) or focus on the program described in the previous bullet in relation to funding emergency preparedness activities. It appears particularly important to engage the most vulnerable First Nations communities and those that face recurring emergencies (such as floods or forest fires).
  
  

These various options do not cover mitigation activities. On this point, the department could consider emergency management criteria in administrating capital programs. This, however, would be done outside of the EMAP's framework.

4.4 Program Results and Outcomes

This evaluation indicates that, generally speaking, when a First Nations community is facing an emergency, it will receive assistance that will be comparable to other surrounding communities. The achievement of this outcome is partly due to EMAP, but it is also due to the widely held organizational culture among emergency management organizations that when help is needed, help is provided. The primary goal of EMAP is to strengthen First Nations communities' ability to actively engage in this process. To this end, progress has been achieved, but considerable challenges remain. This subsection further explores these findings.

4.4.1 Current Performance Measurement

At the time of the evaluation, INAC did not have a formal performance measurement strategy in place for EMAP. As noted throughout this report, during the period covered by this evaluation, the Department has strengthened its internal capacity in the area of emergency management, particularly through the Emergency and Issue Management Directorate. Performance measurement is expected to be part of the ongoing activities of the Directorate. At this point, the Directorate is considering the following performance indicators as a basis for this strategy:

• INAC National Emergency Management Plan in place
  
  
• INAC Regional Emergency Management Plans in place
  
  
• Agreements in place with provinces/territories for emergency management
  
  
• First Nations emergency management plans in place

A more comprehensive performance measurement strategy will need to adequately reflect the progress made in building the capacity of First Nations communities to actively engage in emergency management, as this is a central goal of the program. Also, the performance measurement strategy will need to adequately cover activities initiated at the regional level in all areas of emergency management.

4.4.2 Key Results

The capacity to measure EMAP's results is limited by the gaps in the program's financial information that result from the current funding structure. With this in mind, findings on key results are reviewed for each of the four pillars, and for other activities.

Mitigation

This evaluation has not found any results related to mitigation that can be attributed to EMAP. Evaluation findings indicate INAC has undertaken some infrastructure projects that pursue mitigation goals, but the information about these projects could not be gathered and reviewed. In any case, they were undertaken outside of the program framework.

Preparedness

At the time of the evaluation, a number of First Nations communities had developed an emergency plan. As indicated in Table 6, the Department estimates that between 24% and 100% of First Nations communities located in the various regions of the country had completed an emergency plan. The extent to which EMAP has contributed to this result varies among First Nations communities. Interviews, site visits, and focus groups held as part of this evaluation indicate that, in some communities, this was done without any help from INAC, through EMAP or otherwise. In other cases, communities have greatly benefited from the services rendered by organizations contracted by their respective INAC regional office to help establish these plans.

As widely emphasized during interviews, focus groups, and site visits, elaborating a plan on paper is an important step, but it does not guarantee that a community has actually strengthened its preparedness capacity. An emergency management plan can become a critical tool to focus all activities of the community in that field. But a plan can also sit on a shelf and be meaningless. Evaluation findings indicate that these two scenarios, and many other versions in between, currently exist across Canada.

Preparedness also involves ongoing training and exercises. Some of INAC's regional offices have supported training initiatives and some activities related to emergency simulation. But again, the situation varies across the country. Evaluation findings indicate that some communities have undertaken these activities on their own, while others have successfully obtained support from the Department.

Response

During the period covered by this evaluation, EMAP has successfully provided assistance to First Nations communities facing actual emergencies. Working directly with First Nations communities and emergency management organizations, the Department has succeeded in engaging key partners so that no First Nations community would be left unassisted in case of an actual emergency. During this period, more than $140 million has been provided in response assistance to First Nations communities. Consultations held with First Nations representatives confirm that response services are provided when needed. However, the current funding structure does not provide detailed information on the range and type of activities undertaken in the response area.

Signing agreements with emergency management organizations is one strategy INAC continues to pursue to secure the critical collaboration of first responders. At the time of the evaluation, many provinces and territories had yet to seal their collaboration with INAC through a formal agreement. As indicated in Table 4, there are no agreements in place east of Ontario, and even where agreements have been signed, some are either outdated or just being implemented. In that sense, there is much work to be done.

Recovery

For much of the same reasons as those described above (for response activities), the current funding structure does not allow for a clear identification of activities undertaken in the recovery area. At this point, response and recovery activities are largely treated as part of the same category.

Other Issues

Working directly with regional offices, the Emergency Management and Issues Directorate has built capacity to monitor events that could potentially escalade into emergencies. Events of interest (such as protests or other types of activities undertaken during, for instance, the Aboriginal National Day of Action) are closely monitored and, in cases of escalation, INAC's regional offices are in a position to provide assistance to first responders in order to better understand issues that may have triggered these protests and to mitigate risks to individuals and property.

4.4.2 Best Practices

A first area of best practices that was documented as part of this evaluation was performance measurement. To undertake this type of monitoring activities is challenging. Evaluation findings indicate that leading organizations in this field are still struggling with this issue. As an illustration, the Red Cross, Public Safety Canada, and the Public Health Agency of Canada have all developed performance indicators, but are just beginning to collect baseline data.

Not surprisingly, performance measurement tends to focus on the preparedness pillar.[Note 17] These activities are expected to occur on an ongoing basis, and are therefore predictable and measurable. The performance measurement may cover the quality of the planning done, the effectiveness of training, or the efficiency of exercises.

Another area of best practices relate to the involvement of community stakeholders in emergency management. To be successful, emergency management must involve all key stakeholders who may be affected by a potential disaster. The literature has largely discarded the top-down, hierarchical approach to emergency management. Instead, the widely held view is that emergency management is a process based on consensus and cooperation. In that sense, it must involve more than the actual first responders.

A key benefit in involving stakeholders is to account for cultural values and background. A relevant illustration can be found in Australia, where in dealing with a specific emergency, government officials recommended moving an Indigenous community to higher ground, for security reasons. However, the proposed higher ground was considered sacred to the community, who were opposed to setting foot on the land.[Note 18] In the United States, the Federal Emergency Management Agency adopted a Tribal Policy in 1998 that specifically requires consultations with tribal governments prior to any actions being taken in their communities. In many circumstances, providing cultural education to first responders is seen as an essential step in tailoring an emergency management strategy.

4.4.3 Unintended Impacts

EMAP's current funding structure has had unintended negative impacts on both First Nations communities and INAC's regional offices. The requirement to seek reimbursement through Treasury Board submissions for response and recovery expenditures has forced communities and regional offices to cash-manage these costs, redirecting funding allocated to other programs while waiting for the reimbursement to occur. This triggers the need to internally plan and review current financial commitments to free up the required resources to cover emergency management costs. Even once the reimbursement has occurred, which may take several months, cash-managing is still required to ensure that these resources can be effectively redirected to their original goals.

4.5 Program Efficiency

For the purpose of EMAP, the efficiency issue can be addressed from two angles: the efficiency of emergency response on reserves and the efficiency of the program itself.

At the time of the evaluation, EMAP financial resources were largely centred on response and recovery activities. No funding was directly associated with mitigation activities, and only limited funding appears to have been provided in the areas of preparedness. The four-pillar approach referred to throughout this report is based on the premise that up-front work, through mitigation and preparedness, greatly contributes to the effectiveness of response and recovery activities. EMAP has yet to fully engage in these first two pillars, but the Department has signalled its intention to do so. Achievements in that regard will directly improve program efficiency.

The current program funding structure creates inefficiencies. As described in previous sections of the report, there are alternative models in Canada and in other jurisdictions that could be used to improve the ongoing management of EMAP.

This closing section of the report includes a set of conclusions based on the findings described in the previous sections. When needed, recommendations are included.

5.1 Conclusions

This subsection offers conclusions for each of the evaluation issues and questions that formed the framework of this study. It is worth emphasizing that the purpose of this subsection is not to introduce any new findings. Rather, it is to provide concluding observations, based on evidence already described.

5.1.1 Relevance

This evaluation confirms the need for EMAP. There is an overall trend towards increased  frequency and intensity of emergencies throughout Canada and First Nations communities are considered "high risk" when it comes to disasters due to their small size, social vulnerability and remoteness and isolation.  Many First Nations do not have updated emergency management plans in place leaving them unprepared when emergency events occur.

EMAP is the central tool available to INAC to ensure that required assistance services are provided to First Nations communities facing emergencies. However, the Program, as it is currently designed and delivered, does not meet the needs of First Nations communities in the areas of mitigation, preparedness and recovery.

The EMAP program authorities and objectives (as they are currently outlined in the Logic Model) are largely aligned with government-wide priorities as documented in the 2007 Emergency Management Act, as all are based on the four-pillar approach to emergency management. However, the current program objectives do not appear to capture all departmental priorities.

In recent years, INAC has paid increased attention to civil unrest as part of EMAP. While not strictly defined as an emergency in itself, and events frequently occurring off reserve, civil unrest has the potential to erupt into a situation involving emergency services and First Nations communities.The Department is committed to providing assistance for issues such as civil unrest, and some of the current resources dedicated to emergency management are used for that purpose. However, the current program authority does not include these types of activities (other than search and recovery activities).

EMAP objectives also do not reference Departmental responsibilities in emergency management in the territories. The actual responsibility of INAC when it comes to emergency management in the North has yet to be clearly established.

Search and recovery is similar to civil unrest in that it is not strictly defined as an emergency, but falls under the EMAP program.

One final area of responsibility that is not currently reflected in EMAP's outcomes or authorities is the Department's involvement in emergency activities that are outside INAC's jurisdiction such as pandemic planning. INAC dedicated significant resources to a Health Canada process to have pandemic plans in First Nations communities.

As shown above, INAC's emergency management activities go beyond a strict definition of emergency management.

This evaluation does not point to any substantial issues relating to duplication. In theory, the activities Public Safety Canada currently supports in the area of preparedness (through JEPP) could potentially overlap with some of the activities that EMAP could fund in this area. However, the scope of these two initiatives is limited and no actual instances of duplication have been found as part of this evaluation.

In terms of gaps in delivery, when compared with standard activities of all hazards approaches used in other jurisdictions, EMAP falls short in the areas of mitigation, preparedness and recovery. This is discussed in more detail below.

5.1.2 Performance

At the national level, INAC has established the Emergency and Issue Management Directorate to coordinate the program's activities, and support regional offices and other stakeholders as required. INAC's regional offices are collaborating with provincial and territorial emergency management organizations, as well as with Aboriginal organizations. There are formal agreements in place in approximately half of the jurisdictions, and negotiations are ongoing elsewhere.

EMAP's delivery structure for response and some aspects of recovery is sound as the program essentially supports provincial emergency management organizations that can offer the expertise and resources needed in the area of emergency management. However, the current program delivery mechanisms and structure do not provide the required framework to pursue an all hazards approach to emergency management as required by the Emergency Management Act. There is essentially no structure in place to deal with mitigation-related issues. Various approaches are currently used to support preparedness activities, and while flexibility in this area is required, the current program delivery structure does not provide a clear understanding of the scope of EMAP activities related to preparedness.

The 2007 formative evaluation included close to 40 recommendations, many of which were operational in nature. This evaluation indicates that several of these recommendations helped the work of the Department in developing its departmental emergency management plan. Also, recommendations relating to the funding structure are largely echoed in this evaluation report.

The shortages in appropriate delivery mechanisms and structures identified in answer to question 4 above may be linked to a lack of defined roles and responsibilities. INAC's roles and responsibilities in delivering an all hazards approach to emergency management, especially in the areas of mitigation, preparedness and recovery have not been clearly articulated resulting in inconsistencies in programming across Canada.

At the local level, the distribution of roles and responsibilities becomes more complex. Depending on the community involved and the nature of emergencies occurring, there can be a wide range of stakeholders involved. This evaluation indicates that ambiguities do exist in that regard. In particular, some First Nations communities remain uncertain as to the extent of their responsibility in dealing with emergencies, from declaring the emergency itself to carrying out the required activities under the four pillars of emergency management.

The fact that the Department has extended the scope of emergency management activities to include issues such as civil unrest also adds to the complexity associated with the distribution of roles and responsibilities.

As introduced in the answer to question 2 earlier, the precise role of the Department in an all-hazards approach to emergency management in the three northern territories is also not well defined. Along the same lines, INAC roles and responsibilities with respect to emergency-related activities that fall within the responsibility of another department or jurisdiction (such as health issues) are not clear.

It is important to note that despite these ambiguities in three of the four pillars of emergency management, response services have not been delayed. This evaluation indicates that when faced with an emergency, local stakeholders will proceed and provide the required assistance. Any unresolved administrative issue is addressed after the fact.

EMAP's current funding structure is problematic. It does not provide the required financial base to pursue all of the program's goals and objectives. It also creates inefficiencies in providing the required financial assistance needed to allow INAC to fulfill its legal obligations.

At the time of the evaluation, it was practically impossible to assemble a complete financial picture of EMAP. The requirement to proceed with a new Treasury Board submission every time significant resources are required has triggered unintended negative impacts. In some cases, the Directorate or regional offices need to reallocate funding from other programs to cover some costs. The same situation may occur with band councils. In turn, the incomplete financial picture creates challenges in measuring performance and appropriately documenting the achievements of the program.

Experiences in other settings or jurisdictions confirm that there are a number of options INAC could pursue to improve EMAP's funding structure. Such changes are needed if the program is to successfully pursue program objectives relating to the four pillars of emergency management.

At the time of the evaluation, the Emergency and Issue Management Directorate was collecting only a few indicators related to the number of agreements in place and the number of emergency management plans in place in communities. These indicators measure only a portion of the work being undertaken and do not provide a very useful measure on their own as there are indications that the plans in place are of poor quality, are out dated and have not been tested. Aside from these few indicators, there was no procedure in place to measure and document the program's results, best practices and lessons learned. The Directorate has established founding blocks, such as the development of a departmental emergency management plan, and processes to work and communicate with regional offices. On that basis, the Directorate expects to develop a performance measurement strategy.

At the time of the evaluation, the program's outcomes were concentrated in the area of response and recovery. Despite the lack of agreements in some regions, the Department has succeeded in coordinating and securing the collaboration of emergency management stakeholders to adequately respond to emergencies affecting First Nations communities. However, there were some comments that recovery is focussed primarily on returning evacuees to their communities and restoring damaged infrastructure. It was felt by some that more could be done to help communities deal with the trauma of communities and restoring governance following an event.

The program's outcomes in the area of preparedness are more limited. The Department has provided assistance to some First Nations communities in developing plans and providing training. However, evaluation findings indicate that the need for support in this area far exceeds what the program has offered to date. Also, the evaluation has not documented any program results in the area of mitigation, although infrastructure work continues to be among INAC's priorities.

In accordance with best practices in the field of emergency management, it is expected that more focus will be required on the two up-front pillars of mitigation and preparedness.

The current EMAP funding structure has had unintended negative impacts. It has forced First Nations communities and the Department to cash-manage funding requirements, at the expense of other programs.

The program is in the process of better defining how it intends to fulfill its obligations related to the four pillars of emergency management, particularly as they relate to the first two on mitigation and preparedness. Achievements in these two areas can be expected to enhance program efficiency.

5.2 Recommendations

Recommendation 1:  Roles and responsibilities

It is recommended that INAC clarify the roles and responsibilities of the Department as they relate to emergency management.  This process should consider the current environment of emergency management, specifically the implications of the 2007 Emergency Management Act.  To do so, the Department must define relationships with all external stakeholders and put in place the appropriate governance structures and agreements to ensure fulfillment of responsibilities related to emergency management.  All aspects of emergency management should be considered in this process, with particular emphasis on the following areas:

1. The precise role of the Department in emergency management in the three northern territories.
   
   
2. The precise role of the Department with respect to emergencies that fall within the responsibility of another department or jurisdiction (such as health issues and civil unrest).
   
   
3. The program delivery mechanisms and structure relating to the four pillars of emergency management: mitigation, preparedness, response and recovery activities.
   
   
4. Horizontal engagement of other relevant INAC programs that have a potential to contribute to an all-hazards approach to emergency management, such as capital infrastructure in mitigation projects or land claims in civil unrest issues.
   
   
5. The precise role of First Nations communities in emergency management.

Recommendation 2:   Program funding structure

It is recommended that INAC consider a revised funding structure, to alleviate the impact on regions, other program areas, and communities and provide a secure funding base for the Department's emergency activities.  To facilitate this transition, INAC should document existing INAC funding for emergency management programming and develop forecasts for future expenses relating to an all hazards approach to emergency management.

INAC should also identify appropriate resources in alignment with the Department's roles and responsibilities.  Specifically, ensuring that the department has the ability to provide preparedness and mitigation services in accordance with Departmental obligations under the EMA.

Recommendation 3:  Performance measurement

It is recommended that INAC develop a Performance Measurement Strategy for emergency management programming in consultation with the Evaluation Performance Measurement and Review Branch and in accordance with the principles of the new Treasury Board Policy and Directive on Evaluation.