Evaluation of Aboriginal Economic Development Strategic Partnerships Initiative

1.2.1 Context and Background

1.2.2 Program Objectives and Outcomes

SPI is situated as Program 3.2 under the Lands and Economy Pillar of AANDC's 2014-2015 Program Alignment Architecture.

SPI was launched in June 2010 as a component of the Government of Canada Action Plan to implement the Federal Framework for Aboriginal Economic Development. SPI was designed to help achieve the following strategic goals of the Framework:

• Forging new and effective partnerships – primarily among federal departments and Aboriginal stakeholders, but also with provincial and territorial governments and the private sector; and
• Focusing the role of the federal government – fostering a whole-of-government approach that aligns and targets investments toward market-driven opportunities in key sectors of the Canadian economy.

The stated objective of SPI is to "increase economic development opportunities for Aboriginal entrepreneurs and communities by promoting partnerships between federal and non-federal partners in priority sectors of the economy, including forestry, fisheries, mining, energy and agriculture"^{Footnote 1}. While SPI has primarily sought to advance initiatives within five priority economic sectors – forestry, energy, mining, fisheries and agriculture - many government departments, including AANDC, are being asked to respond to other major development opportunities across the country (such as pipelines, potash, and major hydro development projects). In order to respond to emerging pressure for federal collaboration on these opportunities, SPI has adopted an opportunity-based approach as opposed to the sectoral approach that that was once used. This is intended to lead to greater participation by Aboriginal peoples in the Canadian economy.

SPI expected outcomes are aligned with the strategic priorities, as identified in the Federal Framework for Aboriginal Economic Development, by contributing to opportunity-ready communities, viable Aboriginal businesses and a skilled Aboriginal workforce^{Footnote 2}.

The outcomes and performance indicators identified below are based on the revised Performance Measurement Strategy for the Strategic Partnerships Initiative (July 2013). The revised logic model is presented below.

Text alternative for Logic Model of the Strategic Partnerships Initiative (SPI)

The logic model is oriented from top to bottom. On the left is the title for each row. The first title on top is: Inputs; the second title is: Activities; the third title is: Outputs; the fourth title is Immediate Outcomes; the fifth title is: Intermediate Outcomes and the last title at the bottom is Long-term Outcomes.

The first row on top is composed of two boxes. The box on the left is: Financial and in-kind contributions other resources from non-federal partners. The box on the right is: SPI resources (G&Cs), O&M, other. A dotted line is separating the first row and the second row.

The second row is composed of three boxes. The box on the left is: Building relationship and partnerships; the box in the middle is: Opportunity assessment and priority setting and the box on the right is: Supporting targeted economic development opportunities. An arrow is connecting the box on the left and the box in the middle and an arrow is connecting the box in the middle and the box on the right. Arrows pointing down are connecting all three boxes to all four boxes in the third row.

The third row is composed of four boxes. The box on the left is: Partnership between Federal and non-Federal partners. The second box from the left is: Opportunity profiles, proposals or strategies and studies. The third box from the left is: Community profiles, readiness and capacity assessments. The last box on the right is: Awareness building initiatives in economic development opportunities. Arrows pointing down are connecting the box on the left and the box on the right to all three boxes in the fourth row.

The fourth row is composed of three boxes. The box on the left is: Cooperation and collaboration among Federal and non-Federal partners; the box in the middle is: Simplified government application, monitoring and reporting (single window) and the box on the right is: Aboriginal communities engage members, partners and stakeholders on complex/regional development initiatives. Arrows pointing down is connecting all three boxes to the two boxes in the fifth row.

The fifth row is composed of two boxes. The box on the left is: Aligned federal investments in Aboriginal economic development opportunities. The box on the right is: Aboriginal communities are ready to participate in economic development opportunities. An arrow pointing down is connecting the two boxes to the box in the fifth row. The box in the fifth row is: Aboriginal communities benefit from federal strategic partnerships and investments in economic resource opportunities.

---

SPI aims to achieve the following three immediate outcomes:

• Cooperation and collaboration among federal and non-federal partners;
• Simplified government application, monitoring and reporting (single window); and
• Aboriginal communities engage members, partners and stakeholders on complex/regional development initiatives.

These immediate outcomes are expected to result in the following two intermediate outcomes:

• Aligned federal investments in Aboriginal economic development opportunities; and
• Aboriginal communities are ready to participate in economic development opportunities.

Both intermediate outcomes will contribute to a single long-term outcome: Aboriginal communities benefit from federal strategic partnerships and investments in economic resource opportunities.

1.2.3 Program Management, Key Stakeholders and Beneficiaries

1.2.4 Program Resources

The total funding for this program is $85.6 million (fiscal year 2010-2011 to fiscal year 2014-2015) or $16.9 annually. SPI's annual budget is made up of approximately $14.5 million in Grants and Contributions funds and $2.4 million in Operations and Maintenance funds. The Grants and Contributions funding portion is predominately spent on multi-year initiatives.

Financial Summary Table by Estimates Vote Structure (Dollars)

2.3.1 Planning Phase

This phase was undertaken approximately between July 2013 and December 2013.

Prior to undertaking the evaluation, the evaluation team met with SPI program representatives in order to develop a better understanding of SPI, identify key documents and discuss the development of a Terms of Reference, including the evaluation scope, approach, broad issues and questions and cost. Terms of Reference were approved by the Evaluation, Performance Measurement and Review Committee in September 2013.

2.3.2 Data Collection/Analysis Phase

Data was collected using multiple lines of enquiry and analyzed by triangulating evidence against the issues and questions. This phase was undertaken approximately between January 2014 and June 2014.

2.3.3 Reporting

For each line of evidence, findings from all lines of evidence were analyzed and identified against evaluation issues and questions as key findings by using a findings template format (streamlined technical report). Findings were then cross-identified and triangulated in order to identify emerging themes and key findings.

Several documents and supporting technical-based reports were created to reflect the evidence collected from the evaluation based on evaluation issues as per the evaluation matrix. This phase was undertaken approximately between July 2014 and September 2014.

Preliminary findings were shared with Evaluation Working Group and Advisory Group members and presented to the Evaluation, Performance Measurement and Review Committee in June 2014.

2.4.1 Considerations

Since SPI involves many departments with different roles and responsibilities, collaboration and access to data of those federal departments was essential to the success of this evaluation. To increase success, an evaluation working group and advisory group was established to increase the participation of SPI coordinators and partnered signatories as well as data sharing.

2.4.2 Strengths

As an integral part of the New Federal Framework for Aboriginal Economic Development, comprehensive research was conducted to identify key success factors for Aboriginal business and economic development. SPI is a cornerstone program under the Federal Framework for Aboriginal Economic Development and benefits from the resources and research conducted under the Framework.

Case studies allowed for the opportunity to collect information at a level of detail that was not possible to collect through broader and more representative methods (i.e. a survey) because of cost, schedule and other factors. The case studies highlighted best practices, lessons learned and key successes. The methodology for the case studies included a review of Initiative-specific documentation interviews, administrative data review and on-site observation. To conduct the case studies, evaluators traveled to the regional hub and, where possible, conducted site visits to the Aboriginal community / communities involved in the initiative. Some of the case studies involved multiple remote communities, and numerous partners.

2.4.3 Limitations

The Strategic Partnerships Initiative is a new program for the Department, launched in 2010. Very little longitudinal data exist with which to assess program outcomes, especially in relation to impacts on Aboriginal economic development, job creation, and community well-being. The evaluation therefore focuses on immediate outcomes and the design and delivery aspects associated with SPI.

The SPI Performance Measurement Strategy is currently being updated and may affect the availability of necessary information. Of particular importance, the evaluation mostly relied on the most up-to-date logic model (July 2013) to evaluate the program and not the model originally created for the program. As a result, some of the new outputs in this logic model have only had data collected for them in the past year, limiting longitudinal performance measurement.

In conducting the evaluation, it was apparent that neither strong nor comprehensive performance data was available from participating departments on their initiatives. Additionally, there was concern that different departments may have different ways of determining indicators and measuring outcomes, thus, making cross-departmental programming comparisons difficult. Furthermore, any inconsistencies in data measurement would make it difficult to determine the influence of the SPI initiative over time. Evaluators collected and assessed the performance measurement data available and developed tools that allow for a comprehensive assessment of the state of performance measurement, and the achievements of the program to date. Performance data was mostly qualitative and was assessed with content analysis, with additional financial information used to inform assessments where applicable.

SPI initiatives by their nature are complex and involve multiple players. Coordinating cross-departmental and community-based involvement was slow and cumbersome at times. Further, because projects are complex, the final impacts will become evident only in the long-term and not in the funding period under examination.

Due to the multi-stakeholder nature of the projects that SPI helps leverage, it was difficult to differentiate the impact between SPI dollars and other stakeholders’ contributions. It is important for the purpose of this evaluation to assess the effectiveness of SPI dollars in creating a cross-department leverage effect only and not to evaluate the impact of other stakeholder funds.

The evaluation had neither the resources nor the time to assess each major initiative funded to date.

Community readiness of Aboriginal communities

The current evidence suggests that Aboriginal people are poorly positioned from a capacity and community-readiness perspective to fully benefit from the resource economy. Generally speaking, communities have a limited ability to negotiate with private sector businesses.

According to literature review, a lack of business expertise prevents the participation of some Aboriginal entrepreneurs in opportunities resulting from nearby major projects and can also impede communities from pursuing partnerships, such as equity participation or joint ventures with industry proponents.^{Footnote 4} Further, the literature review highlights capacity limitations with respect to overall business and management and further identified a need for support in accessing major financing, expertise and access to training opportunities.

Evidence from the case studies highlighted the relevance and usefulness of having an established business development team assists communities in developing business plans, obtaining funding and implementing business plans and was cited as a key success factor.

Given the challenges faced by many Aboriginal communities and some Aboriginal organizations with respect to community readiness, case study and interview respondents suggested that SPI provide technical assistance and expert opinions in a number of areas. These most notably included areas related to business planning, legal issues, environmental assessment and knowledge regarding specific sectors, such as forestry, mining and fisheries and the commercialization of energy technologies for example.

Case study and interview respondents also suggested, however, that the needs related to the community readiness of communities, which have an impact on the level of their economic participation are very broad. For example, while the majority of SPI projects have focused on increasing the capacity and preparedness of Aboriginal communities to participate in economic development initiatives, the success of some SPI community readiness projects have been hindered by other social issues, such as housing.

Recognizing that to SPI is not intended to address all social and economic needs of communities, evidence suggests that more clearly articulated outcomes related to the projects supported by the SPI initiatives might help to address this issue. For instance, the case studies found that for initiatives and projects that are more limited in scope, the need being addressed appears to be more clearly defined.

To address this issue, it was suggested by some that communities play a more direct role in articulating the needs related to the Initiatives through increased planning, through band council resolutions, letters of support and other means. Further, case study evidence suggested that community needs be considered more holistically in the context of the broader strategic objectives and expected results identified for SPI as a horizontal initiative.

SPI horizontal approach

The evaluation found a demonstrable need for the horizontal approach SPI employs. This is primarily related to both the complexity of resource-based economic development in the context of Aboriginal participation in economic development initiatives, as well as the fragmented nature of Aboriginal programming.

According to documents reviewed, the landscape of Aboriginal economic development in Canada has changed due to a number of critical factors related to the complexity of resource development opportunities, including their scope, the greater number of resource development initiatives in close proximity to Aboriginal communities and legal requirements for industry and governments to consult with Aboriginal communities prior to economic developments. The involvement of multiple sectors is therefore necessary in order to manage the increased complexity of economic development opportunities.

The need for the involvement of multiple players, however, has resulted in the involvement of a wide variety of stakeholders. Documents reviewed indicate that there are currently more than twenty federal departments and agencies that presently have mandates that include Aboriginal economic development. Documents reviewed and interview respondents have further pointed out that programs offered by these departments have not necessarily encouraged collaboration or have helped to facilitate the identification of shared priorities.

Evidence suggests that without a horizontal and coordinated approach to Aboriginal economic development programming, investments may not be maximizing efforts to be strategic, targeted and timely, with projects and proposals often receiving funding that only address one aspect of the help they need. Further needs may remain unmet and communities are often not successful in preparing for participation in economic development opportunities. Furthermore, an approach that is fragmented and lacking in coordination typically requires that Aboriginal communities and organizations complete multiple applications and meet diverse reporting requirements to access the funds necessary to support a single economic development project. This places additional burden on communities that are already, in many cases, contending with limited capacity with respect to overall business management.

Therefore, a clear and demonstrable need exists for a strategic horizontal approach that will ensure federal investments in Aboriginal economic development are aligned in order to maximize benefits for Aboriginal people. This is supported by the 2007 Report of the Senate Standing Committee on Aboriginal Peoples, which recognized the need for greater coordination within the federal government and recommended it take a lead role in facilitating partnerships between Aboriginal people and the private sector.

Clearly, SPI is addressing a need to support the community readiness of Aboriginal communities in engaging in complex regional economic development opportunities. This is accomplished largely through the SPI process of horizontal collaboration to encourage strategic and targeted investment. However, community readiness with respect to economic participation extends beyond what SPI currently aims to address. Therefore, there is a need to view community readiness more holistically, and continue to consider those needs identified by communities, in the context of economic development planning. Further, SPI is addressing a need to reduce the fragmented nature of economic development programming through its collaborative and horizontal approach, however, there is a need for the SPI to continue to align its objectives and expected results identified for SPI as a horizontal initiative.

Overlap and Duplication

From the perspective of economic programming, case studies and key informant interviewees described SPI as being complementary to other programs and noted that the risk of duplication of programming is lowered by having federal departments at the same table through SPI initiatives. The increased communication between federal departments not only allowed participants to identify instances where potential duplication might occur but also provided opportunities to ensure SPI projects are consistent with and complimentary to other government programming.

Evidence of other federal programming relating to Aboriginal economic development through the promotion of collaboration and partnerships was found, notably AANDC's Community Opportunity Readiness Program (previously named the Community Economic Opportunities Program) and the Community Support Services Program.

It was determined that both the Community Opportunity Readiness Program and the Community Support Services Program do not represent duplication with SPI as both provide a focused approach to project-based funding whereby SPI's funding is intended to fill gaps that cannot be addressed by existing programs. Neither the Community Opportunity Readiness Program nor the Community Support Services Program employs a horizontal approach intended to involve multiple partners through a single window approach to facilitate the involvement of Aboriginal communities in economic development opportunities.

Further, the evaluation found no evidence of duplication among federal or provincial jurisdictions in terms of SPI's horizontal collaborative approach to Aboriginal economic development. Documents reviewed and case study and interviewee respondents could not provide any other examples of programming or initiatives that support Aboriginal participation in the economy through a collaborative, partnership, single-window delivery approach.

AANDC Follow-up Audit of Capacity Development, which included a case study of SPI, identified it as an interesting model that could well be applied to other areas of capacity development, such as band governance and management, infrastructure management, and delivery of services.

Further, the willingness to explore the use of the single-window approach can be seen through the Pangnirtung Project: Making Connections for Youth. This AANDC-led pilot project was created to respond to the federal objective to reduce the administrative burden on grants and contribution recipients, with its goal to establish a single-window program/service delivery model that can be used as a tool to flow funds to northern and remote communities.

Indeed, SPI was cited by many case study and interview respondents as having the potential for being a model, one that could be replicated, for partnering and collaboration that can be extended to other areas beyond economic development.

4.1.1 Program Outcome: Cooperation and collaboration between federal, non-federal partners.

Cooperation and collaboration between federal and non-federal partners was examined using two key indicators as per the SPI Performance Measurement Strategy:

• number of partnerships developed interdepartmental Letters of Agreement and other types of agreements; and
• stakeholder perceptions of changes in understanding, co-operation and collaboration due to partnerships.

SPI has resulted in the cooperation and collaboration amongst 15 federal departments and agencies, including AANDC, involved in the key sectors of the economy targeted by the program:

• Aboriginal Affairs and Northern Development Canada
• Industry Canada
• Federal Economic Development Initiative for Northern Ontario
• Natural Resources Canada
• Agriculture and Agri-Food Canada
• Employment and Social Development Canada
• Canadian Northern Economic Development Agency
• Parks Canada
• Federal Economic Development Agency for Southern Ontario
• Status of Women Canada
• Western Economic Diversification Canada
• Fisheries and Oceans Canada
• Environment Canada^{Footnote 5}
• Canadian Environmental Assessment Agency
• Canada Economic Development for Quebec Regions

SPI, through the Initiatives it supports, has also demonstrated success in the area of cooperation and collaborating through the establishment of partnerships between multiple federal and multiple non-federal partners with the aim of supporting First Nations participation in economic development. In some cases, this collaboration enabled partnerships between organizations that had never existed before. Case study respondents and interviewees indicated, for example, that the SPI initiative Ring of Fire, in 2010-2011, enabled partnerships between the Matawa Tribal Council, the Government of Canada (AANDC, Natural Resources Canada, Federal Economic Development Agency of Northern Ontario and Employment and Social Development Canada), the Government of Ontario (Ministry of Aboriginal Affairs, Ministry of Northern Development and Mines, Ministry of Natural Resources), and industry (Noront Resources, Cliffs Natural Resources, KWG).

The number of partnerships and unique partnerships for each of the sectors and SPI as a whole, based on data reviewed for the period between fiscal year 2010-2011 through to fiscal year 2012-2013 identify that SPI, overall, has resulted in over 100 unique partnerships being established, noting that multiple partnerships with the same entity are only counted once to avoid duplication. The largest share of these new partnerships is in the Forestry and Mining sectors.

In addition to formal partnership arrangements, working partnerships have been established through SPI's governance processes via interdepartmental working groups, regional working groups, and implementation teams established for SPI sectors and projects.

However, while the number of partnerships provides a quantitative value with which to measure the achievement of the Immediate Outcome Cooperation and collaboration between federal and Aboriginal partners, it does not tell us a great deal about the level and value of cooperation and collaboration that occurred. For example, while partnerships may have been formed, it is unclear whether they resulted in full collaboration between the parties, if they resulted in benefits for Aboriginal communities, or if those benefits could have been achieved without the partnership. As a result, SPI could better articulate what it means to have partnerships within a sector, and why those partnerships are indicative of performance.

In addition, the evaluation examined the changes in stakeholder perceptions of understanding, co-operation and collaboration due to the partnerships as an indicator of the level of cooperation and collaboration reached between SPI partners. Stakeholders interviewed during the key informant interview process were nearly unanimous in their agreement that partnerships are necessary for Aboriginal economic development, that SPI has resulted in an increase in the number of partnerships and that more partnerships are necessary in the future. Key informant respondents noted that co-operation and collaboration through leveraging, community engagement, partnerships and agreements have all increased throughout the lifespan of SPI.

Specifically, stakeholders commented that partnerships increased the legitimacy of projects for Aboriginal communities, industry and other levels of government. Businesses and industry were more willing to take part in projects when federal partners were already involved. This increased legitimacy made it easier to leverage funds and increased the chances of success for specific SPI projects. However, it should be noted that despite the positive opinions of stakeholders, no additional evidence was found through the literature or document review to confirm that partnerships increase the success of economic development projects in Aboriginal communities.

Key informant respondents also noted that the increased cooperation and collaboration, particularly through SPI working groups, has resulted in increased communication and an improved ability to both identify opportunities and eliminate potential overlap. Respondents further noted that SPI working groups have shown departments who traditionally have not had a responsibility to promote Aboriginal economic development that there is in fact an important role for them to play. The involvement of a greater number of departments was seen as a positive aspect of SPI as it was noted that the Government cannot continue to work in "silos" and a more coordinated approach to supporting Aboriginal communities is needed. In particular, it was noted that with budget reductions taking place in governments across Canada, partnerships are more important than ever to continue to provide services with reduced resources.

However, while there are positive examples of cooperation and collaboration between federal partners and between federal and non-federal partners, progress has been impeded by weak communication of SPI to federal departments and Aboriginal communities. Specifically, staff, internal and external to AANDC, noted that they spent much of their time educating others about SPI due to a lack of awareness regarding the program and how it operated. In some instances, the lack of communication resulted in missed opportunities for valuable partnerships. In the case of one SPI initiative, the project proponents did not learn the role the new Director General Investment Sub-committee would play until shortly before its 2013-2014 application was due, and as a result, faced challenges.

The challenges relating to communication were also noted in the Governance Review conducted by Patterson Creek Consulting, 2011. The Review noted that at the time, the Federal Coordination Committee had not developed a communications strategy and as a result, the governance review recommended the creation of a communications and marketing director general sub-committee to address the problem, however, some key informant interviewees noted that it took a long time for communications materials to be developed.

Finally, case study interviews identified a challenge relating to ownership as it relates to cooperation and collaboration amongst partners. It was observed in case studies that other federal departments regard SPI as being "owned" by AANDC and not truly horizontal. Further, Aboriginal communities do not regard SPI as offering equal partnership with government and noted the minimal representation of Aboriginal people in the governance structure. Some respondents said there is a lack of true partnership and collaboration in developing annual workplans and it was felt that even when workplans were developed through extensive consultation with communities, they were eventually reduced in scope by the federal government.

4.1.2 Program Outcome: Simplified government application, monitoring and reporting (single-window approach)

The achievement of the program outcome: simplified government application, monitoring and reporting (ie. The single-window approach) was examined using three key indicators as per the SPI Performance Measurement Strategy:

• number of initiatives where "single window" approach was used;
• stakeholder perceptions of the administrative burden of SPI projects compared to other initiatives involving multiple departments; and
• stakeholder perceptions of the value added of the single window aspect of SPI.

According to the documents reviewed, SPI is intended to be different from the usual application based approval and funding process.

SPI allocates funding based on workplans developed collaboratively with Aboriginal communities, federal departments and other partners with a single window approach for recipient communities to access funds and for reporting requirements. In this way, it is hoped that SPI can employ a more coordinated opportunity identification process to eliminate the need for clients to navigate the many federal departments and programs involved with Aboriginal economic development while pursuing specific projects. This coordinated approach is also expected to provide a mechanism to harmonize monitoring and reporting processes as well as the management of financial agreements among relevant partners.

The evaluation found that take-up of the SPI's single-window approach is very limited. Based on the data reviewed and case studies examined, only two of the fourteen SPI Initiatives are currently using the single window approach - the only Northwest Territories Forest Industry and Biomass Initiative and the Atlantic Commercial Fisheries Diversification Initiative. In the case of the Northwest Territories Forest Industry and Biomass Initiative, funding from SPI allocated to AANDC and Natural Resources Canada, along with funding from the regional development agency Canadian Northern Economic Development Agency, all flowed through one contribution agreement between Canadian Northern Economic Development Agency and the Government of the Northwest Territories, due in large part to the persistence of staff in two federal departments.

Overall, respondents felt that in instances where a single window approach had been implemented, it was very beneficial to clients and overall represented an ideal model. In situations where project proponents were not able to fully implement a single window approach, they were able to incorporate some aspects, such as use of the common Terms and Conditions. Many respondents noted that, through SPI, they worked more closely with clients and other departments and this increased collaboration had benefits. Despite the challenges in implementing the single window approach, respondents commented that they believed it would result in a decreased administrative burden and that the decrease would primarily be seen on the client side of the projects.

There are also some examples where simplified reporting and administration have been achieved. For example, the Government of the Northwest Territories provides one annual report to the three federal departments (Canadian Northern Economic Development Agency, Natural Resources Canada, and AANDC), which support the Forest Industry and Biomass Initiative. Additionally, in SPI's tripartite agreement with Ontario as part of the Ring of Fire initiative, the province is using the SPI forms for applications and reporting. But the outcome of a reduced reporting and administrative burden has not been achieved on a large scale. In fact, some partner organizations say that the administrative burden is still significant and consumes the time of their program staff.

While case study and key informant interview respondents generally had positive comments regarding the goal of a single window approach, they identified some challenges.

One hindering factor is the short-term and unpredictable nature of funding for SPI projects. In some cases, projects have been given multi-year funding agreements yet, must have their annual workplans approved each year. In regards to one case study project, respondents from the organization, the federal government and First Nations said the lack of access to stable funding is an impediment. An example provided by one respondent is the difficulty of building a sustainable organization when staff members are all on short-term contracts.

Restrictions on how SPI funding can be used, and the lack of flexibility for re-profiling allocations, are also hindering factors because there is little latitude given for communities and organizations to respond to changing circumstances. Financial planning for complex regional developments requires flexibility since it is impossible to anticipate all contingencies in an annual workplan. For example, multiple interviewees commented that they underestimated the time needed for the community engagement portions of their projects. The time needed to properly engage with communities and gain their support for the project was longer than they expected and as a result, their projects fell behind schedule, which created challenges regarding the re-profiling of funds. Interviewees felt that additional flexibility regarding re-profiling would allow them to adapt to unexpected issues such as these and ensure they continue to effectively deliver their project.

Specifically, the single window approach requires a change in the culture of government and a change in how departments work together. It was noted by respondents that the current culture of governments is one of silos, wherein each department is focused on its own mandate and resources when undertaking a project. While SPI offers a broader and more collaborative approach, there is hesitancy within departments to merge human and financial resources with other organizations and unfamiliarity with the mechanisms to do so. The skepticism within departments regarding the use of collective resources and the confusion around the collaborative mechanisms of SPI are contributing factors to the slow progress in moving towards a single window approach.

A general lack of understanding and knowledge was also noted as a challenge, and may be the root of the skepticism mentioned above. Departments are reluctant to transfer their own funds through the SPI authorities for a range of reasons. The legal and contractual mechanisms for other departments to use the SPI authorities appear to have been poorly communicated. Specifically, interview respondents reported that there is frequently confusion between and within departments regarding how single window financing could work, how it would be reported, the results and measurements that would be required, the Terms and Conditions, and how the project would adhere to the different departmental mandates. Some departments have tried to ensure their work is consistent with the collaborative intent of SPI by delivering projects or programs in close consultation with others; however, they are not willing to commit to the single window approach. Departments interested in taking part in SPI generally have a positive intent to come together to fund projects, yet, acquiring the necessary information regarding the single window approach can be challenging and obtaining approval for such a funding mechanism can be met with resistance. Further, in an environment of deficit reduction departments are wary of signing over funds to another department.

Several misconceptions limit the ability of the single window approach to be used on a wider scale and illustrate the lack of understanding regarding the funding approach. As a result, the departments that do demonstrate some form of cooperation with SPI typically use their own authorities to support Aboriginal organizations and communities instead of the more flexible SPI common authorities. Many respondents spoke of specific departmental funding criteria that must be met, which prevent them from undertaking a single window approach. Specifically, with the exception of the Forest Industry and Biomass Initiative, the regional development agencies, interviewed through case studies, said it would be difficult, if not impossible, for them to use the SPI authorities to flow their funds through another department. A respondent from one of these agencies noted that they still must apply their agency's criteria to Aboriginal economic development, which restricts opportunities. Further, respondents noted that funds cannot be provided without knowledge that they would be used as defined by departmental grants and contributions requirements. However, as the departments and agencies have signed on to the SPI common Terms and Conditions, they do, in fact, have the ability to fund projects using the single window approach even if it does not comply with their regular departmental authorities. This information does not seem to be widely known among departments participating in SPI.

Finally, the evaluation found that it has been challenging to get some federal departments to engage, and participate in, SPI. Interviewees noted that in some instances, departments are fearful that someone else will get the "credit" for collaborative projects undertaken through SPI. The success of SPI partnerships is also hindered by structural impediments that exist in departments. Specifically, respondents noted that some departments were "stuck in their ways" and were unwilling to fund projects that fell outside of their usual funding scope. Further, constricting the ability for departments to partner are the different decision making structures that exist in federal departments. For example, some departments have decision making and funding authorities residing with the Deputy Minister or Minister while others have placed the authority with director generals. As a result, decisions are made by different departments at different times resulting in delays and other challenges to partnerships.

Related to this lack of information is a concern expressed by respondents regarding the high turnover of assistant deputy ministers, director generals and directors involved in SPI, as this often requires a renewed effort in "converting" new people to the value of the SPI single window approach. Respondents commented that there was still an education process to be completed, which would address these misconceptions and knowledge gaps. Additionally, in order to address the perception that SPI is "owned" by AANDC, respondents noted a need for increased support from the Treasury Board of Canada Secretariat to encourage the use of the single window approach and a more coordinated effort at the federal level.

4.1.3 Aboriginal communities engage members/partners and stakeholders on complex regional development initiatives.

The evaluation examined the extent to which the program achieved the following outcome: Aboriginal communities engage members/partners and stakeholders on complex regional development initiatives were examined using two key indicators as per the SPI Performance Measurement Strategy:

• number of Memoranda of Understanding/Impact Benefit Agreements, contractual or other agreements established or in the process of negotiation, between Aboriginal communities and public or private sector partners; and
• stakeholder perceptions of Aboriginal community and private sector engagement in economic development initiatives.

Based on data available for fiscal year 2010-2011 through to fiscal year 2012-2013, the following 14 initiatives have been reviewed^{Footnote 6}:

Modest progress has been made towards supporting Aboriginal communities in engaging members, partners and stakeholders on complex regional development initiatives, specifically through contractual agreements such as Memoranda of Understanding and Impact Benefit Agreements.

For example, the case studies found that SPI had resulted in the following agreements:

• The Forest Industry and Biomass Initiative, undertaken by the Government of the Northwest Territories, has led to a 2014 Forest Management Agreement between the Territorial government and a joint venture company owned by First Nations and Métis;
• The First Nations Power Authority has reached agreements with SaskPower to supply a total of 30 MW of power generation (one agreement is still to be finalized);
• In the Ring of Fire, respondents said that SPI support allowed community chiefs to move ahead on negotiations with the Ontario Government regarding infrastructure for remote communities and some communities have also made significant progress toward Impact Benefit Agreements; and
• In Quebec, the True North Treasure Initiative supported two Aboriginal communities in negotiating successful Impact Benefit Agreements. Other communities are being supported in negotiations underway with mining companies on two other projects.

Case study respondents highlighted specific instances where SPI successfully increased engagement between Aboriginal communities and the private sector regarding economic development:

• In one such instance, it was noted that SPI was successful in addressing conflicts between an Aboriginal community and a local mine regarding an existing Impact Benefit Agreement, which led to a renewed partnership going forward.
• The First Nation Power Authority has also signed an agreement with SaskPower that defines a process for establishing power generation opportunities for First Nations in order to ensure communities, with varying levels of capacity, reach an effective agreement with the provincial utility.
• In the Ring of Fire Initiative, SPI funding has enabled one tribal council to obtain legal advice and training in developing exploration agreements.

Additionally, SPI has focused on laying the groundwork for Aboriginal communities to effectively negotiate future agreements. For example, a current SPI initiative is focused on community planning with the aim of clearly defining community needs in order to form the basis of future negotiations. It is hoped that, by doing so, communities will be better positioned to negotiate Impact Benefit Agreements that truly represent the identified needs of the community rather than focusing on a few specific benefits.

Several interviewees felt, however, that a greater emphasis should be placed on developing partnerships with the private sector going forward. It was felt that a main focus of SPI so far has been partnerships between federal departments and, while that is important, effort should be made to increase the number of private sector partners involved in SPI projects.

4.1.4 Program Outcome: Aboriginal communities are ready to participate in economic development opportunities

The achievement of the program outcome was examined, as per the SPI Performance Measurement Strategy, by:

• evidence of increased community capacity to participate (e.g. Aboriginal people trained or mentored, new business partnerships created, business plans produced and implemented and procurement strategies developed).

Overall, the evidence indicates that by addressing capacity development and community readiness through strategic and targeted investment, SPI has made progress towards this outcome.

Documents reviewed cited a key challenge to ensuring the readiness of Aboriginal communities is the general lack of capacity in the communities, including a need for skilled workers, management capacity, technical capacity of community leaders, and a more thorough knowledge and understanding of the process involved in the developmental projects and their long-term benefits. The Energy Sector Reference Paper from March 2011 noted that few communities had the required skill set in trades, leadership and technical capacity in relation to renewable energy, sustainable development, engagement and project planning^{Footnote 7}.

In case studies as well as interviews, respondents continued to note that capacity building efforts related to engagement and skill and leadership development were required to allow Aboriginal communities to participate more fully in economic development. This increased capacity would allow communities to manage the economic processes themselves, as opposed to letting a developer manage projects for them, and potentially not take the communities' interests into consideration.

A review of data indicates that the majority of reported activities undertaken by initiatives were related to the development and implementation of partnerships, business plans, and procurement strategies, followed by engagement and training activities for both trades and management.

Efforts and progress towards engaging partners and developing relationships were continuously noted in internal progress reports, speaking to the ongoing nature of this particular activity – a pillar of all SPI initiatives. To date, 103 unique partnerships have been established across all initiatives. The collaboration of stakeholders, including Aboriginal organizations, has helped identify a variety of capacity needs at different levels and provided training and engagement activities to target areas, including trade skills development, management skills and training, cultural sensitivity training, negotiations training, and support and guidance in development and implementing business plans.

Across all 19 SPI Initiatives, there were close to 300 reports of activities related to determining needs and increasing the capacity of Aboriginal organizations to benefit from SPI.

The collaborative efforts of the partnerships have resulted in an increased capacity for management as well as skilled workers in the communities. Interviewees and case studies from initiatives that have been concluded indicate that increased capacity has been seen. For example, the Atlantic Commercial Fisheries Diversification Initiative established business development teams to provide technical business development capacity to interested Aboriginal communities. These in turn helped communities to develop proposals, establish business and procurement strategies, as well as establish several new businesses, expand others and create 250 new jobs for the communities involved. The First Nation Power Authority also reflects positive results, particularly noted by interviewees as a successful example of capacity building. Communities that could benefit from the First Nation Power Authority originally did not know how to get involved in the power generation opportunity. However, First Nation Power Authority created a vehicle for agreement negotiation and community education about the power generation industry. This gave the communities the information and capacity to take part in larger economic development opportunities.

Such positive outcomes from earlier initiatives seem to be an encouraging indication for newer initiatives that are just beginning their capacity building and engagement strategies. The British Columbia Liquid Natural Gas project is currently undergoing an engagement strategy that will increase awareness about the Liquid Natural Gas industry and inform communities about the employment opportunities. Similar to earlier initiatives, this will be followed up with feasibility and needs assessments to determine capacity gaps and inform training and business strategies. Another initiative, the Ring of Fire, has also focused on building capacity to increase the communities' ability to participate in discussions with mining companies and the provincial government, as well as increasing environmental capacity and engagement.

However, while planning and implementing business plans and strategies are essential to develop economic business, identifying these factors as indicators of community readiness may be an over extension. Identifying business plans as an indicator may also be problematic as each initiative would involve a business plan of some sort to identify skills and areas of interest. Most initiatives were able to discuss business plans and some others were able to discuss and implement procurement strategies, yet, these tended to revolve around skill development and engagement. As such, a more valid indicator may be required.

The majority of initiatives have demonstrated the considerable effort required before many communities are ready to even start planning for businesses and procurement. Such activities require a healthy community that is willing to engage in the development activities for the longer term, have certain skills within the workforce and management, and have considerable buy-in and understanding from the communities of how the process works and that it will work for them. In light of this, it may be more accurate to determine community readiness via the previously noted factors, and build on them as initiatives have done, as opposed to developing business plans and procurement strategies.

Finally, it was noted during case studies that for unhealthy communities, economic development can be challenging. Sudden increases in economic development and incomes can increase substance abuse, resulting in lost employment, as well as have detrimental impacts on child-rearing^{Footnote 8 Footnote 9}. Further, increased availability of employment can result in decreases in school attendance, as well as a variety in career preparation outside of skills related to the sector-related economic development (i.e. fishing, mining, forestry, etc.)^{Footnote 10}. Other research has indicated that Aboriginal community leaders, due to a lack of understanding in financial management, may struggle with adequately managing an influx of economic resources, resulting in more long-term harm to the community and a loss of investment^{Footnote 11}.

In light of this, a finding from the case studies suggests that an assessment of current social concerns be considered when assessing economic prosperity, and to potentially put in place programs and/or measures to ensure negative impacts are minimized. This in turn may assist the community to maintain ongoing economic prosperity after SPI is no longer investing in and guiding the economic development of the Aboriginal communities involved.

The evaluation recognizes however that as many of these SPI initiatives are in the developmental stages, there is a great deal of groundwork that needs to be completed before communities are fully prepared to participate in the variety of potential economic opportunities that are available. Overall, however, the evidence indicates that SPI has made progress towards Aboriginal communities being ready to participate in economic development opportunities but more consideration of the performance information being collected is required.

4.1.5 Program Outcome: Federal investments in Aboriginal economic development opportunities are aligned

The achievement of this program outcome was examined, as per the SPI Performance Measurement Strategy, by:

• Evidence of alignment (e.g. development /implementation of comprehensive workplans)

Proposals available for 12 of the initiatives include the articulation of overarching goals, current partners, funding received from SPI, estimated allotment of SPI funds for each planned activity, and the independent initiative governance structure, either planned or in existence.

A review of the process indicates that once proposals are approved, a comprehensive workplan is developed which identifies key players – including federal and non-federal organizations – as well as start up activities (i.e. engagement, training, procurement strategies, and negotiations), which were to be aligned with the overall goal (i.e. new and expanded Aboriginal businesses or other economic development opportunities realized).

Progress reports and summaries reflected the considerable investment and progress made on the beginning stages of workplans. Workplans for those initiatives, approved in 2013, were frequently broken down into their component parts, identified as activities, however, concrete numbers on how many workplans and projects existed are not available. Project summary and expenditure tables provided information for 15 initiatives from 2010-2011 until 2012-2013, two of which were renewed and renamed the following year, and two of which were funded for one year exploratory efforts only.

It is important to note that projects did not have a singular "type", and some incorporated a number of objectives that had some degree of interrelation (i.e. one project may identify any combination of engagement, feasibility analysis, capacity building, research, and procurement activities). Further, as the progress reports and project summaries are reported differently, the following values should be viewed as estimates.

The governance structures developed by the initiatives are designed to help alignment and include interdepartmental working groups and steering committees. Working groups bring together federal and provincial departments, private industry and Aboriginal organizations and help ensure that the actions of these partners are aligned with the objectives for SPI. For example, a document review of Ring of Fire indicates that that the governance structure coordinates the actions and investments of 12 partnering departments and agencies. The Aboriginal Business Development Teams by the Aboriginal Aquaculture Initiative are also a good example. These teams provided ongoing support for investments, and provide business development services to those Aboriginal business people who are interested in using the expertise of the Business Development Teams to improve or build their business. The Labrador Trough initiative also identifies a steering committee comprised of three sub-committees, each with a singular focus related to the end goal of Aboriginal Economic Development. Other initiatives identify that a governance structure has been developed or is being planned and will address and coordinate workplan related activities such as infrastructure-related negotiations, workshops and forums for input on skills and training programs related to the sector.

Generally, document review, interviews, and progress reports suggest that federal investments are aligned; however, there is limited quantitative data available to augment the qualitative and largely anecdotal evidence. While workplans provided some information (including the monies supplied by partners, the estimated SPI monies spent on activities, working group details regarding purpose and focus, and the areas of concentration for departments), this information was not available consistently or with the same level of detail for all initiatives. As such, a thorough analysis that would more clearly identify the degree of alignment with Aboriginal economic development was not available at this time.

Ideally, detailed information regarding the funds transferred from departments and where actual funds were spent (related to activity), should be articulated to have an ongoing measure of alignment.

At this stage, using the development or implementation of comprehensive workplan as an indicator of alignment with departmental objectives may not be appropriate. Specific workplans are reported infrequently, and instead, general details are provided regarding efforts made to increase capacity related to the initiative. Alternatively, reporting requirements may need to be altered in order to address this indicator. Specific details identifying a workplan, with its related activities listed, the roles of each partner, the funding contributed to it, and what needs the activities would address would provide more clear information on the project activities, as well as provide more relevant data on multiple lines of questioning for later assessment.

4.1.6 Program Outcome: Aboriginal communities benefit from federal strategic partnerships and investments

Due to the existing capacity of many Aboriginal communities, many initiatives had to "start from scratch" in terms of preparing Aboriginal communities to be ready to engage in economic development. As such, many communities have benefited from involvement with SPI, although perhaps not yet in terms of specific economic opportunities being realized or in businesses being started or expanded. Rather, for many communities, the benefit is in terms of the development that has occurred to make the community ready to engage in economic development opportunities.

Data review indicates that, in total, 489 individual communities have engaged in the initiatives from fiscal year 2010-2011 to fiscal year 2013-2014. In this time, 103 individual partnerships between communities and private, federal, and non-federal organizations have been established. As of yet, the exact number and value of economic development opportunities realized during this timeframe is unavailable, particularly due to SPI still being in its early stages.

Overall, there are 310 reports of progress towards new or expanded Aboriginal Business and other opportunities in Aboriginal communities. This includes feasibility analyses, negotiations and collaboration for business opportunities, inventory assessments, equipment procurement, supply and opportunity assessments, as well as the development of proposals and business plans, resource development specific to business opportunities, and guidance and support supplied to Aboriginal communities to be competitive in a variety of business opportunities. Each of these activities can be translated into benefits for the Aboriginal communities, as they allow greater knowledge sharing and add to the skills and abilities of the people in these communities, which can be applied to community development or with other opportunities.

Due to many of the communities still being in the early stages of initiative development, specific details on opportunities actualized are not yet available. However, one initiative was able to provide more specific details on new or expanded businesses. The Atlantic Commercial Fisheries Diversification Initiative identified 27 businesses that have been impacted by their initiative, either through expansion or creation. Further, there have also been 250 new jobs created, with 31 existing positions maintained. Other SPI Initiatives would do well to establish information collecting mechanisms that allow for this type of meaningful reporting.

As SPI has contributed a portion of the overall investment in these projects, the exact return on investment for SPI cannot be clearly calculated. However, collectively, the Atlantic Commercial Fisheries Diversification Initiative, the British Columbia Tourism Initiative, British Columbia Mining Initiative, Lower Churchill Hydro Initiative, and the First Nation Power Authority, anticipate combined revenues of an estimated $2.2 billion by 2016.

As discussed above, broader impacts, whether economic or social, are very difficult to measure for a number of reasons. Perhaps the most significant limitation is the ability to attribute any observed changes to the incremental contribution of SPI funding. Presumably, social impacts could be measured using time series measures from the National Household Survey to compare changes over time between "like" communities on economic and social indicators. Specifically, one could examine community-level data on income and labour force at multiple census years (i.e., 1996 to 2015) to see whether any increase is above the trends of other communities (controlling for isolation, region and population) and regressing any changes against the relative size of projects supported by SPI. However, with a myriad of other projects related or unrelated to government economic development investments, and other factors that may impact social and economic development, it is likely that impacts of SPI would be lost in measurement error and the measure of its relative impact obscured.

It is therefore essential for SPI investments to include, in Terms and Conditions or Contribution Agreements, some protocols for the measure of economic and social impacts. Specifically, when a project is funded it should have measures in place to assess job growth; proxy measures of economic growth and measures of long-term viability of business opportunities that are spun off of SPI-supported initiatives. It is also important to do this strategically and purposefully so as not to increase the reporting burden to recipients or participants. Thus, consideration should be given to emphasising reporting requirements for performance and reducing other compliance or administrative reporting requirements that may not be necessary.

Improvements/alternatives identified

The evaluation found that SPI is operating efficiently given the unique nature of the program and its objective. As a result, no specific delivery alternatives were proposed to achieve greater efficiency. However, as with most programs, organisations experience significant pressure with delays in funding.

Another important suggestion was that the SPI governance structures engage Aboriginal stakeholders more directly. This suggestion is in line with a recommendation made by Douglas R. Eyford in his report to the Prime Minister Forging Partnerships Building Relationships: Aboriginal Canadians and Energy Development on the importance of early political engagement with First Nations in resource development. The need for better communication was also highlighted, especially around changes to the ways projects are selected and approved and by improving the information flow to communities to deepen their understanding of initiatives and potential impacts.

Lessons Learned

Through key informant interviews, multiple project proponents commented that they experienced unexpected delays due to the time needed for community engagement. Specifically, the time needed to properly engage communities and gain their support for projects was substantially longer than originally anticipated and led to multiple projects falling behind schedule. As a result, project proponents had difficulty spending the allotted funding within the time period. In response to this issue, interviewees suggested multiple potential solutions, including: increasing awareness among SPI applicants regarding the time needed for effective community engagement; increasing the flexibility around the re-profiling of funds to allow programs that have fallen behind schedule to move funds to later years of the project; providing funding for community engagement in addition to other aspects of SPI projects.

As noted earlier in this report, SPI has encountered challenges in establishing and maintaining the involvement of other federal government departments and regional development agencies. In specific cases, senior management from AANDC have reached out to other departments to provide information and education regarding the purpose of SPI and the opportunities the program presents. In such cases, this outreach was found to be effective and further outreach could encourage senior management throughout the federal government to increase their participation in SPI. Consideration should be given to ensuring the timelines for projects are reasonable to allow projects to achieve intended results. SPI is intended to fill gaps in funding and not provide ongoing funding for continuing projects, although some successful projects have resulted in becoming "mainstreamed" and incorporated into the budgets of participating departments, however, this is generally not the intention of the SPI Initiative. For example, in the case of Atlantic Commercial Fisheries Diversification Initiative, the initiative was put in place to respond to a short-term market issue resulting from a reduction in snow crab quota and called for a strategy/investments to diversify the industry. At the end of the three year initiative, the impacts on the community owned businesses operating in the sector were mitigated, and as result, the program was deemed to be a success, in spite of the program ending.

Best Practices

The project governance structure utilized by the Atlantic Commercial Fisheries Diversification Initiative was identified as a best practice. The initiative built upon an ongoing program operated by the Department of Fisheries and Oceans Canada in order to develop a governance structure, which provided assistance to First Nations, applied a rigorous review process to determine eligible projects and above all was trusted and valued by First Nations participants. In this case, business development teams, comprised of subject matter experts, were established in the Ulnooweg Development Group to assist communities in the development of business plans and proposals. A review committee was established to determine what projects would be funded and included representation from two federal departments and two Aboriginal organizations. The decisions made by the committee were informed by the project proposals as well as a proposal review conducted by an independent third party evaluator. Finally, project implementation teams, including community members and members of the business development teams, were established to assist with the establishment and operation of the newly formed businesses. The partnerships formed with regional Aboriginal organizations, the formation of business development teams and project implementation teams provided communities with support to ensure their projects were implemented successfully.

The success of the Atlantic Commercial Fisheries Diversification Initiative reinforces the opinions of interviewees who noted that engaging communities in consultative processes to determine priorities and plan strategically for capacity building and economic development was a best practice. Additionally, investing and partnering with regional Aboriginal organizations was also seen as a best practice, as opposed to attempting to build expertise in each community.

The horizontal nature of SPI and the promotion of partnerships were identified by multiple interviewees as a best practice itself. The Government often encounters challenges as departments operate in "silos" and SPI represents a unique approach to encourage collaboration to ensure Aboriginal participation in large and complex economic development opportunities.