Audit of Economic Development Programs

Delivery of the Aboriginal Business Development Program (ABDP)

As of fiscal 2013-14 AANDC has endeavoured to completely transition away from the direct delivery of the Aboriginal Business Development Program (ABDP) funding to Aboriginal entrepreneurs, businesses, and Aboriginal community enterprises. The new 2013-14 model is designed to have numerous Aboriginal Financial Institutions (AFIs), located regionally across the country, delivering program funding to recipients on behalf of the Department. AANDC is responsible to provide the program funding to the AFIs, referred to as Program Delivery Partners (PDPs), which are then responsible to provide the targeted funding to recipients according to program terms and conditions. AANDC provides the PDPs with program coordination and support through seven regionally located Institutional Development Officers. The Department has a further plan to delegate the ABDP support and program coordination role to the National Aboriginal Capital Corporation Association (NACCA) in 2014-15. This transition would have NACCA distributing the program funding to PDPs for the Aboriginal Developmental Lending Allocation initiative, and providing the coordination and management of program delivery. Based upon NACCA’s performance, the Aboriginal Business Flexible Financing initiative may also be transferred to NACCA in 2015-16.

Economic Development Program Suite Renewal

The LED Sector and regional LED Directorates are operating in an environment characterized by considerable change due to the Deficit Reduction Action Plan (DRAP) and related organization structure, and Economic Development program suite renewal. The audit team took into consideration these upcoming program change initiatives.

In 2009, the Government of Canada released the Federal Framework for Aboriginal Economic Development to better support the full participation of Aboriginal Canadians in the economy. As part of an Action Plan to implement the Framework, AANDC has assessed how well the current suite of Aboriginal lands, environment and economic development programs align with Framework priorities. Three Branches of the LED Sector - Community Opportunities, Aboriginal Entrepreneurship, and Lands and Environmental Management – worked together to review the existing authorities and consolidate them into three program areas under the one authority. The three programs will comprise the following:

• Community Opportunity Readiness Program (CORP) intended to provide project-based funding for First Nation and Inuit Communities for a range of activities to support communities’ pursuit of economic opportunities. Where First Nation and Inuit communities, or their representative organizations, identify opportunities and lack the financial resources to pursue them, CORP will allow them to submit proposals to obtain additional funding;
• Lands and Economic Development Services Program (LEDSP) intended to provide support to First Nation and Inuit Communities to assist with enhancing the economic development, land and environmental capacity of communities and to support the establishment of the conditions for economic development to occur, increasing their participation in the economy; and
• Aboriginal Entrepreneurship Program (AEP) intend to increase the number of viable businesses in Canada owned and controlled by Aboriginal Canadians, provide a supportive business environment for Aboriginal Canadians, support partnerships with various public and private stakeholders and advocate and inform employers about the hiring of Aboriginal peoples.

While this process will realign the organization of programs, many will retain the current approach, design and delivery. For example, core program funding (currently CEDP) will continue to be based on a community funding formula, and the targeted funding will be continue to be proposal/project based.

The following chart depicts how the existing programs included in this audit will be consolidated into one of three programs under the new authority:

Strategic and Operational Planning

The audit noted that EDP governance and oversight was supported through annual program (CEDP, CEOP and CSSP) planning and budgeting, regular management meetings and quarterly reporting of EDP program (Headquarters and Regional) activities and achievements.

Planning at the Program (CEDP, CEOP and CSSP) and regional level was noted consistently for allocation of resources for the in-scope period and across regions and Headquarters.

The audit also noted a good management practice in British Columbia (BC) Region, where LED directorate and economic development unit annual business plans are being completed. These plans identified priorities, key activities and performance targets that were monitored each quarter. These regional directorate and unit plans were aligned with the overall priorities and objectives of the Economic Development programs for LED Sector nationally to help ensure the effective and efficient achievement of objectives. Employee Performance Agreements goals can be prepared with improved alignment to the Directorate and Unit plans to efficiently and effectively align accountabilities.

A strong management practice noted in BC Region included a strategic planning document that aligned available economic development programs to three tiers of community economic development: Tier 1 – Nation Building (CEDP, CSSP) activities include: EDO training initiatives and other capacity building; Tier 2 – Asset Development (CEOP, ABDP, FNLM) activities include economic development plan, strategic plan, land use plan, feasibility studies; and Tier 3 – Access to Capital (Aboriginal Financial Institutions (AFI), ACC, FNFA) activities include equity gap financing, strategic partnerships, capital and business start ups. This approach helped to clarify the Regional strategy and priorities and related activities for the year.

The audit noted that each Region consults with First Nation communities on community development priorities by leveraging a Lands and Economic Development Advisory Committee (LEDAC) or similar advisory body. Management has indicated that these advisory bodies are effective at helping align the EDP priorities of the First Nations communities with those of the Department.

Organization Structure, Accountability and Communication

The audit noted that Regional offices are structured to include the Lands Unit and the Economic Development Unit under one Directorate, each with different management responsibilities, but still having underlying interdependencies. This structure facilitated and supported communication between the Lands and Economic Development Units, while ensuring clear accountability and responsibility for delivery of EDP. Effective communication between land management officers and economic development officers is important since development opportunities are often linked to reserve land as the key asset. The audit noted a variety of management practices across Regions to facilitate this connection and communication. Practices included: co-location of Lands and Economic Development Units; staffing Economic Development Units with land management (Commercial Leasing, Designations) staff; the formation of community teams (including LED and non-LED staff); standing internal management and project meetings between economic development and land management managers and staff; and both formal and ad hoc reporting of issues between economic development officers and lands management officers.

The audit noted LED Sector staff roles and responsibilities for the EDP programs were assigned and communicated through performance management agreements and were consistent with EDP objectives.

In addition, a management control framework was established with clear and communicated roles, responsibilities, accountabilities and delegation of authority. Each Region was consistently structured with a LED Director reporting to the Regional Director General. The Directorate in each Region was supported by Manager positions (both Lands and Economic Development) and Economic Development Officers supporting the management framework.

Management and Oversight Bodies

The audit found that management and oversight bodies exist, which meet regularly, where EDPs can be discussed and issues can be raised. AANDC's Director General Implementation and Operations Committee (DGIOC) provides the key national forum where Director Generals and Regional Director Generals can raise EDP related issues. Regions have regional operations committees to discuss regional program issues. Finally, a regular conference call is conducted that includes all regional LED Directors and representatives from the LED Sector where regions can discuss program delivery with each other, and Headquarters.

Performance Measurement

The audit expected that the LED Sector would have identified appropriate performance measures linked to planned results, and monitored the results of performance to evaluate the effectiveness of EDPs. Regional quarterly reporting included some measures, such as the number of projects funded and the amount of funding for CEDP, CEOP and CSSP.

The audit found that the performance measures currently in place for community based programs (CSSP, CEDP, CEOP) do not adequately demonstrate the impact of the programs on communities. The performance measures currently reported focus on the number of projects approved and the value of funding provided. As noted in the Community Economic Development Program Activity Report for 2012-13, the vision of these programs is to help Aboriginal people to pursue "opportunities for employment, income and wealth creation". Effective performance measures should be in place to monitor and report progress towards this vision. It is understood that LED is in process of revising its Performance Measurement Strategy.

Beginning 2013-14, the delivery of ABDP was transferred from AANDC to AFIs who were successful applicants to be Program Delivery Partners (PDPs). AFIs are required to report on a number of program management performance measures, such as, program activity (number of enquiries, applications received, applications approved), disbursements and cost per case. ABDP has also established strong Performance Measurement Indicators at the project/business level. There is a requirement for AFIs to track, and collect this performance data and provide it to AANDC. As this program delivery design is new in 2013-14, the audit was unable to conclude of the effectiveness of these performance measurement processes. Beginning 2014-15, LED Sector is planning to transition the coordination of ABDP (the current role of AANDC) to NACCA. At the time of the audit, there had not been a formal determination of how performance data of funded projects/businesses by AFIs will be collected and communicated to AANDC. The ABDP provides financing, business support and information to increase the competitiveness and success of Aboriginal businesses in Canadian and world markets. Effective performance measures should be in place to help track and monitor the level of success achieved over time regarding these program objectives.

Recommendation:

1. The Assistant Deputy Minister of the Lands and Economic Development Sector should, as part of revising its Performance Measurement Strategy, design and implement key performance measures and indicators, which provide metrics to objectively evaluate and demonstrate the impact of community-based economic development programs as a key input into the ongoing assessment of the design and delivery of these programs. For the Aboriginal Business Development Program, LED should clarify how project performance data will be collected by third party Program Delivery Partners and communicated to AANDC.

Regional and Project Level

It was noted that risk management is being conducted at the project and regional level. The following examples were noted:

• During the auditing of a sample of individual funded projects, general assessments completed by AANDC were reviewed as part of the project proposal due diligence completed by the Region. The general assessment is completed for each First Nation community and considers risk factors including governance, performance history, financial stability, project complexity and other considerations.
• Each project submitted to Regions for consideration includes a Project Assessment and Summary Report that includes a risk assessment (section 13) that is completed by the Regional Economic Development Officer. The nine risk factors are both qualitative and quantitative and are assessed using a point rated scale with the sum used to derive low, medium or high risk.
• Regional risk assessments were reviewed and noted to be included as part of Regional annual planning and quarterly reporting processes. Most of the regions visited also had included the Regional Risk Profiles as part of the Quarterly Report. The risks include the key regional corporate risks and related regional planning commitments for all programs. An example noted in one Region aligned the regional corporate risk - Government Partnership Risk ("ability to effectively and efficiently work with partners and stakeholders in the development and delivery of policies and programs") to the Regional planning commitment ("Community Development Framework").

National Program Level

Risk management at the project level by LED Sector involved assessment of the risk of each project and determination of type of funding arrangement and amount of monitoring that will be undertaken. As well, a national prioritization framework to ensure transparency of program funding decisions for larger economic opportunities was created to manage the client relationship risk. However, formal creation of a formal risk profile for the program through a facilitated process has not been undertaken for existing programs. The audit expected this to be in place for the EDPs as this is a good management practice and a gap that was noted by a previous audit^{Footnote 1}. Without this consideration of risk, program objectives may not be met due to circumstances occurring (risks) that are not adequately controlled or mitigated and/or risk is not considered in the prioritization and reprioritization of program activities.

Recommendation:

2. The Assistant Deputy Minister of the Lands and Economic Development Sector should include, as part of its annual planning for these programs, identification and assessment of Economic Development risks and determine appropriate mitigation measures required to further support the achievement of program objectives. Where possible and appropriate the program risk work already being done in regions should be leveraged to support the efficiency and effectiveness of this assessment.

Training and Innovation

The audit team found there were appropriate training, tools, resources, information and guidance available to support an effective program design and delivery in-line with EDP.

The management and delivery of the community programs has not been changed over the audit scope period. The audit noted a number of tools and guidance including a training manual, management control regimes and example terms of reference for targeted programming. The approach to training is largely a combination of mentoring, coaching and on the job training. This approach was consistently noted in regions to be effective.

The audit also noted that changes related to the EDP suite renewal and consolidation of programming initiative will impact programming starting 2014-15. LED Sector has drafted tools and resources to assist in the roll-out of the initiative. Examples of these resources include program area descriptions, updated Management Control Frameworks, updated application and reporting templates, and updated program logic models.

For ABDP, the audit noted the development of supporting program documentation for the PDPs that outline program terms and conditions, along with the roles and responsibilities of the various stakeholders.

Strong management practice noted in two Regions (BC and Ontario): Regions have emphasized the use of funding (CEDP, CEOP) by communities to develop 5 year Strategic plans (e.g. strategic economic development plans, land use plans, and community plans). The audit found the emphasis on the development of 5-year strategic plans (economic development, land use) by First Nation communities to be a strong practice for strategically managing Economic Development funding resources. Benefits noted include:

• identification of longer term objectives and planning for economic development projects;
• understanding of projects that are "shovel-ready" which helps with resource re-allocation as circumstances change over time; and
• ability to track individual projects that culminate in economic development over time (e.g. feasibility study to project design to project completion).

The other two Regions were either considering a similar shift to a longer term planning timeframe or were continuing with the annual focus. The audit recognizes benefits and encourages the LED Sector management to consider consistent focus on community strategic plans as a key priority as program re-alignment is rolled-out.

Management of ABDP: Compliance and Quality Assurance

The delivery of the ABDP was transferred from AANDC to PDPs during 2013-14. In all regions visited, the due diligence process to select participating PDPs followed the readiness evaluation and approval process set out by Headquarters. The expectation was that PDPs would take on the management and administration for all ABDP funding. It was noted, however, that in one region the only qualified PDP agreed to manage only a portion of ABDP recipients, and excluded community owned businesses. The management of this ABDP component in the region remained the responsibility of regional staff.

The audit expected clear and communicated compliance requirements for the ABDP, which were not present for the Program. Compliance is a requirement of the Funding Agreement Terms and Conditions between AANDC and the PDP and for any ABDP funding that remains with AANDC (as noted in one region). The regions visited consistently noted that they have not completed any compliance activities of the PDPs. In instances where third party service providers are used to perform program operational activities, the Department is expected to implement a regime that provides assurance over controls exercised by the third party so that the Department has comfort that program funds are being expended according to program terms and conditions. It is expected that Headquarters would provide the expectations for compliance to help ensure consistency across regions.

The audit noted that the ABDP program will be changing again with AANDC delegating to NACCA the management of funding to, and coordination of, the PDPs who deliver ABDP. Management has indicated that as of April 1 2014, NACCA will be distributing the program funding the Aboriginal Developmental Lending Allocation initiative. Based upon NACCA's performance, the Aboriginal Business Flexible Financing initiative, which represents the majority of ABDP's budget, may be transferred to NACCA in 2015-16.

As the program goes through these considerable changes, compliance requirements need to be clarified and implemented.

Recommendation:

3. The Assistant Deputy Minister of the Lands and Economic Development Sector should develop a quality assurance review and compliance framework and practices to provide AANDC with assurance over the effectiveness of Program Delivery Partner program management processes and controls in the delivery of the Aboriginal Business Development Program. The compliance framework and practices should be adjusted to ensure AANDC continues to receive assurance over program delivery once the National Aboriginal Capital Corporation Association takes on the Department's program coordination role.

Results of Testing Program Files (CEDP, CEOP, CSSP and MRED)

A judgemental sample was selected for each of CEDP, CEOP, CSSP and MRED. The audit tested key management control activities that were expected to help ensure the due diligence of CEOP, CEDP, CSSP and MRED project proposal and planning, funding approval, project/ program compliance and monitoring. Evidence was gathered through a combination of examination of documents retained by each Region and through discussions with key regional staff. Management controls over EDPs, including due diligence of funding plans, funding approval, project/ program compliance and monitoring appear to exist and are generally effective and efficient.

The audit found that generally processes and key controls were adequate and effective. The audit team tested 24 CEDP files, 24 CEOP files, 11 CSSP files across 4 regions, along with 2 MRED projects and 1 MRED Funding Arrangement. The audit team was looking for documentation to support the due diligence, approval, compliance and monitoring. The audit found that project proposal and project summary assessment, or similar documents, were received and filed, and displayed formal approval. Section 34 (Financial Administration Act) payment approvals were appropriate, and retained in GCIMS in most cases. No exceptions were noted for compliance and monitoring activities. Testing found that the majority of reporting requirements were received, reviewed and approved. In the few instances where reports were noted to be missing or late, such as Audited Financial Statements, subsequent funding is halted until outstanding reporting is received.

Regions did not note any concerns regarding capacity of delivering targeted, proposal-based funding compared to core formula-based funding. The funding budgeted for targeted and core programs was consistently issued during the year across all regions visited.

Information Management

The audit expected that information would be captured and categorized leveraging the Grants and Contributions Information Management System (GCIMS) as the "system of record", and that a minimum standard for program documentation would be met in aggregate. Test results varied in the degree to which GCIMS was used as the "system of record" for each program. While testing found that file documentation and information was generally complete, information tended to be stored in a number of systems and formats, including GCIMS, Comprehensive Information Document Management system (CIDM), spreadsheets, and hard copy files. The completeness of documentation and evidence captured in GCIMS, particularly for project based funding, was not consistent between programs and between Regions. Of particular note, the use of the GCIMS "Proposal Management module" as a repository of key project documents (proposal/applications, proposal review and approval, project deliverables) varied considerably across programs and regions, from considerable use, to very little.

The audit also noted during file testing that in one region visited, evidence of S.34 program payment approvals for the 2011-12 and 2012-13 sample files tested were not being uploaded into GCIMS in order to evidence that formal approval was executed. The audit team did, however, note that this region has since adopted the practice in 2013-14, and these approvals are now being captured in GCIMS.

• Key documentation in GCIMS should include: Project applications, proposals, proposal review and approval, project reporting and key deliverables, and evidence of formal S.34 program payment approval (scan of batch S.34 payment approval)
• Key evidence should always include: Documentation that evidences why project funding exceeds approved proposal amount.

The management and administration of the EDPs is improved through the use of GCIMS as the common system of record. Benefits resulting from consistent use of GCIMS can be expected to include: national accessibility for programs and projects, completeness of reporting requirements is demonstrated and tracked, and facilitated monitoring of compliance with terms and conditions.

Recommendation:

4. The Assistant Deputy Minister of the Lands and Economic Development Sector should, as program realignment is rolled out and roles and responsibilities are clarified, emphasize the use of Grants and Contributions Information Management System as the system of record, and clarify the need for complete documentation, to support the efficient and effective management of AANDC economic development programs.