Archived - Audit of Implementation of the Urban Aboriginal Strategy

Date: May 2010
Project #09/62

PDF Version (167 Kb, 36 Pages)

Table of Contents

• Initialisms and Abbreviations
• Executive Summary
• 1.0 Statement of Assurance
• 2.0 Introduction
• 3.0 Audit Objective and Scope
• 4.0 Approach and Methodology
• 5.0 Conclusions
• 6.0 Observations
• 7.0 Recommendations
• 8.0 Management Action Plan

Initialisms and Abbreviations

Executive Summary

Background

The Government established the Urban Aboriginal Strategy (UAS) in 1997 to address the challenge of growing numbers of Aboriginal people living in urban centres through improved federal coordination and greater intergovernmental cooperation. At the time, no funding was provided to support community-level projects. However, in 2003 the UAS was allocated $50 million over a four-year period to further the original strategy and to introduce an INAC-managed contribution program aimed at funding projects in a small number of pilot cities. In March 2007, the UAS was renewed for an additional five-year period with annual funding of $13.7M per year.

The objective of the UAS is to promote self-reliance and increase life choices for Aboriginal people in urban centres. In order to accomplish this, UAS funded projects are to strategically focus investment in three priority areas, namely, improving life skills; promoting job training, skills and entrepreneurship; and supporting Aboriginal women, children and families.

The Office of the Federal Interlocutor (OFI) is responsible for the implementation of the UAS. Prior to 2006/07, OFI directed the UAS from its National Capital Region (NCR) office, with on-the-ground service delivery provided through regional offices of Western Economic Diversification Canada and Service Canada. OFI assumed full responsibility for all aspects of the UAS in April 2007, whereupon it established regional offices in Ontario and the four Western provinces. The contribution program is managed and delivered by staff in five OFI regional offices to the 13 urban Aboriginal communities currently targeted by the UAS. Program direction, including policy development and integration with other federal departments, continues to be led by OFI's NCR office.

Objective

The objective of our audit was to provide assurance on the adequacy and appropriateness of the UAS management control framework to support achievement of program outcomes and to ensure that funds are being used for the intended purpose.

The scope of the audit included OFI management processes designed to support UAS implementation, administration of UAS contribution funding, and controls for supporting Community Committees. Our audit did not include assessments of the effectiveness of OFI's controls designed to support collaboration with other federal departments and its coordination with provincial and municipal partners; as this was viewed as overlapping with the objective of an evaluation planned for 2010/11.

Findings and Conclusions

Overall, we found that the Urban Aboriginal Strategy is generally well managed, with some improvements required in how the strategy is implemented, particularly with respect to the management and administration of the UAS contribution program.

Our audit found that UAS policy advice is supported by thorough analysis and that the program delivery model is generally effective in ensuring that funds are spent for the intended purpose. The commitment and personal involvement of all levels of management is clear and the existence of strong direction on strategic aspects of the UAS is evident. While the implementation of the strategy is not without its challenges, OFI proactively identifies areas for improvement and employs an approach of continuous improvement as it gains experience and a better understanding of both inherent challenges of the delivery model and community specific challenges. The most significant challenge identified during our audit is the need to afford communities with flexibility in how contribution funds are allocated with the need for OFI to ensure that the overall UAS priorities are achieved and that policy compliance is achieved.

We identified the need for well defined and defendable community priorities as a key control for ensuring that contribution funds are spent in an accountable manner and that an appropriate balance between flexibility and control is achieved. More specifically, we believe that community priorities should be sufficiently detailed and supported by analysis which demonstrates that they are truly reflective of the issues facing urban Aboriginal people in their respective UAS community. Our audit identified some of the more established UAS communities as having thoroughly documented priorities and plans, while some of the newly added communities have little documented support for their priorities and inadequate plans. In essence, we found that OFI needs to more clearly define its minimum expectations of community plans and priorities and assure itself that these are met by a community prior to agreeing to provide maximum flexibility in the types of projects and expenditures funded.

Our audit identified several other areas for improvement which are reflected in the recommendations that follow.

Recommendations

Our audit provides a number of recommendations intended to address the audit findings.

1. OFI should review the administrative regime it uses to fund new and developing UAS communities. Specifically, consideration should be given to using a risk-based approach that associates the types of projects and expenditures it funds in a community to the ability of the Community Committee to address the issues of urban Aboriginal people in an accountable manner.
2. OFI should establish a consistent expectation for community plans by developing a template that includes the type and level of information required and by providing planning support and guidance as needed. This would ensure that both community and national UAS priorities are addressed, that a shared vision of community success is defined, and that concrete steps are established to achieve objectives and measure progress.
3. When OFI seeks renewal of the UAS spending authority and Terms and Conditions with its federal partners, it should:
   • Having regard for the flexibility intended with the UAS contribution program and the strategic priorities of the UAS, ensure clarity as to the nature of eligible projects, activities and expenditures;
   • Set out expectations for leveraging with provinces and municipalities, how results should be measured, and how leveraging requirements should apply to new and developing communities; and
   • Clarify the definition and approval requirements for capital expenditures.
4. To support consistent understanding of the flexibilities built into the UAS contribution program, OFI should provide guidance to regional staff and Community Committees on eligibility requirements and use of project and recipient risk information to substantiate funding decisions.
5. Annual work plans should be established by UAS HQ and each region. These work plans should establish objectives, regional management and program priorities, planned actions and timelines, targets for recipient and community service standards, and expected results (program priorities and expected results should align with the UAS Performance Measurement Strategy to be developed and implemented following the completion of the summative evaluation which is currently underway). The regional plans should be approved by HQ and progress tracked through a quarterly reporting process.
6. OFI should more fully develop its training regime for Regional Development Officers to ensure that it adequately addresses the dual nature of this role, that being the requirement to support the Community Committees in discharging their responsibilities and the requirement to manage and administer UAS contribution agreements.
7. To ensure efficient and effective processes for the review and approval of projects and the establishment and administration of contribution agreements, OFI should revamp the processes and timelines for soliciting proposals, reviewing project eligibility, preparing and executing funding agreements, processing recipient payments and closing out project files. Service standards should be established and achievement against these standards should be tracked and monitored by OFI HQ on a regular basis.
8. OFI should more fully define the purpose of the Central Fund, as well as the criteria, policies and procedures used in the solicitation, submission, and assessment of Central Fund projects.

1.0 Statement of Assurance

We have completed the Audit of Implementation of the Urban Aboriginal Strategy as managed by the Office of the Federal Interlocutor (OFI). The objective of our audit was to provide assurance on the adequacy and appropriateness of the Urban Aboriginal Strategy (UAS) management control framework to support achievement of program outcomes and to ensure that funds are being used for the intended purpose.

The audit was conducted in accordance with the requirements of the TB Policy on Internal Audit and followed the Institute of Internal Auditors' Standards for the Professional Practice of Internal Auditing.

The scope of the audit included OFI management processes designed to support UAS implementation, administration of UAS contribution funding, and controls for supporting Community Committees. The audit assessed the UAS contribution program controls against the Audit and Evaluation Sector (AES) Grants and Contributions Program Audit Criteria, established in 2007 through consultation with program and regional managers from across the department.

In my professional judgment as Chief Audit and Evaluation Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the conclusions reached and contained in this report. The conclusions were based on a comparison of the situations, as they existed at the time of the audit and against the audit criteria. It should be noted that the conclusions are only applicable for the areas examined.

2.0 Introduction

AES indentified Implementation of the Urban Aboriginal Strategy as an audit project for 2009/10 within its 2009-2012 Risk Based Audit Plan. The UAS was excluded from the Audit of the Office of the Federal Interlocutor in 2007/08.

2.1 Background on the Strategy

The Federal Government established the UAS in 1997 to address the challenge of growing numbers of Aboriginal people living in urban centres. The aim of the strategy is to improve federal coordination, establish greater intergovernmental cooperation, and improve community engagement and the participation of other stakeholders. Recognizing that urban aboriginal communities have traditionally been forced to acquiesce to the policy frameworks of governments, the UAS attempts to meet the unique demands of each community in order to create new relationships with urban Aboriginal peoples and communities.

When the UAS was first established, no OFI contribution funding was provided to support community-level projects. However, in 2003 the UAS was allocated $50 million over a four-year period to further the original strategy and to introduce a contribution program aimed at supporting projects in a small number of pilot cities. Based on what was learned, the UAS was renewed in March 2007 for an additional five-year period with annual funding of $13.7M per year (including $10.0 M per year for contributions to recipients).

The objective of the UAS is to promote self-reliance and increase life choices for Aboriginal people in urban centres. In order to accomplish this, UAS funded projects are to strategically focus investment in three priority areas, namely, improving life skills; promoting job training, skills and entrepreneurship; and supporting Aboriginal women, children and families.

OFI is responsible for the implementation of the UAS. Prior to 2006/07, OFI directed the UAS from its National Capital Region office (headquarters or HQ), with on-the-ground service delivery provided through regional offices of Western Economic Diversification Canada and Service Canada. OFI assumed full responsibility for all aspects of the UAS in April 2007, whereupon it established regional offices in Ontario and the four Western provinces. Presently, the strategy is managed and delivered by staff in five OFI regional offices to urban Aboriginal communities in 13 cities. Program direction, including policy development and integration with other federal departments, continues to be led by OFI HQ.

The UAS is designed to increase coordination within the Government of Canada to maximize its investments and enable greater federal program alignment with provincial and municipal programming. Through sustainable partnerships with provinces, municipalities, Aboriginal organizations and Aboriginal communities, the UAS aims to reduce the level of disparity that urban Aboriginal people face by identifying and addressing the unique needs and priorities of each community.

2.2 Community Based Approach

While broad national priority areas have been established for the strategy, flexibility at the community level in the form of regionally responsive strategies is foundational to the success of the UAS. Management and delivery of UAS projects is to be accomplished through a community based approach in each of the 13 designated UAS cities. Two options for program delivery are available to UAS communities. One is the Community Entity Model in which a single contribution agreement is reached with an incorporated community organization that is responsible for managing and administering project funding on behalf of the community. The other is the Shared Delivery Model in which a UAS steering committee recommends funding allocations to OFI and OFI enters into separate contribution agreements with project proponents and manages and administers those agreements.

Currently, all but one of the UAS targeted urban Aboriginal communities operates under the Shared Delivery Model. With this model, Community Steering Committees (Community Committees) are established, composed of a cross section of individuals from the community and including representatives of the federal government and other levels of government. The primary role of Community Committees is to:

• Identify and prioritize areas of concern within the urban Aboriginal community;
• Develop and implement community strategic plans;
• Solicit and review project proposals submitted by recipients in the community; and
• Assess these proposals against the defined community priorities and plans and recommend projects to OFI.

2.3 Horizontality and Alignment

The UAS strategic objectives of improving federal coordination and achieving greater intergovernmental cooperation are generally referred to by OFI as achieving horizontality and alignment, respectively. Horizontality represents the need for OFI, through the UAS, to promote greater coherence among federal departments by exploring models of enhanced horizontal collaboration, drawing together federal programming. Federal funding collaboration is meant to reduce program duplication and the tendency of programs to be provider-centric rather than effective, coordinated responses to community realities. The UAS Program Terms and Conditions, as amended in March 2008, allow for six other federal departments to use the Terms and Conditions of the UAS where the project or initiative is consistent with the mandate of that department, is community based, is beyond the mandate of any one department, and is consistent with the objectives or goals of the UAS.

Alignment represents the need for OFI, through the UAS, to coordinate federal expenditures directed toward urban Aboriginal issues in UAS cities with provincial and municipal programming, in a manner that both advances federal objectives and effectively responds to local challenges and opportunities. The UAS policy authority established that, for communities to be designated as eligible for the UAS, there must be "real and demonstrable commitments (financial or in kind) from provinces and municipalities on a 50-50 community-by-community basis as partners". The UAS Program Terms and Conditions designate an initial group of 13 urban Aboriginal communities as being eligible and clarify that any new communities added by OFI meet the same requirements established in the policy authority. OFI has not yet designated any new communities as being eligible.

3.0 Audit Objective and Scope

The objective of our audit was to provide assurance on the adequacy and appropriateness of the UAS management control framework to support achievement of program outcomes and to ensure that funds are being used for the intended purpose.

The scope of the audit included OFI management processes designed to support UAS implementation, administration of UAS contribution funding, and controls for supporting Community Committees. Our audit did not include assessments of the effectiveness of OFI's controls designed to support collaboration with other federal departments and its coordination with provincial and municipal partners; as this was viewed as overlapping with the objective of an evaluation planned for 2010/11.

In assessing the controls of the contribution program, the auditors employed the standard audit criteria developed by Audit and Evaluation Sector (AES) for auditing grants and contribution programs. The AES audit criteria for auditing grants and contributions are a set of 37 audit criteria that focus on 7 areas, including: program design and approval; program implementation; program monitoring and reporting; program renewal/close-out; recipient eligibility/evaluation processes; contribution agreement development processes; and contribution agreement monitoring and reporting.

As noted in section 4.0 below, our audit included transaction and document vouching and review in relation to a sample of UAS funded projects in two regional offices and at HQ. The projects reviewed were selected based on a sampling strategy developed during the planning and risk assessment phase and were from the 2008/09 fiscal year. While the standard audit criteria developed by AES for INAC programs were designed to assess management controls in support of contribution programs, they were not sufficient to address the uniqueness of the UAS as a strategy for improving federal coordination and greater intergovernmental cooperation. Accordingly, additional audit tests were designed to address the more unique aspects of the UAS.

4.0 Approach and Methodology

Our audit was conducted in accordance with the requirements of the TB Policy on Internal Audit and followed the Institute of Internal Auditors' Standards for the Professional Practice of Internal Auditing. Sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the opinions provided and contained in this report.

During the planning phase, interviews were conduct with staff at OFI HQ and with Directors from two regional offices. Documentation relevant to the UAS was also reviewed and analyzed. This work was performed in support of the preliminary risk assessment process. The first step in this process identified the significance of each control activity (in relation to each audit criteria), considering both the financial materiality and the importance of the control in supporting an effective control system. The second step assessed the level of residual risk associated with each control activity. A risk score was then assigned to each control activity in order to confirm the audit criteria for inclusion in the audit and to determine the type and nature of audit procedures to be performed in the conduct phase. The planning and risk assessment phase concluded with the completion of the detailed risk assessment, the audit strategy, and the audit program for the audit.

The conduct phase of the audit included the completion of audit procedures at both regional offices as well as at OFI HQ. The principal audit procedures completed by the audit team included:

• Documentation Review – thedocumentation that was subject to examination included, but was not limited to,policy and program authorities; key UAS documentation such as the Integrated Results Based Management and Accountability and Results Based Audit Framework, program policies and procedures including funding and process flowcharts; community specific documentation such as community plans and Community Assessment Tools (CAT); quarterly reports; annual departmental reports; and past UAS evaluation and review reports.
• Transaction and Document Vouching – file documentation was examined for a sample of 34 recipient projects. Project files were selected from the Alberta and Manitoba regions, as well as from HQ region. A grant and contribution testing grid was developed to assess the compliance of projects in relation to relevant audit criteria, UAS Program Terms and Conditions, and the TB Policy on Transfer Payments (2000 and 2008). Audit procedures in relation to the recipient files sampled also included the activities of the OFI Administration Unit at HQ responsible for preparing funding agreements, initiating and issuing payments, and closing out project files. Various discussions were held with a representative from Funding Services Support and Audit Review of INAC's Chief Financial Officer Sector to complete this work.
• Analytical Procedures – financial information in regard to all payments issued under the UAS in fiscal 2008/09 was obtained from INAC's OASIS financial system. These data were analyzed to support audit fieldwork and audit sampling. In addition, analysis was performed in relation to performance and financial information as accumulated in the UAS Project Information Management System (PIMS).
• Regional Questionnaires – a questionnaire was provided to and completed by Regional Directors responsible for a sample of 7 UAS communities as a means of validating specific information on how the Community Committees operate and how OFI supports the committees. The questionnaire also included specific inquiries in regard to the achievement of the UAS requirements of horizontality and alignment.
• Interviews – interviews were conducted with management and personnel responsible for UAS delivery at HQ and in the regions visited. Interview guides were developed for interviews conducted, taking into consideration the objective of the audit and the audit criteria in use.

Audit fieldwork was substantially completed between November 2009 and February 2010.

5.0 Conclusions

Overall, we found that the Urban Aboriginal Strategy is generally well managed, with some improvements required in how the strategy is implemented, particularly with respect to the management and administration of the UAS contribution program.

Our audit found that UAS policy advice is supported by thorough analysis and that the program delivery model is generally effective in ensuring that funds are spent for the intended purpose. The commitment and personal involvement of all levels of management is clear and the existence of strong direction on strategic aspects of the UAS is evident. While the implementation of the strategy is not without its challenges, OFI proactively identifies areas for improvement and employs an approach of continuous improvement as it gains experience and a better understanding of both inherent challenges of the delivery model and community specific challenges. The most significant challenge identified during our audit is the need to afford communities with flexibility in how contribution funds are allocated with the need for OFI to ensure that the overall UAS priorities are achieved and that policy compliance is achieved.

We identified the need for well defined and defendable community priorities as a key control for ensuring that contribution funds are spent in an accountable manner and that an appropriate balance between flexibility and control is achieved. More specifically, we believe that community priorities should be sufficiently detailed and supported by analysis which demonstrates that they are truly reflective of the issues facing urban Aboriginal people in their respective UAS community. Our audit identified some of the more established UAS communities as having thoroughly documented priorities and plans, while some of the newly added communities have little documented support for their priorities and inadequate plans. In essence, we found that OFI needs to more clearly define its minimum expectations of community plans and priorities and assure itself that these are met by a community prior to agreeing to provide maximum flexibility in the types of projects and expenditures funded.

6.0 Observations

The observations of our audit are provided in the two sub-sections of the report that follow. The first (section 6.1) addresses audit observations in regard to the design, implementation and monitoring of the UAS. The second (section 6.2) addresses audit observations at the project level, including the management and administration of recipient contributions for the UAS program. Recommendations in respect of all audit observations are provided in section 7.0 of the report.

During the planning phase of our audit, we identified risks that were inherent to the nature and design of the UAS. Our audit tests were then designed to evaluate whether these significant risk exposures were being adequately mitigated by processes and controls. The inherent risks [Note 1] covered by our audit program included:

• Risk that funded projects will not align with the UAS national priorities for projects funded through the UAS contribution program (irrespective of whether projects align to community priorities);
• Risk that Community Committees do not have the capacity and capability to develop community plans with priorities that are reflective of community need and sufficiently detailed to permit accountable implementation;
• Risk that the emphasis placed on supporting capacity development in new communities by OFI is not sufficient, or that OFI lacks internal capacity, clear guidelines and appropriate tools to support communities effectively and efficiently;
• Risk that provincial and municipal funders do not sufficiently commit to the UAS, resulting in UAS communities falling short of 50-50 leveraging with other levels of government; and
• Risk of real or perceived conflicts of interest among members of Community Committees.

6.1 Design and Implementation of the Urban Aboriginal Strategy

Our audit examined design and approval of the UAS, management controls supporting implementation of the UAS, and monitoring of and reporting on the UAS. Each of these areas is discussed under separate sub-sections below.

6.1.1 Design of the Strategy

Design of the strategy includes the activities conducted by program managers and senior management to design an effective strategy based on identified need and while obtaining the necessary approvals. For the UAS, strategy design does not only include the contribution program, but also the requirement for community engagement, horizontal collaboration with federal departments and program coordination with provincial and municipal partners.

Program Design and Renewal

Our audit included review, analysis and interviews to understand and assess the thoroughness of the process undertaken by OFI to ensure that the UAS concept and delivery model were properly supported and defendable. For greater clarity, our audit did not seek to evaluate the appropriateness of the underlying concept and design of the UAS, but only the due diligence undertaken in forming policy advice and designing the strategy. Evidence obtained of these activities included documented analysis supporting policy advice and Treasury Board submissions, a program evaluation from 2005 and an OFI led review of the strategy from 2009.

Our audit found that OFI employed diligence in how it developed and defended the UAS concept, including identifying and understanding the need for its programming. OFI has also continued to proactively prepare for its next renewal.

UAS Program Terms and Conditions

During the course of our audit, we identified several aspects of the UAS Program Terms and Conditions which were unclear or ambiguous, and where we believe greater clarity is required.

We found that ambiguity exists in terms of how leveraging of OFI contribution funding is to be measured. The UAS policy authority requires that all communities demonstrate 50-50 commitment (financial or in kind) from provinces and municipalities on a community-by-community basis. It is not clear whether the leveraging target must be achieved across all of a community's projects annually, or as OFI defines it, over the five-year period of the Terms and Conditions. There is also ambiguity as to whether the 50-50 target is a measure of leveraging and/or degree of cost-sharing for OFI funded projects, or whether it also considers projects and activities also funded solely by other levels of government.

The Terms and Conditions require that all projects funded by OFI be strategically focused in at least one of three priority areas: improving life skills; promoting job training, skills and entrepreneurship; and supporting Aboriginal women, children and families. The Terms and Conditions provide no further clarity about what is intended within each of the three UAS priorities or on what types of activities are eligible or ineligible. As such, OFI manages the UAS contribution program with considerable flexibility, providing communities with latitude in developing strategies that best meet their needs.

While the Terms and Conditions are very general, the UAS policy authority is far more specific about what is meant by each of the three priority areas. For example, the policy authority explains the first UAS priority of "improving life skills" as, "improving life skills to encourage Aboriginal youth to stay in school, to facilitate the integration of Aboriginal learners who have relocated from other communities into urban schools, and to encourage learning enrichment initiatives outside the formal education system." By simply referring to the Terms and Conditions, one would not likely infer that life skills projects are intended to be aimed at Aboriginal youth. Similar explanations are included in the policy authority for the other two UAS priorities. As a result of these ambiguities, the auditors could not conclude on the eligibility of certain types of projects, including projects aimed at the elderly, housing related projects, cultural activities not focused on women and children, and programming focused on health.

A key principle of the UAS contribution program, as described in the policy authority, is that there should be no overlap or duplication of services with other existing government programs. We noted that the approved Terms and Conditions include no specific condition restricting the funding of projects that are already covered by other federal programming. Notwithstanding this, our audit found that OFI staff generally understand and respect this key principle of the UAS in their project funding decisions.

The Terms and Conditions define the nature and types of eligible costs, providing considerable detail. Through our testing of expenditure eligibility for a sample of UAS contribution agreement files, we determined that the large majority of costs funded by OFI fall clearly within these definitions. There was, however, some ambiguity around the eligibility of a small portion of the expenditures funded by OFI (2.8% of our sample). More specifically, it is ambiguous whether the Terms and Conditions permit the following types of expenditures that are sometimes funded by OFI:

• stipends and/or awards to individuals completing job training and educational programs;
• bursaries for First Nations post-secondary students; and
• child-care and transportation costs for individuals attending job or life skills training courses.

Community Planning and Priority Setting

Our audit considered the extent to which community plans and priorities were well formed to support the effective and accountable implementation of the UAS contribution program. During the planning phase of the audit, community planning was identified as a key control to ensuring that contribution funds are being spent for their intended purpose. In performing our analysis, we reviewed terms of reference and community plans for all UAS communities, conducted interviews with OFI regional and HQ staff, and distributed questionnaires to OFI Regional Directors responsible for overseeing 7 of the 13 UAS communities. We also reviewed various documentation, including Community Assessment Tool (CAT) self-assessment results, reports, meeting agendas, and minutes of Community Committees. Finally, for a sample of UAS projects funded in five communities, we reviewed whether projects were meeting both the community and UAS priorities.

Our audit found considerable inconsistency in the depth and quality of community plans and in the definition of community priorities across the 13 UAS communities. For 5 of the 13 UAS communities, we found that community priorities were not sufficiently detailed to establish the types of projects that would be funded. The content and format of community plans is not mandated and OFI has not established minimum standards for what it considers essential to an effective and complete plan (e.g. clear objectives, defined priorities, types of projects, concrete steps for achieving community objectives, measures and targets for gauging success, etc.)

In light of the varying degrees of demonstrated capacity of Community Committees, we expected that OFI would employ a risk-based approach to funding projects that considers the quality of plans and the extent to which projects undertaken in prior years addressed community priorities. More specifically, an approach that associates the types of projects and expenditures OFI is willing to fund in a community to the demonstrated ability of the Community Committee to address the issues of urban Aboriginal people in an accountable manner. We found that OFI had taken steps in some specific communities where concerns had been identified, but that it does not have an established approach to address this in a consistent and proactive manner.

6.1.2 Program Implementation

Planning and Objective Setting

Our audit expected to find that operational plans were developed by regional offices and the UAS staff at HQ to guide achievement of the UAS objectives. At a minimum, we expected regional work plans to include objectives, priorities, planned actions and timelines, targets for recipient and community service standards, and expected results. Such work plans would also be approved by HQ and allow for the tracking of progress on a regular basis.

Our audit found that while some planning and objective setting activities occur at the regional and HQ levels, they are not commensurate with the complexity of the strategy. Only two regions were found to have established annual priorities and work plans to guide regional operations and no UAS business plan existed. Our audit did find that certain activities (to increase horizontality and improve alignment) are being planned and reported on at the regional level in OFI's Quarterly Reports and Integrated Business Planning regimes. The auditors did not consider these controls to be adequate to support an efficient, effective and coordinated implementation of the strategy.

Policies and Procedures

Our audit also looked at the policies and procedures developed to ensure that the contribution program component of the UAS is applied consistently across all regional and within the UAS Program Terms and Conditions. UAS policies and procedures are documented in the UAS Operational Guidelines that have been established to support project administration and management functions and to ensure a consistent approach to contribution program implementation. These guidelines are supported by extensive templates that have been established to support the project assessment and approval process.

Our audit found the Operations Guidelines and project templates to be essential tools for Development Officers in assessing projects and recipients for funding. Our audit also found the risk based approach established by OFI to define recipient reporting requirements and OFI monitoring requirements to be aligned with the new TB Policy on Transfer Payments (2008). Templates established by OFI include both a Due Diligence Report (DDR) and a Recipient Assessment Report (RAR) that are completed for all projects being considered for UAS funding. The purpose of the DDR is to ensure sufficient data has been collected in order to provide the basis for recommending a proposed project for funding. The purpose of the RAR is to determine the monitoring and reporting requirements for each project. It considers such factors as the organizational capacity and past experience of the recipient and the complexity of the project in assessing project and recipient risk.

Despite the strength of these tools, our audit found that the Operations Guidelines lacked important information in certain key areas to ensure consistent contribution program implementation and adherence to program authorities. We found the information and direction provided by the guidelines to be limited in terms of: project and expenditure eligibility, the recording of source of funds data in PIMS, community plan requirements, and the roles and responsibilities of regional staff. In regard to the Central Fund (with an annual budget of $1.025M), our audit found that neither the guidelines nor any other program documentation, including the Terms and Conditions, fully defined the purpose of the Central Fund, the criteria to be used to evaluate and rank potential Central Fund projects, or how and when projects should be submitted for funding consideration. Our audit found that regional staff had differing views on the use of the Fund. We also noted that many of the projects that were submitted for funding came in late in the fiscal year, after it became apparent that the Fund was not on track to be fully expended.

Training

Our audit assessed the extent to which regional Development Officers receive training in aspects unique to the UAS and more general to the administration of recipient contribution programs. We found that the training currently provided to Development Officers is largely informal on-the-job training, with some training through the Canadian School for Public Service. This was not a significant issue for OFI when the UAS was launched as staff joined from other departments where they had received relevant training and were familiar with the program. However, with turnover since that time, many regional Development Officers have not received sufficient training to address the dual nature of the role, that being the requirement to support Community Committees in discharging their responsibilities and the requirement to manage and administer UAS contribution agreements. Although new staff have access to the UAS Operations Guidelines and on-the-job mentoring, it is insufficient to equip them to handle the complex aspects of the role, particularly those related to supporting Community Committees (e.g. supporting development of plans and priorities). While the development of a formal training program specific to UAS would likely not be cost effective, structured off-site sessions and training on administration of transfer payments should be provided.

Performance Measurement

Our audit considered the extent to which performance measures and reporting processes have been established to ensure that UAS results and contributions to outcomes are identified and measured. We found that activities are clearly linked to UAS objectives and Departmental priorities, and key reporting requirements have been identified in support of the logic model of the UAS. While key reporting requirements have been established, the completeness of performance measurement data is in question as results are not yet being captured in relation to all reporting requirements. This includes post project data that is to be captured in PIMS as well as other program results that are to be captured manually. As a result, while valuable results data may be available in due course to support program renewal, these results are not currently available to support annual planning and goal setting.

Our audit also identified concerns with respect to the accuracy of leveraging information being captured in PIMS. While PIMS is considered to be a valuable tool for the UAS, within the projects sampled we noted numerous instances where the sources of funding as provided for in project work plans (as annexed to funding agreements) were not consistent with the sources of funds as recorded in PIMS. Discrepancies in this data called into question the integrity of leveraging data that may eventually be reported by OFI. We also found that the Operational Guidelines did not provide direction as to what constituted recordable leveraging to ensure consistency and integrity of the information captured, however we found that OFI is working toward defining leveraging

6.1.3 Program Monitoring and Oversight

Program monitoring consists of review and oversight, conducted on a timely basis by program management, to promote the achievement of program objectives and to identify issues of possible concern for resolution. In this regard, our audit considered the extent to which performance and financial data were being collected and analyzed, and results consolidated, to demonstrate UAS performance and effectiveness. Our audit also considered the extent to which monitoring of the UAS contribution program controls was being performed to ensure consistency with policies and procedures.

In respect of the overall strategy, our audit found that OFI senior management provides regular oversight through weekly OFI management calls (with a specific UAS component every second week), quarterly OFI management meetings, and ad hoc bilateral meetings and discussions with Regional Directors. As noted earlier in this report, the effectiveness of these meetings would be improved if regions and UAS HQ had operational work plans and priorities and documented performance reporting. This would provide OFI senior management greater assurance that their direction is understood and implemented, improving the accountability relationship between Regional Directors and their superiors.

In respect of the contribution agreement component of the UAS, we saw limited review of projects that are approved at the regional level (i.e. no cross-regional peer review process or UAS project review committee) and very limited qualitative review and reporting to OFI senior management on whether project funding was effectively addressing all community and UAS priorities. While a defined process is in place for the review, recommendation and approval of all UAS funded projects, improved quality-control is required to ensure that the nature of projects being funded across regions is consistent and in accordance with UAS and policy requirements.

6.2 Contribution Agreement Level Compliance

At the project level, the audit examined controls in support of the evaluation and documentation of recipient and project eligibility, the development of funding agreements, the financial administration of funding agreements, and recipient reporting and monitoring. The adequacy and effectiveness of project level controls was considered through the review of a sample of 34 projects established and managed in Alberta (13), Manitoba (14), and at HQ (7), as well as through interviews. The projects were assessed against the established audit criteria, which considered, among other things, compliance with both the UAS Program Terms and Conditions and the TB Policy on Transfer Payments (2000 and 2008).

Our audit sample included projects funded under each of the three UAS Funds: the CIF, the CCF, and the Central Fund. As previously noted, regional offices are provided with both CIF and CCF budgets, and only have access to the Central Fund once their entire CIF budget has been allocated. CIF and CCF regional budgets are determined based on the number of UAS communities in the region and the size of the communities. For "large" UAS communities, the annual CIF allocation is typically $750,000 and the annual CCF allocation is $185,000. For "small" UAS communities, the annual CIF allocation is typically $400,000 and the annual CCF allocation is $100,000. Regions track commitments against these budgets as projects are assessed and approved. The Central Fund budget is maintained by HQ. The HQ projects sampled by the audit team were therefore all Central Fund projects, while the regional projects sampled were funded under all three Funds.

6.2.1 Funding Decisions and Agreement Development

Funding decisions include the assessment of project funding in a timely, fair and transparent manner, with only eligible recipients and activities approved for funding. Agreement development includes the development of formal funding agreements, containing complete, appropriate and compliant terms and conditions, established with recipients on a timely basis.

Funding Decisions

Our audit found that a structured call for proposal process was employed in 3 of the 5 UAS communities subject to review, with the remaining two communities employing an improvised process for identifying potential projects to recommend for funding. A full call for proposal process was not possible in these communities as one had experienced a recent change in its delivery model and the other had experienced leadership, membership and structure issues within its Community Committees that disrupted standard project identification processes. However, in all five of the communities, the projects that were sampled by the audit team had been reviewed and recommended by the Community Committees as required (note that only CIF projects require such review).

All of the regionally funded projects that were sampled were found to have been subject to an assessment and approval process that included the completion of a DDR and an RAR, and the development of a project work plan to define project particulars in support of funding agreement preparation and project monitoring. While the large majority of the DDRs and RARs completed by regional Development Officers were found to be complete, the audit did note some instances of missing or incomplete documentation. Our audit found that 3 of the 7 HQ sampled projects did not include the completion of a DDR and/or an RAR.

Our audit identified instances where the frequency of recipient reporting, as determined through the risk assessment process, was not consistent with the reporting requirements defined in the funding agreement. However, this was typically the result of funding agreements being established too late in the fiscal year to allow for more frequent reporting.

We were advised that the process employed for the assessment of Central Fund projects at HQ was not well established in 2008/09. We were further advised that this process was improved in 2009/10 and that now, wherever possible, Central Fund projects are assigned to regional offices so that project assessment and administration is conducted by a Development Officer rather that a UAS staff person at HQ that is primarily responsible for policy and planning activities.

Finally, ambiguity in the UAS Program Terms and Conditions, as noted previously in this report, made it difficult for the audit team to verify the eligibility a small number of projects for funding. Similarly, ambiguity in the Terms and Conditions made it difficult for the audit team to verify the eligibility of a small portion of project expenditures.

Establishing Funding Agreements

Our audit found that a model agreement was used to fund all projects, with a work plan annexed to each agreement that provided a description of the project, performance indicators and targets, a detailed budget, and all sources of funding for the project. We also found that all projects were approved by an individual with appropriate delegated financial authority, the only possible exception being those projects that included funding for capital assets. The audit observed that the UAS Program Terms and Conditions only allowed the Associate Deputy Minister (ADM) of OFI to approve arrangements with "capital expenditures". However, the Terms and Conditions do not define capital expenditures. While OFI senior management interprets this requirement as being limited to the funding of land and buildings, we interpreted capital expenditures to include any asset that would have a useful life of greater than one year (e.g. tools and equipment, furniture and commercial kitchen equipment, and computers).

Our audit identified inconsistent and untimely processes for the review and assessment of project proposals and the establishment of funding agreements as areas of concern. In this regard, our analysis considered: 1) the time required by regional offices to assess and approve projects for funding and to submit the necessary project documentation to HQ; and 2) the time required by the OFI Administration Unit to prepare funding agreements and have them executed. Half of the regionally funded projects sampled took over 120 days to assess, approve and submit to HQ for funding, with roughly the same percentage taking over 45 days to have funding agreements prepared, approved and executed by all parties.

Project files did not fully document the reasons for these delays. However, regional staff cited the time required to confirm the support of other funders, the submission of incomplete and untimely documentation by recipients, and their overall workload as reasons for lengthy assessment and approval processes. In terms of agreement preparation, our audit noted that only one person at HQ was dedicated to completing UAS funding agreements, with no defined process in place to cover leave periods or to ensure adherence to defined service standards. As a result of these delays and a lack of attention paid to the importance of meeting service standards, funding agreements are generally established late in the fiscal year.

The charts below illustrate the period in which funding agreements have been signed by OFI in each of the past two fiscal years. The chart on the left illustrates that $6.1M of the $10.0M (61%) in funding agreements established in 2008/09 were established in the last quarter of the fiscal year. The chart on the right illustrates the impact of improvements to the proposal solicitation and review process in 2009/10, with only $3.5M of the $8.4M (42%) in funding agreements established in the last quarter of the fiscal year.

In the diagram, two separate charts provide a comparison of the amounts for the funding agreements signed by the Office of the Federal Interlocutor in each of the two fiscal years 2008-09 and 2009-10.

The first chart shows the total funding under the Urban Aboriginal Strategy provided in 2008-09 broken down by the various periods in which funding agreements were signed. Funding provided in 2008-09 for agreements signed prior to 2008-09 totalled $604,000. Funding provided in 2008-09 for agreements signed in the four quarters of 2008-09 totalled $295,000, $1.4 million, $2.1 million and $6.1 million respectively.

The second chart shows the total funding under the Urban Aboriginal Strategy provided in 2009-10 broken down by the various periods in which funding agreements were signed. Funding provided in 2009-10 for agreements signed prior to 2009-10 totalled $1.6 million. Funding provided in 2009-10 for agreements signed in the four quarters of 2009-10 totalled $480,000, $1.4 million, $2.9 million and $3.5 million respectively.

Funding agreements established late in the fiscal year can negatively impact program effectiveness by creating cash flow constraints for recipients and reducing the value associated with project monitoring and reporting. Virtually all of the projects sampled by the audit team commenced prior to funding agreements being established. Accordingly, recipient organizations were often required to use other sources of funding to support project activities until UAS agreements were in place and initial payments issued. In addition, funding agreements established late in the fiscal year diminished the value Development Officers provided projects through the provision of recipient support and monitoring. Finally, late funding resulted in some recipients receiving as much as 90% of project funding (all but the required 10% holdback) without having ever provided an interim performance or financial report to validate the project activities conducted or the project expenses incurred.

6.2.2 Payments to Recipients and Project Closeout

In addition to funding decisions and agreement development, our audit assessed the adequacy and effectiveness of controls in support of the OFI's financial administration of funding agreements. Activities in this regard are the responsibility of OFI's Administration Unit and include the issuance of initial and subsequent project advances/payments and the close out of project files.

Funding agreement payment schedules are to be established based on cash flow forecasts provided by recipients and, if necessary, are to allow for advance payments in accordance with the TB Policy on Transfer Payments (2000 and 2008). The first payment issued under a UAS agreement is typically issued to a recipient at the time of agreement execution. Subsequent payments are issued on the first of a month and may be tied to the submission of performance and/or financial report by the recipient. However, payments are withheld when a recipient has not met its reporting requirements under an existing or prior year's UAS agreement.

While our audit attempted to consider the timeliness of project payments with respect to the projects sampled, limited conclusions could be drawn. Due to the large number of agreements funded late in the fiscal year, initial payment schedules were often adjusted to reflect the shortened timeline of the agreement. Fourteen of the 34 projects we sampled were funded as one 90% payment. Two other projects (both with the same recipient) had not received any project payments as at the time of our review (December 2009) due to outstanding reports on past UAS agreements. However, it was evident from the processes reviewed and the interviews conducted that project payments were not always issued as per payment schedules due to late recipient reporting, processing delays as a result of staff shortages and bottlenecks within the Administration Unit, and a lack of oversight of the effectiveness and timeliness of financial processes within the unit.

Our audit found these same issues applied to the closeout of project files and the release of final holdbacks. Project holdbacks are to be released in two amounts. The initial amount (half of the 10% holdback) is to be released upon receipt and review of a recipient's final performance and financial reports. These reports are typically due between April and June of the following fiscal year, and are subject to review by OFI staff. The second and final holdback amount is to be released upon receipt and review of the recipient's final statement of revenue and expenses (most often an audited statement). These statements are subject to an initial review (for completeness) by the Senior Financial Advisor in OFI's Administration Unit and are then forwarded to INAC Finance for detailed review. INAC Finance has set a standard of 60 days to complete these reviews.

Our audit found that project closeouts and the release of holdbacks did not occurring in a timely manner. This was due in part to the time required by INAC Finance to complete detailed reviews and the time required by the OFI to analyze and "clear" review comments. Of the 34 projects sampled (all of which were completed on March 31, 2009), initial holdback amounts (the first 50%) had only been released in relation to 4 projects at the time of our review (December 2009), and no final holdbacks had been released and therefore no project files had been closed out. While OFI does not maintain summary information on the closeout of UAS projects, information provided by the Administration Unit as of February 2010 confirmed the following:

• no holdback funds (initial or final) had been released to recipients in regard to 128 of the 165 UAS projects funded in 2008/09;
• initial holdback funds had been released to recipients, but no final holdbacks, in regard to a further 27 of the 165 projects funded in 2008/09; and
• a total of $1.2M in project payables (PAYE) remained outstanding for fiscal 2008/09.

While detailed financial reviews of project expenditures may determine that some of the $1.2M is not payable to recipients, the large majority of these funds represent amounts to be paid to recipients.

6.2.3 Agreement Monitoring and Reporting

Development Officers are responsible for monitoring project recipient activities to ensure compliance with UAS Program Terms and Conditions and funding agreements. This includes ensuring that project implementation is proceeding as intended through site visits and regular, on-going communication. It also includes obtaining and reviewing recipient reports in a timely manner.

As previously noted, our audit found DDRs and RARs to be valuable tools for assessing both recipient and project risk levels, and in defining the frequency of project monitoring and recipient reporting based on risk. In conducting our audit, it was apparent that monitoring activities were taking place that included electronic communications, telephone calls, periodic site-visits, and the review of recipient financial and activity reports. However, these activities were not well documented in the majority of project files. In addition, it was not evident that these activities were conducted as part of a coordinated monitoring plan or that the site visits that were conducted represented structured monitoring activities. Our audit found that the activities to be performed during site visits were not well defined. Our audit also noted that no formal process was in place to ensure that Development Officers followed up and obtain late recipient reports in a timely manner. The majority of recipient reports we reviewed as part of our audit sample were received more than 30 days late, with some not received at all. Our audit did recognize that the value and significance of site visits and interim recipient reporting was negatively impacted by the late completion of funding agreements.

Our audit noted that a very strong monitoring guideline/checklist was under development that, once in place, will provide direction to Development Officers in the completion of effective site visits. In addition to defining specific verification activities to be performed, it will provide a means for recording site visit results. We also noted that OFI conducted audits of UAS recipients for the first time for fiscal 2008/09 and that the selection of agreements for audit was completed with input from regional offices. Recipient audits are important in assessing recipient compliance with Terms and Conditions and funding agreements. OFI's current approach involves obtaining recipient concurrence on audit findings and adjustments prior to finalizing audit reports, which can lead to audit reports being held in draft for extended periods.

7.0 Recommendations

Our audit recommends the following to the OFI to address the findings of the audit:

1. OFI should review the administrative regime it uses to fund new and developing UAS communities. Specifically, consideration should be given to using a risk-based approach that associates the types of projects and expenditures it funds in a community to the ability of the Community Committee to address the issues of urban Aboriginal people in an accountable manner.
2. OFI should establish a consistent expectation for community plans by developing a template that includes the type and level of information required and by providing planning support and guidance as needed. This would ensure that both community and national UAS priorities are addressed, that a shared vision of community success is defined, and that concrete steps are established to achieve objectives and measure progress.
3. When OFI seeks renewal of the UAS spending authority and Terms and Conditions with its federal partners, it should:
   • Having regard for the flexibility intended with the UAS contribution program and the strategic priorities of the UAS, ensure clarity as to the nature of eligible projects, activities and expenditures;
   • Set out expectations for leveraging with provinces and municipalities, how results should be measured, and how leveraging requirements should apply to new and developing communities; and
   • Clarify the definition and approval requirements for capital expenditures.
4. To support consistent understanding of the flexibilities built into the UAS contribution program, OFI should provide guidance to regional staff and Community Committees on eligibility requirements and use of project and recipient risk information to substantiate funding decisions.
5. Annual work plans should be established by UAS HQ and each region. These work plans should establish objectives, regional management and program priorities, planned actions and timelines, targets for recipient and community service standards, and expected results (program priorities and expected results should align with the UAS Performance Measurement Strategy to be developed and implemented following the completion of the summative evaluation which is currently underway). The regional plans should be approved by HQ and progress tracked through a quarterly reporting process.
6. OFI should more fully develop its training regime for Regional Development Officers to ensure that it adequately addresses the dual nature of this role, that being the requirement to support the Community Committees in discharging their responsibilities and the requirement to manage and administer UAS contribution agreements.
7. To ensure efficient and effective processes for the review and approval of projects and the establishment and administration of contribution agreements, OFI should revamp the processes and timelines for soliciting proposals, reviewing project eligibility, preparing and executing funding agreements, processing recipient payments and closing out project files. Service standards should be established and achievement against these standards should be tracked and monitored by OFI HQ on a regular basis.
8. OFI should more fully define the purpose of the Central Fund, as well as the criteria, policies and procedures used in the solicitation, submission, and assessment of Central Fund projects.

8.0 Management Action Plan

Footnotes:

1. Inherent risks are those risks which inherently threaten the achievement of program objectives prior to management having implemented its processes and controls. An adequate and effective management control framework ensures that these inherent risks are mitigated, such that residual risk is reduced to an acceptable level. (return to source paragraph)