Audit of the Management Control Framework of Grants and Contributions (2014-2015) - Focus on the General Assessment Process

Background

In order to support compliance with the Treasury Board Policy on Transfer Payments (PTP) and the management of funding agreements that have been negotiated between Aboriginal Affairs and Northern Development Canada (AANDC or the Department) and funding recipients, a General Assessment (GA) tool was implemented in the Fall of 2010. This tool provides an annual snapshot of a funding recipient's past performance and identifies strengths and emerging risks that may impact the manner in which AANDC manages transfer payments to the recipient.

The GA is a living tool. Version 1 of the GA was implemented and used in the Fall of 2010 for funding agreements for the 2011-2012 funding year. Since that time, Version 2 was developed and has been used for subsequent funding years.

The results of the GA are expected to affect the duration of funding agreements, the frequency and type of departmental monitoring activities (and related reporting requirements), the flexibility of the funding arrangement and a recipient's eligibility for certain funding approaches. As such, the Department must be able to defend the consistency and transparency of the assessment results. Since Part A GA results are to be made public starting in January 2016, there is the potential for additional scrutiny by recipients and other stakeholders; further impacting the need for consistency and defensibility across the Department.

Audit Objective and Scope

The objective of this audit was to provide assurance to senior management of the adequacy and effectiveness of the application of the GA tool across Aboriginal Affairs and Northern Development Canada to support the management of funding agreements with funding recipients.

The scope of this audit focused on an assessment of the adequacy and effectiveness of the GA tool (Part A, Part B, and Part C) examined horizontally across a sample of seven (7) regions for the fiscal years 2014-2015 in order to evaluate the consistency of the application of the GA tool.

Statement of Conformance

The audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

Conclusion

The audit found that GAs are being performed as required by the Department's Directive 410 – General Assessments. However, the audit noted specific inconsistencies, across regions and sectors, in the processes established to develop, review and approve GAs, as well as the documentation and analysis maintained to support assessment results.

In order to improve consistency in the development, review and approval of GAs, as well as their scoring throughout the Department, the audit identified opportunities to improve the tools, guidance and training associated with the GA process.

Further, the audit found that GA scores are not consistently impacting the level of administrative requirements imposed on recipients. Although the intention of the Policy on Transfer Payments was to allow for a risk-based approach to be taken to reporting and monitoring, due to current limitations in program reporting and management regimes, regions and sectors are unable to either adjust the reporting requirements or monitoring performed on recipients based on the GA scores.

Recommendation

The audit team identified areas where the adequacy and effectiveness of the GA tool should be improved, resulting in four (4) recommendations as follows:

1. The Grants and Contributions Management Oversight Committee should develop and coordinate a Department-wide quality assurance/monitoring program in order to provide the Department with a level of assurance that GAs are being completed in a consistent manner and assessment results are supported by adequate justification. The quality assurance/monitoring program should include the review and assessment of Part A, Part B and Part C GAs.
2. The Grants and Contributions Management Oversight Committee should ensure the standardization of program-specific checklists/templates used to support the risk analysis across all regions. These checklists/templates should be assessed for their alignment to the risk considerations and benchmarks outlined in the GA Workbook prior to implementation.
3. The Grants and Contributions Management Oversight Committee should improve alignment of the reporting requirements and monitoring performed on recipients, as well as the flexibility in funding approaches available to recipients based on a recipient's level of risk.
4. The Grants and Contributions Management Oversight Committee should address the recommendations laid out below to address gaps identified with Directive 410 – General Assessments, and the GA Workbook.
   1. Directive 410 – General Assessments should be updated to further define:
      1. The role, responsibilities and accountabilities of Funding Services and program representatives in the development, review and challenge of the programs' components of a Part A GA; and
      2. The role, responsibilities and accountabilities of regional offices in the review and challenge of the impact of Health Canada's input into the GA scores.
   2. The GA Workbook should be updated to include benchmarks for the low-medium and medium-high risk ratings for each risk consideration based on relevant criteria, such as what is already included for the low, medium and high risk ratings.
   3. In order to ensure consistency in the scoring of GAs, the Directive 410 and GA Workbook should be updated to provide additional guidance on how special considerations, such as those identified during the audit, are to be considered/addressed when completing GAs
   4. Directive 410 should be updated toclarify roles and responsibilities over the review and approval of GAs, specifying the appropriate authority level for approvals for Part A and B GAs and ensure their consistent application across regions/sectors.

Management Response

Management is in agreement with the findings, has accepted the recommendations included in the report, and has developed a management action plan to address them. The management action plan has been integrated into this report.

1.1 Grants and Contributions

Aboriginal Affairs and Northern Development Canada (AANDC or the Department) makes funding available to First Nations and other recipients through grants and contributions (G&Cs) for the delivery of programs and services such as education, land management, social development and community infrastructure. Total departmental spending on G&Cs for the 2014-2015 fiscal year was $6.3 billion^{Footnote 1}.

AANDC's transfer payment programs are administered in accordance with the Treasury Board Policy on Transfer Payments (PTP) and Directive on Transfer Payments (DTP), which took effect on October 1, 2008. The objective of the PTP and DTP is to manage transfer payment programs with integrity, transparency and accountability, taking into account the risks, and to ensure that programs are effectively focused on citizens and beneficiaries, and are designed to achieve various Federal Government priorities and expected results. More specifically, the PTP outlines the expectations that risk-based approaches are adapted to the design of: transfer payment programs; program terms and conditions; funding agreements; recipient monitoring; and, recipient auditing.

Within AANDC, the initial implementation of the PTP and DTP took place in March 2011. In order to meet the expectations of the PTP, the Chief Financial Officer (CFO) Sector established the Transfer Payments Centre of Expertise (TPCOE), which has put in place the Management Control Framework (MCF) for Grants and Contributions to ensure effective management and monitoring of G&C programs and to ensure compliance to the PTP and DTP. AANDC's CFO is accountable for the overall management of transfer payment funds and, as such, is the custodian of AANDC's MCF for G&Cs. The MCF establishes roles and responsibilities for the delivery of G&Cs, specifically to program management (the design and implementation of a program) and transfer payment operations (operations of a program with recipients), and sets the Department's expectations of how G&Cs are to be managed across regions and at Headquarters (HQ).

Due to the significance of G&C funding to the overall departmental budget and mandate, an internal audit of the MCF is conducted every year.

1.2 The General Assessment

The General Assessment (GA) is a tool that was implemented in the Fall of 2010 to support compliance with the PTP and the management of funding agreements that have been negotiated between AANDC and funding recipients. The tool provides an overview of a funding recipient's past performance and identifies strengths and emerging risks that may impact the manner in which AANDC manages transfer payments to the recipient. The GA allows AANDC to assess the capacity of recipients to access a variety of funding approaches, including multi-year agreements. The results of the GA may also result in adjustments to AANDC's reporting requirements of the recipient.

Since 2013-2014, AANDC has collaborated with Health Canada (HC) in the development of ongoing multi-program agreements in order to provide funding to common recipients of both AANDC and the First Nations and Inuit Health Branch (FNIHB) of Heath Canada. For the 2014-2015 fiscal year, Part A GAs developed for recipients under joint, ongoing multi-program agreements included an assessment of the recipients' program management activities associated with both AANDC and FNIHB programs.

The GA is a living tool. Version 1 of the GA was implemented and used in the Fall 2010 for funding agreements for the 2011-2012 funding year. Since that time, Version 2 was developed and has been used for subsequent funding years. Version 2 contains refinements to the scoring scale of the considerations, the weightings assigned to each risk factor, and streamlining of the risk factors.

In support of improving the consistency of the completion of GAs, a GA Workbook was created. It outlines the various "considerations", and associated benchmarks, to be assessed for each risk factor (governance, planning, financial management, program management, and other considerations) during the completion of the GA. The GA Workbook supports an equitable and consistent approach to managing funding agreements.

The GA Workbook is divided into two parts: Part A, for more complex funding relationships, and Part B, for less complex funding relationships. Part A is completed for recipients who have an ongoing relationship with AANDC and is completed prior to entering into a funding agreement and once annually thereafter for multi-year agreements in which the recipient is assessed as medium risk or higher. Part A GAs are typically completed by regional offices, with only a limited number of Part A GAs completed by HQ sectors. The Part A GAs include an assessment of the following risk factors: Governance, Planning, Financial Management and Program Management (applicable for each eligible program the recipient is receiving funding for).

In contrast to Part A GAs, Part B GAs are used to assess recipients of a one-time project or an ongoing specific service with its own funding agreement. Part B GAs include an assessment of the governance, performance history, financial stability and planning of the recipient, as well as project complexity for that program/project. Part B does contain an "Other Considerations" risk category which is used when specific program requirements result in the need to evaluate additional areas of risk. For 2014-2015, 30% of the Part B GAs were completed by HQ sectors.

The Part C GA tool is used to assess the risk and capacity of an ongoing, multi-program recipient in support of the decision to move or renew the recipient for a block funding agreement. As a result, Part C GAs contain much more detailed criteria than Part A and B GAs and provide a checklist that determines a numeric assessment score for each risk category. Part C GAs are completed by regional offices.

agreements, the frequency and type of departmental monitoring activities (and related reporting requirements), the flexibility of the funding arrangement and a recipient's eligibility for certain funding approaches, the Department must be able to defend the consistency and transparency of the assessment results. In an effort for increased transparency, the Department has made the decision to publish all Part A GA results starting in January 2016, resulting in the potential for additional scrutiny by recipients and other stakeholders. Due to these factors, the GA was determined to be the focus of this audit.

2.1 Audit Objective

The objective of this audit was to provide assurance to senior management on the adequacy and effectiveness of the application of the GA tool across the Department to support the management of funding agreements with funding recipients.

2.2 Audit Scope

The scope of the audit focused on an assessment of the adequacy and effectiveness of Part A, Part B, and Part C of the GA tool. This included:

• The oversight associated with the development and finalization of the assessment results and scores;
• The selection of an appropriate funding mechanism based on the results of the GA, as applicable^{Footnote 2};
• The activities and participation in the completion of the annual assessment per recipient; and,
• The governance, tools and guidance in place to ensure the consistent application of the assessment across all regions.

The audit scope included examining a sample of Part A, Part B, and Part C GAs for the 2014-2015 fiscal year that covered site visits to three (3) regional offices, as well as a desktop exercise for an additional four (4) regions, in order to evaluate the consistency of the application of the tool.

The scope of the audit did not include an assessment of the GA results completed by Health Canada. The audit scope only covered the review and approval process of AANDC over the completed GA, which includes the components completed by Health Canada.

Selection of Regions for Site Visits

The sampling methodology considered the following factors in the selection of the regional offices to be visited:

• Size (number) of GAs completed by regional offices;
• Size (number) of Low, Medium, and High risk GAs by regional office; and,
• Feedback obtained during the planning phase by individuals interviewed at HQ and regions.

Based on the above analysis and with the objective being to assess the adequacy and effectiveness of the application of the GA tool across the Department, the following regional offices were selected for on-site field testing:

• Atlantic;
• Alberta; and,
• British Columbia.

To gain a comprehensive understanding of the adequacy and effectiveness of Part A, Part B, and Part C of the GA tool, we expanded our testing to include GAs completed in the Ontario, Manitoba, Saskatchewan and Quebec regions. The audit also included interviews in some program areas at HQ focusing on the Part B of the GA Workbook. These included program areas within the Northern Affairs Organization, Lands and Economic Development Sector, and the Treaties and Aboriginal Government Sector.

5.1 Roles, Responsibilities and Accountabilities

Directive 410 – General Assessments (Directive 410), developed by TPCOE, which took effect on April 1, 2011,outlines certain key roles, responsibilities and accountabilities in the completion, review and approval of GAs, as well as the maintenance of the GA framework, including the development of training and working tools. The User Guide, revised in August 2011, also includes the Terms of Reference for the Regional/Sector Transfer Payment Management Committee.

Specifically, Directive 410 assigns responsibility for completing the GAs to Regional Funding Services or program equivalent with oversight provided by Regional Directors General and/or Director Generals of the program branch, the Chief Financial Officer (CFO) sector and the regional and/or sector Transfer Payment Management Committees, or equivalent. The audit observed that although the assigned roles, responsibilities and accountabilities of certain key personnel involved in the GA process are clearly defined, there was a lack of a consistent understanding of some assigned roles. The audit noted that some improvements are necessary to ensure a consistent approach to the completion of programs' components of Part A GAs and HQ sector completed Part B GAs, as well as the review and challenge of Health Canada's assessments.

5.2 Training, Tools and Guidance

The audit team noted gaps in the level of training provided to those assigned with responsibility for completing, reviewing or approving GAs. Additionally, the audit identified opportunities for improving the objectivity of the GA process through enhancements to the GA Workbook.

5.3 GA Assessment Results

5.4 Impact on Monitoring and Reporting Activities

Per the PTP, administrative requirements on recipients, which are required to ensure effective control, transparency and accountability, need to be proportionate to the level of risks specific to the program, the materiality of funding and the risk profile of recipients. The DTP further articulates that the level of monitoring of recipients and the reporting required from recipients should also be impacted by the same risks.

Through the conduct of interviews with representatives from a sample of regional offices and HQ sectors, the audit noted that GA scores are not being consistently used to determine the level of administrative requirements applied to recipients. More specifically, due to limitations imposed by certain programs control frameworks, the level of monitoring of, and reporting required by recipients cannot be tailored based on the results of the GA.

This observation is aligned to a finding outlined in the 2013-2014 Audit of the Management Control Framework for Grants and Contributions – Focus on Program Control Frameworks and Recipient Reporting. That audit had noted that regions and programs were generally not implementing risk-based reporting and management regimes to target limited departmental resources on projects and recipients of highest risk. Similarly, the 2012-2013 Audit of the Management Control Framework for Grants and Contributions – Funding Approaches found that the level of recipient risk was not always adequately considered in the establishment and selection of funding approaches and compliance activities within the Department.

Per Directive 410, the GA was developed to provide a standardized process for assessing a recipient for the purpose of identifying potential issues that impact delivery of AANDC-funded programs in order to provide flexibilities in funding agreement management regimes. As such, the GA should be the main tool used in determining the level of reporting and other administrative requirements imposed on recipients; facilitated by program management control frameworks that provide sufficient flexibility to allow regions to select the appropriate reporting and monitoring requirements based on the GA scores and other relevant factors.

5.5 Review and Approval Process

Directive 410 outlines that GA reports are subject to a multi-level review process to ensure the integrity of individual decision documents and consistency across recipients. The Directive outlines that completed GAs are first subject to a peer group review, then review by a Funding Services Manager or equivalent, and finally, the review and approval by the Chair of the Transfer Payment Management Committee or equivalent.

The GA User Guide – Annex B indicates that GA results should be approved by an established regional committee, or delegated to a sub-committee when appropriate. A risk-based approach should be conducted to determine if circumstances exist where GA approval should be escalated to a Transfer Payment Management Committee.

The audit identified a robust and consistent approach for the review and approval of Part C GAs, with all Part C GAs being reviewed and approved by an established regional oversight committee, comprised of individuals with sufficient authority. However, inconsistencies were noted in the review, challenge, and approval of Part A and Part B GAs across the sampled regional offices and HQ sectors. Of the three regional sites visited, the following was noted:

• In one region visited, no formal peer, supervisory or oversight committee review is performed on all GAs (beyond a manager approval in GCIMS). Only specific GAs that meet a pre-determined variance threshold (5% increase/ decrease against the previously completed risk rating) are reviewed and questioned by the Director of Funding Services. No Part A or B GAs are reviewed or approved by the Regional Operations Committee or a sub-committee.
• In another region, all Part A and B GAs are reviewed first by a Field Services Manager and then by the region's Band Audit and Allegations Management (BAAM) unit. GAs are reviewed and approved either at the Audit Review Committee, or through a BAAM desk review (if very low risk), and then approved in GCIMS by a member of BAAM. Medium and high risk GAs, along with a roll-up of all GA results, are discussed at the Regional Operations Committee.
• In a third region visited, the results of all Part A and B GAs were reviewed by the Funding Services Manager and the region's Risk Assessment Committee. For any Part A and B GAs assessed as Medium-High or High risk, the results were also reviewed by the Transfer Payment Management Committee.

For the three sampled HQ programs, none had established a Transfer Payment Management Committee to review the completed Part B GAs. In all programs, the GAs were reviewed and approved at a Manager-level.

Without a consistent approach for the review and approval of Part A and B GAs, there is an increased risk that unsubstantiated or incomplete GAs are not identified and addressed in a timely manner. Additionally, without a comprehensive review process, which includes the review and challenge of GAs by a committee or individual with sufficient oversight over the completion of all regional/sectoral GAs, inconsistencies in the scoring against risk considerations may not be identified.