Audit of the National Child Benefit Reinvestment and Assisted Living Programs

PDF Version (225 Kb, 26 Pages)

Acronyms
Executive Summary
1. Background
2. Audit Objective and Scope
3. Approach and Methodology
4. Conclusion
5. Findings and Recommendations
6. Management Action Plan
Appendix A: Audit Criteria
Appendix B: Relevant Policies/Directives

Acronyms

AANDC	Aboriginal Affairs and Northern Development Canada
AL	Assisted Living
CFO	Chief Financial Officer
CIMD	Corporate Information Management Directorate
DCI	Data Collection Instruments
ESDPP	Education and Social Development Programs and Partnerships Sector
FN	First Nations
FSO	Funding Services Officers
GCIMS	Grants and Contributions Information Management System
G&C	Grants and Contributions
HQ	Headquarters
ISD	Income Support Directorate
KPI	Key Performance Indicator
MCSS	Ministry of Community and Social Services
NCB	National Child Benefit
NCBS	National Child Benefit Supplement
NCBR	National Child Benefit Reinvestment
OQM	Operations and Quality Management Directorate
PMS	Performance Measurement Strategy
PTP	Policy on Transfer Payments
RO	Regional Operations
SPPB	Social Policy and Programs Branch
TB	Treasury Board
RBAP	Risk-Based Audit Plan
TPCOE	Transfer Payments Centre of Expertise

Executive Summary

Background

National Child Benefit Reinvestment and Assisted Living are two of the five transfer payment programs comprising the Social Development Program at Aboriginal Affairs, Northern Development Canada (AANDC). They are administered in accordance with the Treasury Board (TB) Policy on Transfer Payments (PTP) and the TB Directive on Transfer Payments. The expected outcome of the Social Development Program is "First Nations men, women and children are active participants in social development within their communities".

The NCBR is one component of a broader federal/provincial/territorial National Child Benefit (NCB) initiative that was introduced in 1998. The NCBR provides supports and services for children in low-income families in provinces / territories where social assistance directly paid to families, as part of the NCB, is reduced. The NCBR is currently delivered in provinces other than Manitoba, New Brunswick, Newfoundland and Labrador, and Quebec. In regions where NCBR is delivered, AANDC provides funding for community-based supports and services under five activity areas: child care; child nutrition; support for parents; home to-work transition; and, cultural enrichment. Actual expenditures reported for NCBR in 2012/13 and 2013/14 were approximately $50M and planned expenditures reported in the 2014/15 Report on Plans and Priorities were $54M.

AL provides funding for non-medical, social support services to seniors, adults with chronic illness, and children and adults with disabilities (mental and physical) so that they can maintain functional independence and achieve greater self-reliance. AL is available to individuals ordinarily resident on-reserve who have been formally assessed by a health care professional (in a manner aligned with the relevant provincial or territorial legislation and standards) as requiring services and who do not have the financial means to obtain such services themselves. There are four major components to the AL program: In-Home Care, Adult Foster Care, Institutional Care, and the Disabilities Initiative. Actual expenditures reported for AL in 2012/13 and 2013/14 were approximately $99M, and planned expenditures reported in the 2014/15 Report on Plans and Priorities were $86.9M. Actual expenditures for AL in 2014/15 may be higher than $86.9M because internal funds are generally reallocated to the program throughout the year.

Over time, there has been a misalignment of AANDC AL services with provincial models, particularly with Institutional Care.

Audit Objective and Scope

The objective of the audit was to assess the adequacy and effectiveness of program management controls for the NCBR and AL programs.

The scope of the audit covered the period April 1, 2012 to September 30, 2014 and includes assessments of the:

Implementation of the performance measurement strategy, including design of processes and systems used to collect, review, analyze and report on program performance information;
Appropriateness of policies, procedures, guidelines, tools, learning and development activities, and other supports put in place by the national programs and regional offices in support of program implementation;
Design of risk management processes and program delivery approaches intended to support the risk-based administration of the NCBR and AL programs;
Design of application and proposal assessment processes and funding allocation for the NCBR and AL programs; and
Adequacy of processes in regions to monitor the use of program funds and manage the reinvestment of recipient funding surpluses.

The scope of the audit included coverage of Headquarters program controls and three sampled regions: Yukon, Saskatchewan and Ontario. In respect of the design of funding allocation and proposal assessment processes, all regional processes were reviewed and considered.

Statement of Conformance

This audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

Observations

The Education and Social Development Programs and Partnerships Sector (ESDPP) has taken steps towards updating its National Social Programs Manual and some AL Program management processes to provide guidance to regions in implementing the NCBR and AL Programs. In light of provincial and territorial variations in service delivery models, the AANDC manual provides some flexibility to regions to ensure programming is aligned to corresponding provincial and territorial programs. While this flexibility exists, our audit found that AANDC generally lacks understanding of provincial service delivery models and gaps in its own regional programs, hindering its ability to design programs that are aligned to those of provinces and territories.

There was little evidence of the use of risk-based management practices for administering NCBR and AL Programs at HQ and in regions (i.e. selection of funding instruments, establishing reporting requirements, monitoring recipient program implementation and performance, and auditing recipient compliance). While ESDPP Sector is working toward implementing a risk-based compliance regime for the AL program, the draft regime we reviewed is too heavily focused on verifying prior-year expenditures and needs additional attention on evaluating alignment of program delivery to AANDC or provincial requirements.

While considerable attention has been paid by the HQ program to updating the Performance Measurement Strategy and recipient reporting templates, data validation and management practices are not yet adequate to ensure completeness and accuracy of data. Additional challenges exist in gathering relevant information from provincial organizations that administer programs on AANDC's behalf, as their data does not always align to how AANDC gathers information. ESDPP Sector recognizes these gaps and, as of the end of 2013/14, had not yet begun reporting against its key performance indicators (KPIs) for these programs. Effective 2014/15, the Data Collection Instruments (DCI) had been updated to ensure necessary data is being collected from recipients to support performance measurement reporting.

In addition to performance information gathered for purposes of reporting against KPIs, considerable information is being gathered on NCBR projects (e.g., narrative descriptions on detailed results achieved by every NCBR project) and AL programs (e.g., details of services received and funding allocated to every AL recipient). While this information is useful for purposes of compliance monitoring, very little compliance monitoring is being performed of recipients.

Conclusion

The audit found that management controls for the AL and NCBR programs require strengthening to ensure that the programs are delivered in an effective manner. To support the design and implementation of important changes to the AL and NCBR programs over the medium term, better information provincial/territorial program models and funding support levels is required. More immediate improvements can be made to ensure that requirements placed on recipients are risk-based and that funding surpluses and ineligible expenditures are identified and managed in a timely manner.

Recommendations

The audit team made the following recommendations to improve management controls for the NCBR and AL Programs:

The ADM ESDPP should review the NCBR program policies and procedures to ensure that funding allocation processes are rigorous and aligned to corresponding provincial/territorial program support levels. Further, recipients should be required to demonstrate that their selection of NCBR projects aligns to an assessment of their community's needs and priorities.
The ADM ESDPP should review and gain an understanding of provincial and territorial programs for both NCBR and AL and identify opportunities to better align AANDC's programs and support levels to those of provinces/territories. Further, to inform potential discussions with possible provincial/territorial partners in each region, AANDC should gain an understanding of its existing program delivery gaps and cost variations (e.g., differences in service levels within sub-regions, funding levels for different types of funding agreements, consistency of types of services funded across a region, etc.)
The ADM ESDPP should institute a risk-based administrative regime for the NCBR and AL programs that is considerate of program-level risks and recipient-level risks. Attention should be paid to implementing risk-based regimes for selecting appropriate funding instruments, establishing program and recipient reporting requirements, and performing risk-based monitoring activities.
The ADM ESDPP should review processes for validating and consolidating data collected from NCBR and AL recipients to improve accuracy, usefulness and timeliness of performance information.

Management Response

Management is in agreement with the findings, has accepted the recommendations included in the report, and has developed a management action plan to address them. The management action plan has been integrated into this report.

1. Background

An Audit of National Child Benefit Reinvestment (NCBR) and Assisted Living (AL) Programs was included in Aboriginal Affairs and Northern Development Canada's (AANDC) 2014-2015 to 2016-2017 Risk-Based Audit Plan (RBAP) approved by the Deputy Minister on February 6, 2014. The audit of the NCBR and AL Programs was identified as an audit priority because of the complexity of both programs.

NCBR and AL are two of the five transfer payment programs comprising the Social Development Program at AANDC. They are administered in accordance with the Treasury Board (TB) Policy on Transfer Payments (PTP) and the TB Directive on Transfer Payments. The expected outcome of the Social Development Program is "First Nations men, women and children are active participants in social development within their communities".

The NCBR and AL Programs are important because they serve the needs of those most vulnerable in the Aboriginal population. The NCBR serves children in low-income families, while AL serves low-income seniors, adults with chronic illness, and children and adults with disabilities (mental and physical) so that they can maintain functional independence.

National Child Benefit Reinvestment:

The NCBR is one component of a broader federal/provincial/territorial National Child Benefit (NCB) initiative. AANDC is responsible for the NCBR on reserve in collaboration with Employment and Social Development Canada (ESDC), the lead federal department on the NCB initiative.

The NCB initiative has two components:

A financial benefits component: comprised of the federal Canada Child Tax Benefit (CCTB) and National Child Benefit Supplement (NCBS), and applicable provincial/territorial integrated child benefits. These benefits are paid directly to eligible families by the Canada Revenue Agency.
A reinvestment component - the National Child Benefit Reinvestment (NCBR), which provides services and supports for children in low-income families. This component is administered by provinces/territories/AANDC.

Exhibit 1: NCBR Regional Funding by Fiscal Year (millions of $)

The two components have been designed to work together. The NCBS was introduced by the federal government in 1998 so that provinces, territories and First Nations could choose to reduce (offset) their social assistance payments by an amount up to the full amount of the NCBS without negatively affecting any family's overall level of income. As per the NCB Governance and Accountability Framework and federal NCB authorities, provinces/territories that choose to offset their social assistance benefit rates/payments by the NCBS are required to put the resulting 'savings' toward services/supports for children in low-income families. Referred to as the "reinvestment model", the offset mechanism is what generates funds for the NCBR. AANDC is required to mirror the reinvestment model (i.e., the offset mechanism) of the reference province/territory and an amount equal to the offset must be reinvested into supports and services for children on reserve.

While AANDC must mirror the reinvestment model (offset mechanism) of the reference province/territory, the types of programming provided on reserve and off reserve under the NCBR are different. This is deliberate and consistent with the design of the federal, provincial, and territorial NCB partnership initiative. Provincial/territorial NCBR programming has been designed pursuant to (e.g., adjusted to, and to address gaps in) the suite of services and supports provided by federal, provincial, territorial and municipal bodies off reserve; provincial/territorial NCBR programming has been designed to meet the unique needs of off-reserve communities. AANDC NCBR programming has been designed pursuant to the suite of supports on reserve and to meet the unique needs of First Nations. This design is a strength of the NCB initiative; it allows the NCBR to be adapted to community-identified needs and regional variations in availability of social supports.

AANDC NCBR programming activity areas were developed in collaboration with First Nations. They are periodically reviewed and updated if required, and validated by First Nations. On-reserve NCBR programming continues to differ from off-reserve NCBR activities to avoid duplication of services of programs/benefits that are already available on reserve through other social/health programs. For instance, while the provinces and territories have used the NCBR to provide supplementary health care benefits, on reserve most residents receive those types of benefits through First Nations Non-Insured Health Benefits. To use the NCBR on reserve for these health benefits would be direct duplication of an already available support, and would therefore not be an efficient use of government funds.

Exhibit 2: Per Capita Spending on NCBR Programming

At present, the NCBR is not delivered in Manitoba, New Brunswick or Newfoundland and Labrador, because those provinces have elected to allow the full NCBS to go directly to families without a corresponding reduction in social assistance benefits. Accordingly, the NCBR is not administered on reserve in Manitoba, New Brunswick, or Newfoundland and Labrador. Although the Province of Quebec does not formally participate in the National Child Benefit initiative, it has developed a similar system of family benefits and supports. In 1998, the Province of Quebec and AANDC agreed that the Quebec child benefit be extended to families on reserve provided that AANDC reinvested the savings generated into NCBR supports and services. AANDC provided funding for NCBR on reserve in Quebec from 1998 to 2012. The NCBR no longer operates on reserve in Quebec.

In all other regions, AANDC's NCBR provides funding for community-based supports and services under five activity areas: child care; child nutrition; support for parents; home to-work transition; and, cultural enrichment. The expected outcomes include a reduction in the incidence, depth and effects of child poverty.

On reserve, AANDC administers reinvestment funding to First Nation communities and organizations that in turn deliver the NCBR supports and services. Actual expenditures reported for NCBR in the 2012/13 Departmental Performance Report were $49.9M. In 2013/14, the Departmental Performance Report, actual expenditures were $50.6M and planned expenditures reported in the 2014/15 Report on Plans and Priorities were $54M.

Assisted Living:

Exhibit 3: AL Funding by Fiscal Year (millions of $)

AL is a residency-based program, introduced in 1982, that provides funding for non-medical, social support services to seniors, adults with chronic illness, and children and adults with disabilities (mental and physical) so that they can maintain functional independence and achieve greater self-reliance. AL is an application-based program where persons in need of service/supports request them through a funding recipient (i.e. a delivery agent, such as a First Nation in-home care provider, or institutional care organization, etc).

AL is available to individuals ordinarily resident on-reserve who have been formally assessed by a health care professional (in a manner aligned with the relevant provincial or territorial legislation and standards) as requiring services and who do not have the financial means to obtain such services themselves.

There are four components to the AL program: In-Home Care, Adult Foster Care, Institutional Care, and the Disabilities Initiative. AL's In-Home Care provides financial assistance for non-medical personal care for individuals who need assistance with activities of daily living, complementing Health Canada's Home and Community Care Program which funds medical services on reserve. AL's Adult Foster Care and Institutional Care provide funding for eligible individuals in need of personal non-medical care in supported living environments. AL's Disabilities Initiative provides funding for projects to improve the coordination and accessibility of existing disability programs and community services to persons living on reserve.

Exhibit 4: AL Per-Capita Spending by Region and Fiscal Year

The AL Institutional Care component has experienced "scope creep" where there has been a misalignment between services on and off reserve. Currently AL authorities provide for funding for institutional care for individuals with lower care needs. These individuals were identified as Type 1 and Type 2 clients based on a 1973 federal classification system. AANDC has continued to fund using this classification system, while provinces and territories have now evolved their social assistance programming by separating health and social costs across all levels of care. AANDC has consequently taken on the reimbursement of some health-related care costs. The expected outcome for AL is that in-home, group-home and institutional care supports are accessible to low-income individuals, to help maintain their independence for as long as possible.

Actual expenditures reported for AL in the 2012/13 Departmental Performance Report were $98.3M. In the 2013/14 Departmental Performance Report actual expenditures reported were $99.9M, and planned expenditures reported in the 2014/15 Report on Plans and Priorities were $86.9M.

Table 1: 2013-2014 Assisted Living Expenditures per Program Component by Region, Vote 1, 10 and Statutory Expenses (in Millions)
	AT	QC	ON	MB	SK	AB	BC	YT	HQ	Total
Block Funding	3.18	7.37	3.01	10.86	1.44	4.99	0	2.17	0	33.04
In-Home Care	0.59	2.56	8.17	4.24	3.01	1.73	0.14	5.79	0	26.24
Institutional Care Type 1	1.27	1.76	0.00	4.22	8.74	0	3.92	0	0	19.90
Institutional Care Type 2	0.80	0	0.40	9.33	1.03	0	0.93	2.20	0	14.69
Foster Care	0	0.81	0	0	0	0.03	0.13	0.18	0	1.14
Service Delivery	0.07	1.26	0	0	0	0	0	0.94	0	2.27
Disabilities Initiative	0.06	0.13	0.17	0.15	0.16	0.14	0	0.24	0	1.03
Admin Costs	0.11	0.13	0.09	0.08	0.14	0.03	0.02	0.04	0.89	1.53
Total	6.07	14.02	11.84	28.88	14.52	6.92	5.14	11.57	0.90	99.85
Source: Compiled from the AANDC financial system by the AL and NCBR program.

2. Audit Objective and Scope

2.1 Audit Objective

The objective of the audit was to assess the adequacy and effectiveness of program management controls for the NCBR and AL programs.

2.2 Audit Scope

The scope of the audit covered the period April 1, 2012 to September 30, 2014 and includes assessments of the:

Implementation of the performance measurement strategy, including design of processes and systems used to collect, review, analyze and report on program performance information;
Appropriateness of policies, procedures, guidelines, tools, learning and development activities, and other supports put in place by the national programs and regional offices in support of program implementation;
Design of risk management processes and program delivery approaches intended to support the risk-based administration of the NCBR and AL programs;
Design of application and proposal assessment processes and funding allocation for the NCBR and AL programs; and
Adequacy of processes in regions to monitor the use of program funds and manage the reinvestment of recipient funding surpluses.

3. Approach and Methodology

The audit was conducted in accordance with the requirements of the Policy on Internal Audit and followed the Internal Auditing Standards for the Government of Canada. The audit examined sufficient, relevant evidence to provide a reasonable level of assurance in support of the audit conclusion.

The audit activities followed included:

Interviews with key officials at headquarters (HQ) in the Income Support Directorate (ISD) and Operations and Quality Management Directorate (OQM) within the Social Policy and Programs Branch (SPPB) of Education and Social Development Programs and Partnerships Sector (ESDPP), and at the Corporate Information Management Directorate (CIMD) within the Chief Financial Officer (CFO) Sector with responsibility related to the management of the NCBR and AL Programs;
Interviews with key officials in the three regions visited during the Conduct Phase;
Review of relevant documentation related to the management and delivery of the NCBR and AL Programs;
Review of a sample of 16 recipient agreements files across the three regions visited;
Review of NCBR proposal processes and NCBR / AL recipient funding allocation approaches and methodologies for all AANDC regions; and,
Analysis of NCBR and AL program spending and performance information.

3.1 Selection of Regions for Site Visits

As part of the approach and methodology for addressing the audit criteria, as identified in Appendix A, three regions were selected for site visits. Factors in the selection of the regional offices to be visited included:

The extent to which the office administers NCBR and AL programming;
Variability in NCBR and AL funding levels between 2010/11 to 2012/13;
Coverage of AL programming across the different AL sub-programs (e.g., Institutional Care, In Home Care, Foster Care, and Disabilities Initiative); and
Inclusion of at least one region having an AL sub-component delivered in collaboration with a provincial or territorial government.

Based on the above analysis, the Yukon, Saskatchewan and Ontario Regions were selected for site visits.

3.2 Selection of NCBR and AL Recipient Files for Process Walkthrough

Sixteen (16) recipient agreements and related files were reviewed to gain an understanding of practices employed by regions in administering the NCBR and AL programs. Factors considered in the selection of recipient files included:

Size of the FN community and funding levels, with a view to covering small, medium and large communities;
Recipient risk level based on the General Assessment, with a view to covering the range of recipient risk profiles;
Extent of funding amount variability between 2010/11 to 2012/13, with a view to selecting a few recipients with swings in funding levels; and,
Extent to which AL sub-programs were funded, with a view to maximizing coverage of both Institutional Care and In Home Care sub-programs.

Based on the above analysis 16 recipient files were selected, including:

Four recipients within the Yukon region, two with funding for both NCBR and AL, and two with funding for AL only;
Six recipients within the Ontario region, two with funding for both NCBR and AL, two with funding for NCBR only, and two with funding for AL only; and
Six recipients within the Saskatchewan region, three with funding for both NCBR and AL, one with funding for NCBR only, and two with funding for AL only.

4. Conclusion

5. Findings and Recommendations

Based on the evidence gathered through examination of documentation, interviews and analysis, each audit criterion was assessed and concluded upon. Where a significant difference between the audit criterion and the observed practice was found, the risk of the gap was evaluated and used to develop the conclusion and corresponding recommendations for improvement.

5.1 Program Policies and Procedures

ESDPP Sector has a National Social Programs Manual which serves as a procedural and guidance document for regions implementing social programs, including NCBR and AL. While the manual seeks to establish principles, parameters and guidelines for implementing the AL and NCBR programs, it provides considerable flexibility to regions to ensure programming aligns to the level of support provided by provinces/territories and is sensitive to the circumstances of each region. An update to the manual is currently in draft format and is scheduled to be finalized in the coming months for implementation in the 2015-16 fiscal year.

5.1.1 NCBR Program Policies and Procedures

While the National Social Programs Manual provides some guidance and parameters to regions on eligible recipients and expenditures, it allows regions flexibility in how they administer programs. For example, recipient selection and funding allocation is left to regions and there is no guidance provided on recipient monitoring and compliance. Presently, regions establish maximum recipient funding levels based on formulae or historical funding levels and then request proposals from recipients to demonstrate how their maximum allocation of funds will be spent. While some changes to the manual are envisioned for the 2015-16 fiscal year, they are insufficient to address gaps identified above.

Considering the degree of flexibility available to recipients in selecting NCBR projects (i.e., any activity which supports child care, child nutrition, support for parents, home to-work transition and cultural enrichment), we expected to find that communities were being asked by AANDC, in addition to demonstrating eligibility, to demonstrate due diligence in how they identified community needs and priorities and selected projects that best aligned to their needs. We note that we would not expect AANDC to judge or evaluate community priorities, but rather ascertain that recipients have set priorities and selected projects in a rigorous and transparent manner that considers their community needs and priorities. We found that, that while guidance provided to regions by HQ Program reinforces the importance of tying project selection to identified community needs and priorities, we saw no evidence of this in regional proposal assessment practices. We note that other AANDC funded programs require recipients to following planning processes, examples include Child and Family Services, the Urban Aboriginal Strategy, and Capital Facilities and Maintenance Program.

Having NCBR projects based on community level needs and priorities would reduce the risk that funded NCBR projects do not address the real needs of lower income families in communities.

Recommendation:

1. The ADM ESDPP should review the NCBR program policies and procedures to ensure that funding allocation processes are rigorous and aligned to corresponding provincial/territorial program support levels. Further, recipients should be required to demonstrate that their selection of NCBR projects aligns to an assessment of their community's needs and priorities.

5.1.2 AL Program Policies and Procedures

The National Social Programs Manual provides guidance to regions on recipient and expenditure eligibility requirements for AL programming. Historically, the manner in which the federal government defined AL services was aligned to provinces, however provinces and territories have evolved their programs and definitions of services, resulting in substantial differences at present. As a result, the guidance in the current AANDC manual is of limited utility to most regions. In response to this gap, ESDPP sector is attempting to provide greater guidance in an upcoming update to the manual to address regional variations in program delivery. Considering the complexity of AL programming, ESDPP Sector hosts frequent AL Working Group meetings involving both HQ program staff and regional program staff to discuss and address emerging issues and program implementation challenges.

At the time of the Conduct Phase of this audit, we noted that Program and regional staff had limited information on service delivery gaps between AANDC's existing AL program activities and provincially comparable programming.

It is important to understand that AANDC AL program activities (in-home care and Type I and Type II Institutional Care) are but two elements of a much broader continuum of care (see diagram at Exhibit 5). For the social and health care systems to function efficiently, health care professionals and case managers must identify client needs and ensure they are provided with timely and appropriate services and supports. Furthermore, as a client or patient's needs move along the continuum of care, it is imperative that a health care professional assess their needs in a timely manner and that a case manager (social worker or health care professional) work with the client to ensure they receive timely and appropriate services and supports. Considering that AANDC does not refer or case-manage clients through the continuum of care, and could not be reasonably expected to do so considering its limited scope of services, the department is reliant on Health Canada and provinces/territories to perform assessments and referrals. If certain First Nation communities are underserved by health care professionals and case managers, there is little AANDC can do on its own to ensure that those community members who need AL services are assessed and provided with needed services and supports. In light of this reality, we expected to find that AANDC program and regional officials would be performing analyses on the types and extent of services provided to identify and understand gaps in service delivery and thus inform their discussions with Health Canada and provinces/territories, but found that this analysis has not been undertaken to date.

AANDC AL Institutional Care, In Home Care and Adult Foster Care funding may be provided to FN recipients, private institutions or provincial and territorial governments. We found that the recipients being funded were typically those who have been funded for many years and their funding levels are typically set based on historical costs or previous year actual expenditures. We saw limited proactive work to identify underserved areas and communities within regions.

The lack of understanding of provincial/territorial program support levels and AANDC programming gaps are most likely resulting in some First Nation communities being underserved.

Recommendation:

2. The ADM ESDPP should review and gain an understanding of provincial and territorial programs for both NCBR and AL and identify opportunities to better align AANDC's programs and support levels to those of provinces/territories. Further, to inform potential discussions with possible provincial/territorial partners in each region, AANDC should gain an understanding of its existing program delivery gaps and cost variations (e.g., differences in service levels within sub-regions, funding levels for different types of funding agreements, consistency of types of services funded across a region, etc.)

5.2 Risk-based Program Management

With an aim to ensure strong accountability for public monies and better results for Canadians, the Treasury Board Policy on Transfer Payments requires that transfer payments be managed in a manner that is sensitive to risk, that strikes an appropriate balance between control and flexibility, and that establishes the right combination of good management practices, streamlined administration and clear requirements for performance. The Treasury Board Directive on Transfer Payments establishes that administrative requirements placed on recipients for a given transfer payment program should be considerate of risks specific to the program, the value of funding in relation to administrative costs and the risk profile of recipients.

Our audit found very little evidence of risk-based management practices being implemented in regions in respect to selecting funding instruments, establishing reporting requirements, monitoring recipient program implementation and performance, and auditing recipient compliance. Our findings specific to each of the programs included in the scope of our audit are summarized in sections 5.2.1 and 5.2.2.

5.2.1 NCBR Risk-based Program Management

For the three regions included in the scope of our audit, we found no in-year monitoring of implementation of the NCBR program and only limited post year-end reviews of recipient reports to identify funding surpluses. We observed that reporting requirements placed on NCBR recipients are extensive, going beyond what is necessary for measuring performance. The length of NCBR Reports we reviewed ranged from 20 to 280 pages. Further, the additional information collected from recipients is generally not being used for purposes of recipient compliance monitoring, calling into question its utility. There is an opportunity to implement risk-based reporting regimes to reduce the reporting burden by streamlining the information being gathered in NCBR recipient reports. By streamlining reports, information could be summarized and made more accessible to regional program analysts, allowing them to review reports and provide advice to the FSOs managing the agreements. If additional details on projects are deemed necessary, these could be requested from recipients as required, rather than being requested from all recipients.

5.2.2 AL Risk-based Program Management

For the three regions included in the scope of our audit, we found limited in-year monitoring of implementation of the AL program by First Nation recipients and only limited post year-end reviews of recipient reports to identify funding surpluses. We did identify practices in respect of the management of agreements with private institutions and provinces / territories.

ESDPP Sector's OQM is in the process of developing a risk assessment tool to be applied to every recipient of AL funding and a compliance tool to be applied in evaluating higher risk recipients. We understand that OQM plans to train regional staff and implement the risk assessment and compliance regime in regions for the 2015-16 fiscal year. Based on our review of the proposed processes and tools and the results of our audit work, we found these draft tools to be too broad and burdensome, raising concern about whether they can be implemented by region. Further, we found that ESDPP Sector is working toward implementing a risk-based compliance regime for the AL program that is too heavily focused on verifying prior-year expenditures and would benefit from additional attention being placed on evaluating alignment of program delivery to AANDC or provincial requirements.

Recommendation:

3. The ADM ESDPP should institute a risk-based administrative regime for the NCBR and AL programs that is considerate of program-level risks and recipient-level risks. Attention should be paid to implementing risk-based regimes for selecting appropriate funding instruments, establishing program and recipient reporting requirements, and performing risk-based monitoring activities.

5.3 Program Performance Measurement

The Social Development Programs have developed an umbrella Performance Measurement Strategy (PMS) covering all of the AANDC social programs. This PMS has a significantly reduced number of key performance indicators (KPIs) in comparison to the version that existed prior to 2012-13 fiscal year and focuses on measuring program outcomes using data already available and data that can reasonably be obtained through minor modifications to recipient reports. We reviewed the most recent Departmental Performance Report and ESDPP Quarterly Report and found that performance against the NCBR and AL KPIs is not yet being reported for the NCBR and AL KPIs. Our discussions with ESDPP sector indicate that the program staff is working on improving the integrity of data provided by recipients and collated by regions. Further, some important information will only be available after recipient reports are received for the 2014-15 fiscal year.

NCBR and AL recipient reports, generally referred to as Data Collection Instruments (DCIs), are fillable forms that all recipients must complete annually. Effective fiscal year 2014-15, ESDPP Sector has amended these DCIs with the intent of focusing them on the information required for reporting on the PMS.

5.3.1 NCBR Data Collection

Our audit found that the NCBR DCI includes significant additional reporting requirements, beyond the information necessary to report on program performance. The additional information collected is useful for evaluating compliance with AANDC program requirements and for evaluating the eligibility of specific projects and expenditures undertaken by a given recipient. More specifically, the NCBR DCI is designed for a specific project as opposed to a summary of all projects completed by a recipient. During our site visits to regions, we found there is limited review of these DCIs and little evidence of a challenge role being played by FSOs and NCBR program staff in evaluating eligibility of actual expenditures and alignment of spending with the original recipient project proposals. Some of the larger First Nations and Tribal Council deliver upwards of 30 to 60 projects and submit DCIs ranging up to 280 pages. These NCBR DCIs are of such a size that regional program staff and Funding Services Officers cannot easily identify compliance issues and data integrity problems. Further, the information in these DCIs that is not necessary for reporting on AANDC's performance measures should not reasonably be required from recipients assessed as being lower risk. Rather, recipients could be asked for this information if they are chosen for a mid-year monitoring activity or post year-end compliance review or audit.

In addition, we understand that modifications have been designed into the Income Assistance DCI to provide AANDC with information about the amount of National Child Benefit Supplement (NCBS) offset through Income Assistance for reinvestment (i.e., supports for children in low-income families) in First Nations. Because NCBR funding is supposed to be equal to the NCBS offsets, knowing the amount of funding being offset will permit AANDC regions to better align NCBR funding to provincial / territorial support levels. Based on discussions with regions and program staff, we understand that challenges experienced in relation to reporting for Income Assistance may hinder the completeness and accuracy of this NCBR offset information, thereby impeding the ability of AANDC to align NCBR funding to provincial / territorial support levels.

5.3.2 AL Data Collection

For the delivery of AL, AANDC has funding agreements with First Nations, Tribal Councils, private institutions and provincial and territorial governments.

First Nation, Tribal Council and institutional recipients use a common DCI template to submit DCIs for the three AL programs, In-Home Care, Institutional Care Types 1 and 2, and Adult Foster Care (AL DCI). New data requirements were added to the AL DCI for the 2014/2015 fiscal year to facilitate collection of performance information required to report on the program's updated KPIs. Provincial and territorial government agencies who deliver institutional care and in-home care submit invoices and supporting documentation to AANDC in accordance with the terms set out in their funding agreements. A separate DCI exists for the AL Disabilities, focused on project information and results.

Our audit found that information provided pursuant to an arrangement with a provincial agency in one of the regions visited does not align to the performance information needed to report on the program's KPIs. The AL DCI submitted by other types of recipients requires information on services provided to each individual and includes both personal information and information about the service provided. Some recipients have raised privacy concerns about personal information being reported in relation to the In Home Care sub-program, which has led one region visited to only request aggregated information (i.e. information without personal identifiers). Based on discussions with region and ESDPP staff, we understand that a solution to these challenges has not yet been developed.

During our regional site visits, we noted that the collection and consolidation of data for In Home Care and Institutional Care is a time intensive process that includes uploading DCIs to AANDC's GCIMS system as well as manual entry of every data point in these DCIs into regional Excel spreadsheets. These regional spreadsheets are then consolidated into a national spreadsheet. The lack of timely, complete, accurate and consistently presented data impedes reporting on the achievement of KPIs and ultimately, program outcomes.

Recommendation:

4. The ADM ESDPP should review processes for validating and consolidating data collected from NCBR and AL recipients to improve accuracy, usefulness and timeliness of performance information.

6. Management Action Plan

Recommendations	Management Response / Actions	Responsible Manager (Title)	Planned Implementation Date
1. The ADM ESDPP should review the NCBR program policies and procedures to ensure that funding allocation processes are rigorous and aligned to corresponding provincial/territorial program support levels. Further, recipients should be required to demonstrate that their selection of NCBR projects aligns to an assessment of their community’s needs and priorities.	ESDPP will work collaboratively with ESDC (the federal lead on the NCB initiative) to review the policies and procedures to ensure they best support alignment with program authorities and provincial/ territorial levels of support.	ADM, ESDPP, jointly with SADM RO	March 31, 2017
	ESDPP will determine the appropriate regional NCBR allocations by applying the reinvestment model of the reference province/territory, by conducting Phase 2 of the NCBR Costing Project. As recommended in Phase 1 of the NCBR Costing Project (2013-14), the Income Assistance Data Collection Instrument has been revised to collect the data required to calculate the appropriate allocations in Phase 2.		March 31, 2016
	ESDPP and SADM RO will ensure that program management supports (e.g. through the Management Control Framework) clearly articulate and support First Nation community-based priority-setting of NCBR projects and activities.		March 31, 2017
	ESDPP and SADM RO will continue to work collaboratively with the Strategic Policy Directorate on the Health Canada/AANDC Community Development and Capacity Building Framework to ensure that comprehensive community planning is utilized as a tool by all First Nations Communities to support priority setting. Guidance will be provided in 2015-2016 from PSD through the follow-up of the audit on capacity building.		March 31, 2018
2. The ADM ESDPP should review and gain an understanding of provincial and territorial programs for both NCBR and AL and identify opportunities to better align AANDC’s programs and support levels to those of provinces/territories. Further, to inform potential discussions with possible provincial/territorial partners in each region, AANDC should gain an understanding of its existing program delivery gaps and cost variations (e.g., differences in service levels within sub-regions, funding levels for different types of funding agreements, consistency of types of services funded across a region, etc.)	ESDPP will conduct a scan of provincial and territorial programs and levels of support for both the AL and NCBR programs.	ADM, ESDPP, jointly with SADM RO	March 31, 2016
	ESDPP is developing a joint workplan with Health Canada to identify where there may be gaps in the suite of health and social supports on reserve, and gain a better understanding of opportunities to strengthen the alignment of ESDPP and Health Canada programs with provincial/territorial supports.		March 31, 2016
	ESDPP and Health Canada will gather information that may provide further information and details on programming gaps and cost.		March 31, 2019
3. The ADM ESDPP should institute a risk-based administrative regime for the NCBR and AL programs that is considerate of program-level risks and recipient-level risks. Attention should be paid to implementing risk-based regimes for selecting appropriate funding instruments, establishing program and recipient reporting requirements, and performing risk-based monitoring activities.	ESDPP will ensure that program management supports are clearly identified to support a risk-based administrative regime (e.g., General Assessment Tool).	ADM, ESDPP	March 31, 2016
	ESDPP will continue to collaborate with Audit and Evaluation and the Information Management Branch to streamline reporting requirements and implement risk-based monitoring activities.	ADM, ESDPP	March 31, 2016
4. The ADM ESDPP should review processes for validating and consolidating data collected from NCBR and AL recipients to improve accuracy, usefulness and timeliness of performance information.	ESDPP will continue to collaborate with Audit and Evaluation and the Information Management Branch on departmental initiatives to improve the accuracy, usefulness and timeliness of performance information.	ADM, ESDPP	March 31, 2016

Appendix A: Audit Criteria

To ensure an appropriate level of assurance to meet the audit objective, the following criteria were developed to address the objective as follows:

Criterion #1: NCBR and AL program risks, including risks at the program and recipient levels, are identified, documented, assessed and addressed.

1.1 Risks related to NCBR and AL program management are identified, assessed and addressed with appropriate management controls.

1.2 Appropriate mitigation strategies are identified and implemented based on the results of the general assessment of funding recipients for the NCBR and AL programs.

Criterion #2: The performance measurement regimes of the NCBR and AL programs are appropriately designed to support the measurement of contributions to outcomes.

2.1 The Data Collection Instrument (DCI) reporting requirements for recipients of NCBR and AL program funding are aligned to the performance management strategy.

2.2 Information management systems and processes are established and implemented to efficiently collect, track and report on performance indicators for the NCBR and AL programs.

2.3 Roles, responsibilities and procedures for the collection, tracking and reporting of performance information for the NCBR and AL programs are communicated and understood by program delivery staff.

2.4 Information on performance indicators for the NCBR and AL programs, including DCIs from recipients, is reviewed by program delivery staff for completeness, accuracy, timeliness, and compliance with DCI / reporting obligations.

Criterion #3: Performance information for NCBR and AL programs is reported at an appropriate level and in accordance with the performance measurement strategy.

3.1 Performance information for the NCBR and AL programs is analyzed by program staff in order to inform program and recipient management.

3.2 Processes and controls are in place to consolidate, analyze and report on performance information for the NCBR and AL programs nationally on an accurate, complete and timely basis.

Criterion #4: Guidelines, procedures, tools and ongoing support from Headquarters (HQ) are appropriate to support regional implementation of the NCBR and AL programs.

4.1 Manuals, guides, procedures and tools are aligned with program terms and conditions, and include clearly identified roles, responsibilities and requirements for the management and delivery of the NCBR and AL programs, and are appropriate given the capacity and experience levels of regional staff.

4.2 Program management and delivery staff has an understanding of their roles, responsibilities and requirements for the management and delivery of the NCBR and AL programs.

4.3 A plan is in place to guide medium-term resourcing of the NCBR and AL programs (HQ and/or regional levels).

4.4 Program management and delivery staff have sufficient capabilities to manage and deliver the NCBR and AL programs, and are provided with appropriate supports from HQ as needed.

Criterion #5: Adequate learning and development initiatives are provided to program delivery staff to enable the discharge of their responsibilities in support of the NCBR and AL programs.

5.1 Program management and delivery staff are provided with sufficient learning, development and mentoring to support management and delivery of the NCBR and AL programs in accordance with the program design.

Criterion #6: Regions proactively work with eligible recipients to make them aware of the AL and NCBR programs and support them in program launch and delivery.

6.1 Program delivery staff actively communicates with recipients who are eligible for NCBR and AL program funding but who are not currently delivering the programs.

6.2 Program delivery staff either provides capacity related supports, or ensures that other organizations funded by AANDC provide these supports, to recipients that are seeking to implement the NCBR and AL programs.

Criterion #7: NCBR and AL program funding allocation approaches take into account recipient risk profiles and capacity, and are aligned to program objectives, and terms and conditions.

7.1 Approaches for the assessment of NCBR project proposals, and for funding decisions and allocation of funding to recipients for NCBR and AL, take into account recipient risk profiles and capacity and are aligned to program objectives, and terms and conditions.

7.2 Approaches for the assessment of NCBR project proposals, and for funding decisions and allocation of funding to recipients for NCBR and AL, are communicated and understood by program delivery staff

7.3 Approaches and processes are in place to have NCBR project proposals assessed and funding decisions made in accordance with established procedures.

7.4 Approaches and processes are in place to have NCBR project proposals assessed and funding decisions made in accordance with established procedures.

Criterion #8: Regions take into account recipient risk profiles and capacity in establishing and managing funding agreements with recipients of NCBR and AL funding.

8.1 Funding agreements with recipients (e.g., types of contractual agreements, agreement controls, frequency and type of reporting, cash management schedule, support to recipients, etc.) are established based on consideration of recipient risks and capacity.

8.2 Funding agreements with recipients are managed based on consideration of recipient risks and capacity.

Criterion #9: Financial and non-financial results are monitored to inform program management, including the reinvestment of recipient funding surpluses, and recipient compliance with funding agreements.

9.1 Procedures and requirements for the monitoring of financial and non-financial results of the NCBR and AL programs address informing program management, including the reinvestment of recipient funding surpluses, recipient compliance with funding agreements and the spending of funds for purposes intended.

9.2 Procedures and requirements for the monitoring of financial and non-financial results of the NCBR and AL programs are communicated and understood by program delivery staff.

9.3 Processes and controls are in place to have monitoring of financial and non-financial results of the NCBR and AL programs in accordance with established procedures.

9.4 Monitoring results are reported to appropriate parties, and appropriate program management actions are taken, where required.

Appendix B: Relevant Policies/Directives

The following authoritative sources (i.e. Policies/Directives) were examined and used as a basis for this audit:

Treasury Board Policy on Transfer Payments
Treasury Board Directive on Transfer Payments

Did you find what you were looking for?

What was wrong?

I can't find the information

The information is hard to understand

There was an error or something didn't work

Other reason

Please provide more details

You will not receive a reply. Don't include personal information (telephone, email, SIN, financial, medical, or work details).
Maximum 300 characters

Date modified:: 2016-02-24