Archived - Audit of the Advance Payment Program

Date: December 4, 2008

PDF Version (77 Kb, 14 pages)

Table of Contents

• Executive Summary
• 1.0 Introduction
  • 1.1 Focus of the Audit
• 2.0 Audit Findings
  • 2.1 Audit Criterion: Eligibility of Applicants
  • 2.2 Audit Criterion: Completeness of Application and Supporting Files
  • 2.3 Audit Criterion: Accuracy of IRSRC Payment Details
• 3.0 Conclusion
• Appendix A - Audit Objectives, Criteria and Methodology
• Action Plan

Executive Summary

Indian Residential Schools Resolution Canada [Note 1] (IRSRC) is responsible for administering claims made by former students of the Indian Residential School System. In May 2006, the Government of Canada announced the approval of a final Indian Residential Schools Settlement Agreement (the Agreement), which provides at least $1.9B for common experience payments (CEP) to 80,000 living former students of the schools. In order to advance money to eligible recipients, the Advance Payment (AP) program was implemented on May 10, 2006 and disbursed approximately 10,300 payments to the end of the program on December 31, 2006. Any former student who resided at a listed Indian Residential School and was 65 years of age or older by May 30, 2005 was eligible for this advance payment. The $8,000 provided to claimants as part of the AP process was to be deducted from any future CEP payment.

PricewaterhouseCoopers LLP (PwC) was requested by IRSRC's Internal Audit and Risk Management Services to provide professional services for the audit of compliance of the payments made within the AP Program. The objective of this audit was to determine, based on a sample of AP files, whether payments disbursed under the AP program were accurate regarding recipient information and amount that the application and files to support the payment were complete, and that payments were made only to eligible recipients.

By the end date of the AP program on December 31, 2006, 13,547 applications had been received by IRSRC, of which 3,210 were deemed ineligible for payment. The scope of the audit covered the 10,337 AP files that are associated with a disbursement.

During the course of the audit, three events were identified which impacted the results of the audit work. These events were:

• A SADRE (case management database) system conversion which resulted in the loss of claimant information
• The Summation (Indian Residential Schools manual records research database) consolidation which resulted in changes to record numbering and linkages to SADRE
• Delays in designing process controls due to time constraints in developing and implementing the AP program

Because the newly created IRSRC department had been focused on its infrastructure and delivering its mandate, limited management controls to ensure the accuracy and completeness of payments had been designed and implemented when approximately the first 6,000 payments were processed. As a result of this stratified population, the following sampling strategy was adopted:

• Population 1: Initial 6,000 applications processed and paid prior to the full establishment and implementation of management controls into the AP process; resulting in a random sample of 95 files.
• Population 2: Remaining 4,337 applications processed and paid subsequent to the full establishment and implementation of management controls into the AP process; resulting in a judgemental sample of 25 files from this population.

This internal audit was conducted in accordance with both the Treasury Board Policy on Internal Audit and the Institute of Internal Auditors Standards for the Professional Practice of Internal Auditing. The internal audit conclusions were based on the assessment of findings against pre-established audit criteria agreed to by management and reflects the audit work conducted between August and September of 2007.

Key Findings

• Based on the work performed and information provided, it was concluded that no significant issues were identified regarding the accuracy of payments disbursed under the AP program. Further, in the sample of files reviewed, no instances of overpayments were identified.
• The following exceptions were, however, identified related to the eligibility and completeness of payments made:
  • For 10% of the files reviewed, evidence of eligibility could not be located to validate that the claimant was a resident of a listed Indian Residential School
  • 23% of the files reviewed in the first sample (0% in the second sample) were approved by individuals without proper delegated authority for FAA Section 34.

Recommendations

• IRSRC should ensure the availability of supporting documentation for AP eligibility and provide sufficient assurance that payments were made only to eligible claimants. For the upcoming CEP process, IRSRC should ensure that there is a sufficient audit trail of the Summation data used to validate each claim and that this data can be readily accessed for quality assurance and audit purposes.
• For any future system upgrades or data transfers, IRSRC should ensure that prior to the approval and implementation of the change, minimum data quality validation is completed to ensure relevant records are retained.

1.0 Introduction

Indian Residential Schools Resolution Canada (IRSRC) is responsible for administering claims made by former students of the Indian Residential School System. In May 2006, the Government of Canada announced the approval of a final Indian Residential Schools Settlement Agreement (the Agreement), which provides at least $1.9B for common experience payments (CEP) to 80,000 living former students of the schools. CEP payments include $10,000 for the experience of attending an Indian Residential School plus $3,000 for each year, or part thereof, that they attended a school beyond the first year. The Agreement also provides for an individual assessment process (IAP) to compensate those who suffered sexual, serious psychological and/or physical abuse.

In order to advance money to eligible recipients, the Advance Payment (AP) process was implemented on May 10, 2006 and disbursed approximately 10,300 payments to the end of the program on December 31, 2006. Any former student who resided at a listed Indian Residential School and was 65 years of age or older by May 30, 2005 was eligible for this advance payment. The $8,000 provided to claimants as part of the AP process was to be deducted from any future CEP payment.

There are six main sub-processes within the AP process, as illustrated below:

There are six main sub-processes within the AP process: Intake, Data Entry, Research, Advance Payment S.34, Finance and Records.

The mail room receives all AP applications and completes verification tests to identify any duplicate applications. Once testing is completed, the Records Office creates a file for the application and completes the data entry of the application's relevant information into the RIMS (records management database) and SADRE (case management database). The file is then forwarded to Research, where eligibility is confirmed via manual searches of Summation (a database comprised of the aggregation of individual databases housing the available manual records of Indian Residential Schools). Once eligibility for an Advance Payment is reviewed and confirmed, the file is forwarded to the delegated FAA signing authority, who completes the final approval and Section 34 signature. Once signed, the file is forwarded to Finance who enters the data into OASIS, the financial system, in order to generate the payment to be forwarded to the applicant. Once completed, the file is closed and sent to the Records Office for maintenance.

Subsequent to an initial assessment of the AP program, PricewaterhouseCoopers LLP (PwC) was requested by IRSRC's Audit and Risk Management Services to provide professional services for the audit of compliance of the payments made within the AP program.

The detailed objectives, scope and methodology of this assessment are outlined in Appendix A - Audit Objectives, Criteria and Methodology.

1.1 Focus of the Audit

Audit Objective

The objective of this audit was to determine, based on a sample of AP files, whether payments disbursed under the AP program were accurate in regard to recipient information and amount, and that the application and files to support the payment were complete and that payments were made only to eligible recipients.

Audit Scope

By the end date of the AP program on December 31, 2006, 13,547 applications had been received by IRSRC, of which 3,210 were deemed ineligible for payment. The scope of the audit covered the 10,337 AP files that were associated with a disbursement.

Audit Methodology

Because the newly created IRSRC department had been focused on its infrastructure and delivery on its mandate, limited management controls to ensure the accuracy and completeness of payments had only been designed and implemented when approximately the first 6,000 payments were processed. As a result of this stratified population, the following sampling strategy was adopted:

• Population #1 – Initial 6,000 applications processed and paid prior to the full establishment and implementation of management controls into the AP process
  • Statistical sample with a 95% confidence level, with an assumed 10% margin of error; resulting in a random sample size of 95 files to be reviewed
• Population #2 – Remaining 4,337 files subsequent to the introduction of management controls into the AP process
  • Judgemental sample of 25 files to be reviewed

Statement of Assurance

The audit was conducted in accordance with both the International Standards for the Professional Practice of Internal Auditing as prescribed by the Institute of Internal Auditors (IIA) and the Government of Canada's Policy on Internal Audit.

The internal audit conclusions were based on the assessment of findings against pre-established audit criteria agreed to by management and reflects the audit work conducted between August and September of 2007.

2.0 Audit Findings

All significant audit findings are presented in this section in accordance with the relevant audit objective and criteria.

Audit Objective

The objective of this audit was to determine, based on a sample of AP files, whether payments disbursed under the AP program were accurate according to original application and identification documentation; whether the application and files to support the payment were complete and; that payments were made only to eligible recipients.

2.1 Audit Criterion: Eligibility of Applicants

• Evidence on file that recipients were 65 or older as of May 30, 2005
• Evidence on file that demonstrates the recipient attended an eligible residential school

Evidence of eligibility could not be demonstrated for 10% of the sample tested.

As per the requirements of the AP program, evidence was required to demonstrate that eligible recipients of payments to former students had resided at a listed Indian Residential School (IRS) and were 65 years or older as of May 30, 2005, the day the Settlement Agreement negotiations were initiated.

Based on the files reviewed, it was found that evidence was on file to demonstrate that all applicants were 65 or older at May 30, 2005. However, sufficient evidence was not consistently on file to demonstrate that all recipients attended an eligible residential school.

As part of the AP process, validation of attendance at an eligible residential school was done by a qualified IRSRC researcher verifying the claimant's name against historical records – such as the quarterly returns or daily registers for residential schools. These documents were contained in IRSRC's Summation database and would appear as images to the researcher.

Of 10% of the files reviewed (10 from the first sample and two from the second sample), the source documents referenced on the claimants file (in SADRE) that IRSRC used to prove eligibility could not be located within the Summation database. It was indicated that this may have been a result of the consolidation of the Summation database during the administration of the AP program. Without the documents used to validate the claims of AP applicants, IRSRC cannot prove the eligibility of AP recipients and support the AP payment that was issued as a result.

Recommendations

• IRSRC should ensure the availability of supporting documentation to support AP eligibility and provide sufficient assurance that payments were made only to eligible claimants.
• For the upcoming CEP process, IRSRC should ensure that there is a sufficient audit trail of the Summation data used to validate each claim and can be readily accessed for quality assurance and audit purposes.

2.2 Audit Criterion: Completeness of Application and Supporting Files

• Applicants signature is present on the application on file
• Evidence on file of valid form of identification of applicant (source documentation)
• Consistency of personal information between application and identification on file
• Evidence of appropriate Section 34 approval of payment
• Evidence of appropriate Section 33 approval of payment

The applications and associated supporting files tested were complete, with the exception of two files missing secondary identification.

In the files reviewed, we found that the applicant's signatures were present on all applications. Additionally, we found that almost all applicants used valid identification (as identified in the IRSRC application) and that this information was consistent between the application and the information on file. In 2 of the 120 files tested, valid identification was on file, but there was no evidence of a second piece of identification, as required by the AP program if the first piece was not a birth certificate. Without the appropriate second piece of identification on file, there is a risk that the identification of the applicant was not appropriately validated prior to the disbursement of the payment.

In the first sample, 23% of the payments were approved by an individual without FAA Section 34 delegated signing authority.

Within OASIS, there are two key financial control points in the validation and financial payment process that were used as criteria for this audit – the Financial Administration Act (FAA) Section 34 and Section 33 approvals. Section 34 sign-off confirms that the payee is eligible for or entitled to the payment. Section 33 sign-off authorizes the payment of the $8,000.

A found a number of instances were found where Section 34 approval was on file without the proper delegated authority. Specifically, in the first sample of files tested, 23 did not have an appropriate Section 34 delegated signing authority for the corresponding time period. This was corrected with the delegation of authority for Section 34 effective July 12, 2006. As a result, there was a significant improvement in the second sample of files and all files reviewed from the second sample had an appropriate Section 34 delegated signing authority.

Appropriate delegation of authority for Section 33 and consistent approval was observed in all files reviewed.

Recommendation

No recommendation

2.3 Audit Criterion: Accuracy of IRSRC Payment Details

• Data (i.e. name and address) in SADRE is consistent and complete as compared to source documentation
• Data (i.e. name and address) in OASIS is consistent and complete as compared to source documentation
• Per OASIS, a disbursement of only $8,000 was made to eligible recipients

Information within the OASIS database was consistent with source documentation; however, discrepancies and missing information were identified within the SADRE database.

Claimant information in the OASIS financial system was compared against the claimants' applications for the sample selected and the information in OASIS was found to be complete and accurate. In the sample of files reviewed, no instances of overpayment (i.e. in excess of $8,000) were identified.

We found that the manual data contained in the SADRE database was not always consistent when compared to the source documentation. Inaccuracies identified within the SADRE database included, but were not limited to, date of birth and contact information (mailing address and phone number). Testing results included the following:

• In 20% of files tested in the first sample, applicant information did not match the information captured in SADRE
• In 8% of the files tested in the second sample, applicant information did not match the information captured in SADRE (the improvement in results is attributed to the introduction of key process controls within the AP program)

During the course of the audit, it was discovered that the mailing addresses for 1,514 files had been lost as a result of a system conversion within SADRE. As a result, 8% of the sample files reviewed did not have mailing addresses in SADRE.

As no inaccuracies were identified in the OASIS records tested, there was no impact of the SADRE data inaccuracies on the amounts disbursed.

Recommendation

• For any future system upgrades or data transfers, IRSRC should ensure that prior to the approval and implementation of any changes, minimum data quality validation is completed to ensure relevant records are retained.

For the upcoming CEP process, data entry will be the responsibility of Service Canada, and as such, no recommendation has been made relative to data accuracy. However, accuracy of key data will be critical in order for IRSRC to conduct an effective search of records and validation of claims to determine the appropriate CEP payment to be disbursed.

3.0 Conclusion

Based on the work performed and information provided, we did not identify any significant issues regarding the accuracy of payments disbursed under the AP program. Further, in the sample of files reviewed, no instances of overpayments were identified.

The following exceptions were, however, identified related to the eligibility and completeness of the AP payments and related supporting files:

• No eligibility record could be located to validate that the claimant was a resident of a listed Indian Residential School for 10% of the files reviewed
• 23% of the files reviewed in the first sample (0% in the second sample) were approved by an individual without the proper delegated authority for Section 34

Please see Appendix B Management Action Plans for the response from IRSRC.

Appendix A

Audit Objectives, Criteria and Methodology

This appendix is a detailed listing of all audit objectives and related criteria with a more detailed description of methodology and scope.

Audit Objective

The objective of this audit was to determine, based on a sample of AP files, whether payments disbursed under the AP program were accurate regarding recipient information and amount, the application and files to support the payment were complete and that payments were made only to eligible recipients.

Audit Criteria:

In order to achieve the audit objectives, the following criteria will be evaluated during the course of the audit:

Eligibility of Applicants

1. Evidence on file that recipients were 65 or older as of May 30, 2005
2. Evidence on file that demonstrates the recipient attended an eligible Indian Residential School
   Completeness of Application and Supporting Files
   
3. Applicant's signature is present on the application on file
4. Evidence on file of valid form of identification of applicant (source documentation)
5. Consistency of personal information between application and identification on file
6. Evidence of appropriate Section 34 approval of payment
7. Evidence of appropriate Section 33 approval of payment
   Accuracy of IRSRC Payment Detail
8. Data (i.e. name and address) in SADRE is consistent and complete as compared to source documentation
9. Data (i.e. name and address) in OASIS is consistent and complete as compared to source documentation
10. Per OASIS, a disbursement of only $8,000 was made to eligible recipients

Audit Methodology

Audit Scope

By the end date of the AP program on December 31, 2006, 13,547 applications were received by IRSRC, of which 3,210 were deemed ineligible for payment. The scope of the audit covered the 10,337 AP files that were associated with a disbursement.

Sampling Strategy

Because the newly created IRSRC department had been focused on its infrastructure and delivery of its mandate, limited management controls to ensure the accuracy and completeness of payments had been designed and implemented when approximately the first 6,000 payments were processed. As a result of this stratified population, the following sampling strategy was adopted:

• Population #1 – Initial 6,000 applications processed and paid prior to the full establishment and implementation of management controls into the AP process – statistical sample with a 95% confidence level, with an assumed 10% margin of error; resulting in a sample size of 95 files.
• Population #2 – Remaining 4,337 files processed subsequent to the introduction of management controls into the AP process; judgemental sample of 25 files.

Within each population, a random sampling technique was used to select the files for the audit sample. By using a random sampling technique, no bias occurred in the selection of the files used as part of this sample.

Action Plan

Footnotes

1. As of June 1, 2008, Indian Residential Schools Resolution Canada became part of Indian and Northern Affairs Canada. (return to source paragraph)