Audit of AANDC Support to the Independent Assessment Process

5.1.1 Governance, Oversight, and Strategic Direction

The audit expected the establishment of effective governance mechanisms and oversight bodies with clearly communicated mandates, and that management has clearly defined and communicated the organizations strategic direction, which is aligned to the IAP objectives.

The audit noted that AANDC participates in the "all parties" IAP Oversight Committee (as described in section 1.2).

AANDC established an Indian Residential Schools Settlement Agreement Operations Committee with draft terms of reference (January 10, 2010) which stated the scope of the committee "is to provide governance on all administrative issues and risks to the IRS Settlement Agreement". The Committee is chaired by the Director General of SAO, and membership includes representatives from AANDC, IRSAS, Department of Justice (DOJ), the Court Monitor (Crawford Class Action), and Service Canada.

SAO management noted the use of the IAP Claims Management Committee, which is co-chaired by the Director General of SAO and the Director of Legal Services, to instruct AANDC and DOJ activities and conduct as Canada's representatives to the IAP.

In support of communicating its strategic direction, SAO management complete business plans as part of the broader RIA Sector plan annually. SAO has established key activities including: providing all mandatory documents; attending hearings; conducting negotiated settlements; and processing IAP payments. Annual resource planning (staffing and financial) considers the annual number of planned hearings as communicated by IRSAS (e.g. 4,500 hearings were planned for 2013/14).

The audit concluded that governance mechanisms and oversight bodies are in place and adequate.

5.1.2 Accountability

The audit expected that management has clearly established and communicated expected organizational and individual accountability for services in support of the IAP, including responsibilities and performance expectations to which managers and supervisors are held accountable. These expectations should be formally defined and clearly communicated, and job descriptions and/or performance agreements should in place and maintained.

The audit found that employee performance management plans are prepared by staff members at the beginning of the year to set annual goals against which they will be evaluated. These annual goal are developed with guidance provided by management. The audit reviewed guidance provided for the Litigation Information Analyst and Resolution Manager positions. The guidance includes the departmental strategic objective with example work objectives and performance indicators to establish individual goals aligned to the SAO Branch objectives as reported in the annual RIA business plan.

The audit concluded that accountability practices are adequate and effective.

5.1.3 Results and Performance

The audit expected that Management has identified and monitors appropriate performance measures that are aligned to planned objectives in support of the IAP.

The audit noted the reporting of results and performance in support of the IAP provided in the IAP Dashboard report which includes all key steps in the IAP from claim admission, through document collection from claimant and Canada, negotiated settlement, hearings offered and held, decision and ultimately payment of the claim. This report is a collaboration between SAO, IRSAS and Crawford (Court Monitor), who have responsibilities to provide the reporting for the IAP steps that they support.

SAO also reports on a quarterly basis, the progress made for the key activities in support of the IAP. The key activities reported include: providing Canada's mandatory documentation, locating and contacting alleged perpetrators, case management, attending hearings, negotiating settlements and processing IAP payments. In addition, the Department reports on key IAP activities to Parliamentarians through the Departmental Performance Report.

The audit concluded that management's identification and monitoring of performance measures was adequate and effective.

5.1.4 People – Training and Tools

The audit expected management would have appropriate training, tools, resources and guidance available to staff responsible for providing support services to the IAP.

Management provides technical and supportive training for staff. Technical training is directed to the processes in place to support the IAP (including manuals and IAP training modules) as well as orientation to the Department as needed for hires new to the federal government or AANDC. The AANDC Learning Roadmap provides a number of orientation training courses (mandatory and recommended) that are provided by AANDC and by the Canada School of Public Service. Management noted that the approach to technical training has been developed based on the staff level through a combination of hands on learning, job shadowing, observing hearings and quality control checks. The audit noted the following examples of specific training approaches:

• Clerk level (e.g. CR4) – job shadowing is a primary method used and it is supported by an employee manual, job description and initial information orientation to the IAP.
• Litigation Information Analyst (e.g. EC 3) – Resolution Managers provide initial training combined with an experienced LIA who is dedicated to a new LIA to provide coaching and support.
• Resolution Manager (e.g. EC 5) – training for this position is the most formalized as these individuals represent Canada during the hearing. Training includes mandatory IAP 101 and 201 training modules as well as a three to four month period during which the trainee RM is observed during live hearings and provided feedback and coaching.
• Training and tools are available to staff for use of information systems, such as SADRE and GroupWise. Management noted that a protocol is in place to migrate IAP claim specific documentation captured in GroupWise into SADRE in preparation of the departmental email system transition to Microsoft Outlook. General IAP records are to migrate to CIDM.

Additional available tools include Sharepoint and knowledge sharing sessions. Sharepoint is a knowledge management tool use by the Department of Justice to which SAO has access to as a reference source. Monthly conference calls are arranged (Canada's Representatives Call) to provide a forum in which DOJ and SAO Resolution Managers discuss best practices and knowledge sharing. Management noted there is also a committee responsible (Cross Team Operations) to review the processes and tools.

Supportive training provided includes wellness training that has been made available to staff to help address vicarious trauma that may occur from exposure to details of claimant abuse allegations. The training has been in place since 2008 and was revised in 2014 to focus on resilience.

The audit concluded that training and tools are in place and are adequate.

5.1.5 IAP Completion Planning

The audit expected that management has clearly established and communicated plans to provide support services to the IAP over the wind down period. This would include the identification of organizational goals, priorities, identification of risks, required systems and staff capacity to provide support to completing the IAP over the wind down period.

Planning for the wind down of the IAP was noted by the following:

• As part of the ongoing development of a transition plan, the SAO Branch recently completed a value stream mapping exercise as part of its transition planning process to wind down over the time period 2014-15 to 2019-2020. The Branch has also initiated a Human Resources Management Process as part of the transition process that will look to align resources to evolving organizational needs.
• Management noted work is underway to prepare a proposal to provide funding for the period 2015/16 to 2019/2020; and
• Quarterly reporting 2013-14 included commitment to contribute to the development of a completion strategy produced by the Secretariat and reviewed by the audit. Management noted this contribution encompassed an articulation of SAO's role and general branch commitments.

The audit concluded that IAP completion planning is in place and is adequate.

5.1.6 Risk Management

Settlement Agreement Operations Branch is part of the Residential and Individual Affairs Sector. As such the SAO planning and reporting is included in the broader RIA Business Plan and Quarterly Reporting. The audit noted the identification of risks and mitigation strategies in the RIA Business Plan and Quarterly Reports. The audit did, however, note that a formal and comprehensive risk management framework was not in place in the SAO Branch to manage the support to the IAP.

Risk management is an important element of any large project such as the IAP and can be an effective tool to support setting priorities and resource allocation decisions in the achievement of strategic objectives and mandate. Risk management would include environmental scanning both internal and external to identify risks other risks that could impact the SAO ability to provide the necessary support to the IAP. An example of risk identification was noted in a recent business process mapping exercise, "the worst case scenario for SAO is increasing work load while it loses its specialized expertise and its operating systems begin to fail". Other potential risks that management identified include the risk that litigation arises from the IRSSA, including the IAP or Article 12^{Footnote 3} requests, resulting in additional efforts to provide support beyond the targeted wind-down period. Existing risk identification practices could be augmented to identify other key risks that could impact SAO and its ability to support the IAP to completion. The list of risks should be assessed for "likelihood and impact", with the most significant risks being monitored over time. This process could be included as a component of the annual business planning process.

Recommendation:

1. The audit recommends the Assistant Deputy Minister of the Resolution and Individual Affairs Sector implement a formal risk management framework for the Settlement Agreement Operations Branch. This could be integrated into the annual Settlement Agreement Operations planning process to identify, assess and monitor the most significant risks that may preclude or limit the achievement of effective support of the Independent Assessment Process.

5.1.7 Process Stewardship

The audit expected that processes and procedures are established and formally documented to provide clear guidance and stewardship in the execution of the operational procedures in place to ensure Canada's responsibilities are met throughout the claims process from admission to payment.

5.2.1 Strategic Direction

The audit expected that defined and communicated strategic direction is in place and aligned to the organization's mandate of supporting the IAP, along with clear and communicated operational plans with objectives aimed at achieving the strategic objectives.

The audit noted that IRSAS completed a strategic plan that was approved by the Executive Committee on March 22, 2013. Seven strategic objectives were identified:

1. Process claims in a timely manner;
2. Ensuring a claimant-centred approach
3. Manage resources economically, efficiently and effectively;
4. Manage information effectively;
5. Provide support to the Chief Adjudicator;
6. Promote a healthy work environment/organizational wellness; and
7. Completion of the IAP.

These objectives are reported in the IRSAS annual business plan and quarterly reporting. The strategic objectives were noted to be used to develop unit business plans.

IRSAS developed a resource planning tool to complete the annual operational planning of staffing and financial requirements to support the IAP. The tool is used to identify the planned activities, workloads and results. Against each of the planned activities, each individual Unit within the IRSAS forecasts the staffing required and related salary budget and the operating and maintenance budget. Further details allocate the individual staff to the particular strategic objective that will be supported by the individual activities. This approach was considered by the audit to be a good practice and highlighted by management as a new tool developed and used for planning for fiscal 2014-15 and subsequent fiscal periods.

The audit concluded that strategic and operational planning was adequate and effective.

5.2.2 Accountability

The audit expected that management has clearly established and communicated expected accountability for services in support of the IAP. Accountabilities include responsibilities and performance expectations to which managers and supervisors are to be held accountable. These are to be formally defined and clearly communicated and job descriptions and/or performance agreements should in place and maintained.

Interviews with management noted employee performance management plans are prepared by individual staff members at the beginning of the year to set annual goals against which they will be evaluated. IRSAS provided example work objectives provided to staff as example goals to consider and include, as applicable, in individual employee management plans. The audit reviewed a sample of employee performance management plans including Adjudication Manager, Admission Analyst, Scheduler and Case Management Manager. Work objective goals were found to be aligned to the applicable Unit and IRSAS business plan objectives and demonstrated communication and understanding of, and accountabilities for, goals aligned to strategic objective key activities.

The audit concluded that accountability mechanisms were adequate and effective.

5.2.3 Process Stewardship

The audit expected to find that operational procedures are documented and clear guidance and supporting tools are in place to ensure support services to the IAP are executed effectively. The audit also expected to see process improvements to the IAP support services over time.

IRSAS support services to the IAP are documented and provide guidance in the operational processes and procedures. Examples noted included:

• The IAP process is flowcharted to provide a high level illustration of the steps involve in the process and is used as part of the on-boarding process for newly hired staff;
• Client Services Unit has produced desk guides to support staff responsible to provide assistance to Self-Represented Claimants;
• Chief Adjudicators Office has prepared a CAO Control Framework document that provides CAO staff with the process to be followed for: budgeting, business plans, orientation, employee performance, training, and IAP procedures.
• Decisions Unit has developed a quality control framework that is used to monitor and manage quality of processing by staff. Supervisors in the Unit follow the process outlined to select samples to verify the completeness and accuracy of processing and document handling completed by staff.

The audit noted that IRSAS prepared a report that highlighted numerous process improvements for the IAP that have been identified and implemented since inception. The Report on Process Improvements for the IAP, March 2014 identified a list of ninety two specific process improvement changes impacting claims resolution, addressing priorities of the Settlement Agreement, administration and processing of claims, and adjudication.

The audit concluded that process stewardship support and process improvement mechanisms were in place and adequate.

5.2.4 People – Training and Tools

The audit expected management would have appropriate training, tools, resources and guidance available to staff responsible for providing support services to the IAP.

IRSAS uses a combination of the AANDC Learning Roadmap for training staff who are new to government as well as training that is focused specifically on the processes in place at IRSAS. The Learning Roadmap outlines required training for new public service employees offered by AANDC and the Canada School of Public Service.

Supportive training provided includes wellness training that has been made available to staff to help address vicarious trauma that may occur from exposure to details of claimant abuse allegations. This training is often delivered in coordination with SAO Branch.

The audit concluded that training and tools were in place and were adequate.

5.2.5 IAP Completion Planning

The audit expected that management has clearly established and communicated plans to provide support services to the IAP over the wind down period. This would include the identification of organizational goals, priorities, identification of risks, required systems and staff capacity to provide support to completing the IAP over the wind down period.

IRSAS has prepared and submitted a completion strategy and plan for resolving the remaining IAP caseload, which was provided to the Supervising Courts in December 2013. The strategy included: analysis of the claims admitted, progress made in resolving claim; opportunities to improve the rate of claims resolution; projections to resolve the claims inventory; and risk identification, assessment and mitigations.

IRSAS has also prepared detailed staffing projections integrated into a Completion Action Plan that was presented to the AANDC Workforce Management Board in May 2014. Detail staffing projections were included to present the expected staffing, both indeterminate and term, required over the IAP wind down period.

The audit concluded that IAP completion planning was in place and adequate.

5.2.6 Results and Performance

The audit expected that Management has identified and monitors appropriate performance measures that are aligned to planned objectives in support of the IAP.

The audit noted the reporting of results and performance in support of the IAP provided in the IAP Dashboard report which includes all key steps in the IAP from claim admission, through document collection from claimant and Canada, negotiated settlement, hearings offered and held, decision and ultimately payment of the claim. This report is a collaboration between SAO, IRSAS and Crawford, who have responsibilities to provide the reporting for the IAP steps that they support.

A key management reporting tool is the Quarterly Report which provides a summary of progress against the strategic priorities. Key activities are reported against each of the IRSAS strategic objectives, as on target or behind target based on performance measures and milestones. IRSAS quarterly reporting is done through the AANDC's reporting process, independently to the Treasury Board Secretariat, as well as to the Oversight Committee and the Courts.

Priorities are also reported and included in the executive summary section of the Quarterly Report. Management confirmed that priorities are special projects that are completed within the current year whereas key activities are ongoing across fiscal periods. The priorities reported in the 2013-14 Quarterly Report are identical to those reported in the 2013-16 Strategic Plan and were aligned to the strategic objectives. Some of the priorities for 2014-15 were revised and some remained the same to the prior year.

5.2.7 Risk Management

The audit expected that there are adequate risk management mechanisms in place that are appropriately linked to the IAP objectives. This includes the identification, assessment for likelihood and impact, and ongoing monitoring of risks that may impact or preclude the achievement of IRSAS objectives.

IRSAS has evolved its approach to risk management since the previous audit was conducted and now includes the preparation of a corporate risk profile that includes roles and responsibilities for the Executive Team, Risk Leads, Managers, and the Integration Planning Unit. The risks noted are: Aboriginal relationship; implementation; information management; legal; resource alignment; and capacity and capabilities. Each risk is individually reported with their mitigation strategies and risk leads responsible.

The audit noted good alignment of the Corporate Profile risks and those noted in the Completion Strategy. In reviewing a quarterly report for March 31, 2014, the audit team noted that procurement (risk of not having contract in place) was reported as a high risk, but this risk was not included in risk statements in the Corporate Risk Profile (although procurement was identified as a "risk driver" of the Resource Alignment risk area). Given the level of impact, there may be an opportunity to communicate procurement risks with a higher profile in the Corporate Risk Profile.

Documentation reviewed and interviews with management noted that risk management could be further evolved by including managers and supervisors as part of the risk management process. This would include increasing the breadth of the environmental scan for risk identification and assessment across Units to improve the completeness of identified measured risks. This could be integrated with the annual planning process currently in place. Unit objectives would be assessed for risks that could impact their achievement. The unit level risks should be assessed for "likelihood and impact" to identify the most significant risks that require mitigations to be in place. The allocation of resources should be considered in relation to the controls necessary to mitigate these risks. The top risks should be monitored during the year on a quarterly basis.

Recommendation:

3. The audit recommends the Executive Director of the Indian Residential Schools Adjudication Secretariat continue to evolve the risk management process through the provision of risk management training, and include formal risk identification and assessment processes at the unit level. This can be effectively integrated with the annual planning process to allocate resources, monitored to regularly respond to changes in risks, and used to support decision making.

5.3.1 AANDC Corporate Services (Procurement)

The audit expected to see effective consultation and cooperative relationships between IRSAS, SAO and AANDC CFO Sector relative to procurement.

AANDC CFO Sector provides procurement services support to both SAO and IRSAS. Planning work completed by the audit noted that procurement services were not considered to be a significant risk area from the perspective of SAO Branch, but was a risk area for IRSAS.

IRSAS identified procurement support as a key challenge for IRSAS in providing support to the IAP. The IRSAS Strategic Plan 2013-16 (issued March 22, 2013) noted that "despite having strategies, procedures, and plans in place, we continue to address a set of long-standing corporate challenges that are beyond the Secretariat's control but require collaboration across various sectors of AANDC if progress is to be realized. Management has placed renewed emphasis on human resources and procurement to ensure that the required capacity and services are in place".

IRSAS requires external assistance in support of the IAP from adjudicators, to conduct the hearings and issue decisions, and from lawyers, to provide independent counsel for the Chief Adjudicator and the Secretariat. AANDC Procurement was responsible for providing assistance to the IRSAS to establish the required contracts to have the services in place. However it was recognized by IRSAS and AANDC Procurement that the authority limits for the Adjudicator contracts and legal counsel was not sufficient at AANDC^{Footnote 4}. As a result the procurement support for these contracts requiring renewal and for new external legal counsel contracts was transferred to Public Works and Government Services Canada (PWGSC). IRSAS management indicated that the AANDC procurement group has been very supportive and helpful in assisting IRSAS engage with PWGSC procurement services. The audit did not assess the services being provided by PWGSC, as it is outside of this audit.

5.3.2 AANDC Corporate Services (Human Resources)

The audit expected to see effective consultation and cooperative relationships between IRSAS, SAO and AANDC's Human Resources and Workplace Services Branch (HRWSB) with regards to the provision of Human Resource services.

The Indian Residential Schools Settlement Agreement stipulates that the Government of Canada is to provide sufficient resources to ensure that IAP claims are processed. This means that both SAO and IRSAS are made up of federal employees, with the delegated authority for staffing flowing from the Deputy Minister of AANDC. Therefore all staffing activities must follow Public Service Commission and departmental processes and procedures, with AANDC's HRWSB providing Human Resource support and advice to both SAO and IRSAS.

The audit planning work completed noted that HR staffing has been a challenge for both SAO and for IRSAS. Staffing has been a consistent and significant risk in particular for IRSAS, who have historically experienced high levels of staff vacancy rates, as high as 30%.

The IAP is a process that must be completed in a timely manner and accordance with the Settlement Agreement. The Completion Strategy submitted to the Supervising Courts December 2013, projects that IAP claims admitted will be resolved by 2017-18 with appropriate infrastructure remaining in place to carry out the functions assigned by the Settlement Agreement.

The fact that both SAO and IRSAS are sunsetting organizations that will wind down as the IAP is completed creates a challenge to attract and retain sufficient staff to meet the supporting services required of the IAP. A key reason for staff turnover within both organizations is staff leaving for permanent jobs (i.e. indeterminate) across the federal government. Interviews noted that this tends to occur more often in centres where there is a large federal government presence with greater opportunities to transfer to another organization, such as in the National Capital Region. Many positions within IRSAS and SAO are staffed with term positions, with a "sunsetting" clause, meaning staff will need to find new employment once the IAP is wound down and their term expires. This creates an incentive for term staff to be constantly looking for other employment with more job security. Compounding these organizational challenges is the nature of the work which can expose staff to details of claimant abuse allegations, which can be very troubling and can create a more difficult working environment than experienced elsewhere.

These HR risks also have significant risks and implications for the Government of Canada and for AANDC in particular. These implications can be contradictory or "at odds". For example Canada must provide sufficient human resource support to the IAP to ensure that all claims get processed, otherwise risking the perception that Canada is not meeting their end of the Settlement Agreement. This must be balanced with the fact that all permanent employees remaining from SAO and IRSAS once the IAP winds down must be absorbed or re-deployed by AANDC, at a time when the Department is looking to streamline its workforce in order to manage within current budgets.

The audit noted that IRSAS and SAO have been engaging with AANDC HRWSB to identify opportunities to alleviate some HR pressures and streamline hiring processes. Most recently, between May to June 2014, IRSAS sought, and received, a number of flexibilities from AANDC's Workplace Management Board (WMB) that would help streamline staffing processes and give the Secretariat more staffing options. These include, for example, approving a number of critical indeterminate positions that can continue to be staffed in that manner, and the ability to launch new staffing processes before the applicable pool is depleted. Also, SAO and IRSAS have both been given an option of extending the expiry of staffing pools for term and indeterminate positions to 2 years, from the standard 1 year. The audit team encourages continued identification and implementation by HRWSB, SAO, and IRSAS, of flexibilities, strategies and initiatives that aim to streamline staffing processes in order to help alleviate some of the HR challenges faced by SAO and IRSAS, while remaining in line with the hiring principles of the Public Service Employment Act.

The audit also noted that SAO, IRSAS and HRWSB are experiencing some difficulties managing these staffing challenges cooperatively. For example, the audit noted delays in the implementation of staffing flexibilities provided to IRSAS by the WMB due to confusion and a delay in agreement on the nature of the decision. It is essential that once staffing initiatives are agreed upon, that they get implemented efficiently and effectively to avoid further staffing delays.

The audit observed that there is no formal agreement, such as a Memorandum of Understanding, between these parties that establishes and clarifies the roles and responsibilities, as well as service standards, related to managing HR staffing processes and the provision or HR advice and guidance. Management from HRWSB, IRSAS, and SAO indicated that a formal agreement such as this would be beneficial. Ideally this agreement would articulate the expectations, roles and responsibilities, and service standards of managing HR processes, as well as detail the specific staffing flexibilities and process deviations available to IRSAS and SAO hiring managers. The objective of the agreement would be to clarify expectations around the provision of HR services by HRWSB and requirements of managers at IRSAS and SAO, as well as to help ensure the accurate and timely implementation of staffing process flexibilities in place to support IRSAS and SAO.

Recommendation:

4. The audit recommends that the Director General of HRWSB, Executive Director of IRSAS, and Assistant Deputy Minister of RIA negotiate and implement a formal agreement, such as a Memorandum of Understanding, with regard to HR support services provided by HRWSB. This agreement would clarify the roles and responsibilities and service standards that govern HR staffing, and clarify IAP specific staffing flexibilities.