Archived - Summative Evaluation of the Post-secondary Education Program

Key Findings

The findings indicate a considerable need for AANDC to review its current approach to post-secondary to better ensure short-term successes and long-term benefits to First Nation and Inuit people and to the Canadian economy and society broadly. To that end, this evaluation found the following:

Relevance

1. There is a continued need to fund and otherwise provide resources to First Nation and Inuit students seeking post-secondary education based on a growing Aboriginal population, coupled with projected labour market needs, and the potential for improvements in First Nation and Inuit educational outcomes to contribute to meeting those needs.
2. Education authority activities are generally aligned with Government of Canada priorities; however, recent major reforms are reflective of the need to better align activities and better ensure improvements in student success.
3. The roles and responsibilities of the federal government around post-secondary education remain unclear.
   
   Performance
4. There is evidence of some improvement in the achievement of outcomes for post-secondary with respect to interest in enrollment, and some indications of success. There is a strong need, however, for data collection improvements to allow for a reliable assessment of student enrollment and results.
5. There is evidence to suggest that community-based programming is occurring and providing positive results for students and communities.
6. The most common challenges facing First Nation students include academic readiness for post-secondary studies, access to finances for tuition and other costs, and a difficult transition period. Evidence further suggests the critical importance of family and community support, and the benefits of culturally-relevant curricula and transitional support from institutions.
7. The current approach to funding post-secondary students is problematic in that it does not allocate funding based on need, and that there is little accountability for the degree to which post-secondary funds are used to fund students.
   
   Efficiency and Economy
8. The current approach to programming may not be the most efficient and economic means of achieving the intended objectives of PSE programming.

Based upon these findings, it is recommended that AANDC:

1. Develop a strategic, transparent framework to ensure clear definitions of roles, responsibilities, commitments, funding approaches and accountabilities for post secondary education;
2. Further support partnership development with First Nations, education organizations and institutions to facilitatestudent transitions and support.
3. Work to develop data collection methods that allow for the reliable assessment of student enrollment and success.

1.1 Overview

This summative evaluation of the Post-secondary Education (PSE) Program was conducted in time for consideration of policy authority renewal in 2012-13. It follows a formative evaluation of the PSE Program in 2010, which provided a preliminary examination of the state of information on First Nations education at Aboriginal Affairs and Northern Development Canada (AANDC) and was required to approve the continuation of terms and conditions of the program as of March 31, 2010.

The evaluation was conducted concurrently with the summative Evaluation of Elementary/Secondary (ESE) Programming in order to obtain a holistic understanding of AANDC's suite of education programming and their impact on First Nation and Inuit communities. This approach allowed the evaluation team to minimise costs and reduce the burden for individuals and communities by collecting all the information at one time.

In line with Treasury Board's Directive on the Evaluation Function, the evaluation looked at issues of relevance (continued need, alignment with government priorities, alignment with federal roles and responsibilities), performance (effectiveness), as well as efficiency and economy.

All three components of PSE programming were included as part of this evaluation, namely, the Post-secondary Student Support Program (PSSSP), the University and College Entrance Preparation (UCEP), and Indian Studies Support Program (ISSP).

PSSSP is the primary component of the PSE Program. PSSSP provides financial support to First Nation and Inuit students who are enrolled in post-secondary programs, including: community college and CEGEP diploma or certificate programs; undergraduate programs; and advanced or professional degree programs.

UCEP provides financial support to First Nation and Inuit students who are enrolled in UCEP programs to enable them to attain the academic level required for entrance to degree and diploma credit programs.

ISSP program provides Indian organisations, Indian post-secondary institutions and other eligible Canadian post-secondary institutions with financial support for the research, development and delivery of college and university level courses for First Nation and Inuit students.

1.2 Program Profile

1.3 Current Evaluation

The current evaluations of ESE and PSE looked at the entire suite of education programming offered by AANDC. In the past, AANDC has evaluated components of its education programming, such as the Evaluation of the First Nations SchoolNet Program (2009), the Evaluation of the Special Education Program (2007), the Summative Evaluation of the Cultural Education Centres Program (2005), and the Summative Evaluation of Band-Operated and Federal Schools (2005). While evaluating programs from this lens has its benefits, the result is a lack of understanding of how all programs affect overall student outcomes. It is expected that evaluating Aboriginal education in this light will provide the Department with a clearer sense of the current state of Aboriginal education and insight on future direction.

Internally, the Evaluation, Performance Measurement and Review Branch's (EPMRB) evaluation work is informed by its Engagement Policy that provides Branch staff with a framework for including Aboriginal people and organizations in the evaluation process. The purpose of the policy is to:

• ensure Aboriginal involvement in, and contribution to, relevant and meaningful evaluations;
• define the roles and responsibilities of all parties;
• enhance communication and improve relationships between the Department and Aboriginal people and organizations; and
• ensure that reporting requirements placed on recipients of AANDC programs related to the evaluation are kept to the minimum possible.

The underlying principle is that, for evaluations to be relevant to, and have meaning for, Aboriginal people, Aboriginal people need to be fully involved in the evaluation process. As part of this evaluation, Aboriginal involvement was attained through interviews, surveys and case studies (including focus groups).

Furthermore, an Advisory Committee was established, composed primarily of national First Nation and Inuit education organisations, as well as representation from AANDC. The purpose of the Advisory Committee was to provide guidance and insights on the development of evaluation tools, and on broader interpretation of findings and forming recommendations. The Advisory Committee representatives met and/or were provided with documentation at various stages of the evaluation process to discuss and provide advice on:

1. Terms of Reference
2. Methodology Reports
3. Acquisition of Qualified Consultants
4. Potential Key Informants, Case Study Communities and Literature
5. Technical Reports
6. Evaluation Findings
7. Draft Final Evaluation Report

2.1 Evaluation Scope and Timing

The evaluation examined all three components of the PSE programming: PSSSP, UCEP, and ISSP. The Terms of Reference were approved by AANDC's Evaluation, Performance Measurement and Review Committee (EPMRC) on May 14, 2010. This evaluation provides information on program relevance, performance, and efficiency and economy to support the management of program authorities in compliance with the Policy on Transfer Payments.

Three consultants, Donna Cona Inc., Harris/Decima and the University of Ottawa were contracted to undertake primary data collection (see Section 2.3). Data collection and analysis was undertaken between September 2010 and April 2012.

2.2 Evaluation Issues and Questions

In line with the revised evaluation questions approved at EPMRC in September 2010, the evaluation focused on the following issues:

• Relevance
  • Is there an ongoing need for the program?
  • Is the program consistent with government priorities and AANDC strategic objectives?
  • Is there a legitimate, appropriate and necessary role for the federal government in the program?
• Performance
  • To what extent have intended outcomes been achieved as a result of the program?
  • To what extent has the program influenced the constructive engagement and collaborative networks to further education and skills development?
  • What are the factors (both internal and external) that have facilitated and hindered the achievement of outcomes? Have there been unintended (positive or negative) outcomes? Were actions taken as a result of these?
  • To what extent has the design and delivery of the program facilitated the achievement of outcomes and its overall effectiveness?
• Efficiency and Economy
  • Is the current approach to programming the most economic and efficient means of achieving the intended objectives?
  • How could the program be improved?

2.3 Evaluation Methodology

The following section describes the data collection methods used to perform the evaluation work, as well as the major considerations, strengths and limitations of the report and quality assurance.

2.4 Roles, Responsibilities and Quality Assurance

EPMRB of AANDC's Audit and Evaluation Sector was the project authority for the PSE evaluation, and managed the evaluation in line with EPMRB's Engagement Policy, outlined in Section 1.3, and Quality Assurance Strategy (QAS). The QAS is applied at all stages of the Department's evaluation and review projects, and includes a full set of standards, measures, guides and templates intended to enhance the quality of EPMRB's work.

Oversight of daily activities was the responsibility of the EPMRB evaluation team, headed by a Senior Evaluation Manager. The EPMRB evaluation team was responsible for overseeing all aspects of data collection and validation, as well as composition of the final report and recommendations.

All information from consultants was thoroughly reviewed by the Project Authority (EPMRB) and the Advisory Committee for quality, clarity and accuracy. Subsequently, the consulting firms thoroughly reviewed the analysis of data conducted by EPMRB to ensure their information was well-captured.

In addition, status updates and key findings were regularly provided to EPMRC for strategic advice and direction.

3.1 Continued Need

Is there an ongoing need for investment in First Nations Education programming?

Finding: There is a continued need to fund and otherwise provide resources to First Nation and Inuit students seeking post-secondary education based on a growing Aboriginal population, coupled with projected labour market needs, and the potential for improvements in First Nation and Inuit educational outcomes to contribute to meeting those needs.

The need for continued investment in Education programming is amply demonstrated by the significant gap in educational opportunities and outcomes between First Nation and other Canadian learners.^{Footnote 11} Findings in the current study are consistent with observations made in the Formative Evaluation of PSE Programming^{Footnote 12} - that the expected growth in the on-reserve population will undoubtedly generate greater needs for ongoing investment in First Nation education. Population projections obtained from Statistics Canada show a projected growth in the population of children on reserve of approximately 15 percent, or an additional 30,000 children (see Figure 1) over the next two decades. This does not include the potential implications of persons relocating to reserves as a result of amendments to Bill C-31.^{Footnote 13}

Text description of Figure 1: Projected Population Growth by Age Category of Interest from 2010 to 2026

This figure is a graph indicating the projected population growth by age category each year during the period of 2010 to 2026. There are three age categories represented by independent lines: 0-18 years, 19-64 years, 65 years and over. The population rates are indicated on the 'y' axis and the years are indicated along the 'x' axis.

The 0-18 age category in 2010 is listed at approximately 175,000 and progresses to 200,000 by 2022. By 2026, the population rate for this age category reaches approximately 210,000.

The 19-64 age category had the most pronounced population growth. The 2010 rate began slightly below 250,000 and consistently progressed at a 30 degree trajectory, ending at just under 350,000 by 2026.

The 65 years of age and older category has a slow growth rate, beginning at approximately 25,000 in 2010 and progressing at a consistent rate, reaching 50,000 by 2026.

Historically, Aboriginal students in Canada have been under-represented in post-secondary institutions. The 2006 Census data reveals that 62.5 percent had no certificate, diploma or degree, 25.8 percent achieved a high school certificate or equivalent, 3.7 percent completed an apprenticeship or trades certificate or diploma, 5.5 percent achieved a college, CEGEP or other non-university certificate or diploma, one percent had a university certificate or diploma below the bachelor level and, 1.5 percent completed a bachelors degree or higher.

Over the next decade, the Canadian labour market is anticipated to grow as a result of increased economic activity and an aging population that is entering its retirement years. A Human Resources and Skills Development Canada report, Looking-Ahead: A 10-Year Outlook for the Canadian Labour Market (2008-2017) indicates that between 2008-2017, about 5.5 million jobs are projected to open up due to an increase of economic activity and replacing existing workers - 67.2 percent of these openings will require post-secondary education.^{Footnote 14}

In 2006, 25 percent of the working age Canadian Aboriginal population was 15-24 years old, nine percentage points higher than the non-Aboriginal population (16 percent).^{Footnote 15} Although the Aboriginal population will account for five percent of the total labour force for those between 15-64 years of age in 2026, it is important to note that the projected Aboriginal labour force in the same year will be much larger in certain provinces and more prevalent among the 15-29 age group.^{Footnote 16} For instance, those between the ages of 15-64 in 2026 will account for 28 percent of Saskatchewan's labour force, 22 percent in Manitoba, eight percent in Alberta,^{Footnote 17} and the proportion is larger with respect to the 15-29 year age group.

With respect to the national picture, there are two important observations. First, participation,^{Footnote 18} unemployment and employment rates for non-Aboriginals fared better than for the Aboriginal population for the 15-24 age group, including the Aboriginal population on reserve. Secondly, Canada's Inuit population did not do as well in all three accounts in comparison to the Métis, North American Indian and on-reserve populations. In particular, more than half of the Inuit labour force participants aged 15-24 were unemployed. With respect to the Aboriginal population living on reserve, half of labour force participants aged 15-24 (52 percent) were unemployed, which was higher than the total Aboriginal population (21.6 percent) unemployment rate against a participation rate of 51.9 percent.

Key informants, case study participants and survey respondents emphasized the role of post-secondary education programming as a means of enabling increased Aboriginal participation in the labour force. Key informants often spoke of the cost-benefit ratio of providing post-secondary education to Aboriginal peoples, highlighting that a robust program would enhance their overall capacity to compete in an increasingly globalized and knowledge driven economy. Moreover, some key informants drew attention to the potential for Aboriginal individuals with post-secondary accreditation to participate in Canada's economic development, particularly in the North.

3.2 Alignment with Government Priorities

Is the program consistent with government priorities and AANDC strategic objectives?

Finding: Education authority activities are generally aligned with Government of Canada priorities; however, recent major reforms are reflective of the need to better align activities and better ensure improvements in student success.

Insofar as Education Authority activities relate to better education outcomes for First Nation students, there is a clear strategic alignment with government priorities. The Government's continued investments and stated objectives for First Nations education – as outlined in budgets 2011^{Footnote 19} and 2012^{Footnote 20} – clearly demonstrate recognition of the need for strategic changes and innovative planning needed in order to better align AANDC activities with desired outcomes for First Nation and Inuit students.

Specific recent examples include the formation of the National Panel on First Nations Education; the introduction of a First Nations Education Act by 2014 and the Government's ongoing commitment as reinforced in the 2012 Budget to invest over $300 million per year supports First Nations and Inuit in their post-secondary studies through the PSSSP and the UCEP Program.

Major revisions in education-related activities reflect a common sentiment expressed by government interview participants in this study – that while AANDC education programming outcomes are aligned with the Government's objectives, its broad activities are undergoing significant reform because outcomes are not improving in a satisfactory manner.

In terms of increasing labour market participation, AANDC in 2007-08 began working in partnership with Saskatchewan First Nations, tribal councils, the Province of Saskatchewan, provincial employers, and training institutions to promote a range of active measures strategies focused broadly on First Nation labour market development through a Memorandum of Understanding. AANDC has invested more than $5 million in strategic pilot projects since 2007-2008 in Saskatchewan, including approximately $3 million in 2010-11.^{Footnote 21} This is part of an active measures approach that the Department is currently looking to implement across the country.

3.3 Role of the Federal Government

Is the current role of the Government of Canada legitimate, appropriate and necessary for the improvement of First Nations education success?

Finding: The roles and responsibilities of the federal government around post-secondary education remain unclear.

The federal government obtained its legislative authority in matters pertaining to "Indians, and Lands reserved for Indians" under Section 91(24) of the Constitution Act, 1867. The Indian Act (1985), sections 114 to 122, allows the Minister to enter into agreements for elementary and secondary school services to Indian children living on reserves. Since the Constitution Act and the Indian Act do not make reference to post-secondary education, AANDC considers its involvement in Aboriginal post-secondary education as a matter of social policy rather than a legal responsibility.

There has been significant concern from oversight bodies such as the OAG and the Standing Committee on Public Accounts around AANDC's lack of clear roles and responsibilities surrounding Aboriginal education. They state specifically that "the Department must clarify and formalize its role and responsibilities, otherwise, its accountability for results is weakened and assurances that education funding is being spent in an appropriate fashion are unclear at best."^{Footnote 22}

The majority of key informants did not discuss the role of the federal government, and aside from providing funding for post-secondary students they were unable to indicate any further involvement from the Department. Relationships between the Department and post-secondary institutions are essentially non-existent, though some argue that the Department has a role to play with regards to Aboriginal-controlled post-secondary institutions.^{Footnote 23}.

4.1 Achievement of Intended Outcomes

To what extent have intended outcomes been achieved as a result of AANDC programming?

Finding: There is evidence of some improvement in the achievement of outcomes for post-secondary with respect to interest in enrollment, and some indications of success. There is a strong need, however, for data collection improvements to allow for a reliable assessment of student enrollment and results.

In terms of overall improvement in post-secondary results, about 70 percent of survey respondents suggested that there has been at least some improvement over the past decade, while about 19 percent suggested there has been no change, and 11 percent suggested it has become worse.

It is possible that perceptions in post-secondary outcomes are heavily tied to the general well-being of the community. Whereas, successes in ESE can be immediately observed, successes in PSE may be much less readily evident as many of the students may never return to their communities for work. If the labour market is weak, then it is likely that successful post-secondary graduates will relocate.

In elaborating on improvements in outcomes over the past ten years, it is apparent that one key measure of success for participants is improving motivation and interest among students to pursue post-secondary. Particularly, participants noted that they have far more applicants, and there is gradually increasing emphasis socially on the necessity to pursue post-secondary studies. This positive attitudinal shift is tempered by more tangible outcomes, which participants largely indicated were limited because of inadequate preparation at the high school level. Many participants raised concerns about students being otherwise interested and motivated to pursue post-secondary studies, but lacking in numeracy and literacy essentials.

4.1.1 Progression

Survey participants saw strength in the ability of schools to equip students to progress through elementary school, but less so through high school, and much less so to post-secondary or into the labour market (see Figure 2). This was not attributed to the secondary schools being weaker than elementary; rather to the cumulative skills of students not being attained over time – with respondents suggesting more students are being "pushed ahead" than actually learning. The sentiment across interview, survey and case study participants was that students are not learning through elementary school and are, thus, at a serious disadvantage at the start of secondary, causing a cumulative impediment to progression and very low graduation success. These findings echo those of key informants.

Text description of Figure 2: Participants' Ranking of the Ability of Schools to Adequately Prepare Students

The participants' ranking of schools' ability to adequately prepare students is represented on a scale of 1 to 7, with 1 being very poor, and 7 being very well, represented on the x-axis. The percentage of respondents is represented along the y-axis. For schools' ability to prepare students to advance through elementary school, about 1% ranked '1'; 2% ranked '2'; 14% ranked '3'; 14% ranked '4'; 25% ranked '5'; 28% ranked '6'; and 17% ranked '7'. For schools' ability to prepare students to advance through secondary, 0% ranked '1'; 15% ranked '2'; 17% ranked '3'; 26% ranked '4'; 22% ranked '5'; 14% ranked '6'; and 6% ranked '7.' For schools' ability to prepare students to advance to post-secondary, 2% ranked '1'; 16% ranked '2'; 22% ranked '3'; 25% ranked '4'; 17% ranked '5'; 16% ranked '6'; and 1% ranked '7.' For schools' ability to prepare students to advance to the workforce, 5% ranked '1'; 13% ranked '2'; 23% ranked '3'; 27% ranked '4'; 17% ranked '5'; 13% ranked '6'; and 4% ranked '7.'

Among post-secondary students, only 30 percent felt that the schools in their community were strong in helping students progress from secondary to post-secondary, while 22 percent said they were acceptable and 40 percent said they were weak. They related this primarily to a lack of adequate academic preparation and a lack of culturally-relevant learning.

Similarly, when post-secondary students were asked whether secondary schools adequately prepare those not necessarily interested in post-secondary for the labour market, 39 percent said they were weak while 13 percent said they were acceptable, and 48 percent they were good or excellent.

When post-secondary students were asked how they felt they would fare in the labour market, the vast majority (78 percent) felt they would fare at least as well or better than their non-First Nation counterparts. Among these participants, however, there was a mix of fear of racism and a sentiment that in all probability, they would not be able to return to their First Nation because of a lack of job opportunities.

As for Inuit students, some interview and case study respondents say that the program has allowed Inuit youth to take part in the post-secondary schooling experience, and many have moved onto successful careers as a result. Even if students do not graduate from the program, they have had an opportunity to instill some of the skills required to ensure some level of success. The program allows for Inuit students to experience life in the South, and gain first-hand opportunities for positive life experiences.

4.1.3 Success

Post-secondary success is difficult to gage empirically, as student data are not recorded in the same fashion as elementary/secondary, and because only students funded through AANDC programming are tracked in its databases. Critically, in managing student information from one semester to the next, progression and graduation rates cannot be reliably calculated. The data contain all the students that had been accepted for funding for their post-secondary education as of November 1, and do not contain students that have not been accepted yet for funding, or who are enrolled in winter and/or summer sessions, even if they have been provisionally admitted to a program.

Among funded students, it can be surmised that the number of funded students enrolled in trades or college declined between 2003 and 2009, and remained relatively stable for students enrolled in university (see Figure 3).

Text description of Figure 3: Number of First Year (fall semester) Students Funded by Year, by Type of Program (1 = College/Trades; 2 = University)

The year of enrollment (2003 to 2009) is represented along the x-axis, and the types of program (college/trades versus university) are represented by separate bars within each year. The number of First Nation students funded in each type of program is represented along the y-axis. The bars representing college/trades is a little higher than 7,000 students per year, slowly declining over 2005 and 2006, and leveling off from 2006 onward at about 6,000 students per year up to 2009. The bar representing university enrollment is somewhat steady across the seven years roughly between 3,000 and 3,500 students per year.

Importantly, the number of Inuit students funded and enrolled had been declining sharply between 2003 and 2009 (see Figure 4).

The graduation rate cannot be calculated in the purest sense – that is, the proportion of students who begin post-secondary who ultimately graduate. However, it was possible to examine the proportion of enrolled students who were in their final year of their program, or past their final year (for example a student in a 4-year degree program who was in their fourth year or higher), who were logged as having graduated. As shown in Figure 5, there have been some improvements and the proportional increase has been significant^{Footnote 24} over time. The most considerable improvements were noted in Atlantic region, Ontario and Manitoba.

First year enrollment in program types was steady over time; except Native studies, which has decreased at the college level and increased at the university level; and health sciences, which has increased at the college/trades level.

Text description of Figure 4: Number of Inuit First Year (fall semester) Students Funded by Year, by Type of Program (1 = College/Trades; 2 = University)

The enrollment year (2003 to 2009) is represented along the x-axis, with the type of program (college/trades versus university) represented by two separate bars within each year. The number of Inuit students funded per year is represented along the y-axis. For students funded for college/trades, there are 67 in 2003; 68 in 2004; 45 in 2005; 71 in 2006; 35 in 2007; 27 in 2008 and 37 in 2009. For students funded for university, there are 29 in 2003; 55 in 2004; 51 in 2005; 39 in 2006; 15 in 2007; and 17 in 2008 and 2009.

Text description of Figure 5: Proportion of Students in their final year or beyond their final year who are logged as having graduated, over time, and by Qualification Sought (1 = College/Trade; 2 = Undergraduate Degree; 3 = Graduate Degree)

The enrollment year (2003 to 2009) is represented along the x-axis, with type of qualification (college/trade certificate, undergraduate degree, and graduate degree) represented by three different bars within each year. The percentage of those in, or past, their final year, and having been logged as graduated, is represented along the y-axis. For those seeking a college/trade certificate, the percentage in 2003 is about 26% and slowly increases over time to about 30% in 2009. For those seeking a university degree, the percentage in 2003 is about 30% and slowly increases over time to about 40% in 2008, but shows a decrease to about 35% in 2009. For those seeking a graduate degree, the percentage in 2003 is about 34%, and this decreases to less than 30% in 2004, and slowly increases up to 35% in 2008 and 33% in 2009.

4.2 Program Influence on Collaborative Engagement and Networks

To what extent has the program influenced constructive engagement and collaborative networks to facilitate education and skills development?

Finding: There is evidence to suggest that community-based programming is occurring and providing positive results for students and communities.

Increasingly, post-secondary access and retention initiatives are being delivered in Aboriginal communities. Community-based education initiatives enable students to complete a diploma or degree program without leaving their community. Examples of community-based education initiatives include partnerships between municipal agencies and educational institutions,^{Footnote 25} distance learning programs^{Footnote 26} and tribal controlled post-secondary institutions. In some cases, satellite campus models have proven to be effective.^{Footnote 27}

One example of a tribal college partnership exists between Simon Fraser University (SFU) and Aboriginal-run NVIT. These institutions formed a partnership whereby Aboriginal students with some post-secondary education are given access to several years of work experience and the opportunity to attain a Bachelor's Degree in General Studies. SFU and NVIT worked together to develop similar degree accreditation programs, namely the Aboriginal Community Economic Development Program (ACED) and a Business studies option offered to the Interior Salish people.

Although the ACED program experienced a declining graduation rate, with 80 percent graduating in 2004 and 56 percent in 2006, most graduates elected to work as Economic Development Officers in their communities. A number of institutional differences between SFU and NVIT emerged and the ACED program has been discontinued. Nevertheless, Price & Burtch^{Footnote 28} have identified the ACED example as an opportunity to explore the complexity of collaborative partnerships in the field of Aboriginal education.

Another example of a positive community-based program initiative is the First Nations Partnership Program (FNPP), a generative curriculum model^{Footnote 29} that was developed in response to a request from a group of First Nation communities in central Canada to provide community-based child care training, which both reflected and maintained the cultural practices, values, language, and spirituality of the local population. Through the FNPP, indigenous educational expertise, particularly the knowledge of community Elders, is incorporated into the teaching and learning processes of First Nation students. Furthermore, the generative curriculum model exposes Aboriginal students to Euro-western curricula and encourages discussion regarding the similarities and differences of the two approaches within historical, cultural, political and personal contexts.

In addition to positive retention outcomes, the FNPP has been identified as a mechanism for strengthening community capacity, revitalizing intergenerational teaching and learning roles and advancing First Nations' social development goals.^{Footnote 30}

Bearing in mind that many community-based initiatives are new, data relating to their efficacy is limited. Nevertheless, community-based initiatives have been identified as useful mechanisms for alleviating considerable historical, institutional, geographic and financial barriers to Aboriginal post-secondary education.^{Footnote 31}

4.3 Factors Facilitating or Hindering Achievement of Outcomes

What factors have facilitated or hindered the achievement of outcomes?

Finding: The most common challenges facing First Nation students include academic readiness for post-secondary studies, access to finances for tuition and other costs, and a difficult transition period. Evidence further suggests the critical importance of family and community support, and the benefits of culturally-relevant curricula and transitional support from institutions.

4.3.3 Transition Challenges

Cost of living was the most pervasive issue raised in terms of transition to post-secondary, with 74 percent of student respondents identifying it as a significant challenge. Most participants who discussed this issue suggested that a living allowance is almost completely decimated by rent costs alone. To add to this, parents are rarely in a position to help financially. The limited availability of funds was said to cause stress and distraction from studies. Many First Nation participants noted that funds for living allowances have remained static despite massive increases in inflation, especially in larger cities.

Also pervasive was the issue of adjusting to a new environment. Most respondents indicated students struggle to adjust to city life, mass transit, apartment or residence living, etc. The majority of students said that finding a suitable residence is a challenge. There was a sentiment that many students feel disoriented and alone, often caused by the transition from a more collectivist to individualist culture and the resulting culture shock, which can significantly impede the focus on studies. Many believe this is a major contributor to drop-out, though some suggest that more consistent, well-funded specific First Nations counseling throughout the high school years, preferably on reserve, is key to helping students through this transition. Others suggest that bringing branches of local community colleges either closer to the reserves, or even onto reserves, would have a positive impact on both enrollment rates and graduation rates from these institutions.

Further, while child care considerations can act as a deterrent to pursuing post-secondary altogether, it can also be a serious issue for those eligible and willing to pursue it. Overwhelmingly, participants felt that living allowances for student parents were inadequate to cover actual day care costs, especially in larger centres. Additionally, and consistent with trends across Canada, there are significant issues with availability of child care generally.

As shown in Figure 6, all of these barriers on average were seen as impediments roughly equally. It should be noted though, that similar to the post-secondary students themselves, affordability of living costs was seen as the most pervasive issue by education administrators in terms of the frequency of it being ranked as a significant barrier compared to the other barriers.

Text description of Figure 6: Mean Degree of Barrier (from 1 being not a barrier to 7 being a significant barrier) Faced by Post-secondary Students

Each identified barrier is named along the x-axis. Its average ranking as a barrier on a scale of 1 to 7 is represented along the y-axis. Tuition funds is ranked around 4.7; living costs is ranked around 6; culture is ranked around 5.3; finding a residence is ranked around 5.6; and child care is ranked around 5.8.

4.4 Design and Delivery

To what extent has the design and delivery of the program facilitated the achievement of outcomes and its overall effectiveness?

Finding: The current approach to funding post-secondary students is problematic in that it does not allocate funding based on need, and there is little accountability for the degree to which post-secondary funds are used to fund students.

The design and delivery of post-secondary education planning emerged as a prominent topic of discussion in key informant interviews, with the FNSSP being identified as the most common positive element. Nevertheless, the participants were unable to specify any particular strengths of the program apart from noting that it provides some funding to Aboriginal students. Furthermore, when asked to identify the challenges facing post-secondary education, the vast majority of respondents pointed to issues surrounding funding, particularly, the shortage of available funds to provide access to post-secondary education for all qualified applicants and the insufficiency of existing funds to continue to support current students.

Most First Nations bands manage their own post-secondary funding. Tuition rates and living allowances are based on individual band policies. Case studies revealed that these can play a pivotal role in the extent to which bands support students needs, and when they do not, students may have to drop out with a debt to pay back to the band.

In 2011, the OAG released an update on their recent findings for First Nation programming, which found that AANDC has not explicitly reviewed its post-secondary funding mechanisms despite reviewing some delivery options. In line with key respondents, the report further noted that AANDC allocates its funds to communities without regard for the number of eligible students, and that band governments have the flexibility to allocate the funds for purposes other than education. Overall, the report found that "the current funding mechanism and delivery model used to fund post-secondary education does not ensure that eligible students have equitable access to post-secondary education funding."^{Footnote 36}

Furthermore, key informant and survey respondents frequently reported AANDC's funding cycle as being problematic. Many respondents emphasised that long-term initiatives—particularly, planning and leadership-building projects—cannot be embarked upon when levels of funding fluctuate on an annual basis.

The inclusion of cultural elements, such as the incorporation of community Elders into post-secondary education programming continues to be a challenge, despite some efforts to introduce these elements. Key informants indicated that Elders are not regularly compensated—in the form of honoraria and travel cost reimbursements—for their contributions to enhancing the post-secondary educational experience of Aboriginal students. Moreover, participants emphasized that greater efforts need to be made to incorporate the traditional knowledge of Elders into the design and delivery of post-secondary education programming.

6.1 Conclusions

This evaluation examined PSE program activities, including all three components of the PSE programming, namely, PSSSP, UCEP, and ISSP. In line with Treasury Board requirements, the evaluation looked at issues of relevance (i.e. continued need, alignment with government priorities, alignment with federal roles and responsibilities), performance (i.e. effectiveness), as well as efficiency and economy.

The findings indicate a considerable need for AANDC to review its current approach to post-secondary to better ensure short-term successes and long-term benefits to First Nation and Inuit people and to the Canadian economy and society broadly. To that end, this evaluation found the following:

Relevance

1. There is a continued need to fund and otherwise provide resources to First Nation and Inuit students seeking post-secondary education based on a growing Aboriginal population, coupled with projected labour market needs, and the potential for improvements in First Nation and Inuit educational outcomes to contribute to meeting those needs;
2. Education authority activities are generally aligned with Government of Canada priorities; however, recent major reforms are reflective of the need to better align activities and better ensure improvements in student success; and
3. The roles and responsibilities of the federal government around post-secondary education remain unclear.
   
   Performance
4. There is evidence of some improvement in the achievement of outcomes for post-secondary with respect to interest in enrollment, and some indications of success. There is a strong need, however, for data collection improvements to allow for a reliable assessment of student enrollment and results;
5. There is evidence to suggest that community-based programming is occurring and providing positive results for students and communities;
6. The most common challenges facing First Nation students include academic readiness for post-secondary studies, access to finances for tuition and other costs, and a difficult transition period. Evidence further suggests the critical importance of family and community support, and the benefits of culturally-relevant curricula and transitional support from institutions; and
7. The current approach to funding post-secondary students is problematic in that it does not allocate funding based on need, and that there is little accountability for the degree to which post-secondary funds are used to fund students.
   
   Efficiency and Economy
8. The current approach to programming may not be the most efficient and economic means of achieving the intended objectives of PSE programming.

6.2 Recommendations

It is recommended that AANDC:

1. Develop a strategic, transparent framework to ensure clear definitions of roles, responsibilities, commitments, funding approaches and accountabilities for post secondary education;
2. Further support partnership development with First Nations, education organizations and institutions to facilitatestudent transitions and support; and
3. Work to develop data collection methods that allow for the reliable assessment of student enrollment and success.

APPENDIX – Education Programs Intended Outcomes

Immediate Outcomes

• First Nations students actively participate in elementary and secondary education
• Learning environments with appropriate education programs and support services are in place
• Supported First Nations and Inuit students participate in post-secondary education
  
  Intermediate Outcomes
• First Nations learners progress in their elementary and secondary education, including students with special education needs reaching their full potential
• First Nations and Inuit post-secondary students progress in their program of study
  
  Ultimate Outcome
• First Nations and Inuit learners achieve levels of education comparable to other Canadians
  
  Strategic Outcome
• Individual, Family and Community Well-Being for First Nations and Inuit

AANDC Education Case Studies

School Principals, Managers / Directors of Education

Name of First Nation: ___________________________________________

Position / Type of Respondent: ____________________________________

Length of time in Position: _______________________________________

I would like to lead a discussion with you about local education programs and services. To guide us, I want to discuss several different aspects of the local education programs and services. These areas include:

• Parental involvement (parents volunteer, parents respond to letters sent home)
• Student involvement (volunteers, attend events, sit on committees)
• Community involvement (volunteers, attend events, sit on committees)
• Cultural inclusion (language course, culture in the curriculum)
• First Nation directed and controlled education

The following questions provide a framework for our discussion, and not all questions will apply to each interview. Please offer any information or perspective that you feel would be of value.

Before we begin, I want to mention that this evaluation follows the Personal Information Protection and Electronic Documents Act, as well as the Privacy and Access to Information Acts. As such, we would like to ensure that you fully understand the purpose of this evaluation. If you do not, please feel free to ask me more questions. Further, your name will not be associated with any of the data collected, the data will be securely stored and all information will be disposed of after five years.

I would like to tape our session. We are taping for several reasons: 1) it allows me to hold a conversation with you as opposed to being focused on note-taking; 2) it will allow for an improved record of the information you are willing to share with me today, and 3) it provides for a richer and more comprehensive analysis and reporting of what we are able to record. Your name and identifying information will be removed from any written records created from your interview. A summary of your interview can be emailed to you, for your own use. If you so wish, you may review the notes and make corrections as you see fit and send those along back to us. The data you provide will be stored in a security protected system both on the computer and in hard copy. Do you have any questions before we begin?

1. What is your definition of educating children in preparation for their lives as adults? [refer to laminated Card A to help facilitate discussion] (probes: A. What are their educational needs as infants, children, teenagers, and adults? B. What is the Ideal means for addressing the full educational needs of infants, children, teenagers, and adults? C. What are your limits in understanding what is best; do you feel you could adequately comment on how to best meet the needs?)
2. Please describe the current education system offered through your Band. Please comment on the ways in which each of these types of services are provided:
   1. How does the education authority work? (probes: Does it have strong relationships with the community, parents and students; Does it coordinate an advisory committee, Parent Council, or Student Council; does the authority's office have its own targets for success defined and action taken to meet those goals; How are new programs implemented; do teachers, principals, families, community leaders, or other community members take part in designing the way education is delivered and designed?)
   2. How does/do the school(s) work to meet the basic needs of those students in need? (probes: How many counsellors are employed and how do they work to meet student needs; Do you have child and youth workers and how do they help students; Are there after-school and weekend programs; are there other support programs (e.g., breakfast and/or lunch programs); How are other community service providers linked directly to students in-need of extra supports (e.g., Mentors, Leaders, Guardians, Police, Social Services, a network of providers that case conference).
   3. What kind of outreach and linkages does/do the school(s) do between student groups at school and between the school and the community? (probes: Does the school put on community feasts/gatherings, how often; Are socials held at the school for students or parents; How are parents involved in the day-to-day school environment (e.g., home-visiting, parent supports/counselling).
   4. What types of student data do you collect and how is it stored? (probes: How are student data analyzed; Are data linked back to student success indicators; Is there a system in place that enables you to know if student targets are met).
   5. How are students supported to reach high-levels of self-confidence and academic achievement? (probe: In what ways are tutoring services provided; Are any other programs that target building a belief system around academic achievement offered (e.g., counselling, mentoring programs, guidance from specially trained teachers, specific credit-based courses, scholarships/bursaries, and physical activity); Are there any school courses that are designed around ensuring these elements are met and are there systematic checks to ensure this is happening and they are effective; How are high school students linked to the adult world: such as college courses, volunteer program, work for pay program, summer job opportunities, and exchange programs).
   6. How does the school exemplify an environment with cultural understandings of all staff and students? (probe: Would you characterize the school(s) as having a welcoming environment that promotes and embraces culture and traditions; do teaching staff have opportunities for professional development and access to cultural curriculum and resources specific to some courses, and grades; How are language, culture, history, and cultural-relevance a part of the core subjects; Are Elders and Cultural Teachers brought into the school(s); Is there a Cultural room designated with resources).
3. Now I'd like to show you three models and ask you to place your First Nation's education programs and services within one of the models or to map out the way you see it on a blank model or a blank page. [Interviewer: explain the models, Cards D-F, and give respondent a blank model and a blank page, Cards G/H, and a pen]
4. How does the current AANDC funding and the various programs your Band receives funding under, support and limit this First Nations children's ability to have their full educational needs met? [ask respondent to complete Card C for the programs he/she is aware that are offered within this First Nation] (probes: A. What are the specific strengths and weaknesses of the way you receive funds? B. What are the strengths and weaknesses of what you can do with the AANDC funding you receive? C. What are the strengths and weaknesses of the current resources you have in-place to offer educational services (e.g., other federal/provincial government programs, or private sector, such as colleges/universities)? D. What are the additional needs of students beyond what AANDC provides funding for? E. What needs do students have that you feel are beyond the scope of what an educational system can offer?)
5. To what extent do AANDC's administrative systems and operational practices make it possible for this First Nation to deliver elementary/secondary programming in a cost-effective manner? What are some of the barriers? What are some of the efficiencies?
6. Are the current AANDC Education programs the most economic and efficient means for reaching First Nations student success? Do the costs outweigh the current outcomes or vice versa?
7. How do the various authorities impede or enhance the programs and services this First Nations educational system offers to its membership? Specifically, please discuss each of: A. Band Council, B. regional management organization, C. AANDC region, D. AANDC headquarters, and E. other education authorities (such as the local off-reserve school board/school)?
8. What current mechanisms are in-place that let you know how well student success is being achieved? (probes: A. How do you define student success? B. How are you measuring success rates? C. What are your current rates of success? D. Can you share (give) any data with me? E. What are your success targets for five years from now? F. What key factors will contribute to you improving student success over the next five and ten years?)
9. In what ways have these [show Card B] achievements been recently reached among students from this First Nation?
10. How does this First Nation as a whole, both support and limit student's ability to have their full educational needs met? (probes: A. What barriers, if any, exist that prevent different/alternative educational approaches from flourishing? B. What kind of support does the education department get from the community to support student success? C. In what ways do parents support and get involved in the school, why and why not? D. In what ways do students support and get involved in the school, why and why not?
11. How does/do this First Nations school(s) work to both support and limit student ability to have their full educational needs met? [show Card A] Again, go back to the life cycle and the educational programs and services available for infants, children, teenagers, and adults. (probes: A. What are they doing right? B. What are they doing wrong? C. What areas could they easily change to improve student success? D. What areas are difficult to change that impact student success? E. What could change, but would take more work/money/support?)
12. [if off-reserve school that this reserve's students attend] How does/do the current off-reserve school(s) education programs and services both support and limit First Nation student's ability to have their full educational needs met? [show Card A] Again, discuss the educational programs and services off-reserve that your band members attend for infants, children, teenagers, and adults. (probes: A. What are the programs and services doing right? B. What are they doing wrong? C. What areas could they easily change to improve First Nation student success? D. What areas are difficult to change that impact First Nation student success? E. What could change, but would take more work/money/support?)
13. With regard to student success, think back ten years (what first hit your mind when you thought back to 2000), and five years ago (what comes to mind about the 2005 mark). How have the achievements of the students changed with regard to: [show Card B]
14. Now let's talk about where this First Nations education goes from here? [Interviewer: show the models again, Cards D-F, and give respondent a blank model and a blank page, Cards G/H, and a pen], let's draw this First Nations Education Programs and Services as you think they need to grow in order to better attain optimal student success? Again, you could use a blank paper of the blank model to map it out. (probes: A. What are the needs of students in the future? [show Card A] B. What are the best practices of this future educational system? C. What are the new approaches as well as programs and services you would see being offered?)
15. Do you have any final comments or thoughts you can share?

Text description of The Seven Stages of Life

Graphics developed from teachings shared by Pauline Shirt, Cree, 5th Degree Medawin, Toronto Elder

Good Life 0-6 years
Infant > Child > Youth> Adult > Parent > Grand Parent > Elder

Wondering 7 - 13 years
Infant > Child > Youth > Adult > Parent > Grand Parent > Elder

Fast Life 14 - 20 years
Infant > Child > Youth > Adult > Parent > Grand Parent > Elder

Wandering 21 - 27 years
Infant > Child > Youth > Adult > Parent > Grand Parent > Elder

Parenting 28-34 years
Infant > Child > Youth > Adult > Parent > Grand Parent > Elder Teaching 42-49 years
Infant > Child > Youth > Adult > Parent > Grand Parent > Elder

AANDC Education Programs Intended Outcomes:

Immediate Outcomes

1. First Nations students actively participate in elementary and secondary education
2. Learning environments with appropriate education programs and support services are in place
3. Supported First Nations and Inuit students participate in post-secondary education
   
   Intermediate Outcomes
4. First Nations learners progress in their elementary and secondary education, including students with special education needs reaching their full potential
5. First Nations and Inuit post-secondary students progress in their program of study
   
   Ultimate Outcome
6. First Nations and Inuit learners achieve levels of education comparable to other Canadians
   
   Strategic Outcome
7. Individual, Family and Community Well-Being for First Nations and Inuit

Education Authority
With Minimal Activities

Text description of Education Authority With Minimal Activities

Models adapted from Urban Aboriginal Education Pilot Project: Provincial Evaluators Findings, FINAL REPORT, 03 March 2011, by Johnston Research Inc.

1. Education authority commitment (Secondary Focus)
2. Focus on urgent needs of First Nation students (Secondary Focus)
3. Focus on outreach, Student-Data and linkages for First Nation students (Secondary Focus)
4. Focus on self confidence and academic needs of Focus on outreach, Student-Data and linkages for First Nation students in select settings (Primary Focus)

Education Authority
With Activities

Text description of Education Authority With Activities

Models adapted from Urban Aboriginal Education Pilot Project: Provincial Evaluators Findings, FINAL REPORT, 03 March 2011, by Johnston Research Inc.

1. Increased Education Authority commitment (Secondary Focus)
2. Focus on urgent needs of First Nation students (Secondary Focus)
3. Focus on outreach student-Data and linkages for First Nation students (Primary Focus)
4. Focus on self confidence and academic needs of Focus on outreach, Student-Data and linkages for First Nation students in select settings (Primary Focus)

Education Authority
With Many Activities

School Programming and Data Tool

Name of First Nation: ________________________________________________

I need your assistance in completing this environmental assessment tool that will assess the complex systems within your community and your community's education system.

Principals of each school should first be approached to complete this Tool; they may choose to appoint someone once they have reviewed what we are asking.

Ensure these Names Match: Situational Perspective Tool

School A:  ___________________________________________________________________

2010/11 Enrolment in A: _________ ; if off-reserve 2010/11 add non-FN enrolled: _________

School B:  ___________________________________________________________________

2010/11 Enrolment in B: _________ ; if off-reserve 2010/11 add non-FN enrolled: _________

School C:  ___________________________________________________________________

2010/11 Enrolment in C: _________ ; if off-reserve 2010/11 add non-FN enrolled: _________

School D:  ___________________________________________________________________

2010/11 Enrolment in D: _________ ; if off-reserve 2010/11 add non-FN enrolled: _________

School E:  ___________________________________________________________________

2010/11 Enrolment in E: _________ ; if off-reserve 2010/11 add non-FN enrolled: _________

Basic Descriptors of Key Activities

Students, aged 16 and Over who are Involved at School

Name of First Nation: ___________________________________________

Location of School and Grade: ____________________________________

Length of time in that school: ____________________________________

I would like to lead a discussion with you about local education programs and services. To guide us, I want to discuss several different aspects of the local education programs and services. These areas include:

• Parental involvement (parents volunteer, parents respond to letters sent home)
• Student involvement (volunteers, attend events, sit on committees)
• Community involvement (volunteers, attend events, sit on committees)
• Cultural inclusion (language course, culture in the curriculum)
• First Nation directed and controlled education

The following questions provide a framework for our discussion, and not all questions will apply to each interview. Please offer any information or perspective that you feel would be of value.

Before we begin, I want to mention that this evaluation follows the Personal Information Protection and Electronic Documents Act, as well as the Privacy and Access to Information Acts. As such, we would like to ensure that you fully understand the purpose of this evaluation. If you do not, please feel free to ask me more questions. Further, your name will not be associated with any of the data collected, the data will be securely stored and all information will be disposed of after five years.

I would like to tape our session. We are taping for several reasons: 1) it allows me to hold a conversation with you as opposed to being focused on note-taking; 2) it will allow for an improved record of the information you are willing to share with me today, and 3) it provides for a richer and more comprehensive analysis and reporting of what we are able to record. Your name and identifying information will be removed from any written records created from your interview. A summary of your interview can be emailed to you, for your own use. If you so wish, you may review the notes and make corrections as you see fit and send those along back to us. The data you provide will be stored in a security protected system both on the computer and in hard copy. Do you have any questions before we begin?

1. How should children and youth be educated in preparation for their lives as adults? [refer to laminated Card A to help facilitate discussion] (probes: A. What are their educational needs as infants, children, teenagers, and adults? B. What is the Ideal means for addressing the full educational needs of infants, children, teenagers, and adults? C. What are your limits in understanding what is best; do you feel you could adequately comment on how to best meet the needs?)
2. Where do you go to school (on- or off-reserve)? How well is your school performing, how well are students doing in your school? (probes: A. How do you define a successful student? B. What are the challenges for students in reaching that success? C. What is working well for students?)
3. How well would you say students within this First Nation are doing with reaching these [show Card B] achievements, most recently?
4. How does this First Nation as a whole, both support and limit student's ability to have their full educational needs met? (probes: A. What barriers, if any, exist that prevent different/alternative educational approaches from flourishing? B. What kind of support does the education department get from the community to support student success? C. In what ways do parents support and get involved in the school, why and why not? D. In what ways do students support and get involved in the school, why and why not?
5. How does the school you attend work to both support and limit First Nation students' ability to have their full educational needs met? [show Card A] Again, go back to the life cycle and the educational programs and services available for infants, children, teenagers, and adults. (probes: A. What are they doing right? B. What are they doing wrong? C. What areas could they easily change to improve student success? D. What areas are difficult to change that impact student success? E. What could change, but would take more work/money/support?)
6. Now let's talk about where this First Nations education goes from here? [show Card A] Again, discuss the changes that will best help this First Nations' people do well in school as infants, children, teenagers, and adults? What else should stay that same – what is just right, now?
7. Do you have any final comments or thoughts you can share?

Text description of The Seven Stages of Life

Graphics developed from teachings shared by Pauline Shirt, Cree, 5th Degree Medawin, Toronto Elder

Good Life 0-6 years
Infant > Child > Youth > Adult > Parent > Grand Parent > Elder

Wondering 7 - 13 years
Infant > Child > Youth > Adult > Parent > Grand Parent > Elder

Fast Life 14 - 20 years
Infant > Child > Youth > Adult > Parent > Grand Parent > Elder

Wandering 21 - 27 years
Infant > Child > Youth > Adult > Parent > Grand Parent > Elder

Parenting 28-34 years
Infant > Child > Youth > Adult > Parent > Grand Parent > Elder Teaching 42-49 years
Infant > Child > Youth > Adult > Parent > Grand Parent > Elder

Immediate Outcomes

1. First Nations students actively participate in elementary and secondary education
2. Learning environments with appropriate education programs and support services are in place
3. Supported First Nations and Inuit students participate in post-secondary education
   
   Intermediate Outcomes
4. First Nations learners progress in their elementary and secondary education, including students with special education needs reaching their full potential
5. First Nations and Inuit post-secondary students progress in their program of study
   
   Ultimate Outcome
6. First Nations and Inuit learners achieve levels of education comparable to other Canadians
   
   Strategic Outcome
7. Individual, Family and Community Well-Being for First Nations and Inuit