An Impact Evaluation of the Chrysalis Academy Programme focusing on the contribution of its Outdoor Component

Mkandawire, Sandra

An Impact Evaluation of the Chrysalis Academy Programme focusing on the contribution of its Outdoor Component

Master Thesis

2019

Abstract

This evaluation is an impact evaluation of the Chrysalis Academy (CA) Programme. The CA programme is geared to help youth in the Western Cape develop the skills and resilience needed to transcend the prevailing poverty, inequality, unemployment and crime in their communities. Chrysalis offers a 3-month residential and 5 year After-Care programme for NEET youth aged 18-25 that includes life and vocational skills training as well as counselling and community service. The programme also includes a 2-week outdoor/wilderness component that is deemed crucial to the success of the programme. The evaluation sought to respond to two overarching groups of questions. The first group of questions assessed the long-term impact of the programme with regards to education, employment, crime, drugs as well as family and community relationships. Considering that the outdoor component is regarded as the most crucial component of the programme, the second group of evaluation questions was geared to understand the causal mechanism of the CA outdoor component to assess whether it is consistent with the logic model of successful outdoor programmes and to understand the outcomes that can be expected from this phase. A quasi-experimental impact evaluation design was used to respond to the impact evaluation questions. A database of 14,614 past applicants from the Western Cape formed the basis of a sampling frame from which a random sample of 300 past applicants who received the programme and those not selected into the programme between 2014 and 2016 was drawn. Face-to-face interviews were scheduled with 35 contacted programme recipients and 35 contacted non-selected applicants and interviews were conducted using a structured questionnaire. A final sample of 32 programme applicants and 33 non-selected applicants was analyzed. Differences in the average outcome attainment between past participants versus non-participants were then assessed after Propensity Score Weighting was used to balance the treatment and control group on key variables related to the probability of being selected into the programme. To assess the causal mechanism of the outdoor component, in-depth, semi-structured interviews were conducted with some success and non-success case graduates of the CA programme to assess their experience of the outdoor component and the outcomes that presented as a result. The findings of the evaluation show that the CA programme does not have long-term impact as those who did not receive the programme also attained similar positive outcomes. However, there is a possibility of short-term outcomes attainment from the outdoor component as its causal mechanism is similar to that of successful outdoor programmes. Moreover, participants, regardless of their long-term outcomes, are able to achieve the short-term outcomes expected from the outdoor component. Based on the results, CA should assess how the effect of the outdoor component and possibly other phases of the programme, can be sustained for long periods, possibly by intensifying the After-Care programme. CA should also assess the assumptions around outcomes attainment such as labor market favorability and financial sufficiency and assess how these can be addressed within the scope of the programme.

Keywords

Programme Evaluation

Reference:

Collections

Masters

Full item page