Development of a model for the monitoring and evaluation of nutrition and nutrition-related programmes in South Africa

Abstract

Evaluation research or programme evaluation was developed mainly for general use in social science, notably education and psychology, for understanding and addressing social problems. Rossi eta/. (1999) defined programme evaluation or evaluation research as "... the use of social research procedures to systematically investigate the effectiveness of social intervention programmes that are adapted to their political and organisational environments and designed to inform social action in ways that improve social conditions". There are different types of evaluation, namely the evaluation of need, implementation evaluation, impact evaluation and efficiency evaluation. Several authors and international organisations developed tools and guidelines for measuring performance of nutrition programmes. However, a comprehensive model or framework, based on the existing theory of evaluation research to evaluate nutrition programmes in South Africa, is lacking. The national Primary School Nutrition Program (PSNP) was introduced as one of the 100 day Presidential Lead Projects of the South African Reconstruction and Development Programme in 1994. This programme is implemented in all nine provinces of South Africa and costs the taxpayer approximately R500 million per year. The focus area of the PSNP is school feeding, nutrition education and health promotion through interventions such as parasite control and micronutrient supplementation. An evaluation component was never part of the design of the PSNP. However, in the beginning of 2000 the North West Province, and later in 2000, the National Department of Health, commissioned evaluation of the PSNP. Aims The aim of this study was, therefore, to develop a framework or model for the monitoring and evaluation of nutrition and nutrition-related programmes in South Africa, based on the theory of evaluation research and incorporating the researcher's experience gained in the evaluation of the PSNP. Methods The results of an in-depth study of available literature on evaluation research are synthesized and presented in Chapter 2. This formed the theoretical basis for the development of an evaluation framework. A tender for the evaluation of the PSNP in the North West Province was allocated to the Potchefstroom University for CHE. The researcher acted as project leader and was responsible for the development of the evaluation framework and the execution thereof. The researcher was also part of the core team for the national evaluation of certain aspects of the PSNP and involved in the development of the evaluation framework, the gathering of data, the analysis and interpretation of the data as well as feedback to the relevant stakeholders. For both studies, an evaluation framework was developed based on the tender specifications. Once this was done, appropriate instruments and measuring methods were developed. The fieldwork was done at national, provincial and school level. Once the relevant data were gathered, the data were coded, captured, analysed and interpreted. A final report of the evaluation, including conclusions and recommendations regarding the programme, was written for each evaluation. Consultation with the appropriate stakeholders was held and continuous feedback was given to the North West Province and the National Department of Health for the two evaluations respectively (Chapters 3 and 4). Results A comprehensive framework for the evaluation of nutrition and nutrition-related programmes was developed and discussed in Chapter 5. It is suggested that the theory based evaluation (TBE) approach is followed for the evaluation of these. Weiss (1972; 1988; 1997) described theory-based evaluation as follows: "TBE aims to surface the theoretical underpinnings of the programme in advance and use the theories to help structure the evaluation. It aims to describe the actual mechanisms that are related to good outcomes". The evaluation is expected to collect data to see how well each step of the sequence in fact comes out. The developed framework of the evaluation shows that at least the following four principal components, namely: inputs, outputs, outcomes and impacts should be included. A very important and unique development in the framework is that it links all the steps of the evaluation process to all the phases in the development of a nutrition programme design, protocol and implementation. Conclusions It was possible to develop a framework for the evaluation of nutrition and nutrition-related programmes in South Africa based on TBE and the researcher's experience with the development of an evaluation framework for the PSNP. This framework could serve as a guideline to assist evaluators with the evaluation of nutrition programmes in the future. Recommendations Monitoring and evaluation should form an integral part of any nutrition and nutrition related intervention programme. Especially in South Africa, a developing country with scarce resource, where its serious malnutrition and HIVIAIDS problems are addressed by multi-sectorial, mutli-level and multi-site intervention programmes, appropriate evaluation should be conducted. The evaluation component should be developed as part of the development of the programme. Billions of Rands are spend on these programmes and the funders should demand responsibility and accountability from the programme developers and implementers through appropriate evaluations.