Evaluating development effectiveness assessing and comparing the impact od education intervention in South Africa

Abstract:

This research is a contribution to the field of development evaluation. Much of the evaluation practice in development and public policy remains weak in scientific validity, and challenged by issues of attribution and comparability of results across different studies. After an in-depth review of the existing literature and an analysis of the current shortfalls and knowledge gaps in programme evaluation, the research proposes a methodological framework that allows for the empirical measurement and comparison of the impact of diverse types of interventions aimed at addressing a specific outcome of interest. The evaluation framework informs decision-making in social-economic development processes, by combing elements of theory-based counterfactual evaluation, multiple-treatment meta-analysis, mixed methods, and participatory approaches. The evaluation framework is tested in South Africa by utilising the proposed package of methods through two case studies presented in this thesis, to generate evidence for policy-makers, programme managers, and investors operating in the education sector. The first is an evaluation of the impact of the corporate social investments of Anglo American Platinum in Limpopo and North West provinces, that utilised geo-spatial features of mining operations to conduct a quasiexperiment. The second is a comparative analysis of major interventions implemented in South Africa to improve learning outcomes in public schools. The education meta-analysis is the first of its kind to be conducted in South Africa, and has revealed many locally-produced impact studies which had not previously been captured by international reviews on school interventions in developing countries. The empirical work conducted in this research confirms existing theories and reveals new insights into the role of the private sector, the proximity of schools to mines, psycho-social and economic factors, learner age and home language, educational material, quantity and quality of teachers, school management, and accountability systems, in affecting education outcomes. The research highlights some of the programmes and policies which have been most effective in South Africa’s schooling sector, while cautioning about the contextual factors and methodological design features which influence the effect sizes being reported in the evaluations of development interventions. The research concludes by reflecting on the experiences, data and cost analysis challenges, and the lessons learnt from the application of the proposed evaluation approaches in South Africa’s education sector. It discusses the limitations of the framework, and how this can be further refined for future use in other countries, sectors, and development policy contexts.

Description:

A thesis submitted to the Faculty of Commerce, Law and Management, University of the Witwatersrand, Johannesburg, in fulfilment of the requirements for the degree of Doctor of Philosophy.