What is the most cost-effective way to improve teaching through in-service professional development – intensive coaching, or light-touch training? In this study, Jacobus Cilliers and colleagues provide a head-to-head comparison of these two approaches through a randomised trial carried out among primary government-school teachers in South Africa. The study also provides important evidence of the efficacy of a controversial approach to learning involving the provision of detailed lesson guides – essentially scripts – among teachers.

Scripted lesson plans are often criticised for reducing teacher’s autonomy or de-professionalising teachers, but supporters contend they can be an effective and easily scalable way to raise low teaching quality to acceptable standards. This is especially true in developing countries where teachers themselves often have low levels of human capital. Still, the evidence base of the effectiveness of scripted lessons in low-to-middle income countries is limited.

To develop the programme under study, the authors worked with the Government of South Africa. In both the coaching and training interventions, teachers received the same materials, including detailed daily lesson plans, along with student reading booklets, flash cards, and posters. The goal of the programme was to improve early-grade reading, which is a crucial task in South Africa where just 22 per cent of pupils can read properly after four years of schooling.

The experiment ran in 180 government primary schools, with 50 schools each being assigned to each of the two treatment conditions. In these schools, all teachers received either short, intensive training carried out at a central venue or visits by reading coaches on a monthly basis, who observed their teaching practices and provided feedback. In the remaining 80 schools, teachers went on about their business as usual. In total, almost 3,000 pupils were followed from the baseline to the end-line survey two years after the intervention began.

In the centralised-training intervention, teachers received two sessions, once at the beginning of the academic year and once in the middle of the year, each lasting two days (for a total of 34 hours). The ratio of teachers to facilitators during the training was roughly seven to one. In the coaching intervention, specialist reading coaches instead visited teachers on a monthly basis over the duration of the academic year. The average teacher received ten visits. Each visit lasted around 90 minutes, with an hour devoted to teacher observations, followed by 30 minutes of feedback provision and demonstrations of correct teaching techniques. Information sessions were also held at the start of each term, along with one-to-three afternoon workshops each year. In total, teachers received around 37 hours of support. Three coaches, all of whom held at least an undergraduate degree and had past experiences as teachers and coaches, served 16-17 schools each.

The results show that both interventions had positive effects on pupil performance. After two years, teachers who received coaching improved their pupils’ reading comprehension by the equivalent of around one and a half years of learning more than teachers in the control group, while teachers who received the shorter, centralised training improved their pupils’ performance by around nine months of learning.

To better understand the mechanisms behind the effects, the team analysed what teachers did differently in the classroom as a result of the interventions. It turned out that coached teachers were more likely to split pupils into smaller reading groups, enabling individualised attention and more opportunities to practice reading. Formally, this ‘group-guided reading’ technique could explain as much as 68 per cent of the overall effect on learning.

Interestingly, the effects vary by class size and baseline pupil proficiency. There are essentially no effects of the programme among pupils in the largest 10 per cent of classrooms – or for the weakest fifth of pupils – suggesting that the approach is not a universal solution to raise standards in all contexts.

But what about costs? The coaching intervention cost roughly $43 per pupil annually, compared to $31 for the centralised training – but as the effects of the coaching were so much larger the authors find it to be more cost effective overall.

So should governments switch completely from more traditional, infrequent training to more frequent coaching sessions? Possibly, but the next step should first be to try this type of intervention at scale. Finding three highly skilled coaches is one thing, but you might need hundreds or thousands of them if you were to run a similar programme across an entire country. This, in turn, may decrease any positive effects. Indeed, a recent meta-analysis from the US found good-sized effects for small programmes, but these effects shrank as programmes got larger. Still, the same issue would probably apply to instructors of centralised training. And it certainly seems intuitive that regular feedback is more likely to change behaviour than less frequent training, holding the total time spent roughly constant.

One potential route to scale is through new uses of technology. For example, a study in Brazil found positive impacts of a virtual-coaching programme run via Skype. Such findings are intriguing. But perhaps the most straightforward type of technology to go for is the use of scripts, which this paper suggests have positive effects on learning both when presented through centralised training and intensive coaching. Indeed, if a script is good enough for Hollywood actors, maybe it is also good enough for teachers?