Abstract

Design: Cluster randomised trial with follow up two years after baseline (six months after intervention). A process evaluation investigated the delivery of sex education and broader features of each school.

Setting: Twenty five secondary schools in east Scotland.

Participants: 8430 pupils aged 13-15 years; 7616 completed the baseline questionnaire and 5854 completed the two year follow up questionnaire.

Main outcome measures: Self reported exposure to sexually transmitted disease, use of condoms and contraceptives at first and most recent sexual intercourse, and unwanted pregnancies.

Results: When the intervention group was compared with the conventional sex education group in an intention to treat analysis there were no differences in sexual activity or sexual risk taking by the age of 16 years. However, those in the intervention group reported less regret of first sexual intercourse with most recent partner (young men 9.9% difference, 95% confidence interval −18.7 to −1.0; young women 7.7% difference, −16.6 to 1.2). Pupils evaluated the intervention programme more positively, and their knowledge of sexual health improved. Lack of behavioural effect could not be linked to differential quality of delivery of intervention.

Conclusions: Compared with conventional sex education this specially designed intervention did not reduce sexual risk taking in adolescents.

What is already known on this topic

Despite the widespread assumption that sex education delivered by teachers can reduce sexual risk taking in young people, there have been few randomised trials large enough to show this and none in the United Kingdom

Several quasi-experimental studies have concluded that sex education is effective, but most randomised trials suggest it is not

What this study adds

Improvements in teacher delivered whole class sex education have some beneficial effect on the quality of young people's sexual relationships but do not influence sexual behaviour

Introduction

In Britain problems associated with young people's sexual health include high rates of teenage pregnancy,1 a rising incidence of sexually transmitted diseases, and unsatisfactory early heterosexual relationships. 23 Comprehensive sex education is regarded as essential to complement the provision of sexual health services for young people.1

Several overviews of sexual health programmes for adolescents have concluded that sex education can beneficially affect behaviour, although the evidence comes almost entirely from quasi-experimental studies rather than randomised trials.4–6 One review found only nine randomised trials of school based sex education,4 and only one trial showed a significantly positive effect on behaviour.7 A review of sexual health interventions for primary prevention found that quasi-experimental studies show that interventions work and randomised trials show that they do not.8 However, recent findings from the United States have been more positive.9 No randomised trials in the United Kingdom have been reported, and only one large scale UK evaluation of school sex education has been published.10

Sex education is more likely to influence behaviour if it is narrowly focused, has a clear behavioural message, and develops negotiation skills. 1112 A review of 12 HIV programmes based on psychological theory found them to be effective,13 but only one was school based. To date, school sex education has been delivered by teachers, outside experts, older pupils, or a combination of all three.10 As most UK secondary schools have teachers designated to deliver sex education as part of the curriculum, 114 this is the most sustainable mode of delivery.

Between 1993 and 1996 a sex education programme delivered by teachers was developed for 13-15 year olds in Scotland. We used a randomised trial to evaluate the programme between 1996 and 1999.

SHARE programme

The SHARE intervention (Sexual Health and Relationships: Safe, Happy and Responsible) is a five day teacher training programme plus a 20 session pack: 10 sessions in the third year of secondary school (at 13-14 years) and 10 in the fourth year (at 14-15 years). It is intended to reduce unsafe sexual behaviours, reduce unwanted pregnancies, and improve the quality of sexual relationships. The programme was developed and piloted in Scotland over two years in consultation with teachers, sex education specialists, and education and health promotion departments.15

The psychosocial and sociological theoretical basis of the programme has been set out previously.16 The programme combines active learning (for example, work in small groups and games), information leaflets on sexual health, and development of skills, primarily through the use of interactive video but also through role playing.15 It has the 10 characteristics that Kirby identified as necessary for effective programmes.12 The exercises were developed specifically for the intervention programme or modified from other packs.17–20

In the 12 control schools sex education for third and fourth years varied from seven to 12 lessons in total and was primarily devoted to provision of information and discussion. Only two schools routinely demonstrated how to handle condoms, none systematically developed negotiation skills for sexual encounters, and teachers' training in sex education was generally limited.

Methods

Recruitment and randomisation of schools

We invited all 47 non-Catholic state schools within 24 km of the main cities in Tayside and Lothian regions (excluding pilot schools) to participate. Incentives offered were the full cost of the SHARE teacher training, including supply cover, or in the control arm, the equivalent funds (£2000-£2500) for personal and social education, except sex education. We recruited 25 schools and allocated them by balanced randomisation21 to deliver the intervention programme or to continue with their existing sex education. The main reason the 22 remaining schools gave for not participating was the practical difficulty envisaged in implementing the programme (for example, timetabling), but a few referred to the explicit nature of the programme and research.

Surveys

We developed a self completion questionnaire and validated it in five pilot schools. We recruited two successive cohorts of third year secondary school pupils (aged 13-14 years) in 1996 and 1997 and followed them up at the start of their fifth year (at 15-16 years), about six months after completion of the programme. Three pupils were excluded by teachers because of learning difficulties; others were excluded by only their own or their parents' choice. Parents were informed by letter of the research and the intervention programme and were given the opportunity to withdraw their children. Researchers explained the study to individual classes and answered questions. Pupils had the option to withdraw or to omit questions they did not wish to answer. One education authority with nine study schools objected to some questions; this prevented us from asking about experience of pregnancy.

The questionnaires were administered by trained researchers in exam conditions with no teachers present. Pupils placed completed questionnaires, identified by study number only, in sealed envelopes. Absentees were followed up first within the school and then by post, while those who had left school at follow up were contacted by post.

Process evaluation

We carried out a process evaluation to understand enabling or confounding influences on either the delivery of the programme or its impact on pupils. We investigated the general school context, the extent and quality of delivery of the intervention and control programmes, and pupils' responses to the programmes. We collected data through interviews, questionnaires, group discussions, and classroom observation.

Statistical methods

We calculated the sample size to have 80% power to detect a 33% decrease in the cumulative abortion rate by the age of 20 (at long term follow up) and a 28% decrease in the proportion not using a condom at first intercourse separately for each sex (at six months after the intervention). For the latter outcome we assumed that 27% of the control sample would first have sexual intercourse between the ages of 14 and 1622 (and unpublished data from the national survey of sexual attitudes and lifestyles) and on 60% of these occasions no condom would be used,23 giving an overall rate of 16% for not using a condom at first intercourse. The sample size calculation assumed a design effect of 1.5.

Flow chart of participants; * indicates those for whom demographic data, including social class, were collected at follow up

We completed the analysis protocol and data checking blind to the arm of the trial. In all but one case we used a restricted randomisation test for differences between arms of the trial.21 We assigned the schools to arms of the trial by selecting an allocation that provided a good balance on all measures at school level. The randomisation test uses the set of the 20 000 possible allocations that might have been selected that give the best balance between arms of the trial. For each allocation we calculated the difference (d) in outcome between (potential) arms of the trial. The P value is the percentage of allocations giving a more extreme d than the actual difference (D) for the allocation used in the trial. A test based confidence interval for the effect of treatment (Δ, estimated by D) is found by testing the hypothesis Δ=K for different values of K and finding the values for which the test would give a P value of exactly 0.05 on a two sided test. We used this method instead of the more usual random effects regression models because it is a robust procedure that allows confidence intervals to be calculated directly for the quantities of interest.24

For the outcome of unwanted pregnancy (data unavailable in one local authority) we based tests and confidence intervals on the random effects logistic regression. Behavioural measures denoted by “after first year of programme” are those reported to have occurred since the Easter after baseline, by which time intervention pupils would have received the first half of the programme.

Results

Participant flow and follow up

The eligible population comprised pupils on school registers at the start of the programme. The figure shows the number and proportions of pupils who provided information at baseline (n=7616) and at follow up (n=5854). One school considered the baseline survey to be too explicit for pupils aged 13-14 years but took part in all other aspects of the study. The non-responders at baseline in the other schools (6%) were mainly persistent absentees, with only 32 pupils and seven parents refusing to take part.

Table 1.

Comparison of baseline characteristics according to group and whether follow up data were obtained. Figures are numbers (percentage) of participants

The response rate to the questionnaire after the intervention was lower because some pupils had left school. From official figures we had predicted that about 20% of the sample would have left, but a new work experience scheme increased this to 27%. The response rate was lower for school leavers (41% control, 38% intervention) than for those still at school (82% control, 80% intervention). Non-response among those still at school was primarily among persistent absentees, but a small proportion refused to participate (2%). The response rates were similar in each arm of the trial.

Table 1 compares baseline data between arms of the trial for all pupils in the schools at baseline, and for those with follow up data. In both groups those with follow up data were more likely to come from higher social classes. A slight imbalance in the sex ratio was increased by a further small imbalance in follow up. Other baseline characteristics, including sex, age, experience in sex education, main subject, or seniority of teachers, were similar in both groups. According to the 1991 census data the baseline sample was representative of all 14 year olds in Scotland in terms of social class and family structure, though of course Catholic young people were under-represented.

Delivery of intervention

Initially 80 teachers were trained to deliver the intervention programme. Nearly all welcomed the training and preferred the intervention pack to the courses previously provided. In 10 of the 13 intervention schools almost all pupils received over 15 sessions, including those on sexual negotiation and use of condoms. In three schools timetabling and the low priority attached to sex education meant that most of the pupils did not receive this minimum package.25 In six schools timetabling constraints and teacher mobility led to non-trained teachers delivering the programme to a small minority of classes. Skills based sessions remained challenging even after training, and some teachers modified or cut them.

Sexual behaviour

Overall 41% of young women (1278/3090) and 31% of young men (890/2692) reported having had sexual intercourse by the two year follow up. There were no differences between the groups in any of the main behavioural measures defined in the protocol (table 2). Similar proportions of young men and women in both groups became sexually active after half the programme had been delivered (after one year), and similar proportions used condoms at first intercourse. For none of the other outcomes shown in table 2 (unprotected sex, condom use, oral contraceptive use, and unwanted pregnancy) was there any evidence of a difference between the groups.

The pupils who had left school reported more, and more risky, sexual activity than those still at school. But there was no evidence of a differential effect of the intervention for those who left school compared with those still at school (table 3).

Table 3.

Young people who did not use condom at first intercourse (if this was after first year of programme) according to whether they were still at school. Figures are numbers (percentage) of participants

The random effects analyses found that the most important baseline factors influencing sexual experience at age 16 (as at 1426) were family composition, spending money, and parental monitoring.

Quality of sexual relationships

In relation to their first sexual intercourse and first intercourse with their most recent partner we categorised respondents who reported that “I wish I'd waited longer before having sex” or “It shouldn't have happened at all” as regretting that event. For these two events respondents were asked if either they or their partner had exerted pressure on the other person to have sex. We used a three point scale but have combined all reports of pressure here. Both sexes reported more pressure from young men.

Table 4 shows that there were no differences between groups in regret about or pressure at first intercourse for those experiencing this after the first year of the programme. For those with more than one partner there was evidence that those in the intervention group (especially young men) reported less regret at the timing of their first intercourse with their most recent partner, and fewer young men in the intervention arm reported pressure at this event. This effect was more pronounced for those who were sexually experienced before the programme. The levels of regret and pressure in both groups were lower than those reported at baseline.3 Overall there were high levels of reported enjoyment of most recent sexual intercourse, with no difference between arms of the trial.

Table 4.

Results for quality of sexual relationships by arm of trial. Figures are numbers* (percentage) of participants

Pupils' knowledge and evaluation of sex education

We calculated a mean score from eight questions on practical knowledge about sexual health. Pupils in the intervention arm were more knowledgeable than those in the control arm (table 5), with young men being less informed than young women in each arm. We also calculated a mean score from questions about how well sex education about five practical issues had been covered in school. Pupils in the intervention arm had higher scores (table 5).

Table 5.

Pupils' knowledge about sexual health and evaluation of sex education by arm of trial. Figures are mean scores (number of participants*)

Discussion

In comparison with conventional sex education, a programme specially developed to incorporate current theories on behavioural change had a limited beneficial effect on the quality of relationships but no effect on use of condoms for the third of pupils who have had sexual intercourse by the age of 16 years. These results could be interpreted as evidence of the failure of the programme, the delivery, or the evaluation.

Programme

There are several reasons why this intervention programme might not affect sexual behaviour compared with conventional programmes. Firstly, more of the young people in our study used condoms than we had expected from data from the early 1990s. 2223 This corresponds with other recent findings27 and makes the further reduction of unsafe sex by a new programme much more challenging.

Secondly, the impact of a 20 period school sex education programme might be unimportant compared with long term and pervasive influences12 from, for instance, family, local culture, and the mass media. Skills based exercises in 40-80 minute lessons might be too short to develop sexual interaction skills and too distant to be remembered when needed.

A third possibility is that skills based lessons might require higher motivation to be successful, implying that participants should opt into an intervention. Psychological models of the antecedents of action emphasise motivation, yet in UK secondary schools personal and social education is perceived by pupils to require little attention or effort because there are no exams. If active volunteering is critical to the success of behavioural interventions, however, it would be difficult to recruit young men without innovative approaches.28

Delivery

Possibly the intervention programme may be effective but was not delivered as intended. However, when we analysed our data taking into account the extent and quality of delivery of sex education (summarised above) we got the same results as the intention to treat analysis, suggesting that the lack of effectiveness cannot be attributed to differential quality of delivery.

The intervention might not have been delivered as well as an established programme that had been developed over years to suit teachers' needs. However, the intervention programme was not perceived to have been imposed against teachers' will25: most had been consulted about participating in the trial and the training gave them a sense of ownership of the programme.

Evaluation

At follow up only about one third of the respondents reported having had sexual intercourse and they are likely to be those who are least responsive to interventions delivered by teachers. The programme may have influenced the behaviour of the remaining two thirds of the sample, but this will be detected only in the planned future follow ups. Furthermore, our analysis did not distinguish between those who had received only the first year of the programme and those who had received the full two years before having sex.

Although the groups were well balanced, the design of the study could have been inadequate to detect real effects. Comparison with sex education in control schools might have obscured any effect if some control programmes also influenced behaviour. Furthermore, use of self assessments of sexual relationships as an outcome is problematic because the intervention may have changed perceptions or reporting, or both. However, the good internal consistency of our follow up data does not support this interpretation.

Finally, the intervention programme might have been effective with certain, as yet unidentified, subgroups, but the effects are obscured within the whole sample.

Conclusion

Our analysis does not suggest that the lack of impact on behaviour can be attributed to quality of delivery. The results imply that the potential for teacher delivered, whole class sex education to influence sexual behaviour in adolescents might have already been reached by conventional provision. If behavioural change among this age group is a central objective of school sex education then it should be further refined and other means of delivery should be rigorously evaluated. The intervention programme was rated more positively by pupils than comparison programmes, led to greater practical knowledge about sexual health, and did not encourage earlier sexual activity. We are following up these young people to the age of 20 to assess any effect on the cumulative rate of abortion, an outcome measure uninfluenced by reporting bias or attrition.

Acknowledgments

We thank the young people and teachers involved for their cooperation and support and our administrators and fieldworkers for their conscientious work. We also thank the advisory committee and colleagues in the MRC Social and Public Health Sciences Unit and Applied Statistics Group, Napier University, for their advice and encouragement; Sally Macintyre for helping to design the study and supporting it throughout; Katrina Turner for collecting data on sexual health services; and Geoff Der and Izzy Butcher for their careful reading of the final draft. The research was supported by the UK Medical Research Council, and the development of the SHARE programme funded by the Health Education Board for Scotland. A longer version of this report can be found at www.msoc-mrc.gla.ac.uk/Reports/Pages/share_MAIN.html

Contributors: DW, SS, CA, GMR, and GH designed the study, and MH and KB contributed to subsequent modifications. DW, CA, SS, MH, GMR, and GH designed the questionnaire, and KB helped adapt it for the follow up. MH, DW, and KB collected the data with assistance from GMR in fieldwork and GH in negotiating access. The outcome data were analysed and interpreted by GMR, MH, DW, and CA and the process data by DW, KB, MH, and GMR. DW and GMR drafted the paper, which all authors commented on. DW, GMR, and MH are the guarantors.

Footnotes

Funding UK Medical Research Council. The Health Education Board for Scotland funded the development of the SHARE programme.

Competing interests DW, CA, and SS had a professional interest in the efficacy of the SHARE programme, having led in its design and published on its theoretical basis.