My grandfather had a nearly religious conviction of the value of physical activity. No weekend was complete without a strenuous bout of outdoor exercise, regardless of the weather. New England gets cold and damp, and his five children would prepare themselves for winter excursions with some reluctance. ‘Healthy body, healthy mind!’ my grandfather would say, exhorting them to bear up and show some of the fortitude of their Puritan ancestors. It was perhaps fortunate that their thick scarves muffled any replies.

‘Healthy body, healthy mind’ is from the Latin mens sana in corpore sano, ‘a healthy mind in a healthy body’. It was with some surprise that I learned that the original source doesn’t propose a causal link; the writer is merely listing the things that one should value in life. My grandfather’s interpretation, however, is a widely held belief; in fact, it is tacitly supported by none other than the World Health Organization. In the late 1980s, the WHO introduced the concept of the ‘Health Promoting School’, an institution that supports the development of healthy lifestyles as well as education. The Health Promoting School framework has since been incorporated into many countries’ educational policies.

Earlier this year, the Cochrane Collaboration released a systematic review of ‘the effectiveness of the Health Promoting Schools (HPS) framework in improving the health and well-being of students and their academic achievement’ (Langford et al, 2014).

Health Promoting Schools support the development of healthy lifestyles, as well as education

Methods

An international panel of researchers conducted an extensive and systematic search to identify cluster-randomised controlled trials of HPS interventions aimed at students aged 4 to 18. ‘Cluster randomisation’ means that randomization takes place at the group level; in this case, schools or districts in which interventions were implemented were evaluated against comparable schools or districts in which no intervention took place. In order to qualify as ‘HPS’, an intervention had to include the following:

Introduction of health education into the curriculum

Changes to the school’s ethos or environment or both

Engagement with families or communities, or both

Sixty-seven trials, including a total of 1,443 schools and districts, were found eligible for the review. The researchers grouped the trials according to health topic and approach used. Methodological quality was assessed using GRADE (PDF), which is a widely used standardized system for determining the overall quality of a group of studies. Risk of bias was also considered.

A cluster randomised controlled trial is a type of RCT in which groups of subjects (as opposed to individual subjects) are randomised

Results

Most studies were set in North America (29), Europe (19), or Australasia (11). Interventions were slightly more likely to target younger children (<12). Length varied from eight weeks to two years; the authors noted that shorter interventions were more likely to be aimed at physical activity or nutrition, whereas interventions focusing on substance use, violence, and sexual or mental health tended to be longer.

The overall quality of the evidence was low to moderate, suggesting that additional studies could lead to large changes in the researchers’ conclusions. High attrition and a reliance on self-reported data created a risk of bias in many of the trials.

That said,the authors found that Health Promoting Schools had a:

Positive effect

Little effect

No effect

Insufficient evidence for conclusions

For the following intervention targets:

Body mass index (BMI)

Physical activity

Physical fitness

Fruit and vegetable intake

Tobacco use

Being bullied

Standardized body mass index (zBMI)

Fat intake

Alcohol use

Drug use

Mental health

Violence

Bullying others

Academic, attendance, or school-related outcomes

According to this review, the HPS approach has a positive impact on some health outcomes

Conclusions

The authors conclude:

Overall, we found some evidence to suggest the HPS approach can produce improvements in certain areas of health, but there are not enough data to draw conclusions about its effectiveness for others. We need more studies to find out if this approach can improve other aspects of health and how students perform at school.

Summing Up

The authors’ conclusions, conservative though they are, are already being used to argue for greater investments in Health Promoting Schools. In light of the gaps identified in the review, it is important that some of this funding is allocated to rigorous evaluation. In my view, three areas are particularly in need of further research:

Academic outcomes. According to the authors, ‘the HPS framework is based upon a recognition of the intrinsic link between health and education’, yet few studies examine this link. Implementing a health promotion curriculum in schools implies an unavoidable diversion of resources from traditional educational activities. Evidence suggests that health and academic performance are complementary goals rather than competing ones, but more research is needed. The presence of a correlation between better health and improved academic outcomes would strengthen the business case for Health Promoting Schools, particularly if randomised controlled trials suggested causation. If no correlation were detected, that too would be worth knowing.

Adverse effects. Do obesity prevention campaigns increase disordered eating? Could anti-tobacco initiatives lead more students to cigarettes? Small adverse effects can be significant at the population level, so it’s important to monitor them when large-scale implementations are being considered.

Long term outcomes. Very few trials in this review (10) included follow-up measures. As the authors write, ‘While interventions may be able to produce short-term changes in behaviours or health outcomes, unless these prove sustainable they are likely to be of little public health importance’ (p. 32).

It’s a common mantra in education, but more research really is needed before we can reliably assess the impact of Health Promoting Schools

The suitability of randomised controlled trial designs for evaluating health promotion initiatives and other complex interventions has been questioned (Green, 2000; Tones, 2000 (PDF); WHO European Working Group, 1998 (PDF)). Opponents argue that standardising an intervention in an environment with as many uncontrollable variables as a school is neither feasible nor appropriate. Hawe et al. (2004) (PDF) offers, in my opinion, a sound counter-argument: Randomised controlled trials are suitable, provided we are willing to moderate our definition of ‘standardised’. It’s unnecessary to distribute precisely the same posters and teach exactly the same training sessions in all schools, and one would be foolish to expect good results from doing so. Tailoring an intervention to its target population is an accepted factor in its success. Therefore, “in complex interventions, the function and process of the intervention should be standardised, not the components themselves”(Hawe et al., 2004, p. 1563). This approach avoids placing senseless restrictions on program developers while still allowing us to evaluate interventions in a consistent manner and make some broad statements about which outcomes appear most susceptible to change.

A lengthier discussion of the use of randomised controlled trials with complex interventions appeared in a recent report by Haynes et al. (2012) (PDF). The authors make a persuasive case for the inclusion of randomised controlled trials in policy development. Among their points:

RCTs reduce our dependence on professional opinion and expert intuition, which have been shown to be more fallible than one might expect

RCTs help us determine whether individual components of an intervention are essential or redundant, leading to long term cost savings

RCTs can reveal adverse effects that might otherwise have gone unnoticed, preventing us from unintentionally harming hundreds or thousands of people

Other types of evaluation are, of course, also necessary; the perceptions of students, educators, and administrators are essential to gaining insight into how an intervention worked, or why it didn’t work, which is infinitely useful for program development. It doesn’t undermine the value of qualitative evaluation methods, however, to state that the randomised controlled trial (even the bastardised form used in complex interventions) is deserving of a place.

Alison Jones is a researcher at the University of British Columbia's eHealth Strategy Office in Vancouver, Canada. In 2011, she received a Masters of Library and Information Studies from UBC. Her areas of particular interest include knowledge translation and patient safety.

I am familiar with that review and have been involved with HPS here in New Zealand for several years, so my comments are based on anecdotal experience rather than any RCT or other method of research.

I have long wondered ‘is it really HPS that makes the difference’? I say they because I am also familiar with things like the Hawthorne Effect.

Essentially HPS is a community development model; we target those most in need and have seen incredible behavioural change in terms of student behaviour, attendance at schools, increased physical activity, improved nutrition, better sun safety, etc. However, the question remains, was it HPS that made the change?

Certainly having a process which engages the whole school assists; it also makes it more sustainable than the ‘hit and run’ health education approaches that seem to be common within education. Possibly the most important aspect is that it allows communities to identify THEIR needs, rather than those dictated to them.

Doing an RCT would, IMHO, be unethical. I believe the process does work, but is completely dependent on the school community buy in, something that is not always forthcoming.