July 2, 2016

The Limits of Randomized Controlled Trials: Who Agrees to be Recruited?

Randomised controlled trials (RCTs) of interventions are a gold standard source of evidence in medicine, as people like Ben Goldacre have argued repeatedly. As people are allocated to receiving the intervention at random, this should eliminate many of the biases that come from people self-selecting for interventions they would like.

But RCTs are vulnerable to another sort of bias – that of deciding whether to take part in the trial at all. The study I am discussing here, by Rogers and collaborators, takes a very thorough look at why older people decided to take part in a trial that tested an intervention which is designed to get older people to walk more.

Study participants were recruited from three general practices in relatively affluent parts of England, Oxfordshire and Berkshire. Potential participants were identified from the general practitioners’ records. General practitioners then filtered out those whom they deemed unsuitable for the intervention, and then the invitations to the remaining people were sent out through the practices.

This means that while the researchers did not see the names and addresses of the non-participants, they still had access to some basic demographic information which allowed them to compare who did and did not show interest in the trial. This information included age, gender, whether they were invited on their own or as a couple, and the socioeconomic deprivation index of the area they lived in – but not the area itself.

988 people were contacted initially. Everyone had three options: to take part in the trial, to complete a survey with information about why they chose not to take part, and not to respond at all. 298 (30.2% or one in three) people agreed to participate, and 690 were not interested. Of those 690, 183 (26.5% or one in four) returned the survey, and 77 of the 183 (42%) agreed to be contacted further about their reasons for not participating. Rogers then interviewed 15 of these people herself; the interviews stopped after 15 because no new insights emerged.

Instead of discussing the complex pattern of results that emerged from the study, I would like to highlight two findings that I consider to be the most interesting.

Finding 1: The people who don’t respond at all are very different from the people who will return your non-participation survey.

Table 1 of the paper shows the overall demographic differences between participants and non-participants, while Table 2 looks at the demographic differences between participants and non-participants that returned the survey. The pattern that emerges from Table 1 is that people are less likely to take part if they are male and if they live in a deprived area. Age and whether they were invited as a couple or not did no matter. Table 2, on the other hand, shows no difference at all on any of these four metrics.

Finding 2: Taking part in a trial is hard work for participants.

While the most common reason people cited for not taking part was that they were already physically active (67.3% of the 183 who returned the non-participation survey), the second most important reason was that they just didn’t have the time (44%).

The qualitative interviews provide an insight into the demands that taking part in the trial would place on participants. They would have to

find time in lives that were already full of family commitments and activities

stay with the trial for three months

walk regularly in the dark winter

look after an accelerometer device to measure physical activity

walk regardless of other health issues such as chronic pain, depression, or knee problems

change their existing habits and routines

Conclusion

The people who ended up taking part in the trial were not only more wealthy and more likely to be female, but also more likely to be able to organize their lives around increased physical activity.

What does that mean for clinical practice? While it appears to be very easy to tell people to just be more active, the recruitment patterns for this trial indicate that those who might need help the most don’t necessarily contribute to the evidence base that doctors are told to rely on.