While the necessity of proper control conditions in evaluating the efficacy or effectiveness of psychotherapies, or indeed of any health intervention, is undeniable, what exactly constitutes an appropriate control for an active intervention is far from clear.

In a new meta-analysis published in Acta Psychiatrica Scandinavica, Furukawa and collaborators comparatively examined three types of control conditions (no treatment, waiting list and psychological placebo) in trials of cognitive behavioral therapies (CBT) for adult depression.

Methods

The authors used a meta-analytic methodology called network meta-analysis (NMA). While traditional meta-analysis only allows for the possibility of comparing two interventions at a time, network meta-analysis permits the evaluation of the relative efficacy of all the included interventions, by integrating data both from direct head-to-head comparisons, as well as indirect evidence from studies with common comparators. In this way, NMA borrows strength from the entire network of randomised trials.

The authors included randomised clinical trials in adults, comparing CBT with a control condition. Three control conditions were considered:

Psychological Placebo (PP): the condition was considered as inactive by the researchers conducting the study, but was presented as potentially active to the participants. Moreover, the number of sessions and their duration, as well as the qualifications of the therapists providing the intervention, had to be equivalent to the active treatment in the study

No Treatment (NT): participants received no active treatment during the study and they did not expect to receive it after the study was over

Wait-list (WL): participants received no active treatment during the study, but were told they could receive one such treatment after the study was over

The studies were included if the primary clinical problem treated was depression, evaluated either by a diagnostic interview of by self-report scales. The authors excluded studies focusing on chronic or treatment-resistant depression, and studies where participants had another concurrent primary diagnosis, whether of a mental or physical disorder.

Trials of CBT versus Control for depression were included in the network meta-analysis

The primary outcome of the meta-analysis consisted of the number of patients who responded to treatment, based on changes on the Hamilton Rating Scale for Depression (HAMD), the Beck Depression Inventory (BDI) or any other validated depression scale, at the end of the acute phase of treatment. Analyses were, in as much as possible, intent-to-treat, meaning that they took into account the total number of randomly assigned participants, irrespective of how the investigators in the original studies analyzed their data.

Two independent raters evaluated the included studies for methodological quality, using the Risk of Bias tool developed by Cochrane Collaboration and some additional criteria such as researcher allegiance, therapist allegiance, therapist qualification and treatment fidelity.

The authors first examined individual trials grouped in pairwise, head-to-head meta-analysis and subsequently constituted the evidence network for the NMA and looked at its robustness. They evaluated comparative response rates of CBT and the three control conditions, by reporting odds ratios (OR) and corresponding 95% credible intervals of the OR. Briefly put, the odds of an event represents the number of those who experience it divided by those who don’t and it is expressed by a number from zero (the event will never happen) and infinity (the event will definitely happen).

Results

The results of the NMA indicated a well-connected, homogeneous and consistent network of evidence connecting CBT and the three types of control groups (PP, NT and WL) in the acute phase of treatment of adult depression

Effect sizes for CBT were substantively different, depending on which control condition they were compared to:

Psychological placebo (PP): OR= 1.65 (0.76 to 3.13)

No Treatment (NT): OR= 2.36 (1.31 to 4.26)

Wait-list (WT): OR= 6.26 (3.90 to 10.1)

In other words, WL control may generate bigger effect sizes estimates for CBT than NT or PP control

Moreover, the indirect comparison between NT and WL revealed the former was significantly superior to the latter in eliciting response: OR= 2.9 (1.3-5.7)

However, the authors note that the statistically significant differences between conditions were lost when they applied exploratory sensitivity analyses to correct for small study effects (i.e. small studies having systematically different effects to the large studies)

This analysis suggests that waiting list controls may generate bigger effect size estimates for CBT than No Treatment or Psychological Placebo controls

Conclusions

The authors concluded that:

The currently available best evidence, analysed by use of NMA, suggested that different control conditions lead to substantively different treatment effect estimates and that WL control may generate bigger effect size estimates for CBT than NT or PP.

They also added that:

In other words, WL could be regarded a nocebo condition if it indeed is inferior to NT, that is, doing nothing.

The authors speculate that participants allocated to WL might be more motivated to remain depressed so they can receive their originally desired treatment (CBT) after the completion of the study period, while those allocated to NT may actively seek alternative treatments.

Limitations

The methodological quality of the included studies was sub-optimal, as already remarked by other meta-analyses on CBT and discussed in previous Mental Elf blogs. In fact, only a quarter of the included studies were rated as having a low risk of bias

Importantly, small study effects (i.e. small studies having systematically different effects to the large studies) were notable, particularly in the comparison between CBT and WL

Ioana is Associate Professor at Babes-Bolyai University, Cluj-Napoca and a Research Fellow at the University of Pisa, Italy. She holds a masters degree in Clinical Psychology, a Ph.D. in Psychology, and is a board certified cognitive-behavioral therapist. Her main research interests include critically appraising the efficiency and mechanisms of action of psychotherapy interventions.

That is a very dangerous title – it implies that wait-list in some way enhances the effect of CBT for patients. It doesn’t – and from the results it seems it could be actively harmful to them. This is not a cross-over study, so you can’t tell what happens to patients in the control group once they receive therapy. As far as the analysis goes, all they receive is hope of treatment. And the treatment group won’t have been on the wait list.

Ooh yes, I’d forgotten about that! Thanks for the reminder :-) Good old technology eh? All of your comments will also be stored away in the woodland filing cabinet. The Regimental Elf likes to keep everything organized and properly indexed.

[…] Mental Elf readers are well aware of criticisms directed at psychotherapy’s prodigal son: cognitive behaviour therapy (CBT). Unarguably the most studied and most recommended form of psychotherapy, CBT has nonetheless been been shown to have some problems (see my previous blogs on CBT for adult depression and wait-list control exaggerating the efficacy of CBT). […]