Problems with the PACE Study

Note: The following essay was written in 2012 based only on a reading of the journal paper and its supporting materials. For a more recent critique of the PACE study bringing up even more serious additional problems that have since been revealed as a result of a freedom-of-information request, please see the Virology Blog article series “Trial By Error,” written by investigative journalist David Tuller.

By Lisa Petrison, Ph.D.

In 2011, a paper was published in The Lancet and received a great deal of publicity, causing many people to come away with the mistaken idea that exercise can be substantially helpful or even curative for those with myalgic encephalomyelitis (ME).

Despite the many serious flaws in the paper, it has repeatedly been suggested as being of high-quality and therefore as relevant to the the understanding of both ME and the broader condition of “chronic fatigue syndrome.”

This study was conducted by a team of psychiatrists in the UK between 2005 and 2010. It cost a total of 5 million pounds ($8 million), and was funded by the UK Medical Research Council, the Department of Health for England, the Scottish Chief Scientist Office, and the Department for Work and Pensions.

Participants consisted of 641 individuals meeting the Oxford criteria for chronic fatigue syndrome (with fatigue as the only symptom, accompanied by significant disability in the absence of an exclusionary medical or psychiatric diagnosis). All subjects were age 18 or over, and they had to be well enough to be able to attend 14 or more sessions of therapy over a 52 week period.

All patients received a lecture and a pamphlet from a medical practitioner specializing in the disease. They then were divided into four groups.

Patients in the “cognitive-behavioral therapy” (CBT) condition discussed with therapists how they could arrange their lives so that they could participate in a modest amount of activity and exercise each day. A goal was to have the amount of exercise very gradually increase over time. Topics discussed in the sessions included the importance of planning out each day’s activities in advance and sticking to that schedule as well as ways to reduce general stressors in life.

Patients in the “graded exercise therapy” (GET) condition also were encouraged to engage in modest amounts of exercise each day, and were taught how to use a heart rate monitor to make sure that they did not exceed their limitations. Again, a goal was to have the amount of exercise gradually increase over time. Patients were encouraged to engage in physical routines thought to be conducive to physical conditioning, such as doing extensive stretching before exercising.

Patients in the “adaptive pacing therapy” (APT) condition worked with therapists to monitor their condition, to make sure that they were not doing more than their bodies would allow. Their goal was to conserve energy to the extent that they were able.

Patients in the control condition received no treatment other than the lecture and pamphlet from the medical practitioner.

After 52 weeks, patients were evaluated on a number of dimensions.

The “Chalder Fatigue Scale” had patients rate their own current level of fatigue. Patients in the CBT and GET groups rated their fatigue as significantly (about 15%) lower on average than those in the APT and control groups.

The “Physical Functioning” scale had patients rate their ability to engage in a variety of activities (such as walking or climbing stairs). Patients in the CBT and GET groups rated their ability as significantly (about 10%) higher on average than those in the APT and control groups.

A “Six Minute Walking Test” gauged how far patients could walk in six minutes. The patients in the GET group were able to walk significantly (a little less than 10%) further on average than patients in the other three groups.

Based on this, GET and CBT were deemed by the authors to be successful treatments for CFS. These therapies are currently are incorporated into the NICE (National Institute for Health and Care Excellence) Guidelines as the only accepted treatments for the disease in the UK, and received a substantial amount of favorable media coverage in both the US and UK.

STUDY CRITICISMS

This study has been subject to a wide variety of criticisms, some of which are described here.

Effect Sizes:

One problem is that although the large sample size meant that the findings achieved statistical significance, the actual effect sizes were quite small.

At the beginning of the study, subjects scored an average of about 27 on the 33-point Chalder Fatigue Scale (described below). By the end of the 52 weeks, patients in all the groups improved. Those in the control condition scored 24, compared to 20 in the CBT condition and 21 in the GET condition. While this was a significant difference, the magnitude of difference between groups was small. On average, patients in all the groups stated that they continued to be more fatigued than before they got sick on the equivalent of every item on the scale.

At the beginning of the study, subjects scored about 38 on the 100-point Physical Functioning scale (also described below). By the end of the 52 weeks, patients in all three groups improved. Those in the control group scored an average of 51, compared to an average of 58 in both the CBT and GET groups. On average, patients in all groups remained limited in their ability to do basic activities such as carry groceries, walk up a flight of stairs or bathe themselves.

For the six-minute walking test, patients in the GET condition were able to walk an average of 379 meters (compared to 312 at the beginning of the study). This was a significantly bigger improvement than that of the control group, who could walk an average of 348 meters (compared to 326 at the beginning of the study). However, it was still substantially less than the distance (631 meters) that healthy people aged 65+ were able to walk in six minutes in another study (Troosters T, Gosselink R, Decramer M. Six minute walking distance in healthy elderly subjects. Eur Respir J 1999; 14: 270-274, PMID: 10515400).

Appropriateness of Dependent Variables:

CFS is a multi-systemic illness with a wide variety of symptoms other than fatigue. The authors failed to incorporate that fact into their study. The reported effects centered on three variables — fatigue, physical functioning and (for the GET condition) six-minute walking ability. For instance, cognitive functioning (a key issue in the disease) did not differ significantly across groups at the end of the 52 weeks.

The Chalder Fatigue Scale and Physical Functioning Scale are both subjective measurements. Conceivably, patients who had received 14 sessions of therapy may have been more inclined to report improvements than those who had received no therapy — even if that wasn’t reflective of reality — because they had the underlying desire to feel their time wasn’t wasted or to make it seem like the therapist had been helpful.

The one objective measurement showing an effect — the six-minute walking test — is not reflective of patients’ lives as a whole. By definition, CFS patients’ abilities are constrained less by their inability to participate in activity than by the negative effects that they experience afterwards.

For instance, one study on exercise by another group demonstrated (by having patients wear a device to track all their movements throughout the day) that patients who participated in structured exercise reduced their activity at other times to compensate. (Black CD, McCully KK. Time course of exercise induced alterations in daily activity in chronic fatigue syndrome. Dyn Med. 2005 Oct 28;4:10. PMID: 16255779) Conceivably, patients in the GET condition might have pushed themselves to walk fast for six minutes to demonstrate their success in the program, then spent the next day or week in bed.

Study Population:

Another group of criticisms is related to the patient population who participated in the study.

Patients had to be able to attend 14 or more sessions at a hospital clinic. Thus, those who were severely ill with M.E. (housebound or bedbound) were inherently excluded. Whether any of the study findings might be applicable to them is wholly unknown.

The use of the Oxford definition – which uses no symptoms other than fatigue in the criteria – presents another problem. Fatigue is just one of many symptoms in CFS, and it is characteristic of many other illnesses as well. In particular, it is a symptom of major depression, and the PACE study did not exclude people who were depressed.

The authors reported that some of the people in the study indeed had plain depression rather than CFS. Since people with depression are known to benefit from psychotherapy and exercise, the inclusion of even a few of these individuals could have skewed the results enough to make it look like the interventions were helpful for subjects in general.

The authors re-ran the data to determine if there were interaction effects between a) those patients who met the “international” criteria for CFS vs. those who met the Oxford criteria vs. those with depression and b) the experimental condition (GET, CBT, APT or control). The dependent variables were the Chalder Fatigue Scale and the Physical Function Scale. They found no such interactions.

However, not finding an interaction does not mean that there is no actual underlying interaction. If the authors wanted to show that people meeting the CDC criteria improved more in the GET or CBT conditions than in the control condition, they should have done an analysis of main effects just on those people. As stated in the paper, the authors did not have enough people who met the CDC criteria to prove that the small effect sizes in that group were not due to chance.

The CDC definition of CFS is itself quite broad. For instance, people who do not have post-exertional exhaustion (the recognized cardinal symptom of the illness) or other M.E. symptoms can qualify. Especially since the authors’ results were counter to more than 50 studies in the medical literature showing various negative physiological effects as a result of exercise in CFS patients, the use of a more restrictive definition would have been appropriate.

Study Implementation:

The write-up of the study made it seem that any improvements that occurred were due to increased exercise as a result of physical therapy or psychotherapy. The instructions given to therapists suggest that this may not have been the case, however.

For one thing, subjects in the groups receiving therapy were given counseling that may have helped their condition in ways unrelated to exercise. In the CBT condition, people were encouraged to find ways to reduce stress in their lives – conceivably allowing them to need to push themselves less hard and thus to experience physical improvements as a result of increased rest. In the GET condition, sleep hygiene skills (such as taking a hot bath before bedtime) were discussed. Since sleep is disordered in CFS, this may have helped their physical condition.

Therapists were instructed to remain sympathetic and supportive of the patients. In general, people who are having a difficult time often benefit psychologically from any sort of positive attention, perhaps especially from therapists. It could be that people who were getting GET or CBT felt emotionally better as a result of the interactions and that this affected their perceptions (but not the reality) of how they were doing physically.

Subjects in both the GET and CBT conditions were encouraged to start out with very gentle exercise (relying either on a heart rate monitor or on their physical responses) and to increase the amount very gradually. On the other hand, newly diagnosed CFS patients who are not receiving coaching tend to engage in a push/crash phenomenon — exercising vigorously and then experiencing extended post-exertional exhaustion. Conceivably, patients in the GET and CBT conditions actually might have been exercising less on average than patients in the control condition, with their increased energy and improved physical function resulting from having pushed themselves less hard throughout the year.

Another issue concerns the possibility that GET and CBT are not implemented by other therapists in the ways that they were in the study. Therapists in this study were instructed to be particularly supportive and received in-depth training in the particular methodologies used — for instance, with strong emphasis placed on the idea of extremely gradual increases in activity. Therapists who are less focused on clients’ individual situations and who push hard for faster improvements might generate very different results.

In addition, the 14 sessions of therapy that patients received may be unrealistic in the real world, considering restraints on healthcare spending. Fewer sessions might not provide the same results.

Reporting:

Especially when describing the study to the media, the authors and others promoting it frequently have made the results sound more impressive than they actually are. None of the groups of patients achieved a recovery to anything even close to normal levels of functioning, and the magnitude of differences between getting extensive treatment vs. no treatment were very small.

These researchers and their supporters repeatedly have made statements listing the percentage of subjects in the GET and CBT conditions who have improved, without stating the percentage in the control condition who improved.

For instance, for the Physical Function Scale, about 70% of patients in the GET and CBT conditions improved, compared to 58% in the control condition. For the Chalder Fatigue Scale, 76-80% of patients in the CBT and GET conditions improved, compared to 65% in the control condition. For participant ratings of overall global health, 41% in the GET and CBT conditions improved, compared to 25% in the control condition.

Statements such as “41% of patients receiving GET and CBT improved” thus are misleading, since the majority of those individuals would have improved without any treatment at all.

In addition, the researchers defined the “normal range” as a score of 18 or less on the Chalder Fatigue Score and 60 or higher on the Physical Function Subscale. They then stated that the percentage of individuals meeting this criteria was higher in the CBT group (30%) and the GET group (28%) than in the control group (15%).

These “normal range” scores include patients who demonstrate quite severe problems with cognitive function and fatigue and who are very limited in terms of being able to participate in such basic activities as walking a hundred yards, carrying a bag of groceries and bathing themselves. The idea that patients with such substantial health issues would be described by researchers as “recovered” is very misleading.

In fact, the researchers accepted into the study individuals with Physical Function Subscale scores of up to 65 – even though they noted scores from 60-65 as within the “normal range” in the results section. The fact that some of the study participants were already in the “normal range” prior to the study beginning is an oddity that does not seem to be explained very well in the paper and that needs to be explored further.

Contrast with Findings in Medical Literature:

More than 50 papers on exercise intolerance in CFS patients have been published by medical researchers. These papers demonstrate a wide variety of abnormal negative physiological changes in patients, usually lasting for days (sometimes longer) after the exercise occurs.

The authors of the PACE study state that they are operating under the theory that the problems that patients have with exercise is that they are deconditioned or have an irrational fear of exercise. This is not consistent with the peer-reviewed medical literature, which shows changes related to a wide variety of physiological measures (such as inflammation and oxidative stress) that have nothing to do with muscle conditioning or exercise phobia. It also is not consistent with the observed experience of the illness, in which many extremely fit individuals become permanently bedridden overnight.

*

Chalder Fatigue Scale

33 points possible

1. Do you have problems with tiredness?

2. Do you need to rest more?

3. Do you feel sleepy or drowsy?

4. Do you have problems starting things?

5. Do you lack energy?

6. Do you have less strength in your muscles?

7. Do you feel weak?

8. Do you have difficulty concentrating?

9. Do you make slips of the tongue when speaking?

10. Do you find it more difficult to find the correct word?

11. How is your memory?

Items 1-10: Less than usual, 0; No more than usual, 1; More than usual, 2; Much more than usual, 3.

Item 11: Better than usual, 0; No worse than usual, 1; Worse than usual, 2; Much worse than usual, 3.

*

SF-36 Physical Function Sub-scale

100 points possible

The following questions are about activities you might do during a typical day. Does your health limit you in these activities? If so, how much?

Information on this website is presented by Paradigm Change.Links on this page are in orange (no underlining).

FREE BOOK

Sign up for email updates about M.E. from Paradigm Change and receive a free PDF copy of the book Back from the Edge, summarizing the illness experiences and eventual recovery of a survivor of the Lake Tahoe illness outbreak.