This, from 2009, looks like an important study but I've never noticed it before - anyone know anything about it? I saw one brief comment from Dolphin that almost all the contributing studies are by CBT proponents, and a 2010 'General News' thread received no replies.

It's a vast meta-analysis of symptom data on nearly 38,000 fatigued patients including 1,950 with CFS and was recently cited by van der Meer and Lloyd in their editorial critiquing the ICC to argue that:

chronic fatigue states regardless of exactly how they are defined [ie inc CFS],
share a common and relatively stereotyped set
of symptom domains which can be readily identified
in the community, at all levels of health care, and
across cultures [24].We suggest that there is little to
be gained by reshaping the [Fukuda] diagnostic criteria.

Click to expand...

The study was supported by the CDC as part of the International Chronic Fatigue Syndrome Study Group that came up with the 2003 ('Ambiguities') revision of Fukuda.

From a quick look it appears the majority of CFS cases were Fukuda-defined wiht most of the rest Oxford, and no Empiric.

Google scholar lists only 10 citations of the paper, so maybe it isn't such a big deal.

One problem I recall from the study is that they didn't necessarily ask patients about a full list of symptoms (say 100+ symptoms). And IIRC different lists were used. This restricts the number of factors possible.

Also I'm not convinced that they proved mood disturbance symptoms were a core symptom. All factor analysis does is group together scores that go together. It doesn't prove they are part of the syndrome. So if one had a few questions about following sport, one might find the various questions factor together - people who follow one sport might be more inclined to also follow another sport on average (and people who don't follow one sport might be less inclined to follow another). So one could group those questions together as an "interest in sport" factor - but it doesn't prove they're a part of the syndrome - what they are is a factor group in the questionnaires.

Even if they went one step further and showed the scores on mood disturbance symptoms scored more highly than some other group, that still wouldn't show that it was a core symptom of the condition.

Hi Dolphin, Thanks for the link to Ellen Goudsmit's comments, and your own helpful ones too. I'm still trying to get my mind round what they've done and how it's supposed to support their conclusions; think I'm going to have to work through the paper. But I agree that the general clustering of symptoms (not drawn from a common list) doesn't seem any way to demonstrate core symptoms, especially, as Ellen points out, many of the symptoms are common to many illnesses. And as you say, football fans and rugby fans are not necessarily the same people!

I do think this study is important. As the authors highlight:

In 2003 the International CFS Study Group
recommended a new study of patients with chronic,
unexplained fatigue from which a definition of CFS
could then be derived empirically [8, 'ambiguities' paper]. They also
recommended that the study be international in
nature, encompassing different regions and cultures.

Click to expand...

So they are implying this is it, the 'empirical definition' study, though I'm not quite sure this is quite what the International CFS group meant, and tellingly that 2003 paper added:

To test the validity and reliability of
the CFS case definition as revised, prospective studies of
subjects at high risk for CFS should be undertaken.

The International CFS Study Group recommended
consideration of a revision of the 1994 CFS case
definition based on empirical data gathered from a
large international study [8]. The present findings
indicate that the core dimensions specified in the 1994
definition have construct validity and do not need to
be revised. The International CFS Study Group also
recommended that for research purposes, the diagnosis
of CFS should be made using validated instruments
that allow standardized assessments of the
major symptom domains of the illness. The present
study supports that recommendation and suggests an
empirical diagnostic algorithm similar to that used by
the Centers for Disease Control and Prevention [51].

I don't the Hickie et al. study justifies the Reeves et al. (2005) study at all (nor does it not justify it).

The Reeves et al. (2005) picked thresholds out of the air (bottom 25th percentiles, etc.) for certain measures (not picked randomly) and saw these were better than random thresholds. Not a particular good "empirical" approach - a proper empirical approach would have the data taking the lead and suggesting the thresholds.

--
Obviously wasn't clear enough in my analogy: I was trying to say that there would be some clustering on interest in sports - I was really thinking of team sports.

I'll try another analogy: I would think a questionnaire might find a factor that could be "interest in the performing arts" if there were questions on interest in musicals, ballet, dance, opera, etc. Some people would cluster as having an interest or liking them and some would cluster as not having a big interest in them/not particularly liking them. So if those questions had been in the questionnaire, they could be a factor of the questionnaires. But they would tell zero about CFS.

I don't the Hickie et al. study justifies the Reeves et al. (2005) study at all (nor does it not justify it).

The Reeves et al. (2005) picked thresholds out of the air (bottom 25th percentiles, etc.) for certain measures (not picked randomly) and saw these were better than random thresholds. Not a particular good "empirical" approach - a proper empirical approach would have the data taking the lead and suggesting the thresholds.

I'll try another analogy: I would think a questionnaire might find a factor that could be "interest in the performing arts" if there were questions on interest in musicals, ballet, dance, opera, etc. Some people would cluster as having an interest or liking them and some would cluster as not liking them. So if those questions had been in the questionnaire, they could be a factor of the questionnaires. But they would tell zero about CFS.

Click to expand...

Yes, I'd spotted that apparently-abitrary endorsement of the Reeves study's arbitrary criteria too. At least this study is empirical in the sense of being based on the data, even if their logic is skewed.

Even if they went one step further and showed the scores on mood disturbance symptoms scored more highly than some other group, that still wouldn't show that it was a core symptom of the condition.

Click to expand...

The effect size for 'all subjects with fatigue (n=37 724)' was 0.03, and 0.24 for CFS patients. The effect size is weak, but more importantly the directionality is not demonstrated. One of the differences for categorisation is length of fatigue (ie longer than 3 months, less than six) which means it can be argued that mood disturbance occurs as a result of being fatigued for over 6 months.

The dominant studies were used from:http://www.ncbi.nlm.nih.gov/pubmed/12027042
"The genetic aetiology of somatic distress." n=8392 (Concluded that: These results support previous fndings that somatic symptoms are relatively aetiologically distinct both genetically and environmentally from symptoms of anxiety and depression.)

The dominant studies were used from:http://www.ncbi.nlm.nih.gov/pubmed/12027042
"The genetic aetiology of somatic distress." n=8392 (Concluded that: These results support previous fndings that somatic symptoms are relatively aetiologically distinct both genetically and environmentally from symptoms of anxiety and depression.)

Criteria-based approaches to the diagnosis of CF and related syndromes do not select a homogeneous patient group. While substratification of patients is essential for further aetiological and treatment research, the basis for allocating such subcategories remains controversial.

An important finding?
This study is important, both because it was done as part of the International CFS Group that came up with the 2003 modified Fukuda definition, and because of its startling conclusion that everything from prolonged fatigue (more than one month) through chronic fatigue (6 months+) and CFS have the same types of symptoms - and probably the same underlying pathophysiology:

We suggest that this international study supports the proposition that chronic fatigue states [prolonged fatigue, chronic fatigue and CFS] share a common and stereotyped set of symptom domains, and that these can be readily identified in the community and at all levels of health care. Consequently, it is likely that they share common risk factors, are underpinned by a common pathophysiology, and may respond to common treatment strategies.

Click to expand...

The authors don't report the evidence needed to back up their argument
The study used Factor Analysis, a statistical technique that tries to find underlying but hidden commonalities between variables, in this case symptoms. According to this study, the symptoms clustered into 5 factors common to all fatigue states:

So you might imagine that they ran separate factor analyses on each of Prolonged Fatigue patients, Chronic Fatigue pateints and CFS patients - and found the same 5 factors in each case. But they didn't do this. Instead they analysed all 3 types of fatigue together - 88% of the total sample were prolonged fatigue cases and only 5% were CFS cases. Since almost all the cases were prolonged fatigue the only conclusions that can safely be drawn are those about prolonged fatigue. The study tells us nothing about the factors underlying CFS so the conclusion that CFS share the same 5 clusters of symptoms as other fatigue states is not supported by the evidence presented in this paper.

But maybe they did run the relevant analysis...
Interestingly, what appears to be an earlier version of this paper (by the same authors on the same dataset with much identical text) did include a separate analysis of CFS patients - and these patients had a different factor solution to that for the sample as a whole.

Finally, the 1,387 subjects specifically diagnosed as CFS in secondary/tertiary referral clinics were examined... In this sub-sample, the least satisfactory model was generated. It yielded four symptom factors that were less coherent than the factors in the earlier models, utilized 25 individual items and explained [only] 41% of the variance.

The first factor labelled as general physical health, was unique to this group of subjects. It included a large number of non-specific symptom items that had no outstanding clinical features and manifested diffuse item loadings. The other three factors were labelled similarly to those found in the broader community and healthcare settings. However, the mood disturbance factor included items such as headaches and chills or shivers as well as more typical items such nervous or tense / wound up? And look forward with enjoyment to things? - reflecting heterogeneity in the construct. Interestingly, although fatigue was notionally the central element of the clinical diagnoses, the most pertinent symptom items for fatigue (e.g. feeling tired after rest or relaxation or prolonged tiredness after activity) did not contribute to the factor solution. This suggests that although this group of subjects was drawn from the most homogeneous section of the healthcare system (specialized referral centres), they actually have varied illness experiences, which were not dominated by fatigue and were less uniform than in population and primary care settings.

Thanks to Dolphin for digging out this unpublished information.

Click to expand...

So it appears that CFS does not have the same factor structure as other forms of fatigue, though this analysis has never been formally published.

The lack of a coherent factor structure ties in with an earlier paper by Hickie (which also contributed about half the patients used in the CFS-only analysis above) that concluded:

Criteria-based approaches to the diagnosis of [CFS] do not select a homogeneous patient group.

Click to expand...

Also, the authors didn't evaluate more specific criteria
The other striking omisson in this study is that they didn't attempt to evaluate more specific criteria, or even (it appears) include Post Exertional Fatigue/Malaise as a symptom. Given that the main criticism of Fukuda is that it's too broad, and that they had accounted for less than half the variance with their own factor structure (which suggests there are better explanations they hadn't discovered), it's bizarre the authors go on to conclude that:

there is little to be gained by further reorganization of the diagnostic criteria

Click to expand...

There are numerous other issues with the paper but I'm not sure it's worth going into the detail.

Sorry if I am being naive, but how are they allowed to get away with what appears to be blatantly biased data analysis, particular if as you say the fuller analysis is "known". Is this a feature of the science world's "publication" concept, i.e. that until they publish it is not officially recognised as existing Or is it a case of complex statistics having multiple "valid" interpretations?

As an aside the question of scientists as sub-standard statisticians has come up a couple of times recently. As not all scientists can be expected to be expert statisticians, why is it not standard practice to pass data for statistical analysis by statisticians, rather by the researchers themselves? Is there a valid reason, or is it just historical?

Sorry if I am being naive, but how are they allowed to get away with what appears to be blatantly biased data analysis, particular if as you say the fuller analysis is "known". Is this a feature of the science world's "publication" concept, i.e. that until they publish it is not officially recognised as existing Or is it a case of complex statistics having multiple "valid" interpretations?

As an aside the question of scientists as sub-standard statisticians has come up a couple of times recently. As not all scientists can be expected to be expert statisticians, why is it not standard practice to pass data for statistical analysis by statisticians, rather by the researchers themselves? Is there a valid reason, or is it just historical?

Click to expand...

Can't explain the first part. The published paper makes no sense without the CFS-only analysis, so I'm surprised it was published (the draft may have been declined by the BMJ). One of the authors, Rosane Nisenbaum, is a Biostatistician. The PACE trial has at least 2 biostatisticians on board.

Similar to Fukuda, though I'm not sure they preserved the psychiatric exclusions. Can't tell from here, but you can see the similarities to Fukuda and the US CDC's recommendations based on conflating CFS with CF.

Similar to Fukuda, though I'm not sure they preserved the psychiatric exclusions. Can't tell from here, but you can see the similarities to Fukuda and the US CDC's recommendations based on conflating CFS with CF.

Click to expand...

Thanks for that, and you feedback! Dolphin sent me a copy of the Ozzie criteria (PM me if you want them) and it does appear to drop the psychiatric exclusions, which sems like a major flaw.

I don't necessarily see Fukuda as conflating CF with CFS. The whole strategy of Fukuda is to specifically exclude a whole number of known causes of CF, then to apply a severity threshold (of sorts), and then requires a set of required symptoms (even if it's not a satisfactory set of symptoms). If I'd had the energy, I was going to point out in my original post that this 2009 Hickie paper differs from Fukuda precisely by conflating CFS with CF, and even with prolonged (1 month+) fatigue. Which is quite a shocking thing to do when their own evidence actually points to Fukuda being too broad.

As an aside the question of scientists as sub-standard statisticians has come up a couple of times recently. As not all scientists can be expected to be expert statisticians, why is it not standard practice to pass data for statistical analysis by statisticians, rather by the researchers themselves? Is there a valid reason, or is it just historical?

Click to expand...

In the case of the technique used here, Factor Analysis, there's a lot of user input/judgement required, which is probably why the authors said

A five-factor model of the key symptom domains was preferred. (my italics)

Click to expand...

Different groups could have taken the same data and come up with 4 or 6 factors, and they could all be right, ie it's a matter of opinion - to some extent (with 4, 5 or 6 factor solutions, 4 of the factors would be very similar across all solutions). For that reason, Factor Analysis findings shouldn't be seen as absolute.

It's notable that the authors 'Inflammation' factor included both sharp chest pains and dizziness, neither of which is obviously linked to inflammation (and neither of which is part of Fukuda), which helps to illustrate how the factors are not exactly absolute.

Scope for judgement in Factor Analysis

Some of the techinical detail from the paper:

To determine the number of factors to retain for rotation to an interpretable solution, we used a combination of
1. the eigenvalues,
2. the percentage of the total variance explained by each possible number of factors and the associated scree plot,
3. the reproducibility of the factors, and
4. the clinical meaningfulness of the factors extracted. points numbered for clarity
[i.e. there wasn't a simple formula used. Also:]

- Orthogonal (or varimax) rotation was used to maximize interpretability of factors [this also requires some judgement].
- An arbitrary but conventional threshold of 0.35 for the factor loadings was applied when interpreting and labelling the factors.

No exclusions for known medical and psychiatric illnesses.
There are many causes of chronic fatigue, just look at the standard exclusions in the Fukuda criteria:

The following conditions exclude a patient from the diagnosis of unexplained chronic fatigue:
1. Any active medical condition that may explain the presence of chronic fatigue (31), such as untreated hypothyroidism, sleep apnea, and narcolepsy, and iatrogenic conditions such as side effects of medication... 2... previously treated malignancies and unresolved cases of hepatitis B or C virus infection.
3. Any past or current diagnosis of a major depressive disorder with psychotic or melancholic features; bipolar affective disorders; schizophrenia of any subtype; delusional disorders of any subtype; dementias of any subtype; anorexia nervosa; or bulimia nervosa.
4. Alcohol or other substance abuse

Click to expand...

The authors dataset included "exclusionary and non-exclusionary medical and psychiatric illness" yet no patients with known causes of chronic fatigue were excluded from the data.

So although I agree with the authors to some extent when they say:

Conceptually, the present findings are consistent with the notion that the key symptom phenomena of chronic fatigue states are likely to share common central nervous system mechanisms,

Click to expand...

in that their may be final common pathways eg for fatigue, the causal pathology is likely to be different. Diabetes, cancer and hypothyroidism can all lead to chronic fatigue but you wouldn't treat them all the same way, as the authors propose:

1. All symptoms were dichotomised (ie Yes/No)even if there was frequency/severity data on the symptom in the orginal data. This inevitably increases number of symptoms each patient has, and so increases the level of correlation between symptoms, which will increase the apparent strength of factors. Also, while fukuda specifies headaches of new onset as a symptom, another patient with chronic fatigue may reports headaches, but they may have had that problem well before the fatigue started. Apparently there are technical reasons why dichomisded variables can produce unreliable results in factor analysis.

2.

mean substitution (where missing values were replaced with the mean of the variable) was used for the replacement of missing data values

Click to expand...

Imagine a patient has missing values for symptoms A and B - these will be replaced by the mean value for A and B. So every patient with missing data for symptoms A & B will then be given identical results for A and B, which will again articficially inflate the correlation between A and B (and so the strength of the factors).

3. Because different studies measured symptoms using different questionnaire, the answers to questions from different questionnaires were merged:

Before merging such items,
consideration was given as to whether the item was describing a
clinically similar phenomenon to any other item(s). If so, the
authors (including a psychiatrist, research psychologist, virologist,
biostatistician, epidemiologist and an infectious diseases physician)
made a unanimous decision to merge them (e.g. night sweating
and sweating more than usual), or to leave them separated (e.g.
waking up tired and feeling tired after rest or relaxation). If
clinical consensus could not be reached, polychoric correlations
were used to guide decision-making.

Click to expand...

This adds more uncertainty to the study.

Taking 1-3 together just adds to the doubt about the robustness of the findings.