Recently I have commented on what I see as methodologically poor, biased and spun studies of CBT for psychosis - every time I want to blog on someting else, out comes another questionable study - this time 'in press' at Schizophrenia Research, which I think requires comment - in particular, the gap between the reality and presentation of findings

This is an RCT looking at the use of CBT in UK minority groups with a diagnosis of schizophrenia
A total, n = 33 participants, who were randomly allocated to CBT for psychosis (CBTp n = 16) and treatment as usual (TAU n = 17). Although a relatively small study, the authors did a power analysis based on previous pilot studies and suggest that a minimum of 12 per arm of the trial would be sufficiently powered - obviously they exceed this expectation.

Did the CBT group show any benefit (i.e. reduction of symptoms) over the TAU group?

Conclusion: Participants in the CaCBTp group achieved statistically significant results post-treatment compared to those in the TAU group with some gains maintained at follow-up. High levels of satisfaction with the CaCBTp were reported

Flaming Lips - In the Morning of the Magicians

The key aspect here though is the bolded statement, added casually at the end of the results about 'adjustment'. Table 1 from the paper presented below indicates (as the authors rightly admit) that the TAU group were older, had longer duration of illness and greater medication and they duly adjusted analyses for these variables

However, the abstract results all refer to unadjusted scores - see the Table 2 below - all are taken from the non-adjusted column. If you glance to the final column (Adjusted reduction from baseline) - not a single comparison is significant!

Table 2 Results

Maybe someone can explain to me, what is happening here? Is it blatant author spinning to get further funding (as they mention this in the discussion). Is it collusion from reviewers, who let this through? Failure to spot the obvious by reviewers?
Perhaps I am overly critical - letters on a postcard (in the comments section please)

So the Abstract is factually correct ... but like all effective advertising copy they simply delete the second half of the final sentence, allowing the reader to infer something that isn't actually stated. The full version of the final sentence should presumably be something like, "Adjustment was made for age, gender and antipsychotic medication which rendered all significant effects non-significant"

Errors of ommission and 'selight of hand' are amongst the most pernicious & possibly widespread of problems in psychology research.The problem here is that the authors make massive inferences - to quote:

"1. The current study can inform a fully-powered PHASE III trial."

-Perhaps.... that such a trial is not justified on basis of this study?

"2. An effective treatment based on cultural explanations will ensure better quality of care for this group of patients due to betterengagement and outcomes. Offering CaCBTp offers choice and an evidence based alternative or adjunct to psychotropic medication(DoH:DRE, 2003; Inside Outside: Sashidharan, 2003)."

-CBT not shown to be an effective treatment in this group; and certainly no evidence-base here that it is an alternative (or adjunct) to psychotropic medication

"3. Implications for policy makers include the training of staff and dissemination of this therapy within resource constraints. Provision ofCBT is a cost effective way of managing patients with psychosis in the community as the relapse rate is found to be low (Turkington et al., 2006a,b)."

-massive extrapolations not based on anything here

"4. There are implications for further development of clinical or public health practice due to the role of this treatment in early intervention,relapse prevention and effective community care."

I'm also surprised that there is zero mention of the adjusted results in the paper, I checked it over multiple times just to make sure. Would love to know if I was missing something too. Also, this would be the near the best possible delivery conditions of this therapy since the authors created this particular version, so not showing an actual effect definitely raises serious obstacles to the implications they present.

No JT - they dont mention the 16 nonsignificant adjusted score analyses at all in the discussion or the abstractThe implications they draw are astounding in contentUnless some error has been made in the version that is posted on the 'Schizophrenia Research' website, then I really have to wonder what the reviewers/action editor were thinking when they let this one through as it is!