"I am but mad north-north-west: when the wind is southerly I know a hawk from a handsaw." --Hamlet, Act II, scene ii.

Tuesday, 19 August 2008

A mystery paper...

Just a little bit more on the interview with Dana Ullman that I wrote about here.

Ullman claims that a re-analysis of Shang et al. has been accepted for publication in the Journal of Clinical Epidemiology. The only reference I can find to this study is this, where a study dated 2007, entitled "The conclusions on the effectiveness of homeopathy highly depend on the set of analysed trials" by R Ludtke and ALB Rutten is listed as being 'in press' in the Journal of Clinical Epidemiology.

Here's the list of articles in press in the Journal of Clinical Epidemiology. There is no sign of any such paper. Various searches fail to find any similar papers published anywhere else, or in earlier issues of the Journal of Clinical Epidemiology. The only thing I can find is a paper in Homeopathy called "‘Proof’ against homeopathy in fact supports Homeopathy", in which one Lex Rutten is credited as the first author. Whether this is the same Rutten I cannot say. The main point of the paper seems to be that if you add four positive trials to the Shang dataset, the result would be more positive. And they accuse Shang of cherry-picking. Two of the trials complained about were excluded [PDF] from the Shang meta-analysis: the Fisher et al. paper because it had an ineligible study design, and the Weisenauer and Gaus paper because no matching conventional trial could be found. Of the other two, one by Arnal-Laserre appears to be a French thesis of some description [EDIT: This is a French thesis: it was mentioned in the Cochrane review of "Homoeopathy for the induction of labour". Apparently, the reviewers could not obtain a copy of the thesis, which perhaps explains why Shang et al. did not include it], and the other by Maiwald et al. was not a placebo-controlled trial.

So, does this re-analysis exist, or is it just another figment of the collective homeopathic imagination? And if it ever does get published, is it likely that it will have anything useful to say?

6 comments:

I suspect its just another figment of the collective homeopathic imagination. Interestingly R Ludtke has written an article stating that he sees no point in continued revisiting of systematic reviews as the conclusions don't change. So the question is: do we really gain anything from doing more and more reviews and meta-analyses? Can we really expect to generate new evidence on the effectiveness of homeopathy by continuing on this path? I fear not. One does not have to be clairvoyant to predict that the conclusions of the next systematic review on homeopathy will be: “Due to the small number of included trials up to now there is not enough evidence to say either that homeopathy works or not”.

My plea to the research community is: let's stop that superfluous research and concentrate on generating new, original, data. Let's conduct more clinical trials, not only on the efficacy of homeopathic medicines but also on the effectiveness homeopathy as a package of care in ‘real world’ clinical practice. Let's figure out whether the new ideas on how homeopathy might work really withstand empirical testing. Let's see what we can learn from basic research experiments on highly diluted substances. Let's further explore what really happens in the complex interaction between a homeopath and his or her patient.

What we need at this point in time is a growing body of evidence on homeopathy fueled by original research. The conduct of more reviews will not contribute to this goal.

Of, course, the Carstens Foundation (for whom Dr Ludtke seems to have been working when he wrote the article Gimpy cited) is dedicated to promoting the integration of CAM into mainstream medicine... so it isn't that much of a surprise that he calls for "more new research". One can say similar things about NCCAM in the USA.

The trick always lies in whether they fund research that really asks the critical questions, or instead fund poorly-designed stuff that skirts around the central issue. NCCAM seems to specialise in funding the latter; I don't know about the Carstens Foundation, but I am "default cynical".

Of course, all real scientists also like to call for more research.. and I freely admit to ending several papers and a review that way myself... but one has to ask when the point comes at which we really need more research into things that are repeatedly shown to be placebo therapies. How many times do we need to be told that is what they are? Far better to change the direction of the research and ask "what makes an effective placebo, and for whom?", as Ben Goldacre and others have argued.

Day job...

Fieldwork in Egypt

Welcome to Hawk/Handsaw...

This is the blog of Paul Wilson. You can find science- and pseudoscience-related things here, as well as occasional posts on my current research. There is also some stuff about what I do in my spare time (mostly cycling and complaining about politics).