Tuesday, 5 July 2011

Does Champix cause heart attacks? Hmm, maybe.

The news that Champix/Chantix has been linked to heart disease has been widelyreported in the last day or two, after a meta-analysis found that those who use the controversial stop-smoking drug had a 72% increased risk of adverse cardiovascular events.

The lead author, Sonal Singh, has been getting rather excited about the results.

"This is just like driving a car without brakes,” he said. “Going forward, I don’t know how we will convince our patients to take Champix for what, to increase your risk for heart attack?... People should be concerned. They don’t need Chantix to quit and this is another reason to avoid Chantix all together."

The days of the sober, disinterested academic may be over, but it would nice if researchers could release their results without sounding quite so much like campaigners. Especially when your results really aren't that impressive.

The meta-analysis took 14 studies and found a relative risk of 1.72. That sounds OK until you look at the results of the studies themselves (click to enlarge).

Of the 14 results, only one of them is statistically significant (just) and the rest of them are miles away from achieving significance. This is because the number of people suffering heart attacks is tiny—about one in a thousand overall. In only one of the studies was there more than seven cardiovascular events between the Champix users and the placebo group combined. In six of the studies, there was only one heart attack in the entire cohort. When all studies are combined you still only have 79 heart attacks from over 8,000 people (and 45 of these come from just one study, which clearly used a very different methodology). This, at the very least, indicates that if there is a risk here—and there may be—it is very small in absolute terms. It is simply impossible to take these studies as proof of anything.

This is not the (original) researchers' fault. The studies are too small for enough cases to emerge. If and when larger studies are conducted, it may become apparent that there is a real risk. The fact that 11 of the 14 studies show an increase in risk—albeit a very flimsy one—is suggestive of an effect, but I would put it no more strongly than that. Comparing it to driving a car without brakes is ludicrous and suggests a certain crusading zeal from the study's authors that may have biased their approach.

This is the problem with meta-analysing statistically nonsignificant relative risks. It's a fallacy to think that by combining different studies with different flaws and biases you are creating one good, robust study. You're not. If the original studies don't tell you anything, combining them usually doesn't make them any more informative. Meta-analyses are pretty good at estimating an accurate relative risk when there is strong evidence that a risk exists. When the risk is far from proven—and most of these studies aren't even suggestive—they generate more heat than light.

I realise that this view is far from universal in the debased world of modern epidemiology (not least by those who gather together a similar grab-bag of weak and conflicting secondhand smoke studies to 'prove' various outcomes), but achieving statistical significance by combining a bunch of near-useless studies is, in my view, tantamount to cheating. It is the equivalent of multiplying zero by ten and coming up with two. Consequently, the researchers have found a relative risk of 1.72 that is much higher or much lower than any of the risks found in the studies themselves. That should be a clue that something is wrong.

Lord knows, I hold no brief for the pharmaceutical industry and there are already good reasons not to take Champix without worrying about heart problems, but if the drug really does cause users to have heart attacks, this meta-analysis falls a long way short of proving it. It would be wrong and inconsistent to expect a lower burden of proof against Champix just because it is made by Big Pharma. This is one of those instances where more research really is needed.

7 comments:

Anonymous
said...

"... this meta-analysis falls a long way short of proving it. It would be wrong and inconsistent to expect a lower burden of proof against [Second Hand Smoke] just because it is made by Big [Tobaccl]. This is one of those instances where more research really is needed."

Actually, Chris, the Relative Risk on those numbers (52/4908 Champix, 27/3308 Placebo) is approx 1.11 with a lower CI of 0.9. The 'proper' Odds ratio is approx 1.3 with a lower CI of 0.8 - so neither option achieves any recognisable level of statistical significance.

But of course these researchers used the Peto Odds Ratio, (mathematically, it isn't an odds ratio at all - but that's for another debate), which magically transforms meaningless statistics into press-release headlines.

"was made a Fellow of the Royal Society in 1989 for his contributions to the development of meta-analysis. He is a leading expert on deaths related to tobacco use."

So, the anti-tobacco supremo devised his own statistical procedure that coincidentally enables anti-tobacco to 'improve' the results of their home-grown method of numerical cheating (meta-analysis). Well colour me cynical!

I concur with Anon above; the fact that this methodology is now going to be used to damn big pharma is pungently ironic (or is that ironically pungent?).

Excellent!

I wonder when the ASH knuckle-draggers will be wheeled out to complain about the 'inadequacies' of this study. That I will then be forced to agree with them is somewhat frightening!

Oh, please, meta-analysis in general is a fairly useful tool. It's useful in cases where you have relatively small studies due to inherent sample size limitations or relatively rare outcomes, and want to demonstrate statistical significance.

The specific problem here is that 55% of the weight is given to one study.

About Me

Writer and researcher at the Institute of Economic Affairs. Blogging in a personal capacity.
Author of Selfishness, Greed and Capitalism (2015), The Art of Suppression (2011), The Spirit Level Delusion (2010) and Velvet Glove, Iron Fist (2009).

"Of all tyrannies, a tyranny exercised for the good of its victims may be the most oppressive. It may be better to live under robber barons than under omnipotent moral busybodies. The robber baron's cruelty may sometimes sleep, his cupidity may at some point be satiated; but those who torment us for our own good will torment us without end, for they do so with the approval of their own conscience."