mahiwaga

I'm not really all that mysterious

Aside from the overhyped and clickbaity headlines, one of the basic problems of popular medical journalism is that they generally elevate meta-analyses over randomized controlled trials.1

It isn’t that meta-analyses aren’t as useful, but they can certainly be more unreliable due to having to summarize possibly quite disparate studies with widely varying sample populations with significant differences in experimental design and in expected end-points and measurements. Sure, you can account for a lot of this with statistical analysis, but this tends to introduce additional assumptions that may or may not be warranted, and is another step that’s susceptible to bias.2

Including all relevant material–good, bad, and indifferent–in meta-analysis admits the subjective judgments that meta-analysis was designed to avoid. Several problems arise in meta-analysis: regressions are often non-linear; effects are often multivariate rather than univariate; coverage can be restricted; bad studies may be included; the data summarised may not be homogeneous; grouping different causal factors may lead to meaningless estimates of effects; and the theory-directed approach may obscure discrepancies. Meta-analysis may not be the one best method for studying the diversity of fields for which it has been used.

A meta-analysis is a safer starting point than a single study – but it won’t necessarily be more reliable.

A meta-analysis is usually part of a systematic review. It’s a heavy-duty effort, and it’s often described as the ultimate study, outweighing all others. The last word. A single study becomes a puny thing, to be ignored even.

But while combined results can carry a lot of weight, there are 3 main problems with the idea that a meta-analysis always trumps a single study.

Firstly, a systematic review and meta-analysis isn’t a formal experimental study. It’s a non-experimental or descriptive study. There are subjective judgments every step of the way, giving small teams of like-minded people plenty of room to steer in a desired direction if they want to. A bad or patchy meta-analysis might not come to as reliable conclusions as a well-conducted, adequately powered single study.

Secondly, it’s not at all unusual for a meta-analysis to be heavily dominated by a single study. A study by Paul Glasziou and colleagues in 2010 found that even when there were several trials, the most precise one carried on average half the weight of the results – and around 80% of the time the conclusion of the meta-analysis was pretty much the same as that single study. Understanding and discussing that dominant study is critical.

The third problem is so big, it gets the next place on this listicle: a single new study can overturn the results of a meta-analysis.…

A meta-analysis is a snapshot in time – it can even be out-of-date the day it’s published.

Pope Francis understands that the root cause of terrorism are the inequalities caused by the history of imperialism and colonialism and perpetuated by global capitalism.

I think that in nearly all religions there is always a small fundamentalist group.… We have them…. I don’t like to talk about Islamic violence because every day when I look at the papers I see violence here in Italy - someone killing his girlfriend, someone killing his mother-in-law. These are baptised Catholics. If I speak of Islamic violence, I have to speak of Catholic violence. Not all Muslims are violent.… I know it [is] dangerous to say this but terrorism grows when there is no other option and when money is made and it, instead of the person, is put at the centre of the world economy. That is the first form of terrorism. That is a basic terrorism against all humanity. Let’s talk about that.