meta-analysis bias in location and selection of studies

Matthias Egger, reader in social medicine and epidemiologya(m.egger@bristol.ac.uk),

George Davey Smith, professor of clinical epidemiologya

a Department of Social Medicine, University of Bristol, Bristol BS8 2PR

Correspondence to: Dr Egger

Introdution

Meta-analysis has received a mixed reception since the outset. Some people have rejected what they see as exercises in “mega-silliness,”1 while the purveyors of a highly distinguished series of meta-analyses of perinatal medical care2 have been dismissed as “an obstetrical Baader-Meinhof gang.”3 To some clinicians objecting to the findings of meta-analyses, “a tool has become a weapon.”4 At the other end of the spectrum, the application of a technique that basically consists of calculating a weighted average has been hailed as “Newtonian,”5 and it has been suggested that meta-analysis has left no place for the narrative review article.6 The truth is likely to lie somewhere between these extreme views.

Summary points

Bias can be introduced in many ways into the process of locating and selecting studies for inclusion in meta-analysis

Studies with significant results are more likely to get published than studies without significant results, leading to publication bias

Among published studies, those with significant results are more likely to get published in English, more likely to be cited, and more likely to be published repeatedly, leading to English language bias, citation bias, and multiple publication bias

In less developed countries, studies with significant results may be more likely to get published in a journal indexed in a literature database, which can introduce database bias

Criteria for including studies in a meta-analysis may be influenced by knowledge of the results of the set of potential studies, leading to inclusion bias

The likely presence or absence of bias should be routinely examined in sensitivity analyses and funnel plots

That meta-analysis holds potential problems can be illustrated by contrasting the conclusions of two meta-analyses comparing low molecular weight heparins and standard heparin in the prevention of thrombosis after surgery.78 One group concluded that “low molecular weight …