Papers on meta-analysis

What is meta-analysis

In his best selling book Baby and Child Care, Dr. Benjamin Spock wrote ‘I think it is preferable to accustom a baby to sleeping on his stomach from the beginning if he is willing’. This statement was included in most editions of the book, and in most of the 50 million copies sold from the 1950s into the 1990s. The advice was not unusual, in that many pediatricians made similar recommendations at the time.

During this same period, from the 1950s into the 1990s, more than 100,000 babies died of sudden infant death syndrome (SIDS), also called crib death in the United States and cot death in the United Kingdom, where a seemingly healthy baby goes to sleep and never wakes up.

In the early 1990s, researchers became aware that the risk of SIDS decreased by at least 50% when babies were put to sleep on their backs rather than face down. Governments in various countries launched educational initiatives such as the Back to sleep campaigns in the UK and the US, which led to an immediate and dramatic drop in the number of SIDS deaths.

While the loss of more than 100,000 children would be unspeakably sad in any event, the real tragedy lies in the fact that many of these deaths could have been prevented.

Why perform a meta-analysis

Why perform a meta-analysis? What are the advantages of using statistical methods to synthesize data rather than taking the results that had been reported for each study and then having these collated and synthesized by an expert?

The goal of a synthesis is to understand the results of any study in the context of all the other studies. First, we need to know whether or not the effect size is consistent across the body of data. If it is consistent, then we want to estimate the effect size as accurately as possible and to report that it is robust across the kinds of
studies included in the synthesis. On the other hand, if it varies substantially from study to study, we want to quantify the extent of the variance and consider the implications.

Meta-analysis is able to address these issues whereas the narrative review is not. We start with an example to show how meta-analysis and narrative review would approach the same question, and then use this example to highlight the key differences between the two.

When does it make sense to perform a meta-analysis

In the early days of meta-analysis (at least in its current incarnation) Robert Rosenthal was asked if it makes sense to perform a meta-analysis, given that the studies differ in various ways, and the analysis amounts to combining apples and oranges. Rosenthal answered that combining apples and oranges makes sense if your goal is to produce a fruit salad.

The goal of a meta-analysis is only rarely to synthesize data from a set of identical studies. Almost invariably, the goal is to broaden the base of studies in some way, expand the question, and study the pattern of answers. The question of whether it makes sense to perform a meta-analysis, and the question of what kinds of studies to include, must be asked and answered in the context of specific goals.

The ability to combine data from different studies to estimate the common effect (or mean effect), continues to be an important function of meta-analysis. However, it is not the only function. The goal of some syntheses will be to report the summary effect, but the goal of other syntheses will be to assess the dispersion as well as the mean effect, and the goal of others will be to focus on the dispersion exclusively.

There are two popular statistical models for meta-analysis, the fixed-effect model and the random-effects model. The fact that these two models employ similar sets of formulas to compute statistics, and sometimes yield similar estimates for the various parameters, may lead people to believe that the models are interchangeable. In fact, though, the models represent fundamentally different assumptions about the data.

The selection of the appropriate model is important to ensure that the various statistics are estimated correctly. Additionally, and more fundamentally, the model serves to place the analysis in context. It provides a framework for the goals of the analysis as well as for the interpretation of the statistics.

“I have really enjoyed using the CMA software. Comprehensive Meta-Analysis makes performing a meta-analysis incredibly straightforward. There are numerous options for data analysis and presentation of results. The software was easy to learn, the tutorials were truly useful, and the staff at CMA were always helpful. My meta-analysis using your software has recently been published – Calcif Tissue Int. 2010 Sep;87(3):193-202. Epub 2010 Jun 15.”

“I first encountered Comprehensive Meta-Analysis in an online course in the basics of the statistical technique. The software was intuitive and easy to learn, and it has features that go well beyond the basics. The ability to handle multiple types of data and the comprehensive set of analyses available should satisfy even the most demanding research requirements.”

Scott Segal, MD, MHCM

“I have used Comprehensive Meta-Analysis since I was a student and it has been the basis for dozens of meta-analysis publications since then. The strengths of the software include its intuitive approaches to statistical issues, ease of use, and clear presentation of the data. Journals appreciate that important changes in figures can be made rapidly and clearly. This has also been an excellent tool for teaching students and colleagues about conducting meta-analysis - students enjoy the clarity of presentation and ease of use. With all the ease of use, it is important to note that the software allows a variety of statistical approaches that may be difficult to implement in other software, from choices of meta-regression effects to presentations of heterogeneity. For my group, this software is indispensable.”

“Comprehensive Meta-Analysis program is relatively easy to use even for the statistic-phobes. Directions are easy to follow and guide you through the set-up of your data. The program makes a seemingly complex process fairly straightforward.”

“It is a pleasure to recommend Comprehensive Meta-Analysis. It is dependable, educational and easy to use software that we use often for research and teaching. Using Comprehensive Meta-Analysis is very helpful in my research. Moreover, I have become aware of several new procedures and caveats. Using the software is a beautiful way to understand and build skills and a great teaching tool. It should be used by any epidemiologist who is interested in meta- analysis.”

Shai Linn, MD, DrPH, Dean of the Faculty of Welfare and Health Sciences, University of Haifa