Tip for data extraction for meta-analysis – 28

March 24, 2020

What if the study only reports an effect estimate?

Kathy Taylor

Previously, I highlighted a list of ways where you might find that a summary statistic that you want, for meta-analysis of continuous outcomes, is missing. In my previous post, I looked at the 4th way – neither the summary statistic you want nor a similar statistic is reported. In this post, I’ll focus on another example of the 4th way.

Occasionally, only an effect estimate is given e.g. mean difference. This may be appropriate in some cases, such as for certain study designs, including non-randomised studies to reduce the impact of confounding. Sometimes summary data by treatment arm are not available and this situation might arise in a study abstract. If the reported effect measure is not the one you want to use in your review, then the study cannot be included in a meta-analysis. However, if the effect estimate is the one that you want to use, the study may be included in meta-analysis using the generic inverse variance method, where data are entered in the form of the effect estimate and its standard error (SE). If the effect estimate is reported without a measure of uncertainty (SE, or confidence interval (CI)) or a p-value, the SE can be imputed. If an effect measure is reported with a CI or p-value then these need to be converted to a SE.

The Cochrane Handbook refers to several different effect estimates. Those for continuous outcomes are the standardised mean difference, mean difference (MD), and ratio of means.

A standardised mean difference is reported

Calculating a SE from a confidence interval

As shown previously, for 95% CIs the denominator (D) should be 3.92, for 90% CIs, the denominator should be 3.29 and for 99% CIs the denominator should be 5.15 which are derived from standard normal tables. For a mean difference, confidence intervals should have been calculated from t-distributions and the denominators should therefore be taken from a t-distribution table which I showed previously.

In my previous post I showed how to calculate a SE from a confidence interval or p-value of a MD. The ratio of means, which is a less common effect measure, will be covered in my next blog post. I’ll look at how using this effect estimate can overcome problems with pooling data and some data extraction problems.

Here’s a tip… It’s possible to include a study in a meta-analysis that only reports an effect estimate, provided you know, or you can calculate, its standard error.

Where did the equations come from?(You can skip this if you are only interested in carrying out the calculations)

Figure 1. t-distribution table

The t-value, from the t-distribution (Figure 1) is the number of SEs the value (in this case, the mean difference) is from the mean (which is zero).