Binary Data For a meta-analysis of binary data, which of the following metrics is likely to be more statistically heterogeneous? Incorrect. In most meta-analyses, treatment effects across studies are more likely to be consistent with a multiplicative model for odds or risks when compared with risk difference. When there are differences in the rate of the event in the comparator arms, the risk difference will differ greatly, even when the odds ratio (or the risk ratio) is constant. Correct. In most meta-analyses, treatment effects across studies are more likely to be heterogeneous in the risk difference metric when compared with the risk ratio or the odds ratio. When there is little or no difference in the rate of the event in the comparator arms, all three metrics may be statistically homogeneous.

Reasons for a Meta-analysis Which of the choices listed below is not a reason to carry out a meta-analysis? Incorrect. In addition to obtaining an overall estimate of treatment effect, the major benefit of carrying out a meta-analysis is to understand discrepancies (heterogeneity) of results across studies that address the same question. Incorrect. Obtaining an overall estimate and a measure of uncertainty (confidence interval) is the main reason for carrying out a meta-analysis. Multiple studies that are not statistically significant alone may yield significant results when combined in a meta-analysis. Correct. Most meta-analyses are retrospective exercises that use information reported in publications. Although imputation techniques may be able to fill in missing data, we cannot correct errors inherit to those studies. Thus, it is important to critically appraise the methodological quality and reporting of information of included studies.

Fixed Effects Model Versus Random Effects Model What is the key difference between the fixed effect model and the random effects model? Incorrect. Obviously there are more important differences than spelling between these two models. Incorrect. The random effects model incorporates between-study variations into the calculation and gives a wider confidence interval when heterogeneity is present. When there is no heterogeneity, the between-study component of this weight is zero. In this case, the weights of the fixed effect model and random effects model are identical and will give the same result. Correct. The method weighting studies in the fixed effect model considers only the within-study variation, whereas the random effects model incorporates both within-study variation and between-study variations.

Assessing Study Conclusions This is part of a table from a meta-analysis of genetic factors that concludes: “polymorphisms of both GSTT1 and GSTP1 genes seem [to be] associated with elevated breast cancer risk in a race-specific manner.” This conclusion is based on a significant association among “non-Chinese,” versus a nonsignificant association among “Chinese.” Do you agree with the conclusion? Incorrect. The fact that one meta-analysis is significant and the other is not does not tell us anything about whether the two summary odds ratios are significantly different from each other. Correct. The conclusion is unfounded and entirely unsupported by the performed analyses. It is trivial to show that the summary effects of the two meta-analyses do not differ beyond chance ( p = 0.56, from the reported summaries). Furthermore, the choice of the authors to examine these particular subgroups is not explained. Are there any problems with choosing subgroups arbitrarily? Reference: Sergentanis TN, Economopoulos KP. GSTT1 and GSTP1 polymorphisms and breast cancer risk: a meta-analysis. Breast Cancer Res Treat 2010;121:195-202. http://www.ncbi.nlm.nih.gov/pubmed/19760040

Assessing Study Methods As described in the methods section of a paper, the authors opted to use random or fixed effects models according to whether a heterogeneity test was significant. Do you agree or disagree with their choice? Incorrect. Testing for heterogeneity and then choosing between fixed and random effects is bad practice on many levels. Clinical and methodological heterogeneity are practically always present; therefore, you should opt for random effects. (Exceptions apply for meta-analyses of studies with rare events.) Furthermore, the Q test is insensitive when there are few studies and oversensitive when there are many studies. Correct. Clinical and methodological heterogeneity are practically always present; therefore, one should generally opt for random effects. Exceptions apply for meta-analyses of studies with rare events. Reference: Sergentanis TN, Economopoulos KP. GSTT1 and GSTP1 polymorphisms and breast cancer risk: a meta-analysis. Breast Cancer Res Treat 2010;121:195-202. http://www.ncbi.nlm.nih.gov/pubmed/19760040

Summary

Authors This interactive quiz augments the first module on quantitative synthesis. This quiz was prepared by Joseph Lau, M.D., and Thomas Trikalinos, M.D., Ph.D., members of the Tufts Medical Center Evidence-based Practice Center. It is based on Chapter 9 in Version 1.0 of the Methods Guide for Comparative Effectiveness Reviews (available at: http://www.effectivehealthcare.ahrq.gov/ehc/products/60/294/2009_0805_principles1.pdf).

This quiz was prepared by Joseph Lau, M.D., and Thomas Trikalinos, M.D., Ph.D., members of the Tufts Medical Center Evidence-based Practice Center.

The information in this module is based on Chapter 9 in Version 1.0 of the Methods Guide for Comparative Effectiveness Reviews (available at: http://www.effectivehealthcare.ahrq.gov/repFiles/2007_10DraftMethodsGuide.pdf).