Outline

Introduction

Meanwhile it is well understood that meta-analyses with rare binary (e.g., adverse events or safety) outcomes should not use the standard fixed-effects (FE) or random-effects (RE) model for inference on treatment effects. This has been theoretically proven [Ref.Â 1], [Ref.Â 2] and empirically confirmed in several simulation studies [Ref.Â 3], [Ref.Â 4], [Ref.Â 5], [Ref.Â 6], [Ref.Â 7]. The Mantel-Haenszel (MH) method [Ref.Â 8] has been recommended instead in these sparse data situations. However, these recommendations are frequently overlooked. For example, the three largest systematic reviews [Ref.Â 9], [Ref.Â 10], [Ref.Â 11] on the comparison of off-pump and on-pump surgery in coronary artery bypass surgery, one of the most hotly debated and polarizing issues in cardiac surgery [Ref.Â 12], all use the standard methods. This is especially problematic as the outcomes most relevant to the patient, postoperative death, stroke, myocardial infarction, or renal failure are observed very rarely, in about 1-2% of the patients.

Results

Our initial search yielded 106 publications, of which 57 met the inclusion criteria. These included information from 51 RCTs, 223 single study odds ratios, and 4943 patients. For clinical outcomes which are observed only rarely, the FE and the RE odds ratios always underestimate the treatment effect, in some cases considerably. For example, the FE and the RE odds ratios for postoperative death were 0.94 [0.58, 1.50], but 0.64 [0.36, 1.15] from the MH approach. For postoperative stroke the FE and the RE odds ratios were 0.71 [0.41, 1.23], but 0.50 [0.25, 1.00] from the MH approach.

Discussion

Current meta-analyses for the comparison of off-pump and on-pump surgery in coronary artery bypass surgery underestimate the benefit of the off-pump method, especially for the clinical outcomes which are most relevant to the patient. This is due to the inadequate application of the standard statistical methods. The application of the Mantel-Haenszel method would yield a more realistic picture.

Sellke FW, DiMaio JM, Caplan LR et al. Comparing on-pump and off-pump coronary artery bypass grafting: numerous studies but few conclusions: a scientific statement from the American Heart Association council on cardiovascular surgery and anesthesia in collaboration with the interdisciplinary working group on quality of care and outcomes research. Circulation 2005; 111: 2858-64.