CMA software - Comparisons

How does CMA compare to Revman?

Data entry is much easier with CMA.

The mechanics of data entry are much simpler in CMA – you work with a spreadsheet interface, and can copy-and paste-data as easily as you could in Excel. By contrast, the data entry process in Revman requires the user to set up tables and comparisons before starting data entry.

CMA will compute the effect sizes automatically.

In every meta-analysis you start with the published summary data for each study and compute the treatment effect (or effect size). For example, if a study reports the number of events in each group you might compute the odds ratio. Or, if a study reports means and standard deviations you might compute the standardized mean difference.

Revman will accept summary data in only two formats – events and sample size, or means and standard deviations. If any studies provide data in another format (such as odds ratio and confidence intervals) you would need to compute the effect sizes and variances manually for those studies. By contrast, CMA allows you to enter data in more than 100 formats, and will compute the effect size and variance for all of these formats. Equally important, Revman requires that data for all studies be entered using the same format. By contrast, with CMA you can enter data for each study in its own format, and use as many formats as needed in the same analysis. CMA also supports a much wider range of effect sizes than Revman.

CMA is able to create a customized, high-resolution forest plot.

The forest plot in Revman offers few options for customization. By contrast, CMA allows the user full control over all elements in the forest plot, will create scalable plots (that print at the highest resolution possible for the printer or journal), and allows the user to control the color for every element on the plot. CMA also allows one-click export to other programs such as PowerPoint™ and Word™.

Advanced Functions

CMA allows you to assess the impact of moderator variables. Use analysis of variance to compare the treatment effect across groups (“Is the treatment more effective for acute patients than for chronic patients?”). Use meta-regression to assess the impact of continuous moderators (“Does the treatment effect increase with dosage?”). To assess the potential impact of publication bias CMA includes an array of functions including a funnel plot, where Revman includes only the funnel plot. CMA will run a cumulative meta-analysis to show how the evidence has shifted over time. It will also run a one-study removed analysis to show the impact of each study on the combined effect.

Does CMA offer the same formulas as Revman?

Yes. The development team for CMA includes some of the same people responsible for the development of Revman. CMA includes all of the same computational formulas, has been validated against Revman and provides exactly the same results (see documentation). CMA offers additional options as well, but includes a button to "Use the same options as Revman," which sets all options to match Revman.

“Although my centre develops the statistical software the whole Cochrane Collaboration uses for its systematic reviews and meta-analyses (Review Manager), I have often used Comprehensive Meta-Analysis for my own research projects, as it is easy to import data from Excel, to use effect modifiers, and also because it allows meta-regression and produces bubble graphs that are easy to work with in Word or PowerPoint.”

“Your software is very convenient and useful for systematic reviewers. I used this software to conduct Cochrane systematic review and also helped several postgraduates at University of Ottawa for their research on environmental toxicology and epidemiology. Their manuscripts were all accepted by the internal peer-reviewed journals.”

“I have used Comprehensive Meta-Analysis software within a course on research methods for doctoral students in nursing. With only a 90-minute introduction, students were ready to begin data entry and to conduct basic analyses. Both they and I were pleased with its intuitive interface and with the clarity of its output. Additionally, Biostat, Inc. has provided excellent support in our efforts to employ the software in a classroom setting.”

Stephen J. Walsh, Sc.D., Associate Professor School of Nursing, University of Connecticut

How does CMA compare to Stata?

CMA is a program developed specifically for meta-analysis. As such, it includes functions to automatically compute effect sizes, to perform basic and advanced meta-analyses, and to create publication quality graphics.

Stata is a general purpose statistical package. While Stata has no intrinsic support for meta-analysis, various experts have written macros for meta-analysis, which can be downloaded from the Stata web site and incorporated into Stata. These macros include procedures for basic analysis, for cumulative analysis, for meta-regression, for publication bias, and more. Since the functionality of Stata and CMA are comparable, the main difference is in ease of use and in the options for customizing the output.

Stata is a command-driven language, which means that you type commands, or use a dialog box to create commands, which are then submitted to the program. CMA is a menu-driven program, similar to Excel™.

Computing effect sizes

Stata will accept summary data in only three formats – events and sample size, means and standard deviations, or (in some cases) point estimate and confidence interval. If any studies provide data in another format you would need to compute the effect sizes and variances manually or by writing code. By contrast, CMA allows you to enter data in more than 100 formats, and will compute the effect size and variance for all of these formats. Equally important, Stata requires that data for all studies be entered using the same format. By contrast, with CMA you can enter data for each study in its own format, and use as many formats as needed in the same analysis.

Analyses

Stata computes values and sends these to a DOS-like window for viewing. In CMA the analysis screen is interactive – you can use the screen interactively to explore the impact of different studies, the effect of alternate weighting schemes, and so on.

Forest plots

Stata’s forest plot offers few options for customization. It includes a column for the study name and a symbol representing the point estimate and confidence interval. By contrast, the forest plot in CMA can be customized extensively.

Does CMA offer the same formulas as Stata?

Yes. The development team for CMA includes some of the same people who developed the Stata macros. CMA includes all of the same computational formulas, was validated against Stata and provides exactly the same results (see documentation). CMA offers additional options as well, but includes a button to "Use the same options as Stata," which sets all options to match Stata.

Can CMA import data from Stata?

Yes. You can open the data sheet in Stata, copy the data onto the Windows clipboard, and then paste it into CMA. Then you tell CMA what kind of data is located in each column. The process takes only moments to complete.

“Thank you for developing Comprehensive Meta-Analysis. It is by far the best tool available for today’s meta-analytic researcher. It provides flexibility for all possible types of data, it produces great colorful graphical displays, and the product support is unmatched! I have used a number of different programs for conducting meta-analytic research over the last ten years, and CMA is the only product I recommend to my students and colleagues.”

“Our group has recently begun to conduct meta-analyses within our area of research, i.e. psycho-oncology and health psychology, and we have found CMA extremely useful. CMA distinguishes itself from other available meta-analysis software by the various options to explore and adjust for possible publication bias, as well as by providing several options to explore possible moderators, not only categorical but also continuous. We encourage our PhD students to conduct quantitative systematic reviews as a part of their dissertation whenever possible. The clear menu-driven approach of CMA makes it easy to use for beginners, so that they can focus their energy on the analytical aspects of meta-analysis, rather than on the technical issues of using the software.”

How does CMA compare to SPSS?

CMA is a program developed specifically for meta-analysis. As such, it includes functions to automatically compute effect sizes, to perform basic and advanced meta-analyses, and to create publication quality graphics.

SPSS is a general purpose statistical package with no intrinsic support for meta-analysis. However, David Wilson has written macros that can be incorporated into SPSS and will run a basic meta-analysis, an analysis of variance, and meta-regression.

Computing effect sizes

In every meta-analysis you start with the published summary data for each study and compute the treatment effect (or effect size). For example, if a study reports the number of events in each group you might compute the odds ratio. Or, if a study reports means and standard deviations you might compute the standardized mean difference. Additionally, you need to compute the variance for each effect size.

Wilson’s macros require the user to compute an effect size and variance for each study, and then provide these values to the program. Therefore, the user must either compute these values separately and then enter them as data, or write code to compute these values within SPSS.

By contrast, with CMA you enter the data directly in almost any format(s), and the program computes the effect size and variance automatically.

Analyses and forest plots

Wilson’s macros will report all relevant statistics, but will not create graphics such as a forest plot. CMA is able to create a forest plot, which can play a key role in helping the researcher to interpret the data and to convey it to others.

Does CMA offer the same formulas as SPSS?

SPSS itself does not include any support for meta-analysis. Wilson’s macros use the same formulas as CMA (CMA offers additional options as well), and so will yield identical results. This assumes, of course, that the user has used the same formulas to compute effect sizes and variance for each study.

Can CMA import data from SPSS?

Yes. You can open the data sheet in SPSS, copy the data onto the Windows clipboard, and then paste it into CMA. Then you tell CMA what kind of data is located in each column.

“Comprehensive Meta-Analysis is, literally speaking, the most comprehensive and user friendly meta-analysis “package” program. It excellently accommodates both advanced and beginning users’ needs at the same time and different levels of needs over time. Among numerous laudable features, this package program provides the most comprehensive and advanced solution for testing publication bias. Using Excel-like interfaces, moderator analyses are very easy to do! It would be a great loss for any competent researcher and practitioner to overlook this impressive package program.”

“Comprehensive Meta-Analysis has been the perfect tool for all of my meta-analysis needs. From calculating effect sizes to examining moderator effects to testing publication bias to building visually appealing forest plots, this program has it all. Its simplicity and straightforward design makes it easy to use for those just learning this data analysis technique. Additionally, with all that is included, this program is also great for those wishing to run more sophisticated analyses.”

Joshua Swift, PhD, University of Alaska, Anchorage

How does CMA compare to SAS?

CMA is a program developed specifically for meta-analysis. As such, it includes functions to automatically compute effect sizes, to perform basic and advanced meta-analyses, and to create publication quality graphics.

SAS is a general purpose statistical package with no intrinsic support for meta-analysis. David Wilson has written macros that can be incorporated into SAS and will run a basic meta-analysis, an analysis of variance, and meta-regression.

Computing effect sizes

In every meta-analysis you start with the published summary data for each study and compute the treatment effect (or effect size). For example, if a study reports the number of events in each group you might compute the odds ratio. Or, if a study reports means and standard deviations you might compute the standardized mean difference. Additionally, you need to compute the variance for each effect size.

Wilson’s macros require the user to compute an effect size and variance for each study, and then provide these values to the program. Therefore, the user must either compute these values separately and then enter them as data, or write code to compute these values within SAS.

By contrast, with CMA you enter the data directly in almost any format(s), and the program computes the effect size and variance automatically.

Analyses and forest plots

Wilson’s macros will report all relevant statistics, but will not create graphics such as a forest plot. CMA is able to create a forest plot, which can play a key role in helping the researcher to interpret the data and to convey it to others.

Does CMA offer the same formulas as SAS?

SAS itself does not include any support for meta-analysis. Wilson’s macros use the same formulas as CMA (CMA offers additional options as well), and so will yield identical results. This assumes, of course, that the user has used the same formulas to compute effect sizes and variance for each study.

Can CMA import data from SAS?

Yes. Since SAS does not support copy-and-paste, it is not possible to simply copy the data to CMA. However, SAS is able to export data to a file, which can then be imported to CMA.

“While I was looking for a computer program for meta-analysis, I did a thorough search and compared all available programs I located. I decided to use CMA because of its versatility and authoritativeness. It allows me to code data of various formats and conduct all kinds of analysis – Q tests, meta-regression, sensitivity analysis, etc. It also provides results associated with both the fixed-effects and random-effects models. Furthermore, the program is developed by a team of leading scholars in meta-analysis. I feel lucky to be able to find a program through which I can draw on the expertise of so many outstanding experts.”

“(1) Working with your staff and the software itself made computer installation and start up incredibly easy. (2) Instructing my students (seniors in a capstone research intensive course) on how to use the software was also remarkably easy. They did far more with the software than I had expected and made some elegant looking figures for their senior presentations. (3) Meta-analysis was a new technique for our whole department (PUI institution), and the oral presentations by my students using the CMA software and graphics impressed faculty immensely.”

Susan B. Chaplin, Department of Biology, University of St. Thomas, St. Paul, MN

How does this program compare to Excel?

Excel is a spreadsheet program with no intrinsic support for meta-analysis. While it is possible to program all meta-analysis formulas in Excel, this requires knowledge of the formulas and a substantial investment of time for development and testing. Also, there is no mechanism in Excel that can be used to create a forest plot.

“Comprehensive Meta-Analysis is, in my view, the best meta-analysis software on the market and a "must have" for any meta-analyst. Before I used this software, I was convinced that specialized meta-analysis software was not necessary at all. At that time, I used my own Excel spreadsheets and SPSS to run meta-analysis. In fact, I only tried CMA because I needed a way to make forest plots. Once I tried it, however, I was sold. I could not believe how user friendly it was and how much it could do. Suddenly not only was meta-analysis more efficient, but, more importantly, I could run all types of analyses that previously were not available with the software I had been using. I have since used the program to conduct and publish several meta-analyses. Given how great the program is, I require it when I teach my graduate meta-analysis seminar. Students always seem surprised when they see how easy it is to use, as this is certainly not the norm in statistical software. Thus, our graduate students seem to greatly value CMA as a key resource for conducting meta-analysis.”

“Seeing that I can do with Comprehensive Meta-analysis in 1 week what have taken me 3 weeks when I was using Excel spreadsheets for calculations I have the following: "Comprehensive Meta-Analysis has increased my effectivity to do meta-analysis with 200 %". I am extremely satisfied with this program.”

How does CMA compare to Metawin?

CMA and Metawin are both dedicated meta-analysis programs but CMA incorporates a much wider and more fully developed set of options.

Computing effect sizes

In every meta-analysis you start with the published summary data for each study and compute the treatment effect (or effect size). For example, if a study reports the number of events in each group you might compute the odds ratio. Or, if a study reports means and standard deviations you might compute the standardized mean difference. Additionally, you need to compute the variance for each effect size.

Both Metawin and CMA will allow you to enter summary data and will compute the effect size from that data. However, CMA will work with a much wider array of data formats. Metawin will accept data in a few formats where CMA can accept more than 100. Metawin requires that all studies provide data in the same format while CMA allows you to enter data for each study in its own format. Metawin can work with a few indices of treatment effect (or effect size) where CMA includes more than 15.

Analyses

Metawin will run the analysis and show the computed values. CMA displays all values as part of a scrollable grid which makes the analysis transparent – you can see which studies are included in the analysis, how the studies were weighted, and so on. CMA also includes a much more extensive set of computational options.

Forest plots

In CMA you can fully customize the plot, to ensure that each study stands out clearly, so that the plot is proportioned properly on the page, include all relevant columns, and so on. In Metawin the plot is very basic, allowing one column for study names but little control over formatting.

“I have used Comprehensive Meta-Analysis II for the last 6 years to assist with a number of published meta-analytic studies in the behavioral sciences. I have found the program to be flexible and powerful. I have also used CMA II in two graduate seminars focused on meta-analysis. The students are able to learn the program quickly and appreciate how it structures their data sets and prevents some common mistakes made in meta-analysis.”