Your health insurance plan probably covers anti-inflammatory drugs. But does it cover acupuncture treatments? Should it? Which health services deliver good value for money? Lest you think the debate is limited to the United States (which is an outlier when it comes to health spending), even countries with publicly-run healthcare systems are scrutinizing spending. Devoting dollars to one area (say, hospitals) is effectively a decision not to spend on something else, (perhaps public health programs). All systems, be they public or private, allocate funds in ways to spend money in the most efficient way possible. Thoughtful decisions require a consideration of both benefits and costs.

One of the consistent positions put forward by contributors to this blog is that all health interventions should be evaluated based on the same evidence standard. From this perspective, there is no distinct basket of products and services which are labelled “alternative”, “complementary” or more recently “integrative”. There are only treatments and interventions which have been evaluated to be effective, and those that have not. The idea that these two categories should both be considered valid approaches is a testament to promoters of complementary and alternative medicine (CAM), who, unable to meet the scientific standard, have argued (largely successfully) for different standards and special consideration — be it product regulation (e.g., supplements) or practitioner regulation.

Yet promoters of CAM seek the imprimatur of legitimacy conferred by the tools of science. And in an environment of economic restraint in health spending, they further recognize that showing economic value of CAM is important. Consequently they use the tools of economics to argue a perspective, rather than answer a question. And that’s the case with a recent paper I noticed was being touted by alternative medicine practitioners. Entitled, Are complementary therapies and integrative care cost-effective? A systematic review of economic evaluations, it attempts to summarize economic evaluations conducted on CAM treatments. Why a systematic review? One of the more effective tools for evaluating health outcomes, a systematic review seeks to analyze all published (and unpublished) information on a focused question, using a standardized, transparent approach to evidence analysis. When done well, systematic reviews can sift through thousands of clinical trials to answer focused questions in ways that are less biased than cherry-picking individual studies. The Cochrane Review’s systematic reviews form one of the more respected sources of objective information (with somecaveats) on the efficacy of different health interventions. So there’s been interest in applying the techniques of systematic reviews to questions of economics, where both costs and effects must be measured. Economic evaluations at their core seek to measure the “bang for the buck” of different health interventions. The most accurate economic analyses are built into prospective clinical trials. These studies collect real-world costs and patient consequences, and then allow an accurate evaluation of value-for-money. These types of analyses are rare, however. Most economic evaluations involve modelling (a little to a lot) where health effects and related costs are estimated, to arrive at a calculation of value. Then there’s a discussion of whether that value calculation is “cost-effective”. It’s little wonder that many health professionals look suspiciously at economic analyses: the models are complicated and involve so many variables with subjective inputs that it can be difficult to sort out what the real effects are. Not surprisingly, most economic analyses suggest treatments are cost-effective. Before diving into the study, let’s consider the approach: