Abstract

The assessment of medical practice is evolving rapidly in the United States. An initial focus on structure and process performance measures assessing the quality of medical care is now being supplemented with efficiency measures to quantify the “value” of healthcare delivery. This statement, building on prior work that articulated standards for publicly reported outcomes measures, identifies preferred attributes for measures used to assess efficiency in the allocation of healthcare resources. The attributes identified in this document combined with the previously published standards are intended to serve as criteria for assessing the suitability of efficiency measures for public reporting. This statement identifies the following attributes to be considered for publicly reported efficiency measures: integration of the quality and cost; valid cost measurement and analysis; minimal incentive to provide poor quality care; and proper attribution of the measure. The attributes described in this statement are relevant to a wide range of efforts to profile the efficiency of various healthcare providers, including hospitals, healthcare systems, managed-care organizations, physicians, group practices, and others that deliver coordinated care.

The assessment of medical practice is evolving rapidly in the United States. An initial focus on structure and process performance measures to assess the quality of medical care is now being supplemented with efficiency measures to quantify the value of healthcare delivery. To address the need for standards to govern these efforts, the American Heart Association convened an interdisciplinary expert writing group to identify the essential attributes of measures intended for public reporting of efficiency associated with the delivery of care for patients with cardiovascular disease or stroke. These attributes provide a framework for the development and assessment of efficiency measures.

To date, enthusiasm for measuring and improving efficiency is not matched by a consensus regarding the essential attributes of measures that emphasize efficiency in the allocation of healthcare resources. Reflecting the lack of a common definition of efficiency, current efforts vary from simply measuring and publicly reporting costs, which are not measures of efficiency, to more sophisticated measures that combine aspects of cost and quality. For the purpose of this statement, we focus on measures or combinations of measures that are specifically intended to convey information about the efficient use of healthcare resources, incorporating considerations of both the cost and quality of patient care. The concept of efficiency is distinct from cost alone in that efficiency takes into consideration the results produced, as well as the resources used. Cost implies only consideration of resources expended. Quality includes consideration of what is produced in the way of care or outcomes. We therefore assume that efficiency measures for public reporting should convey a balance of information about resource use and clinical performance or results that extends beyond a narrow focus on cost alone. The idea is to convey information about areas in which low cost and high quality are achieved and to provide a more complete perspective on performance in cost as it relates to quality.

The present statement, which builds on prior work that articulated standards for publicly reported outcomes measures,1 identifies preferred attributes for measures used to assess efficiency in the allocation of healthcare resources. Each of these attributes is, by design, under the control of the measure developers. The attributes identified in the present document, combined with the previously published standards, are intended to guide measure development and serve as criteria for assessing the suitability of efficiency measures for public reporting. The authors of the present document include individuals with expertise in clinical cardiology, quality of care, economics, outcomes research, statistics, health services research, epidemiology, healthcare policy, clinical performance measurement, and public reporting.

Background

Cardiovascular disease and stroke are central to considerations of the efficiency of care because they account for disproportionate disease-related morbidity, mortality, and cost.2 Cardiovascular services accounted for the greatest share of the total change in healthcare spending from 1987 to 2002, far exceeding mental disorders, the next largest diagnostic group.3 The rise in cost per treated heart disease case accounted for ≈70% of the rise in overall medical care spending between 1987 and 2002.

In recognition of the lack of standardized approaches, several organizations, including the National Quality Forum (NQF) and the AQA (formerly Ambulatory Care Quality) Alliance, are attempting to develop working definitions of efficiency in health care so that efficiency at the level of the clinician or institution can be measured objectively.4,5 The AQA Alliance agenda includes 6 priority conditions, including 3 cardiovascular conditions (acute myocardial infarction, congestive heart failure, and coronary artery disease). Furthermore, quality consortia have extended their scope of work to cover measures that include some consideration of cost. For example, the Leapfrog Hospital Insights program, which has been adopted by hospitals throughout the United States, included 2 measures that are related to cost (average length of stay and readmissions).6

Federal and state programs are also developing various approaches to address both cost and quality considerations in measures initiatives. In August 2006, President Bush issued an executive order to “Promote Quality and Efficient Health Care in Federal Programs” that called for greater measurement and new models of reimbursement that reward performance.7 The Centers for Medicare & Medicaid Services (CMS) has stated its goal of improving and reducing variation in efficiency.8 CMS has developed an NQF-approved measure of readmission after hospitalization for heart failure.9 The Medicare Payment Advisory Commission’s June 2007 report to Congress, Promoting Greater Efficiency in Medicare, contains several recommendations regarding efficiency, including measuring hospital readmissions and rewarding hospitals with low readmission rates.10 Readmission rates could be considered to reflect efficiency because a readmission may represent a preventable adverse outcome that is due to poor-quality care and is associated with higher healthcare cost. States have initiated programs that are purported to measure and report on the efficiency of care, although efforts to date are heterogeneous and focus predominantly on cost. At least 30 states have pending or enacted legislation that requires disclosure, transparency, or publication of hospital and healthcare charges and fees.11 Several states are reporting measures of resource use such as average length of stay or insurer payments in conjunction with measures of clinical quality.12,13

Concurrent with these public efforts, private sector initiatives have proliferated under the direction of insurers and consortia of purchasers. Many insurers are measuring the cost of procedures, and admissions at different hospitals and by different providers, and publishing this information online for plan enrollees to view.14–17 To date, there is wide variation in the content and attributes of these measures, some of which are cost-only measures and therefore not true efficiency measures. The marked variability in development of measures from multiple sectors further reinforces the need to define the attributes of efficiency measures regarding suitability for public reporting.

Attributes of Efficiency Measures

We introduce the following domains of attributes to be considered for publicly reported efficiency measures (Table 1): Integration of quality and cost; valid cost measurement and analysis; minimal incentive to provide poor-quality care; and proper attribution of the measure. The attributes described in the present statement are relevant to a wide range of efforts to profile the efficiency of various healthcare providers, including hospitals, healthcare systems, managed-care organizations, physicians, group practices, and others who deliver coordinated care. Ideally, all efficiency measures would, at a minimum, possess these attributes for them to be suitable for public reporting. The authors recognize that measures may conform with several of these attributes to a varying degree, and judgment will need to be applied to determine whether they are suitable efficiency measures. However, measures that only include cost (without any pairing with what is produced), use invalid cost measures, provide incentives for poor-quality care, or cannot be attributed properly should not be considered for public reporting. For all measures, the degree of compliance with each of these attributes should be clear.

Table 1. Standards for Measures Used for Public Reporting of Efficiency in Health Care

These attributes are intended to be considered in addition to those that were published previously for the public reporting of outcomes measures,1 which include the following: 1) the clear and explicit definition of an appropriate patient sample; 2) clinical coherence of the variables used in statistical models; 3) sufficiently high-quality and timely data; 4) designation of an appropriate reference time before which covariates are derived and after which outcomes are measured; 5) use of an appropriate outcome and a standardized period of outcome assessment; 6) application of an analytical approach that accounts for the multilevel structure of data; and 7) disclosure of the methods used to compare outcomes, including disclosure of performance of the risk-adjustment methodology in derivation and validation samples. An underlying assumption is that measures include risk adjustment or a suitable explanation for why it is not necessary.

Integration of Quality and Cost

Any measure of efficiency that is suitable for public reporting should explicitly include consideration of cost/resource use and clinical outputs (eg, quality of care). The incorporation of cost/resource use is fundamental to efficiency measures. Focusing solely on resource considerations, however, fails to account for the consequences; in some cases, providers may achieve efficiency by offering comparably excellent care at lower cost, but in others, lower cost may be associated with underuse of key strategies or misallocation of inputs and results in notably worse outcomes. Only by combining measures of costs and outcomes is it possible to determine the return on, or value of, the healthcare investment. Outcomes in this sense can be understood as the experience of patients in their interaction with the healthcare system and can include clinical events, satisfaction, and intermediate events. Some measures may include outcomes implicitly, such as those that measure practices that increase cost and diminish outcomes (eg, the use of inappropriate procedures that lead to risks that outweigh benefits) or decrease cost and improve outcomes (eg, certain disease management programs for patients with heart failure), but situations in which there are potential tradeoffs between cost and quality require explicit consideration of both aspects of care.

A measure of cost or resource use alone provides only a limited perspective on the true economic performance of the system and may produce a misleading result. For example, measuring the crude average cost of a percutaneous coronary intervention provides no information about whether the system is efficient. Procedures at one institution may cost less than benchmarks for those procedures, but they may be performed for the patients least likely to benefit from them. For example, it is possible that an institution that is treating many low-risk, asymptomatic patients who would receive little to no clinical benefit from the procedure would have a lower average cost than another institution that reserves the procedure for higher-risk, symptomatic patients who would be expected to derive greater benefit. Even the use of risk adjustment in this hypothetical example would not account for the fact that many of the patients in the lower-cost institution should not have had the procedure performed. For these patients, the yield from the cost investment is minimal or nonexistent. Inclusion of the cost with the benefit from the patient perspective will better provide a measure that can be used to improve healthcare delivery. Thus, measures of costs or resource use alone, although commonly referred to as measures of efficiency, are not appropriate individual measures because they do not incorporate the perspective of what is produced for the cost or whether the costs are directed in the proper way.

Sometimes, surrogates for costs are used, but if used alone, they can also be misleading. For example, length of stay is more analogous to a cost measure than an efficiency measure. Similarly, cost per admission is not an efficiency measure because there is no measure of the quality or outcome associated with the admission or its appropriateness. Although the cost per admission may be a useful measure in concert with other measures of what occurs within the admission and afterward, in isolation it does not convey efficiency without an implicit or explicit judgment about the quality of the admission.

The combined consideration of costs and quality also facilitates the identification of waste, those areas in which higher costs do not produce meaningful increases in quality or better outcomes (or may even worsen care and outcomes). These areas are the best targets for improving the efficiency of the healthcare system, because their elimination saves money and may improve care and outcomes. The goals in this case are not to minimize costs for wasteful activities but to avoid them. For example, the American College of Cardiology Foundation is publishing appropriateness criteria for many cardiovascular diagnostic tests, including nuclear imaging, echocardiography, and computed tomography/magnetic resonance imaging, identifying situations for which there was consensus that imaging did not benefit the patient.18–20 A measure that reports rates of inappropriate imaging within practices would contain information regarding both cost and quality, because an inappropriate test results in both higher costs and poorer-quality care. Conversely, a reduction in this rate would simultaneously improve quality and decrease cost. Improvements in this metric should improve the efficiency of the system. Given the importance of this issue, measures of efficiency should clearly state how the dimension of quality relates to cost. Moreover, the method of assessment of quality should be clear, and the implications for providing higher-quality care for a higher cost should be transparent.

Valid Cost Measurement and Analysis

Costs and quality should ideally be assessed concurrently and with valid methods in the same population to provide an accurate assessment of efficiency. Prior work has detailed the characteristics of quality measures.21 Several specific issues are important for the cost/resource assessment and are highlighted in this section. The intent is not to define criteria as much as to make clear what information needs to be highlighted and justified.

Fundamental to cost assessment is a clear description of the case identification and the episode of care that is included. The perspective of the cost/resource assessment also needs to be described explicitly. What could be conceived as efficient from the perspective of a hospital that derives additional revenue from each admission may be extremely inefficient from the perspective of a payer or society. The realities of our payment system force entities to respond in ways that are not always perfectly aligned with societal needs, but publicly reported measures should reflect the interest of society or disclose clearly the perspective they represent.

Efficiency measures should include a clear statement of the time horizon, with an acknowledgment of any limitations that result from the time horizon chosen. In the usual course of measurement, it will not be possible to provide the time horizon necessary to determine with certainty whether a larger short-term investment leads to a better long-term outcome, even if the short-term outcomes are worse or neutral. For example, bypass surgery provides upfront costs and risks that may be offset by longer-term gains. A longer length of stay for patients with heart failure could decrease readmissions or reduce the need for skilled nursing facilities. Thus, the time horizon needs to be clear and justified to the extent possible. The cost assumptions made with the time horizon used should be clear (eg, discounting).

Efficiency measures commonly include resources used in the form of direct healthcare costs that ignore indirect costs. The quantification of costs in efficiency measures presents particular challenges. First, the methods of tabulating costs and resources should be clear, reproducible, and appropriate. If resource utilization is translated into costs, then the approach should be stated clearly. If discounting of future costs is included, then the approach should be stated and justified. Charges (which are list prices) should be avoided because they usually do not reflect costs or expenditures accurately. Similarly, payer costs, which often reflect variable negotiated rates with providers, should be avoided because they are not standardized across payers. All calculations should be stated explicitly.

The scope of the costs and resources should be stated and justified. Considerable attention has been paid to this issue in the cost-effectiveness literature, and the recommendations from that literature should be followed if the form of the measure lends itself to that approach by explicitly measuring costs and outputs.22 An optimal measure of this type would take a comprehensive approach to measuring cost. Failure to do so could result in misleading conclusions. For example, a narrow spectrum of measurement can result in calculations that miss important costs that are just beyond the boundary of the measure. Indeed, Hsu and others23 revealed that drug cost sharing can substantially lower drug costs but with an increase in the risk of an emergency department visit, hospital admission, and death. The net increase in total medical costs was ≈1%. Thus, a measure that concerned itself solely with drug costs would ignore the effect of that policy on patient outcomes and costs in other areas. In another example, Kosiborod and others,24 reporting on trends in the care of hospitalized patients with heart failure in the 1990s, found that dramatic decreases in length of stay were associated with marked increases in readmission rates and discharge to skilled nursing facilities, which could offset the cost savings from a shorter length of stay. A focus solely on length of stay would ignore the overall impact on resource consumption.

Risk adjustment is a critical component of cost estimates, as it is with outcomes measurement, and should account for differences in case mix across units of comparison. Methods of risk adjustment need to be appropriate given the sample size and unit of observation and may include hierarchical modeling and attention to skewed distributions of outcomes and costs. The statistical precision of estimates must also be reported. Although appropriate analysis of the data was highlighted in the prior report on outcomes measures,1 there are additional considerations in models that use costs. Again, the lessons from the cost-effectiveness and econometrics literature are useful.

No or Minimal Incentive to Provide Poor-Quality Care

Any measure of efficiency should have no or only a minimal incentive to provide poor-quality care; that is, the measure should not have a perverse incentive such that excelling in the measure could, from a more comprehensive viewpoint, adversely affect patients and the healthcare system. The overall intent of most efficiency measures should be to provide an opportunity to improve the value of healthcare delivery by eliminating waste and possibly reducing costs where they are not justified on the basis of gains in the quality and appropriateness of care. If costs alone are targeted, then reductions in cost may be associated with worse quality of care delivery and adverse patient outcomes. For example, as described above, a measure that captures only length of stay for a heart failure admission could lead to practices of premature discharge from the hospital, which is subsequently associated with a higher rate of readmission. Such a result could, from a broader perspective, worsen care and increase cost. The incentive to do well on this measure is not necessarily aligned with the best interests of patients or the healthcare system. This measure needs to be combined with other measures to avoid this problem.

Thus, efficiency measures should define how high performance on the measure contributes to insights that will lead to improvements in healthcare delivery, with particular attention to the effect on individual patients and society. If performing well on the measure could have a markedly negative effect on patient outcomes or provide incentives that are counter to high-quality care, then the efficiency measures could perversely affect patients and society, because they push expenditures away from areas in which they produce economically attractive results. Thus, it is crucial for an efficiency measure to be developed with explicit attention given to how improvements in what is measured would reflect improvement in healthcare delivery from a societal perspective.

One approach to minimizing unintended consequences is to develop a battery of complementary measures of cost and quality so that the tradeoffs are explicit, as in a balanced scorecard.25–28 In the example above, it may be useful to include length of stay after a heart failure hospitalization if that measure is combined with other measures of patient outcomes, such that it would be clear whether a reduction in length of stay worsened patient outcomes. In this approach, high performance could be achieved only if length of stay were reduced and patient outcomes did not change or even improved. It might even be better to include a more explicit measure of cost of the hospitalization and the outpatient transition along with patient outcomes. In this case, investment in disease management and the transition to outpatient status would be captured, as would the result of that investment.

The avoidance of perverse incentives does not necessarily mean that measures cannot discourage certain investments, even if they reduce patient outcomes. There are limits to what can be spent to promote patient outcomes. There may be some interventions that are so expensive that they make the health benefit economically unattractive for the healthcare system. At the margin, a measure could provide an incentive to eliminate such a high-cost investment, with the consequence of some diminution of health benefit. It is necessary to see the larger picture to determine whether resources are allocated wisely to produce high-quality care. The point is neither to maximize quality regardless of cost nor to minimize costs regardless of the effect on quality. Rather, the aim is to provide information that allows for the evaluation of any tradeoffs between cost and quality. At some point, costs for a strategy may be so high that they do not justify the better outcomes, or cost savings may be so poorly placed that they cause an unreasonable diminution in patient outcomes. Without both measures, these assessments cannot be made.

Because potential adverse effects may not be anticipated, an explicit plan for evaluation of the impact of efficiency measures should be used, particularly with regard to measuring impact on patients (including their health status), systems of care, and society. Although it may not be possible to demonstrate convincingly that a proposed efficiency measure will not have an adverse effect on patients, it is nevertheless important to address potential unintended consequences. For example, there is still debate about whether the publication of cardiac surgery report cards led to surgeons avoiding operating on higher-risk patients.29,30 To mitigate concerns that an efficiency measure might lead to decreased costs at the expense of decreased quality, one could also present a direct measure of quality, such as patients’ survival and health status for a discrete follow-up period before and after implementation of the publicly reported measure. In any case, such a possibility should be considered in the development and assessment of the measure.

The issue of tradeoffs is not present for all practices in medicine, and at this stage, these measures might best be directed where there are opportunities to decrease cost without an adverse effect on quality and outcomes, or even with an opportunity to improve care. An example of a measure with minimal potential adverse effects would be an assessment of the number of tests performed that have been deemed inappropriate by an American College of Cardiology Foundation expert consensus panel. These panels have addressed the appropriateness of cardiovascular tests and procedures and identified indications of appropriateness.18–20 These indications were developed to identify groups of patients who would not benefit from these tests or procedures. Elimination of these expenditures might even enhance patient outcomes.

No measure would be expected to be free from the possibility of an adverse consequence. In addition, we cannot protect against a few unscrupulous individuals putting their performance on the measure above the interests of patients. Nevertheless, what is important is that this issue be considered and that the measures not strongly encourage such practices.

Proper Attribution of the Measure

Proper attribution, which is relevant to all performance measures, involves correctly assigning the measure to the individual, group, or organization responsible for the decisions, costs, and outcomes. Attribution is often difficult when patients see many practitioners, particularly when an “episode of care” is used to incorporate costs accumulated over a specified time horizon and, in the case of efficiency measures, when those responsible for costs may not completely overlap with those responsible for quality. The assignment of responsibility to any single person may be difficult even in cases in which there is a single leader of the effort, such as may occur with surgery. The surgeon may perform the procedure, but the costs and outcomes are still related to the sum of efforts by many individuals who care for the patient and are influenced by the system in which the care takes place. In the case of 30-day readmission rates for patients discharged with heart failure, the attribution to hospitals may be disputed because the time horizon crosses inpatient and outpatient settings. For costs from a payer’s perspective, even when practitioners make decisions that affect costs, the attribution may be shared with others, because the price of the resources may be negotiated by others and outside their control.

The fact that no single provider was acting alone should not deter measurement, but the procedure for attributing the measure should be stated clearly and justified. The complexities of attribution are best acknowledged explicitly, because it will be rare for costs and outcomes to be easily attributed to a single person, entity, or even healthcare system.

Moreover, the shared nature of attribution of care provides an opportunity for measures to be understood as providing an incentive for increased collaboration across the provision of health care. From the patient perspective, the responsibility for care resides with all the people and institutions that were part of that care. This collective attribution might lead to greater cooperation in the service of the patient and healthcare system, although it is not known whether this result can be achieved in practice.

Conclusions and Recommendations

The implementation of efficiency measures to induce more appropriate and judicious use of healthcare resources is important and challenging. Changing practice to benefit patients and society will require rigorous measures that capture opportunities to improve the efficiency and value of health care. These measures will reflect the resources used and the results obtained.

To protect the interests of patients while illuminating the efficiency of care, it is critically important to create measures that will improve the healthcare system. Measures that reliably expose inefficient care may facilitate internal efficiency improvement and drive patients to more efficient providers. Costly care is not necessarily inefficient care, and confusing the two may lead to a worsening of clinical care and a waste of resources. For example, an emphasis on restraining costs without consideration of the consequences could markedly undermine health outcomes and lead to higher future costs.

With the present statement, we suggest 4 domains that should be considered in any effort to measure and publicly report efficiency. Measures that comply with these standards will be a first step that requires the accompaniment of system changes that encourage practices that are fiscally responsible and oriented toward the best interests of patients and society, with the recognition that these interests are not always aligned. Table 2 shows how some sample measures may be assessed by each of these domains. The measures that are most aligned with these standards are those that target practices that raise costs and have an adverse or neutral effect on outcomes, as seen in the example about inappropriate imaging. Those measures in areas where cost can be constrained, but at the risk of adversely affecting patient outcomes, require explicit examination in the public domain with a clear view of the tradeoffs that may be made in the spirit of more efficient healthcare delivery. These are cases in which the interests of payers, patients, and society can diverge or in which unintended consequences can cause inadvertent harm.

Table 2. Examples of Hospital-Based Measures and Their Properties According to the Standards

In sum, the present document provides a framework for the development and evaluation of efficiency measures. This statement does not propose measures or set firm criteria that measures must pass but identifies domains that need to be addressed as measures are proposed and implemented. In all of these efforts, it will be important to conduct evaluations to assess the impact of implementation and to document the result from the perspective of patients and society.

Footnotes

The American Heart Association and the American College of Cardiology Foundation make every effort to avoid any actual or potential conflicts of interest that may arise as a result of an outside relationship or a personal, professional, or business interest of a member of the writing panel. Specifically, all members of the writing group are required to complete and submit a Disclosure Questionnaire showing all such relationships that might be perceived as real or potential conflicts of interest.

This statement was approved by the American Heart Association Science Advisory and Coordinating Committee on May 30, 2008, and by the American College of Cardiology Foundation Board of Trustees on June 27, 2008.

The American Heart Association requests that this document be cited as follows: Krumholz HM, Keenan PS, Brush JE Jr, Bufalino VJ, Chernew ME, Epstein AJ, Heidenreich PA, Ho V, Masoudi FA, Matchar DB, Normand SLT, Rumsfeld JS, Schuur JD, Smith SC Jr, Spertus JA, Walsh MN. Standards for measures used for public reporting of efficiency in health care: a scientific statement from the American Heart Association Interdisciplinary Council on Quality of Care and Outcomes Research and the American College of Cardiology Foundation. Circulation. 2008;118:1885–1893.

This article is copublished in the Journal of the American College of Cardiology.

Copies: This document is available on the World Wide Web sites of the American Heart Association (my.americanheart.org) and the American College of Cardiology (www.acc.org). A copy of the document is also available at http://www.americanheart.org/presenter.jhtml?identifier=3003999 by selecting either the “topic list” link or the “chronological list” link (No. LS-1823). To purchase additional reprints, call 843-216-2533 or e-mail [email protected]

Expert peer review of AHA Scientific Statements is conducted at the AHA National Center. For more on AHA statements and guidelines development, visit http://www.americanheart.org/presenter.jhtml?identifier=3023366.

Permissions: Multiple copies, modification, alteration, enhancement, and/or distribution of this document are not permitted without the express permission of the American Heart Association. Instructions for obtaining permission are located at http://www.americanheart.org/presenter.jhtml? identifier=4431. A link to the “Permission Request Form” appears on the right side of the page.

National Conference of State Legislatures. State legislation relating to disclosure of health and hospital charges. January 2008. Available at: http://www.ncsl.org/programs/health/Transparency.htm. Accessed August 25, 2008.

Wellpoint. Wellpoint announces new suite of online tools to support consumer-driven health: expands current portfolio of Subimo offerings, which includes the Treatment Cost Advisor ™, to WellPoint members [press release]. Indianapolis, Ind: WellPoint; September 21, 2005. Available at: http://www.redorbit.com/news/health/247299/wellpoint_announces_new_suite_of_online_tools_to_support_consumerdriven/. Accessed August 25, 2008.

Spertus JA, Eagle KA, Krumholz HM, Mitchell KR, Normand SL; American College of Cardiology; American Heart Association Task Force on Performance Measures. American College of Cardiology and American Heart Association methodology for the selection and creation of performance measures for quantifying the quality of cardiovascular care. Circulation. 2005; 111: 1703–1712.