Measuring low-value care

By Tim Badgery-Parker - winner of the Best Health Services and Policy Research Paper by a PhD Student

As patients, we expect the medical care and surgical procedures we receive will benefit us – for example by improving our health or mobility, reducing our pain, or lowering our risk of death. Although we might acknowledge that sometimes doctors make mistakes, we expect the treatments themselves work. But many treatments in use have no evidence of benefit. Other treatments are in use despite evidence that they provide no expected benefit, or even that they more likely harm some patients.

In recent years, awareness of such “low-value care” has increased. One notable initiative is Choosing Wisely, launched in the United States in 2012 by the American Board of Internal Medicine. The stated aim of Choosing Wisely is to “advance a … dialogue on avoiding unnecessary medical tests, treatments and procedures”. The initiative is clinician-led through specialist societies. Each participating society produces a ‘top 5’ list of services to question. In early 2018, Choosing Wisely had spread to 18 countries and claimed 490 recommendations in the US. Currently, Choosing Wisely Australia publishes ‘top 5’ lists from 34 organisations on its website. The Royal Australasian College of Physicians has a similar EVOLVE initiative, with 22 published lists (which are also included in the Choosing Wisely Australia lists).

These lists, and similar recommendations such as the UK National Centre for Health and Care Excellence (NICE) ‘do not do’ recommendations, present a useful resource for investigating low-value care.

My research involves developing measures of low-value care, defined according to Choosing Wisely, NICE ‘do not do’, and other similar recommendations. With these measures, I can explore how much care is low value and the associated financial costs to the health system, examine trends, and look for hospitals with high or low rates of low-value care. This work was a partnership between the Capital Markets CRC, the University of Sydney, and the New South Wales Ministry of Health. The partnership with the Ministry defined the scope of the project as inpatient care in NSW public hospitals.

Another PhD candidate, Kelsey Chalmers, and I reviewed 824 Choosing Wisely recommendations, found 541 relating to care that can occur in the Australian inpatient setting, and published measures for 18 recommendations in BMC Research Notes.

In work published in BMJ Quality & Safety, I used 27 measures based on the NSW hospital admissions data. In financial year 2016-17, these 27 procedures were used in 49,169 episodes across NSW public hospitals, and I identified 5079 (11.00%) to 8855 (19.18%) of these as low value – used in circumstances when Choosing Wisely or other recommendation advised not to use the procedure. These low-value episodes had associated total hospital costs of $49.5 to $99.3 million – funds that could have been spent on better value care.

This is likely only a small fraction of the low-value care in this one setting. From almost 2000 recommendations published by Choosing Wisely, NICE, and other sources, we identified 625 relevant to the inpatient setting, but we have only 27 measures. This is primarily due to the available data. Many recommendations relate to pathology testing, radiological investigations, and prescribing, and these services are not included in the hospital admissions data I used. In other cases, while the specific procedure is recorded, the clinical details to determine if it was low value are not.

This is the major challenge in measuring low-value care. Very few services are low value in every patient. Very few are high value in every patient. Most tests and treatments benefit some patients and do not benefit (or may harm) others. The challenge in my research is to correctly assess whether the treatment was low value, using the information in the hospital claim.

A frequently expressed fear of this work is that we might misclassify appropriate care as low value, and that clinicians and hospitals might be unfairly penalised if the indicators become used for performance monitoring and funding adjustment. We addressed this in several ways. First, we only developed an indicator if we (including clinical advisers) were reasonably confident that the indicator could distinguish low-value care. This is one reason we have only 27 indicators from 625 recommendations.

We also developed two versions of most indicators. Having two definitions explicitly addresses the uncertainty in identifying true low-value care. The differences in results illustrate how changing the definitions changes the measured rate of low-value care, and they indicate the magnitude of change that could be expected if the measures were replaced by completely accurate measures.

In addition, we classified the low-value rates for half of the procedures as “negligible”. Fourteen of the indicators accounted for 98% of the low-value episodes in 2016-17. The other 13 had numbers too small for analysis. The small numbers could also mean that these procedures are only used in the rare cases where they are appropriate. For example, funding for vertebroplasty has been removed from Medicare because of evidence it lacks benefit. However, NICE guidelines state it is appropriate in certain, highly selected patients. We found only 14 vertebroplasties, consistent with use in certain, highly selected patients.

Rates of low-value care varied considerably between hospitals. For example, about 26% of knee arthroscopies were low value, but at individual hospitals the low-value rate ranged from 0% to about 70%. While other procedures did not show such wide ranges, in all cases some hospitals had rates much higher than the state rate, and other hospitals had rates much lower. This variation suggests low-value care can be reduced. Hospitals with low rates may have valuable information on how they achieved those rates. Hospitals with high rates need to consider the reasons for this and implement initiatives to reduce their use of low-value care.