Quality Measures and the Individual Physician

Danielle Ofri, M.D., Ph.D.

The quarterly “report card” sits on my desk. Only 33% of
my patients with diabetes have
glycated hemoglobin levels that
are at goal. Only 44% have cholesterol
levels at goal. A measly
26% have blood pressure at goal.
All my grades are well below my
institution’s targets.

It’s hard not to feel like a failure when the numbers are so
abysmal. We’ve been getting these
reports for more than 2 years
now, and my numbers never budge.
It’s wholly dispiriting.

When I voice concern about the reports, I’m told that these
are simply data, not criticisms,
and that any feedback of data to
doctors is helpful. On the face of
it, this seems logical. How can additional
information be anything
but helpful?

It’s easy, of course, to find
scientific reasons why the data
are less clinically meaningful
than they seem. Success and failure
in these measures tend to be
presented as a binary function,
although clinical risk is almost
always a variable function. My patients
whose blood pressure is
140/85 (quite near the 130/80 goal)
are counted as failures equivalent to patients with a blood pressure of 210/110, even though their risks for adverse cardiovascular outcomes are vastly different.

And although these quality
measures focus on diabetes in
pristine isolation, my patients inconveniently
carry at least five other
diagnoses and routinely have
medication lists in the double
digits. Practicing clinicians know
from experience that micromanagement
of one condition frequently
leads to fallout in another.1,2

What happens when my patients read these data? I wouldn’t blame them if they concluded that I’m a lousy doctor and switched to another physician who manages to get glycated hemoglobin levels at goal for 38% of her patients with diabetes.

The quarterly report card stokes a perennial fear: maybe I really am a substandard doctor, and these statistics simply shed light on what I’ve refused to accept. If I’m doing my patients a disservice, then I’m morally obliged to vacate my office to make room for a more competent practitioner.

I appreciate the efforts and good intentions behind the report cards, and I’m certainly not saying that we shouldn’t have any data at all. But I think we need good evidence that the data measure true quality and that providing data is actually helpful. For individual doctors — as opposed to institutions or countries or populations — the evidence is not convincing.3,4 The possible mandatory use of these quality measures for reimbursement raises a host of other concerns.

If the goal of providing reports to individual physicians is to help them improve their care, it’s critical to understand the baseline assumption about doctors’ performance. Are most doctors doing a reasonable job? If so, then our analytics should aim to weed out the few who are inept. Or are most doctors mediocre, with shoddy clinical skills that put patients at risk? If so, then our data-driven system must prod doctors as a group to up their game.

There isn’t a simple formula for distinguishing good doctors from second-rate ones, nor will there ever be. At least some evidence suggests that when doctors deviate from quality measures, they nearly always have medically valid reasons for doing so.5 I think we should be willing to consider the larger gestalt of medicine, rather than just the minutiae that fit more expediently into a spreadsheet.

Who are the people who choose to enter medicine, and what are their motivations and character? I have yet to meet a medical student, intern, nurse, or doctor who doesn’t feel a powerful sense of professional responsibility. Not every single one is lining up for a Nobel Prize, but overall it is a smart and dedicated group. If Winnicott were selecting a “good enough” cohort for the medical profession, this would be it. I think society accepts that the overwhelming majority of health care workers are in the profession to help patients and are doing a decent job.

Quantitative analysts will chafe at this line of reasoning. They will say that doctors are afraid of being judged on the basis of hard data. They will see it as a sign of medical arrogance that physicians insist that everyone simply trust us to do the right thing because we are such smart and noble people.

I’ve always wanted to ask these analysts how they choose a physician for their sick child or ailing parent. Do they go online and look up doctors’ glycated hemoglobin stats? Do they consult a magazine’s Best Doctor listing? Or do they ask friends and family to recommend a doctor they trust? That trust relies on a host of variables — experience, judgment, thoughtfulness, ethics, intelligence, diligence, compassion, perspective — that are entirely lost in current quality measures. These difficult-to measure traits generally turn out to be the critical components in patient care.

I certainly want to know how my hospital is doing with an overwhelming disease like diabetes. The data do offer a snapshot of the clinical complexities of the disease, the challenges posed by our patients’ cases, and the limits of how much we can alter a disease that is affected by so many variables. And they could highlight fixable systemic impediments to good care.

But pinning the data on individual doctors is different. It purports to make a statement about comparative quality whose objectivity is a fallacy. When it weeds out the rare incompetent, it’s fine. But by and large, it serves only to demoralize doctors.

It offers patients a seductively scientific metric of doctors’ performance — but can easily lead them astray. Relying on these data is like trying to choose which car to purchase, armed with a metallurgic analysis of one square inch of the left rear fender of each car. The numbers are accurate, but they don’t tell you which car will run the best.

We all want our patients to achieve the best health possible, but most doctors don’t actually have control over the challenges of a complicated disease like diabetes — which is probably why my numbers haven’t budged in 2 years.

Sure, I can imagine a few changes that would no doubt improve my patients’ medical care: an hour-long visit instead of 15 minutes, weekly individual nutrition counseling, personal exercise trainers, glucose test strips that are covered by insurance, and medications that don’t cause diarrhea, heart failure, weight gain, or hypoglycemia. But report cards with my stats? So far they haven’t made me a better doctor. They just make me feel like a nihilist, bitterly watching primary care medicine grind down so many of its practitioners.

Doctors who actually practice medicine — as opposed to those who develop many of these benchmarks — know that these statistics cannot possibly capture the totality of what it means to take good care of your patients. They merely measure what is easy to measure.

We teach students and residents that tests that don’t alter clinical management can be harmful and should not be ordered. Regrettably, that is essentially what I’ve concluded about report cards for individual doctors. I don’t even bother checking the results anymore. I just quietly push the reports under my pile of unread journals, phone messages, insurance forms, and prior authorizations. It’s too disheartening, and it chips away at whatever is left of my morale. Besides, there are already five charts in my box — real patients waiting to be seen — and I need my energy for them.

Disclosure forms provided by the author are available with the full text of this article at NEJM.org.

From New York University School of Medicine and Bellevue Hospital, New York.

16-18 May 20148th NatCAM 2014
The 8th NatCAM 2014 from 16 to 18 May 2014 was organized by the Addition Medicine Association of Malaysia and co-hosted by the FPMPAM at the Pullman Hotel, Kuala Lumpur. Continue >>.

8 May 2014Citizen’s Action in Response to Emergencies (CARE) Interview with RTM
On 8 May, Dr Steven Chow and CARE trainers from our partner St John’s Ambulance, Malaysia (SJAM) presented to RTM on the importance of AEDs for saving lives in the event of a cardiac arrest. Continue >>