The VA scandal exposes the folly of metrics

The Veteran’s Administration is under fire for covering up deaths. Men and women who were eligible for care languished on impossibly long waiting lists and even worse when some died waiting for care their deaths were covered up. This is horrific and everyone wants to know how this tragedy could have happened?

Veteran’s hospitals have long waits in my experience because they are underfunded, many (if not all) patients with complex medical issues and often complex social ones, and have systems so complex that you need a lifetime of working there to navigate the system (that’s what happens when you breed the bureaucracy of the military with the bureaucracy of hospital administrators).

I spent a little time at the VA in Colorado and I could never understand the system of who was allowed to get what care or when or how. We had a clinic nurse who cared deeply (and was doing the job of three people) and she served as our universal VA translator. Think dealing with insurance companies is a challenge? The VA makes them look like red tape amateurs.

Goals are important in medicine, whether it’s access to care, choosing the right first-line antibiotic, or reducing bed sores. Without goals and data it’s hard to know if the changes you have implemented are helping your patients. There are many areas for improvement in medicine and metrics can help us see that. However, metrics also have a very dark side because not everyone is honest and some people may start out with the best of intentions but when flummoxed by a seemingly insurmountable challenge don’t always do the right thing.

With the VA fiasco a bonus for the senior staff was tied to the metric of timely care within 14 days. Tying rewards or penalties to metrics seems to encourage some people to think even more about the metric and less about the actual problem. To meet the metric of timely access VA administrators could:

overhaul the system

go public with how it was impossible to fix the system given the rule book and the money allowed

use sleight of hand to drop names from the waiting list

We can all agree that identifying a problem and setting a goal is important. You can’t change your antibiotic prescribing patterns if you don’t know where you are now and where you should be. However, carrots and sticks may not always be the right way to achieve the desired outcome.

Let’s take the urinary tract infection (UTI) example a little further and say that 40% of uncomplicated UTIs are getting the antibiotic ciprofloxacin. Ciprofloxacin is not a first line antibiotic so you want to reduce the prescribing rate to 5%. Ways to go about that include educating physicians, pharmacists, and patients in addition to tracking data, providing feedback and maybe individual problem solving for those physicians who just can’t stop giving ciprofloxacin inappropriately.

Now tie money to that outcome, do you think one (or more) physicians or administrators or pharmacists would be more or less likely to change one or two diagnosis from uncomplicated to complicated urinary tract infection thus satisfying the metric? I’m not saying this would even be intentional. Say Mrs. Smith is insisting that only ciprofloxacin “works for her,” yet you know from the test that nitrofurantoin will work and she has no contraindications to that drug.

Might one doctor somewhere when faced with an ever-growing delay in his/her day as the conversation with Mrs. Smith takes longer and longer convince him or herself that maybe Mrs. Smith actually has a complicated UTI and thus will fall out of the metric so ciprofloxacin is really OK? That would be the wrong decision for many reasons, but I can see it happening and not even driven by anything nefarious like money but rather desperation for the day, exasperation, and a desire to please Mrs. Smith (because pleasing the patient is actually metric too).

Whether it is on purpose or just a crappy day metrics are at risk of being fudged when people start to think more about numbers than the patients they represent. And adding money into the mix? Higher stakes may entice some to do the wrong thing. Some professional athletes dope because they want the gold medal. Some police departments downgrade rapes or don’t investigate them at all to make it look like they are meeting the crime rate metrics (22% of police departments that serve populations >100,000 have severe irregularities in their rape reporting statistics) as for many departments funding is metric driven. Metric madness is not a medicine-only phenomenon.

I really believe that most individuals want to do the right thing, but when faced with what may seem like insurmountable metrics (based on the actual money allotted or the nature of the problem), well, strange things can happen to data under bad leadership.

Metrics sadly teach some people to satisfy the metric, not solve the problem.

The VA scandal exposes the folly of metrics 15 comments

Comments are moderated before they are published. Please read the comment policy.

QQQ

Heads need to roll at the VA, there is no excuse for this level of
incompetence and the atrocious care, or lack of same , given our
veterans. Vets deserve the best care this country can give them. Instead
this admin. lets the VA give bonuses to those who have hidden/lied
about problems with care, Legionaires disease at VA hospitals- like in
Pittsburgh. This wretched care is the responsibility of those from the
top down and all should pay the price for their lying &
incompetence.

Eric Strong

The only reason everyone thinks their health care systems are better than the VA’s is because all of the problems that would result in a national headline if they happen at a VA fly completely under the radar when they happen anywhere else. Problems at the VA are easily politicized for quick and easy points against whatever the current administration is. These opinions about the VA, which are based more on politics rather than based on a sincere desire to provide better medical care to veterans, then get amplified by the media, and quickly transformed into talking points by Congressmen thinking about reelection. Anyone who was sincere about wanting to improve VA care would be focusing on how to recruit high-quality health care professionals to work there (since MD/NP/RN shortages in the VA are responsible for most of the problems) instead of complaining about needing to fire people and complaining about bonuses (most of which is spread among tens of thousands of hard working health care providers, and not to administrators)

I guarantee you that falsification of records, disease outbreaks, and incompetent care, all happen at your hospital too.

rtpinfla

I volunteer once a month at the VA and agree that there needs to be a thorough cleaning but it’s not as simple as lopping off a few heads. That whole system is rotten to the core and is only designed to defend the status quo. It will take even the most idealistic and qualified person on the planet and either1) beat them into submission 2) frustrate them to the point they leave the VA for a position where they can make a difference.
Targeted firings is a little like shoveling a pile of sand on the beach to stop the tide from coming in. The waves will just keep a coming. If you fire the administrators at the very top I promise you the local bureaucracies and fiefdoms that are entrenched in the system will thwart any and all attempts at altering the status quo. Firings and cleanup at the local level will also do nothing because the upper administration will ensure that the appearance high quality care metrics are maintained.
To be sure, there are a lot of caring and dedicated individuals working in the VA. I strongly suspect that a lot of the “bad” apples used to care but have just been beaten into the ground by the system.

EmilyAnon

What lack of money?

“Throughout the VA, nearly $400 million was paid in bonuses in fiscal 2011.”

For some reason, $400 million in bonuses is a far more palatable prospect for administrators to contemplate than the same $400 million in salaries for doctors, nurses and clerks. Go figure.

Eric Strong

Actually, the majority of bonuses given out within the VA system go to actual care providers, and not administrators.

Eric Strong

Tying reimbursement to an easily manipulated metric will necessarily lead to falsification of records and corruption. It happens at the VA; it happens everywhere. We shouldn’t be surprised when this is uncovered; we should be expecting it.

rtpinfla

Also, these metrics are employed by the administrators who only want the metrics to show how good their organization is doing. despite what they say, they say, they are not the least bit interested in identifying problems in the systems or processes that the data is based on. When the data reflects poorly, they point the finger at everyone except themselves. Or they look for ways to manipulate the data to ensure the numbers reflect a positive appearance. Most likely they will do both and then pay themselves a handsome bonus afterwards.

buzzkillerjsmith

You spent time at the VA?! I hope that was in residency, when you had no choice. If it was by your own choice, Dr. G., it bespeaks an insufficient appreciation of geometry and theology.

Eric Strong

The pervasiveness of that attitude is contributing to the problem. A primary root cause of the VA’s issues is insufficient staffing by health care professionals. For example, if VAs had enough doctors and nurses working in the clinics, they would not have needed to resort to cheating on the wait list metrics. VAs across the country have open, unfilled positions all the time. I can’t speak about nursing reimbursement, but reimbursement for internists is competitive with the community. If you really cared about improving veterans’ health, you would consider working for the VA yourself.

Eric Strong

I totally agree. Medical notes constructed within an EMR using templates are a sham. Not just the ROS and PE, but also imported med lists and past med history which are not confirmed with the patient, and imported labs and radiology reports which are not even reviewed. It’s not just bad medicine, but it feels like insurance fraud when an encounter level is supported by a note largely completed by computer algorithm.

Last year I had a very underwhelming encounter as a patient at a local, highly regarded (by reputation) urgent care clinic. The physician seeing me didn’t know my profession. Afterwards, I requested a copy of my H&P (more out of curiosity than anything else). The history and physical as documented did not remotely match what was actually done. (i.e. I would have failed a medical student who had completed the history and exam as actually performed.) I wrote an extensive and specific letter to the medical group complaining that what I saw was bad medicine and essentially fraud, and received a response as underwhelming as the original encounter.

SteveCaley

The demise of Soviet Management Theory was overstated. It just moved overseas.

Eric W Thompson

In the VA the leadership can be fired. Everyone else is in protected status and would have to steal money or assualt someone prior to disciplinary action being taken. You pretty much can’t fire rank and file civil service unless they want to be fired.

Eric Strong

Actually, VA physicians can be fired for conventional reasons. But you are right regarding clerical staff and mid and low level administrators. I don’t know about other care providers (e.g. RNs, NPs, PAs, etc…).

Easier termination of incompetent employees is one of a relatively small number of specific changes which the VA should undertake to get past its current low point.

Joe

Underfunded? There have been multiple reports of the VA health care sector carrying hundreds of millions of dollars of surplus for the last several years. This is not difficult when you don’t provide care and don’t pay your bills when other providers do your job for you.
But yes, I agree with the point about metrics and that they have very little to do with whether or not there is good or efficient patient care going on.