Thursday, April 09, 2009

Leapfrog’s performance scores are meaningless

On the Internet, hospital performance on the Safe Practices Survey is ranked by quartiles, which may suggest to consumers that hospitals in higher quartiles are safer than hospitals in lower quartiles. In this first study of the relationship between survey scores and hospital outcomes, we studied a national sample of hospitals and found no relationship between quartiles of score and in-hospital mortality, regardless of whether we adjusted for expected mortality risk and certain hospital characteristics.

Public reporting has hospitals scrambling to do all sorts of things to buff their profiles. Unfortunately these “improvements” are largely cosmetic and have little beneficial impact on patient outcomes. That’s just one reason the performance movement is a failure.

1 comment:

Too bad you choose the right word: "buff." Scores PROVE that you're doing the right thing, and so it's not your fault if everything goes to hell. The notion of a "score" assumes that you can identify a process and that this process satisfies certain conditions of explicit analyticity. E.g., we know what we're talking about, we know where we started out, we know where we're going, everything we don't measure is an "externality" and therefore somebody else's problem. This Taylor system worked great for Taylor (he didn't have to take a factory job, for example) but everybody else has to live with the consequences.

My question is about the "buffing" in this context. Why would a CEO or a CFO prefer buffing - which, after all, is no more than padding a bra or stuffing your jockey's before a date - to the real thing? To sitting down across a table from the very doctors who saved their grandchild's life the night before, and saying something like, "Dave, God damn it, what the hell is really going on around here? And what do we need to do - together?"