Leapfrog’s report is incomplete, misleading

By Donald E. L. Johnson

Health Care Strategic Management, Feb. 2002, Vol. 20, No. 2

Copyright 2002 by The Business Word Inc.

The Leapfrog Group was created by the Business Roundtable in 2000 to find ways to increase the quality of care delivered by hospitals, but its efforts to date lack the attention to detail or the quality that it is demanding of
hospitals.

Simply put, Leapfrog is so anxious to show progress and to justify its existence that it is taking short
cuts and making claims that don’t stand up.

Hospitals need to challenge its assumptions and question its tactics, because the big corporations and some unions that are backing its efforts are trying to use its information to negotiate with hospitals. Some also are putting its incomplete and misleading information on their Web sites for their employees.

Medical errors overstated
First, Leapfrog is using a discredited Institute of Medicine study on medical errors and deaths in hospitals to give the nation’s hospitals a reputation for poor quality. There are more medical errors and premature deaths in hospitals than there should be, but not the 44,000 to 99,000 avoidable deaths figure that is so loosely and irresponsibly bandied about by Leapfrog and the IOM. (“IOM medical errors study overstates problems,” Health Care Strategic Management, pp. 2-3, August 2000; “The Institute of Medicine Report on Medical Errors: Could it Do Harm?” New England Journal of Medicine, Vol. 342, pp. 1123-1125, April 13, 2000; “Deaths Due to Medical Errors are Exaggerated in Institute of Medicine Report,” Journal of the American Medical Assn., July, 5, 2000). The authors of the IOM study defended it in the same issue of NEJM in which it was challenged.

Survey results incomplete
Second, last month Leapfrog and several other organizations announced the incomplete results of a virtually meaningless survey of hospitals that was designed to show that a few hospitals are using the approaches to quality improvement that it recommends.

The problem is not with the survey design, but that Leapfrog and other organizations are promoting it before all urban hospitals have been surveyed or responded. Before the survey can be used with any confidence, Medstat, which is doing the survey, must get on the phone to every urban hospital and get a response. It should not wait for a hospital to mail a survey.

Only 48%, or 241, out of only 497 surveyed hospitals in six regions of the country responded to the survey. Not only should all urban hospitals be surveyed, but also the response should be 100%. This is not hard to do when so few questions are involved.

So, all of the percentage numbers Leapfrog, the Pacific Business Group on Health and HealthGrades are publishing on various Web sites mean nothing and mislead consumers. They are, however, published below to show how worthless they are.

Computerized physician order entry not easy
Third, executing the computerized physician order entry (CPOE) systems and staffing intensive care units with trained intensivists, as Leapfrog is urging hospitals to do to improve outcomes, are not as feasible or easy as it makes them out to be.

Leapfrog found that 3% of responding hospitals are using CPOE and 30% plan to by 2004. Assume, for a minute, that non-responding hospitals didn’t respond because they aren’t using CPOE and don’t plan to. Then the percentages would be much lower. Software vendors offer CPOE solutions.

But as every health care executive knows, just because CPOE software is available doesn’t mean every physician uses it or that the ones who use it use it all of the time. What percent of the physicians at the 3% of the responding hospitals with CPOE software use it?

“Studies show that CPOE systems can be remarkably effective in reducing the rate of serious medication errors at least 55%,” Leapfrog said in a nationally distributed press release. If the study design is reliable, CPOE may be a valuable tool, but given Leapfrog’s use of other questionable data, one has to wonder.

Similarly, Leapfrog claims that “10% of responding hospitals have intensivists overseeing care in the ICU at least eight hours per day. Another 18% of responding hospitals plan to enlist intensivists by 2004. Studies show that at least one in 10 patients who die every year in ICUs would have an increased chance to live if intensivists managed their care during their stay.”

Leapfrog cites two articles to support its advocacy of CPOE and intensivists: “Effect of computerized physician order entry and a team intervention on prevention of serious medication errors,” JAMA. 1998; 280:1311-16 and, “Potential reduction in mortalty rates using an intensivist model to manage intensive care units,” Effective Clinical Practice. 200; 6:284-289. Hospital strategists should check out these articles.

Shortage of ICU specialists
In addition to the cost and time investments required to implement CPOE, hospitals need to look at the availability of intensivists.

The Society of Critical Care Medicine (SCCM) said that it is “. . .concerned that there are insufficient qualified critical care specialists to comply with Leapfrog standards.”

Also, “While the society applauds the standard it believes implementation will require Herculean efforts to accomplish this feat,” said Ann E. Thompson, M.D., FCCM, president of the society. “In the last decade, studies have clearly demonstrated the ability of intensivists to improve the survival of critically ill patients, maximize the efficient use of costly hospital resources, and decrease the cost of care. Failing to implement the standard will perpetuate the situation that exists in a majority of hospitals today. Our most critically ill patients do not receive optimal care and many are dying unnecessarily.”

Less than one-quarter of ICUs are currently staffed by physicians adequately trained in ICU management, the society said. “And, over the foreseeable future the problem is expected to worsen significantly.”

Can hospitals afford the up-front investments?
Thus, “Even if a majority of hospitals want to comply with the Leapfrog standard, the current supply of
qualified critical care specialists is inadequate to meet those needs.”

“In fact, the shortfall is expected to increase significantly through the year 2030. The nursing shortage is even more severe. In order to increase the existing level of service, more physicians and nurses will be necessary than are currently being trained,” Thompson said.
SCCM says the solution to the staffing shortage is to:

Attract critical care specialists and increase funding for the medical and nursing programs.

Improve working conditions and increase compensation for nurses.

Engage professional societies and the public to advocate for expansion of critical care training programs.

Educate the public on the lifesaving value of appropriately trained ICU physicians.

Another problem with implementing the Leapfrog standards is that many hospitals can’t afford the up-front investments required.

Yes, cutting medical errors and preventing avoidable deaths are important goals that all hospitals are working toward. And improved quality should reduce costs. But money’s tight and setting priorities is difficult for all hospitals. Few buyers are willing to pay for the increased quality they are demanding.

Each hospital has to assess its priorities and opportunities to improve quality and reduce costs. And all hospitals and the American Hospital Assn. should make sure that Leapfrog has its facts right and puts the problems of medical errors and avoidable premature deaths in proper perspective.