Of all the structural (how care is organized) “evidence-based markers of high quality care,” perhaps the most ironclad has been the involvement of critical care physicians in the care of ICU patients. That is, until now.

In a sophisticated study in today’s Annals of Internal Medicine, Levy and colleagues mine a decade-old, 100-hospital, 123-ICU database containing detailed clinical data on more than 100,000 patients to examine the association between ICU staffing models and hospital mortality. The researchers tell us that they began the study expecting to confirm the benefit of intensivists (also called “critical care physicians”). It would have been odd to expect otherwise, since such a benefit has been seen in a number of prior, smaller studies (summarized here).

Levy et al. were really seeking answers to two different but related questions. The first: in those ICUs (n=79, or 64% of the 123 ICUs) in which non-intensivist physicians sometimes called for intensivist help, what was the effect of involving an intensivist on hospital mortality? In the other hospitals (“no choice” institutions), it appeared that patients either virtually always received intensivist consultation and/or management (n=23, 19%; particularly large teaching hospitals) or virtually never did (n=21, 17%). So in these hospitals, the authors were testing which of the two models led to lower mortality rates. Seemed like a no-brainer.

Shockingly (no pun), in the “choice” hospitals, hospital mortality rates were significantly higher when intensivists were called in. Moreover, mortality was also substantially higher in those ICUs in which intensivists managed virtually every patient than in those in which they managed few or none.

You are probably thinking that these must be apples-to-oranges comparisons – patients in intensivist-only ICUs, or those in “choice” hospitals who receive intensivist care, simply must be sicker. A reasonable concern, to be sure, and in fact such patients were more ill. But the investigators, using robust statistical methods to adjust for severity of illness and for the “propensity” to involve an intensivist, found that this case-mix difference explained only a small proportion of the increased mortality.

They also examined several other possible explanations (including that non-intensivists were more likely to discharge patients to hospice or SNF; thus patients would not appear in the “hospital deaths” column), but found that none of them made the overall 40% higher chance of death vanish. If you’re an intensivist looking for any silver lining, the “harm” associated with intensivists was greatest in the least ill ICU patients, and appeared to lessen (but not evaporate) in the sickest quartile of patients. Boo-ya.

In a well-written accompanying editorial, Gordon Rubenfeld and Derek Angus, two of the world’s top ICU researchers, ponder the possible explanations for these stunning results. One, of course, is that the tools to measure, and thus adjust for, severity of illness are imperfect, and that some unmeasured variables (perhaps captured in “the eyeball test”) are associated with both intensivist involvement and mortality, at least in “choice” hospitals. Of course, there is no way to be sure about this without a randomized trial, which would be awfully hard to do (“Sir, you could be randomly assigned to an ICU either with or without experts. Please sign the consent form here.”). In fact, the Annals study could have only been done in the U.S., since virtually every other country’s hospitals are dominated by closed ICUs.

Supporting the “unmeasured confounders” hypothesis, Rubenfeld and Angus highlight the fact that virtually every prior study found that intensivists were beneficial. They also note that Levy and colleagues did not provide data supporting a plausible causal pathway for intensivist-related harm. In a way, they are arguing for a Bayesian approach to the interpretation of clinical research (a point made previously by Browner and Newman): the “pre-test probability” of intensivist benefit is so high (based on face validity and prior studies) that there is a pretty good chance that a single study showing harm is a “false positive.”

A bit haltingly, Rubenfeld and Angus go on to consider an alternative hypothesis: namely, that intensivists kill.

Although we believe that critical care physicians are trained and expertly skilled in the management of critically ill patients, perhaps some routine critical care practices and procedures may not be beneficial or cumulative use of more interventions may take a negative toll.

Although good on ‘em for raising this possibility, their heart isn’t really in it, quite understandably. After discussing various explanations for the Levy results, they conclude that this study is not enough to change practice or policy:

…until someone replicates Levy and colleagues’ results in another cohort and provides evidence for a mechanism by which intensivist-staffed ICUs increase mortality, their study will remain one observation against many.

I happen to agree – if I had a closed ICU that seemed to be working well, I wouldn’t throw open the glass doors tomorrow. And if I ran the Leapfrog Group, I would not take intensivist staffing off my list of evidence-based safety practices, at least not yet. But I would question my assumptions and use this study to motivate further inquiry into the best ways to organize an individual ICU, or all ICUs. As Levy and colleagues conclude…

Although all of the possible explanatory mechanisms we have mentioned [overly aggressive care, infections from indwelling catheters] may seem to portend badly for the practice of critical care medicine, we suggest that, if true, they are amenable to correction or mitigation through such efforts as guideline development and adherence, quality improvement, and systematic efforts to reduce errors. Given the complexity of critical illness, the need for dedicated critical care physicians seems inevitable, and strategies to assure best practices will help them to guarantee the best outcomes possible. Further research is needed to explain these findings and determine whether these results may be explained by unrecognized residual confounders of illness severity.

Seems right to me.

One postscript: though neither the study nor the editorial mention hospitalists, there are two hospitalist angles worthy of reflection. First, SHM surveys tell us that the vast majority of U.S. hospitalists do see ICU patients. Our ICUs at UCSF Medical Center fall into this category – we have a hybrid model in which our hospitalists remain the physicians-of-record for medical ICU patients, with mandatory intensivist consultation (the intensivists assuming the management of the ventilators, the lines, and the sedation). Over the past decade, I have been under intense pressure to “close the unit,” even though most of our hospitalists and intensivists think our present arrangement works quite well (and our model does pass Leapfrog muster for “high intensity intensivist involvement”). The Annals study reminds us that there is much that we do not understand about best practices in ICU organization, and that a model of open ICUs with selective use of intensivists might well prove to be as good as, or better than, the hermetically sealed ICU, with its forced ICU-floor discontinuity.

Secondly, the data from which the Annals study were derived came from Project IMPACT – a study begun by the Society of Critical Care Medicine (with pharma support) in 1996. Although SCCM can’t be thrilled that their database produced these results (you can be sure that the society’s PR firm received an emergency call from the SCCM CEO this week), the study highlights the power of and need for clinical research to answer important clinical and organizational questions. It also underlines the unique ability of medical societies to help organize multicenter studies whose fruits may be harvested over many years. The Society of Hospital Medicine can, and should, be in this business.

If you fly regularly and the pilot comes on and says that the Captain rather than the 1st officer will be in charge on the flight deck it is time to worry. Could a similar phenomenon be at work here in that the Intensivists are removed from seeing the broader care of a patient and therefore might become blinded to other things that are occuring in the same way that the Captains by taking charge infrequently tend to lose their touch? I have no idea but it is an interesting question….this might of course have nothing to do with it and I might simply not have understood what the article was saying.

I’d be careful about drawing larger conclusions about ICU organization from this single study. The Auerbach/Lindenauer study in the NEJM earlier this year found that hospitalists overall did not improve clinical outcomes. Should we then argue that we should go back to having PCP’s see their patients in the hospital?

Your post simplified a complex scientific presentation that I probably would not have read had I not read your review first…sort of like reading a review of a movie before seeing it.

The extensive literature listed in this study’s bibliography showing that CCM docs conveyed benefits were published prior to 2002. They analyzed care administered in the years prior to publication and before it was altered by the proliferation of health IT comprised of CPOE, EMRs, and the other devices you have described in your posts.

Of the one hundred hospitals whose 123 ICUs’ outcomes were part of this study, the questions must be asked: was there complete penetrance of CPOE devices for the entire time frame or were there inter hospital disparities in CPOE deployment? Were there CPOE associated disruptions to care and cognition (Groopman, on your post, June 2) by changes of the ICU IT architecture during the specified time frame? When one considers the time sensitive aspects of critical care therapy and the “Mother of all Excavators” CPOE outcome study (Pediatrics, 2005) to which you referred (“Technology Hype Cycle” May, 13) reporting a threefold higher death rate in critically ill patients managed with CPOE, these rate as important questions. Were the results confounded by variable use of CPOE and other health IT machines, for better or for worse?

I draw your attention to the the IMPACT website that states, for the “foundation” module of IMPACTessentialIT, “data collection typically takes less than 8 minutes per patient”. “ProjectIMPACT” data collection “requires a full time data collector (ideally a critical care nurse) for each 15-20 beds for which data is collected.”

Not everyone is surprised by the results, especially those of us who spent years making social visits to our closed ICU cases, only to see rows of motionless patients with Versed flowing into their veins. Sedation was ordered by house staff by virtue of the requirements of the academic teaching program, who must write the orders, but do so often with minimal supervision. And now there are ICU NPs and PAs happily clicking away on CPOE machines.

I appreciate all the comments. Viz “Vanillablue’s” observation, I had the same thought – in many ways, the Lindenauer NEJM study (which I discussed here) has a similar flavor, in that it was a large database analysis that found a less positive result than many prior, smaller studies.

The difference is that the Lindenauer study showed no benefit (quality) and lower savings than were seen in prior hospitalist research, whereas this week’s Annals study on intensivists showed harm. That’s a horse of a different color.

As a (pediatric) intensivist myself — ouch! But perhaps some of the answer is that intensive care is heavily procedure-oriented, and we do things in the ICU because we can. Capacity drives demand in the ICU, too. I remember the days when every patient got a Swan-Ganz catheter and an arterial line because, well, because they were in the ICU. Now we do those things much less frequently, although everybody still seems to get a central venous catheter, often, I suspect, to make the blood draws easier. We should keep in mind the old saying applied to surgeons: “a surgeon knows how to operate, a good surgeon knows when to operate, and an excellent surgeon knows when not to operate.”

Hmmm, intensivists and harm. It took me a while to figure this one out.
I first take issue with the division and characterization of the groups, but I haven’t seen any critical reviews of the authors’ definitions, so I may be dizzyingly wrong. Bob, maybe you can assist my understanding here. I refer to table 2 of the study under patient characteristics; take CCM to mean critical care management which assumes intensivist involvement and NO CCM to assume primary care physician involvement.

Isn’t CCM for 95% of patients equivalent to NO CCM for 5% of patients? I mean, isn’t it a different way to characterize the same group of patients? If so, how can they be compared to each other? Shouldn’t their evaluation be combined? These are the outliers that the study claims to compare to demonstrate a mortality difference.

I take the same issue with the huge interval of overlap for the 5%-95% CCM and NO CCM group. Logistically, isn’t CCM for 40% of patients the same group as NO CCM for 60% of patients? How can these ICUs be pitted against each other?

Another concern I have is the chosen endpoint of hospital mortality and not ICU mortality. Specifically, an intensivist is doing well if he or she can get a patient to the floor, but what additional confounding elements prevented the patient from surviving his or her hospital stay? Is that the intensivist’s domain and ultimate responsibility? How can we have influence here?

I applaud the study for revealing the very complexity of ICU organization and infrastructure which are imperative to quality critical care delivery. There are many unaccounted for confounders in this study which are difficult to tease out in a database study that cannot capture every variable. I often repeat to my colleagues that intensivists are not “the answer” to all ICU problems, but that the multidisciplinary approach to ICU patient care is the key. We all need to be doing our part competently and collaboratively. Perhaps an emphasis on team dynamics in critical care (and even hospital-based medicine) will become the focal point for study after this review because I’m quite sure organized and protocolized ICUs that don’t use intensivists probably fare well when evaluated.

I refer you to a podcast by the study author Dr. Levy on the Society of Critical Care Medicine website. Thoughtful and provocative.