Friday, September 10, 2010

Why the big fuss over Donald Berwick and integrative medicine?

And besides, what's wrong with a little quackery now and then if patients want it? Those are the questions posed by Shannon Brownlee over at The New Health Dialogue in a post written in response to Kimbal Atwood's expose of Berwick's apparent fondness for unscientific forms of complementary and alternative medicine. Several of her assumptions and accounts of facts are fundamentally flawed. Let's take a closer look at the post.

Early on she says this about Atwood's post:

In Dr. Atwood’s view, a February 2009 appearance by Dr. Berwick at Bravewell Collaborative-sponsored Summit on Integrative Medicine and the Health of the Public, where he shared the podium with Dr. Mehmet Oz, Dr. Dean Ornish, Senator Tom Harkin, was evidence that he was either naïve or had “gone over to the dark side."

The “dark side?” Don Berwick?? C’mon.

OK, let's not judge Berwick by the company he keeps. But there's much more than his mere appearance at the Summit on Integrative Medicine to raise concern about a fondness for, or at least an openness to quackery on his part. His remarks at the Summit and elsewhere are what should be cause for concern. At the Summit he spoke uncritically about homeopathy and acupuncture and said that he viewed integrative medicine as a “distinguished and important new arena.” On another occasion Berwick said that evidence based medicine may have to take a back seat if patients' preferences so dictate. What more do you need?

She goes on:

More to the point, while alternative medicine is undoubtedly mostly quackery, so is a good bit of allopathic medicine, at least if you define quackery as medical practices that are not based on valid evidence.

She's confused. She's confusing quackery with non-evidence based medicine (it's not the same thing) and poor uptake of best evidence (again, it's not the same thing). After some misinformed questions from commenters I once had to explain this. Here is an excerpt from that post (where I used the word woo to denote quackery):

Frequently asked questions on woo and mainstream medicine

“Woo” is a term for certain implausible and outlandish claims of complementary and alternative medicine (CAM). The term has recently been popularized on the blogosphere by Orac and others and has been a subject of several recent posts of mine which have drawn numerous criticisms and questions. Rather than address them piecemeal in my comment threads I decided a more effective way to answer my detractors would be in the form of a frequently asked questions (FAQ) post, so here goes.

Many of mainstream medicine’s conventional treatments are not evidence based. Aren’t they a form of woo?

No. Although some conventional methods fail to measure up to best evidence they are at least based on known anatomy and physiology. They have some plausibility in the observable biophysical model in contrast to the “vital forces”, nebulous “energy fields” and “non-local powers of the mind” which are characteristic of woo.

You seem to focus a lot of your criticism on woo. Isn’t non-evidence based conventional medicine harmful too?

Yes, of course. Moreover, there are harmless forms of woo just as there are harmless conventional breaches of EBM. All departures from best evidence are problematic and need to be addressed, whether woo-based or not.

Then why make a distinction?

Because of important differences in the ways the problems manifest themselves. Mainstream medicine applies a double standard and that’s what I’m trying to expose. People in the mainstream are appropriately critical of conventional deviations from best evidence and are trying to correct the situation. But due to the nature of the problem---a complex interplay of system and cognitive failures---the fix is not easy. In contrast (and here’s where the real hypocrisy comes in) mainstream medicine uncritically embraces woo, applying to it a much easier evidentiary standard and often no standard at all. The remedy for the problem of woo would be much simpler, too. Mainstream medicine could simply say no. Woo, by definition patently implausible, is easy to spot. There’s nothing complicated about it. It’s not a system problem. It’s there in mainstream medicine purely by choice. That fact raises another important distinction. If mainstream departure from EBM is a complex system problem and woo is there by choice then woo constitutes a more serious ethical problem.

The point I was trying to make there is that from both a policy and a purely scientific standpoint, while there may be areas of overlap, there are important distinctions between quackery and non-evidence based medicine. The problem of poor adherence to EBM is very real in mainstream medicine, but it is a complex mixture of inefficient dissemination of knowledge and external barriers---barriers that leaders in mainstream medicine, some of them anyway, are struggling to overcome. This is quite a different situation from those (and, to complicate matters, there are many in mainstream medicine) who knowingly and willfully promote non-evidence based claims without regard to any scientific foundation.

Brownlee's examples of non-evidence based mainstream medicine are misinformed. She writes:

Just look at the recent history of widespread medical practices that turned out not to work -- or worse, not only did they not work, they also harmed patients. High dose chemo for breast cancer comes to mind. And don’t forget lidocaine for cardiac arrythmia and cardiac stents to prevent heart attacks. Bee pollen and homeopathy may turn out to be pseudoscientific bunk, but at least for the most part they’re not as dangerous as some widespread medical therapies that aren't based in sound science and valid evidence.

First off, I'll give her a pass on breast cancer (not my area of expertise by any means; I'll leave that to Orac if he chooses to weigh in). Lidocaine I know a little about. It certainly did work. It's still an alternative in the ACLS algorithms. Prophylactic lidocaine for routine use in myocardial infarction has been abandoned but it did work, in what it was purported to do. Here's the classic paper of Lie and colleagues demonstrating that it when lidocaine was used prophylactically in MI patients it decreased the rate of V fib from 9% to zero. Not only did it work but it worked extraordinarily well. That finding has never been refuted although lidocaine was gradually abandoned for this use for complex reasons. Beta blockers came along. Lidocaine could not be demonstrated to decrease mortality because defibrillation was universally effective. There were side effects---not life threatening when the drug was used properly but they were there. By the mid 80s we were in the reperfusion era, making primary V fib less of a threat.

And cardiac stents to prevent heart attacks? What's she talking about? There are well defined benefits of coronary stenting but no one with any authority in mainstream medicine claimed that stents prevented heart attacks. (With apologies to Orac, holy strawman fallacy, Batman!). In fact, about the time stents came into common use in the late 1980s mainstream thought leaders were saying just the opposite. We already knew by that time that most heart attacks were due to rupture of mildly stenotic plaques, lesions which no one would ever consider targets for revascularization.

To be completely fair, Brownlee could have come up with examples that would have better supported her claim about the shortcomings of mainstream medicine. There was the use of oral antiarrhythmic drugs to suppress ventricular arrhythmias which turned out to be lethal (class Ic drugs, as demonstrated in CAST), not worth the side effects or extracardiac toxicity (tocainide, mexilitine) or not as good as device therapy (amiodarone). The problem here was not that the drugs didn't “work” but that over-reliance on surrogate endpoints was hazardous, even while the recommendations were based on the best scientific principles and evidence then known. CAST was conducted because investigators were aware of proarrhythmia but didn't know it balanced out against potential benefits. The study was completed and lessons were learned. You will never see that kind of scientific progress with any of the forms of woo such as homeopathy or acupuncture.

As to her point that fairy dust at least is not dangerous? Of course it's not, because it does nothing in the patient's body. But that misses an important point. No one that I know of, certainly not Atwood, probably not even Orac, would deny patients the right to avail themselves of harmless quackery. It's quite another thing, though, for supposedly reputable and scientific organizations like the Institute of Medicine, or the leader of the IHI (and now CMS) to endorse it, or, worse, for a government health plan to pay for it.

Brownlee does go on to make a valid point: we in mainstream medicine don't do a good enough job of informing patients about the evidence. And I'm not trying to make excuses but there are external barriers, including time and economic constraints, that make that process very difficult, as I demonstrated here and here.

She concludes:

When it comes to the ethical duties of physicians, do we really need to fret about low risk alternative treatments? It’s probably more important that doctors and patients share decisions, and that we make sure that patients have the tools they need to make their own choices about any treatment, whether it's alternative or mainstream medicine.

No, we don't need to fret about alternative treatments---at least we shouldn't fret about patients seeking them out for themselves, as long as they're harmless. But we should be outraged at the ethical failure of purportedly scientific, esteemed mainstream organizations like the Institute of Medicine, mainstream thought leaders and our government promoting pseudoscience and quackery.

Oh---and for another expose of Berwick's fondness for woo and a brilliant analysis about what's really, really wrong with his extreme consumerism see this post from David Gorski at Science Based Medicine.

13 comments:

Thanks for your thoughtful (and thought-provoking) post. We are in agreement much of the time, but I wanted to take issue with a couple of your points. Let me repeat, bee pollen and homeopathy are probably bunk, and are certainly based in Woo (great term!), but they are not invasive or poisonous. And if I as a patient want to take them, I don't really see any reason my physician should get hot under the collar about it. (By way of full disclosure, I am not a consumer of alternative therapies or a believer in Woo.)

Encainide and flecainide, on the other hand, killed patients. Yes, they were highly effective at suppressing arrythmia, but as you point out, surrogate endpoints are just that, surrogates. The endpoint we care about is the patient's health. So when I say oral antiarrythmics didn’t work, I’m talking about the endpoint that matters to patients.

I concede your point that there is a distinction between non-evidence based medicine and Woo, but the line does not seem as bright to me as it does to you. Medical theories about disease and how the body works are just that, theories. They are indeed verifiable (or really falsifiable), but when medicine doesn't wait for verification before marching in with invasive treatments there’s the risk of harm. Only later do we discover the treatment was worse than the disease. (For those who are interested the history of the high dose chemo fiasco, I recommend False Hope, edited by Richard Rettig.)

As for physicians’ perceptions of the preventive power of stents, Holy strawman, nothing. While it’s doubtless true, as you say, that thought leaders knew that most heart attacks were due to “rupture of mildly stenotic plaques, lesions which no one would ever consider targets for revascularization,” that’s not what the rank and file were thinking. See for example: Roy Poses et al. “Physicians’ Judgments of Survival After Medical Management and Mortality Risk Reduction Due to Revascularization Procedures for Patients With Coronary Artery Disease,” Chest (2002) vol.122 pp 122-133. Richard A. Lange and L.D. Hillis, “Use and Overuse of Angiography and Revascularization for Acute Coronary Syndromes,” NEJM vol. 338 no. 25 pp 1838-1839. Not to mention the paper I link to in my post published in Annals this week, “Patients' and Cardiologists' Perceptions of the Benefits of Percutaneous Coronary Intervention for Stable Coronary Disease.”

We are agreed that patients deserve to know when elective treatments are grounded in valid evidence, and when they're based more on unverified theory or longstanding, but untested practice. They also need to know the potential risks of the various choices they face.

I also agree with your point that the NIH and other august bodies shouldn’t be endorsing quackery. (On the other hand, spending a small percentage of the NIH budget on testing alternative medicines that have the potential for harm is a probably a good idea. St. John’s Wort, for instance.) But if we are going to refuse to pay for untested alternative nostrums, should we also not refuse to pay for some untested mainstream therapies, too? Or at least refuse to pay unless there's a clear demonstration that the patient understands the trade-offs involved, and the poor state of the evidence. Better yet, ask the patient to pay a larger share of the cost when the evidence is poor.

Thanks again for your thoughtful post.

Here are the links to the papers I mention. http://www.ncbi.nlm.nih.gov/pubmed/12114347http://www.ncbi.nlm.nih.gov/pubmedhttp://www.annals.org/content/153/5/307.abstract

As for physicians’ perceptions of the preventive power of stents, Holy strawman, nothing. While it’s true, as you say, thought leaders knew that most heart attacks were due to “rupture of mildly stenotic plaques, lesions which no one would ever consider targets for revascularization,” that’s not what the rank and file were thinking when stents came on the scene. See for example: Roy Poses et al. “Physicians’ Judgments of Survival After Medical Management and Mortality Risk Reduction Due to Revascularization Procedures for Patients With Coronary Artery Disease,” Chest (2002) vol.122 pp 122-133. Richard A. Lange and L.D. Hillis, “Use and Overuse of Angiography and Revascularization for Acute Coronary Syndromes,” NEJM vol. 338 no. 25 pp 1838-1839. Not to mention the paper I link to in my post published in Annals this week, “Patients' and Cardiologists' Perceptions of the Benefits of Percutaneous Coronary Intervention for Stable Coronary Disease.”

We are agreed that patients deserve to know when elective treatments are grounded in valid evidence, and when they're based more on untested theory and longstanding, but untested practice. They also need to know the potential risks of the various choices they face.

I also agree that the NIH and other august bodies shouldn’t be endorsing quackery. (On the other hand, spending a small percentage of the NIH budget on testing alternative medicines that have the potential for harm is a probably a good idea. St. John’s Wort, for instance.) But if we are going to refuse to pay for untested alternative nostrums, shouldn't we also refuse to pay for some untested mainstream therapies, too? Or at least ask patients to pay a larger share of the cost out of pocket?

Thanks again for your excellent post.

Here are the links to the papers I mentioned:http://www.ncbi.nlm.nih.gov/pubmed/12114347http://www.ncbi.nlm.nih.gov/pubmedhttp://www.annals.org/content/153/5/307.abstract

If this is true, how can you ever get a RCT result that can be used over time?

"Time evolution in a dynamic system, like the human body, depends on the total initial state of the organism. Any alteration in the initial state need not hold good as time evolves.1 If these two principles were understood, the concept of controlled studies in medicine, both drug and interventions, would seem unreliable!2 In the conventional controlled studies two matched human beings are compared. The total initial state of both the patient and the control being unknowable: in fact, a large part of the initial state-the genotype and the consciousness-are unknown, the outcome would be fallacious. This is precisely the reason why most, if not all, controlled study results have come to grief on long term audits.3,4,5,6,7,8,9,10,11,12,13 However, short term observations might not show the reality unless, the drug under study is dangerous.14"

I agree entirely with your first paragraph. As I said in my post, I don't get hot under the collar about patients going to see the homeopath down the street. I don't think Atwood would either, or any of the others whom I've joined in the blogging campaign against promotion of woo by mainstream medicine. What we do get upset about is promotion of woo by purportedly scientific institutions or promotion and/or payment for it by the government health care plan, which is more to the point of this discussion.

Regarding the second paragraph, you didn't mention the oral antiarrhythmics in your post, I did. You mentioned lidocaine which is not only not an oral antiarrhythmic, but a different class altogether, a class which was never found to be associated with sudden death. Indeed lidocaine did produce an outcome that mattered to patients in the early stages of a heart attack: a virtually zero chance of having to be zapped with 200 joules of electricity! I brought up encainide and flecainide (by my referral to class 1C drugs) in my post and noted that their use caused death in some patients. I also noted in my post that what we thought were the best scientific principles and evidentiary standards at the time needed to be better. They got better and in just a few more years (1992) they were given a name: evidence based medicine.

Concerning you next paragraph---Without realizing it you may have actually brightened the line between woo and non-evidence based medicine when you pointed out that theories are just theories! That fact, (stated another way, that real science is always tentative) is exactly what the woosters DON'T acknowledge. As Abraham Flexner said 100 years ago, real science knows it's tentative whereas the woosters (he was referring mainly to the homeopaths) deal, by contrast, in presuppositions and dogma. He wrote:

“The modern point of view may be restated as follows: medicine is a discipline, in which the effort is made to use knowledge procured in various ways in order to effect certain practical ends. With abstract general propositions it has nothing to do. It harbors no preconceptions as to diseases or their cure. Instead of starting with a finished and supposedly adequate dogma or principle, it has progressively become less cocksure and more modest. It distrusts general propositions, a priori explanations, grandiose and comforting generalizations.”

Real woo is based on nutty ideas and metaphysical presuppositions that have persisted, unmodified, sometimes for centuries, based on NO observable natural phenomena. Said nutty ideas are either not falsifiable (because no testable or observable mechanism is offered) or so easily falsifiable that even an 11 year old (remember Emily Rosa) could blow them out of the water.

Yes, when medicine marches in without verification there's risk of harm. And your point is? In other words I'm not sure how that point, true though it is, in any way addresses or refutes Atwood's concern that purportedly scientific organizations (IHI, IOM) and government representatives should not promote woo, and that government should not pay for it (as they already are, in big bucks, with their funding of NCCAM).

That said, let's get back to your point about verification. Occasionally, moving ahead without verification may represent a failure. More often, I would submit, it represents a necessity for medical progress. We recently celebrated the 50th anniversary of the first implanted pacemaker. Yes, the pacemaker had risks which we only found out about later, and we could have waited forever for verification, with many lives lost. I could think of many, many other examples.

I will address your points about stents in another comment after checking the references you provided.

First, I didn't say doctors don't have inflated perceptions about technology, stents or other forms. The main point of this whole discussion is how things are promoted and what things are to be paid for, not what doctors in the trenches perceive. No thought leaders in medicine were promoting stents to prevent MI.

What the Poses paper demonstrates is that busy docs in the trenches are not very good at clinical epidemiology, and that they tend to be biased towards technology. But this needs more nuance. Poses compared docs' perceptions against the "best available evidence." But, as you know, the best available evidence in 1996-1997 wasn't very good. It's only in the last few years that we really addressed the question of stenting stable CAD in a systematic fashion, with COURAGE and BARI 2d. Although those trials demonstrated that in VERY stable CAD one can embark on a trial of medication and not immediately jump in with revascularization, 20% or so of those patients (COURAGE) will become unstable and need revascularization on clinical grounds, and there's no way to predict on the front end.

I don't have access to the Hillis paper, but I have heard him speak on this and am familiar with his work, and in fact he was one of the thought leaders I had in mind when I made the point that leaders in mainstream medicine were NOT claiming any preventive power of stents for stable disease, in fact quite the opposite. But the NEJM paper you cite (I was able to access the extract) deals with unstable coronary syndromes in which a heart attack was just completed, in progress or impending and so is not particularly relevant to our discussion about preventing heart attacks. In 2010 the evidence that emergency stents in patients whose heart attacks are diagnosable on arrival (the so called STEMI)are life saving is robust.

I read the Annals paper. Patients' perceptions, more than cardiologists' perceptions, were inflated. This may simply mean that the patients' cardiologists should have spent more time talking with them. Although 40 some odd % of cardiologists apparently thought PCI accomplished more than mere symptom relief that perception may be correct. Even the COURAGE trial (nuclear substudy) showed a subset of patients who had better outcomes, beyond symptom relief, with stenting. Not to mention the medically randomized patients who crossed over to stenting because they became unstable.

As to your final question, there's a difference between untested and totally nutty (implausible). I agree that policy makers will have some hard decisions to make about how to slice the pie of health care resources for conventional medical treatments. But for nutty ideas the job should be easy: just say no.

Here is where we disagree. You say: "Occasionally, moving ahead without verification may represent a failure. More often, I would submit, it represents a necessity for medical progress."

I would argue that moving forward without verification is OFTEN a failure. Most of medicine represents having moved forward without verification. Most of what we currently pay for has not been verified. (I turn to the IOM as my source for this claim: less than half of current practice is backed up by valid science.)

I don't want to be absolutist or silly about this. Some medical practice doesn't need an RCT to show that it's effective. We don't need at RCT to show that arsenic is poisonous, and we don't need an RCT to show that antibiotics are effective against pneumonia.

But that leaves a lot of terra incognita. We know that much of what is being done is useless (the estimates range from 20-50%). Even when a procedure or hospitalization or test is useless, it still poses risk. An unnecessary CT, just to take one example, exposes the patient to radiation, and exposes him or her to the potential for finding an incidentaloma, which must then be worked up. An unnecessary cath exposes the patient to the risk of a perforation and MI. Risk is constant, even when potential benefit is zero.

You wrote: "I read the Annals paper. Patients' perceptions, more than cardiologists' perceptions, were inflated. This may simply mean that the patients' cardiologists should have spent more time talking with them."

Yes, I completely agree, patients hear what they want to hear. We also agree that cardiologists should spend more time talking to patients. (Actually, having just taken my 81 year old mother to the cardiologist, who was fabulous, I think it requires more than time, but that's another discussion.) But the really troubling thing that I've found in spending a fair bit of time talking to a lot of cardiologists is that some (not all) can always find a reason to cath and stent.

They give all sorts of reasons why they need to do elective PCI. For example, you wrote: "Even the COURAGE trial (nuclear substudy) showed a subset of patients who had better outcomes, beyond symptom relief, with stenting." Yup, patients vary, but is this a real result, one that will hold up? Subset analysis is good for generating new hypotheses, but should this be used as a justification do PCI on a lot of patients?

In the end, it is the patient who deserves to know that the benefit is very small in terms of reduction of risk of MI or mortality -- if it exists at all. Meanwhile, the risk stays the same.

I'm circling back to my central point: Patients need to be better informed about what's known and what's hypothesis. They need to better informed about the potential risks as well as potential benefits. And the decision to undergo an elective procedure or test should be shared by patient and physician.

Shannon,I think that while subgroup analysis tends to be weaker than the original trial, the COURAGE substudy was valid. The fact that a lot of patients randomized to medical therapy crossed over to stenting says a lot, too.

If you're making the point that some cardiologists' financial incentives bias them away from best evidence, I won't disagree.

By the way, surveys after publication of COURAGE and BARI2d showed a significant fall in the rate of usage of elective stenting, so at least in the aggregate, cardiologists were paying attention to the evidence.

You asked about the IOM figure of how much of current practice is based in valid science. Here's the direct quote from the IOM:

"Estimates of the proportion of medical care in the United States that is based on, or supported by, adequate evidence range widely [9-14]. However, given concerns about the extent to which this information may be generalized, and the quality of the evidence which is used, some place this figure at well below half."

That IOM statement was nebulous to say the least. To say that "some place" the portion of care based on science at well below half---well, what does that mean, really? That survey data are all over the place and that there are people out there with an opinion that less than half of our care is evidence based. I only skimmed the pdf you linked but I can just imagine how IOM spun that.

Before going further, I think we need to be careful not to conflate two things. One is mainstream medicine accepting certain treatments for which there just wasn't enough evidence---what you refer to as moving ahead without verification---and the other being poor adherence by docs to known best evidence (both under- and over-utilization have been suggested in studies). From what I could see the IOM report dealt mainly with the latter.

References 9-14 in that IOM whitepaper dealt with a few selected conditions and, to my thinking, say little about the overall state of medicine.CONTINUED---

Concerning the two issues under discussion, starting with the problem of moving ahead without verification. Yes, it's been a problem in many areas. To what extent a failure or a necessity of medical progress is one of the fundamental points on which we disagreed. Certainly it's been a mixture of both. We could think of many examples, with differences in interpretation that may be more philosophical than evidential. Some interventions, like the artificial heart, we did "just because we could", appropriately I believe. The same was probably true to a degree of CABG, and although highly questioned in the 70's, its indications were pretty well defined by 3 high quality RCTs reported by the 80s. By the time PTCA came along in the late 70s and later stenting, the situation was complicated. To be considered were studies of intervention vs no intervention, early versus late, various combinations of interventions and numerous antithrombotic strategies, comparative effectiveness vs CABG, and on and on. The combinations were almost endless, and trials numbered in the 100s. Meanwhile, to complicate things further, by the time trial results were reported newer and better technology (how many generations of stents have come out?) had already been developed. Medicine was always a little ahead of the latest clinical trials. But I think patients benefited.

The same could be said for mechanical ventilators, where technology is continuously years ahead of clinical trials. Having worked in ICUs for decades I believe this benefited patients.

Insulin and oral hypoglycemic agents, decades ago, were given to type 2 diabetics in the hope that they would help prevent complications. This was done "without verification", or at least without the type of verification that would meet todays standards of outcome based EBM. It's a good thing for patients we did. Although some agents may increase the risk for macrovascular disease the decrease in microvascular disease is robust as shown in the UKPDS. These are not failures of mainstream medicine. On the flip side there have been failures---HRT and NSAIDs (all classes)come to mind. There are others (digitalis, theophylline) which today we would regard as harmful, but it's a good thing they were put into use without verification when they were, because it was all they had, and although their risks, especially as used in the early days, would be considered unacceptable today, they were better than nothing back then.

As far as the other issue, of docs under- and over-utilizing treatments and not adhering to evidence we already have, the Dartmouth Atlas data reveal this to be a significant problem. It is driven by complex external factors including defensive medicine and inefficient dissemination of best evidence. Organized medicine faces a difficult struggle to overcome this problem, and it is in contrast to the problem of woo. Nutty ideas can sometimes be identified as nutty ideas without a lot of additional research. Mainstream medicine's institutions should just say no.

This last point was the premise of Atwood's post which you criticized. Under discussion in that post was whether policy leaders or purportedly science based institutions should promote quackery. You sidetracked me with a red herring by diverting attention to an issue (conventional medicine's failings) that was not originally under discussion, and warranted discussion separately. Woo AND poor adherence to best evidence regarding conventional treatment are BOTH problems WITHIN conventional medicine but they are separate problems with very little similarity between them in terms of science, policy or solutions.