“Ridiculing RCTs and EBM”

Last week Val Jones posted a short piece on her BetterHealth blog in which she expressed her appreciation for a well-known spoof that had appeared in the British Medical Journal (BMJ) in 2003:

Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials

Dr. Val included the spoof’s abstract in her post linked above. The parachute article was intended to be humorous, and it was. It was a satire, of course. Its point was to call attention to excesses associated with the Evidence-Based Medicine (EBM) movement, especially the claim that in the absence of randomized, controlled trials (RCTs), it is not possible to comment upon the safety or efficacy of a treatment—other than to declare the treatment unproven.

A thoughtful blogger who goes by the pseudonym Laika Spoetnik took issue both with Val’s short post and with the parachute article itself, in a post entitled #NotSoFunny – Ridiculing RCTs and EBM.

Laika, whose real name is Jacqueline, identifies herself as a PhD biologist whose “work is split75%-25% between two jobs: one as a clinical librarian in the Medical Library and one as a Trial Search Coordinator (TSC) for the Dutch Cochrane Centre.” In her post she recalled an experience that would make anyone’s blood boil:

I remember it well. As a young researcher I presented my findings in one of my first talks, at the end of which the chair killed my work with a remark that made the whole room of scientists laugh, but was really beside the point…

This was not my only encounter with scientists who try to win the debate by making fun of a theory, a finding or …people. But it is not only the witty scientist who is to *blame*, it is also the uncritical audience that just swallows it.

I have similar feelings with some journal articles or blog posts that try to ridicule EBM – or any other theory or approach. Funny, perhaps, but often misunderstood and misused by “the audience”.

Jacqueline had this to say about the parachute article:

I found the article only mildly amusing. It is so unrealistic, that it becomes absurd. Not that I don’t enjoy absurdities at times, but absurdities should not assume a life of their own. In this way it doesn’t evoke a true discussion, but only worsens the prejudice some people already have.

A “Double Standard”?

Last week I had planned to write a comprehensive critique of a recent comment by Larry Dossey. He had posted it on Val Jones’s betterhealth website in response to Dr. Val’s essay, “The Decade’s Top 5 Threats To Science In Medicine,” originally posted here on SBM. Much of what Dr. Val had identified as the top threats involved recent dalliances, by government, medical schools, and the media, with the collection of implausible and mostly nonsensical health claims that advocates have dubbed “CAM.” As uncontroversial as Dr. Val’s assertions ought to have been—similar to suggesting that closing one’s eyes and “using the force” would be a threat to safe driving (even if some might quibble over the top threats to science in medicine)—Dr. Dossey demurred by distraction:

Your article implies that conventional medicine is grounded in evidence-based research and that CAM is not. This is grossly overstated, and suggests that a double standard is being applied to these fields.

Dossey trotted out familiar arguments: “Much, if not most, of contemporary medical practice still lacks a scientific foundation”; “the Congressional Office of Technology Assessment (OTA) found that only an estimated 10 to 20% of the techniques that physicians use are empirically proven”; hospital care is “the third leading cause of death in the United States,” accounting for hundreds of thousands of deaths each year.

He concluded with an appeal to fairness, rationality, and collegiality:

Overwhelming evidence reveals that conventional medicine is, on the whole, woefully unscientific. It’s fashionable and easy to deny this, but the facts say otherwise. So, by all means, Dr. Val, be critical of CAM – but do not fall into a double standard. Let us ruthlessly apply science to ALL we do as physicians. Let us challenge ALL areas of medicine to a higher standard. On that, I’m pretty sure we can agree.

Keep up the good work.

Sincerely yours,
Larry Dossey, MD

I procrastinated with my own rebuttal, and in the meantime David Gorski responded to similar language found in an article by Dossey (and two other magical thinkers) titled “The Mythology of Science-Based Medicine,” published by the Huffington Post. I’ll not repeat Dr. Gorski’s able rebuttal in any detail, and I’ve already written about much of what this matter brings to mind. Examples are here, here, and here on the perils of conflating science-based medicine and Evidence-Based Medicine (EBM); here on the false dichotomy of modern medicine vs. “CAM”; here on a concise definition of “CAM”; here and here on the mischief spawned by demands to “ruthlessly apply science,” in the narrow, EBM sense of the word, to implausible health claims; here (point #7) and here regarding the tu quoque fallacy, the “10-20% empirically proven” claim, and the risks of modern health care; here (scroll down to “this week’s entry”) and here, regarding some of Dossey’s own opinions about science and the future of medicine.

For now I’ll elaborate on a few points. These pertain not only to Dr. Dossey but also to myths common to the advocacy of pseudomedicine, so I hope to provide some useful information.

Credibility alert: the following post contains assertions and speculations by yours truly that are subject to, er, different interpretations by those who actually know what the hell they’re talking about when it comes to statistics. With hat in hand, I thank reader BKsea for calling attention to some of them. I have changed some of the wording—competently, I hope—so as not to poison the minds of less wary readers, but my original faux pas are immortalized in BKsea’s comment.

Lies, Damned Lies, and…

A few days ago my colleague, Dr. Harriet Hall, posted an article about acupuncture treatment for chronic prostatitis/chronic pelvic pain syndrome. She discussed a study that had been performed in Malaysia and reported in the American Journal of Medicine. According to the investigators,

After 10 weeks of treatment, acupuncture proved almost twice as likely as sham treatment to improve CP/CPPS symptoms. Participants receiving acupuncture were 2.4-fold more likely to experience long-term benefit than were participants receiving sham acupuncture.

The primary endpoint was to be “a 6-point decrease in NIH-CSPI total score from baseline to week 10.” At week 10, 32 of 44 subjects (73%) in the acupuncture group had experienced such a decrease, compared to 21 of 45 subjects (47%) in the sham acupuncture group. Although the authors didn’t report these statistics per se, a simple “two-proportion Z-test” (Minitab) yields the following:

Sample X N Sample p

1 32 44 0.727273

2 21 45 0.466667

Difference = p (1) – p (2)

Estimate for difference: 0.260606

95% CI for difference: (0.0642303, 0.456982)

Test for difference = 0 (vs not = 0): Z = 2.60 P-Value = 0.009

Fisher’s exact test: P-Value = 0.017

…

Wow! A P-value of 0.009! That’s some serious statistical significance. Even Fisher’s more conservative “exact test” is substantially less than the 0.05 that we’ve come to associate with “rejecting the null hypothesis,” which in this case is that there was no difference in the proportion of subjects who had experienced a 6-point decrease in NIH-CSPI scores at 10 weeks. Surely there is a big difference between getting “real” acupuncture and getting sham acupuncture if you’ve got chronic prostatitis/chronic pelvic pain syndrome, and this study proves it!

The National Center for Complementary and Alternative Medicine (NCCAM) has posted three essays about its latest “strategic planning process,” and has invited “stakeholders” to make comments. I have previously made my own opinions clear,* as have fellow bloggers Gorski, Novella, Lipson, and Sampson: the best strategic plan for the NCCAM would be to extinguish itself. Since politics makes that plan unlikely, there are strategies that could minimize the considerable harm now done by the Center, while possibly offering a modest benefit. In summary:

For both scientific and ethical reasons the NCCAM must dispense with trials of highly implausible claims. It should start by abandoning the ongoing Trial to Assess Chelation Therapy (TACT), its largest and most expensive trial yet, and one that has proven to place experimental subjects in considerable danger. It should publicly acknowledge such mistakes and explain why they must not be repeated—no matter how much political pressure there may be to do so.

The Center should use its website’s Health Information function to explain what’s known, rather than continue its customary practice of putting the best possible slant on most “CAM” claims, no matter how absurd or disproven.

The Center should address aspects of “CAM” advocacy that it has previously avoided, the most important being the close affiliation of such advocacy with the anti-vaccination (and autism quackery) movement. The NCCAM should consider itself an important source of rational information for a public that is currently, and dangerously, misled about immunizations. A related example of mischievous “CAM” advocacy, so far also ignored by the Center’s website, involves an imagined, sinister cartel of physicians, the AMA, pharmaceutical companies, and the FDA. The NCCAM should vigorously debunk such myths by providing facts and data.

The Center should pursue the question of why some people are stubbornly attracted to implausible, unproven, and/or inert treatments. Wally Sampson suggested this idea years ago. It is one of many legacies of the late Barry Beyerstein, among others, whose writings could serve as a template for legitimate NCCAM research topics.

The NCCAM’s Charter and its boosters in Congress make such strategies exceedingly unlikely, as explained here. Therefore, in this and two subsequent postings I’ll address a few of the assertions made in each of the Center’s three “big picture” essays. These will not be comprehensive critiques of those essays, which would require deconstructions of nearly every sentence.

Forgive the departure from my usual verbosity. I’m on my way to a meeting, and I don’t have the time. Today I’ll report disturbing content found in health care bills that are competing for passage in Washington. Thanks to Linda Rosa for keeping our attention on language in one of the Senate bills: “S.1679 – Affordable Health Choices Act,” sponsored by (guess who?) Senator Tom Harkin (D-IA). According to Linda, Harkin and supporters will attempt to merge his bill with Baucus’s. Here are some of the choice passages in Harkin’s 800+ page bill (emphasis added):

A group health plan and a health insurance issuer offering group or individual health insurance coverage shall not discriminate with respect to participation under the plan or coverage against any health care provider who is acting within the scope of that provider’s license or certification under applicable State law. This section shall not require that a group health plan or health insurance issuer contract with any health care provider willing to abide by the terms and conditions for participation established by the plan or issuer. Nothing in this section shall be construed as preventing a group health plan, a health insurance issuer, or the Secretary from establishing varying reimbursement rates based on quality or performance measures.

…(4) ensure that the health team established by the entity includes an interdisciplinary, interprofessional team of health care providers, as determined by the Secretary; such team may include medical specialists, nurses, nutritionists, dieticians, social workers, behavioral and mental health providers (including substance use disorder prevention and treatment providers), doctors of chiropractic, licensed complementary and alternative medicine practitioners, and physicians’ assistants;

…(F) provide coordination of the appropriate use of complementary and alternative (CAM) services to those who request such services;CommentsClose CommentsPermalink

…(H) provide local access to the continuum of health care services in the most appropriate setting, including access to individuals that implement the care plans of patients and coordinate care, such as integrative health care practitioners; (more…)

Last week I discussed the dismal results of the “Gonzalez Trial” for cancer of the pancreas,* as reported in an article recently posted on the website of the Journal of Clinical Oncology. I promised that this week I’d discuss “troubling information, both stated and unstated [in the report],” and also some ethical issues. More has come to light in the past few days, including Nicholas Gonzalez’s own voluminous, angry response to the JCO article. I’ll comment upon that below, but first a brief review.

The trial was begun in 1999 under the auspices of Columbia University, after Rep. Dan Burton had pressured NCI Director Richard Klausner to fund it. It was originally conceived as a randomized, controlled trial comparing the “Gonzalez Regimen” to standard chemotherapy for cancer of the pancreas. In the first year, however, only 2 subjects had been accrued, purportedly because those seeking Gonzalez’s treatment were not willing to risk random assignment to the chemotherapy arm. In 2000, the protocol was changed to a “prospective, cohort study” to allow potential subjects to choose which treatment they would follow. Gonzalez himself was to provide the ‘enzyme’ treatments.

After that there was little public information about the trial for several years, other than a few determination letters from the Office of Human Research Protections and a frightening account of the experience of one subject treated by Gonzalez. By 2006 or so, those of us who pay attention to creeping pseudomedicine in the academy were wondering what had become of it. About a year ago we found out: the trial had been quietly “terminated” in 2005 after it met “pre-determined stopping criteria.” As explained here, that meant that the Gonzalez group had not fared well.

Four years after the trial’s ‘termination,’ the report was finally published: The Gonzalez cohort had not only fared much worse than the cohort that received chemotherapy, but it had fared worse than a comparable group of historical controls. Here, again, is the survival graph from the JCO paper:

The Gonzalez group had also fared much worse in ‘quality of life’ scores, which included a measure of pain.

Now let’s read between the lines. Forgive me for taking shortcuts; I’m a little pressed for time. (more…)

Review

One of the more bizarre and unpleasant “CAM” claims, but one taken very seriously at the NIH, at Columbia University, and on Capitol Hill, is the cancer “detoxification” regimen advocated by Dr. Nicholas Gonzalez:

Patients receive pancreatic enzymes orally every 4 hours and at meals daily on days 1-16, followed by 5 days of rest. Patients receive magnesium citrate and Papaya Plus with the pancreatic enzymes. Additionally, patients receive nutritional supplementation with vitamins, minerals, trace elements, and animal glandular products 4 times per day on days 1-16, followed by 5 days of rest. Courses repeat every 21 days until death despite relapse. Patients consume a moderate vegetarian metabolizer diet during the course of therapy, which excludes red meat, poultry, and white sugar. Coffee enemas are performed twice a day, along with skin brushing daily, skin cleansing once a week with castor oil during the first 6 months of therapy, and a salt and soda bath each week. Patients also undergo a complete liver flush and a clean sweep and purge on a rotating basis each month during the 5 days of rest.

Veteran SBM readers will recall that in the spring of 2008 I posted a series of essays* about this regimen and about the trial that compared it to standard treatment for subjects with cancer of the pancreas. The NIH had funded the trial, to be conducted under the auspices of Columbia, after arm-twisting by Rep. Dan Burton [R-IN], a powerful champion of quackery, and much to the delight of the “Harkinites.”

In the fall of 2008 I posted an addendum based on a little-known determination letter that the Office for Human Research Protections (OHRP) had sent to Columbia during the previous June. The letter revealed that the trial had been terminated in October, 2005, due to “pre-determined stopping criteria.” This demonstrated that Gonzalez’s regimen must have been found to be substantially worse than the current standard of care for cancer of the pancreas, as ineffective as that standard may be. I urge readers who require a review or an introduction to the topic to read that posting, which also considered why no formal report of the trial had yet been made available.

Now, finally, the formal report has been published online by the Journal of Clinical Oncology (JCO):

Readers acquainted with popular culture know that such inane, annoying phrases are typical of American women’s magazines. Thus it may be surprising to learn that only three entries were quoted from sources clearly recognizable as such: numbers 3 and 6 from Cosmopolitan, and number 11 from Glamour. The rest were found in WebMD: the Magazine:

The magazine appears to have been introduced in 2005. According to its masthead page,

WebMD’s mission is to provide objective, trustworthy, and timely health information. Our website and magazine provide credible content, tools, and in-depth reference material about health subjects that matter to you. We are committed to providing information on a wide variety of health topics, all of which are reviewed by our board-certified physicians.

Every physician I know receives a “COMPLIMENTARY WAITING ROOM COPY” each month; the 3 or 4 waiting rooms that I’ve perused have been amply stocked. I suspect that most office managers are happy to be provided with free reading material that seems appropriate for patients, and that most physicians haven’t given the magazine more than a passing glance. The problem is that the magazine, like the consumer website of the same name, offers a mixture of accurate-if-mundane information, misleading health claims, exaggerated nutritional advice, unwarranted fear-mongering, and pseudoscientific nonsense. I’ll limit examples and comments to the final four categories. (more…)

A Neglected Skeptic

Most Westerners—Michael DeBakey and John Bonica being exceptions—who observed ‘acupuncture anesthesia’ in China during the Cultural Revolution seem to have failed to recognize what was going on right under their noses.

I should have added—and I now have—Arthur Taub’s name to that tiny, exceptional group. Taub, a neurologist and neurophysiologist at Yale, was a member of a delegation of Americans sent to China to observe ‘acupuncture anesthesia’ in May of 1974, about a year after Dr. Bonica‘s visit. The delegation included several prominent anesthesiologists. Their report, Acupuncture Anesthesia in the People’s Republic of China:A Trip Report of the American Acupuncture Anesthesia Study Group, was published in 1976 and is available in its entirety here. Excerpts follow (emphasis added):

Pain is a subjective experience. Judging whether an individual is in a state of pain depends on observations of the subject’s behavior, including verbal reports to the observer…When there is no evidence of pain, the observer can adopt one of three positions: (more…)