Skeptimediais a commentary on
mass media treatment of issues concerning science, the
paranormal, and the supernatural.

March 11, 2009.
Last November I wrote about some central
problems with medical
studies and how such studies can be misinterpreted or misapplied. The
problems I focused on concerned such things as statistics,
sample sizes, replication, control groups, randomization, and
what to do about studies that produce contradictory results: the
usual suspects. Two recent media reports remind us that there
are serious problems that can arise, and probably should be
considered, before we begin analyzing and evaluating a new
medical study: bias and fakery.

bias

Medical journals take
it on faith that the researchers who submit their work have
actually done what they say they've done. The journals rely on
the honesty of the scientists, rather than reports of
investigators, when it comes to accepting the data presented to
them. This is really no different from the way journalists
handle much of the information that is handed to them by
interested parties. In most cases, whether the government report
or the corporation press release is true is left up to the
reader to determine. If newspapers, for example, had to
investigate every government report to determine that it wasn't
faked, they'd never publish anything. On the other hand, I take
it for granted (though maybe I shouldn't) that most journalists
are aware that any information given them by an interested party
is going to be biased. Medical journals have assumed that
studies that are industry-funded are more likely to be biased,
in a positive way, than government-funded studies. Because of
the massive quantity of material published each month in medical
journals, readers of those journals are likely to focus on
reading the prestigious ones. Readers are likely to assume that
these prestigious journals would be less likely to have an
industry bias in their selection of articles for publication.
That assumption is questionable, at least in the UK, according
to a new study published in the
British Medical Journal(12 Feb 2009).

The British
Medical Journal this week publishes a complex study that is
quietly one of the most subversive pieces of research ever
printed. It analyses every study ever done on the influenza
vaccine - although it's reasonable to assume that its results
might hold for other subject areas - looking at whether funding
source affected the quality of a study, the accuracy of its
summary, and the eminence of the journal in which it was
published....

We already know that
industry-funded studies are more likely to give a positive
result for the sponsors' drug, and in this case too,
government-funded studies were less likely to have conclusions
favouring the vaccines. We already know that poorer quality
studies are more likely to produce positive results - for drugs,
for homeopathy, for anything - and 70% of the studies they
reviewed were of poor quality. And it has also been shown that
industry-funded studies are more likely to overstate their
results in their conclusions.

But Tom Jefferson and
colleagues looked, for the first time, at where studies are
published. Academics measure the eminence of a journal by its
"impact factor": an indicator of how commonly, on average,
research papers in that journal go on to be referred to by other
research papers. The average journal impact factor for the 92
government-funded studies was 3.74; for the 52 studies wholly or
partly funded by industry, the average impact factor was 8.78.
Studies funded by the pharmaceutical industry are massively more
likely to get into the bigger, more respected journals.

Jefferson et al.
found that there's no difference between industry-funded and
government-funded studies with respect to rigor, quality, size
of samples, or where they submit their articles, yet
drug-company funded articles are more than twice as likely to be
accepted by the high prestige journals. I don't know why this is
the case, but two suggested reasons are: 1. Editors are
influenced by potential advertisers; and 2. Editors are
influenced by authors who are top-tier researchers and drug
companies hire the big names whenever possible. There must be an
"impact factor" for authors based on how many times an author is
cited by other authors. So, it should be easy to determine
whether studies submitted by industry-funded authors are
significantly more eminent than those submitted by
government-funded scientists.

fakery

Baystate Medical
Center in Springfield, Massachusetts, has asked several
anesthesiology journals to retract 21 drug studies published
between
1996 and 2008 by anesthesiologist Scott S. Reuben, M.D., a
pioneer in the area of multimodal analgesia.*
The studies were funded by Pfizer, Merck, and Wyeth. Reuben was
also paid by Pfizer as a speaker to promote its products.*
Raymond F. Kerins Jr., a Pfizer spokesman, said: "It is very
disappointing to learn about Dr. Scott Reuben's alleged
actions."

Reuben is suspected
of falsifying some of his data, including the names of patients
and co-authors. Evan Ekman, MD, an
orthopedic surgeon in Columbia, S.C., told Anesthesiology
News that his name was forged on two of the retracted
papers. Dr. Hal Jenson, Baystate's chief academic officer, said
that the investigation of Rebuen's work found that in many cases
"there was no clinical trial because there were no patients."

"This would be
the largest research fraud in anesthesia," said Dr. Steven
Shafer, editor of the journal Anesthesia and Analgesia.
"Doctors have been using [his] findings very widely. His
findings had a huge impact on the field. The act of fabricating
data is so difficult for me to comprehend. It's beyond my
ability to imagine."*

Reuben is on
"indefinite leave" from the hospital.

Dr. Reuben has been
an extremely active and visible figure in multimodal analgesia,
particularly as an advocate for its use in minimally invasive
orthopedic and spine procedures. His research has provided
support for several mainstays of current anesthetic practice,
such as the use of nonsteroidal anti-inflammatory drugs and
neuropathic agents instead of opioids and preemptive analgesia.
Dr. Reuben has also published and presented data suggesting that
multimodal analgesia can significantly improve long-term
outcomes for patients.*

Dr. Shafer said: “We
are left with a large hole in our understanding of this field.
There are substantial tendrils from this body of work that reach
throughout the discipline of postoperative pain management.
Those tendrils mean that almost every aspect will need to be
carefully thought through.”

Reuben's misconduct
was discovered during an internal review at Bayview that was
triggered during a routine review of research summaries. Reuben
was going to present a summary of work that had not been
approved by an internal hospital review board that oversees
research on patients. That led to a full-scale investigation
that uncovered more than a decade of medical malfeasance by Dr.
Reuben.

Fakery, of course,
occurs much less frequently than bias and has much more impact, but both
can be related to
industry-funded medical research. And, just as medical journal
editors rely on the integrity of those who submit articles to
them, the public relies on the integrity of the journals, the
researchers, and the media. When the faith of the journal
editors is shattered by news of bias or fraud, it is not only
their readers who are affected. Public trust in
science-based medicine is also shattered.