Monday, April 9, 2012

Magnetic Healing and the Importance of Reproducibility, Then and Now

As a scientist, reproducibility is the little demon always lurking in your shadow. At the helm of a series of often delicate experiments and complex analyses, you always wonder what will be discovered by those in your wake. Did you carry out your study properly? Were there any variables you didn't account for? Above all, can an equally competent scientist independently reproduce your results?

In the past couple of years I’ve noticed increasing concern for reproducibility. It’s an essential part of science, since it
helps to cancel out the effects of human fallibility and conflicts of
interest. The more often a research
finding is reproduced, the stronger a foundation it forms for future knowledge-building.

The problem is that most scientific journals do not publish
straightforward reproducibility studies, with occasional high-profile
exceptions -- for example, the apparently erroneous measurements of
faster-than-light neutrinos mentioned in last week’s Nature podcast. Clearly, we need a place for reports like these to be routinely filed so that the value of any and every study can be more
accurately judged. Last week's Science podcast mentioned one such initiative in the field of psychology, called PsychFileDrawer. I think every scientific field needs a
repository like this, and the reproducibility studies should be linked to the
original articles to minimize the time wasted following dead-end roads.

As scientists, we need to be our own worst critics, for two
main reasons:

People outside our own specialties are not
trained to critique our work properly; and

When external audits discover problems, the
whole of science suffers a blow to its reputation.

Of course, problems in scientific reproducibility are not a
new phenomenon. In his Compleated Autobiography,
Ben Franklin mentions the case of a quack named Franz Mesmer who popularized a form
of magnetic therapy based on his “theory” of animal magnetism:

We were much amiss’d in France for the past two years with
the pretended new art of healing by what was called Magnetisme Animale. The professor of this art, Mr. Mesmer, had in
a short time made nearly twenty thousand Louis d’ors by teaching and practicing
it. He proposed the presence of a
universal animal magnetic fluid that could heal the sick and prevent illness
through a simple laying of hands, gestures, and signs… The remedy met with
enough success to encourage the hopes of the ill; and even educated people,
including doctors and surgeons, admitted to the school of magnetism.

Franklin and others were curious and a little skeptical, so
on orders of the King of France, he and a group of independent experimenters of the
Academy of Science carried out their own study of Mesmer’s art of healing:

In our investigation, we tried to detect the presence of
the magnetic fluid; but this fluid was imperceptible to all the senses. The experiments we conducted on ourselves,
including blindfolded subjects, caused us to reject it absolutely as a cure of
illness. We concluded that the system of
magnetism did not cure anything; that both magnetism and his brilliant theory
exist only in the imagination; and that a spectacle such as this seemed to
transport us to the age and the reign of the fairies.

Unfortunately, Mesmer and his successors proceeded to
swindle many credulous patients after this report was published. Perhaps Mesmer earnestly believed in the technique. Maybe his patients derived some comfort from the practice. In any case, at least the study of the Academy of Science helped prevent magnetic healing from
becoming an accepted mainstream medical treatment.

Non-reproducible science is bad science, and bad science is
anti-science. It's better to publish nothing at all than to publish an incorrect result. As Franklin put it:

…one must not be indifferent to the ill-founded reign of
false opinion: the sciences, which grow larger with the truth, have even more
to gain by the suppression of an error.
An error is always a spoiled yeast which ferments and eventually
corrupts the substance in which it is introduced. But when this error leaves the empire of the
sciences to spread among the multitude, and to divide and agitate minds when it
presents a misleading way to heal the sick whom it discourages from looking
elsewhere for help, a good government has an interest in destroying it.

Given the explosion of scientific inquiry in our era, we
need a more concerted, systematic approach to finding the “spoiled yeast” and
removing it before it spreads. I think
these reproducibility repositories are a step in the right direction.

No comments:

Post a Comment

About Me

I'm a scientist, philosophical skeptic, and devotee of E.O. Wilson's notion of consilience -- that is, that the union of disparate branches of inquiry can be more than the sum of its parts. I write about science, technology, and evidence-based approaches to the world's problems.