MOST SCIENCE STUDIES APPEAR TO BE TAINTED BY SLOPPY ANALYSIS

September 18, 2007

We all make mistakes and if you believe medical scholar John Ioannidis -- an epidemiologist who studies research methods at the University of Ioannina School of Medicine in Greece and Tufts University -- scientists make more than their fair share. By his calculations, most published research findings are wrong.

What's more, the hotter the field of research the more likely its published findings should be viewed skeptically, he determined. Take, for example, the discovery that the risk of disease may vary between men and women, depending on their genes:

Studies have prominently reported such sex differences for hypertension, schizophrenia and multiple sclerosis, as well as lung cancer and heart attacks.

In research published recently in the Journal of the American Medical Association, Ioannidis and his colleagues analyzed 432 published research claims concerning gender and genes.

But upon closer scrutiny, almost none of them held up; only one was replicated.

Statistically speaking, science suffers from an excess of significance, says Ioannidas. Overeager researchers often tinker too much with the statistical variables of their analysis to coax any meaningful insight from their data sets. Further, to root out mistakes, scientists rely on each other to be vigilant. But findings too rarely are checked by others or independently replicated. Retractions, while more common, are still relatively infrequent. Findings that have been refuted can linger in the scientific literature for years to be cited unwittingly by other researchers, compounding the errors.