The Why Files -- whyfiles.org

POSTED 4 JUNE 2009

The science of fraud in science

If science is all about facts and theories, hypotheses and proofs, why do we see so much cheating? The high-profile stem cell fraud by Korean researcher Woo Suk Hwang provided glaring evidence that even an august scientific journal like Science can be scammed -- for a while at least.

A protest against cloning fraudster Hwang Woo Suk at his own university, February 20 2006.

Now, a study in Public Library of Science tries to put some numbers on the scientific crimes by summing up results from 21 surveys that asked scientists whether they or their colleagues had falsified research. Daniele Fanelli of the University of Edinburgh says that as he combed through the scientific literature, he was looking for "quantitative survey data assessing how many researchers have committed or observed colleagues committing scientific misconduct in the past."

The many forms
of research misconduct

Fanelli was seeking outright fabrication (invention), falsification (corruption of data) and other "questionable research practices," such as "dropping observations or data points from analyses based on a gut feeling that they were inaccurate" or pitchforking through a heap of data to prove something -- anything. He did not look at plagiarism or other forms of misconduct, like exploiting subordinates (grad students -- are you reading?) or misusing research funds.

All told, Fanelli calculated that 2 percent of scientists had admitted inventing or altering data to support their hypotheses, and that, he says, "is almost certainly an underestimate." Science, after all, is founded on accuracy and dispassionate search for truth, and admitting chicanery is tantamount to a cardinal questioning the primacy of Jesus.

Less clear is how to interpret a second finding: that 14 percent of survey respondents knew of a colleague or associate who had cooked data. That could be an accurate reflection of how much fraud is occurring, but it could also overstate the fraud: Five people could know of one faker, or the respondents could also be more judgmental toward others. "People tend to see the behavior of others as less honest than their own behavior," says Fanelli, who works in the Institute for Study of Science, Technology and Innovation at Edinburgh.

Left: See Keys, A. in bibliography; right: See Yerushalmy et al.

A classic example of cherry picking data. Left: a coronary heart disease researcher omitted countries that didn't fit his predictions. Right: The graph showing all countries in the study is more diffuse, indicating a more complex relationship between diet and coronary heart disease.

Research by Harvard child psychiatrist Joseph Biederman supported a massive increase in prescriptions for antipsychotics for bipolar disorder in children. Biederman failed to disclose most of the $1.6 million he received from drugmakers between 2000 and 2007, according to an investigation by Sen. Charles Grassley. The influential doctor is being investigated by Harvard and the National Institutes of Health (more>>)

Medical (research) malpractice

Disturbingly, Fanelli found the highest rate of reported falsification in medical studies, which may reflect economic pressure from physician-entrepreneurs and especially drug companies that have a profit-and-loss stake in the results. He notes that the U.S. Food and Drug Administration found flaws in 10 percent to 20 percent of studies performed between 1977 and 1990.

This focus did not surprise medical ethicist Norman Fost of the University of Wisconsin-Madison, who told us, "The most worrisome finding is that misconduct is highest in clinical medicine and pharmacology, as these incidents are more likely to have adverse effects on individual's lives than misconduct in basic science, where the connections with human health are more remote."

"Money talks, and nobody walks," seems to be the MO in modern drug research, continued Fost, an expert in research ethics, who points to "the widespread incidence and variety of misconduct by the leading pharmaceutical companies. They now control the research process at almost every stage, and have shown a willingness to produce misleading results at every stage: in the design of studies; collection of data; suppression of unwelcome results; ghostwriting research articles, editorials and letters; misrepresenting results to practicing physicians ... and misleading direct-to-consumer advertisements."

Even the recent emphasis on "disclosing" conflicts of interest "has little apparent effect on these behaviors," says Fost, "and there is evidence that it has a perverse effect, by increasing trust in the audience rather than alerting the audience to be more skeptical: 'If he's disclosing his conflicts, he must be very honest,' and by legitimizing the researcher/author's participation in the misconduct: 'I have disclosed my conflicts; therefore I can consider myself to be behaving ethically.'"

A poor climate for science

The new study may attract attention from interest groups concerned, for example, with politically charged scientific issues. "I've already seen a couple of creationist and climate-skeptic websites mentioning the study," says Fanelli. "I am concerned about this, as my goal is not to undermine the credibility of science, but to clarify the issues."

The scientific community has yet to find a cure-all for questionable data. Some advocate outside audits like those the FDA used to expose flawed research, "but how to do that, and who would pay for it, are open questions," Fanelli says. "It is basically not done."

Even leaving corporate interests aside, bias is unlikely to disappear entirely, Fanelli notes. "Science is made by human beings, who are driven by hopes, desires, expectations and their own interests. There is an intrinsic motivation toward bias in any kind of research. Understanding which factors could drive bias might us help reduce it."