Saturday, January 4, 2014

Is Science Self-Correcting? Not Necessarily

Steven Hsu blogs a 2012 paper by John Ioannidis, the meta-analysis and replication guru who has become famous by identifying the statistical skeleton's in medicine's closet. Improving this is critical not only for medical science's ability to get results, but also in terms of science's credibility in the eyes of the public. This latter consideration has never been so critically important as now.

Excerpted from the abstract:

...self-correction does not always happen to scientific evidence by default....History suggests that major catastrophes in scientific credibility are unfortunately possible...Careful evaluation of the current status of credibility of various scientific fields is important in order to understand any credibility deficits and how one could obtain and establish more trustworthy results. Efficient and unbiased replication mechanisms are essential for maintaining high levels of scientific credibility...In the absence of replication efforts, one is left with unconfirmed (genuine) discoveries and unchallenged fallacies. In several fields of investigation, including many areas of psychological science, perpetuated and unchallenged fallacies may comprise the majority of the circulating evidence.

Words to Live By

"...there is good and bad speculation, and this is not an unparalleled activity in science...Those scientists who have no taste for this sort of speculative enterprise will just have to stay in the trenches and do without it, while the rest of us risk embarrassing mistakes and have a lot of fun." - Dan Dennett