Sunday, October 02, 2011

More evidence that published results are not to be trusted published in Nature. Ionnides found long ago that a large fraction of top medical studies are reversed in five years, and now a study finds that a majority of medical studies in top journals cannot be replicated. Bem's recent demonstration of psychic powers in Cornell undergradutes published in the top psych journal has forced to re-evaluate all experimental data, precisely because he did everything by the book. I've long found that more and more well established results esp in psychology have not held up under scrutiny. Johnathan Schooler has argued the absurdity in Nature of all places this is because the universe reacts whenever humans discover something, when the far more obvious conclusion is that biases in scientific publishing lead us to over-report false positives.

I would like to think economics does better, just because the standards (at least at top journals) are so high that people take years to publish, but I'm sure that if it is better, its only marginally better.

I suppose the only defense I have for Schooler's idea is that this was the premise of Second Foundation, the Asimov sci-fi book that helped inspire me to become an economist, whose precedent was that a bunch of meddling mathematicians (i.e. economists) were secretly manipulating the universe, so the rules of the universe did seem to react to human discoveries. And I suppose even less crazy, since Edmund Phelps won the Nobel Prize in Economics for showing that the Philips Curve (relationship between unemployment and inflation) stops working once people become aware that the Federal Reserve is aware of it. But there the mechanism is clear.