Replicating the Replicators: Is Psychology in Crisis or Not?

This article was originally published on The Psych Report before it became part of the Behavioral Scientist in 2017.

Science recently published a critique of the findings of the Reproducibility Project: Psychology. The Reproducibility Project, conducted by the Open Science Collaboration (OSC), gained significant attention last fall for its sobering assessment of the state of psychological science, reporting that less than half of the 100 studies it attempted to replicate did so.

The recent critique, authored by Dan Gilbert, Gary King, Stephen Pettigrew, and Tim Wilson suggests the authors of the original report neglected to take into account a number of key factors in their study design and analysis, including key sources of random error, underpowered replication attempts, and biases in replication methodology. These failures, Gilbert and his co-authors argue, led the OSC to incorrectly portray the rate of replication within psychology as a “crisis.” Members of the OSC replied to the critique, standing by their original conclusion, and offering a critique of their own on Gilbert and his co-authors’ methods and analyses.

Evan Nesterak is the editor-in-chief at the Behavioral Scientist. In 2013, he co-founded the non-profit psychology news website The Psych Report and in 2016 he helped launch the Philadelphia Behavioral Science Initiative. Evan earned his B.A. from Swarthmore College in 2009, where he studied psychology and statistics.