Replicating the Replicators: Is Psychology in Crisis or Not?

This article was originally published on The Psych Report before it became part of the Behavioral Scientist in 2017.

Science recently published a critique of the findings of the Reproducibility Project: Psychology.  The Reproducibility Project, conducted by the Open Science Collaboration (OSC), gained significant attention last fall for its sobering assessment of the state of psychological science, reporting that less than half of the 100 studies it attempted to replicate did so.

The recent critique, authored by Dan Gilbert, Gary King, Stephen Pettigrew, and Tim Wilson suggests the authors of the original report neglected to take into account a number of key factors in their study design and analysis, including key sources of random error, underpowered replication attempts, and biases in replication methodology. These failures, Gilbert and his co-authors argue, led the OSC to incorrectly portray the rate of replication within psychology as a “crisis.” Members of the OSC replied to the critique, standing by their original conclusion, and offering a critique of their own on Gilbert and his co-authors’ methods and analyses.

Since the critique and response were published, a number of people have attempted to digest the situation in which psychology currently finds itself. Brian Nosek and Elizabeth Gilbert, two of the authors on the original reproducibility report, published a follow-up piece, as did Dan Gilbert and his co-authors here. Uri Simonsohn added his thoughts, as did Sanjay Srivastava. Articles were also published in Nature, the New York Times, The Atlantic and New York Magazine, though Katie Palmer may have summed it up best in her piece for Wired: “Psychology is in Crisis Over Whether It’s in Crisis.” 

Disclosure: Dan Gilbert is on The Psych Report’s Advisory Board.