Friday, 1 April 2016

In the wake of recent failures to repeat some of psychology’s most famous findings, not to mention a few cases of outright research fraud, it’s been claimed that psychological science is in a bit of state. Many psychologists have responded by proposing ways to improve research practices, such as making data freely available online, and preregistering planned methods, to avoid issues of data tinkering later on. However, not everyone actually agrees that psychology is in crisis – at least some psychologists take a more rosy view and think the replication problem has been overblown, as illustrated by a recent ebullient opinion piece published in Science.

We hear a lot of commentary about all this from just a few high profile individuals, but no one really knows what the average psychology researcher really thinks. To find out, a team of psychologists in the UK recruited hundreds of psych researchers around the world to complete what’s known as the “implicit association test” tailored to reveal subconscious attitudes towards psychology. The idea was to find out what psychology researchers think of psychology at a subconscious level. Also, in keeping with the growing awareness of the importance of replicability in science, it was planned in advance that a second team in the USA would subsequently perform the same test with hundreds more international researchers. Dr Cass Andra, a reforming psychologist and leader of the UK arm, said she expected to find that psychologists are in denial about the crisis in their field.

The test involved psychologists pressing one of two keyboard keys as fast as possible whenever they saw different categories of word on-screen – words pertaining to psychology or other sciences, and positive and negative words. On some trials, the same key was allocated to psychology terms (e.g. “social psychology”) and positive words (“robust”), with the other key allocated to other sciences and negative words. On other trials, the set-up was switched.

The main finding in the UK arm of the research is that psychology researchers showed an implicit positive bias towards psychology research – that is, they showed their fastest response times when the same key was allocated to psychology terms and positive words, suggesting that they see psychology research in a positive light. Andra and her colleagues said this was as they expected but also extremely worrying – suggesting that deep down psychologists are confident in their discipline and do not see any need for reform.

However, the American replication attempt failed. These researchers, who also recruited psychology researchers from around the world, made the exact opposite finding – in this case, psychologists were particularly slow to respond when the same key was allocated to psychology terms and positive words, and much quicker when the same key was used for responding to psychology terms and negative words. Professor Polly Anna, who is sceptical about the idea of a replication crisis and a well-known figure through her popular TED talks, said she was disappointed by this result – “Firstly, it’s disappointing from a purely methodological point of view that we failed to replicate the first phase, but also I'm sorry to see that psychologists seem to believe deep-down that their field is in trouble. I think this shows the harm to morale that's been done by all the talk of a replication crisis.”

Unfortunately, despite the initial collaborative spirit, the two teams are now in dispute. In a surprise move, the American team led by Professor Anna, has written a letter to The International Journal of Psychological Research calling for their own replication attempt to be retracted. Anna and her colleagues acknowledged that they might not have sufficient training in the implicit association test, and that it’s possible their own anxieties influenced their participants, thus invalidating their results. “Our finding that psychology researchers think psychology is in crisis is questionable – it can take skill and creativity to get the right results sometimes, and hand on heart, we might have lacked those things here ” Anna and her team told us. "We think the British finding, showing positive views among psychologists toward psychology, should stand, and we want our own replication attempt removed from the record".

But in turn, the British researchers have written a letter to the journal calling for a retraction of the American's retraction letter. “While we would normally hope for a successful replication attempt," the letter states, "we actually welcome the US finding because it helps to show once again the difficulty of conducting replicable psychological science. It may well be the case that their finding that psychologists think psychology is not robust is more robust than our finding that psychologists think psychology is robust. Either way, we hope the message gets through that we need to work together to make psychology more robust."
_________________________________