03 July 2014

Taking responsibility for peer review, STAP edition

I have both hearts and darts for Nature in the wake of its retraction of two stem cell papers from earlier this year, now known as the “STAP” (stimulus-triggered acquisition of pluripotency) papers.

I have many more darts for Nature than hearts, particularly for this astonishing statement:

We have concluded that we and the referees could not have detected the problems that fatally undermined the papers.

Then what is the point of peer review? Sure, I accept that detecting problems may be difficult in practice, but I would feel a lot better if Nature at least acknowledged that it was at least possible that they could have caught the mistakes. It seems strange to claim that nobody could have done better after reviewing this timeline from the PubPeer blog:

Earlier this year, in an article on post-publication peer review, I wrote that when faced with criticism through post-publication peer viewer, about the tendency of journals to “cheerlead for (pre-publication) peer review.”

Even when faced with cases in which peer review failed to detect a highly problematic paper, editors rarely change their journal’s policies to improve the peer review process.

Consequently, I want to congratulate Nature for trying to improve their review processes.

(O)ur approach to policing (image manipulation) was never to do more than to check a small proportion of accepted papers. We are now reviewing our practices to increase such checking greatly, and we will announce our policies when the review is completed.

We will see if this change alone is enough, or whether journals need to go further in upadting peer review. Publishing peer reviews is mentioned in the news article, and that might be a good step to consider.

2 comments:

I've argued before that the main value of peer-review is as a hazing ritual. It doesn’t exist because of any intrinsic value it has, and it certainly isn’t there for the benefit of the recipient. It’s basically a way to draw a line between In and Out. Something for the inductee to endure as a way of proving he’s made of the Right Stuff.

So: the principle value of peer-review is that it provides an opportunity for authors to demonstrate that they are prepared to undergo peer-review.