Menu

Peer Review Gone Wrong, Again

The anonymous peer review process that’s used to determine whether academic research articles are published or presented is supposed to be a neutral process. But research on peer review has revealed many problems with the process, such as biased outcomes, and excessive lengths of time to get articles accepted. This week, there was a stunning example of another problem with the process – sexist reviews.

Researchers Fiona Ingleby and Megan Head conducted a study of gender-based differences in the progression of 244 researchers from Ph.D. programs into postdoctoral jobs. They submitted a research article based on the results of their study to an academic journal operated by the online publisher PLOS ONE. Every submission to PLOS ONE is sent to an academic editor, who then chooses the anonymous peer reviewers to evaluate the submitted manuscript. PLOS ONE uses “single-blind” peer reviews, which means that the reviewers know who the authors of the paper are, but the authors do not know who the reviewers are.

PLOS ONE‘s editorial policy states, “The majority of PLOS ONE submissions are evaluated by two external reviewers, but it is up to the Academic Editor to determine the number of reviews required.” And it appears that Ingleby and Head’s manuscript went to a single reviewer – who said that the article would be better if it had a male co-author. Seriously.

Ingleby used Twitter to post this quote from the anonymous peer review:

It would probably also be better to find one or two male biologists to work with (or at least obtain internal peer review from, but better yet as active co-authors), in order to serve as a possible check against interpretations that may sometimes be drifting too far away from empirical evidence into ideologically based assumptions.

It could perhaps be the case that 99% of female scientists make a decision in mid-life that spending more time with their children is more important to them than doing everything imaginable to try to get one of the rare positions at the utter pinnacle of their field. Or perhaps it is the case that only some small portion of men (and only men) have the kind of egomaniac personality disorder that drives them on to try to become the chief of the world at the expense of all else in life.

The review was accompanied by a rejection letter from the academic editor – which, according to this article, stated: “the qulaity [sic] of the manuscript is por [sic] issues on methodologies and presentation of resulst [sic]”.

One of Fiona Ingleby’s Tweets that included excerpts from the anonymous peer review. (credit: @FionaIngleby/Twitter)

Ingleby and Head filed an appeal of the article’s rejection, based on the blatant sexism of the peer review. After Ingleby posted excerpts from the review on Twitter, there was a storm of outrage – including many sarcastic posts with the #AddMaleAuthorGate tag. PLOS ONE then announced that it had removed the review from its record, sent the paper out for a new review, removed the reviewer from its database, and asked the academic editor to “step down” from its editorial board.

(Although these actions were focused on the sexist peer review, PLOS ONE should also be embarrassed that it appointed an academic editor who thought it was okay to send a communication as illiterate as that rejection letter.)

I can guarantee you that at some point in their career every researcher will receive an anonymous peer review that is unfair, or that reads as if the reviewer didn’t bother to actually read the manuscript, or that recommends rejection of the manuscript because the results don’t fit the reviewer’s or their friends’ view or the world, or…..there’s an endless list of reasons why articles might be rejected – reasons that have nothing to do with the actual quality of the research described in the article.

But I have to admit that in my own experience with anonymous peer reviews – which includes soliciting reviewers and reviews, and making decisions on whether to accept a submission – I have never seen a review as arrogantly clueless as this one. It’s frankly a little worrying that someone, somewhere, not only has this offensive attitude but also feels it’s acceptable to provide such irrelevant and unfounded evaluations of another researcher’s work. And presumably the reviewer is a highly educated person with considerable expertise and research experience. (Here is the list of PLOS ONE‘s editorial board members, to give an idea of the sort of professional qualifications that its peer reviewers and editors are expected to have.) I can’t help but wonder in what other ways this reviewer’s biases might be affecting their work, or how they treat their colleagues, or even how they treat their students.

Ingleby and Head have done a very brave thing in publicizing their experience. Bias might never be completely eliminated from the peer review process – but the damage from biased reviews can only be counteracted if those biases are identified and exposed. Researchers might fear damage to their own careers if they speak out about unfair anonymous reviews of their work, but acknowledgement and discussion of the problem is the only way that it will ever begin to be addressed.

I never heard of PLOS ONE before. I just checked what they have in my field and it seems to be very low quality work but I guess it may be different in other fields. To have documented empirical evidence of bias is certainly important but I’m more interested in what can be done about it? Blind auditions in music were able to reduce gender bias. But nowadays when everything is searchable, “double blinded” reviews are not realistic.