Assessment of Signing Peer Reviews in Principle and in Practice at Public Library of Science (PLOS) Journals

Elizabeth Seiver,1 Helen Atkins1

Objective To investigate the rate at which Public Library of Science (PLOS) peer reviewers at 3 medical journals chose to sign or not sign their reviews, thus revealing or not revealing themselves to the authors of an article, and whether authors’ and reviewers’ stated values about signing reviews matched their behavior.

Design Historical review signing data from 3 PLOS journals (PLOS Computational Biology, PLOS Medicine, and PLOS ONE) from mid-2013 through 2016 were analyzed. In addition, 1-click multiple choice surveys were appended to current system-generated emails, which were sent at the time of manuscript and review submission, to authors and reviewers; these surveys asked them about their general experience with, and preference for, signing reviews. The survey landing page included detailed follow-up questions about their selection, and comments were coded for qualitative analysis. The signing and survey data sets did not share unique identifiers and thus were not matched 1:1.

Results Of 451,306 total reviews analyzed, 34,561 (7.7%) were signed. This was higher for PLOS Medicine than for the other 2 journals (Table). Although the PLOS Computational Biology authors reported that they had received signed reviews (27 of 71 [38.0%]), the reviewers reported that they did not usually sign their reviews (75 of 627 [12.0%]). Combining all 3 PLOS journals, we found that 509 of 1072 authors preferred to receive signed reviews (47.5%) and that 372 of 2359 reviewers reported usually signing them (15.8%). In follow-up comments, the reviewers who reported usually signing reviews made the argument that signing their reviews improved accountability and constructiveness; the reviewers who reported usually not signing cited as their motivation the ability to be more honest and fear of retribution. Many reviewers who had not signed reviews simply stated that they had never been asked before or were not sure of the benefits. Authors who favored receiving signed reviews valued having the additional information on the reviewer’s area of expertise and potentially having more open communication.

Conclusions The 3 PLOS journals represent different research communities with diverging attitudes and behavior toward signing reviews. PLOS Medicine maintains a strong tradition of review signing, whereas the PLOS Computational Biology community, despite familiarity with open review, tends to be fairly conservative about signing reviews. For journals wishing to encourage the adoption of signed reviews, it may be helpful to directly request a signature, provide signing incentives, describe the benefits of signing, or encourage reviewers to understand the author’s perspective.

1Public Library of Science, San Francisco, CA, USA, eseiver@plos.org

Conflict of Interest Disclosures: Dr Seiver and Ms Atkins are employees of PLOS.