A Troubled Tradition

Conclusion

Peer review is a key feature of scientific publication, but it is susceptible to bias, inefficacy and other ethical transgressions. Alternative forms of review have produced only equivocal improvements in fairness and efficacy and have not been tested with respect to other problems. What are the next steps we should take to improve peer review?

First, researchers should receive more education on how to review scientific articles—a skill that is not typically emphasized during research training. Some scientists do show students and postdocs how to review papers, and some research institutions cover peer review in seminars and workshops on research ethics. These practices must become more widespread. In particular, investigators should teach their trainees how to evaluate articles for scientific merit and to follow ethical standards of peer review. Asking young scientists to help review papers is a good way to educate them, provided journal editors give their permission and the process remains confidential.

Journals should also develop and publicize instructions for new reviewers and policies for reviewers and editors, just as they have done for authors. Rules should address confidentiality, fairness, conflict of interest, respect for intellectual property, and professionalism.

Editors should carefully manage the peer review process to prevent or address problems and concerns. They should explicitly inform reviewers about their journals’ peer-review policies, remind reviewers to disclose conflicts of interest and return their reports on time, and delete any personal attacks from reviewers’ comments. If editors have evidence that a reviewer provided a poor review or abused the process, they should not invite that person to do other reviews. These ideals may be complicated when editors have difficulty finding experts willing to review a manuscript and when referees submit their reviews late, overlook errors, or disagree about the quality of a submission. Workshops and conferences on the subject could help editors to cope with these challenges.

Finally, scholars should conduct additional research on the ethics of peer review. Our exploratory study of the experiences of NIEHS researchers suggests that some problems are common, but the results should be confirmed in other settings. Future work should determine how often ethical problems occur and how they affect scientists’ attitudes and behaviors. Studies should also address the causes of unethical behavior in journal peer review and the effectiveness of alternatives, such as double-blind or open review, at preventing various types of transgressions.

There is certainly no perfect solution to the problem of quality control in the scientific record. Despite its flaws, the system adopted by the editors of Philosophical Transactions two and a half centuries ago seems to work as well as any method that has been tried—but its age and pervasiveness must not foster complacency. While journals, editors, and scholars work to understand and regulate peer review, it’s up to every individual scientist to maintain a thoughtful awareness of his or her participation in the process. Such vigilance and professionalism can only improve the quality of reviews, and might even spark new insights into how the review system could eventually be improved.

Acknowledgments

I am grateful to Bruce Androphy, Zubin Master, Christine Flowers and Adil Shamoo for helpful comments. This essay does not represent the views of the NIH, the NIEHS or the U.S. government.

Godlee, F., C. Gale and C. Martyn. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: A randomized controlled trial. Journal of the American Medical Association 280:237–40.

Nylenna, M., P. Riis and Y. Karlsson. 1994. Multiple blinded reviews of the same two manuscripts: Effects of referee characteristics and publication language. Journal of the American Medical Association 272:149–51.