When scientists seek to publish their work, most go through what is called a double blind review. They submit their paper to a journal, and a reviewer does not know who the author is. The reviewer’s identity is also masked, allowing that individual to offer comments without feeling constrained or fearing retaliation. The process is set up to remove bias in publishing.

Recently however, Eden King, of Rice University, and her colleagues offered evidence that bias can still exist with double-blind reviews, and this is especially likely to take place for studies focusing on diversity and inclusion.

The bias can occur in several ways. In some cases, the findings might challenge one’s worldview. Evidence of discrimination and prejudice in the hiring process, for example, could run counter to a reviewer’s beliefs of a just world.

Or, consider research showing that racial minorities and women face systemic barriers that limit their ascension to leadership positions. Such findings could serve to challenge how Whites or men obtained such roles, putting some reviewers on the defensive.

These represent just some of the biases that could creep into the review process.

King and colleagues also conducted an experiment, asking reviewers to assess papers of varying quality. For well-written studies, there was no bias in the review process. However, for papers of average quality, reviewers were much more critical of diversity-focused paper than other those focusing on other topics.

Collectively, this research shows that even when steps are taken to ensure fairness, biases can emerge. Given these patterns, King and colleagues suggest altering reviewers to their possible biases, as doing so might increase their motivation to offer fair evaluations.