Peer review and implicit bias
Review models & DEI
There is no perfect model for peer review that will magically resolve inequities in scholarly publishing. Journals should consider their options with a goal of minimizing biases and promoting constructive feedback.
- Open review: This form of review, where authors and reviewers are aware of the others’ identity, may encourage reviewers to write more empathetic and thoughtful reviews (which may or may not be shared publicly). Some forms of open review aim for inclusion by inviting a wide community to comment and critique an article before or after publication.
- Single-anonymous review: Often used where hiding the identity of the author is very difficult, this model may be prone to conscious or unconscious biases when it comes to factors like race or gender, or institutional prestige. Journals using this method should take special care in recruiting diverse reviewers and keeping a lookout for possible bias.
- Double-anonymous review: Maintaining the anonymity of both author and reviewer is one way to potentially reduce bias, though some reviewers may still make assumptions based on the content or style of the manuscript. One recent study found higher “desk rejection” rates from editors under this model versus a single-anonymous process, which may indicate some level of editorial bias.
- Triple-anonymous review: Editors have many of the same biases as peer reviewers, so this model hides authors’ identities even from journal leadership, with a journal staff member anonymizing submissions even before they hit the editor’s inbox. (OJS users should note the current platform does not easily accommodate triple-anonymous review.)
- Editorial review: Some journal sections—or in some cases entire journals—may not benefit from formal peer review and can rely on editors working in collaboration with authors. Consider where the peer review process is simply gatekeeping and potentially excluding new perspectives, and where it is necessary to improve submissions.
Principles into Action: Blind vs. Anonymous Review
The term “blind review” is ubiquitous throughout scholarly publishing, but how might it reinforce negative perspectives on disability? In a post for the American Philosophical Association blog, Rachel Ades offers a compelling argument for why we should stop equating “blind” with “unknowing” and move toward new terms like “anonymous” or “identity-hidden” to describe our peer review processes.
Recruiting reviewers
Journal staff have a responsibility to seek out a broad pool of reviewers who reflect and promote increased diversity among published authors. In turn, the diversity of editorial board members, who frequently recommend reviewers from their own networks, will have an impact on who reviews for the journal. Researchers have found that reviewer and editor “gatekeepers” tend to demonstrate some bias toward authors with similar characteristics such as gender or geography, which has privileged men and authors from North America and Europe. Some possible ways to help broaden the pool include:
- Take a look at the journal’s list of reviewers over time, considering any known factors such as gender, race or ethnicity. How do these demographics align with editorial board or authorship makeup, as well as intended readership?
- Just as some conferences have explicitly warned against “manels,” or all-male panels, ask editorial staff to avoid sending submissions only to white, male reviewers from North America and Europe.
- Requiring authors (including co-authors) to contribute at least one future peer review per article published.
- Explicitly allowing for “co-review,” where an invited peer reviewer works with a (typically less experienced) researcher to write the review. IOP Publishing launched a co-reviewer policy in 2020. This process can offer experience and credit for early career scholars.
References and resources
Gaston, T. (2019, January 10). Gender diversity in peer review. Wiley. https://www.wiley.com/network/archive/gender-diversity-in-peer-review
Meadows, A. (2018, September 13). Eight ways to tackle diversity and inclusion in peer review. The Scholarly Kitchen, Society for Scholarly Publishing. https://scholarlykitchen.sspnet.org/2018/09/13/eight-ways-to-tackle-diversity-and-inclusion-in-peer-review/
