Peer Review: Rejected - What single-blind is doing to peer review papers

The Peer Review Scoop: Rejected – Reason: Sorry, You’re Female

What is peer review and what are the different types of peer review?

Maybe you’ve never heard of peer review before. Or maybe you’re a bit rusty on the specifics. That’s okay, I’m here to help. Peer review is "to read, check, and give your opinion about something that has been written by another scientist or expert working in the same subject area as you" (Cambridge Dictionary). Within academic journals, there are many who opt to do peer review via single-blind. What that means is that reviewers will know who wrote an article, but the author of the article will not know the reviewer. Another structure, though less common, is double-blind, where the author and the reviewer are ignorant to who the other party is. Finally, the third option, where both the reviewer and the author know who the other party is, at least at some point during the process of review. All of these options are things which occur during scientific research and peer review takes a leaf from scientists’ books in an effort to be more fair and unbiased. But as with many things, no system is perfect.

What are the benefits of single-blind?

To set the scene, single-blind is the standard for journals at the moment. However, there is some question regarding its validity and ability to mitigate bias. Let’s take a look at what that means from an expert - within “Reviewer bias in single- versus double-blind peer review” by Tomkins et al. (2017) there are discussions to the benefit of single-blind peer review: “In one interpretation, single-blind reviewers make use of prior information that may allow them to make better overall judgments” (p.12712). For example, based on a researcher’s prior successes, it might be possible for the reviewer to judge that they will have future research success and deserve funding for a proposal. When it comes to reviewing a completed research paper, reviewers are able to gain insight into prior work by that author to know if they have background in that research area. This can be both good or bad, it really depends on who the researcher is and who the person doing the peer reviewing is. In case you were wondering why the researcher doesn’t get to know who is reviewing them, there can be all sorts of issues with that, not least being interruptions during the review process to influence it or angry letters if the researcher’s article is rejected. But what about double-blind? What are the benefits there?

What are the benefits of double-blind?

Despite there being some benefits to single-blind peer review, there may be more opportunities to have less bias in double-blind peer review. For example, because the reviewer is not able to see any information about who wrote an article, it can be possible to remove possible bias for race, gender, or institution (Tomkins et al., 2017, p.12712). The attempt to mitigate bias in reviewing is the primary call for double-blind peer review, but there is a question from Tomkins et al. on whether this changes any results. While the average person can be biased, and there is no sure way to remove bias from everything, academic publishing does put in an effort to work against any possible bias. Comparatively, Bendiscioli (2019) is able to back up the assessment that revealing a researcher’s identity part way through the review process does result in changes to the peer review: “Indeed, an analysis of the effect of blinding applications in the first stage and disclosing identities in the second stage of the assessment of grant proposals in Spain found that reviewers did change their evaluation once they could see applicants’ identities; the authors of the study suggest that blinding application might help to limit reviewers’ conscious or unconscious bias” (p. 3). The argument for double-blind becomes a bit more credible when there is reasoning behind why journals may want to consider implementing these procedures.

A call for change – why switch to double-blind?

With the background in place, it becomes possible to argue for the switch to double-blind peer review. As mentioned by Bendiscioli (2019), there is an argument for why double-blind peer review is important, due to the studies done about bias when someone’s identity is exposed later in the review process. That alone is a good first step to explain why it is important to implement double-blind peer review. Furthermore, Tomkins et al. (2017) makes the argument that “it may be that other work is disadvantaged, in the sense that two contributions of roughly equal merit might be scored differently by single-blind reviewers, in favour of the one from a top school” (p.12712). Where an article might be of equal merit to another, it is possible that one school may get more attention. The bias of school or university preference might also layer on the judgement regarding someone’s income, someone’s geographic location, or the prestige and expectations behind the assumption that one’s school/university allots a better education than a different school/university.

Both Tomkins et al. (2017) and Bendiscioli (2019) are in agreement that double-blind peer review is worthy to explore further, making way for improvement on the system once it is in place.

Work Cited:

Bendiscioli, S. (2019). The troubles with peer review for allocating research funding: Funders need to experiment with versions of peer review and decision-making. EMBO Reports, 20(12), e49472. https://doi.org/10.15252/embr.201949472

Peer review | English meaning - Cambridge dictionary. (n.d.). https://dictionary.cambridge.org/dictionary/english/peer-review

Tomkins, A., Zhang, M., & Heavlin, W. D. (2017). Reviewer bias in single- versus double-blind peer review. Proceedings of the National Academy of Sciences - PNAS, 114(48), 12708-12713. https://doi.org/10.1073/pnas.1707323114