Essay

New Identities in Peer Review: Who Are They and Why Are They Important?

By: Janaynne Carvalho do Amaral
Carvalho do Amaral J. New identities in peer review: who are they and why are they important?. Sci Ed. 2021;44:128-130.https://doi.org/10.36591/SE-D-4404-129
September 24, 2021
Download Article
Authorship Career Peer Review

Exalted by some people and criticized by others, since the peer review system has been implemented in scholarly journals it has transformed into an effective tool to select manuscripts of scientific merit for publication. Henry Oldenburg, the first Secretary of the Royal Society and creator of the Philosophical Transactions of the Royal Society (founded 1665), was one of the pioneers in implementing peer review in scientific journals. He started the peer-review system in Philosophical Transactions of the Royal Society by inviting 3 members of the Royal Society “who had more knowledge of the matters in question than he, to comment on submissions prior to making the decision about whether to publish.”1 At that time, the development of the peer-review system was linked to the concerns of scientists as science producers and consumers.2 As science producers, they wanted to have their work recognized by publishing in spaces valued by other members of the scientific community. As science consumers, they wanted to make sure that the studies elaborated by other scientists were evaluated with competence.2 The decisions taken by Oldenburg and the Council of the Royal Society to evaluate the quality of the content published in the Philosophical Transactions of the Royal Society constituted the foundations of peer review.2

Throughout the history of scholarly publishing, the peer-review process has typically been done by a small community of peers. These peers are defined in literature as “experienced researchers”3 and selected by the editors based on criteria such as academic seniority, academic degree, involvement in research activities, and scientific production.4 However, the Open Science movement has expanded the community of peers by fostering the participation of new identities in peer review and establishing new criteria to select peer reviewers.

Open Science, Open Access, and Peer Review

The Open Science movement and its components, such as Open Access, open data, open peer review, and citizen science, share values such as transparency, collaboration, sharing, and remixing knowledge.5 This movement is working toward changes in the conditions of production and circulation of information, knowledge, and culture, which have been interfering with the current epistemological and institutional structures, making it necessary to highlight the effect of these changes on the values and practices of scientific dynamics.6 

One of the goals of Open Access advocates is promoting a more democratic access to scholarly journals, potentially giving readers a more active role in scholarly publishing. That is, if the products of scholarly publishing—articles—are more freely available outside academia, it is possible that the role of the peer-review system can expand to promote a more democratic and inclusive participation in science both to the scientific community and anyone interested.

In addition to open peer review potentially reducing some of the bias of anonymized peer review models, promoting a fairer system to researchers, it is also capable of bringing together a multiplicity of voices to collaborate in the evaluation and improvement of manuscripts submitted for publication. Once peer review is open, it can be further expanded to be public, in the community, and crowdsourced,7 which can help give voice to new identities in peer review. It means that “whereas in traditional peer review editors identify and invite specific parties (peers) to review, open participation processes invite interested members of the scholarly community to participate in the review process, either by contributing full, structured reviews or shorter comments.”7 

New Identities of Peer Reviewers

As scientific publishing has changed, two new types of peer reviewers have emerged to bring fresh and important voices to the peer review process.

Early Career Researchers

Early career researchers (ECR) are one of the new identities in peer review. This group can be defined both as undergraduate, graduate or postgraduate students,8 or researchers under the age of 35 who are working toward a doctorate or have recently completed a doctorate.9 ECRs are important to peer review exactly because they bring a different perspective from an earlier career stage of the typical reviewer. They can help improve the peer-review process by identifying gaps in manuscripts and helping ensure they are written in an understandable way. Research conducted by Casnici et al.10 on attitudes of referees in a multidisciplinary journal found that “the disciplinary background and the academic status of the referee have an influence on the report time, the type of recommendation and the acceptance of the reviewing task.” The same study also noted that “senior researchers are harsher in their judgments than junior researchers, and the latter accept requests to review more often and are faster in reporting.”10 ECRs can build a two-way street to improving the-peer review process by learning from “seeing other people’s errors”9 and successes, becoming better authors and reviewers, and helping senior researchers to better communicate research results.

In 2018, BMC journals (https://www.biomedcentral.com/)11 launched a pilot project to engage ECR in the peer review process.12 Called Peer Review Mentoring, the process consists of a professor or senior researcher mentoring an ECR through a peer review, and the report must be assigned to both of them.12 Examples of journals supporting this project are Trials, Systematic Reviews, Pilot and Feasibility Studies, and Journal of Medical Case Reports, among many others.

Nonscientist Peer Reviewers

Healthcare users, patients, lay experts, nonacademic experts, professional communities, readers from non-Anglophone settings, and other interested parties13–14 are some examples of nonscientists in peer review. These people are important to peer review as consumers of scientific information and subjects of research; examples include participants in clinical trials and groups studied by anthropologists. They are able “to detect weaknesses in the reasoning that subject-expert peers may overlook if they are stressed for time or not motivated to produce a careful review.”14 Patients and the public can help to evaluate the quality of the evidence,14 reducing waste in research and ensuring that the design of the research is appropriate, relevant, and beneficial to them.15 In the case of groups studied by anthropologists, they can help to evaluate if their culture and costumes are being described in a correct way, without making them seem exotic or introducing other types of bias.

Research Involvement and Engagement (https://researchinvolvement.biomedcentral.com/)16 is an example of a journal that engages stakeholders, policy makers, service users, and patients in their peer-review process. These various identities are brought in as editors and peer reviewers to coproduce the journal side-by-side with academics. The editors of the journal see the role of the “reviewer patient” as someone who “may comment on the relevance of a study for a particular group, while those with academic training and research knowledge might comment on methodology.”17 

Challenges and the Future

Scientific communication is changing and bringing new challenges to editors, authors, peer reviewers, and publishers. These challenges range from the technological structure of journals, new peer review policies, new ways of managing information, interaction between authors and reviewers and between researchers and society. At the same time, these changes have allowed for the inclusion of new voices and identities in the peer review process. Understanding the role of these new identities is like diving into an infinite sea full of different experiences, with different interests and with different ways of producing knowledge. 

In the context of open science, diversity, equity, and inclusion initiatives in journals and institutions are helping to transform peer review from a tool to select manuscripts for publication to a way to democratize science and embrace the humanity of all actors in the editorial process.

References and Links

  1. The Royal Society Science Policy Centre. Science as an open enterprise. London: The Royal Society Science; 2012. [accessed September 23, 2021]. https://royalsociety.org/~/media/policy/projects/sape/2012-06-20-saoe.pdf
  2. Zuckerman H, Merton RK. Patterns of evaluation in science: functions of the referee system. Minerva. 1991;9:66–100. [accessed September 23, 2021]. http://www.jstor.org/stable/41827004
  3. Meadows AJ. A comunicação científica. Brasília (DF): Briquet de Lemos/Livros; 1999.
  4. Pavan C, Stumpf IRC. Avaliação pelos pares nas revistas brasileiras de ciência da informação: procedimentos e percepções dos atores. Encontros Bibli. 2009;14: 73–92. https://doi.org/10.5007/1518-2924.2009v14n28p73
  5. Ford E. Advancing an open ethos with open peer review. College & Research Libraries. 2007;78:406–412. https://doi.org/10.5860/crl.78.4.406
  6. Albagli S. Ciência aberta, questões abertas. In: Ciência aberta em questão. Brasília (DF): IBICT; Rio de Janeiro: UNIRIO; 2015. p. 9–26.
  7. Ross-Hellauer T. What is open peer review? A systematic review. F1000Research. 2017;6:588. https://doi.org/10.12688/f1000research.11369.1
  8. https://www.ische.org/early-career-researchers/
  9. Rodríguez-Bravo B , Nicholas D , Herman E, Boukacem-Zeghmouri  C, Watkinson A, Xu J, Abrizah A, Świgo M. Peer review: the experience and views of early career researchers. Learned Publ. 2017;30:269–277. https://doi.org/10.1002/leap.1111
  10. Casnici, N, Grimaldo, F, Gilbert, N, Squazzoni, F. Attitudes of referees in a multidisciplinary journal: an empirical analysis. J Assoc Inform Sci Technol. 2016;68:1763–1771. https://doi.org/10.1002/asi.23665
  11. https://www.biomedcentral.com/
  12. https://blogs.biomedcentral.com/bmcblog/2018/06/11/increasing-diversity-peer-review-transparent-mentoring-early-career-researchers/
  13. Bornmann L; Herich H; Joos H; Daniel, H-D. In public peer review of submitted manuscripts, how do reviewer comments differ from comments written by interested members of the scientific community? A content analysis of comments written for Atmospheric Chemistry and Physics. Scientometrics. 2012;93:915–929. https://doi.org/10.1007/s11192-012-0731-8 
  14. Shashok, K. Who’s a peer? Improving peer review by including additional sources of expertise. J Participatory Med. 2010. [accessed September 23, 2021]. https://participatorymedicine.org/journal/opinion/commentary/2010/12/08/1278/
  15. Minogue V, Cooke M, Donskoy A-L, Vicary P, Wells B. Patient and public involvement in reducing health and care research waste. Res Involv Engage. 2018;4:5. https://doi.org/10.1186/s40900-018-0087-1
  16. https://researchinvolvement.biomedcentral.com/
  17. Schroter S,Price A, Flemyng E et al.Perspectives on involvement in the peer-review process: surveys of patient and public reviewers at two journals. BMJ Open. 2018;8:e023357. https://bmjopen.bmj.com/content/8/9/e023357.citation-tools [accessed 2021 Sep 19]

 

Janaynne Carvalho do Amaral, Federal University of Rio de Janeiro.

 

Opinions expressed are those of the authors and do not necessarily reflect the opinions or policies of their employers, the Council of Science Editors or the Editorial Board of Science Editor.