Primer

A Standard Terminology for Peer Review: Supporting Transparency and Trust

By: Nettie Lagace
Lagace N. A standard terminology for peer review: supporting transparency and trust. Sci Ed. 2024;47:20-22.https://doi.org/10.36591/SE-4701-08
February 25, 2024
Download Article
Open Science Peer Review Standards & Guidelines

NISO logoThe National Information Standards Organization (NISO) published ANSI/NISO Z39.106-2023, Standard Terminology for Peer Review1 in July 2023. This publication was the culmination of a NISO working group, consisting of industry stakeholders representing varied organizations and perspectives, examining and testing a terminology originally developed by STM, the International Association of Scientific, Technical and Medical Publishers. This primer article will describe the background and motivation for this standard and detail a few of its aspirations.

STM convened a working group on peer review taxonomy in 2020.2 According to the project lead, Joris van Rossum, Program Director at STM Solutions,3 the purpose of the working group was to recognize the growing calls for transparency and support of open science and determine the best option for communicating peer review across a broad audience of authors, reviewers, and readers when technological innovation in processes and interfaces has resulted in the emergence of so many new models of (open) peer review.4 

Peer review is a required element of quality research and has been since scholarly publication began as a formal endeavor. It is even more important, in today’s abundance of research outputs, to ensure that peer review is an understandable process and worth trusting. This assurance helps authors realize how their work is being evaluated, helps reviewers more effectively contribute to this essential process, and supports readers in their interpretation of published outputs. One straightforward way to foster understanding is to use common terms to describe any processes within individual journal publisher workflows that may be comparable to others across the scholarly landscape and to create terms that demonstrate contrasting processes. Thus, the STM working group reviewed a body of published material and surveyed publisher practices and existing models of peer review and developed an initial set of categories and terms.5 These were further evaluated by the working group and select organizations and brought to NISO for standardization in 2021,6 at the same time a pilot program was underway at several publishers. 

NISO Working Group

NISO is a nonprofit membership organization, based in Baltimore, Maryland, that identifies, develops, publishes, and maintains technical standards and recommendations to manage information and promote interoperability between various systems used by publishers and libraries.7 Since the advent of the Internet more than 30 years ago, it has expanded its participation to include many international organizations, and the reach of its publications has likewise increased. Many of its standards and recommended practices have been fully adopted by large and small publishers in all areas of research, as well as publishers’ system providers and research and government libraries. Consequently, it was an excellent venue for the terminology work to be continued and further appraised.

The NISO working group expanded the input from mainly publishers to other stakeholders in the scholarly landscape, including publisher associations, libraries, platform providers, peer review systems and other scholarly infrastructure providers. The group monitored the pilot—in which several publishers tested the terminology with their staff in specific journals and articles—and discussed the various elements and their practical communicability. As comments and other input from the pilot implementations arrived, the categories and terms were further refined. Then, the NISO process is to obtain formal approval of the finalized draft standard by a NISO leadership group and a voting pool (made up of NISO members appropriately balanced across stakeholder categories) before submission to the Board of Standards Review at the American National Standards Institute (ANSI) for its approval before NISO publication. The NISO process also requires a formal review of the standard 5 years after publication, although comments and requests for changes can be brought to NISO at any time. The Standard Terminology for Peer Review will be managed by a NISO Standing Committee made up of representatives from varied stakeholders.

Peer Review Terminology

The entire Peer Review Terminology fits on 1 sheet of paper!8 It is intended to apply to all review models, although some innovative models, such as the one used at F1000, are fully transparent by design and are not included. In addition, description of the article acceptance process is out of scope.

There are 4 major elements or categories, creating a framework, as follows:

  1. Identity transparency
  2. Reviewer interacts with
  3. Review information published
  4. Post-publication commenting

Terms within these categories indicate the specific conditions of the particular peer review model in use at the journal. 

The first category, “identity transparency,” contains terms (e.g., “single anonymized,” “double anonymized”) that describe the extent to which identities of participants are revealed to each other during the review process. 

The “reviewer interacts with” category describes with whom the reviewer communicates during the process, via whatever means (e.g., submissions systems, email) and may indicate multiple types. 

“Review information published” contains terms that convey information published about the review process on the article page. Examples of these terms include “review summaries” to be indicated when summaries or parts of the reviews or a summary of the review process are published; “author/editor communication” when the decision letter from the editor and reviewer responses (rebuttals) are published; or “reviewer identities” when the identities of the reviewers are published.

“Post publication commenting,” to be used only when applicable, indicates whether commenting is possible on the online-published version or the version of record on the publishing platform. It does not include any possible integrations with third-party platforms such as PubPeer, and includes only 2 possible values, “open” and “on invitation.”

Publishers should apply the Peer Review Terminology at the journal level as well at the published article level; this will communicate the review models used for the journal as well as the kind of review the article itself was subject to. In addition, these should be included in any author guide materials developed by the journal and in any submission system used. An Appendix in the Standard document provides some further examples and implementation advice for publishers.

Implementation

At the present time, many publishers are beginning to implement the Terminology, although due to variations in practices across the stable of journals operated by any single publisher and even within journals themselves,9 implementation can be a detailed task with various considerations and process participants. An example of a journal that has implemented the Terminology is Medical and Veterinary Entomology, published by Wiley,9 which has included it in its Author Guidelines.10

The NISO Peer Review Terminology Standing Committee has also begun to further support the standard and its implementers and is developing plans for its own efforts. Included in its remit are liaisons with other peer review organizations such as the Committee on Publication Ethics (COPE); outreach to publishers via industry meetings, webinars, and articles (such as this one!); and development of case studies and testimonials. Fresh discussions with implementers will undoubtedly elicit further strategies for actions.

Potential future work for the Peer Review Terminology includes expansion to peer review of books or data sets, among other published objects, and determination of a machine-readable version of the terms.

NISO and the Peer Review Terminology industry volunteers are proud of the standard publication, pleased to be supporting it, and eager to connect with colleagues about any questions or issues. 

References and Links

  1. NISO. ANSI/NISO Z39.106-2023, Standard Terminology for Peer Review. 2023. [accessed February 8, 2024.] https://www.niso.org/publications/z39106-2023-peerreview. 
  2. STM. Working Group on Peer Review Taxonomy. [accessed February 8, 2024.] https://www.stm-assoc.org/standards-technology/peer-review-taxonomy-project/. 
  3. NISO. Open Teleconference September 2022 – Peer Review Terminology, September 12, 2022. [accessed February 8, 2024.] https://www.niso.org/events/2022/09/opentelecon-peerreview.  
  4. Ross-Hellauer T. What is open peer review? A systematic review [version 2; peer review: 4 approved]. F1000Research 2017;6:588. https://doi.org/10.12688/f1000research.11369.2. 
  5. STM. A Standard Taxonomy for Peer Review. 2020. [accessed February 8, 2024.]  https://osf.io/hzewj. 
  6. NISO. STM’s Peer Review Taxonomy To Be Formalized As An ANSI/NISO Standard, NISO Press Releases. 2021. [accessed February 8, 2024.] https://www.niso.org/press-releases/2021/07/stms-peer-review-taxonomy-be-formalized-ansiniso-standard. 
  7. NISO. Welcome to NISO! [undated; accessed February 8, 2024.] https://www.niso.org/welcome-to-niso. 
  8. NISO. A Standard Terminology for Peer Review (poster). 2023. [accessed February 8, 2024.] https://groups.niso.org/higherlogic/ws/public/download/28908/Peer_review_terminology.pdf. 
  9. Willis M. What’s in a name? Defining peer review in a standardized way. The Wiley Network, September 25, 2023. [accessed February 8, 2024.] https://www.wiley.com/en-us/network/publishing/research-publishing/editors/whats-in-a-name. 
  10. Royal Entomological Society. Author Guidelines, Peer Review. Med Vet Entomol., 2024. [accessed February 8, 2024.] https://resjournals.onlinelibrary.wiley.com/hub/journal/13652915/homepage/forauthors.html#Peerreview. 

 

Nettie Lagace (https://orcid.org/0000-0002-2302-9119) is Associate Executive Director,
National Information Standards Organization (NISO).

Opinions expressed are those of the authors and do not necessarily reflect the opinions or policies of the Council of Science Editors or the Editorial Board of Science Editor.