Authors in many disciplines favor¹ peer review moving out of the shadows and becoming a more transparent practice. Accordingly, platforms and publishers are increasingly implementing open peer review (OPR) to enable new kinds of discourse within the publishing process. Yet, these systems differ in what is revealed when and to whom.

OPR can operate on many different parts of the review process. It can influence the process of peer review (who can comment on the manuscript, and whether they can communicate with one another) as well as the transparency of information about peer review (the visibility of the manuscript or reviewer names, reports), and it can operate at many different times, from before submission (i.e., preprints) to after publication (i.e., post-publication commenting). One of us undertook a systematic analysis² of definitions of OPR; this uncovered 7 core traits, which were used in 22 distinct configurations. The most frequently used elements of OPR were revealing reviewer identities (open identities) and publishing reviews (open reports).

Growth in open peer-review implementations and experiments

While open peer review has been practiced by publishers such as BMJ, Copernicus, and BMC for almost 20 years, it has gained ground in recent years,³ with EMBO Press, F1000, Nature Communications, eLife, PeerJ, and Royal Society Open Science serving as prominent examples.

The last year has seen a burst in activity in open peer review, some of which has manifested in new workflows and platforms. For example, Wiley launched a trial of a product called Transparent Peer Review⁴; it allows authors of papers in Clinical Genetics to opt in to have reviews posted on Publons. Encouragingly, 83% of authors opted in, and 10 more journals have recently joined the trial.

In addition, BMC has collaborated with Research Square to launch In Review,⁵ a platform that makes manuscripts available for public comment while they are under review, representing experimentation in open participation.

Scholarly activity in open peer review has also made significant progress. In January 2019, results were released from a trial in which 5 Elsevier journals⁶ began publishing all peer reviews. It showed that each journals’ submission rates increased during the trial. While the rate at which reviewers accepted invitations to review declined, these declines matched global trends, so may not have been caused by the review model. Importantly, reviews submitted during the trial were as critical and constructive as those submitted before it. However, less than 10% of reviewers chose to sign their reviews, signalling hesitance to embrace open identities (as predicted by the survey mentioned above) even among reviewers confident in making their reports public.

Importantly, reviews submitted during the trial were as critical and constructive as those submitted before it.

In addition, the authors of this article organized and/or attended a meeting at the Howard Hughes Medical Institute on transparency, recognition, and innovation in peer review⁷ that resulted in the publication of an open letter⁸ signed by over 300 journals that commit to enabling the publication of peer-review reports.

Fine-grain variation

Enthusiasm for open peer review is accompanied by tremendous variation in its implementation. Even within a single broad category of open peer review—for example, open reports—many different implementations are possible:

• Who makes the choice to publish peer reviews? Many journals give authors the choice, while others see adherence to a standard practice as a key element of their editorial process.
• When is this choice made? For example, do authors make the decision upon submissionor after seeing the reviews?
• What exactly is published? Is it the full text of every peer-review report, with nothing left as confidential comments to the editor? Or are the reports compiled into a summary document? For example, eLife publishes a decision letter containing major concerns raised by reviewers.
• How is it published? Does the peer-review report exist as a stand-alone object with its own DOI?

Best practices here are still evolving and will likely often be community-specific, particularly regarding choices about what to make open and when. However, in areas like the publishing of review reports, consensus on best practice and standardized workflows is emerging, as recommended in a recent workshop.⁹ ASAPbio is hoping to explore best practices in greater detail in an upcoming meeting.

All of this diversity applies only to policies pertaining to open peer review; these questions do not address the many other variables introduced by opening commenting, reviewer interaction, or other novel peer-review workflows.

Ideally, it would be easy for authors, journal editors, and other policy makers to survey the landscape both inside and outside their field in order to inform their own peer-review practices.

While this variability in peer-review implementations and experiments is exciting, it can also be confusing for authors. Recently published guidelines,¹⁰ created in collaboration with experts, seek to help guide publishers and editors in implementing such processes for the various facets of OPR. One urgent issue identified was the need to communicate OPR policies in a clear and transparent manner.

Experimentation is no doubt needed to arrive at optimal solutions for individual research communities. Ideally, it would be easy for authors, journal editors, and other policy makers to survey the landscape both inside and outside of their field in order to inform their own peer-review practices.

Transparency in peer-review policies

To assist authors and editors in surveying the landscape of peer-review policies, we created the TRANSPOSE¹¹ database. TRANSPOSE is a grassroots initiative to crowdsource particular journal policies that would benefit from greater clarity and transparency, including policies surrounding open peer review. The goal of the database is to foster new practices while making authors aware of current policies, and we seek to provide resources to assist journals in setting, clarifying, and sharing policies.

Data can be entered into TRANSPOSE through a publicly accessible form, or by requesting from TRANSPOSE organizers a spreadsheet to enter multiple journals with varying policies. All contributions to TRANSPOSE are released under CC0,¹² and by default the most recent version of record will be displayed. All versions are retained and are available for download.

However, we also encourage contributions from journals and publishers in an editor validation process, resulting in records marked as “journal verified.” These records are displayed as such online and can no longer be edited through the publicly-accessible form. Contributors can assert during the submission process that they are an authorized representative of the journal, such as an editor or publisher, and we will contact a representative of the journal before making verified records public.

The benefits of transparent editorial policies

Why would authors and editors use TRANSPOSE? Authors will be able to compare journals to find which policies around open peer review suit their needs and desires in publishing their work. Editors will be able to learn about and compare current practices by searching for journals in related fields.

To make this effort as useful as possible, we plan to study current policies by conducting a landscape study of practices across scholarly fields. We will also work with the community to develop template model policies for use by editors as they update or modify practices according to their needs.

Join us in making policies more transparent

As TRANSPOSE grows, we are eager to receive contributions of information about journal policies directly from the most reliable source: journal editors like you. These contributions signal a willingness to help a community project make the submission and peer-review process more clear and understandable to both authors and readers.

Please feel free to use the online form¹³ to submit, update, or verify a single policy (whether it applies to one or many journals), or get in touch with us (via jessica.polka@asapbio.org) if you have a more complex set of policies that spans across your journal family.

References and Links

1. Ross-Hellauer T, Deppe A, Schmidt B. Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers. PLoS ONE 2017;12:e0189311. https://doi.org/10.1371/journal.pone.0189311.
2. Ross-Hellauer T. What is open peer review? A systematic review [version 1; peer review: 1 approved, 3 approved with reservations]. F1000Research 2017;6:588. https://doi.org/10.12688/f1000research.11369.1.
3. Tennant JP. The state of the art in peer review. FEMS Microbiol Lett 2018;365:fny204. https://doi.org/10.1093/femsle/fny204.
4. https://www.wiley.com/network/researchers/submission-and-navigating-peer-review/progressing-towards-transparency-more-journals-join-our-transparent-peer-review-pilot
5. https://www.biomedcentral.com/p/InReview
6. Bravo G, Grimaldo F, López-Iñesta E, Mehmani B, Squazzoni F. The effect of publishing peer review reports on referee behavior in five scholarly journals. Nat Commun 2019;10:322. https://doi.org/10.1038/s41467-018-08250-2.
7. https://asapbio.org/peer-review/summary
8. https://asapbio.org/letter
9. Beck J, Funk K, Harrison M, McEntyre J. Publishing peer review materials [version 1; peer review: 2 approved]. F1000Research 2018;7:1655. https://doi.org/10.12688/f1000research.16460.1.
10. Ross-Hellauer T, Görögh E. Guidelines for open peer review implementation. Res Integr Peer Rev 2019;4:4. https://doi.org/10.1186/s41073-019-0063-9.
11. https://transpose-publishing.github.io/
12. https://creativecommons.org/share-your-work/public-domain/cc0/
13. https://docs.google.com/spreadsheets/u/1/d/e/2PACX-1vQVQVbwRTFymY1yMMPvHRLzEhlnm0HZ1ZEKvWeNjjbYtvyYuZ4_6eTqrJ0LkQDVF8ASwv62U3uw4V18/pubhtml?gid=1520385021&single=true

Jessica Polka is Executive Director, ASAPbio; Tony Ross-Hellauer is Senior Researcher, Know-Center; and Gary McDowell is Executive Director, Future of Research.