Moderator:
Michael Casp
Production Services Coordinator and Director of Business Development
J&J Editorial
Cary, North Carolina

Speakers:
Annette Flanagin
Executive Managing Editor and Vice President
Editorial Operations, JAMA and The JAMA Network
Chicago, Illinois

Andy Collings
Executive Editor
eLife
Cambridge, United Kingdom

Trish Groves
Head of Research, The BMJ
Editor-in-Chief, BMJ Open
London, United Kingdom

Annette Flanagin opened the session by presenting an overview of both the process of peer review itself and the many “flavors” of peer review: double-blind, single-blind, and open. The presentation provided an overview of the history of peer review as well as the purposes (assessing quality, evaluating scientific soundness, detecting flaws) and weaknesses (unfair, slow, inefficient, expensive) of traditional peer review. She also went over studies that have shown there are very few differences in quality between double-blind, single-blind, and open peer review, although some have found double-blind reviews may better manage some biases.

Flanagin also went over the variations within open peer review in particular (including optional open peer review, prepublication open peer review, and postpublication open peer review), as well as evolving practices and services surrounding the peer-review process, like postpublication commenting, collaborative peer review, and portable/cascading peer review (rejected manuscripts and their reviews are shared with another journal within a group of journals). Another important point Flanagin touched on is the evolving ways journals choose to recognize the service of peer reviewers, including publicly listing reviewers, providing them with free journal subscriptions, and annual best reviewer awards. Some newcomers to this group are publicly compiling reviewer statistics, using ORCID to track reviewer activity, and using citations for published reviews.

Andy Collings presented eLife’s variation of peer review, one he said was driven by their desire to make the peer-review process less painful for authors and to decrease the need for authors to go back to the lab based on peer-review results. Collings also believes many journals are working toward making the peer-review process more efficient and effective, and within this there is room for both improvement and experimentation. As a result, eLife has developed a peer-review process that shares reviewers’ names and comments among the reviewers and also enables them to share their names with the authors if they choose.

The reviewers’ discussions are done on their site, overseen by the editors, while the paper is in peer review. Their experience has shown the whole process takes longer, but the reviewer comments are more constructive as a result, and this helps explain why most papers at eLife only have to go through one round of revisions. Once a paper is published, the decision letter is published online with the paper, along with the author responses, and reviewers can choose to remain anonymous to the author if they wish (although the reviewers will all be known to one another based on their discussions during the peer-review process). In an attempt to encourage this process to remain open, eLife changed the wording of the question about whether reviewers wanted to be anonymous to authors to an encouraging paragraph explaining the process. After the change in wording, they saw a 10% increase in reviewers agreeing to share their names with the authors.

Groves ended the session with an anecdote about open peer review in the real world: BMJ chose to disclose (with permission from all parties) the four reviewers’ comments on a paper they had rejected, along with an explanation of their decision to reject, after the authors took their rejection to the tabloids claiming the reviews had been biased. In fact, the reviewer they had most thought would be biased against them turned out to have returned a positive review; the journal had been flooded with submissions about the same topic and this paper had simply not been a standout among so many similar papers.

Now, more than ever, the publishing industry has the ability to experiment with how the peer-review process works, and with what works best for each particular journal and within individual fields. These presentations show the future of peer review will only continue to develop more “flavors” in the future.