This issue of Science Editor features an overview of recent developments in open peer review from the architects of the new TRANSPOSE database designed to collect and present the various peer review processes at journals. This article has prompted me to consider the various arguments for and against publishing peer review reports, some of which I have collected below in the form of a decision letter and author response for a nonexistent manuscript.

Dear Author,

The editors thank you for submitting to our journal for consideration your manuscript asserting that the scientific publishing and research communities have reached a tipping point regarding the acceptance of open peer review, specifically publishing peer review reports at the time of publication. Your manuscript was reviewed by the editors and two reviewers, and your main arguments in favor of publishing peer review reports have been summarized as follows:

1. Published Reports help expose the inner workings of the black box that is peer review. Peer review remains a somewhat mysterious process, leading many to believe it is responsible for more, and less, than it is in reality. As you state, publishing peer review reports helps shed light on the actual process of peer review and what it can accomplish. As an editor, I particularly appreciated your point that publishing the decision letter along with the reviewer comments helps to clarify the role of an editor, emphasizing how they are not merely “scorekeepers” and are tasked with managing the tough calls, focusing reviewer concerns, making final decisions, and more.

2. They emphasize collaboration. As you note, when the journal publication and peer review process work well it’s a collaboration between authors, editors, and reviewers. Once a journal has determined that the research is interesting, significant, and fits its scope, the reviewers and editors work to establish if the findings are valid, if the message is clear, and if there are any unnecessary gaps. As the editors will attest, this back and forth can be illuminating as all parties, who may not view a research topic in the same way, try to come to a common ground.

3. They can serve as training materials. One of the first things that a reviewer training program does is provide examples of good reviews and reveal the details of the review process. As the author states, published peer review reports serve the same purpose, which if adopted by most journals, will provide an exhaustive set of training materials for early career researchers in all fields. In addition, helping to expose the iterative nature of science, the misunderstandings that need to be addressed, and the tweaks that offer clarity, can provide a simple comfort to trainees as they see further examples of even the most accomplished scientists figuring things out as they go along.

4. They aid in replication. In perhaps the author’s strongest argument, the case is made that the publication of reviewer reports provides another tool that can improve reproducibility. Especially for journals that include the authors’ response, there are usually process details that might not make it to the published article but may prove essential to researchers later. In addition, as noted, if reviewers know that their reviews will be published and available for public scrutiny, they may be more likely to produce more comprehensive and helpful reviews, increasing the quality of the published research.

While it was felt that these are compelling arguments, the reviewers listed the following concerns that the editors believe need to be addressed before acceptance:

1. Publishing peer review reports makes the process appear “messy” and authors will be disinclined to publish in the journal.
2. They provide another possible “gotcha” for people to attack the authors and journal. For example, Reviewer #2 provided the following hypothetical disgruntled tweet: “For shame on the editors of X for publishing this article when reviewer #2 so clearly states why it’s trash.”
3. No one will read them.
4. They place additional strain on an already over-worked staff.

In addition, the editors ask that you please briefly clarify your concerns regarding signing reviews.

If you choose to revise, please provide a detailed Response to Reviewers. We thank you again for your submission and we look forward to receiving your revision.

Sincerely,
Editor

Dear Editor,

Thank you for considering our manuscript and providing this valuable feedback. I appreciate you taking the time to summarize our arguments and giving us the opportunity to address the remaining concerns:

1. Publishing peer review reports makes the process appear “messy” and authors will be disinclined to publish in the journal.

While I understand the concerns that publishing these process documents could be considered “messy,” because they reveal at least the impression of the manuscript in its unfinished, draft state, I am not sure this is necessarily a negative. Messy can be good and science is usually messy, rarely moving in a straight line. Much of the research community has also already embraced this ethos, with an increasing number of researchers posting lab notebooks and protocols along with unreviewed manuscripts to preprint servers. Of course, while there is survey evidence that publishing peer review reports is broadly supported (see eg, Ross-Hellauer et al) input from authors, reviewers, and editors should be considered before making this change.

2. They provide another possible “gotcha” for people to attack the authors and journal.

This is true, and it is an unfortunate side effect that exposing more of the process gives people more to complain about. However, as stated in the manuscript, the increased transparency of this process and the possible increase in the quality of the reviews may lessen this tendency. In the end, it may simply be that “haters gonna hate” and I have expressed this point more professionally in the revised manuscript.

3. No one will read them.

While this is a valid concern, even if it is true, that isn’t enough of a reason not to publish them in and of itself. For example, it is very unlikely that many people read conflict of interest disclosure statements either but it’s important to include them, nonetheless. In addition, early data from the journals that publish peer reviews have shown a range of reader interest, from receiving 10% of the traffic as full articles to as many as a third of all readers clicking to view the peer review reports. In the revision, I’ve included a link to helpful FAQ from ASAPBio with additional examples from journals with experience publishing peer reviews.

4. They place additional strain on an already over-worked staff.

This too is one of my chief remaining concerns, and it’s why initiatives like the TRANSPOSE database described in the recent Science Editor article are so important. Quite a few journals and organizations have already done much of the work to develop efficient processes and the TRANSPOSE database makes it easy for editors to see what other journals are doing and reach out to them as needed. I agree that the process should not be onerous to staff, and with the right guidance, it doesn’t seem like it should have to be.

To address your final point, it does seem that signing reviews is a trickier proposition, and in my opinion, less clear cut. As shown in the survey linked above, this hesitation is common. On the one hand, it can help keep editors (and reviewers) honest. It’s no secret that editors can send manuscripts to reviewers they know will go easy on it if they want to get this author or topic accepted, so exposing the names of reviewers would make those patterns blatantly obvious. On the other hand, we live in a world where people are petty, bias exists, and careers can be ruined by someone holding a grudge. A review is a criticism, a critique, and it’s hard to ignore existing power dynamics when asking people to sign reviews as many accepted articles will receive negative reviews along their way to publication. That said, there are many positives to signing reviews, not least of which is that it provides public recognition for the essential work that peer reviewers provide, and my recommendation is that journals consider allowing reviewers to have the option of signing their reviews if desired.

I would like to thank the editors and reviewers again for their input. I hope that I have sufficiently addressed the remaining concerns, and the final manuscript makes a clear case that publishing peer review reports can be a valuable step toward further increasing the transparency and value of peer review.

Kind regards,
Author

The cover of this issue shows a brain cell as a “ripple” occurs, which is believed to be part of the creation of a memory. The researchers who created this image, including Thanos Siapas and  Brad Hulse of Caltech, are studying how information moves throughout the brain, for example, from “newly coded memories to other brain areas such as the neocortex for safekeeping and long-term storage.” Much like how a peer review report relates to an article, these ripples show not the actual memory, but reveal part of the process that forms the memory, providing insights into how the brain works and possible avenues for addressing alignments.

Elsewhere in this issue, Anna Hatch and Mark Patterson provide an excellent overview on “How journals and publishers can help to reform research assessment”; Resa Roth summarizes her research into “Understanding the Importance of Copyediting in Peer-Reviewed Manuscripts”; Barbara Gastel and co-authors deliver their report on the 2019 AAAS Annual Meeting; Emilie Gunn puts out another Fire of the Week column, this time covering the important topic of “Protecting Patient Privacy Online”; and in her Gatherings of an Infovore column, Barbara Meyers-Ford attempts to answer the question, “PLAN S: Where Is It Now?”

Finally, this issue includes the first half of the annual meeting reports from the 2019 CSE Annual Meeting held in Columbus, Ohio, May 4-7, 2019, starting with a recap article by program co-chairs Mary K Billingsley and Shari Leventhal. As noted in the June 2019 Newsletter, the meeting reports this year have been excellent and provide a great way to review the sessions and the valuable information and tips contained therein. We hope that these reports, and all of the articles in this issue, will provide insights into the process of editing science and further our mission of helping editors and staff run the best version of their journal or other publication in pursuit of improving the scientific literature.

Acknowledgment: Special thanks to Jessica Polka (ASAPBio) for feedback and suggestions.