Notes on Transparency: An Elusive, and Illusive, Goal

Jonathan Schultz

doi:10.36591/SE-D-4301-2

Transparency is a common topic when discussing scientific editing and research rigor, serving as the focus of workshops, initiatives, and more. There’s even a metric now from the Center for Open Science, the TOP Factor, to evaluate how journals are implementing transparency guidelines. In Science Editor, we’ve covered transparency—what it means and how to achieve it—quite often, and this issue is no exception. Reviewing the articles in this issue spurred some additional thoughts on this topic that I’ve collected as follows.

No Panacea. When transparency in research is discussed, it’s common to have it mentioned that it’s not a panacea. Of course not! Nothing is a panacea, to be fair, but transparency has been integral to science since the beginning, so it can’t be expected to fix everything. The first journal articles were letters between scientists explaining their processes; what is happening now is another adjustment of the diopter, bringing more of the research process into greater focus. Science has become more complicated and more collaborative, and the push for greater transparency is necessitated by the former and required for the latter. So many elements and bits of information are required to reproduce or replicate results that asking researchers to spend time tracking them down is effectively preventing that replication from happening, as was shown by initiatives like the Cancer Reproducibility project. Many of the newer transparency guidelines are simply reflecting that the increase in the amount of detail and information is needed to understand and reproduce modern research.

Opportunity, Crimes of. Furthermore, when journals require transparency of data, code, protocols, original figure data, statistical details, etc., etc., it’s not with the expectation that these requirements will eliminate fraud. But they certainly make it harder. For example, a number of basic science biomedical journals now require authors to provide uncut gels and blots, highlighting which lanes were used in the article, as supplemental material. This requirement can’t thwart a highly motivated fraudster, but it may prevent an author from making an improper splice or duplication to make their data appear more compelling. This requirement, like the best transparency guidelines, should be easy to fulfill for the honest and meticulous researcher, but tough for the corrupt or careless.

Motivated Sharing. In many types of research, in addition to being transparent with your processes and data for replicability, sharing research materials can be just as important. As outlined in the article, “How Life Science Journals Can be Champions of Better Material Sharing and Reporting” by Angela Abitua, having access to, for example, specific cell lines, plasmids, or experimental organisms can determine whether results can be successfully replicated and built upon. In the past, these materials were “available upon request,” which required a significant effort on the part of authors to both request and supply them; now, it is becoming increasingly common for repository services to store, validate, and supply materials, removing this burden from authors. This points to an additional benefit of transparency to researchers: the more that is available from third parties, such as repositories or journals, the less time researchers need to spend responding to requests.

On Glass Houses. As journals require increasing transparency from researchers, would it not be appropriate that editorial operations become just as transparent? That is the question raised by Shroyer and coauthors in the article, “Call for Transparency in Top Biomedical Journals’ Publication Practices.” The authors reviewed publication patterns of articles in the New England Journal of Medicine (NEJM) from 2002 through 2017, comparing author characteristics such as gender and institution, and the differences between authors that had only published once in the journal versus those with multiple publications. The authors lament that NEJM choose not to participate directly with their study and provide de-identified journal database information, which they believe would provide a more accurate picture of their publication practices.

I can understand why a journal may not want to turn over even de-identified data to an external researcher, and the results of the study are interesting even without access to the full NEJM submission records. For example, the finding that female first authors are under-represented (only 13%) is an important point, regardless of how many female first author manuscripts were submitted. However, their call for greater transparency of publishing practices is valid and important. Initiatives such as the PEERE protocol are working towards this goal, and it seems like developing a standard for the type of submission, acceptance, and demographic data that journals make publicly available is something that CSE should consider.

Regarding Dark Data. A potentially overlooked aspect of research transparency involves exposing the research that never sees the light of day. This “dark data,” as defined by Sandra Petty, Hugo Stephenson, and Sarah Hadley in their article, “Shining More Light on Dark Data,” are the negative, inconclusive, or confirmatory studies that are left unpublished in a file drawer or lab notebook. This can lead to publication bias, wherein the effects of a particular treatment, drug, or method appear more positive because that’s what is published, but as the authors outline, as science has moved online and become more open, much of this research has been able to move out of the shadows.

Exposed. One of the impediments to transparency is the vulnerability inherent in being more transparent: The more your share, the more people know, the more they can pick apart. Whether it’s sharing your negative results, details of your research process, peer review reports, or journal submission data, when it’s all out there, someone may find something to attack. This tension is clear to anyone who has tried to move a transparent retraction notice or errata through legal review: Details that might seem helpful to independent researchers are sometime viewed as potentially litigious by lawyers (disclaimer: the previous statement was a generalization that in no way reflects a real-life event). However, as greater transparency becomes the norm, not being transparent will likely be seen as suspect on its own. Over time, the exposure that comes with increased transparency will likely become more common and less interesting.

Spoken Words. In the meantime, I find that being transparent can sometimes come easier in person, which is why meetings like the upcoming CSE Annual Meeting can be so valuable. Often, recounting embarrassing details of missteps taken implementing an initiative pour out more freely to a room of colleagues than on the printed page. Likewise, during presentations, questions may be asked revealing worthwhile information speakers didn’t even think to share. This aspect of transparency is at the core of the program put together by co-chairs Emilie Gunn and Peter J Olson of the CSE 2020 Annual Meeting: Advancing Science by Exchanging Knowledge. As they put it, the meeting serves as a place to be open and share the “indispensable experience, innovation, and expertise that provides CSE members with the tools they need to thrive in the ever-evolving, ever-expanding hinterland of scholarly publishing.”

Ever the Point. In the process of developing a transparent peer review pilot, an editor raised a concern that the posted reviewer comments may include criticisms that, for a host of reasons, may not be fully addressed in the final published manuscript. In response, some of the other editors, almost in unison, proclaimed “Well, that’s the point!” Concerns raised by reviewers are likely to be shared by readers so by providing the reviewer comments and the authors’ responses, the hope is that these concerns are acknowledged and questions about the peer review process can be preempted. Readers may still believe the concern was not properly addressed by the authors in their posted response, but it is at least clear that the concern was raised and considered.

Classification. As alluded to earlier, transparency is an aspirational goal, and one that can never truly be achieved. To expose all collected data, every element, decision, and step in the research process, or all parts of the review process, is impossible. It is transparent in comparison to what has been done previously, but never truly transparent in an objective sense. Translucent is probably a more appropriate term; broad outlines can be clearly seen, and maybe a few key details, but it is clear that some obscuring occurs. However, referring to your process as translucent could be interpreted to mean that the obscuring is intentional, so transparency, with a caveat, will have to do.

Ruse. It is important to keep that point in mind as the appearance of transparency can be used to deceive. This is the skill of the stage magician: They make the audience believe they are seeing everything when in fact, they only see what the magician wants them to see. To step behind a curtain and claim to make their assistant disappear fools no one; an audience knows to be skeptical of what they cannot see. But to stand on stage, exposed and alone, and make a person vanish with the snap of their fingers will make an audience believe, even if just for a moment, that something magical occurred. “It had to have happened: I saw it all” you might say. But you didn’t, you just think you did and that makes you less willing to think you were fooled.

Transparency provides editors, reviewers, readers, and researchers with the tools to better adjudicate the quality of the science.

Evolution. That concern should not be considered a flaw of the move for greater transparency, but simply a call to remain skeptical (in the true meaning of the word) at all times. As we are still in a transition period during which new standards of transparency are being established, we may see charlatans cloak their fraud in the guise of transparency as a misdirection from their true intent (“as you can see, I have nothing up my sleeve”). Being more transparent is just one of many indicators of trust in science, but science that is more transparent isn’t inherently truer. Instead, transparency provides editors, reviewers, readers, and researchers with the tools to better adjudicate the quality of the science.

Topics Unrelated. Although not directly related to the specific topic of transparency, many of the other articles in this issue of Science Editor fulfill a similar purpose by providing behind the scene knowledge and insights. For example, Andrés Martin, previous Editor-in-Chief of the Journal of the American Academy of Child and Adolescent Psychiatry, provides details for what he learned as he transitioned the journal to a new EIC, while Peter Olson makes “The Case for Journal Style Guides” and supplies tips for getting them right. This issue also marks the start of three new regular columns: “Style Bites” by Stacy Christiansen and the AMA Manual of Style committee; “Getting Social in Scholarly Publishing” by Jennifer Regala; and the return of “Ethical Editor” by Kelly Hadsell and the CSE Editorial Policy Committee.

Summation. Both Science and Magic may make you exclaim “How did they do that?” but only the magician should be excommunicated for answering the question. To function properly, science needs to be as transparent as possible, providing all the information, data, materials, and more to answer the question.

I hope that Science Editor works in much the same way and readers find the answers to their questions through the transparent sharing of information and insights. If you have a “how did they do that?” question, let us know, and we’ll see if we can publish an answer in an upcoming issue.

References and Links