Introduction

Retractions are not inherently bad. They are, in fact, necessary because research and scholarship are constantly evolving, and the scholarly record must be corrected as errors are uncovered, and new information becomes available. Retraction is meant to be a mechanism for this self-correction, “alerting readers to articles that contain such seriously flawed or erroneous content or data that their findings and conclusions cannot be relied upon.”¹

Despite the importance of these corrective mechanisms, research has shown that this information is often inconsistently displayed,^2–5 meaning that readers may be unaware of the fact that an article has been retracted. Such retracted publications may be subsequently perpetuated through ongoing citations that reinforce their findings.^6–9 This ongoing, inappropriate use of retracted publications undermines the role of retractions as a corrective mechanism. 

Although retraction remains a relatively rare occurrence, the number of retractions is consistently increasing and will likely continue to rise.¹⁰ The decision to retract, remove, or issue an expression of concern is complex, requiring that publishers consider the nature of the concern, the trustworthiness of the findings, and the potential impact of the work. Organizations, such as the Committee on Publication Ethics (COPE), have provided extensive guidance on when retractions are appropriate and how publishers can navigate this decision making process.¹ However, a similar level of guidance has not been available when considering how to communicate these decisions, particularly across systems. 

Effective dissemination of retraction information is essential for a healthy scholarly communication ecosystem. In a predigital era, libraries maintained lists of retracted publications at reference desks and annotated print materials to indicate retracted status.^11,12 These manual, print-based processes established consistency. Although these isolated strategies no longer meet needs, no consistent, cross-industry guidance is available, and inconsistency has emerged. 

NISO and the Communication of Retractions, Removals, and Expressions of Concern

NISO,¹³ a nonprofit membership organization, aims to “identify, develop, maintain and publish technical standards and recommended practices to manage information in today’s continually changing digital environment.”¹⁴ NISO promotes interoperability and communication among libraries, publishers, and vendors worldwide, making it ideally positioned to help address the challenge of communicating retraction-related information.  

Following the 2021 NISO Plus conference, and with support of the Alfred P Sloan Foundation, the NISO Communication of Retractions, Removals, and Expressions of Concern (CREC) Working Group was formed. The working group consisted of over 2 dozen individuals representing publishers, vendors, researchers, and libraries worldwide. Collectively, the group worked to understand the current landscape, including the range of organizational activities and workflows; to articulate best practices, emphasizing a balance between flexibility and consistency; and to outline activities and responsibilities of all actors in the scholarly information ecosystem. The Draft Recommendations were first released for public comment in October 2023, and received over 120 comments from 35 organizations. On June 27, 2024, the culmination of these efforts was published in NISO RP-45-2024, the “Communication of Retractions, Removals, and Expressions of Concern (CREC) Recommended Practice.”¹⁵ 

The CREC Recommended Practice

The focus of the Recommended Practice is not the editorial decision making process underlying postpublication activities, but rather, the timely and transparent communication of those decisions. The 7 sections of the Recommended Practice cover a broad range of retraction-related metadata recommendations, including the creation, transfer, and display of metadata and the responsibilities of multiple actors in the system, as outlined in Responsible, Accountable, Consulted, and Informed (RACI) matrices. 

The Recommended Practice outlines publisher responsibilities, including notifying aggregators and vendors and informing other relevant parties, such as preprint servers and repositories, and encourages the integrating of notifications into standard editorial and production workflows (Figure). Guidance is also provided to vendors and aggregators on metadata receipt and display to ensure visibility across platforms to both human and machine readers. 

Specific recommendations include prepending “RETRACTED:” to the titles of retracted publications, prominently watermarking and labelling content in HTML and PDF format, and ensuring that retraction notices are distinct, easily accessible, and linked to appropriately labelled retracted content. 

The Recommended Practice includes extensive guidance on metadata elements for retracted publications, retraction notices, expressions of concern, and removals. The various elements are classified as Essential, Essential if Available, and Recommended. The Recommended Practice does not establish a new metadata schema, but instead outlines how existing schema and elements can be used to effectively and consistently communicate retraction-related information. To help support implementation, examples of both Journal Article Tag Suite (JATS) and Crossref metadata for retracted publications and retraction notices are provided to illustrate how each element can be used.

The Recommended Practice also offers specific guidance on unique situations, such as items that are retracted prior to formal publication, retraction when the journal has been transferred to a different publisher, retraction in issue-based publishing, and retraction of multiple items as a result of a single investigation. 

Next Steps and Future Work

Although the Recommended Practice has now been published, the work is ongoing. A standing committee will soon be formed, which will be tasked with monitoring ongoing developments, gathering and responding to community feedback, and expanding and refining the Recommended Practice in the future to ensure maximum applicability. Future work may include more substantial guidance on expressions of concern, standardized language around reasons for retraction, and opportunities of automated identification of retracted publications. 

References and Links

1. Committee on Publication Ethics (COPE) Council. COPE guidelines: retraction guidelines. COPE; 2019. https://doi.org/10.24318/cope.2019.1.4. 
2. Suelzer EM, Deal J, Hanus K, Ruggeri BE, Witkowski E. Challenges in identifying the retracted status of an article. JAMA Netw Open. 2021;4:e2115648. https://doi.org/10.1001/jamanetworkopen.2021.15648.
3. Schneider J, Lee J, Zheng H, Salami MO. Assessing the agreement in retraction indexing across 4 multidisciplinary sources: Crossref, Retraction Watch, Scopus, and Web of Science. In: Waltman L, Lai KH, Murat B, Wang J, Weimer V, Noyons E, Luwel M, editors. 27th International Conference on Science, Technology and Innovation Indicators (STI 2023). Center for Science and Technology Studies and European Network of Indicator Developers; 2023. https://doi.org/10.55835/6441e5cae04dbe5586d06a5f.
4. Bakker CJ, Reardon EE, Brown SJ, Theis-Mahon N, Schroter S, Bouter L, Zeegers MP. Identification of retracted publications and completeness of retraction notices in public health. J Clin Epidemiol. 2024;173:111427. https://doi.org/10.1016/j.jclinepi.2024.111427.
5. Boudry C, Howard K, Mouriaux F. Poor visibility of retracted articles: a problem that should no longer be ignored. BMJ. 2023;381:e072929. https://doi.org/10.1136/bmj-2022-072929.
6. Theis-Mahon NR, Bakker CJ. The continued citation of retracted publications in dentistry. J Med Libr Assoc. 2020;108:389–397. https://doi.org/10.5195/jmla.2020.824.
7. Schneider J, Ye D, Hill AM, Whitehorn AS. Continued post-retraction citation of a fraudulent clinical trial report, 11 years after it was retracted for falsifying data. Scientometrics. 2020;125:2877–2913. https://doi.org/10.1007/s11192-020-03631-1.
8. Bakker CJ, Theis-Mahon N, Brown SJ, Zeegers MP. The relationship between methodological quality and the use of retracted publications in evidence syntheses. Syst Rev. 2023;12:168. https://doi.org/10.1186/s13643-023-02316-z.
9. Hsiao TK, Schneider J. Continued use of retracted papers: temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine. Quant Sci Stud. 2021;2:1144–1169. https://doi.org/10.1162/qss_a_00155.
10. Van Noorden R. More than 10,000 research papers were retracted in 2023—a new record. Nature. 2023;624:479–481. https://doi.org/10.1038/d41586-023-03974-8.
11. Duggar DC, Christopher KA, Tucker BE, Jones DA, Watson M, Puckett M, Wood B. Promoting an awareness of retractions: the Louisiana State University Medical Center in Shreveport experience. Med Ref Serv Q. 1995;14:17–32. https://doi.org/10.1300/J115V14N01_03.
12. Hughes C. Academic medical libraries’ policies and procedures for notifying library users of retracted scientific publications. Med Ref Serv Q. 1998;17:37–42. https://doi.org/10.1300/J115v17n02_04.
13. https://www.niso.org/
14. NISO. All About NISO. [accessed August 5, 2024]. https://www.niso.org/what-we-do. 
15. NISO Communication of Retractions, Removals, and Expressions of Concern Working Group. Communication of Retractions, Removals, and Expressions of Concern (CREC) (NISO RP-45-2024). [published online June 27, 2024]. https://doi.org/10.3789/niso-rp-45-2024. 

Caitlin Bakker is Discovery Technologies Librarian at the University of Regina and Co-chair of the NISO Communication of Retractions, Removals, and Expressions of Concern Working Group.