Abstract

Importance: High-impact journals (e.g., New England Journal of Medicine [NEJM]) transform clinical practice; these publications have been commonly used to quantify faculty performance in academic medical centers’ promotion and tenure decisions.
Objectives: To support scientific transparency, the “unwritten” NEJM publication priorities and trends were documented.
Design/Setting: From 2002 to 2017, PubMed extracts for all original NEJM research articles with a structured abstract (n = 2,419) were analyzed. For a sampling of articles, supplementary information was obtained from publicly available resources.
Participants/Exposure: The NEJM author and research project characteristics were compared for the first authors with multiple first author publications (MP) vs. those with a single publication (SP).
Main Outcome(s) and Measure(s): Publication-specific characteristics included National Library of Medicine medical subject headings disease category, clinical trial design, grant funding, coauthor count, collaborating author count, and other study-specific details (e.g., directionality of overall findings). First author-specific characteristics included gender, advanced degrees held, self-designated major clinical specialty, institutional location, and academic rank.
Results: There were 2,065 first authors identified, of which 88% (n = 1,816) were SP first authors; these 1,816 SP first authors represented 75% of publications. Compared to SP first authors, MP first authors more often published clinical trials (96% vs. 80%; P < 0.001), had more collaborators (mean = 195 vs. 100; P = 0.006) since 2008, and were more frequently grant-funded (54% vs. 42%; P < 0.001). For a sampling of abstracts, MP vs. SP publications reported positive findings less often (73% vs. 96%, P = 0.036); MP first authors were more frequently cardiovascular disease-focused (28% vs. 17%, P < 0.001). Overall, female gender was under-represented for both SP and MP first authors (13%).
Conclusions: Given striking differences in NEJM MP vs. SP first authors and publication characteristics, academic faculty hopeful to publish multiple times in a top-tier biomedical research journal should review historical journal-specific publication practices.
Relevance: Given the avalanche of open access journals, the biomedical science academic community now stands at the crossroads of a new “bibliometrics” revolution. These preliminary NEJM-specific patterns raise important research questions; to rigorously document journal-specific publication/authorship variations, biomedical science journals’ enhanced transparency with public reporting now appears warranted.

Background

In academic medicine, performance metrics (i.e., bibliometrics) are increasingly being used to gauge biomedical science research faculty members’ productivity. For example, the H-index is becoming a common indicator of academic output. The H-index calculation includes the number of times that a faculty member’s peer-reviewed publication was cited in other scientific works.¹  Therefore, the likelihood that a faculty member’s publication will become “highly cited” is related to their publication journal’s impact factor, a metric that is based in part on citations received and articles published within the preceding two years.² In addition to readership statistics, impact factors also measure a journal’s importance and potential impact upon transforming future healthcare practice.

The New England Journal of Medicine (NEJM) has the highest current impact factor in biomedical research, trending upward from 55.9 (2014) to 79.3 (2017). In general, biomedical researchers strive to publish in top-tier journals like NEJM; ideally, attaining not just a single first author publication (SP), but multiple first author publications (MP) in NEJM. Some academic institutions even provide financial incentives to publish in top-tier biomedical journals such as NEJM, with Chinese researchers reportedly receiving a prize of 500,000 Chinese Yuan for having a paper published in a highly regarded journal.³

The number and timing of first author publications produced by individual scientists may be complex and challenging to predict, as multiple factors (e.g., use of medical writers) may contribute to an academician’s productivity. Prior studies have identified an inverse relationship between an author’s number of NEJM papers published and the time to a subsequent NEJM publication.⁴ Previous work has shown coauthor team size has more than doubled within the field of medical research over the second half of the 20th century.⁵ However, it is not known whether first authors with larger author teams are more successful in achieving additional first author publications in a top-tier journal. Beyond the number of coauthors, the number of collaborators (i.e., the number of local site investigators participating in a multicenter, randomized, controlled clinical trial) may also be an important factor related to successful research publications. The impact of collaborating author team members has not been previously researched. The purpose of this study was to identify trends in factors related to MP vs. SP NEJM authors to promote greater transparency and potentially provide guidance to future authors wishing to achieve MP author status.

Research Questions and Approach Used

For NEJM original research articles published 2002–2017, this study compared MP vs. SP first authors for the following:

• Differences in coauthor team member count;
• Differences in collaborating team member count;
• Differences in focus across major diseases (based on National Library of Medicine medical subject headings [MeSH]);
• Differences in study designs used (i.e., proportion of clinical trials); and
• Differences in grant funding.

Correspondingly, this study’s null hypothesis was that “…there would be no differences in SP vs. MP authors for their publication’s coauthor counts, collaborating team member counts, the major disease focus, the study design used, as well as the grant support received.”

The Medline records for all NEJM publications from January 1, 2002, through December 31, 2017, were extracted from PubMed; records were identified as being an original journal article (based upon Medline publication type) containing a structured abstract vs. another NEJM publication type (e.g., commentary, editorial, perspective, or case report/case series). Of the 2,484 NEJM original journal articles, 65 (2.6%) did not contain a structured abstract and were excluded. The final study database contained 2,419 NEJM records.

First authors were classified as MP or SP for NEJM publications during the study time period. For NEJM articles that credited a named study group (but not individual authors), the publication’s appendix was reviewed to identify authors. For MP first authors, the time from their initial NEJM original journal article to their second original journal article was calculated. For MP first authors with greater than two publications, the time between the initial and latest NEJM publications (prior to December 31, 2017) was calculated.

For each publication, the coauthor and collaborating author counts, study design (i.e., clinical trial), grant funding, and major disease topic by MeSH classification were identified. All analyses involving collaborating author counts were limited to publications since 2008, as that was the first year that Medline began consistently reporting collaborating authors. Based on proportions of MP vs. SP NEJM articles with these major MeSH classifications, the most frequent MeSH categories were compared. Collaborating authors were defined as those team members mentioned or acknowledged in the manuscript but not included in the author listing. Unless a new study-specific variable was separately defined, standard Medline data field definitions were applied.

Supplementary data, including author-specific characteristics (i.e., gender, advanced degrees held, self-designated major clinical specialty, institutional location, and academic rank) and publication-specific characteristics (e.g., population[s], intervention[s], comparison[s], outcome[s], and directionality of overall findings), were extracted from publicly available websites for a pilot set of records. For detailed methods, see the supplementary Appendix.

Results

Of the NEJM original articles containing a structured abstract, there were 2,419 publications evaluated with a total of 2,065 first authors identified. Of these, 75% (n = 1,816/2,419) of tallied publications were classified as SP first author publications; 25% (n = 603/2,419) were identified as MP first author publications (Figure 1).

Of the first authors identified, 88% (n = 1,816/2,065) were SP first authors; correspondingly, 12% (n = 249/2,065) were MP first authors (Table 1 and Table 2). Of the MP authors, 74% (n = 185/249) had two publications; there were only 3 individuals (1.2%) that had 7, 8, or 9 NEJM publications (Figure 2).

From 2002 (19% MP first authors) to 2017 (24% MP first authors), there was an increasing proportion of NEJM publications from MP first authors (P = 0.037; Figure 3).

For the MP first authors, the average time from initial publication to second publication was 4.2 y (SD = 3.2 y), and the average time from initial publication to last publication (prior to December 31, 2017) was 7.6 y (SD = 4.0 y; MP subgroup N = 64). This 4.2-year gap (between initial to second NEJM publication) appears quite close to the maximum time period (i.e., 5 y) funded by NIH Research Project Grant Program grants.

Overall, the average number of coauthors per publication was not different between MP vs. SP first authors (16 vs. 16, P = 0.221; Supplementary Figure 1). Across early-, mid-, and late-study time periods, the number of coauthors increased over time for both MP and SP; over these time intervals, the linearly increasing rate for coauthor counts was higher for SP vs. MP (P = 0.033).

The average number of collaborators per publication was 130, with significant MP vs. SP differences (216 vs. 100, P < 0.001; Supplementary Figure 2); no significant trend over time in collaborators per publication was observed (P = 0.6882 for SP, P = 0.2615 for MP). In contrast to the coauthor count/publication findings, the linearly increasing rate for collaborating authors/publication was similar between MP and SP authors (P = 0.4580).

The proportion of MP (96%) vs. SP (80%) clinical trials published was different (P < 0.001). From 2002 to 2017, there was an increasing proportion of clinical trials published as time progressed for both MP (relative risk [RR] = 1.006 with 95% confidence interval [CI] [1.001, 1.011]; P = 0.011) and SP first authors (RR = 1.015; CI [1.006, 1.024]; P = 0.002); there was no difference in these trends (P = 0.198). Classified by early/late time periods, there remained an increasing trend over time for both MP (P < 0.001) and SP first authors (P = 0.013), with no significant difference in MP vs. SP patterns (P = 0.322).

Among clinical trials, there were no differences in the MP vs. SP average coauthor counts/publications (SP = 16, MP = 17, P = 0.249); since 2008, however, there were dramatic differences in the average collaborating author counts for publications (SP = 116, MP = 226, P < 0.001; Supplementary Table 1). Correspondingly, the total author counts for publications since 2008 (adding coauthors and collaborating authors) were larger for the MP vs. SP clinical trial publications (244 vs. 134, P < 0.001).

For the supplementary data, the top three MeSH disease-related topics were cardiovascular disease (CVD), neoplasms, and viral diseases varying for MP vs. SP first author publications; the CVD-related publication rates varied for MP (28%) vs. SP first authors (17%; P < 0.001), but CVD-based publication rates did not change over time for either MP or SP first authors (P = 0.139 and P = 0.999, respectively).

The rate of NEJM first authors having at least one grant-funded article was high (43%). Although MP first authors were more likely than SP authors to have reported grant funding (54% vs. 42%; P < 0.001), there were no differences between the SP vs. MP funding trends over time (P = 0.934).

Supplementary data about first authors, institutions, and abstract-specific details were extracted by two authors from publicly available websites for a pilot set of these publications. The inter-rater reliability of the supplementary data capture was evaluated with > 80% agreement and Kappa upwards of 0.7, thus indicating good concordance between these two raters.

Using this supplementary data, the SP vs. MP first author characteristics of gender, institutional location (i.e., North America-based or Massachusetts-based, the state in which NEJM is based), self-designated major clinical specialty, and advanced degrees held (i.e., clinical vs. scientific vs. combined doctoral degrees) were evaluated. For this pilot study evaluating MP and SP first author differences, there was no statistically significant female gender difference (14% vs. 8%; P = 0.554), clinical specialty difference (i.e., medical specialty = 72% vs. 79% and surgical = 10% vs. 8%; P = 0.746], difference in North American location (67% vs. 67%; P = 0.968) or difference in Massachusetts-based institutional affiliation (12% vs. 4%, P = 0.491). Overall, female first authors appeared to be underrepresented for both SP and MP first authors (13%; n = 36/273).

Abstracts were reviewed to summarize each publication’s findings as positive, negative, or no differences found. There was a strong trend against neutral or negative findings being reported, though MP first authors did so more often than SP first authors (27% vs. 4%; P = 0.036). For the subgroup of MP authors, this tendency to report neutral or negative findings more often was not unique to clinical trial-based (P = 0.326) or CVD-related publications (P = 0.129). Although not definitive, these pilot findings suggest there may be a publication bias for first-time studies reporting positive findings submitted to and/or published in NEJM; furthermore, these pilot study results appear consistent with prior findings on publication bias.⁶

Evaluation by multivariable regression analyses reconfirmed that an author publishing clinical trials (odds ratio [OR] = 6.9, 95% CI [2.2, 22.0]; P = 0.0011) with grant-funding (OR = 1.7, 95% CI [1.1, 2.5]; P = 0.0085) was more likely to be an MP author; holding CVD-related topic, coauthor count, and collaborating author count constant.

Limitations

This NEJM-based study may be limited in generalizability. NEJM records from 2002 to 2017 were extracted because 2002 was the earliest date that first authors’ first names were recorded in Medline. Further, the NEJM instructions to authors were revised on July 4, 2002. Previously, instructions stated that, “If more than 12 are listed for a multicenter trial, or more than 8 authors for a study from a single institution, each author must sign a statement attesting that he or she fulfills the authorship criteria of the Uniform Requirements. No more than 12 names will be listed under the title; other names will appear in a footnote.” After July 2002, this wording was removed. Given the 2002 study start-up, it is possible that SP first author publications prior to this date may have been missed; to evaluate for a possible SP author misclassification, all SP authors were searched for any additional publications. Additionally, changes in how an author’s name was published (e.g., addition of middle name) impacted Medline record matches, and it is possible more than one individual with the same name may have had first author publications during the study time period. A careful re-evaluation of all SP first authors estimated the potential misclassification rate at approximately 1.1% (n = 20/1817). Thus, uniform adoption of author-specific identifiers such as ORCID or an author disambiguation system such as Author-ity may support future research evaluating authorship and publication patterns.^7,8

In the natural progression of an academic career, it is also conceivable that some authors may have had a first author publication and then had another paper published in last or senior author role. While this may be a noteworthy achievement and lead to underestimating the number of authors having a prestigious author role for multiple publications, it was beyond the scope of this investigation and thus not considered further here. However, authorship progression (i.e., first to last author position) may prove to be an interesting subject for future work in this field.

For many of the author-specific characteristics explored (e.g., gender differences or differences in Massachusetts-based location), this study was underpowered to detect SP vs. MP first author differences. For gender-specific SP vs. MP differences, a power calculation was performed based on the results of the preliminary data extraction. This showed that, even if the supplemental data capture was performed for all study records, this study would remain underpowered (i.e., estimated power = 0.7173) to detect a SP vs. MP difference among female first authors (assuming alpha = 0.05). Prior studies have similarly suggested a gender-bias for publications may exist for other medical and nonmedical scientific author populations.^9,10 Moreover, it would not be surprising in the future to identify that a larger proportion of MP vs. SP first authors that were Massachusetts-based, as the NEJM is a publication of the Massachusetts Medical Society. These pilot study findings raised important questions as to a potential manuscript selection bias; thus, future access to internal journal editorial office databases will be required to accurately confirm or refute these preliminary findings.

No Access to NEJM’s Internal Databases

Early in the planning phase (e-mail dated November 8, 2017), the NEJM editorial office (Dr EW Campion) was sent an invitation to participate in this proposed study (including a first draft protocol); however, he declined study participation on behalf of his NEJM editorial office’s team. If de-identified NEJM journal database information (e.g., describing author-specific and publication-specific characteristics) had been made available, however, a more comprehensive and timely assessment of the NEJM journal’s author-specific, institution-based, and publication-based MP vs. SP comparisons could have been performed. As access to internal biomedical research journal’s editorial databases is limited generally only to editorial team members, it now appears timely to initiate a dialogue among the key policy makers (e.g., Council of Science Editors or the International Committee of Medical Journal Editors) to establish new open access policies to journal-specific de-identified author and/or publication databases. Additionally, editorial offices should be encouraged to routinely report their own journal’s historical author-related and publication-related characteristics associated with MP vs. SP publications, as well as comparing published author and article characteristics to their rejected articles.

Discussion

For researchers striving to publish in high impact journals, each journal’s unique publication patterns should be researched to identify potential future “success strategies.” For the first time, this study has identified the most important NEJM MP vs. SP first author publication-related differences by aggregating data and examining trends over time. Based on NEJM records extracted from 2002 to 2017, this study found that MP first authors were more likely to publish studies that were grant-funded, trial-related designs, and focused upon the cardiovascular field as compared to SP first authors.

Interestingly, the average coauthor counts for SP vs. MP first author publications were not different; however, coauthor counts did increase over time for both SP and MP first authors. This is consistent with prior work that found highly-productive authors frequently had papers with author counts of 10–100 authors on their curriculum vitae.¹¹ The average collaborating author counts between SP vs. MP first authors were also found to be dramatically different, further highlighting the disparity in characteristics between these two groups.

The editorial teams for NEJM and other top-tier biomedical journals should be cognizant of underrepresentation of female first authors in the scientific work as they evaluate manuscripts for potential publication. Furthermore, future research should investigate factors driving this underrepresentation and identify options to close any gender-related publication gaps.

Although the future NEJM peer-review and manuscript acceptance processes may not conform to these historical patterns, biomedical research faculty hopeful to publish multiple times in NEJM should plan to write research grants to fund large-scale, multi-center clinical trials investigating CVD-related topics with an extensive team of collaborators. New courses in clinical trial designs and management, as well as grant writing, should be offered, complemented by increased professional society networking experiences, to support biomedical research faculty aspiring to publish multiple times in high profile journals.¹² For junior faculty long-term career development planning, potential senior faculty mentors—with a strong MP track record—should be identified to provide wisdom, advice, and oversight.

Similar studies of other top-tier biomedical journals (e.g., The Journal of the American Medical Association and The Lancet) should be performed to confirm or refute the generalizability of these preliminary NEJM findings. Based on data-driven evidence, future generations of biomedical research scientists may be trained and equipped with the appropriate skills (e.g., leadership, writing, and clinical trial management training) necessary to thrive in their respective fields. Moreover, biomedical research faculty should carefully review their targeted journal’s historical publication practices and authors’ characteristics to develop their own academic career development strategy for future promotion and/or tenure success.

Based on documenting publication practices for a leading biomedical science journal, this study has raised several questions worthy of further investigation: Importantly, female first authors appeared to be under-represented (13%). Also, authors with an MD degree (as opposed to other doctoral degrees) comprised the vast majority of all NEJM first authors (95%); therefore, it may be more challenging for PhDs to be published as first authors in NEJM. Interestingly, a first author’s Massachusetts-based location represented a potential advantage—with more than double the projected rate (11%, as compared to ~5% representing a 1/50th expected rate) for US-based authors. Manuscripts reporting positive findings (73.5%) appeared at much higher rates than anticipated; thus, it may be more challenging for articles with no differences found or negative findings to be published in NEJM.

For all submitted and published manuscripts, enhanced transparency along with public access to de-identified biomedical science journals’ databases should be provided to rigorously address these scientific questions raised. Public reporting by top biomedical science journals to describe their publication policies and practices should be strongly encouraged. As the project-specific and author-based characteristics associated with major biomedical science journals’ publication decisions currently remain hidden, it is now time that this historical “glass ceiling” be broken.

As the most highly influential biomedical science journal, the NEJM was selected as the initial focus based upon the latest Journal Citation Reports’ impact factor rankings. As these preliminary findings may be unique to NEJM, however, it is possible that other high impact biomedical science journals may have very different publication practices. Thus, additional bibliometric research comparing these preliminary NEJM findings across other top biomedical science journals now appears warranted.

Acknowledgements

We acknowledge the biostatistical consultation and support provided by the Biostatistical Consulting Core and the Health Science Center Reference Library staff at School of Medicine, Stony Brook University, Stony Brook, New York. Additionally, technical staff support was provided by Dr Rod Jamshidian and Ms Stella (To) Tsui.

References

1. Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A. 2005;102(46):16569–16572. https://doi.org/10.1073/pnas.0507655102.
2. Garfield E. The history and meaning of the journal impact factor. JAMA. 2006;295(1):90–93. https://doi.org/10.1001/jama.295.1.90
3. Don’t pay prizes for published science. Nature. 2017;547:137. https://doi.org/10.1038/547137a
4. Petersen AM, Wang F, Stanley HE. Methods for measuring the citations and productivity of scientists across time and discipline. Phys Rev E Stat Nonlin Soft Matter Phys. 2010;81(3 Pt 2):036114. https://doi.org/10.1103/physreve.81.036114
5. Wuchty S, Jones BF, Uzzi B. The increasing dominance of teams in production of knowledge. Science. 2007;316(5827):1036–1039. https://doi.org/10.1126/science.1136099
6. Fanelli D. Negative results are disappearing from most disciplines and countries. Scientometrics. 2012;90:891–904. https://doi.org/10.1007/s11192-011-0494-7.
7. Credit where credit is due. Nature. 2009;462(7275):825. https://doi.org/10.1038/462825a
8. Lerchenmueller MJ, Sorenson O. Author disambiguation in PubMed: evidence on the precision and recall of author-ity among NIH-funded scientists. PLoS One. 2016;11(7):e0158731. https://doi.org/10.1371/journal.pone.0158731
9. McDonald JS, McDonald RJ, Davenport MS, Jaffe TA, Cook TS, Kallmes DF, Kotsenas AL. Gender and radiology publication productivity: an examination of academic faculty from four health systems in the United States. J Am Coll Radiol. 2017;14(8):1100–1108. https://doi.org/10.1016/j.jacr.2017.04.017.
10. Bendels MHK, Muller R, Brueggmann D, Groneberg DA. Gender disparities in high-quality research revealed by Nature Index journals. PLoS One. 2018;13(1):e0189136. https://doi.org/10.1371/journal.pone.0189136
11. Ioannidis JPA, Klavans R, Boyack KW. Thousands of scientists publish a paper every five days. Nature. 2018;561:167–169. https://doi.org/10.1038/d41586-018-06185-8
12. National Institutes of Health. NIH’s definition of a clinical trial. [accessed March 18, 2020]. https://grants.nih.gov/policy/clinical-trials/definition.htm

BM Carr is from Mayo Clinic, Rochester, Minnesota; JE Krstacic, C Zhu, J Saragossi, J Yang, and AL Shroyer are from Stony Brook University, School of Medicine, Stony Brook, New York.

*Both Dr Carr and Mr Krstacic should be considered as first authors, contributing equally.

**Corresponding author: Dr A. Laurie W. Shroyer, AnnieLaurie.Shroyer@stonybrookmedicine.edu