Disruptive papers - A novel bibliometric tool not yet ready for primetime in pediatric urology

Seth L Teplitsky; Katelyn A Spencer; Rebecca Edwins; Lauren E Robinson; Cayla M Robinson; Adan Z Becerra; Amanda F Buchanan

doi:10.1016/j.jpurol.2024.11.011

Disruptive papers - A novel bibliometric tool not yet ready for primetime in pediatric urology

J Pediatr Urol. 2024 Nov 18:S1477-5131(24)00606-5. doi: 10.1016/j.jpurol.2024.11.011. Online ahead of print.

Authors

Seth L Teplitsky¹, Katelyn A Spencer¹, Rebecca Edwins¹, Lauren E Robinson², Cayla M Robinson², Adan Z Becerra³, Amanda F Buchanan⁴

Affiliations

¹ Department of Urology, University of Kentucky, Lexington, KY, USA.
² William T Young Library, University of Kentucky, Lexington, KY, USA.
³ Department of Surgery, Rush Medical Center, Chicago, IL, USA.
⁴ Department of Urology, University of Kentucky, Lexington, KY, USA. Electronic address: afsaltzman@gmail.com.

PMID: 39613671
DOI: 10.1016/j.jpurol.2024.11.011

Abstract

Background: Historically, word of mouth, total citation count, H-index, and expert opinion were some of the primary metrics used to identify the most influential papers. This method is subject to significant bias. The disruption score was initially created with the intent of measuring the degree to which a publication introduces a new idea, is repeatedly referenced and may be used as an additional publication evaluation metric.

Objective: To use disruptive score, a novel bibliometric index to identify influential papers in pediatric urology.

Methods: A customized search was executed using PubMed to identify papers from NEJM, JAMA, European Urology, Journal of Urology, Urology, Journal of Pediatric Urology, and Journal of Pediatric Surgery between 1954 to 2022 to identify pediatric urology related publications. Once accrued, all papers collected were run through a validated data set to identify a disruption score, where the 50 most disruptive papers were identified. The disruption score ranged from 1 to -1, with a positive score representing papers that are disruptive (challenge current knowledge), whereas a negative score represents developmental papers (help support previous publications). The 50 most disruptive papers were then cross referenced with the NIH iCite tool to identify how many times the article was cited and then the most disruptive articles and the most cited articles were compared.

Results: In total, 12,085 articles were identified and included. The database for analysis of disruption score included articles published from 1954 to 2014, leaving a total of 6544 of the papers for which a disruption score was calculated. Most of the articles identified were published in the Journal of Urology (n = 32) and Journal of Pediatric Surgery (n = 15). Fourteen of the top 50 papers were published prior to 1980 and 7 after 2000. Of the 35 total articles with iCite data available, only 8 articles had greater than 100 total citations.

Discussion: Many of the most disruptive articles identified did not have a large number of citations, highlighting the need to look beyond citation counts when assessing article importance. While this method was able to highlight some lesser-known articles, it does not appear that these citations are all highly impactful, which may be due to the relative youth of the field.

Conclusion: Disruption score provides a new way to assess the impact of publications within the field of pediatric urology. While novel, we feel this metric should be used with caution in pediatric urology at this time.

Keywords: Bibliometric; Pediatric; Urology.