Assessing the Efficacy and Clinical Utility of Artificial Intelligence Scribes in Urology

Joseph Moryousef; Praveen Nadesan; Michael Uy; Danny Matti; Yanbo Guo

doi:10.1016/j.urology.2024.11.061

Assessing the Efficacy and Clinical Utility of Artificial Intelligence Scribes in Urology

Urology. 2024 Nov 30:S0090-4295(24)01112-9. doi: 10.1016/j.urology.2024.11.061. Online ahead of print.

Authors

Joseph Moryousef¹, Praveen Nadesan², Michael Uy¹, Danny Matti³, Yanbo Guo⁴

Affiliations

¹ Department of Surgery, Division of Urology, McMaster University, Hamilton, ON, Canada.
² Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada.
³ Department of Surgery, Division of Urology, Western University, London, ON, Canada.
⁴ Department of Surgery, Division of Urology, McMaster University, Hamilton, ON, Canada. Electronic address: yanbo.guo@medportal.ca.

PMID: 39622441
DOI: 10.1016/j.urology.2024.11.061

Abstract

Objectives: To assess the quality of artificial intelligence scribes and to evaluate their impact on urologic practice.

Methods: Standardized reference consultation notes were created for common urologic referrals (urolithiasis, benign prostate hyperplasia, and prostate specific antigen screening) were created. Audio recordings of these simulated patient encounters were played for five freely accessible AI scribes. The outputs generated by these scribes were evaluated using a standardized survey completed by Canadian urology faculty and trainees across two academic sites.

Results: Twenty urologists responded, of which 75% reported clinical documentation as a significant source of burnout. 90% expressed openness to using AI scribes to facilitate documentation. Among the AI tools assessed, Nabla was ranked the most effective, with a favorable composite score of 68% and lowest critical error composite score (28%). Three-quarters of respondents indicated a willingness to implement AI scribes in their current form, and 89% believed AI scribes would significantly alter their practice in the future.

Conclusions: AI scribes have the potential to substantially alleviate the administrative burden associated with clinical documentation, thereby reducing burnout. However, challenges remain, particularly in ensuring accuracy, and addressing medicolegal and privacy concerns.AI scribes represent a valuable tool for reducing the documentation burden and improving patient-clinician interactions. However, they should complement, rather than replace, clinician-led documentation to maintain the quality and safety of patient care.