Awareness and Use of ChatGPT and Large Language Models: A Prospective Cross-sectional Global Survey in Urology

Michael Eppler; Conner Ganjavi; Lorenzo Storino Ramacciotti; Pietro Piazza; Severin Rodler; Enrico Checcucci; Juan Gomez Rivas; Karl F Kowalewski; Ines Rivero Belenchón; Stefano Puliatti; Mark Taratkin; Alessandro Veccia; Loïc Baekelandt; Jeremy Y-C Teoh; Bhaskar K Somani; Marcelo Wroclawski; Andre Abreu; Francesco Porpiglia; Inderbir S Gill; Declan G Murphy; David Canes; Giovanni E Cacciamani

doi:10.1016/j.eururo.2023.10.014

Awareness and Use of ChatGPT and Large Language Models: A Prospective Cross-sectional Global Survey in Urology

Eur Urol. 2024 Feb;85(2):146-153. doi: 10.1016/j.eururo.2023.10.014. Epub 2023 Nov 4.

Authors

Michael Eppler¹, Conner Ganjavi¹, Lorenzo Storino Ramacciotti¹, Pietro Piazza², Severin Rodler³, Enrico Checcucci⁴, Juan Gomez Rivas⁵, Karl F Kowalewski⁶, Ines Rivero Belenchón⁷, Stefano Puliatti⁸, Mark Taratkin⁹, Alessandro Veccia¹⁰, Loïc Baekelandt¹¹, Jeremy Y-C Teoh¹², Bhaskar K Somani¹³, Marcelo Wroclawski¹⁴, Andre Abreu¹, Francesco Porpiglia⁴, Inderbir S Gill¹, Declan G Murphy¹⁵, David Canes¹⁶, Giovanni E Cacciamani¹⁷

Affiliations

¹ USC Institute of Urology and Catherine and Joseph Aresty Department of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA; AI Center, USC Institute of Urology, University of Southern California, Los Angeles, CA, USA.
² Division of Urology, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy.
³ Department of Urology, Klinikum der Universität München, Munich, Germany.
⁴ Department of Surgery, FPO-IRCCS Candiolo Cancer Institute, Candiolo, Italy.
⁵ Department of Urology, Clinico San Carlos University Hospital, Madrid, Spain.
⁶ Department of Urology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany.
⁷ Urology and Nephrology Department, Virgen del Rocío University Hospital, Seville, Spain.
⁸ Urology Department, University of Modena and Reggio Emilia, Modena, Italy.
⁹ Institute for Urology and Reproductive Health, Sechenov University, Moscow, Russia.
¹⁰ Department of Urology, Azienda Ospedaliera Universitaria Integrata Verona, Verona, Italy.
¹¹ Department of Urology, University Hospitals Leuven, Leuven, Belgium.
¹² Department of Surgery, S.H. Ho Urology Centre, The Chinese University of Hong Kong, Hong Kong, China.
¹³ University Hospital Southampton NHS Foundation Trust, Southampton, UK.
¹⁴ Hospital Israelita Albert Einstein, São Paulo, Brazil; Beneficência Portuguesa de São Paulo, São Paulo, Brazil; Faculdade de Medicina do ABC, Santo Andre, Brazil.
¹⁵ Division of Cancer Surgery, Peter MacCallum Cancer Centre, University of Melbourne, Melbourne, Australia.
¹⁶ Division of Urology, Lahey Hospital & Medical Center, Burlington, MA, USA.
¹⁷ USC Institute of Urology and Catherine and Joseph Aresty Department of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA; AI Center, USC Institute of Urology, University of Southern California, Los Angeles, CA, USA. Electronic address: giovanni.cacciamani@med.usc.edu.

PMID: 37926642
DOI: 10.1016/j.eururo.2023.10.014

Abstract

Background: Since its release in November 2022, ChatGPT has captivated society and shown potential for various aspects of health care.

Objective: To investigate potential use of ChatGPT, a large language model (LLM), in urology by gathering opinions from urologists worldwide.

Design, setting, and participants: An open web-based survey was distributed via social media and e-mail chains to urologists between April 20, 2023 and May 5, 2023. Participants were asked to answer questions related to their knowledge and experience with artificial intelligence, as well as their opinions of potential use of ChatGPT/LLMs in research and clinical practice.

Outcome measurements and statistical analysis: Data are reported as the mean and standard deviation for continuous variables, and the frequency and percentage for categorical variables. Charts and tables are used as appropriate, with descriptions of the chart types and the measures used. The data are reported in accordance with the Checklist for Reporting Results of Internet E-Surveys (CHERRIES).

Results and limitations: A total of 456 individuals completed the survey (64% completion rate). Nearly half (47.7%) reported that they use ChatGPT/LLMs in their academic practice, with fewer using the technology in clinical practice (19.8%). More than half (62.2%) believe there are potential ethical concerns when using ChatGPT for scientific or academic writing, and 53% reported that they have experienced limitations when using ChatGPT in academic practice.

Conclusions: Urologists recognise the potential of ChatGPT/LLMs in research but have concerns regarding ethics and patient acceptance. There is a desire for regulations and guidelines to ensure appropriate use. In addition, measures should be taken to establish rules and guidelines to maximise safety and efficiency when using this novel technology.

Patient summary: A survey asked 456 urologists from around the world about using an artificial intelligence tool called ChatGPT in their work. Almost half of them use ChatGPT for research, but not many use it for patients care. The resonders think ChatGPT could be helpful, but they worry about problems like ethics and want rules to make sure it's used safely.

Keywords: ChatGPT; Ethics; Global; Large language models; OpenAI; Survey; Urology.

MeSH terms

Artificial Intelligence
Cross-Sectional Studies
Humans
Language
Prospective Studies
Urology*