Evaluation of ChatGPT-4's Performance in Therapeutic Decision-Making During Multidisciplinary Oncology Meetings for Head and Neck Squamous Cell Carcinoma

Kenza Alami; Esther Willemse; Marie Quiriny; Samuel Lipski; Celine Laurent; Vincent Donquier; Antoine Digonnet

doi:10.7759/cureus.68808

Evaluation of ChatGPT-4's Performance in Therapeutic Decision-Making During Multidisciplinary Oncology Meetings for Head and Neck Squamous Cell Carcinoma

Cureus. 2024 Sep 6;16(9):e68808. doi: 10.7759/cureus.68808. eCollection 2024 Sep.

Authors

Kenza Alami¹, Esther Willemse², Marie Quiriny², Samuel Lipski², Celine Laurent^{3

4}, Vincent Donquier², Antoine Digonnet²

Affiliations

¹ Otolaryngology, Jules Bordet Institute, Bruxelles, BEL.
² Surgical Oncology, Jules Bordet Institute, Bruxelles, BEL.
³ Otolaryngology - Head and Neck Surgery, Hôpital Ambroise-Paré, Mons, BEL.
⁴ Otolaryngology - Head and Neck Surgery, Hôpital Universitaire de Bruxelles (HUB) Erasme Hospital, Bruxelles, BEL.

Abstract

Objectives First reports suggest that artificial intelligence (AI) such as ChatGPT-4 (Open AI, ChatGPT-4, San Francisco, USA) might represent reliable tools for therapeutic decisions in some medical conditions. This study aims to assess the decisional capacity of ChatGPT-4 in patients with head and neck carcinomas, using the multidisciplinary oncology meeting (MOM) and the National Comprehensive Cancer Network (NCCN) decision as references. Methods This retrospective study included 263 patients with squamous cell carcinoma of the oral cavity, oropharynx, hypopharynx, and larynx who were followed at our institution between January 1, 2016, and December 31, 2021. The recommendation of GPT4 for the first- and second-line treatments was compared to the MOM decision and NCCN guidelines. The degrees of agreement were calculated using the Kappa method, which measures the degree of agreement between two evaluators. Results ChatGPT-4 demonstrated a moderate agreement in first-line treatment recommendations (Kappa = 0.48) and a substantial agreement (Kappa = 0.78) in second-line treatment recommendations compared to the decisions from MOM. A substantial agreement with the NCCN guidelines for both first- and second-line treatments was observed (Kappa = 0.72 and 0.66, respectively). The degree of agreement decreased when the decision included gastrostomy, patients over 70, and those with comorbidities. Conclusions The study illustrates that while ChatGPT-4 can significantly support clinical decision-making in oncology by aligning closely with expert recommendations and established guidelines, ongoing enhancements and training are crucial. The findings advocate for the continued evolution of AI tools to better handle the nuanced aspects of patient health profiles, thus broadening their applicability and reliability in clinical practice.

Keywords: artificial intelligence chatgpt-4; ent - ear nose and throat; head and neck squamous cell carcinoma (hnscc); multidisciplinary oncologic meeting; oral oncology.