Application of a natural language processing artificial intelligence tool in psoriasis: A cross-sectional comparative study on identifying affected areas in patients' data

Abstract

Psoriasis is an immune-mediated skin disease affecting approximately 3% of the global population. Proper management of this condition necessitates the assessment of the body surface area and the involvement of nails and joints. The integration of natural language processing with electronic medical records (EMRs) has recently shown promise in advancing disease classification and research. This study evaluates the performance of ChatGPT-4, a commercial artificial intelligence platform, in analyzing unstructured EMR data of psoriasis patients, particularly in identifying affected body areas. The study analyzed EMR data from 94 patients treated at the Dermatology Department and Psoriasis Outpatient Clinic of Sheba Medical Center between 2008 and 2022. The data were processed using the ChatGPT-4 interface to identify and report the body areas affected by psoriasis. These identified areas were then categorized, and the accuracy of ChatGPT-4's analysis was compared with that of a senior dermatologist. The results revealed that the dermatologist identified 477 psoriasis-affected body areas. ChatGPT-4 accurately recognized 443 (92.8%) of these areas, missed 34, and incorrectly identified 30 areas as affected. From 94 cases, nail involvement was detected in 32 cases (34.0%), with ChatGPT-4 correctly identifying 29 cases. Joint involvement was noted in 25 cases (26.6%), with 24 correctly identified using ChatGPT-4. Complete accuracy was achieved in 54 cases (57.4%), although inaccuracies were observed in 40 cases (42.6%). We found that cases with more characters, words, or identified body areas were more prone to errors, suggesting that increased data complexity heightens the likelihood of inaccuracies in artificial intelligence analysis. ChatGPT-4 demonstrated high performance in analyzing detailed and complex unstructured EMR data from patients with psoriasis, effectively identifying involved body areas, including nails and joints. This highlights the potential of NLP algorithms to enhance the analysis of unstructured EMR data for both clinical follow-up and research purposes.