Standardized Comparison of Voice-Based Information and Documentation Systems to Established Systems in Intensive Care: Crossover Study

Arne Peine; Maike Gronholz; Katharina Seidl-Rathkopf; Thomas Wolfram; Ahmed Hallawa; Annika Reitz; Leo Anthony Celi; Gernot Marx; Lukas Martin

doi:10.2196/44773

Standardized Comparison of Voice-Based Information and Documentation Systems to Established Systems in Intensive Care: Crossover Study

JMIR Med Inform. 2023 Nov 28:11:e44773. doi: 10.2196/44773.

Authors

Arne Peine^{1

2}, Maike Gronholz², Katharina Seidl-Rathkopf², Thomas Wolfram², Ahmed Hallawa¹, Annika Reitz², Leo Anthony Celi^{3

4}, Gernot Marx¹, Lukas Martin^{1

2}

Affiliations

¹ Department of Intensive Care Medicine and Intermediate Care, University Hospital RWTH Aachen, Aachen, Germany.
² Clinomic Group GmbH, Aachen, Germany.
³ Laboratory of Computational Physiology, Harvard-MIT Division of Health Sciences Technology, Cambridge, MA, United States.
⁴ Beth Israel Deaconess Medical Center, Boston, MA, United States.

PMID: 38015593
PMCID: PMC10716746
DOI: 10.2196/44773

Abstract

Background: The medical teams in intensive care units (ICUs) spend increasing amounts of time at computer systems for data processing, input, and interpretation purposes. As each patient creates about 1000 data points per hour, the available information is abundant, making the interpretation difficult and time-consuming. This data flood leads to a decrease in time for evidence-based, patient-centered care. Information systems, such as patient data management systems (PDMSs), are increasingly used at ICUs. However, they often create new challenges arising from the increasing documentation burden.

Objective: New concepts, such as artificial intelligence (AI)-based assistant systems, are hence introduced to the workflow to cope with these challenges. However, there is a lack of standardized, published metrics in order to compare the various data input and management systems in the ICU setting. The objective of this study is to compare established documentation and retrieval processes with newer methods, such as PDMSs and voice information and documentation systems (VIDSs).

Methods: In this crossover study, we compare traditional, paper-based documentation systems with PDMSs and newer AI-based VIDSs in terms of performance (required time), accuracy, mental workload, and user experience in an intensive care setting. Performance is assessed on a set of 6 standardized, typical ICU tasks, ranging from documentation to medical interpretation.

Results: A total of 60 ICU-experienced medical professionals participated in the study. The VIDS showed a statistically significant advantage compared to the other 2 systems. The tasks were completed significantly faster with the VIDS than with the PDMS (1-tailed t₅₉=12.48; Cohen d=1.61; P<.001) or paper documentation (t₅₉=20.41; Cohen d=2.63; P<.001). Significantly fewer errors were made with VIDS than with the PDMS (t₅₉=3.45; Cohen d=0.45; P=.03) and paper-based documentation (t₅₉=11.2; Cohen d=1.45; P<.001). The analysis of the mental workload of VIDS and PDMS showed no statistically significant difference (P=.06). However, the analysis of subjective user perception showed a statistically significant perceived benefit of the VIDS compared to the PDMS (P<.001) and paper documentation (P<.001).

Conclusions: The results of this study show that the VIDS reduced error rate, documentation time, and mental workload regarding the set of 6 standardized typical ICU tasks. In conclusion, this indicates that AI-based systems such as the VIDS tested in this study have the potential to reduce this workload and improve evidence-based and safe patient care.

Keywords: ICU; artificial intelligence; documentation; intensive care medicine; speech-recognition; user perception; workload.

©Arne Peine, Maike Gronholz, Katharina Seidl-Rathkopf, Thomas Wolfram, Ahmed Hallawa, Annika Reitz, Leo Anthony Celi, Gernot Marx, Lukas Martin. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 28.11.2023.