Artificial Intelligence-Based Co-Facilitator (AICF) for Detecting and Monitoring Group Cohesion Outcomes in Web-Based Cancer Support Groups: Single-Arm Trial Study

Yvonne W Leung; Elise Wouterloot; Achini Adikari; Jinny Hong; Veenaajaa Asokan; Lauren Duan; Claire Lam; Carlina Kim; Kai P Chan; Daswin De Silva; Lianne Trachtenberg; Heather Rennie; Jiahui Wong; Mary Jane Esplen

doi:10.2196/43070

Artificial Intelligence-Based Co-Facilitator (AICF) for Detecting and Monitoring Group Cohesion Outcomes in Web-Based Cancer Support Groups: Single-Arm Trial Study

JMIR Cancer. 2024 Jul 22:10:e43070. doi: 10.2196/43070.

Authors

Yvonne W Leung^{1

2

3}, Elise Wouterloot¹, Achini Adikari⁴, Jinny Hong¹, Veenaajaa Asokan¹, Lauren Duan¹, Claire Lam¹, Carlina Kim¹, Kai P Chan¹, Daswin De Silva⁴, Lianne Trachtenberg^{1

5}, Heather Rennie^{1

6}, Jiahui Wong¹, Mary Jane Esplen²

Affiliations

¹ de Souza Institute, University Health Network, Toronto, ON, Canada.
² Department of Psychiatry, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada.
³ College of Professional Studies, Northeastern University, Toronto, ON, Canada.
⁴ Centre for Data Analytics and Cognition, La Trobe University, Melbourne, Australia.
⁵ Centre for Psychology and Emotional Health, Toronto, ON, Canada.
⁶ BC Cancer Agency, Vancouver, BC, Canada.

PMID: 39037754
PMCID: PMC11301110
DOI: 10.2196/43070

Abstract

Background: Commonly offered as supportive care, therapist-led online support groups (OSGs) are a cost-effective way to provide support to individuals affected by cancer. One important indicator of a successful OSG session is group cohesion; however, monitoring group cohesion can be challenging due to the lack of nonverbal cues and in-person interactions in text-based OSGs. The Artificial Intelligence-based Co-Facilitator (AICF) was designed to contextually identify therapeutic outcomes from conversations and produce real-time analytics.

Objective: The aim of this study was to develop a method to train and evaluate AICF's capacity to monitor group cohesion.

Methods: AICF used a text classification approach to extract the mentions of group cohesion within conversations. A sample of data was annotated by human scorers, which was used as the training data to build the classification model. The annotations were further supported by finding contextually similar group cohesion expressions using word embedding models as well. AICF performance was also compared against the natural language processing software Linguistic Inquiry Word Count (LIWC).

Results: AICF was trained on 80,000 messages obtained from Cancer Chat Canada. We tested AICF on 34,048 messages. Human experts scored 6797 (20%) of the messages to evaluate the ability of AICF to classify group cohesion. Results showed that machine learning algorithms combined with human input could detect group cohesion, a clinically meaningful indicator of effective OSGs. After retraining with human input, AICF reached an F₁-score of 0.82. AICF performed slightly better at identifying group cohesion compared to LIWC.

Conclusions: AICF has the potential to assist therapists by detecting discord in the group amenable to real-time intervention. Overall, AICF presents a unique opportunity to strengthen patient-centered care in web-based settings by attending to individual needs.

International registered report identifier (irrid): RR2-10.2196/21453.

Keywords: AI; LIWC; NLP; affect; artificial intelligence; cancer; corpus; emotion; emotion analysis; emotion detection; emotion lining; focus group; group cohesion; integrating human knowledge; language; linguistic; machine learning; named entity recognition; natural language processing; oncology; online support group; online therapist; sentiment analysis; speech tag; speech tagging; spoken language processing; support group; therapy; topic modeling.

©Yvonne W Leung, Elise Wouterloot, Achini Adikari, Jinny Hong, Veenaajaa Asokan, Lauren Duan, Claire Lam, Carlina Kim, Kai P Chan, Daswin De Silva, Lianne Trachtenberg, Heather Rennie, Jiahui Wong, Mary Jane Esplen. Originally published in JMIR Cancer (https://cancer.jmir.org), 22.07.2024.