Using Exploratory Trials to Identify Relevant Contexts and Mechanisms in Complex Electronic Health Interventions: Evaluating the Electronic Patient-Reported Outcome Tool

Carolyn Steele Gray; Janelle Gravesande; Parminder Kaur Hans; Jason X Nie; Sarah Sharpe; Mayura Loganathan; Renee Lyons; Cheryl Cott

doi:10.2196/11950

Using Exploratory Trials to Identify Relevant Contexts and Mechanisms in Complex Electronic Health Interventions: Evaluating the Electronic Patient-Reported Outcome Tool

JMIR Form Res. 2019 Feb 27;3(1):e11950. doi: 10.2196/11950.

Authors

Carolyn Steele Gray^#^{1

2}, Janelle Gravesande^#³, Parminder Kaur Hans^#⁴, Jason X Nie^#⁵, Sarah Sharpe^#⁶, Mayura Loganathan^#^{7

8}, Renee Lyons^#^{1

2}, Cheryl Cott^#⁹

Affiliations

¹ Bridgepoint Collaboratory for Research and Innovation, Lunenfeld-Tanenbaum Research Institute, Sinai Health System, Toronto, ON, Canada.
² Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada.
³ School of Rehabilitation, McMaster University, Hamilton, ON, Canada.
⁴ Corporate Services, Trillium Health Partners, Toronto, ON, Canada.
⁵ Institute for Better Health, Trillium Health Partners, Toronto, ON, Canada.
⁶ QoC Health Inc, Toronto, ON, Canada.
⁷ Ray D Wolfe Department of Family Medicine, Mount Sinai Hospital, Toronto, ON, Canada.
⁸ Department of Family and Community Medicine, Faculty of Medicine, University of Toronto, Toronto, ON, Canada.
⁹ Department of Physical Therapy, University of Toronto, Toronto, ON, Canada.

^# Contributed equally.

PMID: 30810532
PMCID: PMC6414821
DOI: 10.2196/11950

Abstract

Background: Designing appropriate studies for evaluating complex interventions, such as electronic health solutions to support integrated care, remains a methodological challenge. With the many moving parts of complex interventions, it is not always clear how program activities are connected to anticipated and unanticipated outcomes. Exploratory trials can be used to uncover determinants (or mechanisms) to inform content theory that underpins complex interventions before designing a full evaluation plan.

Objective: A multimethod exploratory trial of the electronic patient-reported outcome (ePRO) tool was conducted to uncover contexts, processes and outcome variables, and the mechanisms that link these variables before full-scale evaluation. ePRO is a mobile app and portal designed to support goal-oriented care in interdisciplinary primary health care practices (clinical-level integration). This paper offers evaluation findings and methodological insight on how to use exploratory trial data to identify relevant context, process, and outcome variables, as well as central (necessary to achieving outcomes) versus peripheral (less critical and potentially context dependent) mechanisms at play.

Methods: The 4-month trial was conducted in 2 primary health care practices in Toronto, Canada. The patients were randomized into control and intervention groups and compared pre and post on quality of life and activation outcome measures. Semistructured interviews were conducted with providers and patients in the intervention group. Narrative analysis was used to uncover dominant mechanisms that inform the intervention's content theory (how context and process variables are linked to outcomes).

Results: Overall, 7 providers, 1 administrator, and 16 patients (7-control, 9-intervention) participated in the study. This study uncovered many complex and nuanced context, process, and outcome variables at play in the intervention. Narrative analysis of patient and provider interviews revealed dominant story lines that help to tease apart central and peripheral mechanisms driving the intervention. Provider and patient story lines centered around fitting the new intervention into everyday work and life of patients and providers and meaningfulness of the intervention. These themes were moderated by patient-provider relationships going into and throughout the intervention, their comfort with technology, and the research process.

Conclusions: Identifying dominant story lines using narrative analysis helps to identify the most relevant context and process variables likely to influence study outcomes. Normalization process theory emerges as a useful theory to uncover underlying mechanisms because of its emphasis on the social production and normalization of technological, processual, and social aspects of work; all found to be critical to our intervention. The number of complex, overlapping influencing variables suggests that complex interventions such as ePRO require us to pay careful attention to central versus peripheral mechanisms that will influence study outcomes. The narrative methods presented here are shown to be useful in uncovering these mechanisms and help to guide subsequent larger evaluation studies.

Keywords: clinical trial, phase III; eHealth; health care evaluation mechanisms; mHealth; multiple chronic conditions; narrative analysis; qualitative evaluation; quantitative evaluation.

©Carolyn Steele Gray, Janelle Gravesande, Parminder Kaur Hans, Jason X Nie, Sarah Sharpe, Mayura Loganathan, Renee Lyons, Cheryl Cott. Originally published in JMIR Formative Research (http://formative.jmir.org), 27.02.2019.