Electronic phenotyping of health outcomes of interest using a linked claims-electronic health record database: Findings from a machine learning pilot project

Teresa B Gibson; Michael D Nguyen; Timothy Burrell; Frank Yoon; Jenna Wong; Sai Dharmarajan; Rita Ouellet-Hellstrom; Wei Hua; Yong Ma; Elande Baro; Sarah Bloemers; Cory Pack; Adee Kennedy; Sengwee Toh; Robert Ball

doi:10.1093/jamia/ocab036

Electronic phenotyping of health outcomes of interest using a linked claims-electronic health record database: Findings from a machine learning pilot project

J Am Med Inform Assoc. 2021 Jul 14;28(7):1507-1517. doi: 10.1093/jamia/ocab036.

Authors

Affiliations

¹ Government Health and Human Services, IBM Watson Health, Bethesda, Maryland, USA.
² Food and Drug Administration, Silver Spring, Maryland, USA.
³ Harvard Medical School and Harvard Pilgrim Health Care Institute, Harvard Medical School, Boston, Massachusetts, USA.
⁴ Office of Biostatistics, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, Maryland, USA.
⁵ Division of Epidemiology II, Office of Pharmacovigilance and Epidemiology, Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, Maryland, USA.
⁶ Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, Maryland, USA.
⁷ Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts, USA.
⁸ Office of Surveillance and Epidemiology Center for Drug Evaluation and Research U.S. Food and Drug Administration, Silver Spring, Maryland, USA.

Abstract

Objective: Claims-based algorithms are used in the Food and Drug Administration Sentinel Active Risk Identification and Analysis System to identify occurrences of health outcomes of interest (HOIs) for medical product safety assessment. This project aimed to apply machine learning classification techniques to demonstrate the feasibility of developing a claims-based algorithm to predict an HOI in structured electronic health record (EHR) data.

Materials and methods: We used the 2015-2019 IBM MarketScan Explorys Claims-EMR Data Set, linking administrative claims and EHR data at the patient level. We focused on a single HOI, rhabdomyolysis, defined by EHR laboratory test results. Using claims-based predictors, we applied machine learning techniques to predict the HOI: logistic regression, LASSO (least absolute shrinkage and selection operator), random forests, support vector machines, artificial neural nets, and an ensemble method (Super Learner).

Results: The study cohort included 32 956 patients and 39 499 encounters. Model performance (positive predictive value [PPV], sensitivity, specificity, area under the receiver-operating characteristic curve) varied considerably across techniques. The area under the receiver-operating characteristic curve exceeded 0.80 in most model variations.

Discussion: For the main Food and Drug Administration use case of assessing risk of rhabdomyolysis after drug use, a model with a high PPV is typically preferred. The Super Learner ensemble model without adjustment for class imbalance achieved a PPV of 75.6%, substantially better than a previously used human expert-developed model (PPV = 44.0%).

Conclusions: It is feasible to use machine learning methods to predict an EHR-derived HOI with claims-based predictors. Modeling strategies can be adapted for intended uses, including surveillance, identification of cases for chart review, and outcomes research.

Keywords: administrative claims; electronic health records; electronic phenotyping; healthcare; rhabdomyolysis; supervised machine learning.

Publication types

Research Support, U.S. Gov't, Non-P.H.S.
Research Support, U.S. Gov't, P.H.S.

MeSH terms

Electronic Health Records*
Electronics
Humans
Machine Learning*
Outcome Assessment, Health Care
Pilot Projects

Grants and funding

HHSF223201400030I/FD/FDA HHS/United States