Machine Learning Algorithms Exceed Comorbidity Indices in Prediction of Short-Term Complications After Hip Fracture Surgery

Abstract

Background: Hip fractures are among the most morbid acute orthopaedic injuries often due to accompanying patient frailty. The purpose of this study was to determine the reliability of assessing surgical risk after hip fracture through machine learning (ML) algorithms.

Methods: The American College of Surgeons National Surgical Quality Improvement Program was queried from 2011 to 2018 and the American College of Surgeons National Surgical Quality Improvement Program hip fracture-targeted data set was queried from 2016 to 2018 for all patients undergoing surgical fixation for a diagnosis of an acute primary hip fracture. The data set was randomly split into training (80%) and testing (20%) sets. 3 ML algorithms were used to train models in the prediction of extended hospital length of stay (LOS) >13 days, death, readmissions, home discharge, transfusion, and any medical complication. Testing sets were assessed by receiver operating characteristic, positive predictive value (PPV), and negative predictive value (NPV) and were compared with models constructed from legacy comorbidity indices such as American Society of Anesthesiologists (ASA) score, modified Charlson Comorbidity Index, frailty index, and Nottingham Hip Fracture Score.

Results: Following inclusion/exclusion criteria, 95,745 cases were available in the overall data set and 22,344 in the targeted data set. ML models outperformed comorbidity indices for each complication by area under the curve (AUC) analysis (P < 0.01 for each): medical complications (AUC = 0.65, PPV = 67.5, NPV = 71.7), death (AUC = 0.80, PPV = 46.7, NPV = 94.9), extended LOS (AUC = 0.69, PPV = 71.4, NPV = 94.1), transfusion (AUC = 0.79, PPV = 64.2, NPV = 77.4), readmissions (AUC = 0.63, PPV = 0, NPV = 96.8), and home discharge (AUC = 0.74, PPV = 65.9, NPV = 76.7). In comparison, the best performing legacy index for each complication was medical complication (ASA: AUC = 0.60), death (NHFS: AUC = 0.70), extended LOS (ASA: AUC = 0.62), transfusion (ASA: AUC = 0.57), readmissions (CCI: AUC = 0.58), and home discharge (ASA: AUC = 0.61).

Conclusions: ML algorithms offer an improved method to holistically calculate preoperative risk of patient morbidity, mortality, and discharge destination. Through continued validation, risk calculators using these algorithms may inform medical decision making to providers and payers.