External Validation of the VOCAL-Penn Cirrhosis Surgical Risk Score in 2 Large, Independent Health Systems

Abstract

Cirrhosis poses an increased risk of postoperative mortality, yet it remains challenging to accurately risk stratify patients in clinical practice. The VOCAL-Penn cirrhosis surgical risk score was recently developed and internally validated in the national Veterans Affairs health system; however, to date this score has not been evaluated in independent cohorts. The goal of this study was to compare the predictive performance of the VOCAL-Penn to the Mayo risk, Model for End-Stage Liver Disease (MELD), and MELD-sodium (MELD-Na) scores in 2 large health systems. We performed a retrospective cohort study of patients with cirrhosis undergoing surgical procedures of interest at the Beth Israel Deaconess Medical Center or University of Pennsylvania Health System from January 1, 2008, to October 1, 2015. The outcomes of interest were 30-day and 90-day postoperative mortality. Concordance statistics (C-statistics), calibration curves, Brier scores, and the index of prediction accuracy (IPA) were compared for each predictive model. A total of 855 surgical procedures were identified. The VOCAL-Penn score had the numerically highest C-statistic for 90-day postoperative mortality (eg, 0.82 versus 0.79 Mayo versus 0.78 MELD-Na versus 0.79 MELD), although differences were not statistically significant. Calibrations were excellent for the VOCAL-Penn, MELD, and MELD-Na; however, the Mayo score consistently overestimated risk. The VOCAL-Penn had the lowest Brier score and highest IPA at both time points, suggesting superior overall predictive model performance. In subgroup analyses of patients with higher MELD scores, the VOCAL-Penn had significantly higher C-statistics compared with the MELD and MELD-Na. The VOCAL-Penn score (www.vocalpennscore.com) has excellent discrimination and calibration for postoperative mortality among diverse patients with cirrhosis. Overall performance is superior to the Mayo, MELD, and MELD-Na scores. In contrast to the MELD/MELD-Na, the VOCAL-Penn retains excellent discrimination among patients with higher MELD scores.