Assessing the predictive accuracy of the American College of Surgeons National Surgical Quality Improvement Project Surgical Risk Calculator in open ventral hernia repair

Marten N Basta; Andrew R Bauder; Stephen J Kovach; John P Fischer

doi:10.1016/j.amjsurg.2016.01.034

Assessing the predictive accuracy of the American College of Surgeons National Surgical Quality Improvement Project Surgical Risk Calculator in open ventral hernia repair

Am J Surg. 2016 Aug;212(2):272-81. doi: 10.1016/j.amjsurg.2016.01.034. Epub 2016 May 4.

Authors

Marten N Basta¹, Andrew R Bauder², Stephen J Kovach², John P Fischer³

Affiliations

¹ Department of Plastic Surgery, Brown University and Rhode Island Hospital, Providence, RI, USA.
² Division of Plastic Surgery, Perelman School of Medicine at the University of Pennsylvania, University of Pennsylvania Health System, Philadelphia, PA, USA.
³ Division of Plastic Surgery, Perelman School of Medicine at the University of Pennsylvania, University of Pennsylvania Health System, Philadelphia, PA, USA. Electronic address: john.fischer2@uphs.upenn.edu.

PMID: 27290635
DOI: 10.1016/j.amjsurg.2016.01.034

Abstract

Background: Preoperative surgical risk assessment continues to be a critical component of clinical decision-making. The ACS Universal Risk Calculator estimates risk for several outcomes based on individual risk profiles. Although this represents a tremendous step toward improving outcomes, studies have reported inaccuracies among certain patient populations. This study aimed to assess the predictive accuracy of the American College of Surgeons' (ACS) Risk Calculator in patients undergoing open ventral hernia repair (VHR).

Methods: A review of patients undergoing open, isolated VHR between 7/1/2007 and 7/1/2014 by a single surgeon was performed. Risk factors and outcomes were collected as defined by National Surgical Quality Improvement Project. Thirty-day outcomes included serious complication, venous thromboembolism, medical morbidity, surgical site infection (SSI), unplanned reoperation, mortality, and length of stay (LOS). Patient profiles were entered into the ACS Surgical Risk Calculator and outcome-specific risk predictions recorded. Prediction accuracy was assessed using the Brier score and receiver-operator area under the curve (AUC).

Results: One hundred forty-two patients undergoing open VHR were included. ACS predictions were accurate for cardiac complications (Brier = .02), venous thromboembolism (Brier = .08), reoperation (Brier = .10), and mortality (Brier = .01). Significantly, underestimated outcomes included SSI (Brier = .14), serious complication (Brier = .30), and any complication (Brier = .34). Discrimination ranged from highly accurate (mortality, AUC = .99) to indiscriminate (SSI, AUC = .57). Predicted LOS was 3-fold shorter than observed (2.4 vs 7.4 days, P <.001).

Conclusions: The ACS Surgical Risk Calculator accurately predicted medical complications, reoperation, and 30-day mortality. However, SSIs, serious complications, and LOS were significantly underestimated. These findings suggest that additional considerations are needed to better estimate complications after open VHR.

Keywords: Hernia repair; Risk modeling; Risk stratification; Surgical complications; Surgical outcomes; Ventral hernia.

MeSH terms

Aged
Aged, 80 and over
Female
Hernia, Ventral / surgery*
Herniorrhaphy / adverse effects*
Humans
Male
Middle Aged
Quality Improvement
Retrospective Studies
Risk Assessment
Risk Factors