Using the Surgical Risk Preoperative Assessment System to Define the "High Risk" Surgical Patient

Preoperative Prediction of Unplanned Reoperation in a Broad Surgical Population

INTRODUCTION

Unplanned reoperation is an undesirable outcome with considerable risks and an increasingly assessed quality of care metric. There are no preoperative prediction models for reoperation after an index surgery in a broad surgical population in the literature. The Surgical Risk Preoperative Assessment System (SURPAS) preoperatively predicts 12 postoperative adverse events using 8 preoperative variables, but its ability to predict unplanned reoperation has not been assessed. This study's objective was to determine whether the SURPAS model could accurately predict unplanned reoperation.

METHODS

This was a retrospective analysis of the American College of Surgeons' National Surgical Quality Improvement Program adult database, 2012-2018. An unplanned reoperation was defined as any unintended operation within 30 d of an initial scheduled operation. The 8-variable SURPAS model and a 29-variable "full" model, incorporating all available American College of Surgeons' National Surgical Quality Improvement Program nonlaboratory preoperative variables, were developed using multiple logistic regression and compared using discrimination and calibration metrics: C-indices (C), Hosmer-Lemeshow observed-to-expected plots, and Brier scores (BSs). The internal chronological validation of the SURPAS model was conducted using "training" (2012-2017) and "test" (2018) datasets.

RESULTS

Of 5,777,108 patients, 162,387 (2.81%) underwent an unplanned reoperation. The SURPAS model's C-index of 0.748 was 99.20% of that for the full model (C = 0.754). Hosmer-Lemeshow plots showed good calibration for both models and BSs were similar (BS = 0.0264, full; BS = 0.0265, SURPAS). Internal chronological validation results were similar for the training (C = 0.749, BS = 0.0268) and test (C = 0.748, BS = 0.0250) datasets.

CONCLUSIONS

The SURPAS model accurately predicted unplanned reoperation and was internally validated. Unplanned reoperation can be integrated into the SURPAS tool to provide preoperative risk assessment of this outcome, which could aid patient risk education.

External validation of the Codman score in colorectal surgery: a pragmatic tool to drive quality improvement

AIM

The simple six-variable Codman score is a tool designed to reduce the complexity of contemporary risk-adjusted postoperative mortality rate predictions. We sought to externally validate the Codman score in colorectal surgery.

METHODS

The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) participant user file and colectomy targeted dataset of 2020 were merged. A Codman score (composed of six variables: age, American Society of Anesthesiologists score, emergency status, degree of sepsis, functional status and preoperative blood transfusion) was assigned to every patient. The primary outcome was in-hospital mortality and secondary outcome was morbidity at 30 days. Logistic regression analyses were performed using the Codman score and the ACS NSQIP mortality and morbidity algorithms as independent variables for the primary and secondary outcomes. The predictive performance of discrimination area under receiver operating curve (AUC) and calibration of the Codman score and these algorithms were compared.

RESULTS

A total of 40 589 patients were included and a Codman score was generated for 40 557 (99.02%) patients. The median Codman score was 3 (interquartile range 1-4). To predict mortality, the Codman score had an AUC of 0.92 (95% CI 0.91-0.93) compared to the NSQIP mortality score 0.93 (95% CI 0.92-0.94). To predict morbidity, the Codman score had an AUC of 0.68 (95% CI 0.66-0.68) compared to the NSQIP morbidity score 0.72 (95% CI 0.71-0.73). When body mass index and surgical approach was added to the Codman score, the performance was no different to the NSQIP morbidity score. The calibration of observed versus expected predictions was almost perfect for both the morbidity and mortality NSQIP predictions, and only well fitted for Codman scores of less than 4 and greater than 7.

CONCLUSION

We propose that the six-variable Codman score is an efficient and actionable method for generating validated risk-adjusted outcome predictions and comparative benchmarks to drive quality improvement in colorectal surgery.

Development and validation of a prediction model for conversion of outpatient to inpatient surgery

BACKGROUND

Unplanned hospital admission after intended outpatient surgery is an undesirable outcome. We aimed to develop a prediction model that estimates a patient's risk of conversion from outpatient surgery to inpatient hospitalization.

METHODS

This was a retrospective analysis using the American College of Surgeons National Surgical Quality Improvement Program database, 2005 to 2018. Conversion from outpatient to inpatient surgery was defined as having outpatient surgery and >1 day hospital stay. The Surgical Risk Preoperative Assessment System was developed using multiple logistic regression on a training dataset (2005-2016) and compared to a model using the 26 relevant variables in the American College of Surgeons National Surgical Quality Improvement Program. The Surgical Risk Preoperative Assessment System was validated using a testing dataset (2017-2018). Performance statistics and Hosmer-Lemeshow plots were compared. Two high-risk definitions were compared: (1) the maximum Youden index, and (2) the cohort above the tenth decile of risk on the Hosmer-Lemeshow plot. The sensitivities, specificities, positive predictive values, negative predictive values, and accuracies were compared.

RESULTS

In all, 2,822,379 patients were included; 3.6% of patients unexpectedly converted to inpatient. The 6-variable Surgical Risk Preoperative Assessment System model performed comparably to the 26-variable American College of Surgeons National Surgical Quality Improvement Program model (c-indices = 0.818 vs. 0.823; Brier scores = 0.0308 vs 0.0306, respectively). The Surgical Risk Preoperative Assessment System performed well on internal validation (c-index = 0.818, Brier score = 0.0341). The tenth decile of risk definition had higher specificity, positive predictive values, and accuracy than the maximum Youden index definition, while having lower sensitivity.

CONCLUSION

The Surgical Risk Preoperative Assessment System accurately predicted a patient's risk of unplanned outpatient-to-inpatient conversion. Patients at higher risk should be considered for inpatient surgery, while lower risk patients could safely undergo operations at ambulatory surgery centers.

Development and Validation of a Multivariable Prediction Model for Postoperative Intensive Care Unit Stay in a Broad Surgical Population

IMPORTANCE

Despite limited capacity and expensive cost, there are minimal objective data to guide postoperative allocation of intensive care unit (ICU) beds. The Surgical Risk Preoperative Assessment System (SURPAS) uses 8 preoperative variables to predict many common postoperative complications, but it has not yet been evaluated in predicting postoperative ICU admission.

OBJECTIVE

To determine if the SURPAS model could accurately predict postoperative ICU admission in a broad surgical population.

DESIGN, SETTING, AND PARTICIPANTS

This decision analytical model was a retrospective, observational analysis of prospectively collected patient data from the 2012 to 2018 American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) database, which were merged with individual patients' electronic health record data to capture postoperative ICU use. Multivariable logistic regression modeling was used to determine how the 8 preoperative variables of the SURPAS model predicted ICU use compared with a model inputting all 28 preoperatively available NSQIP variables. Data included in the analysis were collected for the ACS NSQIP at 5 hospitals (1 tertiary academic center, 4 academic affiliated hospitals) within the University of Colorado Health System between January 1, 2012, and December 31, 2018. Included patients were those undergoing surgery in 9 surgical specialties during the 2012 to 2018 period. Data were analyzed from May 29 to July 30, 2021.

EXPOSURE

Surgery in 9 surgical specialties, including general, gynecology, orthopedic, otolaryngology, plastic, thoracic, urology, vascular, and neurosurgery.

MAIN OUTCOMES AND MEASURES

Use of ICU care up to 30 days after surgery.

RESULTS

A total of 34 568 patients were included in the analytical data set: 32 032 (92.7%) in the cohort without postoperative ICU use and 2545 (7.4%) in the cohort with postoperative ICU use (no ICU use: mean [SD] age, 54.9 [16.6] years; 18 188 women [56.8%]; ICU use: mean [SD] age, 60.3 [15.3] years; 1333 men [52.4%]). For the internal chronologic validation of the 7-variable SURPAS model, data from 2012 to 2016 were used as the training data set (n = 24 250, 70.2% of the total sample size of 34 568) and data from 2017 to 2018 were used as the test data set (n = 10 318, 29.8% of the total sample size of 34 568). The C statistic improved in the test data set compared with the training data set (0.933; 95% CI, 0.924-0.941 vs 0.922; 95% CI, 0.917-0.928), whereas the Brier score was slightly worse in the test data set compared with the training data set (0.045; 95% CI, 0.042-0.048 vs 0.045; 95% CI, 0.043-0.047). The SURPAS model compared favorably with the model inputting all 28 NSQIP variables, with both having good calibration between observed and expected outcomes in the Hosmer-Lemeshow graphs and similar Brier scores (model inputting all variables, 0.044; 95% CI, 0.043-0.048; SURPAS model, 0.045; 95% CI, 0.042-0.046) and C statistics (model inputting all variables, 0.929; 95% CI, 0.925-0.934; SURPAS model, 0.925; 95% CI, 0.921-0.930).

CONCLUSIONS AND RELEVANCE

Results of this decision analytical model study revealed that the SURPAS prediction model accurately predicted postoperative ICU use across a diverse surgical population. These results suggest that the SURPAS prediction model can be used to help with preoperative planning and resource allocation of limited ICU beds.