Accurate preoperative prediction of unplanned 30-day postoperative readmission using 8 predictor variables

Postoperative Discharge Destination Impacts 30-Day Outcomes: A National Surgical Quality Improvement Program Multi-Specialty Surgical Cohort Analysis

Surgical patients can be discharged to a variety of facilities which vary widely in intensity of care. Postoperative readmissions have been found to be more strongly associated with post-discharge events than pre-discharge complications, indicating the importance of discharge destination. We sought to evaluate the association between discharge destination and 30-day outcomes. A retrospective cohort study was conducted using the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) database. Patients were dichotomized based on discharge destination: home versus non-home. The main outcome of interest was 30-day unplanned readmission. The secondary outcomes included post-discharge pulmonary, infectious, thromboembolic, and bleeding complications, as well as death. In this cohort study of over 1.5 million patients undergoing common surgical procedures across eight surgical specialties, we found non-home discharge to be associated with adverse 30-day post-operative outcomes, namely, unplanned readmissions, post-discharge pulmonary, infectious, thromboembolic, and bleeding complications, as well as death. Non-home discharge is associated with worse 30-day outcomes among patients undergoing common surgical procedures. Patients and caregivers should be counseled regarding discharge destination, as non-home discharge is associated with adverse post-operative outcomes.

Development and validation of a multivariable preoperative prediction model for postoperative length of stay in a broad inpatient surgical population

BACKGROUND

Postoperative length of stay is a meaningful patient-centered outcome and an important determinant of healthcare costs. The Surgical Risk Preoperative Assessment System preoperatively predicts 12 postoperative adverse events using 8 preoperative variables, but its ability to predict postoperative length of stay has not been assessed. We aimed to determine whether the Surgical Risk Preoperative Assessment System variables could accurately predict postoperative length of stay up to 30 days in a broad inpatient surgical population.

METHODS

This was a retrospective analysis of the American College of Surgeons' National Surgical Quality Improvement Program adult database from 2012 to 2018. A model using the Surgical Risk Preoperative Assessment System variables and a 28-variable "full" model, incorporating all available American College of Surgeons' National Surgical Quality Improvement Program preoperative nonlaboratory variables, were fit to the analytical cohort (2012-2018) using multiple linear regression and compared using model performance metrics. Internal chronological validation of the Surgical Risk Preoperative Assessment System model was conducted using training (2012-2017) and test (2018) datasets.

RESULTS

We analyzed 3,295,028 procedures. The adjusted R2 for the Surgical Risk Preoperative Assessment System model fit to this cohort was 93.3% of that for the full model (0.347 vs 0.372). In the internal chronological validation of the Surgical Risk Preoperative Assessment System model, the adjusted R2 for the test dataset was 97.1% of that for the training dataset (0.3389 vs 0.3489).

CONCLUSION

The parsimonious Surgical Risk Preoperative Assessment System model can preoperatively predict postoperative length of stay up to 30 days for inpatient surgical procedures almost as accurately as a model using all 28 American College of Surgeons' National Surgical Quality Improvement Program preoperative nonlaboratory variables and has shown acceptable internal chronological validation.

Preoperative Prediction of Unplanned Reoperation in a Broad Surgical Population

INTRODUCTION

Unplanned reoperation is an undesirable outcome with considerable risks and an increasingly assessed quality of care metric. There are no preoperative prediction models for reoperation after an index surgery in a broad surgical population in the literature. The Surgical Risk Preoperative Assessment System (SURPAS) preoperatively predicts 12 postoperative adverse events using 8 preoperative variables, but its ability to predict unplanned reoperation has not been assessed. This study's objective was to determine whether the SURPAS model could accurately predict unplanned reoperation.

METHODS

This was a retrospective analysis of the American College of Surgeons' National Surgical Quality Improvement Program adult database, 2012-2018. An unplanned reoperation was defined as any unintended operation within 30 d of an initial scheduled operation. The 8-variable SURPAS model and a 29-variable "full" model, incorporating all available American College of Surgeons' National Surgical Quality Improvement Program nonlaboratory preoperative variables, were developed using multiple logistic regression and compared using discrimination and calibration metrics: C-indices (C), Hosmer-Lemeshow observed-to-expected plots, and Brier scores (BSs). The internal chronological validation of the SURPAS model was conducted using "training" (2012-2017) and "test" (2018) datasets.

RESULTS

Of 5,777,108 patients, 162,387 (2.81%) underwent an unplanned reoperation. The SURPAS model's C-index of 0.748 was 99.20% of that for the full model (C = 0.754). Hosmer-Lemeshow plots showed good calibration for both models and BSs were similar (BS = 0.0264, full; BS = 0.0265, SURPAS). Internal chronological validation results were similar for the training (C = 0.749, BS = 0.0268) and test (C = 0.748, BS = 0.0250) datasets.

CONCLUSIONS

The SURPAS model accurately predicted unplanned reoperation and was internally validated. Unplanned reoperation can be integrated into the SURPAS tool to provide preoperative risk assessment of this outcome, which could aid patient risk education.

Development and validation of a prediction model for conversion of outpatient to inpatient surgery

BACKGROUND

Unplanned hospital admission after intended outpatient surgery is an undesirable outcome. We aimed to develop a prediction model that estimates a patient's risk of conversion from outpatient surgery to inpatient hospitalization.

METHODS

This was a retrospective analysis using the American College of Surgeons National Surgical Quality Improvement Program database, 2005 to 2018. Conversion from outpatient to inpatient surgery was defined as having outpatient surgery and >1 day hospital stay. The Surgical Risk Preoperative Assessment System was developed using multiple logistic regression on a training dataset (2005-2016) and compared to a model using the 26 relevant variables in the American College of Surgeons National Surgical Quality Improvement Program. The Surgical Risk Preoperative Assessment System was validated using a testing dataset (2017-2018). Performance statistics and Hosmer-Lemeshow plots were compared. Two high-risk definitions were compared: (1) the maximum Youden index, and (2) the cohort above the tenth decile of risk on the Hosmer-Lemeshow plot. The sensitivities, specificities, positive predictive values, negative predictive values, and accuracies were compared.

RESULTS

In all, 2,822,379 patients were included; 3.6% of patients unexpectedly converted to inpatient. The 6-variable Surgical Risk Preoperative Assessment System model performed comparably to the 26-variable American College of Surgeons National Surgical Quality Improvement Program model (c-indices = 0.818 vs. 0.823; Brier scores = 0.0308 vs 0.0306, respectively). The Surgical Risk Preoperative Assessment System performed well on internal validation (c-index = 0.818, Brier score = 0.0341). The tenth decile of risk definition had higher specificity, positive predictive values, and accuracy than the maximum Youden index definition, while having lower sensitivity.

CONCLUSION

The Surgical Risk Preoperative Assessment System accurately predicted a patient's risk of unplanned outpatient-to-inpatient conversion. Patients at higher risk should be considered for inpatient surgery, while lower risk patients could safely undergo operations at ambulatory surgery centers.

Development and Validation of a Multivariable Prediction Model for Postoperative Intensive Care Unit Stay in a Broad Surgical Population

IMPORTANCE

Despite limited capacity and expensive cost, there are minimal objective data to guide postoperative allocation of intensive care unit (ICU) beds. The Surgical Risk Preoperative Assessment System (SURPAS) uses 8 preoperative variables to predict many common postoperative complications, but it has not yet been evaluated in predicting postoperative ICU admission.

OBJECTIVE

To determine if the SURPAS model could accurately predict postoperative ICU admission in a broad surgical population.

DESIGN, SETTING, AND PARTICIPANTS

This decision analytical model was a retrospective, observational analysis of prospectively collected patient data from the 2012 to 2018 American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) database, which were merged with individual patients' electronic health record data to capture postoperative ICU use. Multivariable logistic regression modeling was used to determine how the 8 preoperative variables of the SURPAS model predicted ICU use compared with a model inputting all 28 preoperatively available NSQIP variables. Data included in the analysis were collected for the ACS NSQIP at 5 hospitals (1 tertiary academic center, 4 academic affiliated hospitals) within the University of Colorado Health System between January 1, 2012, and December 31, 2018. Included patients were those undergoing surgery in 9 surgical specialties during the 2012 to 2018 period. Data were analyzed from May 29 to July 30, 2021.

EXPOSURE

Surgery in 9 surgical specialties, including general, gynecology, orthopedic, otolaryngology, plastic, thoracic, urology, vascular, and neurosurgery.

MAIN OUTCOMES AND MEASURES

Use of ICU care up to 30 days after surgery.

RESULTS

A total of 34 568 patients were included in the analytical data set: 32 032 (92.7%) in the cohort without postoperative ICU use and 2545 (7.4%) in the cohort with postoperative ICU use (no ICU use: mean [SD] age, 54.9 [16.6] years; 18 188 women [56.8%]; ICU use: mean [SD] age, 60.3 [15.3] years; 1333 men [52.4%]). For the internal chronologic validation of the 7-variable SURPAS model, data from 2012 to 2016 were used as the training data set (n = 24 250, 70.2% of the total sample size of 34 568) and data from 2017 to 2018 were used as the test data set (n = 10 318, 29.8% of the total sample size of 34 568). The C statistic improved in the test data set compared with the training data set (0.933; 95% CI, 0.924-0.941 vs 0.922; 95% CI, 0.917-0.928), whereas the Brier score was slightly worse in the test data set compared with the training data set (0.045; 95% CI, 0.042-0.048 vs 0.045; 95% CI, 0.043-0.047). The SURPAS model compared favorably with the model inputting all 28 NSQIP variables, with both having good calibration between observed and expected outcomes in the Hosmer-Lemeshow graphs and similar Brier scores (model inputting all variables, 0.044; 95% CI, 0.043-0.048; SURPAS model, 0.045; 95% CI, 0.042-0.046) and C statistics (model inputting all variables, 0.929; 95% CI, 0.925-0.934; SURPAS model, 0.925; 95% CI, 0.921-0.930).

CONCLUSIONS AND RELEVANCE

Results of this decision analytical model study revealed that the SURPAS prediction model accurately predicted postoperative ICU use across a diverse surgical population. These results suggest that the SURPAS prediction model can be used to help with preoperative planning and resource allocation of limited ICU beds.

Associations between preoperative risks of postoperative complications: Results of an analysis of 4.8 Million ACS-NSQIP patients

BACKGROUND

Surgical Risk Preoperative Assessment System (SURPAS) estimates patient's preoperative risk of 12 postoperative complications, yet little is known about associations between these probabilities- We sought to examine relationships between predicted probabilities.

METHODS

Risk of 12 postoperative complications was calculated using SURPAS and the 2012-2018 ACS-NSQIP database. Pearson correlation coefficients (r) were computed to examine relationships between predicted outcomes. "High-risk" was predicted risk in the 10th decile.

RESULTS

4,777,267 patients were included. 71.1% were not high risk, 10.7% were high risk for 1, and 18.2% were high risk for ≥2 complications. High mortality risk was associated with high risk for pulmonary (r = 0.94), cardiac (r = 0.98), renal (r = 0.93), and stroke (0.96) complications. Patients high-risk for ≥2 complications had the most comorbidities and actual adverse outcomes.

CONCLUSIONS

High preoperative risk for certain postoperative complications had strong correlations. 18.2% of patients were high-risk for ≥2 complications and could be targeted for risk reduction interventions.

Comparison of Preoperative Surgical Risk Estimated by Thoracic Surgeons Versus a Standardized Surgical Risk Prediction Tool

Considerable variability exists between surgeons' assessments of a patient's individual pre-operative surgical risk. Surgical risk calculators are not routinely used despite their validation. We sought to compare thoracic surgeons' prediction of patients' risk of postoperative adverse outcomes versus a surgical risk calculator, the Surgical Risk Preoperative Assessment System (SURPAS). We developed vignettes from 30 randomly selected patients who underwent thoracic surgery in the American College of Surgeons' National Surgical Quality Improvement Program (NSQIP) database. Twelve thoracic surgeons estimated patients' preoperative risks of postoperative morbidity and mortality. These were compared to SURPAS estimates of the same vignettes. C-indices and Brier scores were calculated for the surgeons' and SURPAS estimates. Agreement between surgeon estimates was examined using intraclass correlation coefficients (ICCs). Surgeons estimated higher morbidity risk compared to SURPAS for low-risk patients (ASA classes 1-2, 11.5% vs. 5.1%, p=<0.001) and lower morbidity risk compared to SURPAS for high-risk patients (ASA class 5, 37.6% vs. 69.8%, p<0.001). This trend also occurred in high-risk patients for mortality (ASA 5, 11.1% vs. 44.3%, p<0.001). C-indices for SURPAS vs. surgeons were 0.84 vs. 0.76 (p=0.3) for morbidity and 0.98 vs. 0.85 (p=0.001) for mortality. Brier scores for SURPAS vs. surgeons were 0.1579 vs. 0.1986 for morbidity (p=0.03) and 0.0409 vs. 0.0543 for mortality (p=0.006). ICCs showed that surgeons had moderate risk agreement for morbidity (ICC=0.654) and mortality (ICC=0.507). Thoracic surgeons and patients could benefit from using a surgical risk calculator to better estimate patients' surgical risks during the informed consent process.

Using the Surgical Risk Preoperative Assessment System to Define the "High Risk" Surgical Patient

BACKGROUND

Defining a "high risk" surgical population remains challenging. Using the Surgical Risk Preoperative Assessment System (SURPAS), we sought to define "high risk" groups for adverse postoperative outcomes.

MATERIALS AND METHODS

We retrospectively analyzed the 2009-2018 American College of Surgeons National Surgical Quality Improvement Program database. SURPAS calculated probabilities of 12 postoperative adverse events. The Hosmer Lemeshow graphs of deciles of risk and maximum Youden index were compared to define "high risk."

RESULTS

Hosmer-Lemeshow plots suggested the "high risk" patient could be defined by the 10th decile of risk. Maximum Youden index found lower cutoff points for defining "high risk" patients and included more patients with events. This resulted in more patients classified as "high risk" and higher number needed to treat to prevent one complication. Some specialties (thoracic, vascular, general) had more "high risk" patients, while others (otolaryngology, plastic) had lower proportions.

CONCLUSIONS

SURPAS can define the "high risk" surgical population that may benefit from risk-mitigating interventions.

Accuracy of the surgical risk preoperative assessment system universal risk calculator in predicting risk for patients undergoing selected operations in 9 specialty areas

BACKGROUND

The universal Surgical Risk Preoperative Assessment System prediction models for postoperative adverse outcomes have good accuracy for estimating risk in broad surgical populations and for surgical specialties. The accuracy in individual operations has not yet been assessed. The objective of this study was to evaluate the Surgical Risk Preoperative Assessment System in predicting adverse outcomes for selected individual operations.

METHODS

The Surgical Risk Preoperative Assessment System models were applied to the top 2 most frequent common procedural terminology codes in 9 surgical specialties and 5 additional common general surgical operations in the 2009 to 2018 database of the American College of Surgeons National Surgical Quality Improvement Program. Goodness of fit statistics were estimated, including c-indices for discrimination, Hosmer-Lemeshow graphs and P values for calibration, overall observed versus expected event rates, and Brier scores.

RESULTS

The total sample size was 2,020,172, which represented 29% of the 6.9 million operations in the American College of Surgeons National Surgical Quality Improvement Program database. Average c-indices across 12 outcomes were acceptable (≥0.70) for 13 (56.5%) of the 23 operations. Overall observed-to-expected rates were similar for mortality and overall morbidity across the 23 operations. Hosmer-Lemeshow graphs over quintiles of risk comparing observed-to-expected rates of mortality and overall morbidity were similar for 52% and 70% of operations, respectively. Model performance was better in less complex operations and those done in patients with lower preoperative risk.

CONCLUSION

Surgical Risk Preoperative Assessment System displayed accuracy in estimating postoperative adverse events for some of the 23 operations studied, but not all. In the procedures where Surgical Risk Preoperative Assessment System was not accurate, developing disease or operation-specific risk models might be appropriate.

Development and validation of prediction scores for nosocomial infections, reoperations, and adverse events in the daily clinical setting of neurosurgical patients with cerebral and spinal tumors

OBJECTIVE

Various quality indicators are currently under investigation, aiming at measuring the quality of care in neurosurgery; however, the discipline currently lacks practical scoring systems for accurately assessing risk. The aim of this study was to develop three accurate, easy-to-use risk scoring systems for nosocomial infections, reoperations, and adverse events for patients with cerebral and spinal tumors.

METHODS

The authors developed a semiautomatic registry with administrative and clinical data and included all patients with spinal or cerebral tumors treated between September 2017 and May 2019. Patients were further divided into development and validation cohorts. Multivariable logistic regression models were used to develop risk scores by assigning points based on β coefficients, and internal validation of the scores was performed.

RESULTS

In total, 1000 patients were included. An unplanned 30-day reoperation was observed in 6.8% of patients. Nosocomial infections were documented in 7.4% of cases and any adverse event in 14.5%. The risk scores comprise variables such as emergency admission, nursing care level, ECOG performance status, and inflammatory markers on admission. Three scoring systems, NoInfECT for predicting the incidence of nosocomial infections (low risk, 1.8%; intermediate risk, 8.1%; and high risk, 26.0% [p < 0.001]), LEUCut for 30-day unplanned reoperations (low risk, 2.2%; intermediate risk, 6.8%; and high risk, 13.5% [p < 0.001]), and LINC for any adverse events (low risk, 7.6%; intermediate risk, 15.7%; and high risk, 49.5% [p < 0.001]), showed satisfactory discrimination between the different outcome groups in receiver operating characteristic curve analysis (AUC ≥ 0.7).

CONCLUSIONS

The proposed risk scores allow efficient prediction of the likelihood of adverse events, to compare quality of care between different providers, and further provide guidance to surgeons on how to allocate preoperative care.

Performance Comparison Between SURPAS and ACS NSQIP Surgical Risk Calculator in Pulmonary Resection

BACKGROUND

Accurate preoperative risk assessment is critical for informed decision making. The Surgical Risk Preoperative Assessment System (SURPAS) and the National Surgical Quality Improvement Program (NSQIP) Surgical Risk Calculator (SRC) predict risks of common postoperative complications. This study compares observed and predicted outcomes after pulmonary resection between SURPAS and NSQIP SRC.

METHODS

Between January 2016 and December 2018, 2514 patients underwent pulmonary resection and were included. We entered the requisite patient demographics, preoperative risk factors, and procedural details into the online NSQIP SRC and SURPAS formulas. Performance of the prediction models was assessed by discrimination and calibration.

RESULTS

No statistically significant differences were found between the 2 models in discrimination performance for 30-day mortality, urinary tract infection, readmission, and discharge to a nursing or rehabilitation facility. The ability to discriminate between a patient who will develop a complication and a patient who will not was statistically indistinguishable between NSQIP and SURPAS, except for renal failure. With a C index closer to 1.0, the NSQIP performed significantly better than the SURPAS SRC in discriminating risk of renal failure (C index, 0.798 vs 0.694; P = .003). The calibration curves of predicted and observed risk for each model demonstrate similar performance with a tendency toward overestimation of risk, apart from renal failure.

CONCLUSIONS

Overall, SURPAS and NSQIP SRC performed similarly in predicting outcomes for pulmonary resections in this large, single-center validation study with moderate to good discrimination of outcomes. Notably, SURPAS uses a smaller set of input variables to generate the preoperative risk assessment. The addition of thoracic-specific input variables may improve performance.

The value of the "Surgical Risk Preoperative Assessment System" (SURPAS) in preoperative consultation for elective surgery: a pilot study

Background

Risk assessment is essential to informed decision making in surgery. Preoperative use of the Surgical Risk Preoperative Assessment System (SURPAS) providing individualized risk assessment, may enhance informed consent. We assessed patient and provider perceptions of SURPAS as a risk assessment tool.

Methods

A convergent mixed-methods study assessed SURPAS’s trial implementation, concurrently collecting quantitative and qualitative data, separately analyzing it, and integrating the results. Patients and providers were surveyed and interviewed on their opinion of how SURPAS impacted the preoperative encounter. Relationships between patient risk and patient and provider assessment of SURPAS were examined.

Results

A total of 197 patients were provided their SURPAS postoperative risk estimates in nine surgeon’s clinics. Of the total patients, 98.8% reported they understood their surgical risks very or quite well after exposure to SURPAS; 92.7% reported SURPAS was very helpful or helpful. Providers shared that 83.4% of the time they reported SURPAS was very or somewhat helpful; 44.7% of the time the providers reported it changed their interaction with the patient and this change was beneficial 94.3% of the time. As patient risk increased, providers reported that SURPAS was increasingly helpful (p < 0.0001).

Conclusions

Patients and providers reported the use of SURPAS helpful and informative during the preoperative risk assessment of patients, thus improving the surgical decision making process. Patients thought that SURPAS was helpful regardless of their risk level, whereas providers thought that SURPAS was more helpful in higher risk patients.