Accuracy of the ACS NSQIP Surgical Risk Calculator for Predicting Postoperative Complications in Gastric Cancer Following Open Gastrectomy

INTRODUCTION

The prediction of complications before gastric surgery is of utmost importance in shared decision making and proper counseling of the patient in order to minimize postoperative complications. Our aim was to evaluate the predictive validity of American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) risk calculator in gastric cancer patients who underwent gastrectomy.

METHODS

Preoperative assessment data of 432 patients were retrospectively reviewed and manually entered into the calculator. The accuracy of the calculator was evaluated using Pearson's chi-squared test, C-statistic, Brier score, and Hosmer-Lemeshow test.

RESULTS

The lowest Brier scores were observed in urinary tract infection, renal failure, venous thromboembolism, pneumonia, and cardiac complications. Best results were obtained for predicting sepsis, discharge to rehabilitation facility, and death (low Brier scores, C-statistic >.7, and Hosmer-Lemeshow P > .05).

CONCLUSION

The calculator had a strong performance in predicting sepsis, discharge to the rehabilitation facility, and death. However, it performed poor in predicting the most commonly observed events (any or serious complication and surgical site infection).

Machine learning in gastrointestinal surgery

Machine learning (ML) is a collection of algorithms allowing computers to learn directly from data without predetermined equations. It is used widely to analyze "big data". In gastrointestinal surgery, surgeons deal with various data such as clinical parameters, surgical videos, and pathological images, to stratify surgical risk, perform safe surgery and predict patient prognosis. In the current "big data" era, the accelerating accumulation of a large amount of data drives studies using ML algorithms. Three subfields of ML are supervised learning, unsupervised learning, and reinforcement learning. In this review, we summarize applications of ML to surgical practice in the preoperative, intraoperative, and postoperative phases of care. Prediction and stratification using ML is promising; however, the current overarching concern is the availability of ML models. Information systems that can manage "big data" and integrate ML models into electronic health records are essential to incorporate ML into daily practice. ML is fundamental technology to meaningfully process data that exceeds the capacity of the human mind to comprehend. The accelerating accumulation of a large amount of data is changing the nature of surgical practice fundamentally. Artificial intelligence (AI), represented by ML, is being incorporated into daily surgical practice.

The Need for Accurate Risk Assessment in a High-Risk Patient Population: A NSQIP Study Evaluating Outcomes of Cholecystectomy in the Patient With Cancer

BACKGROUND

Cholecystectomy is considered a low-risk procedure with proven safety in many high-risk patient populations. However, the risk of cholecystectomy in patients with active cancer has not been established.

METHODS

The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) database was queried to identify all patients with disseminated cancer who underwent cholecystectomy from 2005 to 2016. Postcholecystectomy outcomes were defined for patients with cancer and those without by comparing several outcomes measures. A multivariate model was used to estimate the odds of 30-d mortality.

RESULTS

We compared outcomes in 3097 patients with disseminated cancer to a matched cohort of patients without cancer. Patients with cancer had more comorbidities at baseline: dyspnea (10.5% versus 7.0%, P < 0.0001), steroid use (10.1% versus 3.0%, P < 0.0001), and loss of >10% body weight in 6-mo prior (9.3% versus 1.6%, P < 0.0001). Patients with cancer sustained higher rates of wound (2.3% versus 5.6%, P < 0.0001), respiratory (1.4% versus 3.9%, P < 0.0001), and cardiovascular (2.0% versus 6.8%, P < 0.0001) complications. In addition, patients with disseminated cancer experienced a longer length of stay and higher 30-d mortality. Multivariate modeling showed that the odds of 30-d mortality was 3.3 times greater in patients with cancer.

CONCLUSIONS

Compared to patients without cancer, those with disseminated cancer are at higher risk of complication and mortality following cholecystectomy. Traditional treatment algorithms should be used with caution and care decisions individualized based on the patient's disease status and treatment goals.

Validation of the ACS NSQIP Surgical Risk Calculator for Patients with Early Gastric Cancer Treated with Laparoscopic Gastrectomy

Purpose

The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) risk calculator is useful in predicting postoperative adverse events. However, its accuracy in specific disorders is unclear. We validated the ACS NSQIP risk calculator in patients with gastric cancer undergoing curative laparoscopic surgery.

Materials and Methods

We included 207 consecutive early gastric cancer patients who underwent laparoscopic gastrectomy between January 2018 and January 2019. The preoperative characteristics and risks of the patients were reviewed and entered into the ACS NSQIP calculator. The estimated risks of postoperative outcomes were compared with the observed outcomes using C-statistics and Brier scores.

Results

Most of the patients underwent distal gastrectomy with Roux-en-Y reconstruction (74.4%). We did not observe any cases of mortality, venous thromboembolism, urinary tract infection, renal failure, or cardiac complications. The other outcomes assessed were complications such as pneumonia, surgical site infections, any complications requiring re-operation or hospital readmission, the rates of discharge to nursing homes/rehabilitation centers, and the length of stay. All C-statistics were <0 and the highest was for pneumonia (0.65; 95% confidence interval: 0.58–0.71). Brier scores ranged from 0.01 for pneumonia to 0.155 for other complications. Overall, the risk calculator was inconsistent in predicting the outcomes.

Conclusions

The ACS NSQIP surgical risk calculator showed low predictive ability for postoperative adverse events after laparoscopic gastrectomy for patients with early gastric cancer. Further research to adjust the risk calculator for these patients may improve its predictive ability.

Retroperitoneal sarcoma perioperative risk stratification: A United States Sarcoma Collaborative evaluation of the ACS-NSQIP risk calculator

BACKGROUND

The ACS-NSQIP risk calculator predicts perioperative risk. This study tested the calculator's ability to predict risk for outcomes following retroperitoneal sarcoma (RPS) resection.

METHODS

The United States Sarcoma Collaborative database was queried for adults who underwent RPS resection. Estimated risk for outcomes was calculated twice in the risk calculator, once using sarcoma-specific CPT codes and once using codes indicative of most comorbid organ resection (eg nephrectomy). ROC curves were generated, with area under the curve (AUC) and Brier scores reported to assess discrimination and calibration. An AUC < 0.6 was considered ineffective discrimination. A negative ▲ Brier indicated improved performance relative to baseline outcome rates.

RESULTS

In total, 482 patients were identified with a 42.3% 90-day complication rate. Discrimination was poor for all outcomes except "all complications" and "renal failure." Baseline outcome rates were better predictors than calculator estimates except for "discharge to nursing or rehab facility" and "renal failure." Replacing sarcoma-specific CPT codes with resection-specific codes did not improve performance.

CONCLUSION

The ACS-NSQIP risk calculator poorly predicted outcomes following RPS resection. Changing sarcoma-specific CPT to resection-specific codes did not improve performance. Comorbidities in the calculator may not effectively capture perioperative risk. Future work should evaluate a sarcoma-specific calculator.

Use of Machine Learning for Prediction of Patient Risk of Postoperative Complications After Liver, Pancreatic, and Colorectal Surgery

BACKGROUND

Surgical resection is the only potentially curative treatment for patients with colorectal, liver, and pancreatic cancers. Although these procedures are performed with low mortality, rates of complications remain relatively high following hepatopancreatic and colorectal surgery.

METHODS

The American College of Surgeons (ACS) National Surgical Quality Improvement Program was utilized to identify patients undergoing liver, pancreatic and colorectal surgery from 2014 to 2016. Decision tree models were utilized to predict the occurrence of any complication, as well as specific complications. To assess the variability of the performance of the classification trees, bootstrapping was performed on 50% of the sample.

RESULTS

Algorithms were derived from a total of 15,657 patients who met inclusion criteria. The algorithm had a good predictive ability for the occurrence of any complication, with a C-statistic of 0.74, outperforming the ASA (C-statistic 0.58) and ACS-Surgical Risk Calculator (C-statistic 0.71). The algorithm was able to predict with high accuracy thirteen out of the seventeen complications analyzed. The best performance was in the prediction of stroke (C-statistic 0.98), followed by wound dehiscence, cardiac arrest, and progressive renal failure (all C-statistic 0.96). The algorithm had a good predictive ability for superficial SSI (C-statistic 0.76), organ space SSI (C-statistic 0.76), sepsis (C-statistic 0.79), and bleeding requiring transfusion (C-statistic 0.79).

CONCLUSION

Machine learning was used to develop an algorithm that accurately predicted patient risk of developing complications following liver, pancreatic, or colorectal surgery. The algorithm had very good predictive ability to predict specific complications and demonstrated superiority over other established methods.

Impact of preoperative risk factors on outcome after gastrectomy

Background

Gastrectomy is associated with relevant postoperative morbidity. However, outcome of surgery can be improved by careful selection of patients. The objective of the current study was therefore to identify preoperative risk factors that might impact on patients’ further outcome after surgical resection.

Methods

Preoperative risk factors having respectively different surgical risk scores for major complex surgery (including Cologne Risk Score, p-/o-POSSUM, and NSQIP risk score) of patients that underwent gastrectomy for AEG II/III tumors and gastric cancer were correlated with complications according to Clavien-Dindo and outcome. Patients who underwent surgery in palliative intention were excluded from further analysis.

Results

Subtotal gastrectomy was performed in 23%, gastrectomy in 59%, and extended gastrectomy in 18% in a total of 139 patients (mean age: 64 years old). Thirty six percent experienced a minor complication (Dindo I-II) and 24% a major complication (Dindo III-V), which resulted in a prolonged hospital stay (p < 0.001). In-hospital mortality (=Dindo V) was 2.5%. Besides age, type of surgical procedure impacted on complications with extended gastrectomy showing the highest risk (p = 0.005). The o-POSSUM score failed to predict mortality accurately. We observed a highly positive correlation between predicted morbidity respectively mortality and occurrence of complications estimated by p-POSSUM (p = 0.005), Cologne Risk (p = 0.007), and NSQIP scores (p < 0.001).

Conclusion

The results demonstrate a significant association between different risk scores and occurrence of complications following gastrectomy. The p-POSSUM, Cologne Risk, and NSQIP score exhibited superior performance than the o-POSSUM score. Therefore, these scores might allow identification and selection of high-risk patients and thus might be highly useful for clinical decision making.

Evaluating the ACS NSQIP Risk Calculator in Primary Pancreatic Neuroendocrine Tumor: Results from the US Neuroendocrine Tumor Study Group

BACKGROUND

In a changing health care environment where patient outcomes will be more closely scrutinized, the ability to predict surgical complications is becoming increasingly important. The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) online risk calculator is a popular tool to predict surgical risk. This paper aims to assess the applicability of the ACS NSQIP calculator to patients undergoing surgery for pancreatic neuroendocrine tumors (PNETs).

METHODS

Using the US Neuroendocrine Tumor Study Group (USNET-SG), 890 patients who underwent pancreatic procedures between 1/1/2000-12/31/2016 were evaluated. Predicted and actual outcomes were compared using C-statistics and Brier scores.

RESULTS

The most commonly performed procedure was distal pancreatectomy, followed by standard and pylorus-preserving pancreaticoduodenectomy. For the entire group of patients studied, C-statistics were highest for discharge destination (0.79) and cardiac complications (0.71), and less than 0.7 for all other complications. The Brier scores for surgical site infection (0.1441) and discharge to nursing/rehabilitation facility (0.0279) were below the Brier score cut-off, while the rest were equal to or above and therefore not useful for interpretation.

CONCLUSION

This work indicates that the ACS NSQIP risk calculator is a valuable tool that should be used with caution and in coordination with clinical assessment for PNET clinical decision-making.