External validation of Cleveland Clinic Foundation colorectal cancer model in a University Clinic in terms of predicting operative mortality

Machine learning-based prediction of postoperative mortality in emergency colorectal surgery: A retrospective, multicenter cohort study using Tokushukai medical database

Background

Although prognostic factors associated with mortality in patients with emergency colorectal surgery have been identified, an accurate mortality risk assessment is still necessary to determine the range of therapeutic resources in accordance with the severity of patients. We established machine-learning models to predict in-hospital mortality for patients who had emergency colorectal surgery using clinical data at admission and attempted to identify prognostic factors associated with in-hospital mortality.

Methods

This retrospective cohort study included adult patients undergoing emergency colorectal surgery in 42 hospitals between 2012 and 2020. We employed logistic regression and three supervised machine-learning models: random forests, gradient-boosting decision trees (GBDT), and multilayer perceptron (MLP). The area under the receiver operating characteristics curve (AUROC) was calculated for each model. The Shapley additive explanations (SHAP) values are also calculated to identify the significant variables in GBDT.

Results

There were 8792 patients who underwent emergency colorectal surgery. As a result, the AUROC values of 0.742, 0.782, 0.814, and 0.768 were obtained for logistic regression, random forests, GBDT, and MLP. According to SHAP values, age, colorectal cancer, use of laparoscopy, and some laboratory variables, including serum lactate dehydrogenase serum albumin, and blood urea nitrogen, were significantly associated with in-hospital mortality.

Conclusion

We successfully generated a machine-learning prediction model, including GBDT, with the best prediction performance and exploited the potential for use in evaluating in-hospital mortality risk for patients who undergo emergency colorectal surgery.

Risk prediction models for postoperative outcomes of colorectal cancer surgery in the older population - a systematic review

BACKGROUND

An increasing number of patients with Colorectal Cancer (CRC) is 65 years or older. We aimed to systematically review existing clinical prediction models for postoperative outcomes of CRC surgery, study their performance in older patients and assess their potential for preoperative decision making.

METHODS

A systematic search in Pubmed and Embase for original studies of clinical prediction models for outcomes of CRC surgery. Bias and relevance for preoperative decision making with older patients were assessed using the CHARMS guidelines.

RESULTS

26 prediction models from 25 publications were included. The average age of included patients ranged from 61 to 76. Two models were exclusively developed for 65 and older. Common outcomes were mortality (n = 10), anastomotic leakage (n = 7) and surgical site infections (n = 3). No prediction models for quality of life or physical functioning were identified. Age, gender and ASA score were common predictors; 12 studies included intraoperative predictors. For the majority of the models, bias for model development and performance was considered moderate to high.

CONCLUSIONS

Prediction models are available that address mortality and surgical complications after CRC surgery. Most models suffer from methodological limitations, and their performance for older patients is uncertain. Models that contain intraoperative predictors are of limited use for preoperative decision making. Future research should address the predictive value of geriatric characteristics to improve the performance of prediction models for older patients.

Model for risk adjustment of postoperative mortality in patients with colorectal cancer

Background

A model was developed for risk adjustment of postoperative mortality in patients with colorectal cancer in order to make fair comparisons between healthcare providers. Previous models were derived in relatively small studies with the use of suboptimal modelling techniques.

Methods

Data from adults included in a national study of major surgery for colorectal cancer were used to develop and validate a logistic regression model for 90-day mortality. The main risk factors were identified from a review of the literature. The association with age was modelled as a curved continuous relationship. Bootstrap resampling was used to select interactions between risk factors.

Results

A model based on data from 62 314 adults was developed that was well calibrated (absolute differences between observed and predicted mortality always smaller than 0·75 per cent in deciles of predicted risk). It discriminated well between low- and high-risk patients (C-index 0·800, 95 per cent c.i. 0·793 to 0·807). An interaction between age and metastatic disease was included as metastatic disease was found to increase postoperative risk in young patients aged 50 years (odds ratio 3·53, 95 per cent c.i. 2·66 to 4·67) far more than in elderly patients aged 80 years (odds ratio 1·48, 1·32 to 1·66).

Conclusion

Use of this model, estimated in the largest number of patients with colorectal cancer to date, is recommended when comparing postoperative mortality of major colorectal cancer surgery between hospitals, clinical teams or individual surgeons.

Postoperative 30-day mortality in patients undergoing surgery for colorectal cancer: development of a prognostic model using administrative claims data

PURPOSE

To develop a prognostic model to predict 30-day mortality following colorectal cancer (CRC) surgery using the Surveillance, Epidemiology, and End Results (SEER)-Medicare-linked data and to assess whether race/ethnicity, neighborhood, and hospital characteristics influence model performance.

METHODS

We included patients aged 66 years and older from the linked 2000-2005 SEER-Medicare database. Outcome included 30-day mortality, both in-hospital and following discharge. Potential prognostic factors included tumor, treatment, sociodemographic, hospital, and neighborhood characteristics (census-tract-poverty rate). We performed a multilevel logistic regression analysis to account for nesting of CRC patients within hospitals. Model performance was assessed using the area under the receiver operating characteristic curve (AUC) for discrimination and the Hosmer-Lemeshow goodness-of-fit test for calibration.

RESULTS

In a model that included all prognostic factors, important predictors of 30-day mortality included age at diagnosis, cancer stage, and mode of presentation. Race/ethnicity, census-tract-poverty rate, and hospital characteristics were independently associated with 30-day mortality, but they did not influence model performance. Our SEER-Medicare model achieved moderate discrimination (AUC = 0.76), despite suboptimal calibration.

CONCLUSIONS

We developed a prognostic model that included tumor, treatment, sociodemographic, hospital, and neighborhood predictors. Race/ethnicity, neighborhood, and hospital characteristics did not improve model performance compared with previously developed models.

Predicting postoperative mortality after colorectal surgery: a novel clinical model

AIM

The aim of this study was to develop and externally validate a clinically, practical and discriminative prediction model designed to estimate in-hospital mortality of patients undergoing colorectal surgery.

METHOD

All consecutive patients who underwent elective or emergency colorectal surgery from 1990 to 2005, at the Zaandam Medical Centre, The Netherlands, were included in this study. Multivariate logistic regression analysis was performed to estimate odds ratios (ORs) and 95% confidence intervals (CIs) linking the explanatory variables to the outcome variable in-hospital mortality, and a simplified Identification of Risk in Colorectal Surgery (IRCS) score was constructed. The model was validated in a population of patients who underwent colorectal surgery from 2005 to 2011 in Barcelona, Spain. Predictive performance was estimated by calculating the area under the receiver operating characteristic curve.

RESULTS

The strongest predictors of in-hospital mortality were emergency surgery (OR = 6.7, 95% CI 4.7-9.5), tumour stage (OR = 3.2, 95% CI 2.8-4.6), age (OR = 13.1, 95% CI 6.6-26.0), pulmonary failure (OR = 4.9, 95% CI 3.3-7.1) and cardiac failure (OR = 3.7, 95% CI 2.6-5.3). These parameters were included in the prediction model and simplified scoring system. The IRCS model predicted in-hospital mortality and demonstrated a predictive performance of 0.83 (95% CI 0.79-0.87) in the validation population. In this population the predictive performance of the CR-POSSUM score was 0.76 (95% CI 0.71-0.81).

CONCLUSIONS

The results of this study have shown that the IRCS score is a good predictor of in-hospital mortality after colorectal surgery despite the relatively low number of model parameters.