Evaluation of prognostic risk models for postoperative pulmonary complications in adult patients undergoing major abdominal surgery: a systematic review and international external validation cohort study

External validation of the CARDOT score for predicting respiratory complications after thoracic surgery

BACKGROUND

The CARDOT scores have been developed for prediction of respiratory complications after thoracic surgery. This study aimed to externally validate the CARDOT score and assess the predictive value of preoperative neutrophil-to-lymphocyte ratio (NLR) for postoperative respiratory complication.

METHODS

A retrospective cohort study of consecutive thoracic surgical patients at a single tertiary hospital in northern Thailand was conducted. The development and validation datasets were collected between 2006 and 2012 and from 2015 to 2021, respectively. Six prespecified predictive factors were identified, and formed a predictive score, the CARDOT score (chronic obstructive pulmonary disease, American Society of Anesthesiologists physical status, right-sided operation, duration of surgery, preoperative oxygen saturation on room air, thoracotomy), was calculated. The performance of the CARDOT score was evaluated in terms of discrimination by using the area under the receiver operating characteristic (AuROC) curve and calibration.

RESULTS

There were 1086 and 1645 patients included in the development and validation datasets. The incidence of respiratory complications was 15.7% (171 of 1086) and 22.5% (370 of 1645) in the development and validation datasets, respectively. The CARDOT score had good discriminative ability for both the development and validation datasets (AuROC 0.789 (95% CI 0.753-0.827) and 0.758 (95% CI 0.730-0.787), respectively). The CARDOT score showed good calibration in both datasets. A high NLR (≥ 4.5) significantly increased the risk of respiratory complications after thoracic surgery (P < 0.001). The AuROC curve of the validation cohort increased to 0.775 (95% CI 0.750-0.800) when the score was combined with a high NLR. The AuROC of the CARDOT score with the NLR showed significantly greater discrimination power than that of the CARDOT score alone (P = 0.008).

CONCLUSIONS

The CARDOT score showed a good discriminative performance in the external validation dataset. An addition of a high NLR significantly increases the predictive performance of CARDOT score. The utility of this score is valuable in settings with limited access to preoperative pulmonary function testing.

Machine learning and preoperative risk prediction: the machines are coming

Preoperative risk prediction is an important component of perioperative medicine. Machine learning is a powerful tool that could lead to increasingly complex risk prediction models with improved predictive performance. Careful consideration is required to guide the machine learning approach to ensure appropriate decisions are made with regard to what we are trying to predict, when we are trying to predict it, and what we seek to do with the results.

Utilising intraoperative respiratory dynamic features for developing and validating an explainable machine learning model for postoperative pulmonary complications

BACKGROUND

Timely detection of modifiable risk factors for postoperative pulmonary complications (PPCs) could inform ventilation strategies that attenuate lung injury. We sought to develop, validate, and internally test machine learning models that use intraoperative respiratory features to predict PPCs.

METHODS

We analysed perioperative data from a cohort comprising patients aged 65 yr and older at an academic medical centre from 2019 to 2023. Two linear and four nonlinear learning models were developed and compared with the current gold-standard risk assessment tool ARISCAT (Assess Respiratory Risk in Surgical Patients in Catalonia Tool). The Shapley additive explanation of artificial intelligence was utilised to interpret feature importance and interactions.

RESULTS

Perioperative data were obtained from 10 284 patients who underwent 10 484 operations (mean age [range] 71 [65-98] yr; 42% female). An optimised XGBoost model that used preoperative variables and intraoperative respiratory variables had area under the receiver operating characteristic curves (AUROCs) of 0.878 (0.866-0.891) and 0.881 (0.879-0.883) in the validation and prospective cohorts, respectively. These models outperformed ARISCAT (AUROC: 0.496-0.533). The intraoperative dynamic features of respiratory dynamic system compliance, mechanical power, and driving pressure were identified as key modifiable contributors to PPCs. A simplified model based on XGBoost including 20 variables generated an AUROC of 0.864 (0.852-0.875) in an internal testing cohort. This has been developed into a web-based tool for further external validation (https://aorm.wchscu.cn/).

CONCLUSIONS

These findings suggest that real-time identification of surgical patients' risk of postoperative pulmonary complications could help personalise intraoperative ventilatory strategies and reduce postoperative pulmonary complications.

Development and validation of a prediction model for postoperative pneumonia in patients who received spinal surgery: A retrospective study

Objectives

To develop and validate a risk prediction model by identifying the preoperative factors associated with an increased risk of pneumonia after spinal surgery.

Methods

This study included patients with spinal disease from two hospitals between January 2021 and June 2023. The patients were divided into the training and validation sets, which were categorized as postoperative pneumonia (POP) or non-POP, respectively. This study identified the independent risk variables for POP using a multivariate logistic regression analysis. A nomogram prediction model was developed and validated using risk factors, receiver operating characteristic (ROC) curves, calibration curves, and decision curve analysis (DCA) to assess predictive performance.

Results

Following exclusion, 2223 patients from Changzheng Hospital were enrolled in the training set and 357 patients from the No. 905 Hospital of PLA Navy were enrolled in the validation set. Univariate and multivariate logistic regression analyses revealed that operation time, American Society of Anesthesiologists (ASA) grade, smoking, non-wearing of medical masks, lack of preoperative respiratory training, chronic obstructive pulmonary disease (COPD), underlying diseases, and spinal section were risk factors for POP development in patients with spinal diseases. The area under the ROC curve of the training set was 0.950, whereas that of the validation set was 0.879. The model calibration curves demonstrated good agreement, and the DCA indicated a high expected net benefit value.

Conclusion

The POP risk prediction model has high accuracy and efficiency in predicting POP in patients with spinal diseases. POP development is influenced by factors such as operation length, ASA grade, smoking, non-wearing of medical masks, lack of preoperative respiratory training, COPD, underlying diseases, and lumbar surgery.

Risk factors for postoperative pulmonary complications in elderly patients undergoing video-assisted thoracoscopic surgery lobectomy under general anesthesia: a retrospective study

BACKGROUND

The objective of this study is to identify and evaluate the risk factors associated with the development of postoperative pulmonary complications (PPCs) in elderly patients undergoing video-assisted thoracoscopic surgery lobectomy under general anesthesia.

METHODS

The retrospective study consecutively included elderly patients (≥ 70 years old) who underwent thoracoscopic lobectomy at Xuanwu Hospital of Capital Medical University from January 1, 2018 to August 31, 2023. The demographic characteristics, the preoperative, intraoperative and postoperative parameters were collected and analyzed using multivariate logistic regression to identify the prediction of risk factors for PPCs.

RESULTS

322 patients were included for analysis, and 115 patients (35.7%) developed PPCs. Multifactorial regression analysis showed that ASA ≥ III (P = 0.006, 95% CI: 1.230 ∼ 3.532), duration of one-lung ventilation (P = 0.033, 95% CI: 1.069 ∼ 4.867), smoking (P = 0.027, 95% CI: 1.072 ∼ 3.194) and COPD (P = 0.015, 95% CI: 1.332 ∼ 13.716) are independent risk factors for PPCs after thoracoscopic lobectomy in elderly patients.

CONCLUSION

Risk factors for PPCs are ASA ≥ III, duration of one-lung ventilation, smoking and COPD in elderly patients over 70 years old undergoing thoracoscopic lobectomy. It is necessary to pay special attention to these patients to help optimize the allocation of resources and enhance preventive efforts.

A Transcriptomic Approach to Understand Patient Susceptibility to Pneumonia After Abdominal Surgery

OBJECTIVE

To describe immune pathways and gene networks altered following major abdominal surgery and to identify transcriptomic patterns associated with postoperative pneumonia.

BACKGROUND

Nosocomial infections are a major healthcare challenge, developing in over 20% of patients aged 45 or over undergoing major abdominal surgery, with postoperative pneumonia associated with an almost 5-fold increase in 30-day mortality.

METHODS

From a prospective consecutive cohort (n=150) undergoing major abdominal surgery, whole-blood RNA was collected preoperatively and at 3 time-points postoperatively (2-6, 24, and 48 h). Twelve patients diagnosed with postoperative pneumonia and 27 matched patients remaining infection-free were identified for analysis with RNA-sequencing.

RESULTS

Compared to preoperative sampling, 3639 genes were upregulated and 5043 downregulated at 2 to 6 hours. Pathway analysis demonstrated innate-immune activation with neutrophil degranulation and Toll-like-receptor signaling upregulation alongside adaptive-immune suppression. Cell-type deconvolution of preoperative RNA-sequencing revealed elevated S100A8/9-high neutrophils alongside reduced naïve CD4 T-cells in those later developing pneumonia. Preoperatively, a gene-signature characteristic of neutrophil degranulation was associated with postoperative pneumonia acquisition ( P =0.00092). A previously reported Sepsis Response Signature (SRSq) score, reflecting neutrophil dysfunction and a more dysregulated host response, at 48 hours postoperatively, differed between patients subsequently developing pneumonia and those remaining infection-free ( P =0.045). Analysis of the novel neutrophil gene-signature and SRSq scores in independent major abdominal surgery and polytrauma cohorts indicated good predictive performance in identifying patients suffering later infection.

CONCLUSIONS

Major abdominal surgery acutely upregulates innate-immune pathways while simultaneously suppressing adaptive-immune pathways. This is more prominent in patients developing postoperative pneumonia. Preoperative transcriptomic signatures characteristic of neutrophil degranulation and postoperative SRSq scores may be useful predictors of subsequent pneumonia risk.

Nomogram for predicting postoperative pulmonary complications in spinal tumor patients

OBJECTIVES

Although several independent risk factors for postoperative pulmonary complications (PPCs) after spinal tumor surgery have been studied, a simple and valid predictive model for PPC occurrence after spinal tumor surgery has not been developed.

PATIENTS AND METHODS

We collected data from patients who underwent elective spine surgery for a spinal tumor between 2013 and 2020 at a tertiary hospital in China. Data on patient characteristics, comorbidities, preoperative examinations, intraoperative variables, and clinical outcomes were collected. We used univariable and multivariable logistic regression models to assess predictors of PPCs and developed and validated a nomogram for PPCs. We evaluated the performance of the nomogram using the area under the receiver operating characteristic curve (ROC), calibration curves, the Brier Score, and the Hosmer-Lemeshow (H-L) goodness-of-fit test. For clinical use, decision curve analysis (DCA) was conducted to identify the model's performance as a tool for supporting decision-making.

RESULTS

Among the participants, 61 (12.4%) individuals developed PPCs. Clinically significant variables associated with PPCs after spinal tumor surgery included BMI, tumor location, blood transfusion, and the amount of blood lost. The nomogram incorporating these factors showed a concordance index (C-index) of 0.755 (95% CI: 0.688-0.822). On internal validation, bootstrapping with 1000 resamples yielded a bias-corrected area under the receiver operating characteristic curve of 0.733, indicating the satisfactory performance of the nomogram in predicting PPCs. The calibration curve demonstrated accurate predictions of observed values. The decision curve analysis (DCA) indicated a positive net benefit for the nomogram across most predicted threshold probabilities.

CONCLUSIONS

We have developed a new nomogram for predicting PPCs in patients who undergo spinal tumor surgery.

Development and validation of an interpretable Markov-embedded multilabel model for predicting risks of multiple postoperative complications among surgical inpatients: a multicenter prospective cohort study

BACKGROUND

When they encounter various highly related postoperative complications, existing risk evaluation tools that focus on single or any complications are inadequate in clinical practice. This seriously hinders complication management because of the lack of a quantitative basis. An interpretable multilabel model framework that predicts multiple complications simultaneously is urgently needed.

MATERIALS AND METHODS

The authors included 50 325 inpatients from a large multicenter cohort (2014-2017). The authors separated patients from one hospital for external validation and randomly split the remaining patients into training and internal validation sets. A MARKov-EmbeDded (MARKED) multilabel model was proposed, and three models were trained for comparison: binary relevance, a fully connected network (FULLNET), and a deep neural network. Performance was mainly evaluated using the area under the receiver operating characteristic curve (AUC). The authors interpreted the model using Shapley Additive Explanations. Complication-specific risk and risk source inference were provided at the individual level.

RESULTS

There were 26 292, 6574, and 17 459 inpatients in the training, internal validation, and external validation sets, respectively. For the external validation set, MARKED achieved the highest average AUC (0.818, 95% CI: 0.771-0.864) across eight outcomes [compared with binary relevance, 0.799 (0.748-0.849), FULLNET, 0.806 (0.756-0.856), and deep neural network, 0.815 (0.765-0.866)]. Specifically, the AUCs of MARKED were above 0.9 for cardiac complications [0.927 (0.894-0.960)], neurological complications [0.905 (0.870-0.941)], and mortality [0.902 (0.867-0.937)]. Serum albumin, surgical specialties, emergency case, American Society of Anesthesiologists score, age, and sex were the six most important preoperative variables. The interaction between complications contributed more than the preoperative variables, and formed a hierarchical chain of risk factors, mild complications, and severe complications.

CONCLUSION

The authors demonstrated the advantage of MARKED in terms of performance and interpretability. The authors expect that the identification of high-risk patients and the inference of the risk source for specific complications will be valuable for clinical decision-making.

Development of an automated, general-purpose prediction tool for postoperative respiratory failure using machine learning: A retrospective cohort study

STUDY OBJECTIVE

Postoperative respiratory failure is a major surgical complication and key quality metric. Existing prediction tools underperform, are limited to specific populations, and necessitate manual calculation. This limits their implementation. We aimed to create an improved, machine learning powered prediction tool with ideal characteristics for automated calculation.

DESIGN, SETTING, AND PATIENTS

We retrospectively reviewed 101,455 anesthetic procedures from 1/2018 to 6/2021. The primary outcome was the Standardized Endpoints in Perioperative Medicine consensus definition for postoperative respiratory failure. Secondary outcomes were respiratory quality metrics from the National Surgery Quality Improvement Sample, Society of Thoracic Surgeons, and CMS. We abstracted from the electronic health record 26 procedural and physiologic variables previously identified as respiratory failure risk factors. We randomly split the cohort and used the Random Forest method to predict the composite outcome in the training cohort. We coined this the RESPIRE model and measured its accuracy in the validation cohort using area under the receiver operating curve (AUROC) analysis, among other measures, and compared this with ARISCAT and SPORC-1, two leading prediction tools. We compared performance in a validation cohort using score cut-offs determined in a separate test cohort.

MAIN RESULTS

The RESPIRE model exhibited superior accuracy with an AUROC of 0.93 (95% CI, 0.92-0.95) compared to 0.82 for both ARISCAT and SPORC-1 (P-for-difference < 0.0001 for both). At comparable 80-90% sensitivities, RESPIRE had higher positive predictive value (11%, 95% CI: 10-12%) and lower false positive rate (12%, 95% CI: 12-13%) compared to 4% and 37% for both ARISCAT and SPORC-1. The RESPIRE model also better predicted the established quality metrics for postoperative respiratory failure.

CONCLUSIONS

We developed a general-purpose, machine learning powered prediction tool with superior performance for research and quality-based definitions of postoperative respiratory failure.

Performance Comparison of Pulmonary Risk Scoring Systems in Lung Resection

OBJECTIVE

To validate and compare the performance of different pulmonary risk scoring systems to predict postoperative pulmonary complications (PPCs) in lung resection surgery.

DESIGN

Retrospective cohort study SETTING: A historic single-center cohort of lung resection surgeries PARTICIPANTS: Adult patients undergoing lung resection surgery under 1-lung ventilation.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The accuracy of the following pulmonary risk scoring systems were used to predict pulmonary complications: the ARISCAT (Assess respiratory RIsk in Surgical patients in CATalonia), the LAS VEGAS (Local Assessment of VEntilatory management during General Anesthesia for Surgery), the SPORC (Score for Prediction of Postoperative Respiratory Complications), and a recent thoracic-specific risk score, named CARDOT. Discrimination and calibration were assessed using the concordance (c) index and the intercept of LOESS (locally estimated scatterplot)-smoothed curves, respectively. Additional models were constructed that incorporated predicted postoperative forced expiratory volume (ppoFEV1) into each scoring system. Of the 2,104 patients undergoing lung surgery, 123 developed postoperative pulmonary complications (PPCs; 5.9%). All scoring systems had poor discriminatory power to predict PPCs (ARISCAT c-index 0.60, 95% confidence interval [CI] 0.55-0.65; LAS VEGAS c-index 0.68, 95% CI 0.63-0.73; SPORC c-index 0.63, 95% CI 0.59-0.68; CARDOT c-index 0.64, 95% CI 0.58-0.70), but the inclusion of ppoFEV1 slightly improved the performance of LAS VEGAS (c-index 0.70, 95% CI 0.66-0.75) and CARDOT (c-index 0.68, 95% CI 0.62-0.73). Analysis of calibration showed a slight overestimation when using ARISCAT (intercept -0.28) and LAS VEGAS (intercept -0.27).

CONCLUSIONS

None of the scoring systems appeared to have adequate discriminatory power to predict PPCs among patients undergoing lung resection. An alternative risk score is necessary to better predict patients at risk of PPCs after thoracic surgery.

Study management strategies to optimize student- and trainee-led collaborative research

The collaborative research model is a powerful approach to answering surgical research questions that empowers and inspires medical students and trainees. The Opioid PrEscRiptions and usage After Surgery (OPERAS) study is a student- and trainee-led international multi-centre prospective cohort study developed in Australia and Aotearoa New Zealand. In this article, we will discuss (i) how the OPERAS study was conceptualized and structured; (ii) the channels through which information and education were communicated to collaborators; (iii) how data was stored in a secure and user-friendly fashion and (iv) the lessons learned and expected goals for the future. We aim to describe how collaborative research studies can be designed to support early career researchers to make valuable contributions to the literature.