Comparison of logistic regression and machine learning methods for predicting postoperative delirium in elderly patients: A retrospective study

Machine learning with clinical and intraoperative biosignal data for predicting postoperative delirium after cardiac surgery

Early identification of patients at high risk of delirium is crucial for its prevention. Our study aimed to develop machine learning models to predict delirium after cardiac surgery using intraoperative biosignals and clinical data. We introduced a novel approach to extract relevant features from continuously measured intraoperative biosignals. These features reflect the patient's overall or baseline status, the extent of unfavorable conditions encountered intraoperatively, and beat-to-beat variability within the data. We developed a soft voting ensemble machine learning model using retrospective data from 1,912 patients. The model was then prospectively validated with data from 202 additional patients, achieving a high performance with an area under the receiver operating characteristic curve of 0.887 and an accuracy of 0.881. According to the SHapley Additive exPlanation method, several intraoperative biosignal features had high feature importance, suggesting that intraoperative patient management plays a crucial role in preventing delirium after cardiac surgery.

Interpretable machine learning model for early prediction of delirium in elderly patients following intensive care unit admission: a derivation and validation study

Background and objective

Delirium is the most common neuropsychological complication among older adults admitted to the intensive care unit (ICU) and is often associated with a poor prognosis. This study aimed to construct and validate an interpretable machine learning (ML) for early delirium prediction in older ICU patients.

Methods

This was a retrospective observational cohort study and patient data were extracted from the Medical Information Mart for Intensive Care-IV database. Feature variables associated with delirium, including predisposing factors, disease-related factors, and iatrogenic and environmental factors, were selected using least absolute shrinkage and selection operator regression, and prediction models were built using logistic regression, decision trees, support vector machines, extreme gradient boosting (XGBoost), k-nearest neighbors and naive Bayes methods. Multiple metrics were used for evaluation of performance of the models, including the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, recall, F1 score, calibration plot, and decision curve analysis. SHapley Additive exPlanations (SHAP) were used to improve the interpretability of the final model.

Results

Nine thousand seven hundred forty-eight adults aged 65 years or older were included for analysis. Twenty-six features were selected to construct ML prediction models. Among the models compared, the XGBoost model demonstrated the best performance including the highest AUC (0.836), accuracy (0.765), sensitivity (0.713), recall (0.713), and F1 score (0.725) in the training set. It also exhibited excellent discrimination with AUC of 0.810, good calibration, and had the highest net benefit in the validation cohort. The SHAP summary analysis showed that Glasgow Coma Scale, mechanical ventilation, and sedation were the top three risk features for outcome prediction. The SHAP dependency plot and SHAP force analysis interpreted the model at both the factor level and individual level, respectively.

Conclusion

ML is a reliable tool for predicting the risk of critical delirium in elderly patients. By combining XGBoost and SHAP, it can provide clear explanations for personalized risk prediction and more intuitive understanding of the effect of key features in the model. The establishment of such a model would facilitate the early risk assessment and prompt intervention for delirium.

Development and validation of a nomogram to predict postoperative delirium in older patients after major abdominal surgery: a retrospective case-control study

BACKGROUND

Postoperative delirium is a common complication in older patients, with poor long-term outcomes. This study aimed to investigate risk factors and develop a predictive model for postoperative delirium in older patients after major abdominal surgery.

METHODS

This study retrospectively recruited 7577 patients aged ≥ 65 years who underwent major abdominal surgery between January 2014 and December 2018 in a single hospital in Beijing, China. Patients were divided into a training cohort (n = 5303) and a validation cohort (n = 2224) for univariate and multivariate logistic regression analyses and to build a nomogram. Data were collected for 43 perioperative variables, including demographics, medical history, preoperative laboratory results, imaging, and anesthesia information.

RESULTS

Age, chronic obstructive pulmonary disease, white blood cell count, glucose, total protein, creatinine, emergency surgery, and anesthesia time were associated with postoperative delirium in multivariate analysis. We developed a nomogram based on the above 8 variables. The nomogram achieved areas under the curve of 0.731 and 0.735 for the training and validation cohorts, respectively. The discriminatory ability of the nomogram was further assessed by dividing the cases into three risk groups (low-risk, nomogram score < 175; medium-risk, nomogram score 175~199; high-risk, nomogram score > 199; P < 0.001). Decision curve analysis revealed that the nomogram provided a good net clinical benefit.

CONCLUSIONS

We developed a nomogram that could predict postoperative delirium with high accuracy and stability in older patients after major abdominal surgery.

Application of Machine Learning in Predicting Perioperative Outcomes in Patients with Cancer: A Narrative Review for Clinicians

This narrative review explores the utilization of machine learning (ML) and artificial intelligence (AI) models to enhance perioperative cancer care. ML and AI models offer significant potential to improve perioperative cancer care by predicting outcomes and supporting clinical decision-making. Tailored for perioperative professionals including anesthesiologists, surgeons, critical care physicians, nurse anesthetists, and perioperative nurses, this review provides a comprehensive framework for the integration of ML and AI models to enhance patient care delivery throughout the perioperative continuum.

Development and validation of delirium prediction models for noncardiac surgery patients

STUDY OBJECTIVE

Postoperative delirium is associated with morbidity and mortality, and its incidence varies widely. Using known predisposing and precipitating factors, we sought to develop postoperative delirium prediction models for noncardiac surgical patients.

DESIGN

Retrospective prediction model study.

SETTING

Major quaternary medical center.

PATIENTS

Our January 2016 to June 2020 training dataset included 51,677 patients of whom 2795 patients had delirium. Our July 2020 to January 2022 validation dataset included 14,438 patients of whom 912 patients had delirium.

INTERVENTIONS

None.

MEASUREMENTS

We trained and validated two static prediction models and one dynamic delirium prediction model. For the static models, we used random survival forests and traditional Cox proportional hazard models to predict postoperative delirium from preoperative variables, or from a combination of preoperative and intraoperative variables. We also used landmark modeling to dynamically predict postoperative delirium using preoperative, intraoperative, and postoperative variables before onset of delirium.

MAIN RESULTS

In the validation analyses, the static random forest model had a c-statistic of 0.81 (95% CI: 0.79, 0.82) and a Brier score of 0.04 with preoperative variables only, and a c-statistic of 0.86 (95% CI: 0.84, 0.87) and a Brier score of 0.04 when preoperative and intraoperative variables were combined. The corresponding Cox models had similar discrimination metrics with slightly better calibration. The dynamic model - using all available data, i.e., preoperative, intraoperative and postoperative data - had an overall c-index of 0.84 (95% CI: 0.83, 0.85).

CONCLUSIONS

Using preoperative and intraoperative variables, simple static models performed as well as a dynamic delirium prediction model that also included postoperative variables. Baseline predisposing factors thus appear to contribute far more to delirium after noncardiac surgery than intraoperative or postoperative variables. Improved postoperative data capture may help improve delirium prediction and should be evaluated in future studies.

The association between the fibrinogen-to-albumin ratio and delirium after deep brain stimulation surgery in Parkinson's disease

Introduction

Postoperative delirium (POD) remains one of the most prevalent neuropsychiatric complications after deep brain stimulation (DBS) surgery. The fibrinogen-to-albumin ratio (FAR) has been shown to significantly correlate with the prognosis of many diseases related to inflammation. However, the association between FAR and POD remains unclear. We aimed to explore the association between POD and FAR in patients with Parkinson's disease (PD) undergoing DBS surgery.

Methods

Patients with PD who underwent DBS surgery in our hospital were included in this retrospective study. FAR was calculated from the blood sample collected on admission. The association between baseline FAR and delirium after surgery was assessed by binary logistic regression analysis, interaction analysis, and stratified analyses.

Results

Of 226 patients, 37 (16.4%) suffered from delirium after surgery. The average age of the participants was 63.3 ± 7.2 years, and 51.3% were male patients. Multivariate logistic regression analysis indicated that patients in the highest FAR tertile had a higher risk of POD compared with patients in the lowest FAR tertile (OR = 3.93, 95% CI: 1.24 ~ 12.67). Subgroup analysis demonstrated that FAR and the preoperative Mini-Mental State Examination score (p = 0.013) had an association with delirium after surgery.

Conclusion

Our data suggest that a higher preoperative FAR was significantly associated with delirium after DBS surgery. FAR on admission is a useful candidate biomarker to identify patients with PD who are at a high risk of delirium following DBS surgery.

The construction of a nomogram to predict the prognosis and recurrence risks of UPJO

Objective

This study was conducted to explore the risk factors for the prognosis and recurrence of ureteropelvic junction obstruction (UPJO).

Methods

The correlation of these variables with the prognosis and recurrence risks was analyzed by binary and multivariate logistic regression. Besides, a nomogram was constructed based on the multivariate logistic regression calculation. After the model was verified by the C-statistic, the ROC curve was plotted to evaluate the sensitivity of the model. Finally, the decision curve analysis (DCA) was conducted to estimate the clinical benefits and losses of intervention measures under a series of risk thresholds.

Results

Preoperative automated peritoneal dialysis (APD), preoperative urinary tract infection (UTI), preoperative renal parenchymal thickness (RPT), Mayo adhesive probability (MAP) score, and surgeon proficiency were the high-risk factors for the prognosis and recurrence of UPJO. In addition, a nomogram was constructed based on the above 5 variables. The area under the curve (AUC) was 0.8831 after self cross-validation, which validated that the specificity of the model was favorable.

Conclusion

The column chart constructed by five factors has good predictive ability for the prognosis and recurrence of UPJO, which may provide more reasonable guidance for the clinical diagnosis and treatment of this disease.

Predicting risk of preterm birth in singleton pregnancies using machine learning algorithms

We aimed to develop, train, and validate machine learning models for predicting preterm birth (<37 weeks' gestation) in singleton pregnancies at different gestational intervals. Models were developed based on complete data from 22,603 singleton pregnancies from a prospective population-based cohort study that was conducted in 51 midwifery clinics and hospitals in Wenzhou City of China between 2014 and 2016. We applied Catboost, Random Forest, Stacked Model, Deep Neural Networks (DNN), and Support Vector Machine (SVM) algorithms, as well as logistic regression, to conduct feature selection and predictive modeling. Feature selection was implemented based on permutation-based feature importance lists derived from the machine learning models including all features, using a balanced training data set. To develop prediction models, the top 10%, 25%, and 50% most important predictive features were selected. Prediction models were developed with the training data set with 5-fold cross-validation for internal validation. Model performance was assessed using area under the receiver operating curve (AUC) values. The CatBoost-based prediction model after 26 weeks' gestation performed best with an AUC value of 0.70 (0.67, 0.73), accuracy of 0.81, sensitivity of 0.47, and specificity of 0.83. Number of antenatal care visits before 24 weeks' gestation, aspartate aminotransferase level at registration, symphysis fundal height, maternal weight, abdominal circumference, and blood pressure emerged as strong predictors after 26 completed weeks. The application of machine learning on pregnancy surveillance data is a promising approach to predict preterm birth and we identified several modifiable antenatal predictors.

Prediction models for postoperative delirium in elderly patients with machine-learning algorithms and SHapley Additive exPlanations

Postoperative delirium (POD) is a common and severe complication in elderly patients with hip fractures. Identifying high-risk patients with POD can help improve the outcome of patients with hip fractures. We conducted a retrospective study on elderly patients (≥65 years of age) who underwent orthopedic surgery with hip fracture between January 2014 and August 2019. Conventional logistic regression and five machine-learning algorithms were used to construct prediction models of POD. A nomogram for POD prediction was built with the logistic regression method. The area under the receiver operating characteristic curve (AUC-ROC), accuracy, sensitivity, and precision were calculated to evaluate different models. Feature importance of individuals was interpreted using Shapley Additive Explanations (SHAP). About 797 patients were enrolled in the study, with the incidence of POD at 9.28% (74/797). The age, renal insufficiency, chronic obstructive pulmonary disease (COPD), use of antipsychotics, lactate dehydrogenase (LDH), and C-reactive protein are used to build a nomogram for POD with an AUC of 0.71. The AUCs of five machine-learning models are 0.81 (Random Forest), 0.80 (GBM), 0.68 (AdaBoost), 0.77 (XGBoost), and 0.70 (SVM). The sensitivities of the six models range from 68.8% (logistic regression and SVM) to 91.9% (Random Forest). The precisions of the six machine-learning models range from 18.3% (logistic regression) to 67.8% (SVM). Six prediction models of POD in patients with hip fractures were constructed using logistic regression and five machine-learning algorithms. The application of machine-learning algorithms could provide convenient POD risk stratification to benefit elderly hip fracture patients.

Prediction of Complications and Prognostication in Perioperative Medicine: A Systematic Review and PROBAST Assessment of Machine Learning Tools

BACKGROUND

The utilization of artificial intelligence and machine learning as diagnostic and predictive tools in perioperative medicine holds great promise. Indeed, many studies have been performed in recent years to explore the potential. The purpose of this systematic review is to assess the current state of machine learning in perioperative medicine, its utility in prediction of complications and prognostication, and limitations related to bias and validation.

METHODS

A multidisciplinary team of clinicians and engineers conducted a systematic review using the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) protocol. Multiple databases were searched, including Scopus, Cumulative Index to Nursing and Allied Health Literature (CINAHL), the Cochrane Library, PubMed, Medline, Embase, and Web of Science. The systematic review focused on study design, type of machine learning model used, validation techniques applied, and reported model performance on prediction of complications and prognostication. This review further classified outcomes and machine learning applications using an ad hoc classification system. The Prediction model Risk Of Bias Assessment Tool (PROBAST) was used to assess risk of bias and applicability of the studies.

RESULTS

A total of 103 studies were identified. The models reported in the literature were primarily based on single-center validations (75%), with only 13% being externally validated across multiple centers. Most of the mortality models demonstrated a limited ability to discriminate and classify effectively. The PROBAST assessment indicated a high risk of systematic errors in predicted outcomes and artificial intelligence or machine learning applications.

CONCLUSIONS

The findings indicate that the development of this field is still in its early stages. This systematic review indicates that application of machine learning in perioperative medicine is still at an early stage. While many studies suggest potential utility, several key challenges must be first overcome before their introduction into clinical practice.

EDITOR’S PERSPECTIVE

Preoperative prognostic nutritional index predicts postoperative delirium in aged patients after surgery: A matched cohort study

OBJECTIVE

Prognostic nutritional index (PNI) is an indicator to evaluate the nutritional immune status of patients. This study aimed to assess whether preoperative PNI could predict the occurrence of postoperative POD in aged patients undergoing non-neurosurgery and non-cardiac surgery.

METHOD

The aged patients undergoing non-neurosurgery and non-cardiac surgery between January 2014 and August 2019 were included in the retrospective cohort study. The correlation between POD and PNI was investigated by univariate and multivariable logistic regression analysis, propensity score matching (PSM), inverse probability of treatment weighting (IPTW), and subgroup analysis.

RESULTS

In the cohort (n = 29,814), the cutoff value of PNI was 46.01 determined by the receiver operating characteristic (ROC) curve. In univariate and three multivariable regression analysis, the ORs of PNI ≤ 46.01 was 2.573(95% CI:2.261-2.929, P < 0.001),1.802 (95% CI:1.567-2.071, P < 0.001),1.463(95% CI:1.246-1.718, P < 0.001),1.370(95% CI:1.165-1.611, P < 0.001). In the PSM model and IPTW model, the ORs of PNI ≤ 46.01 were 1.424(95% CI:1.172-1.734, P < 0.001) and 1.356(95% CI:1.223-1.505, P < 0.001).

CONCLUSION

The PNI was found to have a predictive value for POD in patients undergoing non-neurosurgery and non-cardiac surgery. Improving preoperative nutritional status may be beneficial in preventing POD for aged patients.

Predicting pediatric emergence delirium using data-driven machine learning applied to electronic health record dataset at a quaternary care pediatric hospital

Objectives

Pediatric emergence delirium is an undesirable outcome that is understudied. Development of a predictive model is an initial step toward reducing its occurrence. This study aimed to apply machine learning (ML) methods to a large clinical dataset to develop a predictive model for pediatric emergence delirium.

Materials and Methods

We performed a single-center retrospective cohort study using electronic health record data from February 2015 to December 2019. We built and evaluated 4 commonly used ML models for predicting emergence delirium: least absolute shrinkage and selection operator, ridge regression, random forest, and extreme gradient boosting. The primary outcome was the occurrence of emergence delirium, defined as a Watcha score of 3 or 4 recorded at any time during recovery.

Results

The dataset included 54 776 encounters across 43 830 patients. The 4 ML models performed similarly with performance assessed by the area under the receiver operating characteristic curves ranging from 0.74 to 0.75. Notable variables associated with increased risk included adenoidectomy with or without tonsillectomy, decreasing age, midazolam premedication, and ondansetron administration, while intravenous induction and ketorolac were associated with reduced risk of emergence delirium.

Conclusions

Four different ML models demonstrated similar performance in predicting postoperative emergence delirium using a large pediatric dataset. The prediction performance of the models draws attention to our incomplete understanding of this phenomenon based on the studied variables. The results from our modeling could serve as a first step in designing a predictive clinical decision support system, but further optimization and validation are needed.

Clinical trial number and registry URL

Not applicable.

Top-Down Proteomics of Human Saliva, Analyzed with Logistic Regression and Machine Learning Methods, Reveal Molecular Signatures of Ovarian Cancer

Ovarian cancer (OC) is the most lethal of all gynecological cancers. Due to vague symptoms, OC is mostly detected at advanced stages, with a 5-year survival rate (SR) of only 30%; diagnosis at stage I increases the 5-year SR to 90%, suggesting that early diagnosis is essential to cure OC. Currently, the clinical need for an early, reliable diagnostic test for OC screening remains unmet; indeed, screening is not even recommended for healthy women with no familial history of OC for fear of post-screening adverse events. Salivary diagnostics is considered a major resource for diagnostics of the future. In this work, we searched for OC biomarkers (BMs) by comparing saliva samples of patients with various stages of OC, breast cancer (BC) patients, and healthy subjects using an unbiased, high-throughput proteomics approach. We analyzed the results using both logistic regression (LR) and machine learning (ML) for pattern analysis and variable selection to highlight molecular signatures for OC and BC diagnosis and possibly re-classification. Here, we show that saliva is an informative test fluid for an unbiased proteomic search of candidate BMs for identifying OC patients. Although we were not able to fully exploit the potential of ML methods due to the small sample size of our study, LR and ML provided patterns of candidate BMs that are now available for further validation analysis in the relevant population and for biochemical identification.