Predicting adverse outcomes of cardiac surgery with the application of artificial neural networks

Comparative analysis of machine learning vs. traditional modeling approaches for predicting in-hospital mortality after cardiac surgery: temporal and spatial external validation based on a nationwide cardiac surgery registry

AIMS

Preoperative risk assessment is crucial for cardiac surgery. Although previous studies suggested machine learning (ML) may improve in-hospital mortality predictions after cardiac surgery compared to traditional modeling approaches, the validity is doubted due to lacking external validation, limited sample sizes, and inadequate modeling considerations. We aimed to assess predictive performance between ML and traditional modelling approaches, while addressing these major limitations.

METHODS AND RESULTS

Adult cardiac surgery cases (n = 168 565) between 2013 and 2018 in the Chinese Cardiac Surgery Registry were used to develop, validate, and compare various ML vs. logistic regression (LR) models. The dataset was split for temporal (2013-2017 for training, 2018 for testing) and spatial (geographically-stratified random selection of 83 centers for training, 22 for testing) experiments, respectively. Model performances were evaluated in testing sets for discrimination and calibration. The overall in-hospital mortality was 1.9%. In the temporal testing set (n = 32 184), the best-performing ML model demonstrated a similar area under the receiver operating characteristic curve (AUC) of 0.797 (95% CI 0.779-0.815) to the LR model (AUC 0.791 [95% CI 0.775-0.808]; P = 0.12). In the spatial experiment (n = 28 323), the best ML model showed a statistically better but modest performance improvement (AUC 0.732 [95% CI 0.710-0.754]) than LR (AUC 0.713 [95% CI 0.691-0.737]; P = 0.002). Varying feature selection methods had relatively smaller effects on ML models. Most ML and LR models were significantly miscalibrated.

CONCLUSION

ML provided only marginal improvements over traditional modelling approaches in predicting cardiac surgery mortality with routine preoperative variables, which calls for more judicious use of ML in practice.

Machine learning using institution-specific multi-modal electronic health records improves mortality risk prediction for cardiac surgery patients

Background

The Society of Thoracic Surgeons risk scores are widely used to assess risk of morbidity and mortality in specific cardiac surgeries but may not perform optimally in all patients. In a cohort of patients undergoing cardiac surgery, we developed a data-driven, institution-specific machine learning-based model inferred from multi-modal electronic health records and compared the performance with the Society of Thoracic Surgeons models.

Methods

All adult patients undergoing cardiac surgery between 2011 and 2016 were included. Routine electronic health record administrative, demographic, clinical, hemodynamic, laboratory, pharmacological, and procedural data features were extracted. The outcome was postoperative mortality. The database was randomly split into training (development) and test (evaluation) cohorts. Models developed using 4 classification algorithms were compared using 6 evaluation metrics. The performance of the final model was compared with the Society of Thoracic Surgeons models for 7 index surgical procedures.

Results

A total of 6392 patients were included and described by 4016 features. Overall mortality was 3.0% (n = 193). The XGBoost algorithm using only features with no missing data (336 features) yielded the best-performing predictor. When applied to the test set, the predictor performed well (F-measure = 0.775; precision = 0.756; recall = 0.795; accuracy = 0.986; area under the receiver operating characteristic curve = 0.978; area under the precision-recall curve = 0.804). eXtreme Gradient Boosting consistently demonstrated improved performance over the Society of Thoracic Surgeons models when evaluated on index procedures within the test set.

Conclusions

Machine learning models using institution-specific multi-modal electronic health records may improve performance in predicting mortality for individual patients undergoing cardiac surgery compared with the standard-of-care, population-derived Society of Thoracic Surgeons models. Institution-specific models may provide insights complementary to population-derived risk predictions to aid patient-level decision making.

Machine learning methods for perioperative anesthetic management in cardiac surgery patients: a scoping review

Background

Machine learning (ML) is developing fast with promising prospects within medicine and already has several applications in perioperative care. We conducted a scoping review to examine the extent and potential limitations of ML implementation in perioperative anesthetic care, specifically in cardiac surgery patients.

Methods

We mapped the current literature by searching three databases: MEDLINE (Ovid), EMBASE (Ovid), and Cochrane Library. Articles were eligible if they reported on perioperative ML use in the field of cardiac surgery with relevance to anesthetic practices. Data on the applicability of ML and comparability to conventional statistical methods were extracted.

Results

Forty-six articles on ML relevant to the work of the anesthesiologist in cardiac surgery were identified. Three main categories emerged: (I) event and risk prediction, (II) hemodynamic monitoring, and (III) automation of echocardiography. Prediction models based on ML tend to behave similarly to conventional statistical methods. Using dynamic hemodynamic or ultrasound data in ML models, however, shifts the potential to promising results.

Conclusions

ML in cardiac surgery is increasingly used in perioperative anesthetic management. The majority is used for prediction purposes similar to conventional clinical scores. Remarkable ML model performances are achieved when using real-time dynamic parameters. However, beneficial clinical outcomes of ML integration have yet to be determined. Nonetheless, the first steps introducing ML in perioperative anesthetic care for cardiac surgery have been taken.

Remote patient monitoring after cardiac surgery: The utility of a novel telemedicine system

Objective

We examined cardiac surgery patients who underwent monitoring of postoperative vital parameters using medical monitoring devices which transferred data to a mobile application and a web‐based software.

Methods

From November 2017 to November 2020, a total of 2340 patients were enrolled in the remote patient monitoring system after undergoing cardiac surgery. The medical devices recorded vital parameters, such as blood pressure, pulse rate, saturation, body temperature, blood glucose, and electrocardiography were measured via the Health Monitor DakikApp and Holter ECG DakikApp devices which reported data to web‐based software and a mobile application (DakikApp Mobile Systems, Remscheid, Germany). During the follow‐up period, patients were contacted daily through text and voice messages, and video conferences. Remote Medical Evaluations (RMEs) concerning patients' medical states were performed. Medication reminders, daily treatment were communicated to the patients with the DakikApp Mobile Systems Software.

Results

During a mean follow‐up period of 78.9 ± 107.1 (10–395) days, a total of 135,786 patient contacts were recorded (782 video conferences, 2805 voice messaging, and 132,199 text correspondence). The number of RMEs handled by the Telemedicine Team was 79,560. A total of 105,335 vital parameter measurements were performed and 5024 hospital application requests (6.3% per RME) were addressed successfully and hospitalization was avoided. A total of 144 (6.1%) potentially life‐threatening complications were found to have been diagnosed early using the Telemedicine System.

Conclusion

Remote Patient Monitoring Systems combined with professional medical devices are feasible, effective, and safe for the purpose of improving postoperative outcomes.

Machine learning improves mortality risk prediction after cardiac surgery: Systematic review and meta-analysis

BACKGROUND

Interest in the usefulness of machine learning (ML) methods for outcomes prediction has continued to increase in recent years. However, the advantage of advanced ML model over traditional logistic regression (LR) remains controversial. We performed a systematic review and meta-analysis of studies comparing the discrimination accuracy between ML models versus LR in predicting operative mortality following cardiac surgery.

METHODS

The present systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis statement. Discrimination ability was assessed using the C-statistic. Pooled C-statistics and its 95% credibility interval for ML models and LR were obtained were obtained using a Bayesian framework. Pooled estimates for ML models and LR were compared to inform on difference between the 2 approaches.

RESULTS

We identified 459 published citations of which 15 studies met inclusion criteria and were used for the quantitative and qualitative analysis. When the best ML model from individual study was used, meta-analytic estimates showed that ML were associated with a significantly higher C-statistic (ML, 0.88; 95% credibility interval, 0.83-0.93 vs LR, 0.81; 95% credibility interval, 0.77-0.85; P = .03). When individual ML algorithms were instead selected, we found a nonsignificant trend toward better prediction with each of ML algorithms. We found no evidence of publication bias (P = .70).

CONCLUSIONS

The present findings suggest that when compared with LR, ML models provide better discrimination in mortality prediction after cardiac surgery. However, the magnitude and clinical influence of such an improvement remains uncertain.

Artificial Intelligence and Machine Learning in Anesthesiology

Commercial applications of artificial intelligence and machine learning have made remarkable progress recently, particularly in areas such as image recognition, natural speech processing, language translation, textual analysis, and self-learning. Progress had historically languished in these areas, such that these skills had come to seem ineffably bound to intelligence. However, these commercial advances have performed best at single-task applications in which imperfect outputs and occasional frank errors can be tolerated.The practice of anesthesiology is different. It embodies a requirement for high reliability, and a pressured cycle of interpretation, physical action, and response rather than any single cognitive act. This review covers the basics of what is meant by artificial intelligence and machine learning for the practicing anesthesiologist, describing how decision-making behaviors can emerge from simple equations. Relevant clinical questions are introduced to illustrate how machine learning might help solve them-perhaps bringing anesthesiology into an era of machine-assisted discovery.

Off-pump versus On-pump Coronary Artery Bypass Grafting in Frail Patients: Study Protocol for the FRAGILE Multicenter Randomized Controlled Trial

Introduction

Advances in modern medicine have led to people living longer and healthier lives. Frailty is an emerging concept in medicine yet to be explored as a risk factor in cardiac surgery. When it comes to CABG surgery, randomized controlled clinical trials have primarily focused on low-risk (ROOBY, CORONARY), elevated-risk (GOPCABE) or high-risk patients (BBS), but not on frail patients. Therefore, we believe that off-pump CABG could be an important technique in patients with limited functional capacity to respond to surgical stress. In this study, the authors introduce the new national, multicenter, randomized, controlled trial "FRAGILE", to be developed in the main cardiac surgery centers of Brazil, to clarify the potential benefit of off-pump CABG in frail patients.

Methods

FRAGILE is a two-arm, parallel-group, multicentre, individually randomized (1:1) controlled trial which will enroll 630 patients with blinded outcome assessment (at 30 days, 6 months, 1 year, 2 years and 3 years), which aims to compare adverse cardiac and cerebrovascular events after off-pump versus on-pump CABG in pre-frail and frail patients. Primary outcomes will be all-cause mortality, acute myocardial infarction, cardiac arrest with successful resuscitation, low cardiac output syndrome/cardiogenic shock, stroke, and coronary reintervention. Secondary outcomes will be major adverse cardiac and cerebrovascular events, operative time, mechanical ventilation time, hyperdynamic shock, new onset of atrial fibrillation, renal replacement therapy, reoperation for bleeding, pneumonia, length of stay in intensive care unit, length of stay in hospital, number of units of blood transfused, graft patency, rate of complete revascularization, neurobehavioral outcomes after cardiac surgery, quality of life after cardiac surgery and costs.

Discussion

FRAGILE trial will determine whether off-pump CABG is superior to conventional on-pump CABG in the surgical treatment of pre-frail and frail patients.

Trial registration

ClinicalTrials.gov, ID: NCT02338947. Registered on August 29^th 2014; last updated on March 21^st 2016.

Complementarity of Clinician Judgment and Evidence Based Models in Medical Decision Making: Antecedents, Prospects, and Challenges

Early accounts of the development of modern medicine suggest that the clinical skills, scientific competence, and doctors' judgment were the main impetus for treatment decision, diagnosis, prognosis, therapy assessment, and medical progress. Yet, clinician judgment has its own critics and is sometimes harshly described as notoriously fallacious and an irrational and unfathomable black box with little transparency. With the rise of contemporary medical research, the reputation of clinician judgment has undergone significant reformation in the last century as its fallacious aspects are increasingly emphasized relative to the evidence based options. Within the last decade, however, medical forecasting literature has seen tremendous change and new understanding is emerging on best ways of sharing medical information to complement the evidence based medicine practices. This review revisits and highlights the core debate on clinical judgments and its interrelations with evidence based medicine. It outlines the key empirical results of clinician judgments relative to evidence based models and identifies its key strengths and prospects, the key limitations and conditions for the effective use of clinician judgment, and the extent to which it can be optimized and professionalized for medical use.

Neural Network Prediction of ICU Length of Stay Following Cardiac Surgery Based on Pre-Incision Variables

Background

Advanced predictive analytical techniques are being increasingly applied to clinical risk assessment. This study compared a neural network model to several other models in predicting the length of stay (LOS) in the cardiac surgical intensive care unit (ICU) based on pre-incision patient characteristics.

Methods

Thirty six variables collected from 185 cardiac surgical patients were analyzed for contribution to ICU LOS. The Automatic Linear Modeling (ALM) module of IBM-SPSS software identified 8 factors with statistically significant associations with ICU LOS; these factors were also analyzed with the Artificial Neural Network (ANN) module of the same software. The weighted contributions of each factor (“trained” data) were then applied to data for a “new” patient to predict ICU LOS for that individual.

Results

Factors identified in the ALM model were: use of an intra-aortic balloon pump; O₂ delivery index; age; use of positive cardiac inotropic agents; hematocrit; serum creatinine ≥ 1.3 mg/deciliter; gender; arterial pCO₂. The r² value for ALM prediction of ICU LOS in the initial (training) model was 0.356, p <0.0001. Cross validation in prediction of a “new” patient yielded r² = 0.200, p <0.0001. The same 8 factors analyzed with ANN yielded a training prediction r² of 0.535 (p <0.0001) and a cross validation prediction r² of 0.410, p <0.0001. Two additional predictive algorithms were studied, but they had lower prediction accuracies. Our validated neural network model identified the upper quartile of ICU LOS with an odds ratio of 9.8(p <0.0001).

Conclusions

ANN demonstrated a 2-fold greater accuracy than ALM in prediction of observed ICU LOS. This greater accuracy would be presumed to result from the capacity of ANN to capture nonlinear effects and higher order interactions. Predictive modeling may be of value in early anticipation of risks of post-operative morbidity and utilization of ICU facilities.

Predicting reintubation, prolonged mechanical ventilation and death in post-coronary artery bypass graft surgery: a comparison between artificial neural networks and logistic regression models

INTRODUCTION

In coronary artery bypass (CABG) surgery, the common complications are the need for reintubation, prolonged mechanical ventilation (PMV) and death. Thus, a reliable model for the prognostic evaluation of those particular outcomes is a worthwhile pursuit. The existence of such a system would lead to better resource planning, cost reductions and an increased ability to guide preventive strategies. The aim of this study was to compare different methods - logistic regression (LR) and artificial neural networks (ANNs) - in accomplishing this goal.

MATERIAL AND METHODS

Subjects undergoing CABG (n = 1315) were divided into training (n = 1053) and validation (n = 262) groups. The set of independent variables consisted of age, gender, weight, height, body mass index, diabetes, creatinine level, cardiopulmonary bypass, presence of preserved ventricular function, moderate and severe ventricular dysfunction and total number of grafts. The PMV was also an input for the prediction of death. The ability of ANN to discriminate outcomes was assessed using receiver-operating characteristic (ROC) analysis and the results were compared using a multivariate LR.

RESULTS

The ROC curve areas for LR and ANN models, respectively, were: for reintubation 0.62 (CI: 0.50-0.75) and 0.65 (CI: 0.53-0.77); for PMV 0.67 (CI: 0.57-0.78) and 0.72 (CI: 0.64-0.81); and for death 0.86 (CI: 0.79-0.93) and 0.85 (CI: 0.80-0.91). No differences were observed between models.

CONCLUSIONS

The ANN has similar discriminating power in predicting reintubation, PMV and death outcomes. Thus, both models may be applicable as a predictor for these outcomes in subjects undergoing CABG.

Mortality predicted accuracy for hepatocellular carcinoma patients with hepatic resection using artificial neural network

The aim of this present study is firstly to compare significant predictors of mortality for hepatocellular carcinoma (HCC) patients undergoing resection between artificial neural network (ANN) and logistic regression (LR) models and secondly to evaluate the predictive accuracy of ANN and LR in different survival year estimation models. We constructed a prognostic model for 434 patients with 21 potential input variables by Cox regression model. Model performance was measured by numbers of significant predictors and predictive accuracy. The results indicated that ANN had double to triple numbers of significant predictors at 1-, 3-, and 5-year survival models as compared with LR models. Scores of accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC) of 1-, 3-, and 5-year survival estimation models using ANN were superior to those of LR in all the training sets and most of the validation sets. The study demonstrated that ANN not only had a great number of predictors of mortality variables but also provided accurate prediction, as compared with conventional methods. It is suggested that physicians consider using data mining methods as supplemental tools for clinical decision-making and prognostic evaluation.

Addition of Frailty and Disability to Cardiac Surgery Risk Scores Identifies Elderly Patients at High Risk of Mortality or Major Morbidity

Background—

Cardiac surgery risk scores perform poorly in elderly patients, in part because they do not take into account frailty and disability which are critical determinants of health status with advanced age. There is an unmet need to combine established cardiac surgery risk scores with measures of frailty and disability to provide a more complete model for risk prediction in elderly patients undergoing cardiac surgery.

Methods and Results—

This was a prospective, multicenter cohort study of elderly patients (≥70 years) undergoing coronary artery bypass and/or valve surgery in the United States and Canada. Four different frailty scales, 3 disability scales, and 5 cardiac surgery risk scores were measured in all patients. The primary outcome was the STS composite end point of in-hospital postoperative mortality or major morbidity. A total of 152 patients were enrolled, with a mean age of 75.9±4.4 years and 34% women. Depending on the scale used, 20–46% of patients were found to be frail, and 5–76% were found to have at least 1 disability. The most predictive scale in each domain was: 5-meter gait speed ≥6 seconds as a measure of frailty (odds ratio [OR], 2.63; 95% confidence interval [CI], 1.17–5.90), ≥3 impairments in the Nagi scale as a measure of disability (OR, 2.98; 95% CI, 1.35–6.56) and either the Parsonnet score (OR, 1.08; 95% CI, 1.04–1.13) or Society of Thoracic Surgeons Predicted Risk of Mortality or Major Morbidity (STS-PROMM) (OR, 1.05; 95% CI, 1.01–1.09) as a cardiac surgery risk score. Compared with the Parsonnet score or STS-PROMM alone, (area under the curve, 0.68–0.72), addition of frailty and disability provided incremental value and improved model discrimination (area under the curve, 0.73–0.76).

Conclusions—

Clinicians should use an integrative approach combining frailty, disability, and risk scores to better characterize elderly patients referred for cardiac surgery and identify those that are at increased risk.