Predictive Utility of a Machine Learning Algorithm in Estimating Mortality Risk in Cardiac Surgery

The Emerging and Important Role of Artificial Intelligence in Cardiac Surgery

Artificial Intelligence (AI) has greatly affected our everyday lives and holds great promise to change the landscape of medicine. AI is particularly positioned to improve care for the increasingly complex patients undergoing cardiac surgery utilizing immense amount of data generated in the course of their care. When deployed, AI can be used to analyze this information at the patient's bedside more expediently and accurately, all while providing new insights. This review summarizes the current applications of AI in cardiac surgery, from the vantage point of a patient's journey. Applications of AI include pre-operative risk assessment, intraoperative planning, post-operative patient care and out-patient telemonitoring, encompassing the spectrum of cardiac surgical care. Offloading of administrative processes and enhanced experience with information gathering also represent a unique and underrepresented avenue for future utilization of AI. As clinicians, understanding the nomenclature and applications of AI is important to contextualize problems, to ensure problem-driven solutions and for clinical benefit. Precision medicine, and thus clinically relevant AI, remains dependent on data curation and warehousing to gather insights from large multicenter repositories while treating privacy with the utmost importance. AI tasks should not be siloed but rather holistically integrated into clinical workflow to retain context and relevance. As cardiac surgeons, AI allows us to look forward to a bright future of more efficient utilization of our clinical expertise toward high-level decision making and technical prowess.

Artificial intelligence in cardiac surgery: A systematic review

BACKGROUND

Artificial intelligence (AI) has emerged as a tool to potentially increase the efficiency and efficacy of cardiovascular care and improve clinical outcomes. This study aims to provide an overview of applications of AI in cardiac surgery.

METHODS

A systematic literature search on AI applications in cardiac surgery from inception to February 2024 was conducted. Articles were then filtered based on the inclusion and exclusion criteria and the risk of bias was assessed. Key findings were then summarized.

RESULTS

A total of 81 studies were found that reported on AI applications in cardiac surgery. There is a rapid rise in studies since 2020. The most popular machine learning technique was random forest (n = 48), followed by support vector machine (n = 33), logistic regression (n = 32), and eXtreme Gradient Boosting (n = 31). Most of the studies were on adult patients, conducted in China, and involved procedures such as valvular surgery (24.7%), heart transplant (9.4%), coronary revascularization (11.8%), congenital heart disease surgery (3.5%), and aortic dissection repair (2.4%). Regarding evaluation outcomes, 35 studies examined the performance, 26 studies examined clinician outcomes, and 20 studies examined patient outcomes.

CONCLUSION

AI was mainly used to predict complications following cardiac surgeries and improve clinicians' decision-making by providing better preoperative risk assessment, stratification, and prognostication. While the application of AI in cardiac surgery has greatly progressed in the last decade, further studies need to be conducted to verify accuracy and ensure safety before use in clinical practice.

Development and preliminary assessment of a machine learning model to predict myocardial infarction and cardiac arrest after major operations

INTRODUCTION

Accurate prediction of complications often informs shared decision-making. Derived over 10 years ago to enhance prediction of intra/post-operative myocardial infarction and cardiac arrest (MI/CA), the Gupta score has been criticized for unreliable calibration and inclusion of a wide spectrum of unrelated operations. In the present study, we developed a novel machine learning (ML) model to estimate perioperative risk of MI/CA and compared it to the Gupta score.

METHODS

Patients undergoing major operations were identified from the 2016-2020 ACS-NSQIP. The Gupta score was calculated for each patient, and a novel ML model was developed to predict MI/CA using ACS NSQIP-provided data fields as covariates. Discrimination (C-statistic) and calibration (Brier score) of the ML model were compared to the existing Gupta score within the entire cohort and across operative subgroups.

RESULTS

Of 2,473,487 patients included for analysis, 25,177 (1.0%) experienced MI/CA (55.2% MI, 39.1% CA, 5.6% MI and CA). The ML model, which was fit using a randomly selected training cohort, exhibited higher discrimination within the testing dataset compared to the Gupta score (C-statistic 0.84 vs 0.80, p < 0.001). Furthermore, the ML model had significantly better calibration in the entire cohort (Brier score 0.0097 vs 0.0100). Model performance was markedly improved among patients undergoing thoracic, aortic, peripheral vascular and foregut surgery.

CONCLUSIONS

The present ML model outperformed the Gupta score in the prognostication of MI/CA across a heterogenous range of operations. Given the growing integration of ML into healthcare, such models may be readily incorporated into clinical practice and guide benchmarking efforts.

Personalizing patient risk of a life-altering event: An application of machine learning to hemiarch surgery

OBJECTIVE

The study objective was to assess a machine learning model's ability to predict the occurrence of life-altering events in hemiarch surgery and determine contributing patient characteristics and intraoperative factors.

METHODS

In total, 602 patients who underwent hemiarch replacement at a high-volume aortic center from 2009 to 2022 were included. Patients were randomly divided into training (80%) and testing (20%) sets with various eXtreme gradient boosting candidate models constructed to predict the risk of experiencing life-altering events, including stroke, mortality, or new renal replacement therapy requirement. A total of 64 input parameters from the index hospitalization were identified, including 24 demographic characteristics as well as 8 preoperative and 32 intraoperative variables. A SHapley Additive exPlanation beeswarm plot was generated to identify and interpret the impact of individual features on the predictions of the final model.

RESULTS

A life-altering event was noted in 15% (90/602) of patients who underwent hemiarch replacement, including urgent/emergency cases and dissections. The final eXtreme Gradient Boosting model demonstrated a cross-validation accuracy of 88% on the testing set and was well calibrated as evidenced by a low Brier score of 0.12. The best performing model achieved an area under the receiver operating characteristic curve of 0.76 and an area under the precision recall curve of 0.55. The SHapley Additive exPlanation beeswarm plot provided insights into key features that significantly influenced model prediction.

CONCLUSIONS

Machine learning demonstrated superior accuracy in predicting hemiarch patients who would experience a life-altering event. This model may help to guide patients and clinicians in stratifying risk on an individual basis, which may in turn influence clinical decision-making.

Prediction of coronary artery bypass graft outcomes using a single surgical note: An artificial intelligence-based prediction model study

BACKGROUND

Healthcare providers currently calculate risk of the composite outcome of morbidity or mortality associated with a coronary artery bypass grafting (CABG) surgery through manual input of variables into a logistic regression-based risk calculator. This study indicates that automated artificial intelligence (AI)-based techniques can instead calculate risk. Specifically, we present novel numerical embedding techniques that enable NLP (natural language processing) models to achieve higher performance than the risk calculator using a single preoperative surgical note.

METHODS

The most recent preoperative surgical consult notes of 1,738 patients who received an isolated CABG from July 1, 2014 to November 1, 2022 at a single institution were analyzed. The primary outcome was the Society of Thoracic Surgeons defined composite outcome of morbidity or mortality (MM). We tested three numerical-embedding techniques on the widely used TextCNN classification model: 1a) Basic embedding, treat numbers as word tokens; 1b) Basic embedding with a dataloader that Replaces out-of-context (ROOC) numbers with a tag, where context is defined as within a number of tokens of specified keywords; 2) ScaleNum, an embedding technique that scales in-context numbers via a learned sigmoid-linear-log function; and 3) AttnToNum, a ScaleNum-derivative that updates the ScaleNum embeddings via multi-headed attention applied to local context. Predictive performance was measured via area under the receiver operating characteristic curve (AUC) on holdout sets from 10 random-split experiments. For eXplainable-AI (X-AI), we calculate SHapley Additive exPlanation (SHAP) values at an ngram resolution (SHAP-N). While the analyses focus on TextCNN, we execute an analogous performance pipeline with a long short-term memory (LSTM) model to test if the numerical embedding advantage is robust to model architecture.

RESULTS

A total of 567 (32.6%) patients had MM following CABG. The embedding performances are as follows with the TextCNN architecture: 1a) Basic, mean AUC 0.788 [95% CI (confidence interval): 0.768-0.809]; 1b) ROOC, 0.801 [CI: 0.788-0.815]; 2) ScaleNum, 0.808 [CI: 0.785-0.821]; and 3) AttnToNum, 0.821 [CI: 0.806-0.834]. The LSTM architecture produced a similar trend. Permutation tests indicate that AttnToNum outperforms the other embedding techniques, though not statistically significant verse ScaleNum (p-value of .07). SHAP-N analyses indicate that the model learns to associate low blood urine nitrate (BUN) and creatinine values with survival. A correlation analysis of the attention-updated numerical embeddings indicates that AttnToNum learns to incorporate both number magnitude and local context to derive semantic similarities.

CONCLUSION

This research presents both quantitative and clinical novel contributions. Quantitatively, we contribute two new embedding techniques: AttnToNum and ScaleNum. Both can embed strictly positive and bounded numerical values, and both surpass basic embeddings in predictive performance. The results suggest AttnToNum outperforms ScaleNum. With regards to clinical research, we show that AI methods can predict outcomes after CABG using a single preoperative note at a performance that matches or surpasses the current risk calculator. These findings reveal the potential role of NLP in automated registry reporting and quality improvement.

A machine learning approach to predicting 30-day mortality following paediatric cardiac surgery: findings from the Australia New Zealand Congenital Outcomes Registry for Surgery (ANZCORS)

OBJECTIVES

We aim to develop the first risk prediction model for 30-day mortality for the Australian and New Zealand patient populations and examine whether machine learning (ML) algorithms outperform traditional statistical approaches.

METHODS

Data from the Australia New Zealand Congenital Outcomes Registry for Surgery, which contains information on every paediatric cardiac surgical encounter in Australian and New Zealand for patients aged <18 years between January 2013 and December 2021, were analysed (n = 14 343). The outcome was mortality within the 30-day period following a surgical encounter, with ∼30% of the observations randomly selected to be used for validation of the final model. Three different ML methods were used, all of which employed five-fold cross-validation to prevent overfitting, with model performance judged primarily by the area under the receiver operating curve (AUC).

RESULTS

Among the 14 343 30-day periods, there were 188 deaths (1.3%). In the validation data, the gradient-boosted tree obtained the best performance [AUC = 0.87, 95% confidence interval = (0.82, 0.92); calibration = 0.97, 95% confidence interval = (0.72, 1.27)], outperforming penalized logistic regression and artificial neural networks (AUC of 0.82 and 0.81, respectively). The strongest predictors of mortality in the gradient boosting trees were patient weight, STAT score, age and gender.

CONCLUSIONS

Our risk prediction model outperformed logistic regression and achieved a level of discrimination comparable to the PRAiS2 and Society of Thoracic Surgery Congenital Heart Surgery Database mortality risk models (both which obtained AUC = 0.86). Non-linear ML methods can be used to construct accurate clinical risk prediction tools.

Comparison of machine learning techniques in prediction of mortality following cardiac surgery: analysis of over 220 000 patients from a large national database

OBJECTIVES

To perform a systematic comparison of in-hospital mortality risk prediction post-cardiac surgery, between the predominant scoring system-European System for Cardiac Operative Risk Evaluation (EuroSCORE) II, logistic regression (LR) retrained on the same variables and alternative machine learning techniques (ML)-random forest (RF), neural networks (NN), XGBoost and weighted support vector machine.

METHODS

Retrospective analyses of prospectively routinely collected data on adult patients undergoing cardiac surgery in the UK from January 2012 to March 2019. Data were temporally split 70:30 into training and validation subsets. Mortality prediction models were created using the 18 variables of EuroSCORE II. Comparisons of discrimination, calibration and clinical utility were then conducted. Changes in model performance, variable-importance over time and hospital/operation-based model performance were also reviewed.

RESULTS

Of the 227 087 adults who underwent cardiac surgery during the study period, there were 6258 deaths (2.76%). In the testing cohort, there was an improvement in discrimination [XGBoost (95% confidence interval (CI) area under the receiver operator curve (AUC), 0.834-0.834, F1 score, 0.276-0.280) and RF (95% CI AUC, 0.833-0.834, F1, 0.277-0.281)] compared with EuroSCORE II (95% CI AUC, 0.817-0.818, F1, 0.243-0.245). There was no significant improvement in calibration with ML and retrained-LR compared to EuroSCORE II. However, EuroSCORE II overestimated risk across all deciles of risk and over time. The calibration drift was lowest in NN, XGBoost and RF compared with EuroSCORE II. Decision curve analysis showed XGBoost and RF to have greater net benefit than EuroSCORE II.

CONCLUSIONS

ML techniques showed some statistical improvements over retrained-LR and EuroSCORE II. The clinical impact of this improvement is modest at present. However the incorporation of additional risk factors in future studies may improve upon these findings and warrants further study.

Machine learning using institution-specific multi-modal electronic health records improves mortality risk prediction for cardiac surgery patients

Background

The Society of Thoracic Surgeons risk scores are widely used to assess risk of morbidity and mortality in specific cardiac surgeries but may not perform optimally in all patients. In a cohort of patients undergoing cardiac surgery, we developed a data-driven, institution-specific machine learning-based model inferred from multi-modal electronic health records and compared the performance with the Society of Thoracic Surgeons models.

Methods

All adult patients undergoing cardiac surgery between 2011 and 2016 were included. Routine electronic health record administrative, demographic, clinical, hemodynamic, laboratory, pharmacological, and procedural data features were extracted. The outcome was postoperative mortality. The database was randomly split into training (development) and test (evaluation) cohorts. Models developed using 4 classification algorithms were compared using 6 evaluation metrics. The performance of the final model was compared with the Society of Thoracic Surgeons models for 7 index surgical procedures.

Results

A total of 6392 patients were included and described by 4016 features. Overall mortality was 3.0% (n = 193). The XGBoost algorithm using only features with no missing data (336 features) yielded the best-performing predictor. When applied to the test set, the predictor performed well (F-measure = 0.775; precision = 0.756; recall = 0.795; accuracy = 0.986; area under the receiver operating characteristic curve = 0.978; area under the precision-recall curve = 0.804). eXtreme Gradient Boosting consistently demonstrated improved performance over the Society of Thoracic Surgeons models when evaluated on index procedures within the test set.

Conclusions

Machine learning models using institution-specific multi-modal electronic health records may improve performance in predicting mortality for individual patients undergoing cardiac surgery compared with the standard-of-care, population-derived Society of Thoracic Surgeons models. Institution-specific models may provide insights complementary to population-derived risk predictions to aid patient-level decision making.

Prediction of operative mortality for patients undergoing cardiac surgical procedures without established risk scores

OBJECTIVE

Current cardiac surgery risk models do not address a substantial fraction of procedures. We sought to create models to predict the risk of operative mortality for an expanded set of cases.

METHODS

Four supervised machine learning models were trained using preoperative variables present in the Society of Thoracic Surgeons (STS) data set of the Massachusetts General Hospital to predict and classify operative mortality in procedures without STS risk scores. A total of 424 (5.5%) mortality events occurred out of 7745 cases. Models included logistic regression with elastic net regularization (LogReg), support vector machine, random forest (RF), and extreme gradient boosted trees (XGBoost). Model discrimination was assessed via area under the receiver operating characteristic curve (AUC), and calibration was assessed via calibration slope and expected-to-observed event ratio. External validation was performed using STS data sets from Brigham and Women's Hospital (BWH) and the Johns Hopkins Hospital (JHH).

RESULTS

Models performed comparably with the highest mean AUC of 0.83 (RF) and expected-to-observed event ratio of 1.00. On external validation, the AUC was 0.81 in BWH (RF) and 0.79 in JHH (LogReg/RF). Models trained and applied on the same institution's data achieved AUCs of 0.81 (BWH: LogReg/RF/XGBoost) and 0.82 (JHH: LogReg/RF/XGBoost).

CONCLUSIONS

Machine learning models trained on preoperative patient data can predict operative mortality at a high level of accuracy for cardiac surgical procedures without established risk scores. Such procedures comprise 23% of all cardiac surgical procedures nationwide. This work also highlights the value of using local institutional data to train new prediction models that account for institution-specific practices.

Optimizing the Definition of Proximal Junctional Kyphosis: A Sensitivity Analysis

STUDY DESIGN

Diagnostic binary threshold analysis.

OBJECTIVE

(1) Perform a sensitivity analysis demonstrating the test performance metrics for any combination of proximal junctional angle (PJA) magnitude and change; (2) Propose a new proximal junctional kyphosis (PJK) criteria.

SUMMARY OF BACKGROUND DATA

Previous definitions of PJK have been arbitrarily selected and then tested through retrospective case series, often showing little correlation with clinical outcomes.

MATERIALS AND METHODS

Surgically treated adult spinal deformity patients (≥4 levels fused) enrolled into a prospective, multicenter database were evaluated at a minimum 2-year follow-up for proximal junctional failure (PJF). Using PJF as the outcome of interest, test performance metrics including sensitivity, positive predictive value, and F1 metrics (harmonic mean of precision and recall) were calculated for all combinations of PJA magnitude and change using different combinations of perijunctional vertebrae. The combination with the highest F1 score was selected as the new PJK criteria. Performance metrics of previous PJK definitions and the new PJK definition were compared.

RESULTS

Of the total, 669 patients were reviewed. PJF rate was 10%. Overall, the highest F1 scores were achieved when the upper instrumented vertebrae -1 (UIV-1)/UIV+2 angle was measured. For lower thoracic cases, out of all the PJA and magnitude/change combinations tested, a UIV-1/UIV+2 magnitude of -28° and a change of -20° was associated with the highest F1 score. For upper thoracic cases, a UIV-1/UIV+2 magnitude of -30° and a change of -24° were associated with the highest F1 score. Using PJF as the outcome, patients meeting this new criterion (11.5%) at 6 weeks had the lowest survival rate (74.7%) at 2 years postoperative, compared with Glattes (84.4%) and Bridwell (77.4%).

CONCLUSIONS

Out of all possible PJA magnitude and change combinations, without stratifying by upper thoracic versus lower thoracic fusions, a magnitude of ≤-28° and a change of ≤-22° provide the best test performance metrics for predicting PJF.

Machine learning-based prediction of 1-year mortality in hypertensive patients undergoing coronary revascularization surgery

BACKGROUND

Machine learning (ML) has shown promising results in all fields of medicine, including preventive cardiology. Hypertensive patients are at higher risk of mortality after coronary artery bypass graft (CABG) surgery; thus, we aimed to design and evaluate five ML models to predict 1-year mortality among hypertensive patients who underwent CABG.

HYOTHESIS

ML algorithms can significantly improve mortality prediction after CABG.

METHODS

Tehran Heart Center's CABG data registry was used to extract several baseline and peri-procedural characteristics and mortality data. The best features were chosen using random forest (RF) feature selection algorithm. Five ML models were developed to predict 1-year mortality: logistic regression (LR), RF, artificial neural network (ANN), extreme gradient boosting (XGB), and naïve Bayes (NB). The area under the curve (AUC), sensitivity, and specificity were used to evaluate the models.

RESULTS

Among the 8,493 hypertensive patients who underwent CABG (mean age of 68.27 ± 9.27 years), 303 died in the first year. Eleven features were selected as the best predictors, among which total ventilation hours and ejection fraction were the leading ones. LR showed the best prediction ability with an AUC of 0.82, while the least AUC was for the NB model (0.79). Among the subgroups, the highest AUC for LR model was for two age range groups (50-59 and 80-89 years), overweight, diabetic, and smoker subgroups of hypertensive patients.

CONCLUSIONS

All ML models had excellent performance in predicting 1-year mortality among CABG hypertension patients, while LR was the best regarding AUC. These models can help clinicians assess the risk of mortality in specific subgroups at higher risk (such as hypertensive ones).

Deep Learning to Predict Mortality After Cardiothoracic Surgery Using Preoperative Chest Radiographs

BACKGROUND

The Society of Thoracic Surgeons Predicted Risk of Mortality (STS-PROM) estimates mortality risk only for certain common procedures (eg, coronary artery bypass or valve surgery) and is cumbersome, requiring greater than 60 inputs. We hypothesized that deep learning can estimate postoperative mortality risk based on a preoperative chest radiograph for cardiac surgeries in which STS-PROM scores were available (STS index procedures) or unavailable (non-STS index procedures).

METHODS

We developed a deep learning model (CXR-CTSurgery) to predict postoperative mortality based on preoperative chest radiographs in 9283 patients at Massachusetts General Hospital (MGH) having cardiac surgery before April 8, 2014. CXR-CTSurgery was tested on 3615 different MGH patients and externally tested on 2840 patients from Brigham and Women's Hospital (BWH) having surgery after April 8, 2014. Discrimination for mortality was compared with the STS-PROM using the C-statistic. Calibration was assessed using the observed-to-expected ratio (O/E ratio).

RESULTS

For STS index procedures, CXR-CTSurgery had a C-statistic similar to STS-PROM at MGH (CXR-CTSurgery: 0.83 vs STS-PROM: 0.88; P = .20) and BWH (0.74 vs 0.80; P = .14) testing cohorts. The CXR-CTSurgery C-statistic for non-STS index procedures was similar to STS index procedures in the MGH (0.87 vs 0.83) and BWH (0.73 vs 0.74) testing cohorts. For STS index procedures, CXR-CTSurgery had better calibration than the STS-PROM in the MGH (O/E ratio: 0.74 vs 0.52) and BWH (O/E ratio: 0.91 vs 0.73) testing cohorts.

CONCLUSIONS

CXR-CTSurgery predicts postoperative mortality based on a preoperative CXR with similar discrimination and better calibration than the STS-PROM. This may be useful when the STS-PROM cannot be calculated or for non-STS index procedures.

A machine learning approach to high-risk cardiac surgery risk scoring

INTRODUCTION

In patients undergoing high-risk cardiac surgery, the uncertainty of outcome may complicate the decision process to intervene. To augment decision-making, a machine learning approach was used to determine weighted personalized factors contributing to mortality.

METHODS

American College of Surgeons National Surgical Quality Improvement Program was queried for cardiac surgery patients with predicted mortality ≥10% between 2012 and 2019. Multiple machine learning models were investigated, with significant predictors ultimately used in gradient boosting machine (GBM) modeling. GBM-trained data were then used for local interpretable model-agnostic explanations (LIME) modeling to provide individual patient-specific mortality prediction.

RESULTS

A total of 194 patient deaths among 1291 high-risk cardiac surgeries were included. GBM performance was superior to other model approaches. The top five factors contributing to mortality in LIME modeling were preoperative dialysis, emergent cases, Hispanic ethnicity, steroid use, and ventilator dependence. LIME results individualized patient factors with model probability and explanation of fit.

CONCLUSIONS

The application of machine learning techniques provides individualized predicted mortality and identifies contributing factors in high-risk cardiac surgery. Employment of this modeling to the Society of Thoracic Surgeons database may provide individualized risk factors contributing to mortality.

Prediction of Postoperative Deterioration in Cardiac Surgery Patients Using Electronic Health Record and Physiologic Waveform Data

BACKGROUND

Postoperative hemodynamic deterioration among cardiac surgical patients can indicate or lead to adverse outcomes. Whereas prediction models for such events using electronic health records or physiologic waveform data are previously described, their combined value remains incompletely defined. The authors hypothesized that models incorporating electronic health record and processed waveform signal data (electrocardiogram lead II, pulse plethysmography, arterial catheter tracing) would yield improved performance versus either modality alone.

METHODS

Intensive care unit data were reviewed after elective adult cardiac surgical procedures at an academic center between 2013 and 2020. Model features included electronic health record features and physiologic waveforms. Tensor decomposition was used for waveform feature reduction. Machine learning-based prediction models included a 2013 to 2017 training set and a 2017 to 2020 temporal holdout test set. The primary outcome was a postoperative deterioration event, defined as a composite of low cardiac index of less than 2.0 ml min-1 m-2, mean arterial pressure of less than 55 mmHg sustained for 120 min or longer, new or escalated inotrope/vasopressor infusion, epinephrine bolus of 1 mg or more, or intensive care unit mortality. Prediction models analyzed data 8 h before events.

RESULTS

Among 1,555 cases, 185 (12%) experienced 276 deterioration events, most commonly including low cardiac index (7.0% of patients), new inotrope (1.9%), and sustained hypotension (1.4%). The best performing model on the 2013 to 2017 training set yielded a C-statistic of 0.803 (95% CI, 0.799 to 0.807), although performance was substantially lower in the 2017 to 2020 test set (0.709, 0.705 to 0.712). Test set performance of the combined model was greater than corresponding models limited to solely electronic health record features (0.641; 95% CI, 0.637 to 0.646) or waveform features (0.697; 95% CI, 0.693 to 0.701).

CONCLUSIONS

Clinical deterioration prediction models combining electronic health record data and waveform data were superior to either modality alone, and performance of combined models was primarily driven by waveform data. Decreased performance of prediction models during temporal validation may be explained by data set shift, a core challenge of healthcare prediction modeling.

EDITOR’S PERSPECTIVE

Comparison of Machine Learning Models Including Preoperative, Intraoperative, and Postoperative Data and Mortality After Cardiac Surgery

IMPORTANCE

A variety of perioperative risk factors are associated with postoperative mortality risk. However, the relative contribution of routinely collected intraoperative clinical parameters to short-term and long-term mortality remains understudied.

OBJECTIVE

To examine the performance of multiple machine learning models with data from different perioperative periods to predict 30-day, 1-year, and 5-year mortality and investigate factors that contribute to these predictions.

DESIGN, SETTING, AND PARTICIPANTS

In this prognostic study using prospectively collected data, risk prediction models were developed for short-term and long-term mortality after cardiac surgery. Included participants were adult patients undergoing a first-time valve operation, coronary artery bypass grafting, or a combination of both between 1997 and 2017 in a single center, the University Medical Centre Groningen in the Netherlands. Mortality data were obtained in November 2017. Data analysis took place between February 2020 and August 2021.

EXPOSURE

Cardiac surgery.

MAIN OUTCOMES AND MEASURES

Postoperative mortality rates at 30 days, 1 year, and 5 years were the primary outcomes. The area under the receiver operating characteristic curve (AUROC) was used to assess discrimination. The contribution of all preoperative, intraoperative hemodynamic and temperature, and postoperative factors to mortality was investigated using Shapley additive explanations (SHAP) values.

RESULTS

Data from 9415 patients who underwent cardiac surgery (median [IQR] age, 68 [60-74] years; 2554 [27.1%] women) were included. Overall mortality rates at 30 days, 1 year, and 5 years were 268 patients (2.8%), 420 patients (4.5%), and 612 patients (6.5%), respectively. Models including preoperative, intraoperative, and postoperative data achieved AUROC values of 0.82 (95% CI, 0.78-0.86), 0.81 (95% CI, 0.77-0.85), and 0.80 (95% CI, 0.75-0.84) for 30-day, 1-year, and 5-year mortality, respectively. Models including only postoperative data performed similarly (30 days: 0.78 [95% CI, 0.73-0.82]; 1 year: 0.79 [95% CI, 0.74-0.83]; 5 years: 0.77 [95% CI, 0.73-0.82]). However, models based on all perioperative data provided less clinically usable predictions, with lower detection rates; for example, postoperative models identified a high-risk group with a 2.8-fold increase in risk for 5-year mortality (4.1 [95% CI, 3.3-5.1]) vs an increase of 11.3 (95% CI, 6.8-18.7) for the high-risk group identified by the full perioperative model. Postoperative markers associated with metabolic dysfunction and decreased kidney function were the main factors contributing to mortality risk.

CONCLUSIONS AND RELEVANCE

This study found that the addition of continuous intraoperative hemodynamic and temperature data to postoperative data was not associated with improved machine learning-based identification of patients at increased risk of short-term and long-term mortality after cardiac operations.

Expression Analysis of Ligand-Receptor Pairs Identifies Cell-to-Cell Crosstalk between Macrophages and Tumor Cells in Lung Adenocarcinoma

Background

Lung adenocarcinoma is one of the most commonly diagnosed malignancies worldwide. Macrophage plays crucial roles in the tumor microenvironment, but its autocrine network and communications with tumor cell are still unclear.

Methods

We acquired single-cell RNA sequencing (scRNA-seq) (n = 30) and bulk RNA sequencing (n = 1480) samples of lung adenocarcinoma patients from previous literatures and publicly available databases. Various cell subtypes were identified, including macrophages. Differentially expressed ligand-receptor gene pairs were obtained to explore cell-to-cell communications between macrophages and tumor cells. Furthermore, a machine-learning predictive model based on ligand-receptor interactions was built and validated.

Results

A total of 159,219 single cells (18,248 tumor cells and 29,520 macrophages) were selected in this study. We identified significantly correlated autocrine ligand-receptor gene pairs in tumor cells and macrophages, respectively. Furthermore, we explored the cell-to-cell communications between macrophages and tumor cells and detected significantly correlated ligand-receptor signaling pairs. We determined that some of the hub gene pairs were associated with patient prognosis and constructed a machine-learning model based on the intercellular interaction network.

Conclusion

We revealed significant cell-to-cell communications (both autocrine and paracrine network) within macrophages and tumor cells in lung adenocarcinoma. Hub genes with prognostic significance in the network were also identified.

Pre-operative Machine Learning for Heart Transplant Patients Bridged with Temporary Mechanical Circulatory Support

Background: Existing prediction models for post-transplant mortality in patients bridged to heart transplantation with temporary mechanical circulatory support (tMCS) perform poorly. A more reliable model would allow clinicians to provide better pre-operative risk assessment and develop more targeted therapies for high-risk patients. Methods: We identified adult patients in the United Network for Organ Sharing database undergoing isolated heart transplantation between 01/2009 and 12/2017 who were supported with tMCS at the time of transplant. We constructed a machine learning model using extreme gradient boosting (XGBoost) with a 70:30 train:test split to predict 1-year post-operative mortality. All pre-transplant variables available in the UNOS database were included to train the model. Shapley Additive Explanations was used to identify and interpret the most important features for XGBoost predictions. Results: A total of 1584 patients were included, with a median age of 56 (interquartile range: 46-62) and 74% male. Actual 1-year mortality was 12.1%. Out of 498 available variables, 43 were selected for the final model. The area under the receiver operator characteristics curve (AUC) for the XGBoost model was 0.71 (95% CI: 0.62-0.78). The most important variables predictive of 1-year mortality included recipient functional status, age, pulmonary capillary wedge pressure (PCWP), cardiac output, ECMO usage, and serum creatinine. Conclusions: An interpretable machine learning model trained on a large clinical database demonstrated good performance in predicting 1-year mortality for patients bridged to heart transplantation with tMCS. Machine learning may be used to enhance clinician judgement in the care of markedly high-risk transplant recipients.

Machine Learning to Predict Outcomes and Cost by Phase of Care After Coronary Artery Bypass Grafting

BACKGROUND

Machine learning may enhance prediction of outcomes after coronary artery bypass grafting (CABG). We sought to develop and validate a dynamic machine learning model to predict CABG outcomes at clinically relevant pre- and postoperative time points.

METHODS

The Society of Thoracic Surgeons (STS) registry data elements from 2086 isolated CABG patients were divided into training and testing datasets and input into Extreme Gradient Boosting decision-tree machine learning algorithms. Two prediction models were developed based on data from preoperative (80 parameters) and postoperative (125 parameters) phases of care. Outcomes included operative mortality, major morbidity or mortality, high cost, and 30-day readmission. Machine learning and STS model performance were assessed using accuracy and the area under the precision-recall curve (AUC-PR).

RESULTS

Preoperative machine learning models predicted mortality (accuracy, 98%; AUC-PR = 0.16; F1 = 0.24), major morbidity or mortality (accuracy, 75%; AUC-PR = 0.33; F1 = 0.42), high cost (accuracy, 83%; AUC-PR = 0.51; F1 = 0.52), and 30-day readmission (accuracy, 70%; AUC-PR = 0.47; F1 = 0.49) with high accuracy. Preoperative machine learning models performed similarly to the STS for prediction of mortality (STS AUC-PR = 0.11; P = .409) and outperformed STS for prediction of mortality or major morbidity (STS AUC-PR = 0.28; P < .001). Addition of intraoperative parameters further improved machine learning model performance for major morbidity or mortality (AUC-PR = 0.39; P < .01) and high cost (AUC-PR = 0.64; P < .01), with cross-clamp and bypass times emerging as important additive predictive parameters.

CONCLUSIONS

Machine learning can predict mortality, major morbidity, high cost, and readmission after isolated CABG. Prediction based on the phase of care allows for dynamic risk assessment through the hospital course, which may benefit quality assessment and clinical decision-making.

Parsimonious machine learning models to predict resource use in cardiac surgery across a statewide collaborative

Objective

Methods

Results

Conclusions

Machine learning algorithms for predicting mortality after coronary artery bypass grafting

Background

As the era of big data analytics unfolds, machine learning (ML) might be a promising tool for predicting clinical outcomes. This study aimed to evaluate the predictive ability of ML models for estimating mortality after coronary artery bypass grafting (CABG).

Materials and methods

Various baseline and follow-up features were obtained from the CABG data registry, established in 2005 at Tehran Heart Center. After selecting key variables using the random forest method, prediction models were developed using: Logistic Regression (LR), Support Vector Machine (SVM), Naïve Bayes (NB), K-Nearest Neighbors (KNN), Extreme Gradient Boosting (XGBoost), and Random Forest (RF) algorithms. Area Under the Curve (AUC) and other indices were used to assess the performance.

Results

A total of 16,850 patients with isolated CABG (mean age: 67.34 ± 9.67 years) were included. Among them, 16,620 had one-year follow-up, from which 468 died. Eleven features were chosen to train the models. Total ventilation hours and left ventricular ejection fraction were by far the most predictive factors of mortality. All the models had AUC &gt; 0.7 (acceptable performance) for 1-year mortality. Nonetheless, LR (AUC = 0.811) and XGBoost (AUC = 0.792) outperformed NB (AUC = 0.783), RF (AUC = 0.783), SVM (AUC = 0.738), and KNN (AUC = 0.715). The trend was similar for two-to-five-year mortality, with LR demonstrating the highest predictive ability.

Conclusion

Various ML models showed acceptable performance for estimating CABG mortality, with LR illustrating the highest prediction performance. These models can help clinicians make decisions according to the risk of mortality in patients undergoing CABG.

Machine learning algorithms to predict major bleeding after isolated coronary artery bypass grafting

Objectives

Postoperative major bleeding is a common problem in patients undergoing cardiac surgery and is associated with poor outcomes. We evaluated the performance of machine learning (ML) methods to predict postoperative major bleeding.

Methods

A total of 1,045 patients who underwent isolated coronary artery bypass graft surgery (CABG) were enrolled. Their datasets were assigned randomly to training (70%) or a testing set (30%). The primary outcome was major bleeding defined as the universal definition of perioperative bleeding (UDPB) classes 3–4. We constructed a reference logistic regression (LR) model using known predictors. We also developed several modern ML algorithms. In the test set, we compared the area under the receiver operating characteristic curves (AUCs) of these ML algorithms with the reference LR model results, and the TRUST and WILL-BLEED risk score. Calibration analysis was undertaken using the calibration belt method.

Results

The prevalence of postoperative major bleeding was 7.1% (74/1,045). For major bleeds, the conditional inference random forest (CIRF) model showed the highest AUC [0.831 (0.732–0.930)], and the stochastic gradient boosting (SGBT) and random forest models demonstrated the next best results [0.820 (0.742–0.899) and 0.810 (0.719–0.902)]. The AUCs of all ML models were higher than [0.629 (0.517–0.641) and 0.557 (0.449–0.665)], as achieved by TRUST and WILL-BLEED, respectively.

Conclusion

ML methods successfully predicted major bleeding after cardiac surgery, with greater performance compared with previous scoring models. Modern ML models may enhance the identification of high-risk major bleeding subpopulations.

Machine Learning Methods for Predicting Long-Term Mortality in Patients After Cardiac Surgery

Objective:

This study aims to construct and validate several machine learning (ML) algorithms to predict long-term mortality and identify risk factors in unselected patients post-cardiac surgery.

Methods

The Medical Information Mart for Intensive Care (MIMIC-III) database was used to perform a retrospective administrative database study. Candidate predictors consisted of the demographics, comorbidity, vital signs, laboratory test results, scoring systems, and treatment information on the first day of ICU admission. Four-year mortality was set as the study outcome. We used the ML methods of logistic regression (LR), artificial neural network (NNET), naïve bayes (NB), gradient boosting machine (GBM), adapting boosting (Ada), random forest (RF), bagged trees (BT), and eXtreme Gradient Boosting (XGB). The prognostic capacity and clinical utility of these ML models were compared using the area under the receiver operating characteristic curves (AUC), calibration curves, and decision curve analysis (DCA).

Results

Of 7,368 patients in MIMIC-III included in the final cohort, a total of 1,337 (18.15%) patients died during a 4-year follow-up. Among 65 variables extracted from the database, a total of 25 predictors were selected using recursive feature elimination and included in the subsequent analysis. The Ada model performed best among eight models in both discriminatory ability with the highest AUC of 0.801 and goodness of fit (visualized by calibration curve). Moreover, the DCA shows that the net benefit of the RF, Ada, and BT models surpassed that of other ML models for almost all threshold probability values. Additionally, through the Ada technique, we determined that red blood cell distribution width (RDW), blood urea nitrogen (BUN), SAPS II, anion gap (AG), age, urine output, chloride, creatinine, congestive heart failure, and SOFA were the Top 10 predictors in the feature importance rankings.

Conclusions

The Ada model performs best in predicting 4-year mortality after cardiac surgery among the eight ML models, which might have significant application in the development of early warning systems for patients following operations.

Development of machine learning models for mortality risk prediction after cardiac surgery

BACKGROUND

We developed machine learning models that combine preoperative and intraoperative risk factors to predict mortality after cardiac surgery.

METHODS

Machine learning involving random forest, neural network, support vector machine, and gradient boosting machine was developed and compared with the risk scores of EuroSCORE I and II, Society of Thoracic Surgeons (STS), as well as a logistic regression model. Clinical data were collected from patients undergoing adult cardiac surgery at the First Medical Centre of Chinese PLA General Hospital between December 2008 and December 2017. The primary outcome was post-operative mortality. Model performance was estimated using several metrics, including sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve (AUC). The visualization algorithm was implemented using Shapley's additive explanations.

RESULTS

A total of 5,443 patients were enrolled during the study period. The mean EuroSCORE II score was 3.7%, and the actual in-hospital mortality rate was 2.7%. For predicting operative mortality after cardiac surgery, the AUC scores were 0.87, 0.79, 0.81, and 0.82 for random forest, neural network, support vector machine, and gradient boosting machine, compared with 0.70, 0.73, 0.71, and 0.74 for EuroSCORE I and II, STS, and logistic regression model. Shapley's additive explanations analysis of random forest yielded the top-20 predictors and individual-level explanations for each prediction.

CONCLUSIONS

Machine learning models based on available clinical data may be superior to clinical scoring tools in predicting postoperative mortality in patients following cardiac surgery. Explanatory models show the potential to provide personalized risk profiles for individuals by accounting for the contribution of influencing factors. Additional prospective multicenter studies are warranted to confirm the clinical benefit of these machine learning-driven models.

Development and validation of a machine learning method to predict intraoperative red blood cell transfusions in cardiothoracic surgery

Accurately predicting red blood cell (RBC) transfusion requirements in cardiothoracic (CT) surgery could improve blood inventory management and be used as a surrogate marker for assessing hemorrhage risk preoperatively. We developed a machine learning (ML) method to predict intraoperative RBC transfusions in CT surgery. A detailed database containing time-stamped clinical variables for all CT surgeries from 5/2014-6/2019 at a single center (n = 2410) was used for model development. After random forest feature selection, surviving features were inputs for ML algorithms using five-fold cross-validation. The dataset was updated with 437 additional cases from 8/2019-8/2020 for validation. We developed and validated a hybrid ML method given the skewed nature of the dataset. Our Gaussian Process (GP) regression ML algorithm accurately predicted RBC transfusion amounts of 0 and 1-3 units (root mean square error, RMSE 0.117 and 1.705, respectively) and our GP classification ML algorithm accurately predicted 4 + RBC units transfused (area under the curve, AUC = 0.826). The final prediction is the regression result if classification predicted < 4 units transfused, or the classification result if 4 + units were predicted. We developed and validated an ML method to accurately predict intraoperative RBC transfusions in CT surgery using local data.

Machine learning methods for perioperative anesthetic management in cardiac surgery patients: a scoping review

Background

Machine learning (ML) is developing fast with promising prospects within medicine and already has several applications in perioperative care. We conducted a scoping review to examine the extent and potential limitations of ML implementation in perioperative anesthetic care, specifically in cardiac surgery patients.

Methods

We mapped the current literature by searching three databases: MEDLINE (Ovid), EMBASE (Ovid), and Cochrane Library. Articles were eligible if they reported on perioperative ML use in the field of cardiac surgery with relevance to anesthetic practices. Data on the applicability of ML and comparability to conventional statistical methods were extracted.

Results

Forty-six articles on ML relevant to the work of the anesthesiologist in cardiac surgery were identified. Three main categories emerged: (I) event and risk prediction, (II) hemodynamic monitoring, and (III) automation of echocardiography. Prediction models based on ML tend to behave similarly to conventional statistical methods. Using dynamic hemodynamic or ultrasound data in ML models, however, shifts the potential to promising results.

Conclusions

ML in cardiac surgery is increasingly used in perioperative anesthetic management. The majority is used for prediction purposes similar to conventional clinical scores. Remarkable ML model performances are achieved when using real-time dynamic parameters. However, beneficial clinical outcomes of ML integration have yet to be determined. Nonetheless, the first steps introducing ML in perioperative anesthetic care for cardiac surgery have been taken.

Supplementing Existing Societal Risk Models for Surgical Aortic Valve Replacement With Machine Learning for Improved Prediction

Background

This study evaluated the role of supplementing Society of Thoracic Surgeons (STS) risk models for surgical aortic valve replacement with machine learning (ML).

Methods and Results

Adults undergoing isolated surgical aortic valve replacement in the STS National Database between 2007 and 2017 were included. ML models for operative mortality and major morbidity were previously developed using extreme gradient boosting. Concordance and discordance in predicted risk between ML and STS models were defined using equal‐size tertile‐based thresholds of risk. Calibration metrics and discriminatory capability were compared between concordant and discordant patients. A total of 243 142 patients were included. Nearly all calibration metrics were improved in cases of concordance. Similarly, concordance indices improved substantially in cases of concordance for all models with the exception of deep sternal wound infection. The greatest improvements in concordant versus discordant cases were in renal failure: ML model (concordance index, 0.660 [95% CI, 0.632–0.687] discordant versus 0.808 [95% CI, 0.794–0.822] concordant) and STS model (concordance index, 0.573 [95% CI, 0.549–0.576] discordant versus 0.797 [95% CI, 0.782–0.811] concordant) (each P<0.001). Excluding deep sternal wound infection, the concordance indices ranged from 0.549 to 0.660 for discordant cases and 0.674 to 0.808 for concordant cases.

Conclusions

Supplementing ML models with existing STS models for surgical aortic valve replacement may have an important role in risk prediction and should be explored further. In particular, for the roughly 25% to 50% of patients demonstrating discordance in estimated risk between ML and STS, there appears to be a substantial decline in predictive performance suggesting vulnerability of the existing models in these patient subsets.

Comparison of risk scores for predicting outcomes after isolated tricuspid valve surgery

BACKGROUND

Risk models play important roles in stratification and decision-making towards cardiac surgery. Isolated tricuspid valve surgery is a high risk but increasingly performed the operation, however, the performance of risk models has not been externally evaluated in these patients. We compared the prognostic utility of contemporary risk scores for isolated tricuspid valve surgery.

METHODS

Consecutive patients undergoing isolated tricuspid valve surgery at Cleveland Clinic during 2004-2018 were evaluated in this cohort study. EuroSCORE II, Society of Thoracic Surgeon's tricuspid (STS-TVS) score, and the Model for End-stage Liver Disease (MELD) score were retrospectively calculated, and their performance for predicting operative mortality, postoperative complications, and mortality during follow-up was assessed.

RESULTS

Amongst 207 patients studied, the mean age was 54.1 ± 17.9 years, 116 (56.0%) were female, 92 (44.4%) had secondary tricuspid regurgitation, and 151 (72.9%) had a surgical repair. Mean EuroSCORE II, STS-TVS, and MELD scores were 6.3 ± 6.6%, 5.5 ± 6.2%, and 9.8 ± 4.7, respectively. C-statistics (95% confidence intervals) for operative mortality were 0.83 (0.74-0.93) for EuroSCORE II, 0.60 (0.45-0.75) for STS-TVS score, and 0.74 (0.58-0.89) for MELD score, while observed/expected ratios were 0.78 and 0.89 for the first two scores. All three scores were associated with mortality during follow-up and discriminated most postoperative complications.

CONCLUSION

EuroSCORE II was superior to STS-tricuspid score for isolated TVS risk assessment. Although surgical risk scores traditionally underestimated operative mortality after isolated tricuspid valve surgery, they did not in our cohort, reflecting the excellent surgical results. The simple MELD score performed similarly to the EuroSCORE II, especially for discriminating morbidities.

Machine learning models for mitral valve replacement: A comparative analysis with the Society of Thoracic Surgeons risk score

BACKGROUND

Current Society of Thoracic Surgeons (STS) risk models for predicting outcomes of mitral valve surgery (MVS) assume a linear and cumulative impact of variables. We evaluated postoperative MVS outcomes and designed mortality and morbidity risk calculators to supplement the STS risk score.

METHODS

Data from the STS Adult Cardiac Surgery Database for MVS was used from 2008 to 2017. The data included 383,550 procedures and 89 variables. Machine learning (ML) algorithms were employed to train models to predict postoperative outcomes for MVS patients. Each model's discrimination and calibration performance were validated using unseen data against the STS risk score.

RESULTS

Comprehensive mortality and morbidity risk assessment scores were derived from a training set of 287,662 observations. The area under the curve (AUC) for mortality ranged from 0.77 to 0.83, leading to a 3% increase in predictive accuracy compared to the STS score. Logistic Regression and eXtreme Gradient Boosting achieved the highest AUC for prolonged ventilation (0.82) and deep sternal wound infection (0.78 and 0.77) respectively. EXtreme Gradient Boosting performed the best with an AUC of 0.815 for renal failure. For permanent stroke prediction all models performed similarly with an AUC around 0.67. The ML models led to improved calibration performance for mortality, prolonged ventilation, and renal failure, especially in cases of reconstruction/repair and replacement surgery.

CONCLUSIONS

The proposed risk models complement existing STS models in predicting mortality, prolonged ventilation, and renal failure, allowing healthcare providers to more accurately assess a patient's risk of morbidity and mortality when undergoing MVS.

Machine learning in gastrointestinal surgery

Machine learning (ML) is a collection of algorithms allowing computers to learn directly from data without predetermined equations. It is used widely to analyze "big data". In gastrointestinal surgery, surgeons deal with various data such as clinical parameters, surgical videos, and pathological images, to stratify surgical risk, perform safe surgery and predict patient prognosis. In the current "big data" era, the accelerating accumulation of a large amount of data drives studies using ML algorithms. Three subfields of ML are supervised learning, unsupervised learning, and reinforcement learning. In this review, we summarize applications of ML to surgical practice in the preoperative, intraoperative, and postoperative phases of care. Prediction and stratification using ML is promising; however, the current overarching concern is the availability of ML models. Information systems that can manage "big data" and integrate ML models into electronic health records are essential to incorporate ML into daily practice. ML is fundamental technology to meaningfully process data that exceeds the capacity of the human mind to comprehend. The accelerating accumulation of a large amount of data is changing the nature of surgical practice fundamentally. Artificial intelligence (AI), represented by ML, is being incorporated into daily surgical practice.

Machine learning: principles and applications for thoracic surgery

OBJECTIVES

Machine learning (ML) has experienced a revolutionary decade with advances across many disciplines. We seek to understand how recent advances in ML are going to specifically influence the practice of surgery in the future with a particular focus on thoracic surgery.

METHODS

Review of relevant literature in both technical and clinical domains.

RESULTS

ML is a revolutionary technology that promises to change the way that surgery is practiced in the near future. Spurred by an advance in computing power and the volume of data produced in healthcare, ML has shown remarkable ability to master tasks that had once been reserved for physicians. Supervised learning, unsupervised learning and reinforcement learning are all important techniques that can be leveraged to improve care. Five key applications of ML to cardiac surgery include diagnostics, surgical skill assessment, postoperative prognostication, augmenting intraoperative performance and accelerating translational research. Some key limitations of ML include lack of interpretability, low quality and volumes of relevant clinical data, ethical limitations and difficulties with clinical implementation.

CONCLUSIONS

In the future, the practice of cardiac surgery will be greatly augmented by ML technologies, ultimately leading to improved surgical performance and better patient outcomes.

The cardiac surgeon's guide to artificial intelligence

PURPOSE OF REVIEW

Artificial intelligence is the ability for machines to perform intelligent tasks. Artificial intelligence is already penetrating many aspects of medicine including cardiac surgery. Here, we offer a platform introduction to artificial intelligence for cardiac surgeons to understand the implementations of this transformative tool.

RECENT FINDINGS

Artificial intelligence has contributed greatly to the automation of cardiac imaging, including echocardiography, cardiac computed tomography, cardiac MRI and most recently, in radiomics. There are also several artificial intelligence based clinical prediction tools that predict complex outcomes after cardiac surgery. Waveform analysis, specifically, automated electrocardiogram analysis, has seen significant strides with promise in wearables and remote monitoring. Experimentally, artificial intelligence has also entered the operating room in the form of augmented reality and automated robotic surgery.

SUMMARY

Artificial intelligence has many potential exciting applications in cardiac surgery. It can streamline physician workload and help make medicine more human again by placing the physician back at the bedside. Here, we offer cardiac surgeons an introduction to this transformative tool so that they may actively participate in creating clinically relevant implementations to improve our practice.

Using machine learning to improve survival prediction after heart transplantation

BACKGROUND

This study investigates the use of modern machine learning (ML) techniques to improve prediction of survival after orthotopic heart transplantation (OHT).

METHODS

Retrospective study of adult patients undergoing primary, isolated OHT between 2000 and 2019 as identified in the United Network for Organ Sharing (UNOS) registry. The primary outcome was 1-year post-transplant survival. Patients were randomly divided into training (80%) and validation (20%) sets. Dimensionality reduction and data re-sampling were employed during training. Multiple machine learning algorithms were combined into a final ensemble ML model. The discriminatory capability was assessed using the area under receiver-operating-characteristic curve (AUROC), net reclassification index (NRI), and decision curve analysis (DCA).

RESULTS

A total of 33,657 OHT patients were evaluated. One-year mortality was 11% (n = 3738). In the validation cohort, the AUROC of singular logistic regression was 0.649 (95% CI, 0.628-0.670) compared to 0.691 (95% CI, 0.671-0.711) with random forest, 0.691 (95% CI, 0.671-0.712) with deep neural network, and 0.653 (95% CI, 0.632-0.674) with Adaboost. A final ensemble ML model was created that demonstrated the greatest improvement in AUROC: 0.764 (95% CI, 0.745-0.782) (p < .001). The ensemble ML model improved predictive performance by 72.9% ±3.8% (p < .001) as assessed by NRI compared to logistic regression. DCA showed the final ensemble method improved risk prediction across the entire spectrum of predicted risk as compared to all other models (p < .001).

CONCLUSIONS

Modern ML techniques can improve risk prediction in OHT compared to traditional approaches. This may have important implications in patient selection, programmatic evaluation, allocation policy, and patient counseling and prognostication.

A review of machine learning for cardiology

This paper reviews recent cardiology literature and reports how Artificial Intelligence Tools (specifically, Machine Learning techniques) are being used by physicians in the field. Each technique is introduced with enough details to allow the understanding of how it works and its intent, but without delving into details that do not add immediate benefits and require expertise in the field. We specifically focus on the principal Machine Learning based risk scores used in cardiovascular research. After introducing them and summarizing their assumptions and biases, we discuss their merits and shortcomings. We report on how frequently they are adopted in the field and suggest why this is the case based on our expertise in Machine Learning. We complete the analysis by reviewing how corresponding statistical approaches compare with them. Finally, we discuss the main open issues in applying Machine Learning tools to cardiology tasks, also drafting possible future directions. Despite the growing interest in these tools, we argue that there are many still underutilized techniques: while Neural Networks are slowly being incorporated in cardiovascular research, other important techniques such as Semi-Supervised Learning and Federated Learning are still underutilized. The former would allow practitioners to harness the information contained in large datasets that are only partially labeled, while the latter would foster collaboration between institutions allowing building larger and better models.

Can machine learning improve mortality prediction following cardiac surgery?

OBJECTIVES

Interest in the clinical usefulness of machine learning for risk prediction has bloomed recently. Cardiac surgery patients are at high risk of complications and therefore presurgical risk assessment is of crucial relevance. We aimed to compare the performance of machine learning algorithms over traditional logistic regression (LR) model to predict in-hospital mortality following cardiac surgery.

METHODS

A single-centre data set of prospectively collected information from patients undergoing adult cardiac surgery from 1996 to 2017 was split into 70% training set and 30% testing set. Prediction models were developed using neural network, random forest, naive Bayes and retrained LR based on features included in the EuroSCORE. Discrimination was assessed using area under the receiver operating characteristic curve, and calibration analysis was undertaken using the calibration belt method. Model calibration drift was assessed by comparing Goodness of fit χ2 statistics observed in 2 equal bins from the testing sample ordered by procedure date.

RESULTS

A total of 28 761 cardiac procedures were performed during the study period. The in-hospital mortality rate was 2.7%. Retrained LR [area under the receiver operating characteristic curve 0.80; 95% confidence interval (CI) 0.77-0.83] and random forest model (0.80; 95% CI 0.76-0.83) showed the best discrimination. All models showed significant miscalibration. Retrained LR proved to have the weakest calibration drift.

CONCLUSIONS

Our findings do not support the hypothesis that machine learning methods provide advantage over LR model in predicting operative mortality after cardiac surgery.

An interpretable machine learning prognostic system for locoregionally advanced nasopharyngeal carcinoma based on tumor burden features

OBJECTIVES

We aimed to build a survival system by combining a highly-accurate machine learning (ML) model with explainable artificial intelligence (AI) techniques to predict distant metastasis in locoregionally advanced nasopharyngeal carcinoma (NPC) patients using magnetic resonance imaging (MRI)-based tumor burden features.

MATERIALS AND METHODS

1643 patients from three hospitals were enrolled according to set criteria. We employed ML to develop a survival model based on tumor burden signatures and all clinical factors. Shapley Additive exPlanations (SHAP) was utilized to explain prediction results and interpret the complex non-linear relationship among features and distant metastasis. We also constructed other models based on routinely used cancer stages, Epstein-Barr virus (EBV) DNA, or other clinical features for comparison. Concordance index (C-index), receiver operating curve (ROC) analysis and decision curve analysis (DCA) were executed to assess the effectiveness of the models.

RESULTS

Our proposed system consistently demonstrated promising performance across independent cohorts. The concordance indexes were 0.773, 0.766 and 0.760 in the training, internal validation and external validation sets. SHAP provided personalized protective and risk factors for each NPC patient and uncovered some novel non-linear relationships between features and distant metastasis. Furthermore, high-risk patients who received induction chemotherapy (ICT) and concurrent chemoradiotherapy (CCRT) had better 5-year distant metastasis-free survival (DMFS) than those who only received CCRT, whereas ICT + CCRT and CCRT had similar DMFS in low-risk patients.

CONCLUSIONS

The interpretable machine learning system demonstrated superior performance in predicting metastasis in locoregionally advanced NPC. High-risk patients might benefit from ICT.

Current Therapeutic Options in Aortic Stenosis

Aortic stenosis is the most common valvular disease requiring valve replacement. Valve replacement therapies have undergone progressive evolution since the 1960s. Over the last 20 years, transcatheter aortic valve replacement has radically transformed the care of aortic stenosis, such that it is now the treatment of choice for many, particularly elderly, patients. This review provides an overview of the pathophysiology, presentation, diagnosis, indications for intervention, and current therapeutic options for aortic stenosis.

Machine Learning for Prediction and Risk Stratification of Lupus Nephritis Renal Flare

BACKGROUND

Renal flare of lupus nephritis (LN) is strongly associated with poor kidney outcomes, and predicting renal flare and stratifying its risk are important for clinical decision-making and individualized management to reduce LN flare.

METHODS

We randomly divided 1,694 patients with biopsy-proven LN, who had achieved remission after treatment, into a derivation cohort (n = 1,186) and an internal validation cohort (n = 508), at a ratio of 7:3. The risk of renal flare 5 years after remission was predicted using an eXtreme Gradient Boosting (XGBoost) method model, developed from 59 variables, including demographic, clinical, immunological, pathological, and therapeutic characteristics. A simplified risk score prediction model (SRSPM) was developed from important variables selected by XGBoost model using stepwise Cox regression for practical convenience.

RESULTS

The 5-year relapse rates were 39.5% and 38.2% in the derivation and internal validation cohorts, respectively. Both the XGBoost model and the SRSPM had good predictive performance, with a C-index of 0.819 (95% confidence interval [CI]: 0.774-0.857) and 0.746 (95% CI: 0.697-0.795), respectively, in the validation cohort. The SRSPM comprised 6 variables, including partial remission and endocapillary hypercellularity at baseline, age, serum Alb, anti-dsDNA, and serum complement C3 at the point of remission. Using Kaplan-Meier analysis, the SRSPM identified significant risk stratification for renal flares (p < 0.001).

CONCLUSIONS

Renal flare of LN can be readily predicted using the XGBoost model and the SRSPM, and the SRSPM can also stratify flare risk. Both models are useful for clinical decision-making and individualized management in LN.

Cardiac Operative Risk in Latin America: A Comparison of Machine Learning Models vs EuroSCORE-II

BACKGROUND

Machine learning is a useful tool for predicting medical outcomes. This study aimed to develop a machine learning-based preoperative score to predict cardiac surgical operative mortality.

METHODS

We developed various models to predict cardiac operative mortality using machine learning techniques and compared each model to European System for Cardiac Operative Risk Evaluation-II (EuroSCORE-II) using the area under the receiver operating characteristic (ROC) and precision-recall (PR) curves (ROC AUC and PR AUC) as performance metrics. The model calibration in our population was also reported with all models and in high-risk groups for gradient boosting and EuroSCORE-II. This study is a retrospective cohort based on a prospectively collected database from July 2008 to April 2018 from a single cardiac surgical center in Bogotá, Colombia.

RESULTS

Model comparison consisted of hold-out validation: 80% of the data were used for model training, and the remaining 20% of the data were used to test each model and EuroSCORE-II. Operative mortality was 6.45% in the entire database and 6.59% in the test set. The performance metrics for the best machine learning model, gradient boosting (ROC: 0.755; PR: 0.292), were higher than those of EuroSCORE-II (ROC: 0.716, PR: 0.179), with a P value of .318 for the AUC of the ROC and .137 for the AUC of the PR.

CONCLUSIONS

The gradient boosting model was more precise than EuroSCORE-II in predicting mortality in our population based on ROC and PR analyses, although the difference was not statistically significant.

Using machine learning to improve risk prediction in durable left ventricular assist devices

Risk models have historically displayed only moderate predictive performance in estimating mortality risk in left ventricular assist device therapy. This study evaluated whether machine learning can improve risk prediction for left ventricular assist devices. Primary durable left ventricular assist devices reported in the Interagency Registry for Mechanically Assisted Circulatory Support between March 1, 2006 and December 31, 2016 were included. The study cohort was randomly divided 3:1 into training and testing sets. Logistic regression and machine learning models (extreme gradient boosting) were created in the training set for 90-day and 1-year mortality and their performance was evaluated after bootstrapping with 1000 replications in the testing set. Differences in model performance were also evaluated in cases of concordance versus discordance in predicted risk between logistic regression and extreme gradient boosting as defined by equal size patient tertiles. A total of 16,120 patients were included. Calibration metrics were comparable between logistic regression and extreme gradient boosting. C-index was improved with extreme gradient boosting (90-day: 0.707 [0.683–0.730] versus 0.740 [0.717–0.762] and 1-year: 0.691 [0.673–0.710] versus 0.714 [0.695–0.734]; each p<0.001). Net reclassification index analysis similarly demonstrated an improvement of 48.8% and 36.9% for 90-day and 1-year mortality, respectively, with extreme gradient boosting (each p<0.001). Concordance in predicted risk between logistic regression and extreme gradient boosting resulted in substantially improved c-index for both logistic regression and extreme gradient boosting (90-day logistic regression 0.536 versus 0.752, 1-year logistic regression 0.555 versus 0.726, 90-day extreme gradient boosting 0.623 versus 0.772, 1-year extreme gradient boosting 0.613 versus 0.742, each p<0.001). These results demonstrate that machine learning can improve risk model performance for durable left ventricular assist devices, both independently and as an adjunct to logistic regression.

Machine Learning Compared With Conventional Statistical Models for Predicting Myocardial Infarction Readmission and Mortality: A Systematic Review

BACKGROUND

Machine learning (ML) methods are increasingly used in addition to conventional statistical modelling (CSM) for predicting readmission and mortality in patients with myocardial infarction (MI). However, the two approaches have not been systematically compared across studies of prognosis in patients with MI.

METHODS

Following PRISMA guidelines, we systematically reviewed the literature via Medline, EPub, Cochrane Central, Embase, Inspec, ACM Digital Library, and Web of Science. Eligible studies included primary research articles published from January 2000 to March 2020, comparing ML and CSM for prognostication after MI.

RESULTS

Of 7,348 articles, 112 underwent full-text review, with the final set composed of 24 articles representing 374,365 patients. ML methods included artificial neural networks (n = 12 studies), random forests (n = 11), decision trees (n = 8), support vector machines (n = 8), and Bayesian techniques (n = 7). CSM included logistic regression (n = 19 studies), existing CSM-derived risk scores (n = 12), and Cox regression (n = 2). Thirteen of 19 studies examining mortality reported higher C-indexes with the use of ML compared with CSM. One study examined readmissions at 2 different time points, with C-indexes that were higher for ML than CSM. Across all studies, a total of 29 comparisons were performed, but the majority (n = 26, 90%) found small (< 0.05) absolute differences in the C-index between ML and CSM. With the use of a modified CHARMS checklist, sources of bias were identifiable in the majority of studies, and only 2 were externally validated.

CONCLUSION

Although ML algorithms tended to have higher C-indexes than CSM for predicting death or readmission after MI, these studies exhibited threats to internal validity and were often unvalidated. Further comparisons are needed, with adherence to clinical quality standards for prognosis research. (Trial registration: PROSPERO CRD42019134896).

Machine learning, artificial intelligence and mechanical circulatory support: A primer for clinicians

Artificial intelligence (AI) refers to the ability of machines to perform intelligent tasks, and machine learning (ML) is a subset of AI describing the ability of machines to learn independently and make accurate predictions. The application of AI combined with "big data" from the electronic health records, is poised to impact how we take care of patients. In recent years, an expanding body of literature has been published using ML in cardiovascular health care, including mechanical circulatory support (MCS). This primer article provides an overview for clinicians on relevant concepts of ML and AI, reviews predictive modeling concepts in ML and provides contextual reference to how AI is being adapted in the field of MCS. Lastly, it explains how these methods could be incorporated in the practices of medicine to improve patient outcomes.

[Machine Learning Methods for Prediction of Hospital Mortality in Patients with Coronary Heart Disease after Coronary Artery Bypass Grafting]

Aim      To compare the accuracy of predicting an in-hospital fatal outcome for models based on current machine-learning technologies in patients with ischemic heart disease (IHD) after coronary bypass (CB) surgery.Material and methods  A retrospective analysis of 866 electronic medical records was performed for patients (685 men and 181 women) who have had a CB surgery for IHD in 2008-2018. Results of clinical, laboratory, and instrumental evaluations obtained prior to the CB surgery were analyzed. Patients were divided into two groups: group 1 included 35 (4 %) patients who died within the first 20 days of CB, and group 2 consisted of 831 (96 %) patients with a beneficial outcome of the surgery. Predictors of the in-hospital fatal outcome were identified by a multistep selection procedure with analysis of statistical hypotheses and calculation of weight coefficients. For construction of models and verification of predictors, machine-learning methods were used, including the multifactorial logistic regression (LR), random forest (RF), and artificial neural networks (ANN). Model accuracy was evaluated by three metrics: area under the ROC curve (AUC), sensitivity, and specificity. Cross validation of the models was performed on test samples, and the control validation was performed on a cohort of patients with IHD after CB, whose data were not used in development of the models.Results The following 7 risk factors for in-hospital fatal outcome with the greatest predictive potential were isolated from the EuroSCORE II scale: ejection fraction (EF) &lt;30 %, EF 30-50 %, age of patients with recent MI, damage of peripheral arterial circulation, urgency of CB, functional class III-IV chronic heart failure, and 5 additional predictors, including heart rate, systolic blood pressure, presence of aortic stenosis, posterior left ventricular (LV) wall relative thickness index (RTI), and LV relative mass index (LVRMI). The models developed by the authors using LR, RF and ANN methods had higher AUC values and sensitivity compared to the classical EuroSCORE II scale. The ANN models including the RTI and LVRMI predictors demonstrated a maximum level of prognostic accuracy, which was illustrated by values of the quality metrics, AUC 93 %, sensitivity 90 %, and specificity 96 %. The predictive robustness of the models was confirmed by results of the control validation.Conclusion      The use of current machine-learning technologies allowed developing a novel algorithm for selection of predictors and highly accurate models for predicting an in-hospital fatal outcome after CB.

A Machine Learning-Based Prediction of Hospital Mortality in Patients With Postoperative Sepsis

Introduction: The incidence of postoperative sepsis is continually increased, while few studies have specifically focused on the risk factors and clinical outcomes associated with the development of sepsis after surgical procedures. The present study aimed to develop a mathematical model for predicting the in-hospital mortality among patients with postoperative sepsis. Materials and Methods: Surgical patients in Medical Information Mart for Intensive Care (MIMIC-III) database who simultaneously fulfilled Sepsis 3.0 and Agency for Healthcare Research and Quality (AHRQ) criteria at ICU admission were incorporated. We employed both extreme gradient boosting (XGBoost) and stepwise logistic regression model to predict the in-hospital mortality among patients with postoperative sepsis. Consequently, the model performance was assessed from the angles of discrimination and calibration. Results: We included 3,713 patients who fulfilled our inclusion criteria, in which 397 (10.7%) patients died during hospitalization, and 3,316 (89.3%) patients survived through discharge. Fluid-electrolyte disturbance, coagulopathy, renal replacement therapy (RRT), urine output, and cardiovascular surgery were important features related to the in-hospital mortality. The XGBoost model had a better performance in both discriminatory ability (c-statistics, 0.835 vs. 0.737 and 0.621, respectively; AUPRC, 0.418 vs. 0.280 and 0.237, respectively) and goodness of fit (visualized by calibration curve) compared to the stepwise logistic regression model and baseline model. Conclusion: XGBoost model has a better performance in predicting hospital mortality among patients with postoperative sepsis in comparison to the stepwise logistic regression model. Machine learning-based algorithm might have significant application in the development of early warning system for septic patients following major operations.

Development and external-validation of a nomogram for predicting the survival of hospitalised HIV/AIDS patients based on a large study cohort in western China

The aim of this study was to develop and externally validate a simple-to-use nomogram for predicting the survival of hospitalised human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS) patients (hospitalised person living with HIV/AIDS (PLWHAs)). Hospitalised PLWHAs (n = 3724) between January 2012 and December 2014 were enrolled in the training cohort. HIV-infected inpatients (n = 1987) admitted in 2015 were included as the external-validation cohort. The least absolute shrinkage and selection operator method was used to perform data dimension reduction and select the optimal predictors. The nomogram incorporated 11 independent predictors, including occupation, antiretroviral therapy, pneumonia, tuberculosis, Talaromyces marneffei, hypertension, septicemia, anaemia, respiratory failure, hypoproteinemia and electrolyte disturbances. The Likelihood χ² statistic of the model was 516.30 (P = 0.000). Integrated Brier Score was 0.076 and Brier scores of the nomogram at the 10-day and 20-day time points were 0.046 and 0.071, respectively. The area under the curves for receiver operating characteristic were 0.819 and 0.828, and precision-recall curves were 0.242 and 0.378 at two time points. Calibration plots and decision curve analysis in the two sets showed good performance and a high net benefit of nomogram. In conclusion, the nomogram developed in the current study has relatively high calibration and is clinically useful. It provides a convenient and useful tool for timely clinical decision-making and the risk management of hospitalised PLWHAs.