Machine learning improves mortality risk prediction after cardiac surgery: Systematic review and meta-analysis

Machine learning prediction of one-year mortality after percutaneous coronary intervention in acute coronary syndrome patients

BACKGROUND

Machine learning (ML) models have the potential to accurately predict outcomes and offer novel insights into inter-variable correlations. In this study, we aimed to design ML models for the prediction of 1-year mortality after percutaneous coronary intervention (PCI) in patients with acute coronary syndrome.

METHODS

This study was performed on 13,682 patients at Tehran Heart Center from 2015 to 2021. Patients were split into 70:30 for testing and training. Four ML models were designed: a traditional Logistic Regression (LR) model, Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Ada Boost models. The importance of features was calculated using the RF feature selector and SHAP based on the XGBoost model. The Area Under the Receiver Operating Characteristic Curve (AUC-ROC) for the prediction on the testing dataset was the main measure of the model's performance.

RESULTS

From a total of 9,073 patients with >1-year follow-up, 340 participants died. Higher age and higher rates of comorbidities were observed in these patients. Body mass index and lipid profile demonstrated a U-shaped correlation with the outcome. Among the models, RF had the best discrimination (AUC 0.866), while the highest sensitivity (80.9%) and specificity (88.3%) were for LR and XGBoost models, respectively. All models had AUCs of >0.8.

CONCLUSION

ML models can predict 1-year mortality after PCI with high performance. A classic LR statistical approach showed comparable results with other ML models. The individual-level assessment of inter-variable correlations provided new insights into the non-linear contribution of risk factors to post-PCI mortality.

Performance Drift in Machine Learning Models for Cardiac Surgery Risk Prediction: Retrospective Analysis

Background

The Society of Thoracic Surgeons and European System for Cardiac Operative Risk Evaluation (EuroSCORE) II risk scores are the most commonly used risk prediction models for in-hospital mortality after adult cardiac surgery. However, they are prone to miscalibration over time and poor generalization across data sets; thus, their use remains controversial. Despite increased interest, a gap in understanding the effect of data set drift on the performance of machine learning (ML) over time remains a barrier to its wider use in clinical practice. Data set drift occurs when an ML system underperforms because of a mismatch between the data it was developed from and the data on which it is deployed.

Objective

In this study, we analyzed the extent of performance drift using models built on a large UK cardiac surgery database. The objectives were to (1) rank and assess the extent of performance drift in cardiac surgery risk ML models over time and (2) investigate any potential influence of data set drift and variable importance drift on performance drift.

Methods

We conducted a retrospective analysis of prospectively, routinely gathered data on adult patients undergoing cardiac surgery in the United Kingdom between 2012 and 2019. We temporally split the data 70:30 into a training and validation set and a holdout set. Five novel ML mortality prediction models were developed and assessed, along with EuroSCORE II, for relationships between and within variable importance drift, performance drift, and actual data set drift. Performance was assessed using a consensus metric.

Results

A total of 227,087 adults underwent cardiac surgery during the study period, with a mortality rate of 2.76% (n=6258). There was strong evidence of a decrease in overall performance across all models (P<.0001). Extreme gradient boosting (clinical effectiveness metric [CEM] 0.728, 95% CI 0.728-0.729) and random forest (CEM 0.727, 95% CI 0.727-0.728) were the overall best-performing models, both temporally and nontemporally. EuroSCORE II performed the worst across all comparisons. Sharp changes in variable importance and data set drift from October to December 2017, from June to July 2018, and from December 2018 to February 2019 mirrored the effects of performance decrease across models.

Conclusions

All models show a decrease in at least 3 of the 5 individual metrics. CEM and variable importance drift detection demonstrate the limitation of logistic regression methods used for cardiac surgery risk prediction and the effects of data set drift. Future work will be required to determine the interplay between ML models and whether ensemble models could improve on their respective performance advantages.

Machine learning improves prediction of postoperative outcomes after gastrointestinal surgery: a systematic review and meta-analysis

BACKGROUND

Machine learning (ML) approaches have become increasingly popular in predicting surgical outcomes. However, it is unknown whether they are superior to traditional statistical methods such as logistic regression (LR). This study aimed to perform a systematic review and meta-analysis to compare the performance of ML vs LR models in predicting postoperative outcomes for patients undergoing gastrointestinal (GI) surgery.

METHODS

A systematic search of Embase, MEDLINE, Cochrane, Web of Science, and Google Scholar was performed through December 2022. The primary outcome was the discriminatory performance of ML vs LR models as measured by the area under the receiver operating characteristic curve (AUC). A meta-analysis was then performed using a random effects model.

RESULTS

A total of 62 LR models and 143 ML models were included across 38 studies. On average, the best-performing ML models had a significantly higher AUC than the LR models (ΔAUC, 0.07; 95% CI, 0.04-0.09; P < .001). Similarly, on average, the best-performing ML models had a significantly higher logit (AUC) than the LR models (Δlogit [AUC], 0.41; 95% CI, 0.23-0.58; P < .001). Approximately half of studies (44%) were found to have a low risk of bias. Upon a subset analysis of only low-risk studies, the difference in logit (AUC) remained significant (ML vs LR, Δlogit [AUC], 0.40; 95% CI, 0.14-0.66; P = .009).

CONCLUSION

We found a significant improvement in discriminatory ability when using ML over LR algorithms in predicting postoperative outcomes for patients undergoing GI surgery. Subsequent efforts should establish standardized protocols for both developing and reporting studies using ML models and explore the practical implementation of these models.

Development and validation of algorithms to predict left ventricular ejection fraction class from healthcare claims data

AIMS

The use of large medical or healthcare claims databases is very useful for population-based studies on the burden of heart failure (HF). Clinical characteristics and management of HF patients differ according to categories of left ventricular ejection fraction (LVEF), but this information is often missing in such databases. We aimed to develop and validate algorithms to identify LVEF in healthcare databases where the information is lacking.

METHODS AND RESULTS

Algorithms were built by machine learning with a random forest approach. Algorithms were trained and reinforced using the French national claims database [Système National des Données de Santé (SNDS)] and a French HF registry. Variables were age, gender, and comorbidities, which could be identified by medico-administrative code-based proxies, Anatomical Therapeutic Chemical codes for drug delivery, International Classification of Diseases (Tenth Revision) coding for hospitalizations, and administrative codes for any other type of reimbursed care. The algorithms were validated by cross-validation and against a subset of the SNDS that includes LVEF information. The areas under the receiver operating characteristic curve were 0.84 for the algorithm identifying LVEF ≤ 40% and 0.79 for the algorithms identifying LVEF < 50% and ≥50%. For LVEF ≤ 40%, the reinforced algorithm identified 50% of patients in the validation dataset with a positive predictive value of 0.88 and a specificity of 0.96. The most important predictive variables were delivery of HF medication, sex, age, hospitalization, and testing for natriuretic peptides with different orders of positive or negative importance according to the LVEF category.

CONCLUSIONS

The algorithms identify reduced or preserved LVEF in HF patients within a nationwide healthcare claims database with high positive predictive value and low rates of false positives.

The State of Artificial Intelligence in Pediatric Surgery: A Systematic Review

BACKGROUND

Artificial intelligence (AI) has been recently shown to improve clinical workflows and outcomes - yet its potential in pediatric surgery remains largely unexplored. This systematic review details the use of AI in pediatric surgery.

METHODS

Nine medical databases were searched from inception until January 2023, identifying articles focused on AI in pediatric surgery. Two authors reviewed full texts of eligible articles. Studies were included if they were original investigations on the development, validation, or clinical application of AI models for pediatric health conditions primarily managed surgically. Studies were excluded if they were not peer-reviewed, were review articles, editorials, commentaries, or case reports, did not focus on pediatric surgical conditions, or did not employ at least one AI model. Extracted data included study characteristics, clinical specialty, AI method and algorithm type, AI model (algorithm) role and performance metrics, key results, interpretability, validation, and risk of bias using PROBAST and QUADAS-2.

RESULTS

Authors screened 8178 articles and included 112. Half of the studies (50%) reported predictive models (for adverse events [25%], surgical outcomes [16%] and survival [9%]), followed by diagnostic (29%) and decision support models (21%). Neural networks (44%) and ensemble learners (36%) were the most commonly used AI methods across application domains. The main pediatric surgical subspecialties represented across all models were general surgery (31%) and neurosurgery (25%). Forty-four percent of models were interpretable, and 6% were both interpretable and externally validated. Forty percent of models had a high risk of bias, and concerns over applicability were identified in 7%.

CONCLUSIONS

While AI has wide potential clinical applications in pediatric surgery, very few published AI algorithms were externally validated, interpretable, and unbiased. Future research needs to focus on developing AI models which are prospectively validated and ultimately integrated into clinical workflows.

LEVEL OF EVIDENCE

2A.

A Machine Learning Algorithm using Clinical and Demographic Data for All-Cause Preterm Birth Prediction

OBJECTIVE

 Preterm birth remains the predominant cause of perinatal mortality throughout the United States and the world, with well-documented racial and socioeconomic disparities. To develop and validate a predictive algorithm for all-cause preterm birth using clinical, demographic, and laboratory data using machine learning.

STUDY DESIGN

 We performed a cohort study of pregnant individuals delivering at a single institution using prospectively collected information on clinical conditions, patient demographics, laboratory data, and health care utilization. Our primary outcome was all-cause preterm birth before 37 weeks. The dataset was randomly divided into a derivation cohort (70%) and a separate validation cohort (30%). Predictor variables were selected amongst 33 that had been previously identified in the literature (directed machine learning). In the derivation cohort, both statistical (logistic regression) and machine learning (XG-Boost) models were used to derive the best fit (C-Statistic) and then validated using the validation cohort. We measured model discrimination with the C-Statistic and assessed the model performance and calibration of the model to determine whether the model provided clinical decision-making benefits.

RESULTS

 The cohort includes a total of 12,440 deliveries among 12,071 individuals. Preterm birth occurred in 2,037 births (16.4%). The derivation cohort consisted of 8,708 (70%) and the validation cohort consisted of 3,732 (30%). XG-Boost was chosen due to the robustness of the model and the ability to deal with missing data and collinearity between predictor variables. The top five predictor variables identified as drivers of preterm birth, by feature importance metric, were multiple gestation, number of emergency department visits in the year prior to the index pregnancy, initial unknown body mass index, gravidity, and prior preterm delivery. Test performance characteristics were similar between the two populations (derivation cohort area under the curve [AUC] = 0.70 vs. validation cohort AUC = 0.63).

CONCLUSION

 Clinical, demographic, and laboratory information can be useful to predict all-cause preterm birth with moderate precision.

KEY POINTS

· Machine learning can be used to create models to predict preterm birth.. · In our model, all-cause preterm birth can be predicted with moderate precision.. · Clinical, demographic, and laboratory information can be useful to predict all-cause preterm birth..

Personalizing patient risk of a life-altering event: An application of machine learning to hemiarch surgery

OBJECTIVE

The study objective was to assess a machine learning model's ability to predict the occurrence of life-altering events in hemiarch surgery and determine contributing patient characteristics and intraoperative factors.

METHODS

In total, 602 patients who underwent hemiarch replacement at a high-volume aortic center from 2009 to 2022 were included. Patients were randomly divided into training (80%) and testing (20%) sets with various eXtreme gradient boosting candidate models constructed to predict the risk of experiencing life-altering events, including stroke, mortality, or new renal replacement therapy requirement. A total of 64 input parameters from the index hospitalization were identified, including 24 demographic characteristics as well as 8 preoperative and 32 intraoperative variables. A SHapley Additive exPlanation beeswarm plot was generated to identify and interpret the impact of individual features on the predictions of the final model.

RESULTS

A life-altering event was noted in 15% (90/602) of patients who underwent hemiarch replacement, including urgent/emergency cases and dissections. The final eXtreme Gradient Boosting model demonstrated a cross-validation accuracy of 88% on the testing set and was well calibrated as evidenced by a low Brier score of 0.12. The best performing model achieved an area under the receiver operating characteristic curve of 0.76 and an area under the precision recall curve of 0.55. The SHapley Additive exPlanation beeswarm plot provided insights into key features that significantly influenced model prediction.

CONCLUSIONS

Machine learning demonstrated superior accuracy in predicting hemiarch patients who would experience a life-altering event. This model may help to guide patients and clinicians in stratifying risk on an individual basis, which may in turn influence clinical decision-making.

A systematic review of prediction models on arteriovenous fistula: Risk scores and machine learning approaches

OBJECTIVE

Failure-to-mature and early stenosis remains the Achille's heel of hemodialysis arteriovenous fistula (AVF) creation. The maturation and patency of an AVF can be influenced by a variety of demographic, comorbidity, and anatomical factors. This study aims to review the prediction models of AVF maturation and patency with various risk scores and machine learning models.

DATA SOURCES AND REVIEW METHODS

Literature search was performed on PubMed, Scopus, and Embase to identify eligible articles. The quality of the studies was assessed using the Prediction model Risk Of Bias ASsessment (PROBAST) Tool. The performance (discrimination and calibration) of the included studies were extracted.

RESULTS

Fourteen studies (seven studies used risk score approaches; seven studies used machine learning approaches) were included in the review. Among them, 12 studies were rated as high or unclear "risk of bias." Six studies were rated as high concern or unclear for "applicability." C-statistics (Model discrimination metric) was reported in five studies using risk score approach (0.70-0.886) and three utilized machine learning methods (0.80-0.85). Model calibration was reported in three studies. Failure-to-mature risk score developed by one of the studies has been externally validated in three different patient populations, however the model discrimination degraded significantly (C-statistics: 0.519-0.53).

CONCLUSION

The performance of existing predictive models for AVF maturation/patency is underreported. They showed satisfactory performance in their own study population. However, there was high risk of bias in methodology used to build some of the models. The reviewed models also lack external validation or had reduced performance in external cohort.

Application of machine learning for lung cancer survival prognostication-A systematic review and meta-analysis

Introduction

Machine learning (ML) techniques have gained increasing attention in the field of healthcare, including predicting outcomes in patients with lung cancer. ML has the potential to enhance prognostication in lung cancer patients and improve clinical decision-making. In this systematic review and meta-analysis, we aimed to evaluate the performance of ML models compared to logistic regression (LR) models in predicting overall survival in patients with lung cancer.

Methods

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement. A comprehensive search was conducted in Medline, Embase, and Cochrane databases using a predefined search query. Two independent reviewers screened abstracts and conflicts were resolved by a third reviewer. Inclusion and exclusion criteria were applied to select eligible studies. Risk of bias assessment was performed using predefined criteria. Data extraction was conducted using the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS) checklist. Meta-analytic analysis was performed to compare the discriminative ability of ML and LR models.

Results

The literature search resulted in 3,635 studies, and 12 studies with a total of 211,068 patients were included in the analysis. Six studies reported confidence intervals and were included in the meta-analysis. The performance of ML models varied across studies, with C-statistics ranging from 0.60 to 0.85. The pooled analysis showed that ML models had higher discriminative ability compared to LR models, with a weighted average C-statistic of 0.78 for ML models compared to 0.70 for LR models.

Conclusion

Machine learning models show promise in predicting overall survival in patients with lung cancer, with superior discriminative ability compared to logistic regression models. However, further validation and standardization of ML models are needed before their widespread implementation in clinical practice. Future research should focus on addressing the limitations of the current literature, such as potential bias and heterogeneity among studies, to improve the accuracy and generalizability of ML models for predicting outcomes in patients with lung cancer. Further research and development of ML models in this field may lead to improved patient outcomes and personalized treatment strategies.

Research Concepts and Opportunities for Early-Career Investigators in Cardiac Surgery

Basic, translational or clinic, research is a key component of cardiac surgery. Understanding basic cellular and molecular mechanisms is key to improving patient outcomes, and cardiac surgical procedures must be compared with nonsurgical alternatives. However, guidance for early-career investigators interested in cardiac surgery research is limited. This opinion piece aims at providing basic guidance and principles based on the authors' experience.

Comparative analysis of machine learning vs. traditional modeling approaches for predicting in-hospital mortality after cardiac surgery: temporal and spatial external validation based on a nationwide cardiac surgery registry

AIMS

Preoperative risk assessment is crucial for cardiac surgery. Although previous studies suggested machine learning (ML) may improve in-hospital mortality predictions after cardiac surgery compared to traditional modeling approaches, the validity is doubted due to lacking external validation, limited sample sizes, and inadequate modeling considerations. We aimed to assess predictive performance between ML and traditional modelling approaches, while addressing these major limitations.

METHODS AND RESULTS

Adult cardiac surgery cases (n = 168 565) between 2013 and 2018 in the Chinese Cardiac Surgery Registry were used to develop, validate, and compare various ML vs. logistic regression (LR) models. The dataset was split for temporal (2013-2017 for training, 2018 for testing) and spatial (geographically-stratified random selection of 83 centers for training, 22 for testing) experiments, respectively. Model performances were evaluated in testing sets for discrimination and calibration. The overall in-hospital mortality was 1.9%. In the temporal testing set (n = 32 184), the best-performing ML model demonstrated a similar area under the receiver operating characteristic curve (AUC) of 0.797 (95% CI 0.779-0.815) to the LR model (AUC 0.791 [95% CI 0.775-0.808]; P = 0.12). In the spatial experiment (n = 28 323), the best ML model showed a statistically better but modest performance improvement (AUC 0.732 [95% CI 0.710-0.754]) than LR (AUC 0.713 [95% CI 0.691-0.737]; P = 0.002). Varying feature selection methods had relatively smaller effects on ML models. Most ML and LR models were significantly miscalibrated.

CONCLUSION

ML provided only marginal improvements over traditional modelling approaches in predicting cardiac surgery mortality with routine preoperative variables, which calls for more judicious use of ML in practice.

Interpretable machine learning-based predictive modeling of patient outcomes following cardiac surgery

BACKGROUND

The clinical applicability of machine learning predictions of patient outcomes following cardiac surgery remains unclear. We applied machine learning to predict patient outcomes associated with high morbidity and mortality after cardiac surgery and identified the importance of variables to the derived model's performance.

METHODS

We applied machine learning to the Society of Thoracic Surgeons Adult Cardiac Surgery Database to predict postoperative hemorrhage requiring reoperation, venous thromboembolism (VTE), and stroke. We used permutation feature importance to identify variables important to model performance and a misclassification analysis to study the limitations of the model.

RESULTS

The study dataset included 662,772 subjects who underwent cardiac surgery between 2015 and 2017 and 240 variables. Hemorrhage requiring reoperation, VTE, and stroke occurred in 2.9%, 1.2%, and 2.0% of subjects, respectively. The model performed remarkably well at predicting all 3 complications (area under the receiver operating characteristic curve, 0.92-0.97). Preoperative and intraoperative variables were not important to model performance; instead, performance for the prediction of all 3 outcomes was driven primarily by several postoperative variables, including known risk factors for the complications, such as mechanical ventilation and new onset of postoperative arrhythmias. Many of the postoperative variables important to model performance also increased the risk of subject misclassification, indicating internal validity.

CONCLUSIONS

A machine learning model accurately and reliably predicts patient outcomes following cardiac surgery. Postoperative, as opposed to preoperative or intraoperative variables, are important to model performance. Interventions targeting this period, including minimizing the duration of mechanical ventilation and early treatment of new-onset postoperative arrhythmias, may help lower the risk of these complications.

Machine-learning-based prediction of survival and mitral regurgitation recurrence in patients undergoing mitral valve repair

OBJECTIVES

This study was conducted to assess long-term clinical outcomes after mitral valve repair using machine-learning techniques.

METHODS

We retrospectively evaluated 436 consecutive patients (mean age: 54.7 ± 15.4; 235 males) who underwent mitral valve repair between January 2000 and December 2017. Actuarial survival and freedom from significant (≥ moderate) mitral regurgitation (MR) were clinical end points. To evaluate the independent risk factors, random survival forest (RSF), extreme gradient boost (XGBoost), support vector machine, Cox proportional hazards model and general linear models with elastic net regularization were used. Concordance indices (C-indices) of each model were estimated.

RESULTS

The operative mortality was 0.9% (N = 4). Reoperation was required in 15 patients (3.5%). In terms of C-index, the overall performance of the XGBoost (C-index 0.806) and RSF models (C-index 0.814) was better than that of the Cox model (C-index 0.733) in overall survival. For the recurrent MR, the C-index for XGBoost was 0.718, which was the highest among the 5 models. Compared to the Cox model (C-index 0.545), the C-indices of the XGBoost (C-index 0.718) and RSF models (C-index 0.692) were higher.

CONCLUSIONS

Machine-learning techniques can be a useful tool for both prediction and interpretation in the survival and recurrent MR. From the machine-learning techniques examined here, the long-term clinical outcomes of mitral valve repair were excellent. The complexity of MV increased the risk of late mitral valve-related reoperation.

Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network

Machine learning (ML) is increasingly applied to predict adverse postoperative outcomes in cardiac surgery. Commonly used ML models fail to translate to clinical practice due to absent model explainability, limited uncertainty quantification, and no flexibility to missing data. We aimed to develop and benchmark a novel ML approach, the uncertainty-aware attention network (UAN), to overcome these common limitations. Two Bayesian uncertainty quantification methods were tested, generalized variational inference (GVI) or a posterior network (PN). The UAN models were compared with an ensemble of XGBoost models and a Bayesian logistic regression model (LR) with imputation. The derivation datasets consisted of 153,932 surgery events from the Australian and New Zealand Society of Cardiac and Thoracic Surgeons (ANZSCTS) Cardiac Surgery Database. An external validation consisted of 7343 surgery events which were extracted from the Medical Information Mart for Intensive Care (MIMIC) III critical care dataset. The highest performing model on the external validation dataset was a UAN-GVI with an area under the receiver operating characteristic curve (AUC) of 0.78 (0.01). Model performance improved on high confidence samples with an AUC of 0.81 (0.01). Confidence calibration for aleatoric uncertainty was excellent for all models. Calibration for epistemic uncertainty was more variable, with an ensemble of XGBoost models performing the best with an AUC of 0.84 (0.08). Epistemic uncertainty was improved using the PN approach, compared to GVI. UAN is able to use an interpretable and flexible deep learning approach to provide estimates of model uncertainty alongside state-of-the-art predictions. The model has been made freely available as an easy-to-use web application demonstrating that by designing uncertainty-aware models with innately explainable predictions deep learning may become more suitable for routine clinical use.

A machine learning approach to predicting 30-day mortality following paediatric cardiac surgery: findings from the Australia New Zealand Congenital Outcomes Registry for Surgery (ANZCORS)

OBJECTIVES

We aim to develop the first risk prediction model for 30-day mortality for the Australian and New Zealand patient populations and examine whether machine learning (ML) algorithms outperform traditional statistical approaches.

METHODS

Data from the Australia New Zealand Congenital Outcomes Registry for Surgery, which contains information on every paediatric cardiac surgical encounter in Australian and New Zealand for patients aged <18 years between January 2013 and December 2021, were analysed (n = 14 343). The outcome was mortality within the 30-day period following a surgical encounter, with ∼30% of the observations randomly selected to be used for validation of the final model. Three different ML methods were used, all of which employed five-fold cross-validation to prevent overfitting, with model performance judged primarily by the area under the receiver operating curve (AUC).

RESULTS

Among the 14 343 30-day periods, there were 188 deaths (1.3%). In the validation data, the gradient-boosted tree obtained the best performance [AUC = 0.87, 95% confidence interval = (0.82, 0.92); calibration = 0.97, 95% confidence interval = (0.72, 1.27)], outperforming penalized logistic regression and artificial neural networks (AUC of 0.82 and 0.81, respectively). The strongest predictors of mortality in the gradient boosting trees were patient weight, STAT score, age and gender.

CONCLUSIONS

Our risk prediction model outperformed logistic regression and achieved a level of discrimination comparable to the PRAiS2 and Society of Thoracic Surgery Congenital Heart Surgery Database mortality risk models (both which obtained AUC = 0.86). Non-linear ML methods can be used to construct accurate clinical risk prediction tools.

Tree-based survival analysis improves mortality prediction in cardiac surgery

Objectives

Machine learning (ML) classification tools are known to accurately predict many cardiac surgical outcomes. A novel approach, ML-based survival analysis, remains unstudied for predicting mortality after cardiac surgery. We aimed to benchmark performance, as measured by the concordance index (C-index), of tree-based survival models against Cox proportional hazards (CPH) modeling and explore risk factors using the best-performing model.

Methods

144,536 patients with 147,301 surgery events from the Australian and New Zealand Society of Cardiac and Thoracic Surgeons (ANZSCTS) national database were used to train and validate models. Univariate analysis was performed using Student's T-test for continuous variables, Chi-squared test for categorical variables, and stratified Kaplan-Meier estimation of the survival function. Three ML models were tested, a decision tree (DT), random forest (RF), and gradient boosting machine (GBM). Hyperparameter tuning was performed using a Bayesian search strategy. Performance was assessed using 2-fold cross-validation repeated 5 times.

Results

The highest performing model was the GBM with a C-index of 0.803 (0.002), followed by RF with 0.791 (0.003), DT with 0.729 (0.014), and finally CPH with 0.596 (0.042). The 5 most predictive features were age, type of procedure, length of hospital stay, drain output in the first 4 h (ml), and inotrope use greater than 4 h postoperatively.

Conclusion

Tree-based learning for survival analysis is a non-parametric and performant alternative to CPH modeling. GBMs offer interpretable modeling of non-linear relationships, promising to expose the most relevant risk factors and uncover new questions to guide future research.

Comparing Decentralized Learning Methods for Health Data Models to Nondecentralized Alternatives: Protocol for a Systematic Review

BACKGROUND

Considering the soaring health-related costs directed toward a growing, aging, and comorbid population, the health sector needs effective data-driven interventions while managing rising care costs. While health interventions using data mining have become more robust and adopted, they often demand high-quality big data. However, growing privacy concerns have hindered large-scale data sharing. In parallel, recently introduced legal instruments require complex implementations, especially when it comes to biomedical data. New privacy-preserving technologies, such as decentralized learning, make it possible to create health models without mobilizing data sets by using distributed computation principles. Several multinational partnerships, including a recent agreement between the United States and the European Union, are adopting these techniques for next-generation data science. While these approaches are promising, there is no clear and robust evidence synthesis of health care applications.

OBJECTIVE

The main aim is to compare the performance among health data models (eg, automated diagnosis and mortality prediction) developed using decentralized learning approaches (eg, federated and blockchain) to those using centralized or local methods. Secondary aims are comparing the privacy compromise and resource use among model architectures.

METHODS

We will conduct a systematic review using the first-ever registered research protocol for this topic following a robust search methodology, including several biomedical and computational databases. This work will compare health data models differing in development architecture, grouping them according to their clinical applications. For reporting purposes, a PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 flow diagram will be presented. CHARMS (Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies)-based forms will be used for data extraction and to assess the risk of bias, alongside PROBAST (Prediction Model Risk of Bias Assessment Tool). All effect measures in the original studies will be reported.

RESULTS

The queries and data extractions are expected to start on February 28, 2023, and end by July 31, 2023. The research protocol was registered with PROSPERO, under the number 393126, on February 3, 2023. With this protocol, we detail how we will conduct the systematic review. With that study, we aim to summarize the progress and findings from state-of-the-art decentralized learning models in health care in comparison to their local and centralized counterparts. Results are expected to clarify the consensuses and heterogeneities reported and help guide the research and development of new robust and sustainable applications to address the health data privacy problem, with applicability in real-world settings.

CONCLUSIONS

We expect to clearly present the status quo of these privacy-preserving technologies in health care. With this robust synthesis of the currently available scientific evidence, the review will inform health technology assessment and evidence-based decisions, from health professionals, data scientists, and policy makers alike. Importantly, it should also guide the development and application of new tools in service of patients' privacy and future research.

TRIAL REGISTRATION

PROSPERO 393126; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=393126.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/45823.

Comparison of machine learning techniques in prediction of mortality following cardiac surgery: analysis of over 220 000 patients from a large national database

OBJECTIVES

To perform a systematic comparison of in-hospital mortality risk prediction post-cardiac surgery, between the predominant scoring system-European System for Cardiac Operative Risk Evaluation (EuroSCORE) II, logistic regression (LR) retrained on the same variables and alternative machine learning techniques (ML)-random forest (RF), neural networks (NN), XGBoost and weighted support vector machine.

METHODS

Retrospective analyses of prospectively routinely collected data on adult patients undergoing cardiac surgery in the UK from January 2012 to March 2019. Data were temporally split 70:30 into training and validation subsets. Mortality prediction models were created using the 18 variables of EuroSCORE II. Comparisons of discrimination, calibration and clinical utility were then conducted. Changes in model performance, variable-importance over time and hospital/operation-based model performance were also reviewed.

RESULTS

Of the 227 087 adults who underwent cardiac surgery during the study period, there were 6258 deaths (2.76%). In the testing cohort, there was an improvement in discrimination [XGBoost (95% confidence interval (CI) area under the receiver operator curve (AUC), 0.834-0.834, F1 score, 0.276-0.280) and RF (95% CI AUC, 0.833-0.834, F1, 0.277-0.281)] compared with EuroSCORE II (95% CI AUC, 0.817-0.818, F1, 0.243-0.245). There was no significant improvement in calibration with ML and retrained-LR compared to EuroSCORE II. However, EuroSCORE II overestimated risk across all deciles of risk and over time. The calibration drift was lowest in NN, XGBoost and RF compared with EuroSCORE II. Decision curve analysis showed XGBoost and RF to have greater net benefit than EuroSCORE II.

CONCLUSIONS

ML techniques showed some statistical improvements over retrained-LR and EuroSCORE II. The clinical impact of this improvement is modest at present. However the incorporation of additional risk factors in future studies may improve upon these findings and warrants further study.

Machine Learning and Artificial Intelligence in Surgical Research

Machine learning, a subtype of artificial intelligence, is an emerging field of surgical research dedicated to predictive modeling. From its inception, machine learning has been of interest in medical and surgical research. Built on traditional research metrics for optimal success, avenues of research include diagnostics, prognosis, operative timing, and surgical education, in a variety of surgical subspecialties. Machine learning represents an exciting and developing future in the world of surgical research that will not only allow for more personalized and comprehensive medical care.

Machine learning-based prediction of 1-year mortality in hypertensive patients undergoing coronary revascularization surgery

BACKGROUND

Machine learning (ML) has shown promising results in all fields of medicine, including preventive cardiology. Hypertensive patients are at higher risk of mortality after coronary artery bypass graft (CABG) surgery; thus, we aimed to design and evaluate five ML models to predict 1-year mortality among hypertensive patients who underwent CABG.

HYOTHESIS

ML algorithms can significantly improve mortality prediction after CABG.

METHODS

Tehran Heart Center's CABG data registry was used to extract several baseline and peri-procedural characteristics and mortality data. The best features were chosen using random forest (RF) feature selection algorithm. Five ML models were developed to predict 1-year mortality: logistic regression (LR), RF, artificial neural network (ANN), extreme gradient boosting (XGB), and naïve Bayes (NB). The area under the curve (AUC), sensitivity, and specificity were used to evaluate the models.

RESULTS

Among the 8,493 hypertensive patients who underwent CABG (mean age of 68.27 ± 9.27 years), 303 died in the first year. Eleven features were selected as the best predictors, among which total ventilation hours and ejection fraction were the leading ones. LR showed the best prediction ability with an AUC of 0.82, while the least AUC was for the NB model (0.79). Among the subgroups, the highest AUC for LR model was for two age range groups (50-59 and 80-89 years), overweight, diabetic, and smoker subgroups of hypertensive patients.

CONCLUSIONS

All ML models had excellent performance in predicting 1-year mortality among CABG hypertension patients, while LR was the best regarding AUC. These models can help clinicians assess the risk of mortality in specific subgroups at higher risk (such as hypertensive ones).

Machine learning to predict adverse outcomes after cardiac surgery: A systematic review and meta-analysis

BACKGROUND

Machine learning (ML) models are promising tools for predicting adverse postoperative outcomes in cardiac surgery, yet have not translated to routine clinical use. We conducted a systematic review and meta-analysis to assess the predictive performance of ML approaches.

METHODS

We conducted an electronic search to find studies assessing ML and traditional statistical models to predict postoperative outcomes. Our primary outcome was the concordance (C-) index of discriminative performance. Using a Bayesian meta-analytic approach we pooled the C-indices with the 95% credible interval (CrI) across multiple outcomes comparing ML methods to logistic regression (LR) and clinical scoring tools. Additionally, we performed critical difference and sensitivity analysis.

RESULTS

We identified 2792 references from the search of which 51 met inclusion criteria. Two postoperative outcomes were amenable for meta-analysis: 30-day mortality and in-hospital mortality. For 30-day mortality, the pooled C-index and 95% CrI were 0.82 (0.79-0.85), 0.80 (0.77-0.84), 0.78 (0.74-0.82) for ML models, LR, and scoring tools respectively. For in-hospital mortality, the pooled C-index was 0.81 (0.78-0.84) and 0.79 (0.73-0.84) for ML models and LR, respectively. There were no statistically significant results indicating ML superiority over LR.

CONCLUSION

In cardiac surgery patients, for the prediction of mortality, current ML methods do not have greater discriminative power over LR as measured by the C-index.

Prediction of Postoperative Deterioration in Cardiac Surgery Patients Using Electronic Health Record and Physiologic Waveform Data

BACKGROUND

Postoperative hemodynamic deterioration among cardiac surgical patients can indicate or lead to adverse outcomes. Whereas prediction models for such events using electronic health records or physiologic waveform data are previously described, their combined value remains incompletely defined. The authors hypothesized that models incorporating electronic health record and processed waveform signal data (electrocardiogram lead II, pulse plethysmography, arterial catheter tracing) would yield improved performance versus either modality alone.

METHODS

Intensive care unit data were reviewed after elective adult cardiac surgical procedures at an academic center between 2013 and 2020. Model features included electronic health record features and physiologic waveforms. Tensor decomposition was used for waveform feature reduction. Machine learning-based prediction models included a 2013 to 2017 training set and a 2017 to 2020 temporal holdout test set. The primary outcome was a postoperative deterioration event, defined as a composite of low cardiac index of less than 2.0 ml min-1 m-2, mean arterial pressure of less than 55 mmHg sustained for 120 min or longer, new or escalated inotrope/vasopressor infusion, epinephrine bolus of 1 mg or more, or intensive care unit mortality. Prediction models analyzed data 8 h before events.

RESULTS

Among 1,555 cases, 185 (12%) experienced 276 deterioration events, most commonly including low cardiac index (7.0% of patients), new inotrope (1.9%), and sustained hypotension (1.4%). The best performing model on the 2013 to 2017 training set yielded a C-statistic of 0.803 (95% CI, 0.799 to 0.807), although performance was substantially lower in the 2017 to 2020 test set (0.709, 0.705 to 0.712). Test set performance of the combined model was greater than corresponding models limited to solely electronic health record features (0.641; 95% CI, 0.637 to 0.646) or waveform features (0.697; 95% CI, 0.693 to 0.701).

CONCLUSIONS

Clinical deterioration prediction models combining electronic health record data and waveform data were superior to either modality alone, and performance of combined models was primarily driven by waveform data. Decreased performance of prediction models during temporal validation may be explained by data set shift, a core challenge of healthcare prediction modeling.

EDITOR’S PERSPECTIVE

Machine Learning to Predict Outcomes and Cost by Phase of Care After Coronary Artery Bypass Grafting

BACKGROUND

Machine learning may enhance prediction of outcomes after coronary artery bypass grafting (CABG). We sought to develop and validate a dynamic machine learning model to predict CABG outcomes at clinically relevant pre- and postoperative time points.

METHODS

The Society of Thoracic Surgeons (STS) registry data elements from 2086 isolated CABG patients were divided into training and testing datasets and input into Extreme Gradient Boosting decision-tree machine learning algorithms. Two prediction models were developed based on data from preoperative (80 parameters) and postoperative (125 parameters) phases of care. Outcomes included operative mortality, major morbidity or mortality, high cost, and 30-day readmission. Machine learning and STS model performance were assessed using accuracy and the area under the precision-recall curve (AUC-PR).

RESULTS

Preoperative machine learning models predicted mortality (accuracy, 98%; AUC-PR = 0.16; F1 = 0.24), major morbidity or mortality (accuracy, 75%; AUC-PR = 0.33; F1 = 0.42), high cost (accuracy, 83%; AUC-PR = 0.51; F1 = 0.52), and 30-day readmission (accuracy, 70%; AUC-PR = 0.47; F1 = 0.49) with high accuracy. Preoperative machine learning models performed similarly to the STS for prediction of mortality (STS AUC-PR = 0.11; P = .409) and outperformed STS for prediction of mortality or major morbidity (STS AUC-PR = 0.28; P < .001). Addition of intraoperative parameters further improved machine learning model performance for major morbidity or mortality (AUC-PR = 0.39; P < .01) and high cost (AUC-PR = 0.64; P < .01), with cross-clamp and bypass times emerging as important additive predictive parameters.

CONCLUSIONS

Machine learning can predict mortality, major morbidity, high cost, and readmission after isolated CABG. Prediction based on the phase of care allows for dynamic risk assessment through the hospital course, which may benefit quality assessment and clinical decision-making.

Machine learning algorithms to predict major bleeding after isolated coronary artery bypass grafting

Objectives

Postoperative major bleeding is a common problem in patients undergoing cardiac surgery and is associated with poor outcomes. We evaluated the performance of machine learning (ML) methods to predict postoperative major bleeding.

Methods

A total of 1,045 patients who underwent isolated coronary artery bypass graft surgery (CABG) were enrolled. Their datasets were assigned randomly to training (70%) or a testing set (30%). The primary outcome was major bleeding defined as the universal definition of perioperative bleeding (UDPB) classes 3–4. We constructed a reference logistic regression (LR) model using known predictors. We also developed several modern ML algorithms. In the test set, we compared the area under the receiver operating characteristic curves (AUCs) of these ML algorithms with the reference LR model results, and the TRUST and WILL-BLEED risk score. Calibration analysis was undertaken using the calibration belt method.

Results

The prevalence of postoperative major bleeding was 7.1% (74/1,045). For major bleeds, the conditional inference random forest (CIRF) model showed the highest AUC [0.831 (0.732–0.930)], and the stochastic gradient boosting (SGBT) and random forest models demonstrated the next best results [0.820 (0.742–0.899) and 0.810 (0.719–0.902)]. The AUCs of all ML models were higher than [0.629 (0.517–0.641) and 0.557 (0.449–0.665)], as achieved by TRUST and WILL-BLEED, respectively.

Conclusion

ML methods successfully predicted major bleeding after cardiac surgery, with greater performance compared with previous scoring models. Modern ML models may enhance the identification of high-risk major bleeding subpopulations.

Predicting major adverse cardiovascular events in acute coronary syndrome: A scoping review of machine learning approaches

BACKGROUND

Acute coronary syndrome is the topmost cause of death worldwide; therefore, it is necessary to predict major adverse cardiovascular events and cardiovascular deaths in patients with acute coronary syndrome to make correct and timely clinical decisions.

OBJECTIVE

The current review aimed to highlight algorithms and important predictor variables through examining those studies which used machine learning algorithms for predicting major adverse cardiovascular events in patients with acute coronary syndrome.

METHODS

In order to predict major adverse cardiovascular events in patients with acute coronary syndrome, the preferred reporting items for scoping reviews guidelines were used. PubMed, Embase, Web of Science, Scopus, Springer, and IEEE Xplore databases were searched for articles published between 2005 and 2021. The findings of the studies are presented in the form of a narrative synthesis of evidence.

RESULTS

According to the results, 14 (63.64%) studies did not perform external validation and only used registry data. The algorithms used in this study comprised, inter alia, Regression Logistic, Random Forest, Boosting Ensemble, Non-Boosting Ensemble, Decision Trees, and Naive Bayes. Multiple studies (N=20) achieved a high Area under the ROC Curve between 0.8 to 0.99 in predicting mortality and major adverse cardiovascular events. The predictor variables used in these studies were divided into demographic, clinical, and therapeutic features. However, no study reported the integration of machine learning model into clinical practice.

CONCLUSION

Machine learning algorithms rendered acceptable results to predict major adverse cardiovascular events and mortality outcomes in patients with acute coronary syndrome. However, these approaches have never been integrated into clinical practice. Further research is required to develop feasible and effective machine learning prediction models to measure their potentially important implications for optimizing the quality of care in patients with acute coronary syndrome.

Machine Learning Methods for Predicting Long-Term Mortality in Patients After Cardiac Surgery

Objective:

This study aims to construct and validate several machine learning (ML) algorithms to predict long-term mortality and identify risk factors in unselected patients post-cardiac surgery.

Methods

The Medical Information Mart for Intensive Care (MIMIC-III) database was used to perform a retrospective administrative database study. Candidate predictors consisted of the demographics, comorbidity, vital signs, laboratory test results, scoring systems, and treatment information on the first day of ICU admission. Four-year mortality was set as the study outcome. We used the ML methods of logistic regression (LR), artificial neural network (NNET), naïve bayes (NB), gradient boosting machine (GBM), adapting boosting (Ada), random forest (RF), bagged trees (BT), and eXtreme Gradient Boosting (XGB). The prognostic capacity and clinical utility of these ML models were compared using the area under the receiver operating characteristic curves (AUC), calibration curves, and decision curve analysis (DCA).

Results

Of 7,368 patients in MIMIC-III included in the final cohort, a total of 1,337 (18.15%) patients died during a 4-year follow-up. Among 65 variables extracted from the database, a total of 25 predictors were selected using recursive feature elimination and included in the subsequent analysis. The Ada model performed best among eight models in both discriminatory ability with the highest AUC of 0.801 and goodness of fit (visualized by calibration curve). Moreover, the DCA shows that the net benefit of the RF, Ada, and BT models surpassed that of other ML models for almost all threshold probability values. Additionally, through the Ada technique, we determined that red blood cell distribution width (RDW), blood urea nitrogen (BUN), SAPS II, anion gap (AG), age, urine output, chloride, creatinine, congestive heart failure, and SOFA were the Top 10 predictors in the feature importance rankings.

Conclusions

The Ada model performs best in predicting 4-year mortality after cardiac surgery among the eight ML models, which might have significant application in the development of early warning systems for patients following operations.

Comparing Machine Learning Models and Statistical Models for Predicting Heart Failure Events: A Systematic Review and Meta-Analysis

Objective

To compare the performance, clinical feasibility, and reliability of statistical and machine learning (ML) models in predicting heart failure (HF) events.

Background

Although ML models have been proposed to revolutionize medicine, their promise in predicting HF events has not been investigated in detail.

Methods

A systematic search was performed on Medline, Web of Science, and IEEE Xplore for studies published between January 1, 2011 to July 14, 2021 that developed or validated at least one statistical or ML model that could predict all-cause mortality or all-cause readmission of HF patients. Prediction Model Risk of Bias Assessment Tool was used to assess the risk of bias, and random effect model was used to evaluate the pooled c-statistics of included models.

Result

Two-hundred and two statistical model studies and 78 ML model studies were included from the retrieved papers. The pooled c-index of statistical models in predicting all-cause mortality, ML models in predicting all-cause mortality, statistical models in predicting all-cause readmission, ML models in predicting all-cause readmission were 0.733 (95% confidence interval 0.724–0.742), 0.777 (0.752–0.803), 0.678 (0.651–0.706), and 0.660 (0.633–0.686), respectively, indicating that ML models did not show consistent superiority compared to statistical models. The head-to-head comparison revealed similar results. Meanwhile, the immoderate use of predictors limited the feasibility of ML models. The risk of bias analysis indicated that ML models' technical pitfalls were more serious than statistical models'. Furthermore, the efficacy of ML models among different HF subgroups is still unclear.

Conclusions

ML models did not achieve a significant advantage in predicting events, and their clinical feasibility and reliability were worse.

Machine Learning for the Prediction of Complications in Patients After Mitral Valve Surgery

Background: This study intended to use a machine learning model to identify critical preoperative and intraoperative variables and predict the risk of several severe complications (myocardial infarction, stroke, renal failure, and hospital mortality) after cardiac valvular surgery.

Study Design and Methods: A total of 1,488 patients undergoing cardiac valvular surgery in eight large tertiary hospitals in China were examined. Fifty-four perioperative variables, such as essential demographic characteristics, concomitant disease, preoperative laboratory indicators, operation type, and intraoperative information, were collected. Machine learning models were developed and validated by 10-fold cross-validation. In each fold, Recursive Feature Elimination was used to select key variables. Ten machine learning models and logistic regression were developed. The area under the receiver operating characteristic (AUROC), accuracy (ACC), Youden index, sensitivity, specificity, F1-score, positive predictive value (PPV), and negative predictive value (NPV) were used to compare the prediction performance of different models. The SHapley Additive ex Planations package was applied to interpret the best machine learning model. Finally, a model was trained on the whole dataset with the merged key variables, and a web tool was created for clinicians to use.

Results: In this study, 14 vital variables, namely, intraoperative total input, intraoperative blood loss, intraoperative colloid bolus, Classification of New York Heart Association (NYHA) heart function, preoperative hemoglobin (Hb), preoperative platelet (PLT), age, preoperative fibrinogen (FIB), intraoperative minimum red blood cell volume (Hct), body mass index (BMI), creatinine, preoperative Hct, intraoperative minimum Hb, and intraoperative autologous blood, were finally selected. The eXtreme Gradient Boosting algorithms (XGBOOST) algorithm model presented a significantly better predictive performance (AUROC: 0.90) than the other models (ACC: 81%, Youden index: 70%, sensitivity: 89%, specificity: 81%, F1-score:0.26, PPV: 15%, and NPV: 99%).

Conclusion: A model for predicting several severe complications after cardiac valvular surgery was successfully developed using a machine learning algorithm based on 14 perioperative variables, which could guide clinical physicians to take appropriate preventive measures and diminish the complications for patients at high risk.

Electrocardiography-Based Artificial Intelligence Algorithm Aids in Prediction of Long-term Mortality After Cardiac Surgery

OBJECTIVE

To assess whether an electrocardiography-based artificial intelligence (AI) algorithm developed to detect severe ventricular dysfunction (left ventricular ejection fraction [LVEF] of 35% or below) independently predicts long-term mortality after cardiac surgery among patients without severe ventricular dysfunction (LVEF>35%).

METHODS

Patients who underwent valve or coronary bypass surgery at Mayo Clinic (1993-2019) and had documented LVEF above 35% on baseline electrocardiography were included. We compared patients with an abnormal vs a normal AI-enhanced electrocardiogram (AI-ECG) screen for LVEF of 35% or below on preoperative electrocardiography. The primary end point was all-cause mortality.

RESULTS

A total of 20,627 patients were included, of whom 17,125 (83.0%) had a normal AI-ECG screen and 3502 (17.0%) had an abnormal AI-ECG screen. Patients with an abnormal AI-ECG screen were older and had more comorbidities. Probability of survival at 5 and 10 years was 86.2% and 68.2% in patients with a normal AI-ECG screen vs 71.4% and 45.1% in those with an abnormal screen (log-rank, P<.01). In the multivariate Cox survival analysis, the abnormal AI-ECG screen was independently associated with a higher all-cause mortality overall (hazard ratio [HR], 1.31; 95% CI, 1.24 to 1.37) and in subgroups of isolated valve surgery (HR, 1.30; 95% CI, 1.18 to 1.42), isolated coronary artery bypass grafting (HR, 1.29; 95% CI, 1.20 to 1.39), and combined coronary artery bypass grafting and valve surgery (HR, 1.19; 95% CI, 1.08 to 1.32). In a subgroup analysis, the association between abnormal AI-ECG screen and mortality was consistent in patients with LVEF of 35% to 55% and among those with LVEF above 55%.

CONCLUSION

A novel electrocardiography-based AI algorithm that predicts severe ventricular dysfunction can predict long-term mortality among patients with LVEF above 35% undergoing valve and/or coronary bypass surgery.

Clinical impact and quality of randomized controlled trials involving interventions evaluating artificial intelligence prediction tools: a systematic review

The evidence of the impact of traditional statistical (TS) and artificial intelligence (AI) tool interventions in clinical practice was limited. This study aimed to investigate the clinical impact and quality of randomized controlled trials (RCTs) involving interventions evaluating TS, machine learning (ML), and deep learning (DL) prediction tools. A systematic review on PubMed was conducted to identify RCTs involving TS/ML/DL tool interventions in the past decade. A total of 65 RCTs from 26,082 records were included. A majority of them had model development studies and generally good performance was achieved. The function of TS and ML tools in the RCTs mainly included assistive treatment decisions, assistive diagnosis, and risk stratification, but DL trials were only conducted for assistive diagnosis. Nearly two-fifths of the trial interventions showed no clinical benefit compared to standard care. Though DL and ML interventions achieved higher rates of positive results than TS in the RCTs, in trials with low risk of bias (17/65) the advantage of DL to TS was reduced while the advantage of ML to TS disappeared. The current applications of DL were not yet fully spread performed in medicine. It is predictable that DL will integrate more complex clinical problems than ML and TS tools in the future. Therefore, rigorous studies are required before the clinical application of these tools.

Prediction of short-term mortality in acute heart failure patients using minimal electronic health record data

Background

Acute heart failure (AHF) is associated with significant morbidity and mortality. Effective patient risk stratification is essential to guiding hospitalization decisions and the clinical management of AHF. Clinical decision support systems can be used to improve predictions of mortality made in emergency care settings for the purpose of AHF risk stratification. In this study, several models for the prediction of seven-day mortality among AHF patients were developed by applying machine learning techniques to retrospective patient data from 236,275 total emergency department (ED) encounters, 1881 of which were considered positive for AHF and were used for model training and testing. The models used varying subsets of age, sex, vital signs, and laboratory values. Model performance was compared to the Emergency Heart Failure Mortality Risk Grade (EHMRG) model, a commonly used system for prediction of seven-day mortality in the ED with similar (or, in some cases, more extensive) inputs. Model performance was assessed in terms of area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity.

Results

When trained and tested on a large academic dataset, the best-performing model and EHMRG demonstrated test set AUROCs of 0.84 and 0.78, respectively, for prediction of seven-day mortality. Given only measurements of respiratory rate, temperature, mean arterial pressure, and FiO₂, one model produced a test set AUROC of 0.83. Neither a logistic regression comparator nor a simple decision tree outperformed EHMRG.

Conclusions

A model using only the measurements of four clinical variables outperforms EHMRG in the prediction of seven-day mortality in AHF. With these inputs, the model could not be replaced by logistic regression or reduced to a simple decision tree without significant performance loss. In ED settings, this minimal-input risk stratification tool may assist clinicians in making critical decisions about patient disposition by providing early and accurate insights into individual patient’s risk profiles.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13040-021-00255-w.