Prediction of Cardiac Arrest in the Emergency Department Based on Machine Learning and Sequential Characteristics: Model Development and Retrospective Clinical Validation Study

Explainable Artificial Intelligence Warning Model Using an Ensemble Approach for In-Hospital Cardiac Arrest Prediction: Retrospective Cohort Study

BACKGROUND

Cardiac arrest (CA) is the leading cause of death in critically ill patients. Clinical research has shown that early identification of CA reduces mortality. Algorithms capable of predicting CA with high sensitivity have been developed using multivariate time series data. However, these algorithms suffer from a high rate of false alarms, and their results are not clinically interpretable.

OBJECTIVE

We propose an ensemble approach using multiresolution statistical features and cosine similarity-based features for the timely prediction of CA. Furthermore, this approach provides clinically interpretable results that can be adopted by clinicians.

METHODS

Patients were retrospectively analyzed using data from the Medical Information Mart for Intensive Care-IV database and the eICU Collaborative Research Database. Based on the multivariate vital signs of a 24-hour time window for adults diagnosed with heart failure, we extracted multiresolution statistical and cosine similarity-based features. These features were used to construct and develop gradient boosting decision trees. Therefore, we adopted cost-sensitive learning as a solution. Then, 10-fold cross-validation was performed to check the consistency of the model performance, and the Shapley additive explanation algorithm was used to capture the overall interpretability of the proposed model. Next, external validation using the eICU Collaborative Research Database was performed to check the generalization ability.

RESULTS

The proposed method yielded an overall area under the receiver operating characteristic curve (AUROC) of 0.86 and area under the precision-recall curve (AUPRC) of 0.58. In terms of the timely prediction of CA, the proposed model achieved an AUROC above 0.80 for predicting CA events up to 6 hours in advance. The proposed method simultaneously improved precision and sensitivity to increase the AUPRC, which reduced the number of false alarms while maintaining high sensitivity. This result indicates that the predictive performance of the proposed model is superior to the performances of the models reported in previous studies. Next, we demonstrated the effect of feature importance on the clinical interpretability of the proposed method and inferred the effect between the non-CA and CA groups. Finally, external validation was performed using the eICU Collaborative Research Database, and an AUROC of 0.74 and AUPRC of 0.44 were obtained in a general intensive care unit population.

CONCLUSIONS

The proposed framework can provide clinicians with more accurate CA prediction results and reduce false alarm rates through internal and external validation. In addition, clinically interpretable prediction results can facilitate clinician understanding. Furthermore, the similarity of vital sign changes can provide insights into temporal pattern changes in CA prediction in patients with heart failure-related diagnoses. Therefore, our system is sufficiently feasible for routine clinical use. In addition, regarding the proposed CA prediction system, a clinically mature application has been developed and verified in the future digital health field.

Advancing prognostic precision in pulmonary embolism: A clinical and laboratory-based artificial intelligence approach for enhanced early mortality risk stratification

BACKGROUND

Acute pulmonary embolism (PE) is a critical medical emergency that necessitates prompt identification and intervention. Accurate prognostication of early mortality is vital for recognizing patients at elevated risk for unfavourable outcomes and administering suitable therapy. Machine learning (ML) algorithms hold promise for enhancing the precision of early mortality prediction in PE patients.

OBJECTIVE

To devise an ML algorithm for early mortality prediction in PE patients by employing clinical and laboratory variables.

METHODS

This study utilized diverse oversampling techniques to improve the performance of various machine learning models including ANN, SVM, DT, RF, and AdaBoost for early mortality prediction. Appropriate oversampling methods were chosen for each model based on algorithm characteristics and dataset properties. Predictor variables included four lab tests, eight physiological time series indicators, and two general descriptors. Evaluation used metrics like accuracy, F1_score, precision, recall, Area Under the Curve (AUC) and Receiver Operating Characteristic (ROC) curves, providing a comprehensive view of models' predictive abilities.

RESULTS

The findings indicated that the RF model with random oversampling exhibited superior performance among the five models assessed, achieving elevated accuracy and precision alongside high recall for predicting the death class. The oversampling approaches effectively equalized the sample distribution among the classes and enhanced the models' performance.

CONCLUSIONS

The suggested ML technique can efficiently prognosticate mortality in patients afflicted with acute PE. The RF model with random oversampling can aid healthcare professionals in making well-informed decisions regarding the treatment of patients with acute PE. The study underscores the significance of oversampling methods in managing imbalanced data and emphasizes the potential of ML algorithms in refining early mortality prediction for PE patients.

Real-time machine learning model to predict in-hospital cardiac arrest using heart rate variability in ICU

Predicting in-hospital cardiac arrest in patients admitted to an intensive care unit (ICU) allows prompt interventions to improve patient outcomes. We developed and validated a machine learning-based real-time model for in-hospital cardiac arrest predictions using electrocardiogram (ECG)-based heart rate variability (HRV) measures. The HRV measures, including time/frequency domains and nonlinear measures, were calculated from 5 min epochs of ECG signals from ICU patients. A light gradient boosting machine (LGBM) algorithm was used to develop the proposed model for predicting in-hospital cardiac arrest within 0.5-24 h. The LGBM model using 33 HRV measures achieved an area under the receiver operating characteristic curve of 0.881 (95% CI: 0.875-0.887) and an area under the precision-recall curve of 0.104 (95% CI: 0.093-0.116). The most important feature was the baseline width of the triangular interpolation of the RR interval histogram. As our model uses only ECG data, it can be easily applied in clinical practice.

Artificial Intelligence in Ventricular Arrhythmias and Sudden Death

Sudden cardiac arrest due to lethal ventricular arrhythmias is a major cause of mortality worldwide and results in more years of potential life lost than any individual cancer. Most of these sudden cardiac arrest events occur unexpectedly in individuals who have not been identified as high-risk due to the inadequacy of current risk stratification tools. Artificial intelligence tools are increasingly being used to solve complex problems and are poised to help with this major unmet need in the field of clinical electrophysiology. By leveraging large and detailed datasets, artificial intelligence-based prediction models have the potential to enhance the risk stratification of lethal ventricular arrhythmias. This review presents a synthesis of the published literature and a discussion of future directions in this field.

Development of a machine learning-based clinical decision support system to predict clinical deterioration in patients visiting the emergency department

This study aimed to develop a machine learning-based clinical decision support system for emergency departments based on the decision-making framework of physicians. We extracted 27 fixed and 93 observation features using data on vital signs, mental status, laboratory results, and electrocardiograms during emergency department stay. Outcomes included intubation, admission to the intensive care unit, inotrope or vasopressor administration, and in-hospital cardiac arrest. eXtreme gradient boosting algorithm was used to learn and predict each outcome. Specificity, sensitivity, precision, F1 score, area under the receiver operating characteristic curve (AUROC), and area under the precision-recall curve were assessed. We analyzed 303,345 patients with 4,787,121 input data, resampled into 24,148,958 1 h-units. The models displayed a discriminative ability to predict outcomes (AUROC > 0.9), and the model with lagging 6 and leading 0 displayed the highest value. The AUROC curve of in-hospital cardiac arrest had the smallest change, with increased lagging for all outcomes. With inotropic use, intubation, and intensive care unit admission, the range of AUROC curve change with the leading 6 was the highest according to different amounts of previous information (lagging). In this study, a human-centered approach to emulate the clinical decision-making process of emergency physicians has been adopted to enhance the use of the system. Machine learning-based clinical decision support systems customized according to clinical situations can help improve the quality of care.

A Real-World Exploration into Clinical Outcomes of Direct Oral Anticoagulant Dosing Regimens in Morbidly Obese Patients Using Data-Driven Approaches

INTRODUCTION

The clinical outcomes of direct oral anticoagulant (DOAC) dosage regimens in morbid obesity are uncertain due to limited clinical evidence. This study seeks to bridge this evidence gap by identifying the factors associated with clinical outcomes following the dosing of DOACs in morbidly obese patients.

METHOD

A data-driven observational study was carried out using supervised machine learning (ML) models with a dataset extracted from electronic health records and preprocessed. Following 70%:30% partitioning of the overall dataset via stratified sampling, the selected ML classifiers (e.g., random forest, decision trees, bootstrap aggregation) were applied to the training dataset (70%). The outcomes of the models were evaluated against the test dataset (30%). Multivariate regression analysis explored the association between DOAC regimens and clinical outcomes.

RESULTS

A sample of 4,275 morbidly obese patients was extracted and analysed. The decision trees, random forest, and bootstrap aggregation classifiers achieved acceptable (excellent) values of precision, recall, and F1 scores in terms of their contribution to clinical outcomes. The length of stay, treatment days, and age were ranked highest for relevance to mortality and stroke. Among DOAC regimens, apixaban 2.5 mg twice daily ranked highest for its association with mortality, increasing the mortality risk by 43% (odds ratio [OR] 1.430, 95% confidence interval [CI] 1.181-1.732, p = 0.001). On the other hand, apixaban 5 mg twice daily reduced the odds of mortality by 25% (OR 0.751, 95% CI 0.632-0.905, p = 0.003) but increased the odds of stroke events. No clinically relevant non-major bleeding events occurred in this group.

CONCLUSION

Data-driven approaches can identify key factors associated with clinical outcomes following the dosing of DOACs in morbidly obese patients. This will help design further studies to explore well tolerated and effective DOAC doses for morbidly obese patients.

Development of a machine-learning algorithm to predict in-hospital cardiac arrest for emergency department patients using a nationwide database

In this retrospective observational study, we aimed to develop a machine-learning model using data obtained at the prehospital stage to predict in-hospital cardiac arrest in the emergency department (ED) of patients transferred via emergency medical services. The dataset was constructed by attaching the prehospital information from the National Fire Agency and hospital factors to data from the National Emergency Department Information System. Machine-learning models were developed using patient variables, with and without hospital factors. We validated model performance and used the SHapley Additive exPlanation model interpretation. In-hospital cardiac arrest occurred in 5431 of the 1,350,693 patients (0.4%). The extreme gradient boosting model showed the best performance with area under receiver operating curve of 0.9267 when incorporating the hospital factor. Oxygen supply, age, oxygen saturation, systolic blood pressure, the number of ED beds, ED occupancy, and pulse rate were the most influential variables, in that order. ED occupancy and in-hospital cardiac arrest occurrence were positively correlated, and the impact of ED occupancy appeared greater in small hospitals. The machine-learning predictive model using the integrated information acquired in the prehospital stage effectively predicted in-hospital cardiac arrest in the ED and can contribute to the efficient operation of emergency medical systems.

Emerging role of artificial intelligence in cardiac electrophysiology

Artificial intelligence (AI) and machine learning (ML) have significantly impacted the field of cardiovascular medicine, especially cardiac electrophysiology (EP), on multiple fronts. The goal of this review is to familiarize readers with the field of AI and ML and their emerging role in EP. The current review is divided into 3 sections. In the first section, we discuss the definitions and basics of AI, ML, and big data. In the second section, we discuss their application to EP in the context of detection, prediction, and management of arrhythmias. Finally, we discuss the regulatory issues, challenges, and future directions of AI in EP.

Advantage of Vital Sign Monitoring Using a Wireless Wearable Device for Predicting Septic Shock in Febrile Patients in the Emergency Department: A Machine Learning-Based Analysis

Intermittent manual measurement of vital signs may not rapidly predict sepsis development in febrile patients admitted to the emergency department (ED). We aimed to evaluate the predictive performance of a wireless monitoring device that continuously measures heart rate (HR) and respiratory rate (RR) and a machine learning analysis in febrile but stable patients in the ED. We analysed 468 patients (age, ≥18 years; training set, n = 277; validation set, n = 93; test set, n = 98) having fever (temperature >38 °C) and admitted to the isolation care unit of the ED. The AUROC of the fragmented model with device data was 0.858 (95% confidence interval [CI], 0.809−0.908), and that with manual data was 0.841 (95% CI, 0.789−0.893). The AUROC of the accumulated model with device data was 0.861 (95% CI, 0.811−0.910), and that with manual data was 0.853 (95% CI, 0.803−0.903). Fragmented and accumulated models with device data detected clinical deterioration in febrile patients at risk of septic shock 9 h and 5 h 30 min earlier, respectively, than those with manual data. Continuous vital sign monitoring using a wearable device could accurately predict clinical deterioration and reduce the time to recognise potential clinical deterioration in stable ED patients with fever.

Explainable and efficient deep early warning system for cardiac arrest prediction from electronic health records

Cardiac arrest (CA) is a fatal acute event. The development of new CA early warning system based on time series of vital signs from electronic health records (EHR) has great potential to reduce CA damage. In this process, recursive architecture-based deep learning, as a powerful tool for time series data processing, enables automatically extract features from various monitoring clinical parameters and to further improve the performance for acute critical illness prediction. However, the unexplainable nature and excessive time caused by black box structure with poor parallelism are the limitations of its development, especially in the CA clinical application with strict requirement of emergency treatment and low hidden dangers. In this study, we present an explainable and efficient deep early warning system for CA prediction, which features are captured by an efficient temporal convolutional network (TCN) on EHR clinical parameters sequence and explained by deep Taylor decomposition (DTD) theoretical framework. To demonstrate the feasibility of our method and further evaluate its performance, prediction and explanation experiments were performed. Experimental results show that our method achieves superior CA prediction accuracy compared with standard national early warning score (NEWS), in terms of overall AUROC (0.850 Vs. 0.476) and F1-Score (0.750 Vs. 0.450). Furthermore, our method improves the interpretability and efficiency of deep learning-based CA early warning system. It provides the relevance of prediction results for each clinical parameter and about 1.7 times speed enhancement for system calculation compared with the long short-term memory network.

Prediction of cardiac arrest in critically ill patients based on bedside vital signs monitoring

PURPOSE

Cardiac arrest (CA) is the most serious death-related event in critically ill patients and the early detection of CA is beneficial to reduce mortality according to clinical research. This study aims to develop and verify a real-time, interpretable machine learning model, namely cardiac arrest prediction index (CAPI), to predict CA of critically ill patients based on bedside vital signs monitoring.

METHODS

A total of 1,860 patients were analyzed retrospectively from the Medical Information Mart for Intensive Care III (MIMIC-III) database. Based on vital signs, we extracted a total of 43 features for building machine learning model. Extreme Gradient Boosting (XGBoost) was used to develop a real-time prediction model. Three-fold cross validation determined the consistency of model accuracy. SHAP value was used to capture the overall and real-time interpretability of the model.

RESULTS

On the test set, CAPI predicted 95% of CA events, 80% of which were identified more than 25 min in advance, resulting in an area under the receiver operating characteristic curve (AUROC) of 0.94. The sensitivity, specificity, area under the precision-recall curve (AUPRC) and F1-score were 0.86, 0.85, 0.12 and 0.05, respectively.

CONCLUSION

CAPI can help predict patients with CA in the vital signs monitoring at bedside. Compared with previous studies, CAPI can give more timely notifications to doctors for CA events. However, current performance was at the cost of alarm fatigue. Future research is still needed to achieve better clinical application.

Potential applications and performance of machine learning techniques and algorithms in clinical practice: A systematic review

PURPOSE

The advent of clinically adapted machine learning algorithms can solve numerous problems ranging from disease diagnosis and prognosis to therapy recommendations. This systematic review examines the performance of machine learning (ML) algorithms and evaluates the progress made to date towards their implementation in clinical practice.

METHODS

Systematic searching of databases (PubMed, MEDLINE, Scopus, Google Scholar, Cochrane Library and WHO Covid-19 database) to identify original articles published between January 2011 and October 2021. Studies reporting ML techniques in clinical practice involving humans and ML algorithms with a performance metric were considered.

RESULTS

Of 873 unique articles identified, 36 studies were eligible for inclusion. The XGBoost (extreme gradient boosting) algorithm showed the highest potential for clinical applications (n = 7 studies); this was followed jointly by random forest algorithm, logistic regression, and the support vector machine, respectively (n = 5 studies). Prediction of outcomes (n = 33), in particular Inflammatory diseases (n = 7) received the most attention followed by cancer and neuropsychiatric disorders (n = 5 for each) and Covid-19 (n = 4). Thirty-three out of the thirty-six included studies passed more than 50% of the selected quality assessment criteria in the TRIPOD checklist. In contrast, none of the studies could achieve an ideal overall bias rating of 'low' based on the PROBAST checklist. In contrast, only three studies showed evidence of the deployment of ML algorithm(s) in clinical practice.

CONCLUSIONS

ML is potentially a reliable tool for clinical decision support. Although advocated widely in clinical practice, work is still in progress to validate clinically adapted ML algorithms. Improving quality standards, transparency, and interpretability of ML models will further lower the barriers to acceptability.

Artificial Intelligence in Predicting Cardiac Arrest: Scoping Review

BACKGROUND

Cardiac arrest is a life-threatening cessation of activity in the heart. Early prediction of cardiac arrest is important, as it allows for the necessary measures to be taken to prevent or intervene during the onset. Artificial intelligence (AI) technologies and big data have been increasingly used to enhance the ability to predict and prepare for the patients at risk.

OBJECTIVE

This study aims to explore the use of AI technology in predicting cardiac arrest as reported in the literature.

METHODS

A scoping review was conducted in line with the guidelines of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) extension for scoping reviews. Scopus, ScienceDirect, Embase, the Institute of Electrical and Electronics Engineers, and Google Scholar were searched to identify relevant studies. Backward reference list checks of the included studies were also conducted. Study selection and data extraction were independently conducted by 2 reviewers. Data extracted from the included studies were synthesized narratively.

RESULTS

Out of 697 citations retrieved, 41 studies were included in the review, and 6 were added after backward citation checking. The included studies reported the use of AI in the prediction of cardiac arrest. Of the 47 studies, we were able to classify the approaches taken by the studies into 3 different categories: 26 (55%) studies predicted cardiac arrest by analyzing specific parameters or variables of the patients, whereas 16 (34%) studies developed an AI-based warning system. The remaining 11% (5/47) of studies focused on distinguishing patients at high risk of cardiac arrest from patients who were not at risk. Two studies focused on the pediatric population, and the rest focused on adults (45/47, 96%). Most of the studies used data sets with a size of <10,000 samples (32/47, 68%). Machine learning models were the most prominent branch of AI used in the prediction of cardiac arrest in the studies (38/47, 81%), and the most used algorithm was the neural network (23/47, 49%). K-fold cross-validation was the most used algorithm evaluation tool reported in the studies (24/47, 51%).

CONCLUSIONS

AI is extensively used to predict cardiac arrest in different patient settings. Technology is expected to play an integral role in improving cardiac medicine. There is a need for more reviews to learn the obstacles to the implementation of AI technologies in clinical settings. Moreover, research focusing on how to best provide clinicians with support to understand, adapt, and implement this technology in their practice is also necessary.

Machine learning based early mortality prediction in the emergency department

BACKGROUND

It is a great challenge for emergency physicians to early detect the patient's deterioration and prevent unexpected death through a large amount of clinical data, which requires sufficient experience and keen insight.

OBJECTIVE

To evaluate the performance of machine learning models in quantifying the severity of emergency department (ED) patients and identifying high-risk patients.

METHODS

Using routinely-available demographics, vital signs and laboratory tests extracted from electronic health records (EHRs), a framework based on machine learning and feature engineering was proposed for mortality prediction. Patients who had one complete record of vital signs and laboratory tests in ED were included. The following patients were excluded: pediatric patients aged < 18 years, pregnant woman, and patients died or were discharged or hospitalized within 12 h after admission. Based on 76 original features extracted, 9 machine learning models were adopted to validate our proposed framework. Their optimal hyper-parameters were fine-tuned using the grid search method. The prediction results were evaluated on performance metrics (i.e., accuracy, area under the curve (AUC), recall and precision) with repeated 5-fold cross-validation (CV). The time window from patient admission to the prediction was analyzed at 12 h, 24 h, 48 h, and entire stay.

RESULTS

We studied a total of 1114 ED patients with 71.54% (797/1114) survival and 28.46% (317/1114) death in the hospital. The results revealed a more complete time window leads to better prediction performance. Using the entire stay records, the LightGBM model with refined feature engineering demonstrated high discrimination and achieved 93.6% (±0.008) accuracy, 97.6% (±0.003) AUC, 97.1% (±0.008) recall, and 94.2% (±0.006) precision, even if no diagnostic information was utilized.

CONCLUSIONS

This study quantifies the criticality of ED patients and appears to have significant potential as a clinical decision support tool in assisting physicians in their clinical routine. While the model requires validation before use elsewhere, the same methodology could be used to create a strong model for the new hospital.

Predicting Antituberculosis Drug-Induced Liver Injury Using an Interpretable Machine Learning Method: Model Development and Validation Study

Background

Tuberculosis (TB) is a pandemic, being one of the top 10 causes of death and the main cause of death from a single source of infection. Drug-induced liver injury (DILI) is the most common and serious side effect during the treatment of TB.

Objective

We aim to predict the status of liver injury in patients with TB at the clinical treatment stage.

Methods

We designed an interpretable prediction model based on the XGBoost algorithm and identified the most robust and meaningful predictors of the risk of TB-DILI on the basis of clinical data extracted from the Hospital Information System of Shenzhen Nanshan Center for Chronic Disease Control from 2014 to 2019.

Results

In total, 757 patients were included, and 287 (38%) had developed TB-DILI. Based on values of relative importance and area under the receiver operating characteristic curve, machine learning tools selected patients’ most recent alanine transaminase levels, average rate of change of patients’ last 2 measures of alanine transaminase levels, cumulative dose of pyrazinamide, and cumulative dose of ethambutol as the best predictors for assessing the risk of TB-DILI. In the validation data set, the model had a precision of 90%, recall of 74%, classification accuracy of 76%, and balanced error rate of 77% in predicting cases of TB-DILI. The area under the receiver operating characteristic curve score upon 10-fold cross-validation was 0.912 (95% CI 0.890-0.935). In addition, the model provided warnings of high risk for patients in advance of DILI onset for a median of 15 (IQR 7.3-27.5) days.

Conclusions

Our model shows high accuracy and interpretability in predicting cases of TB-DILI, which can provide useful information to clinicians to adjust the medication regimen and avoid more serious liver injury in patients.