Machine learning for the prediction of successful extubation among patients with mechanical ventilation in the intensive care unit: A retrospective observational study

Predictors of weaning failure in ventilated intensive care patients: a systematic evidence map

BACKGROUND

Ventilator weaning is of great importance for intensive care patients in order to avoid complications caused by prolonged ventilation. However, not all patients succeed in weaning immediately. Their spontaneous breathing may be insufficient, resulting in extubation failure and the subsequent need for reintubation. To identify patients at high risk for weaning failure, a variety of potential predictors has already been examined in individual studies and meta-analyses over the last decades. However, an overview of all the predictors investigated is missing.

AIM

To provide an overview of empirically investigated predictors for weaning failure.

METHODS

A systematic evidence map was developed. To this end, we conducted a systematic search in the Medline, Cochrane, and CINAHL databases in December 2023 and added a citation search and a manual search in June 2024. Studies on predictors for weaning failure in adults ventilated in the intensive care unit were included. Studies on children, outpatients, non-invasive ventilation, or explanatory factors of weaning failure were excluded. Two reviewers performed the screening and data extraction independently. Data synthesis followed an inductive approach in which the predictors were thematically analyzed, sorted, and clustered.

RESULTS

Of the 1388 records obtained, 140 studies were included in the analysis. The 112 prospective and 28 retrospective studies investigated a total of 145 predictors. These were assigned to the four central clusters 'Imaging procedures' (n = 22), 'Physiological parameters' (n = 61), 'Scores and indices' (n = 53), and 'Machine learning models' (n = 9). The most frequently investigated predictors are the rapid shallow breathing index, the diaphragm thickening fraction, the respiratory rate, the P/F ratio, and the diaphragm excursion.

CONCLUSION

Predictors for weaning failure are widely researched. To date, 145 predictors have been investigated with varying intensity in 140 studies that are in line with the current weaning definition. It is no longer just individual predictors that are investigated, but more comprehensive assessments, indices and machine learning models in the last decade. Future research should be conducted in line with international weaning definitions and further investigate poorly researched predictors. Registration, Protocol: https://doi.org/10.17605/OSF.IO/2KDYU.

Predicting ICU Interventions: A Transparent Decision Support Model Based on Multivariate Time Series Graph Convolutional Neural Network

In this study, we present a novel approach for predicting interventions for patients in the intensive care unit using a multivariate time series graph convolutional neural network. Our method addresses two critical challenges: the need for timely and accurate decisions based on changing physiological signals, drug administration information, and static characteristics; and the need for interpretability in the decision-making process. Drawing on real-world ICU records from the MIMIC-III dataset, we demonstrate that our approach significantly improves upon existing machine learning and deep learning methods for predicting two targeted interventions, mechanical ventilation and vasopressors. Our model achieved an accuracy improvement from 81.6% to 91.9% and a F1 score improvement from 0.524 to 0.606 for predicting mechanical ventilation interventions. For predicting vasopressor interventions, our model achieved an accuracy improvement from 76.3% to 82.7% and a F1 score improvement from 0.509 to 0.619. We also assessed the interpretability by performing an adjacency matrix importance analysis, which revealed that our model uses clinically meaningful and appropriate features for prediction. This critical aspect can help clinicians gain insights into the underlying mechanisms of interventions, allowing them to make more informed and precise clinical decisions. Overall, our study represents a significant step forward in the development of decision support systems for ICU patient care, providing a powerful tool for improving clinical outcomes and enhancing patient safety.

Performance of a Large Language Model on Japanese Emergency Medicine Board Certification Examinations

BACKGROUND

Emergency physicians need a broad range of knowledge and skills to address critical medical, traumatic, and environmental conditions. Artificial intelligence (AI), including large language models (LLMs), has potential applications in healthcare settings; however, the performance of LLMs in emergency medicine remains unclear.

METHODS

To evaluate the reliability of information provided by ChatGPT, an LLM was given the questions set by the Japanese Association of Acute Medicine in its board certification examinations over a period of 5 years (2018-2022) and programmed to answer them twice. Statistical analysis was used to assess agreement of the two responses.

RESULTS

The LLM successfully answered 465 of the 475 text-based questions, achieving an overall correct response rate of 62.3%. For questions without images, the rate of correct answers was 65.9%. For questions with images that were not explained to the LLM, the rate of correct answers was only 52.0%. The annual rates of correct answers to questions without images ranged from 56.3% to 78.8%. Accuracy was better for scenario-based questions (69.1%) than for stand-alone questions (62.1%). Agreement between the two responses was substantial (kappa = 0.70). Factual error accounted for 82% of the incorrectly answered questions.

CONCLUSION

An LLM performed satisfactorily on an emergency medicine board certification examination in Japanese and without images. However, factual errors in the responses highlight the need for physician oversight when using LLMs.

Usefulness of Self-Selected Scenarios for Simple Triage and Rapid Treatment Method Using Virtual Reality

BACKGROUND

Repeated triage training is necessary to maintain and improve the accuracy of simple triage and rapid treatment (START), a popular triage method. Virtual reality (VR) may be more effective than conventional training methods. This study aimed to verify the educational usefulness of START using VR originally developed for students.

METHODS

A VR was initially developed with a function that allowed students to select the triage procedure and its evaluation. Triage was performed using a simple modified START method, and eight scenarios were developed. The participants included 70 paramedic students classified into VR and live lecture groups. They took a 20-question written test that evaluated their academic ability before the course. After the course, a practical test and a 20-question written test were conducted. The total score of the practical test was 43 points. Triage procedure (1 point), observation and evaluation (1-5 points), and triage categories (1 point) were evaluated in this test.

RESULTS

The VR and live lecture groups consisted of 33 and 29 participants, respectively. No significant differences were observed pre- and post-test. In the practical test, the median (interquartile range) score was 29 (26-32) and 25 (23-29) for the VR and live lecture groups, respectively, with the VR group scoring significantly higher (P=0.03).

CONCLUSION

Our results confirmed the educational usefulness of selective VR for active learning of START. Therefore, VR combined with live lectures and simulations would be an optimal educational technique.

Using Artificial Intelligence to Predict Mechanical Ventilation Weaning Success in Patients with Respiratory Failure, Including Those with Acute Respiratory Distress Syndrome

The management of mechanical ventilation (MV) remains a challenge in intensive care units (ICUs). The digitalization of healthcare and the implementation of artificial intelligence (AI) and machine learning (ML) has significantly influenced medical decision-making capabilities, potentially enhancing patient outcomes. Acute respiratory distress syndrome, an overwhelming inflammatory lung disease, is common in ICUs. Most patients require MV. Prolonged MV is associated with an increased length of stay, morbidity, and mortality. Shortening the MV duration has both clinical and economic benefits and emphasizes the need for better MV weaning management. AI and ML models can assist the physician in weaning patients from MV by providing predictive tools based on big data. Many ML models have been developed in recent years, dealing with this unmet need. Such models provide an important prediction regarding the success of the individual patient's MV weaning. Some AI models have shown a notable impact on clinical outcomes. However, there are challenges in integrating AI models into clinical practice due to the unfamiliar nature of AI for many physicians and the complexity of some AI models. Our review explores the evolution of weaning methods up to and including AI and ML as weaning aids.

FT-GAT: Graph neural network for predicting spontaneous breathing trial success in patients with mechanical ventilation

BACKGROUND AND OBJECTIVES

Intensive care unit (ICU) physicians perform weaning procedures considering complex clinical situations and weaning protocols; however, liberating critical patients from mechanical ventilation (MV) remains challenging. Therefore, this study aims to aid physicians in deciding the early liberation of patients from MV by developing an artificial intelligence model that predicts the success of spontaneous breathing trials (SBT).

METHODS

We retrospectively collected data of 652 critical patients (SBT success: 641, SBT failure: 400) who received MV at the Chungbuk National University Hospital (CBNUH) ICU from July 2020 to July 2022, including mixed and trauma ICUs. Patients underwent SBTs according to the CBNUH weaning protocol or physician's decision, and SBT success was defined as extubation performed by the physician on the SBT day. Additionally, our dataset comprised 11 numerical and 2 categorical features that can be obtained for any ICU patient, such as vital signs and MV setting values. To predict SBT success, we analyzed tabular data using a graph neural network-based approach. Specifically, the graph structure was designed considering feature correlation, and a novel deep learning model, called feature tokenizer graph attention network (FT-GAT), was developed for graph analysis. FT-GAT transforms the input features into high-dimensional embeddings and analyzes the graph via the attention mechanism.

RESULTS

The quantitative evaluation results indicated that FT-GAT outperformed conventional models and clinical indicators by achieving the following model performance (AUROC): FT-GAT (0.80), conventional models (0.69-0.79), and clinical indicators (0.65-0.66) CONCLUSIONS: Through timely detection critical patients who can succeed in SBTs, FT-GAT can help prevent long-term use of MV and potentially lead to improvement in patient outcomes.

Comprehensive breathing variability indices enhance the prediction of extubation failure in patients on mechanical ventilation

BACKGROUND AND OBJECTIVE

Despite the numerous studies on extubation readiness assessment for patients who are invasively ventilated in the intensive care unit, a 10-15% extubation failure rate persists. Although breathing variability has been proposed as a potential predictor of extubation failure, it is mainly assessed using simple statistical metrics applied to basic respiratory parameters. Therefore, the complex pattern of breathing variability conveyed by continuous ventilation waveforms may be underexplored.

METHODS

Here, we aimed to develop novel breathing variability indices to predict extubation failure among invasively ventilated patients. First, breath-to-breath basic and comprehensive respiratory parameters were computed from continuous ventilation waveforms 1 h before extubation. Subsequently, the basic and advanced variability methods were applied to the respiratory parameter sequences to derive comprehensive breathing variability indices, and their role in predicting extubation failure was assessed. Finally, after reducing the feature dimensionality using the forward search method, the combined effect of the indices was evaluated by inputting them into the machine learning models, including logistic regression, random forest, support vector machine, and eXtreme Gradient Boosting (XGBoost).

RESULTS

The coefficient of variation of the dynamic mechanical power per breath (CV-MPd[J/breath]) exhibited the highest area under the receiver operating characteristic curve (AUC) of 0.777 among the individual indices. Furthermore, the XGBoost model obtained the best AUC (0.902) by combining multiple selected variability indices.

CONCLUSIONS

These results suggest that the proposed novel breathing variability indices can improve extubation failure prediction in invasively ventilated patients.

Machine learning for prediction of in-hospital mortality in lung cancer patients admitted to intensive care unit

BACKGROUNDS

The in-hospital mortality in lung cancer patients admitted to intensive care unit (ICU) is extremely high. This study intended to adopt machine learning algorithm models to predict in-hospital mortality of critically ill lung cancer for providing relative information in clinical decision-making.

METHODS

Data were extracted from the Medical Information Mart for Intensive Care-IV (MIMIC-IV) for a training cohort and data extracted from the Medical Information Mart for eICU Collaborative Research Database (eICU-CRD) database for a validation cohort. Logistic regression, random forest, decision tree, light gradient boosting machine (LightGBM), eXtreme gradient boosting (XGBoost), and an ensemble (random forest+LightGBM+XGBoost) model were used for prediction of in-hospital mortality and important feature extraction. The AUC (area under receiver operating curve), accuracy, F1 score and recall were used to evaluate the predictive performance of each model. Shapley Additive exPlanations (SHAP) values were calculated to evaluate feature importance of each feature.

RESULTS

Overall, there were 653 (24.8%) in-hospital mortality in the training cohort, and 523 (21.7%) in-hospital mortality in the validation cohort. Among the six machine learning models, the ensemble model achieved the best performance. The top 5 most influential features were the sequential organ failure assessment (SOFA) score, albumin, the oxford acute severity of illness score (OASIS) score, anion gap and bilirubin in random forest and XGBoost model. The SHAP summary plot was used to illustrate the positive or negative effects of the top 15 features attributed to the XGBoost model.

CONCLUSION

The ensemble model performed best and might be applied to forecast in-hospital mortality of critically ill lung cancer patients, and the SOFA score was the most important feature in all models. These results might offer valuable and significant reference for ICU clinicians' decision-making in advance.

An artificial intelligence system to predict the optimal timing for mechanical ventilation weaning for intensive care unit patients: A two-stage prediction approach

Background

For the intensivists, accurate assessment of the ideal timing for successful weaning from the mechanical ventilation (MV) in the intensive care unit (ICU) is very challenging.

Purpose

Using artificial intelligence (AI) approach to build two-stage predictive models, namely, the try-weaning stage and weaning MV stage to determine the optimal timing of weaning from MV for ICU intubated patients, and implement into practice for assisting clinical decision making.

Methods

AI and machine learning (ML) technologies were used to establish the predictive models in the stages. Each stage comprised 11 prediction time points with 11 prediction models. Twenty-five features were used for the first-stage models while 20 features were used for the second-stage models. The optimal models for each time point were selected for further practical implementation in a digital dashboard style. Seven machine learning algorithms including Logistic Regression (LR), Random Forest (RF), Support Vector Machines (SVM), K Nearest Neighbor (KNN), lightGBM, XGBoost, and Multilayer Perception (MLP) were used. The electronic medical records of the intubated ICU patients of Chi Mei Medical Center (CMMC) from 2016 to 2019 were included for modeling. Models with the highest area under the receiver operating characteristic curve (AUC) were regarded as optimal models and used to develop the prediction system accordingly.

Results

A total of 5,873 cases were included in machine learning modeling for Stage 1 with the AUCs of optimal models ranging from 0.843 to 0.953. Further, 4,172 cases were included for Stage 2 with the AUCs of optimal models ranging from 0.889 to 0.944. A prediction system (dashboard) with the optimal models of the two stages was developed and deployed in the ICU setting. Respiratory care members expressed high recognition of the AI dashboard assisting ventilator weaning decisions. Also, the impact analysis of with- and without-AI assistance revealed that our AI models could shorten the patients' intubation time by 21 hours, besides gaining the benefit of substantial consistency between these two decision-making strategies.

Conclusion

We noticed that the two-stage AI prediction models could effectively and precisely predict the optimal timing to wean intubated patients in the ICU from ventilator use. This could reduce patient discomfort, improve medical quality, and lower medical costs. This AI-assisted prediction system is beneficial for clinicians to cope with a high demand for ventilators during the COVID-19 pandemic.

Dynamic prediction of life-threatening events for patients in intensive care unit

Background

Early prediction of patients’ deterioration is helpful in early intervention for patients at greater risk of deterioration in Intensive Care Unit (ICU). This study aims to apply machine learning approaches to heterogeneous clinical data for predicting life-threatening events of patients in ICU.

Methods

We collected clinical data from a total of 3151 patients admitted to the Medical Intensive Care Unit of Peking Union Medical College Hospital in China from January 1st, 2014, to October 1st, 2019. After excluding the patients who were under 18 years old or stayed less than 24 h at the ICU, a total of 2170 patients were enrolled in this study. Multiple machine learning approaches were utilized to predict life-threatening events (i.e., death) in seven 24-h windows (day 1 to day 7) and their performance was compared.

Results

Light Gradient Boosting Machine showed the best performance. We found that life-threatening events during the short-term windows can be better predicted than those in the medium-term windows. For example, death in 24 h can be predicted with an Area Under Curve of 0.905. Features like infusion pump related fluid input were highly related to life-threatening events. Furthermore, the prediction power of static features such as age and cardio-pulmonary function increased with the extended prediction window.

Conclusion

This study demonstrates that the integration of machine learning approaches and large-scale high-quality clinical data in ICU could accurately predict life-threatening events for ICU patients for early intervention.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-02026-x.

Machine learning for predicting successful extubation in patients receiving mechanical ventilation

Ventilator liberation is one of the most critical decisions in the intensive care unit; however, prediction of extubation failure is difficult, and the proportion thereof remains high. Machine learning can potentially provide a breakthrough in the prediction of extubation success. A total of seven studies on the prediction of extubation success using machine learning have been published. These machine learning models were developed using data from electronic health records, 8–78 features, and algorithms such as artificial neural network, LightGBM, and XGBoost. Sensitivity ranged from 0.64 to 0.96, specificity ranged from 0.73 to 0.85, and area under the receiver operating characteristic curve ranged from 0.70 to 0.98. The features deemed most important included duration of mechanical ventilation, PaO₂, blood urea nitrogen, heart rate, and Glasgow Coma Scale score. Although the studies had limitations, prediction of extubation success by machine learning has the potential to be a powerful tool. Further studies are needed to assess whether machine learning prediction reduces the incidence of extubation failure or prolongs the duration of ventilator use, thereby increasing tracheostomy and ventilator-related complications and mortality.

A Simple Algorithm Using Ventilator Parameters to Predict Successfully Rapid Weaning Program in Cardiac Intensive Care Unit Patients

BACKGROUND

Ventilator weaning is one of the most significant challenges in the intensive care unit (ICU). Approximately 30% of patients fail to wean, resulting in prolonged use of ventilators and increased mortality. There are numerous high-performance prediction models available today, but they require a large number of parameters to predict and are thus impractical in clinical practice.

OBJECTIVES

This study aims to create an artificial intelligence (AI) model for predicting weaning time and to identify the most simplified key predictors that will allow the model to achieve adequate accuracy with as few parameters as possible.

METHODS

This is a retrospective study of to-be-weaned patients (n = 1439) hospitalized in the cardiac ICU of Cheng Hsin General Hospital's Department of Cardiac Surgery from November 2018 to August 2020. The patients were divided into two groups based on whether they could be weaned within 24 h (i.e., "patients weaned within 24 h" (n = 1042) and "patients not weaned within 24 h" (n = 397)). Twenty-eight variables were collected including demographic characteristics, arterial blood gas readings, and ventilation set parameters. We created a prediction model using logistic regression and compared it to other machine learning techniques such as decision tree, random forest, support vector machine (SVM), extreme gradient boosting, and artificial neural network. Forward, backward, and stepwise selection methods were used to identify significant variables, and the receiver operating characteristic curve was used to assess the accuracy of each AI model.

RESULTS

The SVM [receiver operating characteristic curve (ROC-AUC) = 88%], logistic regression (ROC-AUC = 86%), and XGBoost (ROC-AUC = 85%) models outperformed the other five machine learning models in predicting weaning time. The accuracies in predicting patient weaning within 24 h using seven variables (i.e., expiratory minute ventilation, expiratory tidal volume, ventilation rate set, heart rate, peak pressure, pH, and age) were close to those using 28 variables.

CONCLUSIONS

The model developed in this research successfully predicted the weaning success of ICU patients using a few and easily accessible parameters such as age. Therefore, it can be used in clinical practice to identify difficult-to-wean patients to improve their treatment.

Machine Learning and Antibiotic Management

Machine learning and cluster analysis applied to the clinical setting of an intensive care unit can be a valuable aid for clinical management, especially with the increasing complexity of clinical monitoring. Providing a method to measure clinical experience, a proxy for that automatic gestalt evaluation that an experienced clinician sometimes effortlessly, but often only after long, hard consideration and consultation with colleagues, relies upon for decision making, is what we wanted to achieve with the application of machine learning to antibiotic therapy and clinical monitoring in the present work. This is a single-center retrospective analysis proposing methods for evaluation of vitals and antimicrobial therapy in intensive care patients. For each patient included in the present study, duration of antibiotic therapy, consecutive days of treatment and type and combination of antimicrobial agents have been assessed and considered as single unique daily record for analysis. Each parameter, composing a record was normalized using a fuzzy logic approach and assigned to five descriptive categories (fuzzy domain sub-sets ranging from “very low” to “very high”). Clustering of these normalized therapy records was performed, and each patient/day was considered to be a pertaining cluster. The same methodology was used for hourly bed-side monitoring. Changes in patient conditions (monitoring) can lead to a shift of clusters. This can provide an additional tool for assessing progress of complex patients. We used Fuzzy logic normalization to descriptive categories of parameters as a form nearer to human language than raw numbers.

Predictors for extubation failure in COVID-19 patients using a machine learning approach

INTRODUCTION

Determining the optimal timing for extubation can be challenging in the intensive care. In this study, we aim to identify predictors for extubation failure in critically ill patients with COVID-19.

METHODS

We used highly granular data from 3464 adult critically ill COVID patients in the multicenter Dutch Data Warehouse, including demographics, clinical observations, medications, fluid balance, laboratory values, vital signs, and data from life support devices. All intubated patients with at least one extubation attempt were eligible for analysis. Transferred patients, patients admitted for less than 24 h, and patients still admitted at the time of data extraction were excluded. Potential predictors were selected by a team of intensive care physicians. The primary and secondary outcomes were extubation without reintubation or death within the next 7 days and within 48 h, respectively. We trained and validated multiple machine learning algorithms using fivefold nested cross-validation. Predictor importance was estimated using Shapley additive explanations, while cutoff values for the relative probability of failed extubation were estimated through partial dependence plots.

RESULTS

A total of 883 patients were included in the model derivation. The reintubation rate was 13.4% within 48 h and 18.9% at day 7, with a mortality rate of 0.6% and 1.0% respectively. The grandient-boost model performed best (area under the curve of 0.70) and was used to calculate predictor importance. Ventilatory characteristics and settings were the most important predictors. More specifically, a controlled mode duration longer than 4 days, a last fraction of inspired oxygen higher than 35%, a mean tidal volume per kg ideal body weight above 8 ml/kg in the day before extubation, and a shorter duration in assisted mode (< 2 days) compared to their median values. Additionally, a higher C-reactive protein and leukocyte count, a lower thrombocyte count, a lower Glasgow coma scale and a lower body mass index compared to their medians were associated with extubation failure.

CONCLUSION

The most important predictors for extubation failure in critically ill COVID-19 patients include ventilatory settings, inflammatory parameters, neurological status, and body mass index. These predictors should therefore be routinely captured in electronic health records.

Biosignal-Based Digital Biomarkers for Prediction of Ventilator Weaning Success

We evaluated new features from biosignals comprising diverse physiological response information to predict the outcome of weaning from mechanical ventilation (MV). We enrolled 89 patients who were candidates for weaning from MV in the intensive care unit and collected continuous biosignal data: electrocardiogram (ECG), respiratory impedance, photoplethysmogram (PPG), arterial blood pressure, and ventilator parameters during a spontaneous breathing trial (SBT). We compared the collected biosignal data's variability between patients who successfully discontinued MV (n = 67) and patients who did not (n = 22). To evaluate the usefulness of the identified factors for predicting weaning success, we developed a machine learning model and evaluated its performance by bootstrapping. The following markers were different between the weaning success and failure groups: the ratio of standard deviations between the short-term and long-term heart rate variability in a Poincaré plot, sample entropy of ECG and PPG, α values of ECG, and respiratory impedance in the detrended fluctuation analysis. The area under the receiver operating characteristic curve of the model was 0.81 (95% confidence interval: 0.70-0.92). This combination of the biosignal data-based markers obtained during SBTs provides a promising tool to assist clinicians in determining the optimal extubation time.