Machine learning predicts the short-term requirement for invasive ventilation among Australian critically ill COVID-19 patients

Derivation and external validation of predictive models for invasive mechanical ventilation in intensive care unit patients with COVID-19

BACKGROUND

This study aimed to develop prognostic models for predicting the need for invasive mechanical ventilation (IMV) in intensive care unit (ICU) patients with COVID-19 and compare their performance with the Respiratory rate-OXygenation (ROX) index.

METHODS

A retrospective cohort study was conducted using data collected between March 2020 and August 2021 at three hospitals in Rio de Janeiro, Brazil. ICU patients aged 18 years and older with a diagnosis of COVID-19 were screened. The exclusion criteria were patients who received IMV within the first 24 h of ICU admission, pregnancy, clinical decision for minimal end-of-life care and missing primary outcome data. Clinical and laboratory variables were collected. Multiple logistic regression analysis was performed to select predictor variables. Models were based on the lowest Akaike Information Criteria (AIC) and lowest AIC with significant p values. Assessment of predictive performance was done for discrimination and calibration. Areas under the curves (AUC)s were compared using DeLong's algorithm. Models were validated externally using an international database.

RESULTS

Of 656 patients screened, 346 patients were included; 155 required IMV (44.8%), 191 did not (55.2%), and 207 patients were male (59.8%). According to the lowest AIC, arterial hypertension, diabetes mellitus, obesity, Sequential Organ Failure Assessment (SOFA) score, heart rate, respiratory rate, peripheral oxygen saturation (SpO2), temperature, respiratory effort signals, and leukocytes were identified as predictors of IMV at hospital admission. According to AIC with significant p values, SOFA score, SpO2, and respiratory effort signals were the best predictors of IMV; odds ratios (95% confidence interval): 1.46 (1.07-2.05), 0.81 (0.72-0.90), 9.13 (3.29-28.67), respectively. The ROX index at admission was lower in the IMV group than in the non-IMV group (7.3 [5.2-9.8] versus 9.6 [6.8-12.9], p < 0.001, respectively). In the external validation population, the area under the curve (AUC) of the ROX index was 0.683 (accuracy 63%), the AIC model showed an AUC of 0.703 (accuracy 69%), and the lowest AIC model with significant p values had an AUC of 0.725 (accuracy 79%).

CONCLUSIONS

In the development population of ICU patients with COVID-19, SOFA score, SpO2, and respiratory effort signals predicted the need for IMV better than the ROX index. In the external validation population, although the AUCs did not differ significantly, the accuracy was higher when using SOFA score, SpO2, and respiratory effort signals compared to the ROX index. This suggests that these variables may be more useful in predicting the need for IMV in ICU patients with COVID-19.

CLINICALTRIALS

GOV IDENTIFIER

NCT05663528.

Predicting invasive mechanical ventilation in COVID 19 patients: A validation study

INTRODUCTION

The decision to intubate and ventilate a patient is mainly clinical. Both delaying intubation (when needed) and unnecessarily invasively ventilating (when it can be avoided) are harmful. We recently developed an algorithm predicting respiratory failure and invasive mechanical ventilation in COVID-19 patients. This is an internal validation study of this model, which also suggests a categorized "time-weighted" model.

METHODS

We used a dataset of COVID-19 patients who were admitted to Rabin Medical Center after the algorithm was developed. We evaluated model performance in predicting ventilation, regarding the actual endpoint of each patient. We further categorized each patient into one of four categories, based on the strength of the prediction of ventilation over time. We evaluated this categorized model performance regarding the actual endpoint of each patient.

RESULTS

881 patients were included in the study; 96 of them were ventilated. AUC of the original algorithm is 0.87-0.94. The AUC of the categorized model is 0.95.

CONCLUSIONS

A minor degradation in the algorithm accuracy was noted in the internal validation, however, its accuracy remained high. The categorized model allows accurate prediction over time, with very high negative predictive value.

Machine Learning for Benchmarking Critical Care Outcomes

OBJECTIVES

Enhancing critical care efficacy involves evaluating and improving system functioning. Benchmarking, a retrospective comparison of results against standards, aids risk-adjusted assessment and helps healthcare providers identify areas for improvement based on observed and predicted outcomes. The last two decades have seen the development of several models using machine learning (ML) for clinical outcome prediction. ML is a field of artificial intelligence focused on creating algorithms that enable computers to learn from and make predictions or decisions based on data. This narrative review centers on key discoveries and outcomes to aid clinicians and researchers in selecting the optimal methodology for critical care benchmarking using ML.

METHODS

We used PubMed to search the literature from 2003 to 2023 regarding predictive models utilizing ML for mortality (592 articles), length of stay (143 articles), or mechanical ventilation (195 articles). We supplemented the PubMed search with Google Scholar, making sure relevant articles were included. Given the narrative style, papers in the cohort were manually curated for a comprehensive reader perspective.

RESULTS

Our report presents comparative results for benchmarked outcomes and emphasizes advancements in feature types, preprocessing, model selection, and validation. It showcases instances where ML effectively tackled critical care outcome-prediction challenges, including nonlinear relationships, class imbalances, missing data, and documentation variability, leading to enhanced results.

CONCLUSIONS

Although ML has provided novel tools to improve the benchmarking of critical care outcomes, areas that require further research include class imbalance, fairness, improved calibration, generalizability, and long-term validation of published models.