External validation in an intermediate unit of a respiratory decompensation model trained in an intensive care unit

Predicting respiratory decompensation in mechanically ventilated adult ICU patients

Introduction: Mechanical ventilation is a life-saving treatment in the Intensive Care Unit (ICU), but often causes patients to be at risk of further respiratory complication. We created a statistical model utilizing electronic health record and physiologic vitals data to predict the Center for Disease Control and Prevention (CDC) defined Ventilator Associated Complications (VACs). Further, we evaluated the effect of data temporal resolution and feature generation method choice on the accuracy of such a constructed model. Methods: We constructed a random forest model to predict occurrence of VACs using health records and chart events from adult patients in the Medical Information Mart for Intensive Care III (MIMIC-III) database. We trained the machine learning models on two patient populations of 1921 and 464 based on low and high frequency data availability. Model features were generated using both basic statistical summaries and tsfresh, a python library that generates a large number of derived time-series features. Classification to determine whether a patient will experience VAC one hour after 35 h of ventilation was performed using a random forest classifier. Two different sample spaces conditioned on five varying feature extraction techniques were evaluated to identify the most optimal selection of features resulting in the best VAC discrimination. Each dataset was assessed using K-folds cross-validation (k = 10), giving average area under the receiver operating characteristic curves (AUROCs) and accuracies. Results: After feature selection, hyperparameter tuning, and feature extraction, the best performing model used automatically generated features on high frequency data and achieved an average AUROC of 0.83 ± 0.11 and an average accuracy of 0.69 ± 0.10. Discussion: Results show the potential viability of predicting VACs using machine learning, and indicate that higher-resolution data and the larger feature set generated by tsfresh yield better AUROCs compared to lower-resolution data and manual statistical features.

Beyond prediction: Off-target uses of artificial intelligence-based predictive analytics in a learning health system

Introduction

Artificial-intelligence (AI)-based predictive analytics provide new opportunities to leverage rich sources of continuous data to improve patient care through early warning of the risk of clinical deterioration and improved situational awareness.Part of the success of predictive analytic implementation relies on integration of the analytic within complex clinical workflows. Pharmaceutical interventions have off-target uses where a drug indication has not been formally studied for a different indication but has potential for clinical benefit. An analog has not been described in the context of AI-based predictive analytics, that is, when a predictive analytic has been trained on one outcome of interest but is used for additional applications in clinical practice.

Methods

In this manuscript we present three clinical vignettes describing off-target use of AI-based predictive analytics that evolved organically through real-world practice.

Results

Off-target uses included:real-time feedback about treatment effectiveness, indication of readiness to discharge, and indication of the acuity of a hospital unit.

Conclusion

Such practice fits well with the learning health system goals to continuously integrate data and experience to provide.

Nursing and precision predictive analytics monitoring in the acute and intensive care setting: An emerging role for responding to COVID-19 and beyond

As the global response to COVID-19 continues, nurses will be tasked with appropriately triaging patients, responding to events of clinical deterioration, and developing family-centered plans of care within a healthcare system exceeding capacity. Predictive analytics monitoring, an artificial intelligence (AI)-based tool that translates streaming clinical data into a real-time visual estimation of patient risks, allows for evolving acuity assessments and detection of clinical deterioration while the patient is in pre-symptomatic states. While nurses are on the frontline for the COVID-19 pandemic, the use of AI-based predictive analytics monitoring may help cognitively complex clinical decision-making tasks and pave a pathway for early detection of patients at risk for decompensation. We must develop strategies and techniques to study the impact of AI-based technologies on patient care outcomes and the clinical workflow. This paper outlines key concepts for the intersection of nursing and precision predictive analytics monitoring.

External validation of a novel signature of illness in continuous cardiorespiratory monitoring to detect early respiratory deterioration of ICU patients

Objective:

The goal of predictive analytics monitoring is the early detection of patients at high risk of subacute potentially catastrophic illnesses. An excellent example of a targeted illness is respiratory failure leading to urgent unplanned intubation, where early detection might lead to interventions that improve patient outcomes. Previously, we identified signatures of this illness in the continuous cardiorespiratory monitoring data of intensive care unit (ICU) patients and devised algorithms to identify patients at rising risk. Here, we externally validated three logistic regression models to estimate the risk of emergency intubation developed in Medical and Surgical ICUs at the University of Virginia.

Approach:

We calculated the model outputs for more than 8000 patients in the University of California—San Francisco ICUs, 240 of whom underwent emergency intubation as determined by individual chart review.

Main results:

We found that the AUC of the models exceeded 0.75 in this external population, and that the risk rose appreciably over the 12 h before the event.

Significance:

We conclude that there are generalizable physiological signatures of impending respiratory failure in the continuous cardiorespiratory monitoring data.

Predicting the need for intubation in the first 24 h after critical care admission using machine learning approaches

Early and accurate prediction of the need for intubation may provide more time for preparation and increase safety margins by avoiding high risk late intubation. This study evaluates whether machine learning can predict the need for intubation within 24 h using commonly available bedside and laboratory parameters taken at critical care admission. We extracted data from 2 large critical care databases (MIMIC-III and eICU-CRD). Missing variables were imputed using autoencoder. Machine learning classifiers using logistic regression and random forest were trained using 60% of the data and tested using the remaining 40% of the data. We compared the performance of logistic regression and random forest models to predict intubation in critically ill patients. After excluding patients with limitations of therapy and missing data, we included 17,616 critically ill patients in this retrospective cohort. Within 24 h of admission, 2,292 patients required intubation, whilst 15,324 patients were not intubated. Blood gas parameters (P_aO₂, P_aCO₂, HCO₃⁻), Glasgow Coma Score, respiratory variables (respiratory rate, S_pO₂), temperature, age, and oxygen therapy were used to predict intubation. Random forest had AUC 0.86 (95% CI 0.85–0.87) and logistic regression had AUC 0.77 (95% CI 0.76–0.78) for intubation prediction performance. Random forest model had sensitivity of 0.88 (95% CI 0.86–0.90) and specificity of 0.66 (95% CI 0.63–0.69), with good calibration throughout the range of intubation risks. The results showed that machine learning could predict the need for intubation in critically ill patients using commonly collected bedside clinical parameters and laboratory results. It may be used in real-time to help clinicians predict the need for intubation within 24 h of intensive care unit admission.

Impact of predictive analytics based on continuous cardiorespiratory monitoring in a surgical and trauma intensive care unit

Predictive analytics monitoring, the use of patient data to provide continuous risk estimation of deterioration, is a promising new application of big data analytical techniques to the care of individual patients. We tested the hypothesis that continuous display of novel electronic risk visualization of respiratory and cardiovascular events would impact intensive care unit (ICU) patient outcomes. In an adult tertiary care surgical trauma ICU, we displayed risk estimation visualizations on a large monitor, but in the medical ICU in the same institution we did not. The risk estimates were based solely on analysis of continuous cardiorespiratory monitoring. We examined 4275 individual patient records within a 7 month time period preceding and following data display. We determined cases of septic shock, emergency intubation, hemorrhage, and death to compare rates per patient care pre-and post-implementation. Following implementation, the incidence of septic shock fell by half (p < 0.01 in a multivariate model that included age and APACHE) in the surgical trauma ICU, where the data were continuously on display, but by only 10% (p = NS) in the control Medical ICU. There were no significant changes in the other outcomes. Display of a predictive analytics monitor based on continuous cardiorespiratory monitoring was followed by a reduction in the rate of septic shock, even when controlling for age and APACHE score.

Advancing Continuous Predictive Analytics Monitoring: Moving from Implementation to Clinical Action in a Learning Health System

In the intensive care unit, clinicians monitor a diverse array of data inputs to detect early signs of impending clinical demise or improvement. Continuous predictive analytics monitoring synthesizes data from a variety of inputs into a risk estimate that clinicians can observe in a streaming environment. For this to be useful, clinicians must engage with the data in a way that makes sense for their clinical workflow in the context of a learning health system (LHS). This article describes the processes needed to evoke clinical action after initiation of continuous predictive analytics monitoring in an LHS.

Identifying the low risk patient in surgical intensive and intermediate care units using continuous monitoring

BACKGROUND

Continuous predictive monitoring has been employed successfully to predict subclinical adverse events. Should low values on these models, however, reassure us that a patient will not have an adverse outcome? Negative predictive values of such models could help predict safe patient discharge. The goal of this study was to validate the negative predictive value of an ensemble model for critical illness (using previously developed models for respiratory instability, hemorrhage, and sepsis) based on bedside monitoring data in the intensive care units and intermediate care unit.

METHODS

We calculated the relative risk of 3 critical illnesses for all patients every 15 minutes (n= 124,588) for 2,924 patients downgraded from the surgical intensive care units and intermediate care unit between May 2014 to May 2016. We constructed an ensemble model to estimate at the time of intensive care units or intermediate care unit discharge the probability of favorable outcome after downgrade.

RESULTS

Outputs form the ensemble model stratified patients by risk of favorable and bad outcomes in both intensive care units/intermediate care unit; area under the receiver operating characteristic curve = .639/.629 respectively for favorable outcomes and .645/.641 for adverse events. These performance characteristics are commensurate with published models for predicting readmission. The ensemble model remained a statistically significant predictor after adjusting for hospital duration of stay and admitting service. The rate of favorable outcome in the highest and lowest deciles in the intensive care units were 76.2% and 27.3% (2.8-fold decrease) and 88.3% and 33.2% in the intermediate care unit (2.7-fold decrease), respectively.

CONCLUSION

An ensemble model for critical illness predicts favorable outcome after downgrade and safe patient discharge (hospital stay <7 days, no readmission, upgrade, or death).