1
|
Tan Y, Young M, Girish A, Hu B, Kurian Z, Greenstein JL, Kim H, Winslow RL, Fackler J, Bergmann J. Predicting respiratory decompensation in mechanically ventilated adult ICU patients. Front Physiol 2023; 14:1125991. [PMID: 37123253 PMCID: PMC10140580 DOI: 10.3389/fphys.2023.1125991] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Accepted: 04/06/2023] [Indexed: 05/02/2023] Open
Abstract
Introduction: Mechanical ventilation is a life-saving treatment in the Intensive Care Unit (ICU), but often causes patients to be at risk of further respiratory complication. We created a statistical model utilizing electronic health record and physiologic vitals data to predict the Center for Disease Control and Prevention (CDC) defined Ventilator Associated Complications (VACs). Further, we evaluated the effect of data temporal resolution and feature generation method choice on the accuracy of such a constructed model. Methods: We constructed a random forest model to predict occurrence of VACs using health records and chart events from adult patients in the Medical Information Mart for Intensive Care III (MIMIC-III) database. We trained the machine learning models on two patient populations of 1921 and 464 based on low and high frequency data availability. Model features were generated using both basic statistical summaries and tsfresh, a python library that generates a large number of derived time-series features. Classification to determine whether a patient will experience VAC one hour after 35 h of ventilation was performed using a random forest classifier. Two different sample spaces conditioned on five varying feature extraction techniques were evaluated to identify the most optimal selection of features resulting in the best VAC discrimination. Each dataset was assessed using K-folds cross-validation (k = 10), giving average area under the receiver operating characteristic curves (AUROCs) and accuracies. Results: After feature selection, hyperparameter tuning, and feature extraction, the best performing model used automatically generated features on high frequency data and achieved an average AUROC of 0.83 ± 0.11 and an average accuracy of 0.69 ± 0.10. Discussion: Results show the potential viability of predicting VACs using machine learning, and indicate that higher-resolution data and the larger feature set generated by tsfresh yield better AUROCs compared to lower-resolution data and manual statistical features.
Collapse
Affiliation(s)
- Yvette Tan
- Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Michael Young
- Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Akanksha Girish
- Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Beini Hu
- Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Zina Kurian
- Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Joseph L. Greenstein
- Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
- Institute for Computational Medicine, Johns Hopkins University, Baltimore, MD, United States
| | - Han Kim
- Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Raimond L Winslow
- Institute for Computational Medicine, Johns Hopkins University, Baltimore, MD, United States
| | - James Fackler
- Department of Anesthesiology and Critical Care Medicine, School of Medicine, Johns Hopkins University, Baltimore, MD, United States
| | - Jules Bergmann
- Department of Anesthesiology and Critical Care Medicine, School of Medicine, Johns Hopkins University, Baltimore, MD, United States
- *Correspondence: Jules Bergmann,
| |
Collapse
|
2
|
Keim‐Malpass J, Moorman LP, Monfredi OJ, Clark MT, Bourque JM. Beyond prediction: Off-target uses of artificial intelligence-based predictive analytics in a learning health system. Learn Health Syst 2023; 7:e10323. [PMID: 36654806 PMCID: PMC9835046 DOI: 10.1002/lrh2.10323] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2021] [Revised: 06/03/2022] [Accepted: 06/11/2022] [Indexed: 01/21/2023] Open
Abstract
Introduction Artificial-intelligence (AI)-based predictive analytics provide new opportunities to leverage rich sources of continuous data to improve patient care through early warning of the risk of clinical deterioration and improved situational awareness.Part of the success of predictive analytic implementation relies on integration of the analytic within complex clinical workflows. Pharmaceutical interventions have off-target uses where a drug indication has not been formally studied for a different indication but has potential for clinical benefit. An analog has not been described in the context of AI-based predictive analytics, that is, when a predictive analytic has been trained on one outcome of interest but is used for additional applications in clinical practice. Methods In this manuscript we present three clinical vignettes describing off-target use of AI-based predictive analytics that evolved organically through real-world practice. Results Off-target uses included:real-time feedback about treatment effectiveness, indication of readiness to discharge, and indication of the acuity of a hospital unit. Conclusion Such practice fits well with the learning health system goals to continuously integrate data and experience to provide.
Collapse
Affiliation(s)
- Jessica Keim‐Malpass
- School of NursingUniversity of VirginiaCharlottesvilleVirginiaUSA
- Center for Advanced Medical AnalyticsUniversity of VirginiaCharlottesvilleVirginiaUSA
| | | | - Oliver J. Monfredi
- Center for Advanced Medical AnalyticsUniversity of VirginiaCharlottesvilleVirginiaUSA
- Division of Cardiovascular Medicine, School of MedicineUniversity of VirginiaCharlottesvilleVirginiaUSA
| | | | - Jamieson M. Bourque
- Center for Advanced Medical AnalyticsUniversity of VirginiaCharlottesvilleVirginiaUSA
- Division of Cardiovascular Medicine, School of MedicineUniversity of VirginiaCharlottesvilleVirginiaUSA
| |
Collapse
|
3
|
Keim-Malpass J, Moorman LP. Nursing and precision predictive analytics monitoring in the acute and intensive care setting: An emerging role for responding to COVID-19 and beyond. INTERNATIONAL JOURNAL OF NURSING STUDIES ADVANCES 2021; 3:100019. [PMID: 33426534 PMCID: PMC7781904 DOI: 10.1016/j.ijnsa.2021.100019] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2020] [Revised: 12/16/2020] [Accepted: 12/29/2020] [Indexed: 12/23/2022] Open
Abstract
As the global response to COVID-19 continues, nurses will be tasked with appropriately triaging patients, responding to events of clinical deterioration, and developing family-centered plans of care within a healthcare system exceeding capacity. Predictive analytics monitoring, an artificial intelligence (AI)-based tool that translates streaming clinical data into a real-time visual estimation of patient risks, allows for evolving acuity assessments and detection of clinical deterioration while the patient is in pre-symptomatic states. While nurses are on the frontline for the COVID-19 pandemic, the use of AI-based predictive analytics monitoring may help cognitively complex clinical decision-making tasks and pave a pathway for early detection of patients at risk for decompensation. We must develop strategies and techniques to study the impact of AI-based technologies on patient care outcomes and the clinical workflow. This paper outlines key concepts for the intersection of nursing and precision predictive analytics monitoring.
Collapse
Affiliation(s)
- Jessica Keim-Malpass
- School of Nursing, Department of Acute and Specialty Care, University of Virginia, Charlottesville, VA, USA,Center for Advanced Medical Analytics, University of Virginia, Charlottesville, VA, USA,Corresponding author at: University of Virginia School of Nursing, P.O. Box 800782, Charlottesville, VA 22908 USA
| | - Liza P. Moorman
- AMP3D: Advanced Medical Predictive Devices, Diagnostics and Displays, Inc., Charlottesville, VA, USA
| |
Collapse
|
4
|
Callcut RA, Xu Y, Moorman JR, Tsai C, Villaroman A, Robles AJ, Lake DE, Hu X, Clark MT. External validation of a novel signature of illness in continuous cardiorespiratory monitoring to detect early respiratory deterioration of ICU patients. Physiol Meas 2021; 42. [PMID: 34580242 PMCID: PMC9548299 DOI: 10.1088/1361-6579/ac2264] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2021] [Accepted: 08/31/2021] [Indexed: 12/23/2022]
Abstract
Objective: The goal of predictive analytics monitoring is the early detection of patients at high risk of subacute potentially catastrophic illnesses. An excellent example of a targeted illness is respiratory failure leading to urgent unplanned intubation, where early detection might lead to interventions that improve patient outcomes. Previously, we identified signatures of this illness in the continuous cardiorespiratory monitoring data of intensive care unit (ICU) patients and devised algorithms to identify patients at rising risk. Here, we externally validated three logistic regression models to estimate the risk of emergency intubation developed in Medical and Surgical ICUs at the University of Virginia. Approach: We calculated the model outputs for more than 8000 patients in the University of California—San Francisco ICUs, 240 of whom underwent emergency intubation as determined by individual chart review. Main results: We found that the AUC of the models exceeded 0.75 in this external population, and that the risk rose appreciably over the 12 h before the event. Significance: We conclude that there are generalizable physiological signatures of impending respiratory failure in the continuous cardiorespiratory monitoring data.
Collapse
Affiliation(s)
- Rachael A Callcut
- University of California, Davis, Department of Surgery, Davis, CA, United States of America
| | - Yuan Xu
- University of California, San Francisco, Department of Surgery, San Francisco, CA, United States of America
| | - J Randall Moorman
- University of Virginia, UVa Center for Advanced Medical Analytics, Charlottesville, VA, United States of America.,University of Virginia, Cardiovascular Division, Charlottesville, VA, United States of America
| | - Christina Tsai
- University of California, San Francisco, Department of Surgery, San Francisco, CA, United States of America
| | - Andrea Villaroman
- University of California, San Francisco, Department of Surgery, San Francisco, CA, United States of America
| | - Anamaria J Robles
- University of California, San Francisco, Department of Surgery, San Francisco, CA, United States of America
| | - Douglas E Lake
- University of Virginia, UVa Center for Advanced Medical Analytics, Charlottesville, VA, United States of America.,University of Virginia, Cardiovascular Division, Charlottesville, VA, United States of America
| | - Xiao Hu
- Duke University, School of Nursing, United States of America
| | - Matthew T Clark
- University of Virginia, UVa Center for Advanced Medical Analytics, Charlottesville, VA, United States of America.,Advanced Medical Predictive Devices, Diagnostics, and Displays, Charlottesville, VA, United States of America
| |
Collapse
|
5
|
Predicting the need for intubation in the first 24 h after critical care admission using machine learning approaches. Sci Rep 2020; 10:20931. [PMID: 33262391 PMCID: PMC7708470 DOI: 10.1038/s41598-020-77893-3] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2020] [Accepted: 11/09/2020] [Indexed: 12/20/2022] Open
Abstract
Early and accurate prediction of the need for intubation may provide more time for preparation and increase safety margins by avoiding high risk late intubation. This study evaluates whether machine learning can predict the need for intubation within 24 h using commonly available bedside and laboratory parameters taken at critical care admission. We extracted data from 2 large critical care databases (MIMIC-III and eICU-CRD). Missing variables were imputed using autoencoder. Machine learning classifiers using logistic regression and random forest were trained using 60% of the data and tested using the remaining 40% of the data. We compared the performance of logistic regression and random forest models to predict intubation in critically ill patients. After excluding patients with limitations of therapy and missing data, we included 17,616 critically ill patients in this retrospective cohort. Within 24 h of admission, 2,292 patients required intubation, whilst 15,324 patients were not intubated. Blood gas parameters (PaO2, PaCO2, HCO3−), Glasgow Coma Score, respiratory variables (respiratory rate, SpO2), temperature, age, and oxygen therapy were used to predict intubation. Random forest had AUC 0.86 (95% CI 0.85–0.87) and logistic regression had AUC 0.77 (95% CI 0.76–0.78) for intubation prediction performance. Random forest model had sensitivity of 0.88 (95% CI 0.86–0.90) and specificity of 0.66 (95% CI 0.63–0.69), with good calibration throughout the range of intubation risks. The results showed that machine learning could predict the need for intubation in critically ill patients using commonly collected bedside clinical parameters and laboratory results. It may be used in real-time to help clinicians predict the need for intubation within 24 h of intensive care unit admission.
Collapse
|
6
|
Ruminski CM, Clark MT, Lake DE, Kitzmiller RR, Keim-Malpass J, Robertson MP, Simons TR, Moorman JR, Calland JF. Impact of predictive analytics based on continuous cardiorespiratory monitoring in a surgical and trauma intensive care unit. J Clin Monit Comput 2018; 33:703-711. [PMID: 30121744 DOI: 10.1007/s10877-018-0194-4] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2018] [Accepted: 08/02/2018] [Indexed: 01/10/2023]
Abstract
Predictive analytics monitoring, the use of patient data to provide continuous risk estimation of deterioration, is a promising new application of big data analytical techniques to the care of individual patients. We tested the hypothesis that continuous display of novel electronic risk visualization of respiratory and cardiovascular events would impact intensive care unit (ICU) patient outcomes. In an adult tertiary care surgical trauma ICU, we displayed risk estimation visualizations on a large monitor, but in the medical ICU in the same institution we did not. The risk estimates were based solely on analysis of continuous cardiorespiratory monitoring. We examined 4275 individual patient records within a 7 month time period preceding and following data display. We determined cases of septic shock, emergency intubation, hemorrhage, and death to compare rates per patient care pre-and post-implementation. Following implementation, the incidence of septic shock fell by half (p < 0.01 in a multivariate model that included age and APACHE) in the surgical trauma ICU, where the data were continuously on display, but by only 10% (p = NS) in the control Medical ICU. There were no significant changes in the other outcomes. Display of a predictive analytics monitor based on continuous cardiorespiratory monitoring was followed by a reduction in the rate of septic shock, even when controlling for age and APACHE score.
Collapse
Affiliation(s)
- Caroline M Ruminski
- University of Virginia School of Medicine, P.O. Box 800158, Charlottesville, VA, 22908, USA
| | - Matthew T Clark
- Advanced Medical Predictive Devices, Diagnostics, Displays (AMP3D), Charlottesville, VA, USA
| | - Douglas E Lake
- University of Virginia School of Medicine, P.O. Box 800158, Charlottesville, VA, 22908, USA
| | | | | | | | | | - J Randall Moorman
- University of Virginia School of Medicine, P.O. Box 800158, Charlottesville, VA, 22908, USA.
| | - J Forrest Calland
- University of Virginia School of Medicine, P.O. Box 800158, Charlottesville, VA, 22908, USA
| |
Collapse
|
7
|
Keim-Malpass J, Kitzmiller RR, Skeeles-Worley A, Lindberg C, Clark MT, Tai R, Calland JF, Sullivan K, Randall Moorman J, Anderson RA. Advancing Continuous Predictive Analytics Monitoring: Moving from Implementation to Clinical Action in a Learning Health System. Crit Care Nurs Clin North Am 2018; 30:273-287. [PMID: 29724445 DOI: 10.1016/j.cnc.2018.02.009] [Citation(s) in RCA: 26] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
In the intensive care unit, clinicians monitor a diverse array of data inputs to detect early signs of impending clinical demise or improvement. Continuous predictive analytics monitoring synthesizes data from a variety of inputs into a risk estimate that clinicians can observe in a streaming environment. For this to be useful, clinicians must engage with the data in a way that makes sense for their clinical workflow in the context of a learning health system (LHS). This article describes the processes needed to evoke clinical action after initiation of continuous predictive analytics monitoring in an LHS.
Collapse
Affiliation(s)
- Jessica Keim-Malpass
- Department of Acute and Specialty Care, School of Nursing, University of Virginia, PO Box 800782, Charlottesville, VA 22908, USA; Department of Medicine, School of Medicine, University of Virginia, 1215 Lee Street, Charlottesville, VA 22908, USA.
| | - Rebecca R Kitzmiller
- School of Nursing, University of North Carolina, Carrington Hall, South Columbia Street, Chapel Hill, NC 27599, USA
| | - Angela Skeeles-Worley
- School of Education, University of Virginia, 405 Emmet Street South, Charlottesville, VA 22903, USA
| | - Curt Lindberg
- Billings Clinic, 801 North 29th Street, Billings, MT 59101, USA
| | - Matthew T Clark
- Advanced Medical Predictive Devices, Diagnostics, Displays, Charlottesville, VA 22903, USA
| | - Robert Tai
- School of Education, University of Virginia, 405 Emmet Street South, Charlottesville, VA 22903, USA
| | - James Forrest Calland
- Department of Medicine, School of Medicine, University of Virginia, 1215 Lee Street, Charlottesville, VA 22908, USA
| | - Kevin Sullivan
- Department of Computer Science, School of Engineering, University of Virginia, Engineer's Way, Charlottesville, VA 22903, USA
| | - J Randall Moorman
- Department of Medicine, School of Medicine, University of Virginia, 1215 Lee Street, Charlottesville, VA 22908, USA; Advanced Medical Predictive Devices, Diagnostics, Displays, Charlottesville, VA 22903, USA
| | - Ruth A Anderson
- School of Nursing, University of North Carolina, Carrington Hall, South Columbia Street, Chapel Hill, NC 27599, USA
| |
Collapse
|
8
|
Blackburn HN, Clark MT, Moorman JR, Lake DE, Calland JF. Identifying the low risk patient in surgical intensive and intermediate care units using continuous monitoring. Surgery 2018; 163:811-818. [PMID: 29433853 DOI: 10.1016/j.surg.2017.08.022] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2017] [Revised: 07/27/2017] [Accepted: 08/30/2017] [Indexed: 12/23/2022]
Abstract
BACKGROUND Continuous predictive monitoring has been employed successfully to predict subclinical adverse events. Should low values on these models, however, reassure us that a patient will not have an adverse outcome? Negative predictive values of such models could help predict safe patient discharge. The goal of this study was to validate the negative predictive value of an ensemble model for critical illness (using previously developed models for respiratory instability, hemorrhage, and sepsis) based on bedside monitoring data in the intensive care units and intermediate care unit. METHODS We calculated the relative risk of 3 critical illnesses for all patients every 15 minutes (n= 124,588) for 2,924 patients downgraded from the surgical intensive care units and intermediate care unit between May 2014 to May 2016. We constructed an ensemble model to estimate at the time of intensive care units or intermediate care unit discharge the probability of favorable outcome after downgrade. RESULTS Outputs form the ensemble model stratified patients by risk of favorable and bad outcomes in both intensive care units/intermediate care unit; area under the receiver operating characteristic curve = .639/.629 respectively for favorable outcomes and .645/.641 for adverse events. These performance characteristics are commensurate with published models for predicting readmission. The ensemble model remained a statistically significant predictor after adjusting for hospital duration of stay and admitting service. The rate of favorable outcome in the highest and lowest deciles in the intensive care units were 76.2% and 27.3% (2.8-fold decrease) and 88.3% and 33.2% in the intermediate care unit (2.7-fold decrease), respectively. CONCLUSION An ensemble model for critical illness predicts favorable outcome after downgrade and safe patient discharge (hospital stay <7 days, no readmission, upgrade, or death).
Collapse
Affiliation(s)
- Holly N Blackburn
- UVA Center for Advanced Medical Analytics, University of Virginia, Charlottesville, VA, USA
| | - Matthew T Clark
- UVA Center for Advanced Medical Analytics, University of Virginia, Charlottesville, VA, USA
| | - J Randall Moorman
- Advanced Medical Predictive Devices, Diagnostics, and Displays; University of Virginia, Charlottesville, VA, USA
| | - Douglas E Lake
- Advanced Medical Predictive Devices, Diagnostics, and Displays; University of Virginia, Charlottesville, VA, USA
| | - J Forrest Calland
- Advanced Medical Predictive Devices, Diagnostics, and Displays; University of Virginia, Charlottesville, VA, USA.
| |
Collapse
|