Coronavirus Disease 2019 Calls for Predictive Analytics Monitoring-A New Kind of Illness Scoring System

Detecting Patient Deterioration Early Using Continuous Heart rate and Respiratory rate Measurements in Hospitalized COVID-19 Patients

BACKGROUND

Presenting symptoms of COVID-19 patients are unusual compared with many other illnesses. Blood pressure, heart rate, and respiratory rate may stay within acceptable ranges as the disease progresses. Consequently, intermittent monitoring does not detect deterioration as it is happening. We investigated whether continuously monitoring heart rate and respiratory rate enables earlier detection of deterioration compared with intermittent monitoring, or introduces any risks.

METHODS

When available, patients admitted to a COVID-19 ward received a wireless wearable sensor which continuously measured heart rate and respiratory rate. Two intensive care unit (ICU) physicians independently assessed sensor data, indicating when an intervention might be necessary (alarms). A third ICU physician independently extracted clinical events from the electronic medical record (EMR events). The primary outcome was the number of true alarms. Secondary outcomes included the time difference between true alarms and EMR events, interrater agreement for the alarms, and severity of EMR events that were not detected.

RESULTS

In clinical practice, 48 (EMR) events occurred. None of the 4 ICU admissions were detected with the sensor. Of the 62 sensor events, 13 were true alarms (also EMR events). Of these, two were related to rapid response team calls. The true alarms were detected 39 min (SD = 113) before EMR events, on average. Interrater agreement was 10%. Severity of the 38 non-detected events was similar to the severity of 10 detected events.

CONCLUSION

Continuously monitoring heart rate and respiratory rate does not reliably detect deterioration in COVID-19 patients when assessed by ICU physicians.

Beyond prediction: Off-target uses of artificial intelligence-based predictive analytics in a learning health system

Introduction

Artificial-intelligence (AI)-based predictive analytics provide new opportunities to leverage rich sources of continuous data to improve patient care through early warning of the risk of clinical deterioration and improved situational awareness.Part of the success of predictive analytic implementation relies on integration of the analytic within complex clinical workflows. Pharmaceutical interventions have off-target uses where a drug indication has not been formally studied for a different indication but has potential for clinical benefit. An analog has not been described in the context of AI-based predictive analytics, that is, when a predictive analytic has been trained on one outcome of interest but is used for additional applications in clinical practice.

Methods

In this manuscript we present three clinical vignettes describing off-target use of AI-based predictive analytics that evolved organically through real-world practice.

Results

Off-target uses included:real-time feedback about treatment effectiveness, indication of readiness to discharge, and indication of the acuity of a hospital unit.

Conclusion

Such practice fits well with the learning health system goals to continuously integrate data and experience to provide.

Signatures of illness in children requiring unplanned intubation in the pediatric intensive care unit: A retrospective cohort machine-learning study

Acute respiratory failure requiring the initiation of invasive mechanical ventilation remains commonplace in the pediatric intensive care unit (PICU). Early recognition of patients at risk for respiratory failure may provide clinicians with the opportunity to intervene and potentially improve outcomes. Through the development of a random forest model to identify patients at risk for requiring unplanned intubation, we tested the hypothesis that subtle signatures of illness are present in physiological and biochemical time series of PICU patients in the early stages of respiratory decompensation. We included 116 unplanned intubation events as recorded in the National Emergency Airway Registry for Children in 92 PICU admissions over a 29-month period at our institution. We observed that children have a physiologic signature of illness preceding unplanned intubation in the PICU. Generally, it comprises younger age, and abnormalities in electrolyte, hematologic and vital sign parameters. Additionally, given the heterogeneity of the PICU patient population, we found differences in the presentation among the major patient groups - medical, cardiac surgical, and non-cardiac surgical. At four hours prior to the event, our random forest model demonstrated an area under the receiver operating characteristic curve of 0.766 (0.738 for medical, 0.755 for cardiac surgical, and 0.797 for non-cardiac surgical patients). The multivariable statistical models that captured the physiological and biochemical dynamics leading up to the event of urgent unplanned intubation in a PICU can be repurposed for bedside risk prediction.

The principles of whole-hospital predictive analytics monitoring for clinical medicine originated in the neonatal ICU

In 2011, a multicenter group spearheaded at the University of Virginia demonstrated reduced mortality from real-time continuous cardiorespiratory monitoring in the neonatal ICU using what we now call Artificial Intelligence, Big Data, and Machine Learning. The large, randomized heart rate characteristics trial made real, for the first time that we know of, the promise that early detection of illness would allow earlier and more effective intervention and improved patient outcomes. Currently, though, we hear as much of failures as we do of successes in the rapidly growing field of predictive analytics monitoring that has followed. This Perspective aims to describe the principles of how we developed heart rate characteristics monitoring for neonatal sepsis and then applied them throughout adult ICU and hospital medicine. It primarily reflects the work since the 1990s of the University of Virginia group: the theme is that sudden and catastrophic deteriorations can be preceded by subclinical but measurable physiological changes apparent in the continuous cardiorespiratory monitoring and electronic health record.

Individual illness dynamics: An analysis of children with sepsis admitted to the pediatric intensive care unit

Illness dynamics and patterns of recovery may be essential features in understanding the critical illness course. We propose a method to characterize individual illness dynamics in patients who experienced sepsis in the pediatric intensive care unit. We defined illness states based on illness severity scores generated from a multi-variable prediction model. For each patient, we calculated transition probabilities to characterize movement among illness states. We calculated the Shannon entropy of the transition probabilities. Using the entropy parameter, we determined phenotypes of illness dynamics based on hierarchical clustering. We also examined the association between individual entropy scores and a composite variable of negative outcomes. Entropy-based clustering identified four illness dynamic phenotypes in a cohort of 164 intensive care unit admissions where at least one sepsis event occurred. Compared to the low-risk phenotype, the high-risk phenotype was defined by the highest entropy values and had the most ill patients as defined by a composite variable of negative outcomes. Entropy was significantly associated with the negative outcome composite variable in a regression analysis. Information-theoretical approaches to characterize illness trajectories offer a novel way of assessing the complexity of a course of illness. Characterizing illness dynamics with entropy offers additional information in conjunction with static assessments of illness severity. Additional attention is needed to test and incorporate novel measures representing the dynamics of illness.

Ideal algorithms in healthcare: Explainable, dynamic, precise, autonomous, fair, and reproducible

Established guidelines describe minimum requirements for reporting algorithms in healthcare; it is equally important to objectify the characteristics of ideal algorithms that confer maximum potential benefits to patients, clinicians, and investigators. We propose a framework for ideal algorithms, including 6 desiderata: explainable (convey the relative importance of features in determining outputs), dynamic (capture temporal changes in physiologic signals and clinical events), precise (use high-resolution, multimodal data and aptly complex architecture), autonomous (learn with minimal supervision and execute without human input), fair (evaluate and mitigate implicit bias and social inequity), and reproducible (validated externally and prospectively and shared with academic communities). We present an ideal algorithms checklist and apply it to highly cited algorithms. Strategies and tools such as the predictive, descriptive, relevant (PDR) framework, the Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence (SPIRIT-AI) extension, sparse regression methods, and minimizing concept drift can help healthcare algorithms achieve these objectives, toward ideal algorithms in healthcare.

Continuous cardiorespiratory monitoring is a dominant source of predictive signal in machine learning for risk stratification and clinical decision support. Physiol Meas 2021;42. [PMID: 34580243 DOI: 10.1088/1361-6579/ac2130]

Beaulieu-Jones and coworkers propose a litmus test for the field of predictive analytics-performance improvements must be demonstrated to be the result of non-clinician-initiated data, otherwise, there should be caution in assuming that predictive models could improve clinical decision-making (Beaulieu-Joneset al2021). They demonstrate substantial prognostic information in unsorted physician orders made before the first midnight of hospital admission, and we are persuaded that it is fair to ask-if the physician thought of it first, what exactly is machine learning for in-patient risk stratification learning about? While we want predictive analytics to represent the leading indicators of a patient's illness, does it instead merely reflect the lagging indicators of clinicians' actions? We propose that continuous cardiorespiratory monitoring-'routine telemetry data,' in Beaulieu-Jones' terms-represents the most valuable non-clinician-initiated predictive signal present in patient data, and the value added to patient care justifies the efforts and expense required. Here, we present a clinical and a physiological point of view to support our contention.

External validation of a novel signature of illness in continuous cardiorespiratory monitoring to detect early respiratory deterioration of ICU patients

Objective:

The goal of predictive analytics monitoring is the early detection of patients at high risk of subacute potentially catastrophic illnesses. An excellent example of a targeted illness is respiratory failure leading to urgent unplanned intubation, where early detection might lead to interventions that improve patient outcomes. Previously, we identified signatures of this illness in the continuous cardiorespiratory monitoring data of intensive care unit (ICU) patients and devised algorithms to identify patients at rising risk. Here, we externally validated three logistic regression models to estimate the risk of emergency intubation developed in Medical and Surgical ICUs at the University of Virginia.

Approach:

We calculated the model outputs for more than 8000 patients in the University of California—San Francisco ICUs, 240 of whom underwent emergency intubation as determined by individual chart review.

Main results:

We found that the AUC of the models exceeded 0.75 in this external population, and that the risk rose appreciably over the 12 h before the event.

Significance:

We conclude that there are generalizable physiological signatures of impending respiratory failure in the continuous cardiorespiratory monitoring data.

Principles for Real-World Implementation of Bedside Predictive Analytics Monitoring

A new development in the practice of medicine is Artificial Intelligence-based predictive analytics that forewarn clinicians of future deterioration of their patients. This proactive opportunity, though, is different from the reactive stance that clinicians traditionally take. Implementing these tools requires new ideas about how to educate clinician users to facilitate trust and adoption and to promote sustained use. Our real-world hospital experience implementing a predictive analytics monitoring system that uses electronic health record and continuous monitoring data has taught us principles that we believe to be applicable to the implementation of other such analytics systems within the health care environment. These principles are mentioned below:• To promote trust, the science must be understandable.• To enhance uptake, the workflow should not be impacted greatly.• To maximize buy-in, engagement at all levels is important.• To ensure relevance, the education must be tailored to the clinical role and hospital culture.• To lead to clinical action, the information must integrate into clinical care.• To promote sustainability, there should be periodic support interactions after formal implementation.

The Quest for System-Theoretical Medicine in the COVID-19 Era

Precision medicine and molecular systems medicine (MSM) are highly utilized and successful approaches to improve understanding, diagnosis, and treatment of many diseases from bench-to-bedside. Especially in the COVID-19 pandemic, molecular techniques and biotechnological innovation have proven to be of utmost importance for rapid developments in disease diagnostics and treatment, including DNA and RNA sequencing technology, treatment with drugs and natural products and vaccine development. The COVID-19 crisis, however, has also demonstrated the need for systemic thinking and transdisciplinarity and the limits of MSM: the neglect of the bio-psycho-social systemic nature of humans and their context as the object of individual therapeutic and population-oriented interventions. COVID-19 illustrates how a medical problem requires a transdisciplinary approach in epidemiology, pathology, internal medicine, public health, environmental medicine, and socio-economic modeling. Regarding the need for conceptual integration of these different kinds of knowledge we suggest the application of general system theory (GST). This approach endorses an organism-centered view on health and disease, which according to Ludwig von Bertalanffy who was the founder of GST, we call Organismal Systems Medicine (OSM). We argue that systems science offers wider applications in the field of pathology and can contribute to an integrative systems medicine by (i) integration of evidence across functional and structural differentially scaled subsystems, (ii) conceptualization of complex multilevel systems, and (iii) suggesting mechanisms and non-linear relationships underlying the observed phenomena. We underline these points with a proposal on multi-level systems pathology including neurophysiology, endocrinology, immune system, genetics, and general metabolism. An integration of these areas is necessary to understand excess mortality rates and polypharmacological treatments. In the pandemic era this multi-level systems pathology is most important to assess potential vaccines, their effectiveness, short-, and long-time adverse effects. We further argue that these conceptual frameworks are not only valid in the COVID-19 era but also important to be integrated in a medicinal curriculum.

Dynamic Transitions of Pediatric Sepsis: A Markov Chain Analysis

Pediatric sepsis is a heterogeneous disease with varying physiological dynamics associated with recovery, disability, and mortality. Using risk scores generated from a sepsis prediction model to define illness states, we used Markov chain modeling to describe disease dynamics over time by describing how children transition among illness states. We analyzed 18,666 illness state transitions over 157 pediatric intensive care unit admissions in the 3 days following blood cultures for suspected sepsis. We used Shannon entropy to quantify the differences in transition matrices stratified by clinical characteristics. The population-based transition matrix based on the sepsis illness severity scores in the days following a sepsis diagnosis can describe a sepsis illness trajectory. Using the entropy based on Markov chain transition matrices, we found a different structure of dynamic transitions based on ventilator use but not age group. Stochastic modeling of transitions in sepsis illness severity scores can be useful in describing the variation in transitions made by patient and clinical characteristics.