Logistic early warning scores to predict death, cardiac arrest or unplanned intensive care unit re-admission after cardiac surgery

Continuous Data-Driven Monitoring in Critical Congenital Heart Disease: Clinical Deterioration Model Development

BACKGROUND

Critical congenital heart disease (cCHD)-requiring cardiac intervention in the first year of life for survival-occurs globally in 2-3 of every 1000 live births. In the critical perioperative period, intensive multimodal monitoring at a pediatric intensive care unit (PICU) is warranted, as their organs-especially the brain-may be severely injured due to hemodynamic and respiratory events. These 24/7 clinical data streams yield large quantities of high-frequency data, which are challenging in terms of interpretation due to the varying and dynamic physiology innate to cCHD. Through advanced data science algorithms, these dynamic data can be condensed into comprehensible information, reducing the cognitive load on the medical team and providing data-driven monitoring support through automated detection of clinical deterioration, which may facilitate timely intervention.

OBJECTIVE

This study aimed to develop a clinical deterioration detection algorithm for PICU patients with cCHD.

METHODS

Retrospectively, synchronous per-second data of cerebral regional oxygen saturation (rSO₂) and 4 vital parameters (respiratory rate, heart rate, oxygen saturation, and invasive mean blood pressure) in neonates with cCHD admitted to the University Medical Center Utrecht, the Netherlands, between 2002 and 2018 were extracted. Patients were stratified based on mean oxygen saturation during admission to account for physiological differences between acyanotic and cyanotic cCHD. Each subset was used to train our algorithm in classifying data as either stable, unstable, or sensor dysfunction. The algorithm was designed to detect combinations of parameters abnormal to the stratified subpopulation and significant deviations from the patient's unique baseline, which were further analyzed to distinguish clinical improvement from deterioration. Novel data were used for testing, visualized in detail, and internally validated by pediatric intensivists.

RESULTS

A retrospective query yielded 4600 hours and 209 hours of per-second data in 78 and 10 neonates for, respectively, training and testing purposes. During testing, stable episodes occurred 153 times, of which 134 (88%) were correctly detected. Unstable episodes were correctly noted in 46 of 57 (81%) observed episodes. Twelve expert-confirmed unstable episodes were missed in testing. Time-percentual accuracy was 93% and 77% for, respectively, stable and unstable episodes. A total of 138 sensorial dysfunctions were detected, of which 130 (94%) were correct.

CONCLUSIONS

In this proof-of-concept study, a clinical deterioration detection algorithm was developed and retrospectively evaluated to classify clinical stability and instability, achieving reasonable performance considering the heterogeneous population of neonates with cCHD. Combined analysis of baseline (ie, patient-specific) deviations and simultaneous parameter-shifting (ie, population-specific) proofs would be promising with respect to enhancing applicability to heterogeneous critically ill pediatric populations. After prospective validation, the current-and comparable-models may, in the future, be used in the automated detection of clinical deterioration and eventually provide data-driven monitoring support to the medical team, allowing for timely intervention.

The risk assessment tool for intensive care unit readmission: A systematic review and meta-analysis

OBJECTIVE

To review and evaluate existing risk assessment tools for intensive care unitreadmission.

METHODS

Nine electronic databases (Medline, CINAHL, Web of Science, Cochrane Library, Embase, Sino Med, CNKI, VIP, and Wan fang) were systematically searched from their inception to September 2022. Two authors independently extracted data from the literature included. Meta-analysis was performed under the bivariate modeling and summary receiver operating characteristic curve method.

RESULTS

A total of 29 studies were included in this review, among which 11 were quantitatively Meta-analyzed. The results showed Stability and Workload Index for Transfer: Sensitivity = 0.55, Specificity = 0.65, Area under curve = 0.63. And Early warning score: Sensitivity = 0.78, Specificity = 0.83, Area under curve = 0.88. The remaining tools included scores, nomograms, machine learning models, and deep learning models. These studies, with varying reports on thresholds, case selection, data preprocessing, and model performance, have a high risk of bias.

CONCLUSION

We cannot identify a tool that can be used directly in intensive care unit readmission risk assessment. Scores based on early warning score are moderately accurate in predicting readmission, but there is heterogeneity and publication bias that requires model adjustment for local factors such as resources, demographics, and case mix. Machine learning models present a promising modeling technique but have a high methodological bias and require further validation.

IMPLICATIONS FOR CLINICAL PRACTICE

Using reliable risk assessment tools is essential for the early identification of unplanned intensive care unit readmission risk in critically ill patients.  A reliable risk assessment tool must be developed, which is the focus of further research.

Predictive Modeling for Readmission to Intensive Care: A Systematic Review

To evaluate the methodologic rigor and predictive performance of models predicting ICU readmission; to understand the characteristics of ideal prediction models; and to elucidate relationships between appropriate triage decisions and patient outcomes.

DATA SOURCES

PubMed, Web of Science, Cochrane, and Embase.

STUDY SELECTION

Primary literature that reported the development or validation of ICU readmission prediction models within from 2010 to 2021.

DATA EXTRACTION

Relevant study information was extracted independently by two authors using the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies checklist. Bias was evaluated using the Prediction model Risk Of Bias ASsessment Tool. Data sources, modeling methodology, definition of outcomes, performance, and risk of bias were critically evaluated to elucidate relevant relationships.

DATA SYNTHESIS

Thirty-three articles describing models were included. Six studies had a high overall risk of bias due to improper inclusion criteria or omission of critical analysis details. Four other studies had an unclear overall risk of bias due to lack of detail describing the analysis. Overall, the most common (50% of studies) source of bias was the filtering of candidate predictors via univariate analysis. The poorest performing models used existing clinical risk or acuity scores such as Acute Physiologic Assessment and Chronic Health Evaluation II, Sequential Organ Failure Assessment, or Stability and Workload Index for Transfer as the sole predictor. The higher-performing ICU readmission prediction models used homogenous patient populations, specifically defined outcomes, and routinely collected predictors that were analyzed over time.

CONCLUSIONS

Models predicting ICU readmission can achieve performance advantages by using longitudinal time series modeling, homogenous patient populations, and predictor variables tailored to those populations.

Failure to Rescue: A Quality Metric for Cardiac Surgery and Cardiovascular Critical Care

Failure to rescue, defined as mortality after a surgical complication, is a widely accepted quality metric across many specialties and is becoming an important metric in cardiac surgery. The failure to rescue metric provides a target for improvements in patient outcomes after complications occur. To be used appropriately, the failure to rescue metric must be defined using a prespecified set of life-threatening and rescuable complications. Successful patient rescue requires a systematic approach of complication recognition, timely escalation of care, effective medical management, and mitigation of additional complications. This process requires contributions from cardiac surgeons, intensivists, and other specialists including cardiologists, neurologists, and anaesthesiologists. Factors that affect failure to rescue rates in cardiac surgery and cardiovascular critical care include nurse staffing ratios, intensivist coverage, advanced specialist support, hospital and surgical volume, the presence of trainees, and patient comorbidities. Strategies to improve patient rescue include working to understand the mechanisms of failure to rescue, anticipating postoperative complications, prioritizing microsystem factors, enhancing early escalation of care, and educating and empowering junior clinicians. When used appropriately, the failure to rescue quality metric can help institutions focus on improving processes of care that minimize morbidity and mortality from rescuable complications after cardiac surgery.

Forecasting medical state transition using machine learning methods

Early circulatory failure detection is an effective way to reduce medical fatigue and improve state pre-warning ability. Instead of using 0-1 original state, a transformed state is proposed in this research, which reflects how the state is transformed. The performance of the proposed method is compared with the original method under three models, including logistic regression, AdaBoost and XGBoost. The results show that the model XGBoost generally has the best performance measured by AUC, F1 and Sensitivity with values around 0.93, 0.91 and 0.90, at the prediction gaps 5, 10 and 20 separately. Under the model XGBoost, the method with transformed response variable has significantly better performance than that with the original response variable, with the performance metrics being around 1% to 4% higher, and the t values are all significant under the level 0.01. In order to explore the model performance under different baseline information, a subgroup analysis is conducted under sex, age, weight and height. The results demonstrate that sex and age have more significant influence on the model performance especially at the higher gaps than weight and height.

Dynamic early warning scores for predicting clinical deterioration in patients with respiratory disease

Background

The National Early Warning Score-2 (NEWS-2) is used to detect patient deterioration in UK hospitals but fails to take account of the detailed granularity or temporal trends in clinical observations. We used data-driven methods to develop dynamic early warning scores (DEWS) to address these deficiencies, and tested their accuracy in patients with respiratory disease for predicting (1) death or intensive care unit admission, occurring within 24 h (D/ICU), and (2) clinically significant deterioration requiring urgent intervention, occurring within 4 h (CSD).

Methods

Clinical observations data were extracted from electronic records for 31,590 respiratory in-patient episodes from April 2015 to December 2020 at a large acute NHS Trust. The timing of D/ICU was extracted for all episodes. 1100 in-patient episodes were annotated manually to record the timing of CSD, defined as a specific event requiring a change in treatment. Time series features were entered into logistic regression models to derive DEWS for each of the clinical outcomes. Area under the receiver operating characteristic curve (AUROC) was the primary measure of model accuracy.

Results

AUROC (95% confidence interval) for predicting D/ICU was 0.857 (0.852–0.862) for NEWS-2 and 0.906 (0.899–0.914) for DEWS in the validation data. AUROC for predicting CSD was 0.829 (0.817–0.842) for NEWS-2 and 0.877 (0.862–0.892) for DEWS. NEWS-2 ≥ 5 had sensitivity of 88.2% and specificity of 54.2% for predicting CSD, while DEWS ≥ 0.021 had higher sensitivity of 93.6% and approximately the same specificity of 54.3% for the same outcome. Using these cut-offs, 315 out of 347 (90.8%) CSD events were detected by both NEWS-2 and DEWS, at the time of the event or within the previous 4 h; 12 (3.5%) were detected by DEWS but not by NEWS-2, while 4 (1.2%) were detected by NEWS-2 but not by DEWS; 16 (4.6%) were not detected by either scoring system.

Conclusion

We have developed DEWS that display greater accuracy than NEWS-2 for predicting clinical deterioration events in patients with respiratory disease. Prospective validation studies are required to assess whether DEWS can be used to reduce missed deteriorations and false alarms in real-life clinical settings.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12931-022-02130-6.

FDG-PET/CT Assessment of the Cerebral Protective Effects of Hydrogen in Rabbits with Cardiac Arrest

BACKGROUND

Anatomical imaging methods and histological examinations have limited clinical value for early monitoring of brain function damage after cardiac arrest (CA) in vivo.

OBJECTIVE

We aimed to assess the cerebral protective effects of hydrogen in rabbits with CA by using fluorodeoxyglucose-positron emission tomography/computed tomography (FDG-PET/CT).

METHODS

Male rabbits were divided into the hydrogen-treated (n=6), control (n=6), and sham (n=3) groups. Maximum standardized uptake values (SUVmax) were measured by FDG-PET/CT at baseline and post-resuscitation. Blood Ubiquitin C-terminal hydrolase-L1 (UCH-L1) and neuron specific enolase (NSE) were measured before and after the operation. After surgical euthanasia, brain tissues were extracted for Nissl staining.

RESULTS

SUVmax values first decreased at 2 and 24 h after resuscitation before rising in the hydrogen-treated and control groups. SUVmax values in the frontal, occipital, and left temporal lobes and in the whole brain were significantly different between the hydrogen and control groups at 2 and 24 h post-resuscitation (P<0.05). The neurological deficit scores at 24 and 48 h were lower in the hydrogen-treated group (P<0.05). At 24 h, the serum UCH-L1 and NSE levels were increased in the hydrogen and control groups (P<0.05), but not in the sham group. At 48 and 72 h post-CA, the plasma UCH-L1 and NSE levels in the hydrogen and control groups gradually decreased. Neuronal damage was smaller in the hydrogen group compared with the control group at 72 h.

CONCLUSION

FDG-PET/CT could be used to monitor early cerebral damage, indicating a novel method for evaluating the protective effects of hydrogen on the brain after CA.

Prediction of severe adverse event from vital signs for post-operative patients

Monitoring post-operative patients is important for preventing severe adverse events (SAE), which increases morbidity and mortality. Conventional bedside monitoring system has demonstrated the difficulty in long term monitoring of those patients because majority of them are ambulatory. With development of wearable system and advanced data analytics, those patients would benefit greatly from continuous and predictive monitoring. In this study, we aim to predict SAE based on monitoring of vital signs. Heart rate, respiration rate, and blood oxygen saturation were continuously acquired by wearable devices and blood pressure was measured intermittently from 453 post-operative patients. SAEs from various complications were extracted from patients' database. The trends of vital signs were first extracted with moving average. Then four descriptive statistics were calculated from trend of each modality as features. Finally, a machine learning approach based on support vector machine was employed for prediction of SAE. It has shown the averaged accuracy of 89%, sensitivity of 80%, specificity of 93% and the area under receiver operating characteristic curve (AUROC) of 93%. These findings are promising and demonstrate the feasibility of predicting SAE from vital signs acquired with wearable devices and measured intermittently.

Prediction of mortality in COVID-19 through combing CT severity score with NEWS, qSOFA, or peripheral perfusion index

INTRODUCTION

The assessment of disease severity and the prediction of clinical outcomes at early disease stages can contribute to decreased mortality in patients with Coronavirus disease 2019 (COVID-19). This study was conducted to develop and validate a multivariable risk prediction model for mortality with using a combination of computed tomography severity score (CT-SS), national early warning score (NEWS), and quick sequential (sepsis-related) organ failure assessment (qSOFA) in COVID-19 patients.

METHODS

We retrospectively collected medical data from 655 adult COVID-19 patients admitted to our hospital between July and November 2020. Data on demographics, clinical characteristics, and laboratory and radiological findings measured as part of standard care at admission were used to calculate NEWS, qSOFA score, CT-SS, peripheral perfusion index (PPI) and shock index (SI). Logistic regression and Cox proportional hazard models were used to predict mortality, which was our primary outcome. The predictive accuracy of distinct scoring systems was evaluated by the receiver-operating characteristic (ROC) curve analysis.

RESULTS

The median age was 50.0 years [333 males (50.8%), 322 females (49.2%)]. Higher NEWS and SI was associated with time-to-death within 90-days, whereas higher age, CT-SS and lower PPI were significantly associated with time-to-death within both 14 days and 90 days in the adjusted Cox regression model. The CT-SS predicted different mortality risk levels within each stratum of NEWS and qSOFA and improved the discrimination of mortality prediction models. Combining CT-SS with NEWS score yielded more accurate 14 days (DBA: -0.048, p = 0.002) and 90 days (DBA: -0.066, p < 0.001) mortality prediction.

CONCLUSION

Combining severity tools such as CT-SS, NEWS and qSOFA improves the accuracy of predicting mortality in patients with COVID-19. Inclusion of these tools in decision strategies might provide early detection of high-risk groups, avoid delayed medical attention, and improve patient outcomes.

Introduction of a standardised maternity early warning system: indicative data from a before-and-after study at a large pilot site before national rollout in Aotearoa New Zealand

Strong evidence now demonstrates that recognition and response systems using standardised early warning scores can help prevent harm associated with in-hospital clinical deterioration in non-pregnant adult patients. However, a standardised maternity-specific early warning system has not yet been agreed in the UK. In Aotearoa New Zealand, following the nationwide implementation of the standardised New Zealand Early Warning Score (NZEWS) for adult inpatients, a modified maternity-specific variation (NZMEWS) was piloted in a major tertiary hospital in Auckland, before national rollout. Following implementation in July 2018, we observed a significant and sustained reduction in severe maternal morbidity as measured by emergency response calls to women who were very unwell (emergency response team call), and a non-significant reduction in cardiorespiratory arrest team calls. Emergency response team calls to maternity wards fell from a median of 0.8 per 100 births at baseline (January 2017-May 2018) to 0.6 per 100 births monthly (from March 2019 to December 2020) (p < 0.0001). Cardiorespiratory arrest team calls to maternity wards fell from 0.14 per 100 births per quarter (quarter 1 2017-quarter 2 2018) to 0.09 calls per 100 births per quarter after NZMEWS was introduced (quarter 3 2018-quarter 4 2020) (p = 0.2593). These early results provide evidence that NZMEWS can detect and prevent deterioration of pregnant women, although there are multiple factors that may have contributed to the reduction in emergency response calls noted.

A predictive framework in healthcare: Case study on cardiac arrest prediction

Data-driven healthcare uses predictive analytics to enhance decision-making and personalized healthcare. Developing prognostic models is one of the applications of predictive analytics in medical environments. Various studies have used machine learning techniques for this purpose. However, there is no specific standard for choosing prediction models for different medical purposes. In this paper, the ISAF framework was proposed for choosing appropriate prediction models with regard to the properties of the classification methods. As one of the case study applications, a prognostic model for predicting cardiac arrests in sepsis patients was developed step by step through the ISAF framework. Finally, a new modified stacking model produced the best results. We predict 85 % of heart arrest cases one hour before the incidence (sensitivity> = 0.85) and 73 % of arrest cases 25 h before the occurrence (sensitivity> = 0.73). The results indicated that the proposed prognostic model has significantly improved the prediction results compared to the two standard systems of APACHE II and MEWS. Furthermore, compared to previous research, the proposed model has extended the prediction interval and improved the performance criteria.

Machine Learning-Based Early Warning Systems for Clinical Deterioration: Systematic Scoping Review

Background

Timely identification of patients at a high risk of clinical deterioration is key to prioritizing care, allocating resources effectively, and preventing adverse outcomes. Vital signs–based, aggregate-weighted early warning systems are commonly used to predict the risk of outcomes related to cardiorespiratory instability and sepsis, which are strong predictors of poor outcomes and mortality. Machine learning models, which can incorporate trends and capture relationships among parameters that aggregate-weighted models cannot, have recently been showing promising results.

Objective

This study aimed to identify, summarize, and evaluate the available research, current state of utility, and challenges with machine learning–based early warning systems using vital signs to predict the risk of physiological deterioration in acutely ill patients, across acute and ambulatory care settings.

Methods

PubMed, CINAHL, Cochrane Library, Web of Science, Embase, and Google Scholar were searched for peer-reviewed, original studies with keywords related to “vital signs,” “clinical deterioration,” and “machine learning.” Included studies used patient vital signs along with demographics and described a machine learning model for predicting an outcome in acute and ambulatory care settings. Data were extracted following PRISMA, TRIPOD, and Cochrane Collaboration guidelines.

Results

We identified 24 peer-reviewed studies from 417 articles for inclusion; 23 studies were retrospective, while 1 was prospective in nature. Care settings included general wards, intensive care units, emergency departments, step-down units, medical assessment units, postanesthetic wards, and home care. Machine learning models including logistic regression, tree-based methods, kernel-based methods, and neural networks were most commonly used to predict the risk of deterioration. The area under the curve for models ranged from 0.57 to 0.97.

Conclusions

In studies that compared performance, reported results suggest that machine learning–based early warning systems can achieve greater accuracy than aggregate-weighted early warning systems but several areas for further research were identified. While these models have the potential to provide clinical decision support, there is a need for standardized outcome measures to allow for rigorous evaluation of performance across models. Further research needs to address the interpretability of model outputs by clinicians, clinical efficacy of these systems through prospective study design, and their potential impact in different clinical settings.

Update on early warning scores

Early warning scores (EWS) have the objective to provide a preventive approach for detecting those patients in general wards at risk of deterioration before it begins. Well implemented and combined with a tiered response, the EWS expect to be a relevant tool for patient safety. Most of the evidence for their use has been published for the general EWS. Their strengths, such as objectivity and systematic response, health provider training, universal applicability and automatization potential need to be highlighted to counterbalance the weakness and limitations that have also been described. The near future will probably increase availability of EWS, reliability and predictive value through the spread and acceptability of continuous monitoring in general ward, its integration in decision support algorithms with automatic alerts and the elaboration of temporal vital signs patterns that will finally allow to perform a personal modelling depending on individual patient characteristics.

Dynamic individual vital sign trajectory early warning score (DyniEWS) versus snapshot national early warning score (NEWS) for predicting postoperative deterioration

Aims

International early warning scores (EWS) including the additive National Early Warning Score (NEWS) and logistic EWS currently utilise physiological snapshots to predict clinical deterioration. We hypothesised that a dynamic score including vital sign trajectory would improve discriminatory power.

Methods

Multicentre retrospective analysis of electronic health record data from postoperative patients admitted to cardiac surgical wards in four UK hospitals. Least absolute shrinkage and selection operator-type regression (LASSO) was used to develop a dynamic model (DyniEWS) to predict a composite adverse event of cardiac arrest, unplanned intensive care re-admission or in-hospital death within 24 h.

Results

A total of 13,319 postoperative adult cardiac patients contributed 442,461 observations of which 4234 (0.96%) adverse events in 24 h were recorded. The new dynamic model (AUC = 0.80 [95% CI 0.78−0.83], AUPRC = 0.12 [0.10−0.14]) outperforms both an updated snapshot logistic model (AUC = 0.76 [0.73−0.79], AUPRC = 0.08 [0.60−0.10]) and the additive National Early Warning Score (AUC = 0.73 [0.70−0.76], AUPRC = 0.05 [0.02−0.08]). Controlling for the false alarm rates to be at current levels using NEWS cut-offs of 5 and 7, DyniEWS delivers a 7% improvement in balanced accuracy and increased sensitivities from 41% to 54% at NEWS 5 and 18% to –30% at NEWS 7.

Conclusions

Using an advanced statistical approach, we created a model that can detect dynamic changes in risk of unplanned readmission to intensive care, cardiac arrest or in-hospital mortality and can be used in real time to risk-prioritise clinical workload.