Performance of universal early warning scores in different patient subgroups and clinical settings: a systematic review

Evaluating the performance of the National Early Warning Score in different diagnostic groups

BACKGROUND

The National Early Warning Score (NEWS) is used in hospitals across the UK to detect deterioration of patients within care pathways. It is used for most patients, but there are relatively few studies validating its performance in groups of patients with specific conditions.

METHODS

The performance of NEWS was evaluated against 36 other Early Warning Scores, in 123 patient groups, through use of the area under the receiver operating characteristic (AUROC) curve technique, to compare the abilities of each Early Warning Score to discriminate an outcome within 24hrs of vital sign recording. Outcomes evaluated were death, ICU admission, or a combined outcome of either death or ICU admission within 24 hours of an observation set.

RESULTS

The National Early Warning Score 2 performs either best or joint best within 120 of the 123 patient groups evaluated and is only outperformed in prediction of unanticipated ICU admission. When outperformed by other Early Warning Scores in the remaining 3 patient groups, the performance difference was marginal.

CONCLUSIONS

Consistently high performance indicates that NEWS is a suitable early warning score to use for all diagnostic groups considered by this analysis, and patients are not disadvantaged through use of NEWS in comparison to any of the other evaluated Early Warning Scores.

Development and validation of the early warning system scores ontology

BACKGROUND

Clinical early warning scoring systems, have improved patient outcomes in a range of specializations and global contexts. These systems are used to predict patient deterioration. A multitude of patient-level physiological decompensation data has been made available through the widespread integration of early warning scoring systems within EHRs across national and international health care organizations. These data can be used to promote secondary research. The diversity of early warning scoring systems and various EHR systems is one barrier to secondary analysis of early warning score data. Given that early warning score parameters are varied, this makes it difficult to query across providers and EHR systems. Moreover, mapping and merging the parameters is challenging. We develop and validate the Early Warning System Scores Ontology (EWSSO), representing three commonly used early warning scores: the National Early Warning Score (NEWS), the six-item modified Early Warning Score (MEWS), and the quick Sequential Organ Failure Assessment (qSOFA) to overcome these problems.

METHODS

We apply the Software Development Lifecycle Framework-conceived by Winston Boyce in 1970-to model the activities involved in organizing, producing, and evaluating the EWSSO. We also follow OBO Foundry Principles and the principles of best practice for domain ontology design, terms, definitions, and classifications to meet BFO requirements for ontology building.

RESULTS

We developed twenty-nine new classes, reused four classes and four object properties to create the EWSSO. When we queried the data our ontology-based process could differentiate between necessary and unnecessary features for score calculation 100% of the time. Further, our process applied the proper temperature conversions for the early warning score calculator 100% of the time.

CONCLUSIONS

Using synthetic datasets, we demonstrate the EWSSO can be used to generate and query health system data on vital signs and provide input to calculate the NEWS, six-item MEWS, and qSOFA. Future work includes extending the EWSSO by introducing additional early warning scores for adult and pediatric patient populations and creating patient profiles that contain clinical, demographic, and outcomes data regarding the patient.

Performance of digital early warning score (NEWS2) in a cardiac specialist setting: retrospective cohort study

INTRODUCTION

Patients with cardiovascular diseases (CVD) are at significant risk of developing critical events. Early warning scores (EWS) are recommended for early recognition of deteriorating patients, yet their performance has been poorly studied in cardiac care settings. Standardisation and integrated National Early Warning Score 2 (NEWS2) in electronic health records (EHRs) are recommended yet have not been evaluated in specialist settings.

OBJECTIVE

To investigate the performance of digital NEWS2 in predicting critical events: death, intensive care unit (ICU) admission, cardiac arrest and medical emergencies.

METHODS

Retrospective cohort analysis.

STUDY COHORT

Individuals admitted with CVD diagnoses in 2020; including patients with COVID-19 due to conducting the study during the COVID-19 pandemic.

MEASURES

We tested the ability of NEWS2 in predicting the three critical outcomes from admission and within 24 hours before the event. NEWS2 was supplemented with age and cardiac rhythm and investigated. We used logistic regression analysis with the area under the receiver operating characteristic curve (AUC) to measure discrimination.

RESULTS

In 6143 patients admitted under cardiac specialties, NEWS2 showed moderate to low predictive accuracy of traditionally examined outcomes: death, ICU admission, cardiac arrest and medical emergency (AUC: 0.63, 0.56, 0.70 and 0.63, respectively). Supplemented NEWS2 with age showed no improvement while age and cardiac rhythm improved discrimination (AUC: 0.75, 0.84, 0.95 and 0.94, respectively). Improved performance was found of NEWS2 with age for COVID-19 cases (AUC: 0.96, 0.70, 0.87 and 0.88, respectively).

CONCLUSION

The performance of NEWS2 in patients with CVD is suboptimal, and fair for patients with CVD with COVID-19 to predict deterioration. Adjustment with variables that strongly correlate with critical cardiovascular outcomes, that is, cardiac rhythm, can improve the model. There is a need to define critical endpoints, engagement with clinical experts in development and further validation and implementation studies of EHR-integrated EWS in cardiac specialist settings.

Implementation of a digital early warning score (NEWS2) in a cardiac specialist and general hospital settings in the COVID-19 pandemic: a qualitative study

OBJECTIVES

To evaluate implementation of digital National Early Warning Score 2 (NEWS2) in a cardiac care setting and a general hospital setting in the COVID-19 pandemic.

DESIGN

Thematic analysis of qualitative semistructured interviews using the non-adoption, abandonment, scale-up, spread, sustainability framework with purposefully sampled nurses and managers, as well as online surveys from March to December 2021.

SETTINGS

Specialist cardiac hospital (St Bartholomew's Hospital) and general teaching hospital (University College London Hospital, UCLH).

PARTICIPANTS

Eleven nurses and managers from cardiology, cardiac surgery, oncology and intensive care wards (St Bartholomew's) and medical, haematology and intensive care wards (UCLH) were interviewed and 67 were surveyed online.

RESULTS

Three main themes emerged: (1) implementing NEWS2 challenges and supports; (2) value of NEWS2 to alarm, escalate and during the pandemic; and (3) digitalisation: electronic health record (EHR) integration and automation. The value of NEWS2 was partly positive in escalation, yet there were concerns by nurses who undervalued NEWS2 particularly in cardiac care. Challenges, like clinicians' behaviours, lack of resources and training and the perception of NEWS2 value, limit the success of this implementation. Changes in guidelines in the pandemic have led to overlooking NEWS2. EHR integration and automated monitoring are improvement solutions that are not fully employed yet.

CONCLUSION

Whether in specialist or general medical settings, the health professionals implementing early warning score in healthcare face cultural and system-related challenges to adopting NEWS2 and digital solutions. The validity of NEWS2 in specialised settings and complex conditions is not yet apparent and requires comprehensive validation. EHR integration and automation are powerful tools to facilitate NEWS2 if its principles are reviewed and rectified, and resources and training are accessible. Further examination of implementation from the cultural and automation domains is needed.

Evaluating a novel, integrative dashboard for health professionals' performance in managing deteriorating patients: a quality improvement project

BACKGROUND

The quality of recording and documentation of deteriorating patient management by health professionals has been challenged during the COVID-19 pandemic. Non-adherence to escalation and documentation guidelines increases risk of serious adverse events. Electronic health record (EHR)-integrated dashboards are auditing tools of patients' status and clinicians' performance, but neither the views nor the performance of health professionals have been assessed, relating to management of deteriorating patients.

OBJECTIVE

To develop and evaluate a real-time dashboard of deteriorating patients' assessment, referral and therapy.

SETTINGS

Five academic hospitals in the largest National Health Service (NHS) trust in the UK (Barts Health NHS Trust).

INTERVENTION

The dashboard was developed from EHR data to investigate patients with National Early Warning Score (NEWS2)>5, assessment, and escalation of deteriorating patients. We adopted the Plan, Do, Study, Act model and Standards for Quality Improvement Reporting Excellence framework to evaluate the dashboard.

DESIGN

Mixed methods: (1) virtual, face-to-face, interviews and (2) retrospective descriptive EHR data analysis.

RESULTS

We interviewed three nurses (two quality and safety and one informatics specialists). Participants perceived the dashboard as a facilitator for auditing NEWS2 recording and escalation of care to improve practice; (2) there is a need for guiding clinicians and adjusting data sources and metrics to enhance the functionality and usability. Data analysis (2019-2022) showed: (1) NEWS2 recording has gradually improved (May 2021-April 2022) from 64% to 83%;(2) referral and assessment completion increased (n: 170-6800 and 23-540, respectively).

CONCLUSION

The dashboard is an effective real-time data-driven method for improving the quality of managing deteriorating patients. Integrating health systems, a wider analysis NEWS2 and escalation of care metrics, and clinicians' learning digital solutions will enhance functionality and experience to boost its value. There is a need to examine the generalisability of the dashboard through further validation and quality improvement studies.

National Early Warning Score and New-Onset Atrial Fibrillation for Predicting In-Hospital Mortality or Transfer to the Intensive Care Unit in Emergency Department Patients with Suspected Bacterial Infections

Purpose

There are conflicting data regarding the role of the National Early Warning Score 2 (NEWS2) in predicting adverse outcomes in patients with infectious diseases. New-onset atrial fibrillation (NO-AF) has been suggested as a sepsis-defining sign of organ dysfunction. This study aimed to examine the prognostic accuracy of NEWS2 and whether NO-AF can provide prognostic information in emergency department (ED) patients with suspected bacterial infections.

Patients and Methods

Secondary analyses of data from a prospective observational cohort study of adults admitted in a 6-month period with suspected bacterial infections. We used the composite endpoint of in-hospital mortality or transfer to the intensive care unit as the primary outcome. The prognostic accuracy of NEWS2 and quick sequential organ failure assessment (qSOFA) and covariate-adjusted area under the receiver operating curves (AAUROC) were used to describe the performance of the scores. Logistic regression analysis was used to examine the association between NO-AF and the composite endpoint.

Results

A total of 2055 patients were included in this study. The composite endpoint was achieved in 198 (9.6%) patients. NO-AF was observed in 80 (3.9%) patients. The sensitivity and specificity for NEWS2 ≥5 were 70.2% (63.3-76.5) and 60.2% (57.9-62.4), respectively, and those for qSOFA ≥2 were 26.3% (20.3-33.0) and 91.0% (89.6-92.3), respectively. AAUROC for NEWS2 and qSOFA were 0.68 (0.65-0.73) and 0.63 (0.59-0.68), respectively. The adjusted odds ratio for achieving the composite endpoint in 48 patients with NO-AF who fulfilled the NEWS2 ≥5 criteria was 2.71 (1.35-5.44).

Conclusion

NEWS2 had higher sensitivity but lower specificity and better, albeit poor, discriminative ability to predict the composite endpoint compared to qSOFA. NO-AF can provide important prognostic information.

Comparison of Nine Early Warning Scores for Identification of Short-Term Mortality in Acute Neurological Disease in Emergency Department

(1) Background: The aim was screening the performance of nine Early Warning Scores (EWS), to identify patients at high-risk of premature impairment and to detect intensive care unit (ICU) admissions, as well as to track the 2-, 7-, 14-, and 28-day mortality in a cohort of patients diagnosed with an acute neurological condition. (2) Methods: We conducted a prospective, longitudinal, observational study, calculating the EWS [Modified Early Warning Score (MEWS), National Early Warning Score (NEWS), VitalPAC Early Warning Score (ViEWS), Modified Rapid Emergency Medicine Score (MREMS), Early Warning Score (EWS), Hamilton Early Warning Score (HEWS), Standardised Early Warning Score (SEWS), WHO Prognostic Scored System (WPSS), and Rapid Acute Physiology Score (RAPS)] upon the arrival of patients to the emergency department. (3) Results: In all, 1160 patients were included: 808 patients were hospitalized, 199 cases (17%) required ICU care, and 6% of patients died (64 cases) within 2 days, which rose to 16% (183 cases) within 28 days. The highest area under the curve for predicting the need for ICU admissions was obtained by RAPS and MEWS. For predicting mortality, MREMS obtained the best scores for 2- and 28-day mortality. (4) Conclusions: This is the first study to explore whether several EWS accurately identify the risk of ICU admissions and mortality, at different time points, in patients with acute neurological disorders. Every score analyzed obtained good results, but it is suggested that the use of RAPS, MEWS, and MREMS should be preferred in the acute setting, for patients with neurological impairment.

Performance of Modified Pediatric Early Warning Score in General Medical Conditions and Disease Subgroups

Numerous existing Pediatric Early Warning Scores (PEWs) have varying degrees of reliability and validity, which are used in variety diseases of patients. This study is a prospective diagnosis study which involved the pediatric nurse evaluation of patient status using modified Pediatric Early Warning Score (NU-PEWS) until the patient was discharged or transferred to PICU. A total of 824 pediatric patients were admitted, 407 participants were enrolled in this study. The NU-PEWS demonstrated the most accurate cut-off point at greater than 3, with 90.5% sensitivity and 89.1% specificity. The receiver operating characteristic (ROC) indicated positive results in the general medical condition (ROC 0.958), gastrointestinal, respiratory, and hematologic diseases (ROC 0.94-0.97) whereas lowest in neurological disease (ROC 0.843). This study validated that NU-PEWS has good performance in detecting deteriorating patients, and that prediction utilizations are good in almost every subgroup of disease, with the exception of neurological disease.