Performance of digital early warning score (NEWS2) in a cardiac specialist setting: retrospective cohort study

INTRODUCTION

Patients with cardiovascular diseases (CVD) are at significant risk of developing critical events. Early warning scores (EWS) are recommended for early recognition of deteriorating patients, yet their performance has been poorly studied in cardiac care settings. Standardisation and integrated National Early Warning Score 2 (NEWS2) in electronic health records (EHRs) are recommended yet have not been evaluated in specialist settings.

OBJECTIVE

To investigate the performance of digital NEWS2 in predicting critical events: death, intensive care unit (ICU) admission, cardiac arrest and medical emergencies.

METHODS

Retrospective cohort analysis.

STUDY COHORT

Individuals admitted with CVD diagnoses in 2020; including patients with COVID-19 due to conducting the study during the COVID-19 pandemic.

MEASURES

We tested the ability of NEWS2 in predicting the three critical outcomes from admission and within 24 hours before the event. NEWS2 was supplemented with age and cardiac rhythm and investigated. We used logistic regression analysis with the area under the receiver operating characteristic curve (AUC) to measure discrimination.

RESULTS

In 6143 patients admitted under cardiac specialties, NEWS2 showed moderate to low predictive accuracy of traditionally examined outcomes: death, ICU admission, cardiac arrest and medical emergency (AUC: 0.63, 0.56, 0.70 and 0.63, respectively). Supplemented NEWS2 with age showed no improvement while age and cardiac rhythm improved discrimination (AUC: 0.75, 0.84, 0.95 and 0.94, respectively). Improved performance was found of NEWS2 with age for COVID-19 cases (AUC: 0.96, 0.70, 0.87 and 0.88, respectively).

CONCLUSION

The performance of NEWS2 in patients with CVD is suboptimal, and fair for patients with CVD with COVID-19 to predict deterioration. Adjustment with variables that strongly correlate with critical cardiovascular outcomes, that is, cardiac rhythm, can improve the model. There is a need to define critical endpoints, engagement with clinical experts in development and further validation and implementation studies of EHR-integrated EWS in cardiac specialist settings.

Implementation of a digital early warning score (NEWS2) in a cardiac specialist and general hospital settings in the COVID-19 pandemic: a qualitative study

OBJECTIVES

To evaluate implementation of digital National Early Warning Score 2 (NEWS2) in a cardiac care setting and a general hospital setting in the COVID-19 pandemic.

DESIGN

Thematic analysis of qualitative semistructured interviews using the non-adoption, abandonment, scale-up, spread, sustainability framework with purposefully sampled nurses and managers, as well as online surveys from March to December 2021.

SETTINGS

Specialist cardiac hospital (St Bartholomew's Hospital) and general teaching hospital (University College London Hospital, UCLH).

PARTICIPANTS

Eleven nurses and managers from cardiology, cardiac surgery, oncology and intensive care wards (St Bartholomew's) and medical, haematology and intensive care wards (UCLH) were interviewed and 67 were surveyed online.

RESULTS

Three main themes emerged: (1) implementing NEWS2 challenges and supports; (2) value of NEWS2 to alarm, escalate and during the pandemic; and (3) digitalisation: electronic health record (EHR) integration and automation. The value of NEWS2 was partly positive in escalation, yet there were concerns by nurses who undervalued NEWS2 particularly in cardiac care. Challenges, like clinicians' behaviours, lack of resources and training and the perception of NEWS2 value, limit the success of this implementation. Changes in guidelines in the pandemic have led to overlooking NEWS2. EHR integration and automated monitoring are improvement solutions that are not fully employed yet.

CONCLUSION

Whether in specialist or general medical settings, the health professionals implementing early warning score in healthcare face cultural and system-related challenges to adopting NEWS2 and digital solutions. The validity of NEWS2 in specialised settings and complex conditions is not yet apparent and requires comprehensive validation. EHR integration and automation are powerful tools to facilitate NEWS2 if its principles are reviewed and rectified, and resources and training are accessible. Further examination of implementation from the cultural and automation domains is needed.

Evaluating a novel, integrative dashboard for health professionals' performance in managing deteriorating patients: a quality improvement project

BACKGROUND

The quality of recording and documentation of deteriorating patient management by health professionals has been challenged during the COVID-19 pandemic. Non-adherence to escalation and documentation guidelines increases risk of serious adverse events. Electronic health record (EHR)-integrated dashboards are auditing tools of patients' status and clinicians' performance, but neither the views nor the performance of health professionals have been assessed, relating to management of deteriorating patients.

OBJECTIVE

To develop and evaluate a real-time dashboard of deteriorating patients' assessment, referral and therapy.

SETTINGS

Five academic hospitals in the largest National Health Service (NHS) trust in the UK (Barts Health NHS Trust).

INTERVENTION

The dashboard was developed from EHR data to investigate patients with National Early Warning Score (NEWS2)>5, assessment, and escalation of deteriorating patients. We adopted the Plan, Do, Study, Act model and Standards for Quality Improvement Reporting Excellence framework to evaluate the dashboard.

DESIGN

Mixed methods: (1) virtual, face-to-face, interviews and (2) retrospective descriptive EHR data analysis.

RESULTS

We interviewed three nurses (two quality and safety and one informatics specialists). Participants perceived the dashboard as a facilitator for auditing NEWS2 recording and escalation of care to improve practice; (2) there is a need for guiding clinicians and adjusting data sources and metrics to enhance the functionality and usability. Data analysis (2019-2022) showed: (1) NEWS2 recording has gradually improved (May 2021-April 2022) from 64% to 83%;(2) referral and assessment completion increased (n: 170-6800 and 23-540, respectively).

CONCLUSION

The dashboard is an effective real-time data-driven method for improving the quality of managing deteriorating patients. Integrating health systems, a wider analysis NEWS2 and escalation of care metrics, and clinicians' learning digital solutions will enhance functionality and experience to boost its value. There is a need to examine the generalisability of the dashboard through further validation and quality improvement studies.

Performance of universal early warning scores in different patient subgroups and clinical settings: a systematic review

OBJECTIVE

To assess predictive performance of universal early warning scores (EWS) in disease subgroups and clinical settings.

DESIGN

Systematic review.

DATA SOURCES

Medline, CINAHL, Embase and Cochrane database of systematic reviews from 1997 to 2019.

INCLUSION CRITERIA

Randomised trials and observational studies of internal or external validation of EWS to predict deterioration (mortality, intensive care unit (ICU) transfer and cardiac arrest) in disease subgroups or clinical settings.

RESULTS

We identified 770 studies, of which 103 were included. Study designs and methods were inconsistent, with significant risk of bias (high: n=16 and unclear: n=64 and low risk: n=28). There were only two randomised trials. There was a high degree of heterogeneity in all subgroups and in national early warning score (I2=72%-99%). Predictive accuracy (mean area under the curve; 95% CI) was highest in medical (0.74; 0.74 to 0.75) and surgical (0.77; 0.75 to 0.80) settings and respiratory diseases (0.77; 0.75 to 0.80). Few studies evaluated EWS in specific diseases, for example, cardiology (n=1) and respiratory (n=7). Mortality and ICU transfer were most frequently studied outcomes, and cardiac arrest was least examined (n=8). Integration with electronic health records was uncommon (n=9).

CONCLUSION

Methodology and quality of validation studies of EWS are insufficient to recommend their use in all diseases and all clinical settings despite good performance of EWS in some subgroups. There is urgent need for consistency in methods and study design, following consensus guidelines for predictive risk scores. Further research should consider specific diseases and settings, using electronic health record data, prior to large-scale implementation.

PROSPERO REGISTRATION NUMBER

PROSPERO CRD42019143141.