Evaluation of NEWS2 response thresholds in a retrospective observational study from a UK acute hospital

Real-world implementation of the National Early Warning Score-2 in an acute respiratory unit

INTRODUCTION

The National Early Warning Score-2 (NEWS-2) is used to detect deteriorating patients in hospital settings. We aimed to understand how NEWS-2 functions in the real-life setting of an acute respiratory unit.

METHODS

Clinical observations data were extracted for adult patients (age ≥18 years), admitted under the care of respiratory medicine services from July to December 2019, who had at least one recorded task relating to clinical deterioration. The timing and nature of urgent out-of-hours medical reviews (escalations) were extracted through manual review of the case notes.

RESULTS

The data set comprised 765 admission episodes (48.9% women) with a mean (SD) age of 69.3 (14.8). 8971 out of 35 991 out-of-hours observation sets (24.9%) had a NEWS-2 ≥5, and 586 of these (6.5%) led to an escalation. Out of 687 escalations, 101 (14.7%) were associated with observation sets with NEWS-2<5. Rising oxygen requirement and extreme values of individual observations were associated with an increased risk of escalation. 57.6% of escalations resulted in a change in treatment. Inpatient mortality was higher in patients who were escalated at least once, compared with those who were not escalated.

CONCLUSIONS

Most observation sets with NEWS-2 scores ≥5 did not lead to a medical escalation in an acute respiratory setting out-of-hours, but more than half of escalations resulted in a change in treatment. Rising oxygen requirement is a key indicator of respiratory patient acuity which appears to influence the decision to request urgent out-of-hours medical reviews.

Explainable hierarchical clustering for patient subtyping and risk prediction

We present a pipeline in which machine learning techniques are used to automatically identify and evaluate subtypes of hospital patients admitted between 2017 and 2021 in a large UK teaching hospital. Patient clusters are determined using routinely collected hospital data, such as those used in the UK's National Early Warning Score 2 (NEWS2). An iterative, hierarchical clustering process was used to identify the minimum set of relevant features for cluster separation. With the use of state-of-the-art explainability techniques, the identified subtypes are interpreted and assigned clinical meaning, illustrating their robustness. In parallel, clinicians assessed intracluster similarities and intercluster differences of the identified patient subtypes within the context of their clinical knowledge. For each cluster, outcome prediction models were trained and their forecasting ability was illustrated against the NEWS2 of the unclustered patient cohort. These preliminary results suggest that subtype models can outperform the established NEWS2 method, providing improved prediction of patient deterioration. By considering both the computational outputs and clinician-based explanations in patient subtyping, we aim to highlight the mutual benefit of combining machine learning techniques with clinical expertise.

Variability and performance of NHS England's 'reason to reside' criteria in predicting hospital discharge in acute hospitals in England: a retrospective, observational cohort study

OBJECTIVES

NHS England (NHSE) advocates 'reason to reside' (R2R) criteria to support discharge planning. The proportion of patients without R2R and their rate of discharge are reported daily by acute hospitals in England. R2R has no interoperable standardised data model (SDM), and its performance has not been validated. We aimed to understand the degree of intercentre and intracentre variation in R2R-related metrics reported to NHSE, define an SDM implemented within a single centre Electronic Health Record to generate an electronic R2R (eR2R) and evaluate its performance in predicting subsequent discharge.

DESIGN

Retrospective observational cohort study using routinely collected health data.

SETTING

122 NHS Trusts in England for national reporting and an acute hospital in England for local reporting.

PARTICIPANTS

6 602 706 patient-days were analysed using 3-month national data and 1 039 592 patient-days, using 3-year single centre data.

MAIN OUTCOME MEASURES

Variability in R2R-related metrics reported to NHSE. Performance of eR2R in predicting discharge within 24 hours.

RESULTS

There were high levels of intracentre and intercentre variability in R2R-related metrics (p<0.0001) but not in eR2R. Informedness of eR2R for discharge within 24 hours was low (J-statistic 0.09-0.12 across three consecutive years). In those remaining in hospital without eR2R, 61.2% met eR2R criteria on subsequent days (76% within 24 hours), most commonly due to increased NEWS2 (21.9%) or intravenous therapy administration (32.8%).

CONCLUSIONS

Reported R2R metrics are highly variable between and within acute Trusts in England. Although case-mix or community care provision may account for some variability, the absence of a SDM prevents standardised reporting. Following the development of a SDM in one acute Trust, the variability reduced. However, the performance of eR2R was poor, prone to change even when negative and unable to meaningfully contribute to discharge planning.

Effect of implementing the NEWS2 escalation protocol in a large acute NHS trust: a retrospective cohort analysis of mortality, workload and ability of early warning score to predict death within 24 hours

OBJECTIVES

To describe the inpatient population, establish patterns in admission and mortality over a 4-year period in different cohorts and assess the prognostic ability and workload implications of introducing the National Early Warning Score 2 (NEWS2) and associated escalation protocol.

DESIGN

Retrospective cohort analyses of medical and surgical inpatient admissions.

SETTING

Large teaching hospital with tertiary inpatient care and a major trauma centre employing an electronic observations platform, initially with a local early warning score, followed by NEWS2 introduction in June 2019.

PARTICIPANTS

332 682 adult patients were admitted between 1 January 2016 and 31 December 2019.

OUTCOME MEASURES

Mortality, workload and ability of early warning score to predict death within 24 hours.

RESULTS

Admissions rose by 19% from 76 055 in 2016 to 90 587 in 2019. Total bed days rose by 10% from 433 382 to 477 485. Mortality fell from 3.7% to 3.1% and was significantly lower in patients discharged from a surgical specialty, 1.0%-1.2% (p<0.001). Total observations recorded increased by 14% from 1 976 872 in 2016 to 2 249 118 in 2019. 65% of observations were attributable to patients under medical specialties, 34% to patients under surgical specialties. Recorded escalations to the registrar were stable from January 2016 to May 2019 but trebled following the introduction of NEWS2 in June 2019.

CONCLUSIONS

There was an increase in hospital inpatient activity between 2016 and 2019, associated with a reduction in mortality and percentage of observations calculated as reaching threshold NEWS2 score of 7 for escalation to the registrar. The introduction of the NEWS2, with a higher sensitivity and lower specificity, when allied to its escalation protocol, was associated with a significant increase in actual recorded escalations to the registrar. This was more marked in the surgical population and would support refining threshold scores based on admission characteristics when developing the next iteration of NEWS.

Dynamic early warning scores for predicting clinical deterioration in patients with respiratory disease

Background

The National Early Warning Score-2 (NEWS-2) is used to detect patient deterioration in UK hospitals but fails to take account of the detailed granularity or temporal trends in clinical observations. We used data-driven methods to develop dynamic early warning scores (DEWS) to address these deficiencies, and tested their accuracy in patients with respiratory disease for predicting (1) death or intensive care unit admission, occurring within 24 h (D/ICU), and (2) clinically significant deterioration requiring urgent intervention, occurring within 4 h (CSD).

Methods

Clinical observations data were extracted from electronic records for 31,590 respiratory in-patient episodes from April 2015 to December 2020 at a large acute NHS Trust. The timing of D/ICU was extracted for all episodes. 1100 in-patient episodes were annotated manually to record the timing of CSD, defined as a specific event requiring a change in treatment. Time series features were entered into logistic regression models to derive DEWS for each of the clinical outcomes. Area under the receiver operating characteristic curve (AUROC) was the primary measure of model accuracy.

Results

AUROC (95% confidence interval) for predicting D/ICU was 0.857 (0.852–0.862) for NEWS-2 and 0.906 (0.899–0.914) for DEWS in the validation data. AUROC for predicting CSD was 0.829 (0.817–0.842) for NEWS-2 and 0.877 (0.862–0.892) for DEWS. NEWS-2 ≥ 5 had sensitivity of 88.2% and specificity of 54.2% for predicting CSD, while DEWS ≥ 0.021 had higher sensitivity of 93.6% and approximately the same specificity of 54.3% for the same outcome. Using these cut-offs, 315 out of 347 (90.8%) CSD events were detected by both NEWS-2 and DEWS, at the time of the event or within the previous 4 h; 12 (3.5%) were detected by DEWS but not by NEWS-2, while 4 (1.2%) were detected by NEWS-2 but not by DEWS; 16 (4.6%) were not detected by either scoring system.

Conclusion

We have developed DEWS that display greater accuracy than NEWS-2 for predicting clinical deterioration events in patients with respiratory disease. Prospective validation studies are required to assess whether DEWS can be used to reduce missed deteriorations and false alarms in real-life clinical settings.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12931-022-02130-6.