Derivation and internal validation of the screening to enhance prehospital identification of sepsis (SEPSIS) score in adults on arrival at the emergency department

Prehospital early warning scores for adults with suspected sepsis: the PHEWS observational cohort and decision-analytic modelling study

Background

Guidelines for sepsis recommend treating those at highest risk within 1 hour. The emergency care system can only achieve this if sepsis is recognised and prioritised. Ambulance services can use prehospital early warning scores alongside paramedic diagnostic impression to prioritise patients for treatment or early assessment in the emergency department.

Objectives

To determine the accuracy, impact and cost-effectiveness of using early warning scores alongside paramedic diagnostic impression to identify sepsis requiring urgent treatment.

Design

Retrospective diagnostic cohort study and decision-analytic modelling of operational consequences and cost-effectiveness.

Setting

Two ambulance services and four acute hospitals in England.

Participants

Adults transported to hospital by emergency ambulance, excluding episodes with injury, mental health problems, cardiac arrest, direct transfer to specialist services, or no vital signs recorded.

Interventions

Twenty-one early warning scores used alongside paramedic diagnostic impression, categorised as sepsis, infection, non-specific presentation, or other specific presentation.

Main outcome measures

Proportion of cases prioritised at the four hospitals; diagnostic accuracy for the sepsis-3 definition of sepsis and receiving urgent treatment (primary reference standard); daily number of cases with and without sepsis prioritised at a large and a small hospital; the minimum treatment effect associated with prioritisation at which each strategy would be cost-effective, compared to no prioritisation, assuming willingness to pay £20,000 per quality-adjusted life-year gained.

Results

Data from 95,022 episodes involving 71,204 patients across four hospitals showed that most early warning scores operating at their pre-specified thresholds would prioritise more than 10% of cases when applied to non-specific attendances or all attendances. Data from 12,870 episodes at one hospital identified 348 (2.7%) with the primary reference standard. The National Early Warning Score, version 2 (NEWS2), had the highest area under the receiver operating characteristic curve when applied only to patients with a paramedic diagnostic impression of sepsis or infection (0.756, 95% confidence interval 0.729 to 0.783) or sepsis alone (0.655, 95% confidence interval 0.63 to 0.68). None of the strategies provided high sensitivity (> 0.8) with acceptable positive predictive value (> 0.15). NEWS2 provided combinations of sensitivity and specificity that were similar or superior to all other early warning scores. Applying NEWS2 to paramedic diagnostic impression of sepsis or infection with thresholds of > 4, > 6 and > 8 respectively provided sensitivities and positive predictive values (95% confidence interval) of 0.522 (0.469 to 0.574) and 0.216 (0.189 to 0.245), 0.447 (0.395 to 0.499) and 0.274 (0.239 to 0.313), and 0.314 (0.268 to 0.365) and 0.333 (confidence interval 0.284 to 0.386). The mortality relative risk reduction from prioritisation at which each strategy would be cost-effective exceeded 0.975 for all strategies analysed.

Limitations

We estimated accuracy using a sample of older patients at one hospital. Reliable evidence was not available to estimate the effectiveness of prioritisation in the decision-analytic modelling.

Conclusions

No strategy is ideal but using NEWS2, in patients with a paramedic diagnostic impression of infection or sepsis could identify one-third to half of sepsis cases without prioritising unmanageable numbers. No other score provided clearly superior accuracy to NEWS2. Research is needed to develop better definition, diagnosis and treatments for sepsis.

Study registration

This study is registered as Research Registry (reference: researchregistry5268).

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 17/136/10) and is published in full in Health Technology Assessment; Vol. 28, No. 16. See the NIHR Funding and Awards website for further award information.

Prehospital early warning scores for adults with suspected sepsis: retrospective diagnostic cohort study

BACKGROUND

Ambulance services need to identify and prioritise patients with sepsis for early hospital assessment. We aimed to determine the accuracy of early warning scores alongside paramedic diagnostic impression to identify sepsis that required urgent treatment.

METHODS

We undertook a retrospective diagnostic cohort study involving adult emergency medical cases transported to Sheffield Teaching Hospitals ED by Yorkshire Ambulance Service in 2019. We used routine ambulance service data to calculate 21 early warning scores and categorise paramedic diagnostic impressions as sepsis, infection, non-specific presentation or other presentation. We linked cases to hospital records and identified those meeting the sepsis-3 definition who received urgent hospital treatment for sepsis (reference standard). Analysis determined the accuracy of strategies that combined early warning scores at varying thresholds for positivity with paramedic diagnostic impression.

RESULTS

We linked 12 870/24 955 (51.6%) cases and identified 348/12 870 (2.7%) with a positive reference standard. None of the strategies provided sensitivity greater than 0.80 with positive predictive value greater than 0.15. The area under the receiver operating characteristic curve for the National Early Warning Score, version 2 (NEWS2) applied to patients with a diagnostic impression of sepsis or infection was 0.756 (95% CI 0.729, 0.783). No other early warning score provided clearly superior accuracy to NEWS2. Paramedic impression of sepsis or infection had sensitivity of 0.572 (0.519, 0.623) and positive predictive value of 0.156 (0.137, 0.176). NEWS2 thresholds of >4, >6 and >8 applied to patients with a diagnostic impression of sepsis or infection, respectively, provided sensitivities and positive predictive values of 0.522 (0.469, 0.574) and 0.216 (0.189, 0.245), 0.447 (0.395, 0.499) and 0.274 (0.239, 0.313), and 0.314 (0.268, 0.365) and 0.333 (0.284, 0.386).

CONCLUSION

No strategy is ideal but using NEWS2 alongside paramedic diagnostic impression of infection or sepsis could identify one-third to half of sepsis cases without prioritising unmanageable numbers. No other score provided clearly superior accuracy to NEWS2.

TRIAL REGISTRATION NUMBER

researchregistry5268, https://www.researchregistry.com/browse-the-registry%23home/registrationdetails/5de7bbd97ca5b50015041c33/.

Evidence for Use of Validated Sepsis Screening Tools in the Prehospital Population: A Scoping Review

INTRODUCTION

Early detection and treatment of sepsis improves chances of survival; however, sepsis is often difficult to diagnose initially. This is especially true in the prehospital setting, where resources are scarce, yet time is of great significance. Early warning scores (EWS) based on vital signs were originally developed to guide medical practitioners in determining the degree of illness of a patient in the in-patient setting. These EWS were adapted for use in the prehospital setting to predict critical illness and sepsis. We performed a scoping review to evaluate the existing evidence for use of validated EWS to identify prehospital sepsis.

METHODS

We performed a systematic search using the CINAHL, Embase, Ovid-MEDLINE, and PubMed databases on September 1, 2022. Articles that examined the use of EWS to identify prehospital sepsis were included and assessed.

RESULTS

Twenty-three studies were included in this review: one validation study, two prospective studies, two systematic reviews, and 18 retrospective studies. Study characteristics, classification statistics, and primary conclusions of each article were extracted and tabulated. Classification statistics varied markedly for prehospital sepsis identification across all included EWS: sensitivities ranged from 0.02-1.00, specificities from 0.07-1.00, and PPV and NPV from 0.19-0.98 and 0.32-1.00, respectively.

CONCLUSIONS

All studies demonstrated inconsistency for the identification of prehospital sepsis. The variety of available EWS and study design heterogeneity suggest it is unlikely that new research can identify a single gold standard score. Based on our findings in this scoping review, we recommend future efforts focus on combining standardized prehospital care with clinical judgment to provide timely interventions for unstable patients where infection is considered a likely etiology, in addition to improving sepsis education for prehospital clinicians. At most, EWS can be used as an adjunct to these efforts, but they should not be relied on alone for prehospital sepsis identification.

Supporting the ambulance service to safely convey fewer patients to hospital by developing a risk prediction tool: Risk of Adverse Outcomes after a Suspected Seizure (RADOSS)-protocol for the mixed-methods observational RADOSS project

INTRODUCTION

Ambulances services are asked to further reduce avoidable conveyances to emergency departments (EDs). Risk of Adverse Outcomes after a Suspected Seizure seeks to support this by: (1) clarifying the risks of conveyance and non-conveyance, and (2) developing a risk prediction tool for clinicians to use 'on scene' to estimate the benefits an individual would receive if conveyed to ED and risks if not.

METHODS AND ANALYSIS

Mixed-methods, multi-work package (WP) project. For WP1 and WP2 we shall use an existing linked data set that tracks urgent and emergency care (UEC) use of persons served by one English regional ambulance service. Risk tools are specific to clinical scenarios. We shall use suspected seizures in adults as an exemplar.WP1: Form a cohort of patients cared for a seizure by the service during 2019/2020. It, and nested Knowledge Exchange workshops with clinicians and service users, will allow us to: determine the proportions following conveyance and non-conveyance that die and/or recontact UEC system within 3 (/30) days; quantify the proportion of conveyed incidents resulting in 'avoidable ED attendances' (AA); optimise risk tool development; and develop statistical models that, using information available 'on scene', predict the risk of death/recontact with the UEC system within 3 (/30) days and the likelihood of an attendance at ED resulting in an AA.WP2: Form a cohort of patients cared for a seizure during 2021/2022 to 'temporally' validate the WP1 predictive models.WP3: Complete the 'next steps' workshops with stakeholders. Using nominal group techniques, finalise plans to develop the risk tool for clinical use and its evaluation.

ETHICS AND DISSEMINATION

WP1a and WP2 will be conducted under database ethical approval (IRAS 307353) and Confidentiality Advisory Group (22/CAG/0019) approval. WP1b and WP3 have approval from the University of Liverpool Central Research Ethics Committee (11450). We shall engage in proactive dissemination and knowledge mobilisation to share findings with stakeholders and maximise evidence usage.

Screening strategies to identify sepsis in the prehospital setting: a validation study

BACKGROUND

In the prehospital setting, differentiating patients who have sepsis from those who have infection but no organ dysfunction is important to initiate sepsis treatments appropriately. We aimed to identify which published screening strategies for paramedics to use in identifying patients with sepsis provide the most certainty for prehospital diagnosis.

METHODS

We identified published strategies for screening by paramedics through a literature search. We then conducted a validation study in Alberta, Canada, from April 2015 to March 2016. For adult patients (≥ 18 yr) who were transferred by ambulance, we linked records to an administrative database and then restricted the search to patients with infection diagnosed in the emergency department. For each patient, the classification from each strategy was determined and compared with the diagnosis recorded in the emergency department. For all strategies that generated numeric scores, we constructed diagnostic prediction models to estimate the probability of sepsis being diagnosed in the emergency department.

RESULTS

We identified 21 unique prehospital screening strategies, 14 of which had numeric scores. We linked a total of 131 745 eligible patients to hospital databases. No single strategy had both high sensitivity (overall range 0.02-0.85) and high specificity (overall range 0.38-0.99) for classifying sepsis. However, the Critical Illness Prediction (CIP) score, the National Early Warning Score (NEWS) and the Quick Sepsis-Related Organ Failure Assessment (qSOFA) score predicted a low to high probability of a sepsis diagnosis at different scores. The qSOFA identified patients with a 7% (lowest score) to 87% (highest score) probability of sepsis diagnosis.

INTERPRETATION

The CIP, NEWS and qSOFA scores are tools with good predictive ability for sepsis diagnosis in the prehospital setting. The qSOFA score is simple to calculate and may be useful to paramedics in screening patients with possible sepsis.