Performance of seven different paediatric early warning scores to predict critical care admission in febrile children presenting to the emergency department: a retrospective cohort study

Assessing children who are acutely ill in general practice using the National PEWS and LqSOFA clinical scores: a retrospective cohort study

BACKGROUND

Clinical tools are needed in general practice to help identify children who are seriously ill. The Liverpool quick Sequential Organ Failure Assessment (LqSOFA) was validated in an emergency department and performed well. The National Paediatric Early Warning System (PEWS) has been introduced in hospitals throughout England with hopes for implementation in general practice.

AIM

To validate the LqSOFA and National PEWS in general practice.

DESIGN AND SETTING

Secondary analysis of 6703 children aged <5 years presenting to 225 general practices in England and Wales with acute illnesses, linked to hospital data.

METHOD

Variables from the LqSOFA and National PEWS were mapped onto study data to calculate score totals. A primary outcome of admission within 2 days of GP consultation was used to calculate sensitivity, specificity, negative predictive values (NPVs), positive predictive values (PPVs), and area under the receiver operating characteristic curve (AUC).

RESULTS

A total of 104/6703 children were admitted to hospital within 2 days (pre-test probability 1.6%) of GP consultation. The sensitivity of the LqSOFA was 30.6% (95% confidence interval [CI] = 21.8% to 41.0%), with a specificity of 84.7% (95% CI = 83.7% to 85.6%), PPV of 3.0% (95% CI = 2.1% to 4.4%), NPV of 98.7% (95% CI = 98.4% to 99.0%), and AUC of 0.58 (95% CI = 0.53 to 0.63). The sensitivity of the National PEWS was 81.0% (95% CI = 71.0% to 88.1%), with a specificity of 32.5% (95% CI = 31.2% to 33.8%), PPV of 1.9% (95% CI = 1.5% to 2.5%), NPV of 99.1% (95% CI = 98.4% to 99.4%), and AUC of 0.66 (95% CI = 0.59 to 0.72).

CONCLUSION

Although the NPVs appear useful, owing to low pre-test probabilities rather than discriminative ability, neither tool accurately identified admissions to hospital. Unconsidered use by GPs could result in unsustainable referrals.

Susceptibility to childhood sepsis, contemporary management, and future directions

Sepsis disproportionally affects children across all health-care settings and is one of the leading causes of morbidity and mortality in neonatal and paediatric age groups. As shown in the first paper in this Series, the age-specific incidence of sepsis is highest during the first years of life, before approaching adult incidence rates during adolescence. In the second paper in this Series, we focus on the unique susceptibility of paediatric patients to sepsis and how the underlying dysregulated host response relates to developmental aspects of children's immune system, genetic, perinatal, and environmental factors, and comorbidities and socioeconomic determinants of health, which often differ between children and adults. State-of-the-art clinical management of paediatric sepsis is organised around three treatment pillars-diagnosis, early resuscitation, and titration of advanced care-and we examine available treatment guidelines and the limitations of their supporting evidence. Serious evidence gaps remain in key areas of paediatric sepsis care, especially surrounding recognition, common interventions, and survivor support, and to this end we offer a research roadmap for the next decade that could accelerate targeted diagnostics and personalised use of immunomodulation. However, improving outcomes for children with sepsis relies fundamentally on systematic quality improvement in both recognition and treatment, which is the theme of the third paper in this Series. Digital health, as shown in the fourth and final paper of this Series, holds promising potential in breaking down the barriers that hinder progress in paediatric sepsis care and, ultimately, global child health.

Quality improvement programmes in paediatric sepsis from a global perspective

Sepsis is a major contributor to poor child health outcomes around the world. The high morbidity, mortality, and societal cost associated with paediatric sepsis render it a global health priority, as summarised in Paper 1 of this Series. Sepsis is characterised by a dysregulated host response to infection that manifests as organ failure, and children are uniquely susceptible to sepsis, as discussed in Paper 2. The focus of this third Series paper is quality improvement in paediatric sepsis. The 2017 WHO resolution on sepsis outlined key aims to reduce the burden of sepsis. As of 2024, only a small number of countries have implemented systematic, paediatric-focused quality improvement programmes to raise sepsis awareness, enhance recognition of sepsis, promote timely treatment, and provide long-term support for paediatric sepsis survivors. We examine programme successes and systematic barriers to quality improvement targeting paediatric sepsis. We highlight the need for programme design to consider the entire patient journey, starting with prevention, caregiver awareness, recognition at home, education of the health-care workforce, development of health-care systems, and establishment of long-term family and survivor support extending beyond the intensive care unit. Building on lessons learnt from existing quality improvement programmes, we outline implementation strategies and measures to enable benchmarking. Ultimately, quality improvement on a global scale can only be accelerated through a global learning platform focusing on paediatric sepsis.

Predicting transfers to intensive care in children using CEWT and other early warning systems

Background and Objective

The Children's Early Warning Tool (CEWT), developed in Australia, is widely used in many countries to monitor the risk of deterioration in hospitalized children. Our objective was to compare CEWT prediction performance against a version of the Bedside Pediatric Early Warning Score (Bedside PEWS), Between the Flags (BTF), and the pediatric Calculated Assessment of Risk and Triage (pCART).

Methods

We conducted a retrospective observational study of all patient admissions to the Comer Children's Hospital at the University of Chicago between 2009-2019. We compared performance for predicting the primary outcome of a direct ward-to-intensive care unit (ICU) transfer within the next 12 h using the area under the receiver operating characteristic curve (AUC). Alert rates at various score thresholds were also compared.

Results

Of 50,815 ward admissions, 1,874 (3.7%) experienced the primary outcome. Among patients in Cohort 1 (years 2009-2017, on which the machine learning-based pCART was trained), CEWT performed slightly worse than Bedside PEWS but better than BTF (CEWT AUC 0.74 vs. Bedside PEWS 0.76, P < 0.001; vs. BTF 0.66, P < 0.001), while pCART performed best for patients in Cohort 2 (years 2018-2019, pCART AUC 0.84 vs. CEWT AUC 0.79, P < 0.001; vs. BTF AUC 0.67, P < 0.001; vs. Bedside PEWS 0.80, P < 0.001). Sensitivity, specificity, and positive predictive values varied across all four tools at the examined thresholds for alerts.

Conclusion

CEWT has good discrimination for predicting which patients will likely be transferred to the ICU, while pCART performed the best.

Delay Between Actual Occurrence of Patient Vital Sign and the Nominal Appearance in the Electronic Health Record: Single-Center, Retrospective Study of PICU Data, 2014-2018

OBJECTIVES

Patient vital sign data charted in the electronic health record (EHR) are used for time-sensitive decisions, yet little is known about when these data become nominally available compared with when the vital sign was actually measured. The objective of this study was to determine the magnitude of any delay between when a vital sign was actually measured in a patient and when it nominally appears in the EHR.

DESIGN

We performed a single-center retrospective cohort study.

SETTING

Tertiary academic children's hospital.

PATIENTS

A total of 5,458 patients were admitted to a PICU from January 2014 to December 2018.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We analyzed entry and display times of all vital signs entered in the EHR. The primary outcome measurement was time between vital sign occurrence and nominal timing of the vital sign in the EHR. An additional outcome measurement was the frequency of batch charting. A total of 9,818,901 vital sign recordings occurred during the study period. Across the entire cohort the median (interquartile range [IQR]) difference between time of occurrence and nominal time in the EHR was in hours:minutes:seconds, 00:41:58 (IQR 00:13:42-01:44:10). Lag in the first 24 hours of PICU admission was 00:47:34 (IQR 00:15:23-02:19:00), lag in the last 24 hours was 00:38:49 (IQR 00:13:09-01:29:22; p < 0.001). There were 1,892,143 occurrences of batch charting.

CONCLUSIONS

This retrospective study shows a lag between vital sign occurrence and its appearance in the EHR, as well as a frequent practice of batch charting. The magnitude of the delay-median ~40 minutes-suggests that vital signs available in the EHR for clinical review and incorporation into clinical alerts may be outdated by the time they are available.

Evaluating the performance of the National Early Warning Score in different diagnostic groups

BACKGROUND

The National Early Warning Score (NEWS) is used in hospitals across the UK to detect deterioration of patients within care pathways. It is used for most patients, but there are relatively few studies validating its performance in groups of patients with specific conditions.

METHODS

The performance of NEWS was evaluated against 36 other Early Warning Scores, in 123 patient groups, through use of the area under the receiver operating characteristic (AUROC) curve technique, to compare the abilities of each Early Warning Score to discriminate an outcome within 24hrs of vital sign recording. Outcomes evaluated were death, ICU admission, or a combined outcome of either death or ICU admission within 24 hours of an observation set.

RESULTS

The National Early Warning Score 2 performs either best or joint best within 120 of the 123 patient groups evaluated and is only outperformed in prediction of unanticipated ICU admission. When outperformed by other Early Warning Scores in the remaining 3 patient groups, the performance difference was marginal.

CONCLUSIONS

Consistently high performance indicates that NEWS is a suitable early warning score to use for all diagnostic groups considered by this analysis, and patients are not disadvantaged through use of NEWS in comparison to any of the other evaluated Early Warning Scores.

Impact of parental and healthcare professional concern on the diagnosis of pediatric sepsis: a diagnostic accuracy study

Objective

The Surviving Sepsis Campaign recommends systematic screening for sepsis. Although many sepsis screening tools include parent or healthcare professional concern, there remains a lack of evidence to support this practice. We aimed to test the diagnostic accuracy of parent and healthcare professional concern in relation to illness severity, to diagnose sepsis in children.

Design

This prospective multicenter study measured the level of concern for illness severity as perceived by the parent, treating nurse and doctor using a cross-sectional survey. The primary outcome was sepsis, defined as a pSOFA score >0. The unadjusted area under receiver-operating characteristic curves (AUC) and adjusted Odds Ratios (aOR) were calculated.

Setting

Two specialised pediatric Emergency Departments in Queensland.

Patients

Children aged 30 days to 18 years old that were evaluated for sepsis.

Intervention

None.

Main Results

492 children were included in the study, of which 118 (23.9%) had sepsis. Parent concern was not associated with sepsis (AUC 0.53, 95% CI: 0.46-0.61, aOR: 1.18; 0.89-1.58) but was for PICU admission (OR: 1.88, 95% CI: 1.17-3.19) and bacterial infection (aOR: 1.47, 95% CI: 1.14-1.92). Healthcare professional concern was associated with sepsis in both unadjusted and adjusted models (nurses: AUC 0.57, 95% CI-0.50, 0.63, aOR: 1.29, 95% CI: 1.02-1.63; doctors: AUC 0.63, 95% CI: 0.55, 0.70, aOR: 1.61, 95% CI: 1.14-2.19).

Conclusions

While our study does not support the broad use of parent or healthcare professional concern in isolation as a pediatric sepsis screening tool, measures of concern may be valuable as an adjunct in combination with other clinical data to support sepsis recognition.

Clinical Trial Registration

ACTRN12620001340921.

Validation of a paediatric sepsis screening tool to identify children with sepsis in the emergency department: a statewide prospective cohort study in Queensland, Australia

OBJECTIVE

The Surviving Sepsis Campaign guidelines recommend the implementation of systematic screening for sepsis. We aimed to validate a paediatric sepsis screening tool and derive a simplified screening tool.

DESIGN

Prospective multicentre study conducted between August 2018 and December 2019. We assessed the performance of the paediatric sepsis screening tool using stepwise multiple logistic regression analyses with 10-fold cross-validation and evaluated the final model at defined risk thresholds.

SETTING

Twelve emergency departments (EDs) in Queensland, Australia.

PARTICIPANTS

3473 children screened for sepsis, of which 523 (15.1%) were diagnosed with sepsis.

INTERVENTIONS

A 32-item paediatric sepsis screening tool including rapidly available information from triage, risk factors and targeted physical examination.

PRIMARY OUTCOME MEASURE

Senior medical officer-diagnosed sepsis combined with the administration of intravenous antibiotics in the ED.

RESULTS

The 32-item paediatric sepsis screening tool had good predictive performance (area under the receiver operating characteristic curve (AUC) 0.80, 95% CI 0.78 to 0.82). A simplified tool containing 16 of 32 criteria had comparable performance and retained an AUC of 0.80 (95% CI 0.78 to 0.82). To reach a sensitivity of 90% (95% CI 87% to 92%), the final model achieved a specificity of 51% (95% CI 49% to 53%). Sensitivity analyses using the outcomes of sepsis-associated organ dysfunction (AUC 0.84, 95% CI 0.81 to 0.87) and septic shock (AUC 0.84, 95% CI 0.81 to 0.88) confirmed the main results.

CONCLUSIONS

A simplified paediatric sepsis screening tool performed well to identify children with sepsis in the ED. Implementation of sepsis screening tools may improve the timely recognition and treatment of sepsis.

Mid-Regional Pro-Adrenomedullin in Combination With Pediatric Early Warning Scores for Risk Stratification of Febrile Children Presenting to the Emergency Department: Secondary Analysis of a Nonprespecified United Kingdom Cohort Study

OBJECTIVES

Current sepsis guidelines do not provide good risk stratification of subgroups in whom prompt IV antibiotics and fluid resuscitation might of benefit. We evaluated the utility of mid-regional pro-adrenomedullin (MR-proADM) in identification of patient subgroups at risk of requiring PICU or high-dependency unit (HDU) admission or fluid resuscitation.

DESIGN

Secondary, nonprespecified analysis of prospectively collected dataset.

SETTING

Pediatric Emergency Department in a United Kingdom tertiary center.

PATIENTS

Children less than 16 years old presenting with fever and clinical indication for venous blood sampling ( n = 1,183).

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Primary outcome measures were PICU/HDU admission or administration of fluid resuscitation, with a secondary outcome of definite or probable bacterial infection. Biomarkers were measured on stored plasma samples and children phenotyped into bacterial and viral groups using a previously published algorithm. Of the 1,183 cases, 146 children (12.3%) required fluids, 48 (4.1%) were admitted to the PICU/HDU, and 244 (20.6%) had definite or probable bacterial infection. Area under the receiver operating characteristic (AUC) was used to assess performance. MR-proADM better predicted fluid resuscitation (AUC, 0.73; 95% CI, 0.67-0.78), than both procalcitonin (AUC, 0.65; 95% CI, 0.59-0.71) and Pediatric Early Warning Score (PEWS: AUC, 0.62; 95% CI, 0.56-0.67). PEWS alone showed good accuracy for PICU/HDU admission 0.83 (0.78-0.89). Patient subgroups with high MR-proADM (≥ 0.7 nmol/L) and high procalcitonin (≥ 0.5 ng/mL) had increased association with PICU/HDU admission, fluid resuscitation, and bacterial infection compared with subgroups with low MR-proADM (< 0.7 nmol/L). For children with procalcitonin less than 0.5 ng/mL, high MR-proADM improved stratification for fluid resuscitation only.

CONCLUSIONS

High MR-proADM and high procalcitonin were associated with increased likelihood of subsequent disease progression. Incorporating MR-proADM into clinical risk stratification may be useful in clinician decision-making regarding initiation of IV antibiotics, fluid resuscitation, and escalation to PICU/HDU admission.

Health professionals' initial experiences and perceptions of the acceptability of a whole-hospital, pro-active electronic paediatric early warning system (the DETECT study): a qualitative interview study

Background

Paediatric early warning systems (PEWS) alert health professionals to signs of a child’s deterioration with the intention of triggering an urgent review and escalating care. They can reduce unplanned critical care transfer, cardiac arrest, and death. Electronic systems may be superior to paper-based systems. The objective of the study was to critically explore the initial experiences and perceptions of health professionals about the acceptability of DETECT e-PEWS, and what factors influence its acceptability.

Methods

A descriptive qualitative study (part of The DETECT study) was undertaken February 2020–2021. Single, semi-structured telephone interviews were used. The setting was a tertiary children’s hospital, UK. The participants were health professionals working in study setting and using DETECT e-PEWS. Sampling was undertaken using a mix of convenience and snowballing techniques. Participants represented two user-groups: ‘documenting vital signs’ (D-VS) and ‘responding to vital signs’ (R-VS). Perceptions of clinical utility and acceptability of DETECT e-PEWS were derived from thematic analysis of transcripts.

Results

Fourteen HPs (12 nurses, 2 doctors) participated; seven in D-VS and seven in the R-VS group. Three main themes were identified: complying with DETECT e-PEWS, circumventing DETECT e-PEWS, and disregarding DETECT e-PEWS. Overall clinical utility and acceptability were deemed good for HPs in the D-VS group but there was diversity in perception in the R-VS group (nurses found it more acceptable than doctors). Compliance was better in the D-VS group where use of DETECT e-PEWS was mandated and used more consistently. Some health professionals circumvented DETECT e-PEWS and fell back into old habits. Doctors (R-VS) did not consistently engage with DETECT e-PEWS, which reduced the acceptability of the system, even in those who thought the system brought benefits.

Conclusions

Speed and accuracy of real-time data, automation of triggering alerts and improved situational awareness were key factors that contributed to the acceptability of DETECT e-PEWS. Mandating use of both recording and responding aspects of DETECT e-PEWS is needed to ensure full implementation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12887-022-03411-1.