Development and initial validation of the Bedside Paediatric Early Warning System score

Validation of the Children's Hospital Early Warning System for Critical Deterioration Recognition

OBJECTIVE

Early warning scores, such as the Children's Hospital Early Warning Score (CHEWS), are used by hospitals to identify patients at risk for critical deterioration and trigger clinicians to intervene and prevent further deterioration. This study's objectives were to validate the CHEWS and to compare the CHEWS to the previously validated Brighton Pediatric Early Warning Score (PEWS) for early detection of critical deterioration in hospitalized, non-cardiac patients at a pediatric hospital.

DESIGN AND METHODS

A retrospective cohort study reviewed medical and surgical patients at a quaternary academic pediatric hospital. CHEWS scores and abstracted PEWS scores were obtained on cases (n=360) and a randomly selected comparison sample (n=776). Specificity, sensitivity, area under the receiver-operating characteristic curves (AUROC) and early warning times were calculated for both scoring tools.

RESULTS

The AUROC for CHEWS was 0.902 compared to 0.798 for PEWS (p<0.001). Sensitivity for scores ≥3 was 91.4% for CHEWS and 73.6% for PEWS with specificity of 67.8% for CHEWS and 88.5% for PEWS. Sensitivity for scores ≥5 was 75.6% for CHEWS and 38.9% for PEWS with specificity of 88.5% for CHEWS and 93.9% for PEWS. The early warning time from critical score (≥5) to critical deterioration was 3.8h for CHEWS versus 0.6h for PEWS (p<0.001).

CONCLUSION

The CHEWS system demonstrated higher discrimination, higher sensitivity and longer early warning time than the PEWS for identifying children at risk for critical deterioration.

Poor inter-observer agreement in the measurement of respiratory rate in children: a prospective observational study

OBJECTIVE

To determine the inter-observer agreement of a respiratory rate (RR) count on a child when assessed by three independent observers.

DESIGN

The RR of 169 children (age range: 3 days to 15 years) was measured by three independent observers over a 3-month period. The first RR was taken by different healthcare professionals (HCPs) from within the hospital using their own preferred method of measurement. A further count of RR was then taken by two observers from the research team simultaneously within 30 min of the first measurement, using the WHO-recommended method of measurement.

RESULTS

507 RR measurements were taken on 169 children. Median RR showed a 4 beats per minute (bpm) difference between the HCP (median RR 32 bpm) and the researchers (median RR 28 bpm). The 95% limits of agreement between the first measurement and second and third measurements were -10.2 to 17.7 bpm and -11.4 to 18.7 bpm, respectively. For simultaneous measurements, the 95% limits of agreement were -7.1 to 7.0 bpm. 81 children had a RR > 95th centile for their age and an even poorer level of agreement was seen in these children than in those whose RR was within normal range. In only 27 of these 81 children (33%) did all three observers agree on the presence of a raised RR.

CONCLUSIONS

Inter-observer agreement for the measurement of RR in children is poor. The effect that this variation has on the clinical assessment and subsequent management of a child may be significant. These findings highlight the need for a robust review of our current measurement methods and interpretation of an important vital sign.

Predicting Severe Pneumonia Outcomes in Children

BACKGROUND

Substantial morbidity and excessive care variation are seen with pediatric pneumonia. Accurate risk-stratification tools to guide clinical decision-making are needed.

METHODS

We developed risk models to predict severe pneumonia outcomes in children (<18 years) by using data from the Etiology of Pneumonia in the Community Study, a prospective study of community-acquired pneumonia hospitalizations conducted in 3 US cities from January 2010 to June 2012. In-hospital outcomes were organized into an ordinal severity scale encompassing severe (mechanical ventilation, shock, or death), moderate (intensive care admission only), and mild (non-intensive care hospitalization) outcomes. Twenty predictors, including patient, laboratory, and radiographic characteristics at presentation, were evaluated in 3 models: a full model included all 20 predictors, a reduced model included 10 predictors based on expert consensus, and an electronic health record (EHR) model included 9 predictors typically available as structured data within comprehensive EHRs. Ordinal regression was used for model development. Predictive accuracy was estimated by using discrimination (concordance index).

RESULTS

Among the 2319 included children, 21% had a moderate or severe outcome (14% moderate, 7% severe). Each of the models accurately identified risk for moderate or severe pneumonia (concordance index across models 0.78-0.81). Age, vital signs, chest indrawing, and radiologic infiltrate pattern were the strongest predictors of severity. The reduced and EHR models retained most of the strongest predictors and performed as well as the full model.

CONCLUSIONS

We created 3 risk models that accurately estimate risk for severe pneumonia in children. Their use holds the potential to improve care and outcomes.

Development and validation of an electronic medical record-based alert score for detection of inpatient deterioration outside the ICU

BACKGROUND

Patients in general medical-surgical wards who experience unplanned transfer to the intensive care unit (ICU) show evidence of physiologic derangement 6-24h prior to their deterioration. With increasing availability of electronic medical records (EMRs), automated early warning scores (EWSs) are becoming feasible.

OBJECTIVE

To describe the development and performance of an automated EWS based on EMR data.

MATERIALS AND METHODS

We used a discrete-time logistic regression model to obtain an hourly risk score to predict unplanned transfer to the ICU within the next 12h. The model was based on hospitalization episodes from all adult patients (18years) admitted to 21 Kaiser Permanente Northern California (KPNC) hospitals from 1/1/2010 to 12/31/2013. Eligible patients met these entry criteria: initial hospitalization occurred at a KPNC hospital; the hospitalization was not for childbirth; and the EMR had been operational at the hospital for at least 3months. We evaluated the performance of this risk score, called Advanced Alert Monitor (AAM) and compared it against two other EWSs (eCART and NEWS) in terms of their sensitivity, specificity, negative predictive value, positive predictive value, and area under the receiver operator characteristic curve (c statistic).

RESULTS

A total of 649,418 hospitalization episodes involving 374,838 patients met inclusion criteria, with 19,153 of the episodes experiencing at least one outcome. The analysis data set had 48,723,248 hourly observations. Predictors included physiologic data (laboratory tests and vital signs); neurological status; severity of illness and longitudinal comorbidity indices; care directives; and health services indicators (e.g. elapsed length of stay). AAM showed better performance compared to NEWS and eCART in all the metrics and prediction intervals. The AAM AUC was 0.82 compared to 0.79 and 0.76 for eCART and NEWS, respectively. Using a threshold that generated 1 alert per day in a unit with a patient census of 35, the sensitivity of AAM was 49% (95% CI: 47.6-50.3%) compared to the sensitivities of eCART and NEWS scores of 44% (42.3-45.1) and 40% (38.2-40.9), respectively. For all three scores, about half of alerts occurred within 12h of the event, and almost two thirds within 24h of the event.

CONCLUSION

The AAM score is an example of a score that takes advantage of multiple data streams now available in modern EMRs. It highlights the ability to harness complex algorithms to maximize signal extraction. The main challenge in the future is to develop detection approaches for patients in whom data are sparser because their baseline risk is lower.

CLARIPED: a new tool for risk classification in pediatric emergencies

Objective:

To present a new pediatric risk classification tool, CLARIPED, and describe its development steps.

Methods:

Development steps: (i) first round of discussion among experts, first prototype; (ii) pre-test of reliability, 36 hypothetical cases; (iii) second round of discussion to perform adjustments; (iv) team training; (v) pre-test with patients in real time; (vi) third round of discussion to perform new adjustments; (vii) final pre-test of validity (20% of medical treatments in five days).

Results:

CLARIPED features five urgency categories: Red (Emergency), Orange (very urgent), Yellow (urgent), Green (little urgent) and Blue (not urgent). The first classification step includes the measurement of four vital signs (VIPE score); the second step consists in the urgency discrimination assessment. Each step results in assigning a color, selecting the most urgent one for the final classification. Each color corresponds to a maximum waiting time for medical care and referral to the most appropriate physical area for the patient's clinical condition. The interobserver agreement was substantial (kappa=0.79) and the final pre-test, with 82 medical treatments, showed good correlation between the proportion of patients in each urgency category and the number of used resources (p<0.001).

Conclusions:

CLARIPED is an objective and easy-to-use tool for simple risk classification, of which pre-tests suggest good reliability and validity. Larger-scale studies on its validity and reliability in different health contexts are ongoing and can contribute to the implementation of a nationwide pediatric risk classification system.

Identifying High-Risk Children in the Emergency Department

PURPOSE

Early warning scores (EWS) identify high-risk hospitalized patients prior to clinical deterioration; however, their ability to identify high-risk pediatric patients in the emergency department (ED) has not been adequately evaluated. We sought to determine the association between modified pediatric EWS (MPEWS) in the ED and inpatient ward-to-pediatric intensive care unit (PICU) transfer within 24 hours of admission.

METHODS

This is a case-control study of 597 pediatric ED patients admitted to the inpatient ward at Seattle Children's Hospital between July 1, 2010, and December 31, 2011. Cases were children subsequently transferred to the PICU within 24 hours, whereas controls remained hospitalized on the inpatient ward. The association between MPEWS in the ED and ward-to-PICU transfer was determined by chi-square analysis.

RESULTS

Fifty children experienced ward-to-PICU transfer within 24 hours of admission. The area under the receiver-operator characteristic curve was 0.691. Children with MPEWS > 7 in the ED were more likely to experience ward-to-PICU transfer (odds ratio 8.36, 95% confidence interval 2.98-22.08); however, the sensitivity was only 18.0% with a specificity of 97.4%. Using MPEWS >7 for direct PICU admission would have led to 167 unnecessary PICU admissions and identified only 9 of 50 patients who required PICU care.

CONCLUSIONS

Elevated MPEWS in the ED is associated with increased risk of ward-to-PICU transfer within 24 hours of admission; however, an MPEWS threshold of 7 is not sufficient to identify more than a small proportion of ward-admitted children with subsequent clinical deterioration.

Description of PICU Unplanned Readmission

OBJECTIVES

ICU readmission within 48 hours of discharge is associated with increased mortality. The objectives of this study were to describe the frequency of, factors associated with, and outcomes associated with unplanned PICU readmission.

DESIGN

A retrospective case-control study was performed. We evaluated 13 candidate risk factors and report patient outcomes following readmission. Subgroup analyses were performed for patients discharged from the cardiac PICU and medical-surgical PICU.

SETTING

The study was undertaken at the Hospital for Sick Children, Department of Critical Care Medicine.

PATIENTS

Eligible patients were discharged from the PICU to an inpatient ward between December 2006 and January 2013. Case patients were readmitted to the PICU within 48 hours of discharge.

MEASUREMENTS AND MAIN RESULTS

There were 10,422 eligible patient discharges; 264 (2.5%) were readmitted within 48 hours. In the univariable analysis, unplanned readmission was associated with PICU patient admissions of younger age, lower weight, greater duration of PICU stay, greater cumulative stay in PICU in the past 2 years, higher Pediatric Logistic Organ Dysfunction score on PICU discharge, discharge outside goal discharge time (06:00-11:59 hr), use of extracorporeal organ support during ICU stay, greater Bedside Pediatric Early Warning Score, at discharge and discharge from the cardiac PICU. In the multivariable analysis, the factors most significantly associated with unplanned PICU readmission were length of stay more than 48 hours, greater cumulative length of PICU stay in the past 2 years, discharge from cardiac PICU, and higher Pediatric Logistic Organ Dysfunction and Bedside Pediatric Early Warning Scores on index discharge. Mortality was 1.8 times (p = 0.03) higher in patients with an unplanned PICU readmission compared with patients during their index PICU admission.

CONCLUSIONS

The only potentially modifiable factors we found associated with PICU readmission within 48 hours of discharge were discharge time of day and the Pediatric Logistic Organ Dysfunction and Bedside Pediatric Early Warning Scores at the time of PICU discharge.

Diagnostic accuracy of PAT-POPS and ManChEWS for admissions of children from the emergency department

Background

The Pennine Acute Trust (PAT) Paediatric Observation Priority Score (PAT-POPS) is a specific emergency department (ED) physiological and observational aggregate scoring system, with scores of 0–18. A higher score indicates greater likelihood of admission. The Manchester Children's Early Warning System (ManChEWS) assesses six physiological observations to create a trigger score, classified as Green, Amber or Red.

Methods

Prospectively collected data were used to calculate PAT-POPS and ManChEWS on 2068 patients aged under 16 years (mean 5.6 years, SD 4.6) presenting over 1 month to a UK District General Hospital Paediatric ED. Receiver operating characteristics (ROC) comparison, using STATA V.13, was used to investigate the ability of ManChEWS and PAT-POPS to predict admission to hospital within 72 h of presentation to the ED.

Results

Comparison of the area under the ROC curve indicates that the ManChEWS ROC is 0.67 (95% CI 0.64 to 0.70) and the PAT-POPS ROC is 0.72 (95% CI 0.68 to 0.75). The difference is statistically significant. At a PAT-POPS cut-off of ≥2, 80% of patients had their admission risk correctly classified (positive likelihood ratio 3.40, 95% CI 2.90 to 3.98) whereas for ManChEWS with a cut off of ≥Amber only 71% of patients were correctly classified (positive likelihood ratio 2.18, 95% CI 1.94 to 2.45).

Conclusions

PAT-POPS is a more accurate predictor of admission risk than ManChEWS. Replacing ManChEWS with PAT-POPS would appear to be clinically appropriate in a paediatric ED. This needs validation in a multicentre study.

CLARIPED: a new tool for risk classification in pediatric emergencies

OBJECTIVE

To present a new pediatric risk classification tool, CLARIPED, and describe its development steps.

METHODS

Development steps: (i) first round of discussion among experts, first prototype; (ii) pre-test of reliability, 36 hypothetical cases; (iii) second round of discussion to perform adjustments; (iv) team training; (v) pre-test with patients in real time; (vi) third round of discussion to perform new adjustments; (vii) final pre-test of validity (20% of medical treatments in five days).

RESULTS

CLARIPED features five urgency categories: Red (Emergency), Orange (very urgent), Yellow (urgent), Green (little urgent) and Blue (not urgent). The first classification step includes the measurement of four vital signs (Vipe score); the second step consists in the urgency discrimination assessment. Each step results in assigning a color, selecting the most urgent one for the final classification. Each color corresponds to a maximum waiting time for medical care and referral to the most appropriate physical area for the patient's clinical condition. The interobserver agreement was substantial (kappa=0.79) and the final pre-test, with 82 medical treatments, showed good correlation between the proportion of patients in each urgency category and the number of used resources (p<0.001).

CONCLUSIONS

CLARIPED is an objective and easy-to-use tool for simple risk classification, of which pre-tests suggest good reliability and validity. Larger-scale studies on its validity and reliability in different health contexts are ongoing and can contribute to the implementation of a nationwide pediatric risk classification system.

Evaluation and management of bradycardia in neonates and children

Heart rate is commonly used in pediatric early warning scores. Age-related changes in the anatomy and physiology of infants and children produce normal ranges for electrocardiogram features that differ from adults and vary with age. Bradycardia is defined as a heart rate below the lowest normal value for age. Pediatric bradycardia most commonly manifests as sinus bradycardia, junctional bradycardia, or atrioventricular block. As a result of several different etiologies, it may occur in an entirely structurally normal heart or in association with concomitant congenital heart disease. Genetic variants in multiple genes have been described to date in the pathogenesis of inherited sinus node dysfunction or progressive cardiac conduction disorders. Management and eventual prognosis of bradycardia in the young are entirely dependent upon the underlying cause. Reasons to intervene for bradycardia are the association of related symptoms and/or the downstream risk of heart failure or pause-dependent tachyarrhythmia. The simplest aspect of severe bradycardia management is reflected in the Pediatric and Advanced Life Support (PALS) guidelines.

CONCLUSION

Early diagnosis and appropriate management are critical in many cases in order to prevent sudden death, and this review critically assesses our current practice for evaluation and management of bradycardia in neonates and children.

WHAT IS KNOWN

Bradycardia is defined as a heart rate below the lowest normal value for age. Age related changes in the anatomy and physiology of infants and children produce normal ranges for electrocardiogram features that differ from adults and vary with age. Pediatric bradycardia most commonly manifests as sinus bradycardia, junctional bradycardia, or atrioventricular block.

WHAT IS NEW

Management and eventual prognosis of bradycardia in the young are entirely dependent upon the underlying cause. Bradycardia may occur in a structurally normal heart or in association with congenital heart disease. Genetic variants in multiple genes have been described. Reasons to intervene for bradycardia are the association of related symptoms and/or the downstream risk of heart failure or pause-dependent tachyarrhythmia. Early diagnosis and appropriate management are critical in order to prevent sudden death.

An Ethnographic Observational Study to Evaluate and Optimize the Use of Respiratory Acoustic Monitoring in Children Receiving Postoperative Opioid Infusions

BACKGROUND

Respiratory depression in children receiving postoperative opioid infusions is a significant risk because of the interindividual variability in analgesic requirement. Detection of respiratory depression (or apnea) in these children may be improved with the introduction of automated acoustic respiratory rate (RR) monitoring. However, early detection of adverse events must be balanced with the risk of alarm fatigue. Our objective was to evaluate the use of acoustic RR monitoring in children receiving opioid infusions on a postsurgical ward and identify the causes of false alarm and optimal alarm thresholds.

METHODS

A video ethnographic study was performed using an observational, mixed methods approach. After surgery, an acoustic RR sensor was placed on the participant's neck and attached to a Rad87 monitor. The monitor was networked with paging for alarms. Vital signs data and paging notification logs were obtained from the central monitoring system. Webcam videos of the participant, infusion pump, and Rad87 monitor were recorded, stored on a secure server, and subsequently analyzed by 2 research nurses to identify the cause of the alarm, response, and effectiveness. Alarms occurring within a 90-second window were grouped into a single-alarm response opportunity.

RESULTS

Data from 49 patients (30 females) with median age 14 (range, 4.4-18.8) years were analyzed. The 896 bedside vital sign threshold alarms resulted in 160 alarm response opportunities (44 low RR, 74 high RR, and 42 low SpO2). In 141 periods (88% of total), for which video was available, 65% of alarms were deemed effective (followed by an alarm-related action within 10 minutes). Nurses were the sole responders in 55% of effective alarms and the patient or parent in 20%. Episodes of desaturation (SpO2 < 90%) were observed in 9 patients: At the time of the SpO2 paging trigger, the RR was >10 bpm in 6 of 9 patients. Based on all RR samples observed, the default alarm thresholds, to serve as a starting point for each patient, would be a low RR of 6 (>10 years of age) and 10 (4-9 years of age).

CONCLUSIONS

In this study, the use of RR monitoring did not improve the detection of respiratory depression. An RR threshold, which would have been predictive of desaturations, would have resulted in an unacceptably high false alarm rate. Future research using a combination of variables (e.g., SpO2 and RR), or the measurement of tidal volumes, may be needed to improve patient safety in the postoperative ward.

Rapid Response Team Calls and Unplanned Transfers to the Pediatric Intensive Care Unit in a Pediatric Hospital

BACKGROUND

Variability in disposition of children according to the time of rapid response calls is unknown.

OBJECTIVE

To evaluate times and disposition of rapid response alerts and outcomes for children transferred from acute care to intensive care.

METHODS

Deidentified data on demographics, time and disposition of the child after activation of a rapid response, time of transfer to intensive care, and patient outcomes were reviewed retrospectively. Data for rapid-response patients on time of activation of the response and unplanned transfers to the intensive care unit were compared with data on other patients admitted to the unit.

RESULTS

Of 542 rapid responses activated, 321 (59.2%) were called during the daytime. Out of all rapid response activations, 323 children (59.6%) were transferred to intensive care, 164 (30.3%) remained on the general unit, and 19 (3.5%) required resuscitation. More children were transferred to intensive care after rapid response alerts (P = .048) during the daytime (66%) than at night (59%). During the same period, 1313 patients were transferred to intensive care from acute care units. Age, sex, risk of mortality, length of stay, and mortality rate did not differ according to the time of transfer. Mortality among unplanned transfers (3.8%) was significantly higher (P < .001) than among other intensive care patients (1.4%).

CONCLUSION

Only 25% of transfers from acute care units to the intensive care unit occurred after activation of a rapid response team. Most rapid responses were called during daytime hours. Mortality was significantly higher among unplanned transfers from acute care than among other intensive care admissions.

A pragmatic checklist to identify pediatric ICU patients at risk for cardiac arrest or code bell activation

BACKGROUND

In-hospital cardiac arrest is a rare event associated with significant morbidity and mortality. The ability to identify the ICU patients at risk for cardiac arrest could allow the clinical team to prepare staff and equipment in anticipation.

METHODS

This pilot study was completed at a large tertiary care pediatric intensive care unit to determine the feasibility of a simple checklist of clinical variables to predict deterioration. The daily checklist assessed patient risk for critical deterioration defined as cardiac arrest or code bell activation within 24h of the checklist screen. The Phase I checklist was developed by expert consensus and evaluated to determine standard diagnostic test performance. A modified Phase II checklist was developed to prospectively test the feasibility and bedside provider "number needed to train".

RESULTS

For identifying patients requiring code bell activation, both checklists demonstrated a sensitivity of 100% with specificity of 76.0% during Phase I and 97.7% during Phase II. The positive likelihood ratio improved from 4.2 to 43.7. For identifying patients that had a cardiac arrest within 24h, the Phase I and II checklists demonstrated a sensitivity of 100% with specificity again improving from 75.7% to 97.6%. There was an improved positive likelihood ratio from 4.1 in Phase I to 41.9 in Phase II, with improvement of "number needed to train" from 149 to 7.4 providers.

CONCLUSIONS

A novel high-risk clinical indicators checklist is feasible and provides timely and accurate identification of the ICU patients at risk for cardiac arrest or code bell activation.

Heart rate in pediatric trauma: rethink your strategy

BACKGROUND

The optimal heart rate (HR) for children after trauma is based on values derived at rest for a given age. As the stages of shock are based in part on HR, a better understanding of how HR varies after trauma is necessary. Admission HRs of pediatric trauma patients were analyzed to determine which ranges were associated with lowest mortality.

MATERIALS AND METHODS

The National Trauma Data Bank was used to evaluate all injured patients ages 1-14 years admitted between 2007 and 2011. Patients were stratified into eight groups based on age. Clinical characteristics and outcomes were recorded, and regression analysis was used to determine mortality odds ratios (ORs) for HR ranges within each age group.

RESULTS

A total of 214,254 pediatric trauma patients met inclusion criteria. The average admission HR and systolic blood pressure were 104.7 and 120.4, respectively. Overall mortality was 0.8%. The HR range associated with lowest mortality varied across age groups and, in children ages 7-14, was narrower than accepted resting HR ranges. The lowest risk of mortality for patients ages 5-14 was captured at HR 80-99.

CONCLUSIONS

The HR associated with lowest mortality after pediatric trauma frequently differs from resting HR. Our data suggest that a 7y old with an HR of 115 bpm may be in stage III shock, whereas traditional HR ranges suggest that this is a normal rate for this child. Knowing when HR is critically high or low in the pediatric trauma population will better guide treatment.

Accuracy of Bedside Paediatric Early Warning System (BedsidePEWS) in a Pediatric Stem Cell Transplant Unit

Hospital mortality in children who undergo stem cell transplant (SCT) is high. Early warning scores aim at identifying deteriorating patients and at preventing adverse outcomes. The bedside pediatric early warning system (BedsidePEWS) is a pediatric early warning score based on 7 clinical indicators, ranging from 0 (all indicators within normal ranges for age) to 26. The aim of this case-control study was to assess the performance of BedsidePEWS in identifying clinical deterioration events among children admitted to an SCT unit. Cases were defined as clinical deterioration events; controls were all the other patients hospitalized on the same ward at the time of case occurrence. BedsidePEWS was retrospectively measured at 4-hour intervals in cases and controls 24 hours before an event (T4-T24). We studied 19 cases and 80 controls. The score significantly increased in cases from a median of 4 at T24 to a median of 14 at T4. The proportion of correctly classified cases and controls was >90% since T8. The area under the curve receiver operating characteristic was 0.9. BedsidePEWS is an accurate screening tool to predict clinical deterioration in SCT patients.

A protocol for developing early warning score models from vital signs data in hospitals using ensembles of decision trees

INTRODUCTION

Multiple early warning scores (EWS) have been developed and implemented to reduce cardiac arrests on hospital wards. Case-control observational studies that generate an area under the receiver operator curve (AUROC) are the usual validation method, but investigators have also generated EWS with algorithms with no prior clinical knowledge. We present a protocol for the validation and comparison of our local Hamilton Early Warning Score (HEWS) with that generated using decision tree (DT) methods.

METHODS AND ANALYSIS

A database of electronically recorded vital signs from 4 medical and 4 surgical wards will be used to generate DT EWS (DT-HEWS). A third EWS will be generated using ensemble-based methods. Missing data will be multiple imputed. For a relative risk reduction of 50% in our composite outcome (cardiac or respiratory arrest, unanticipated intensive care unit (ICU) admission or hospital death) with a power of 80%, we calculated a sample size of 17,151 patient days based on our cardiac arrest rates in 2012. The performance of the National EWS, DT-HEWS and the ensemble EWS will be compared using AUROC.

ETHICS AND DISSEMINATION

Ethics approval was received from the Hamilton Integrated Research Ethics Board (#13-724-C). The vital signs and associated outcomes are stored in a database on our secure hospital server. Preliminary dissemination of this protocol was presented in abstract form at an international critical care meeting. Final results of this analysis will be used to improve on the existing HEWS and will be shared through publication and presentation at critical care meetings.

Evaluation of Electronic Medical Record Vital Sign Data Versus a Commercially Available Acuity Score in Predicting Need for Critical Intervention at a Tertiary Children's Hospital

OBJECTIVES

Evaluate the ability of vital sign data versus a commercially available acuity score adapted for children (pediatric Rothman Index) to predict need for critical intervention in hospitalized pediatric patients to form the foundation for an automated early warning system.

DESIGN

Retrospective review of electronic medical record data.

SETTING

Academic children's hospital.

PATIENTS

A total of 220 hospitalized children 6.7 ± 6.7 years old experiencing a cardiopulmonary arrest (condition A) and/or requiring urgent intervention with transfer (condition C) to the ICU between January 2006 and July 2011.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Physiologic data 24 hours preceding the event were extracted from the electronic medical record. Vital sign predictors were constructed using combinations of age-adjusted abnormalities in heart rate, systolic and diastolic blood pressures, respiratory rate, and peripheral oxygen saturation to predict impending deterioration. Sensitivity and specificity were determined for vital sign-based predictors by using 1:1 age-matched and sex-matched non-ICU control patients. Sensitivity and specificity for a model consisting of any two vital sign measurements simultaneously outside of age-adjusted normal ranges for condition A, condition C, and condition A or C were 64% and 54%, 57% and 53%, and 59% and 54%, respectively. The pediatric Rothman Index (added to the electronic medical record in April 2009) was evaluated in a subset of these patients (n = 131) and 16,138 hospitalized unmatched non-ICU control patients for the ability to predict condition A or C, and receiver operating characteristic curves were generated. Sensitivity and specificity for a pediatric Rothman Index cutoff of 40 for condition A, condition C, and condition A or C were 56% and 99%, 13% and 99%, and 28% and 99%, respectively.

CONCLUSIONS

A model consisting of simultaneous vital sign abnormalities and the pediatric Rothman Index predict condition A or C in the 24-hour period prior to the event. Vital sign only prediction models have higher sensitivity than the pediatric Rothman Index but are associated with a high false-positive rate. The high specificity of the pediatric Rothman Index merits prospective evaluation as an electronic adjunct to human-triggered early warning systems.

Comparison of Pediatric Early Warning Score to Physician Opinion for Deteriorating Patients

BACKGROUND

This study compares a Pediatric Early Warning Score (PEWS) to physician opinion in identifying patients at risk for deterioration.

METHODS

Maximum PEWS recorded during each admission was retrospectively ascertained from electronic medical record data. Physician opinion regarding risk of subsequent deterioration was determined by assignment to an institutional "senior sign-out" (SSO) list that highlights patients whom senior pediatric residents have identified as at risk. Deterioration events were defined as intubation, initiation of high flow nasal cannula, inotropes, noninvasive mechanical ventilation, or aggressive fluid resuscitation within 12 hours of transfer to the PICU. We assessed the relationships of sociodemographic variables, PEWS, and SSO assignment with subsequent deterioration events using multivariate regression analysis to control for a number of covariates.

RESULTS

There were 97 patients with nonelective transfers to the PICU who were eligible for placement on the SSO lists before transfer, 51 of whom experienced qualifying deterioration events. Maximum recorded PEWS was significantly higher for patients with a subsequent deterioration event during the first 12 hours after transfer, compared with those who were transferred but did not experience a deterioration event in the first 12 hours (mean [SD]: 3.9 [2.0] vs 2.9 [2.0]; P = .01). This association persisted even after multivariate adjustment. SSO assignment was only marginally associated with risk of deterioration among this patient population, with or without adjustment for covariates.

CONCLUSIONS

The PEWS was significantly associated with ICU deterioration, whereas physician opinion was not. Used alone or in conjunction with physician assessment, PEWS is a valuable tool for identifying patients vulnerable to acute deterioration.

New Insights Into Multicenter PICU Mortality Among Pediatric Hematopoietic Stem Cell Transplant Patients

OBJECTIVES

Over 2,500 children undergo hematopoietic stem cell transplantation in the United States each year, and up to 35% require PICU support for life-threatening complications. PICU mortality has dropped from 85% to 44%, but interpretation is confounded by significant cohort heterogeneity. Reports conflict regarding outcomes for patients with different underlying hematopoietic stem cell transplantation indications, and the burden of infectious complications for these patients has not been evaluated. We aim to describe infections, critical care interventions, and mortality for pediatric hematopoietic stem cell transplantation patients requiring PICU admission.

DESIGN

A retrospective multicenter cohort analysis.

SETTING

One hundred twelve centers in the Virtual PICU Systems database, January 1, 2009, to June 30, 2012.

PATIENTS

A total of 1,782 admissions for patients who are 21 years old or younger with prior hematopoietic stem cell transplantation.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Pediatric Index of Mortality-2, Pediatric Risk of Mortality-3, transplant indication, infections, interventions, and mortality were recorded from admission through PICU death or discharge. Pediatric hematopoietic stem cell transplantation patients comprised 0.7% of all PICU admissions (1,782/246,346), which resulted in 16.2% mortality compared with 2.4% mortality for non-hematopoietic stem cell transplantation admissions (odds ratio, 7.8; 95% CI, 6.8-8.8; p < 0.001). Mortality for admissions with underlying hematologic malignancy (22.7%) was similar to that of admissions with primary immunodeficiency (19.4%; p = 0.41) but significantly greater than admissions with underlying nonmalignant non-primary immunodeficiency hematologic disease (15.4%; p = 0.020), metabolic disorder (8.1%; p < 0.001), or solid malignancy (5.7%; p < 0.001). Infection was documented in 45.7% of admissions with 22.2% mortality; viral and fungal mortality were 28.5% and 33.7%, respectively. Invasive positive pressure ventilation and renal replacement therapy were used in only 34.6% and 11.9% of admissions, with mortality of 42.5% and 51.9%, respectively.

CONCLUSIONS

PICU mortality for pediatric hematopoietic stem cell transplantation patients may be as low as 16.2% but higher for those receiving intubation (42.5%) or replacement therapy (51.9%). Hematologic malignancy and primary immunodeficiency had greater risk for mortality than other transplant indications. Greater understanding of other risk factors affecting mortality and the need for critical care support is needed.

Predicting mortality in sick African children: the FEAST Paediatric Emergency Triage (PET) Score

BACKGROUND

Mortality in paediatric emergency care units in Africa often occurs within the first 24 h of admission and remains high. Alongside effective triage systems, a practical clinical bedside risk score to identify those at greatest risk could contribute to reducing mortality.

METHODS

Data collected during the Fluid As Expansive Supportive Therapy (FEAST) trial, a multi-centre trial involving 3,170 severely ill African children, were analysed to identify clinical and laboratory prognostic factors for mortality. Multivariable Cox regression was used to build a model in this derivation dataset based on clinical parameters that could be quickly and easily assessed at the bedside. A score developed from the model coefficients was externally validated in two admissions datasets from Kilifi District Hospital, Kenya, and compared to published risk scores using Area Under the Receiver Operating Curve (AUROC) and Hosmer-Lemeshow tests. The Net Reclassification Index (NRI) was used to identify additional laboratory prognostic factors.

RESULTS

A risk score using 8 clinical variables (temperature, heart rate, capillary refill time, conscious level, severe pallor, respiratory distress, lung crepitations, and weak pulse volume) was developed. The score ranged from 0-10 and had an AUROC of 0.82 (95 % CI, 0.77-0.87) in the FEAST trial derivation set. In the independent validation datasets, the score had an AUROC of 0.77 (95 % CI, 0.72-0.82) amongst admissions to a paediatric high dependency ward and 0.86 (95 % CI, 0.82-0.89) amongst general paediatric admissions. This discriminative ability was similar to, or better than other risk scores in the validation datasets. NRI identified lactate, blood urea nitrogen, and pH to be important prognostic laboratory variables that could add information to the clinical score.

CONCLUSIONS

Eight clinical prognostic factors that could be rapidly assessed by healthcare staff for triage were combined to create the FEAST Paediatric Emergency Triage (PET) score and externally validated. The score discriminated those at highest risk of fatal outcome at the point of hospital admission and compared well to other published risk scores. Further laboratory tests were also identified as prognostic factors which could be added if resources were available or as indices of severity for comparison between centres in future research studies.

Pediatric Early Warning Score and unplanned readmission to the pediatric intensive care unit

BACKGROUND AND OBJECTIVES

Early unplanned Pediatric Intensive Care Unit (PICU) readmission is associated with greater length of stay and mortality. No tools exist to identify children at risk for PICU readmission. The Pediatric Early Warning Score (PEWS) currently identify children at risk for deterioration on the ward. Our primary objective was to evaluate the ability of PEWS to identify children at risk for unplanned PICU readmission.

METHODS

A single-center case-control study of 189 children (38 cases and 151 age-matched controls) 18years or younger transferred from the PICU to the pediatric ward from January 1, 2010-March 30, 2013, at an urban tertiary care children's hospital was conducted.

RESULTS

Thirty-eight cases had unplanned PICU readmission within 48hours of transfer to pediatric ward, whereas 151 controls were not readmitted. The PEWS assigned prior to PICU discharge and first PEWS assigned on the ward were collected for cases and controls. Each 1-point increase in the PEWS score significantly increased risk of PICU readmission (odds ratios [95% confidence intervals], 1.6 [1.12-2.27; P = .009] and 1.89 [1.33-2.69; P < .001], respectively). Discrimination ability of PEWS for PICU readmission improved when chronic diagnoses were included.

CONCLUSIONS

Higher PEWS scores were associated with increased risk of unplanned PICU readmission. However, cutoff scores are not sensitive or specific enough to be clinically useful. Adding chronic disease variables may improve the clinical utility of cutoff PEWS scores.

Evaluating processes of care and outcomes of children in hospital (EPOCH): study protocol for a randomized controlled trial

Background

The prevention of near and actual cardiopulmonary arrest in hospitalized children is a patient safety imperative. Prevention is contingent upon the timely identification, referral and treatment of children who are deteriorating clinically. We designed and validated a documentation-based system of care to permit identification and referral as well as facilitate provision of timely treatment. We called it the Bedside Paediatric Early Warning System (BedsidePEWS). Here we describe the rationale for the design, intervention and outcomes of the study entitled Evaluating Processes and Outcomes of Children in Hospital (EPOCH).

Methods/Design

EPOCH is a cluster-randomized trial of the BedsidePEWS. The unit of randomization is the participating hospital. Eligible hospitals have a Pediatric Intensive Care Unit (PICU), are anticipated to have organizational stability throughout the study, are not using a severity of illness score in hospital wards and are willing to be randomized. Patients are >37 weeks gestational age and <18 years and are hospitalized in inpatient ward areas during all or part of their hospital admission.

Randomization is to either BedsidePEWS or control (no severity of illness score) in a 1:1 ratio within two strata (<200, ≥200 hospital beds). All-cause hospital mortality is the selected primary outcome. It is objective, independent of do-not-resuscitate status and can be reliably measured. The secondary outcomes include (1) clinical outcomes: clinical deterioration, severity of illness at and during ICU admission, and potentially preventable cardiac arrest; (2) processes of care outcomes: immediate calls for assistance, hospital and ICU readmission, and perceptions of healthcare professionals; and (3) resource utilization: ICU days and use of ICU therapies.

Discussion

Following funding by the Canadian Institutes of Health Research and local ethical approvals, site enrollment started in 2010 and was closed in February 2014. Patient enrollment is anticipated to be complete in July 2015. The results of EPOCH will strengthen the scientific basis for local, regional, provincial and national decision-making and for the recommendations of national and international bodies. If negative, the costs of hospital-wide implementation can be avoided. If positive, EPOCH will have provided a scientific justification for the major system-level changes required for implementation.

Trial registration: NCT01260831 ClinicalTrials.gov date: 14 December 2010.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-015-0712-3) contains supplementary material, which is available to authorized users.

Time to clinical stability among children hospitalized with pneumonia

We evaluated the performance of time to clinical stability (TCS), a longitudinal outcome measure using 4 physiologic parameters (temperature, heart rate, respiratory rate, and use of supplemental oxygen), among children enrolled in a prospective study of pneumonia hospitalizations. We calculated the time from admission to normalization for each of the 4 parameters individually along with various combinations of these parameters (≥2 parameters). We assessed for agreement between the combined TCS measures and both hospital length of stay and an ordinal severity scale (nonsevere, severe, and very severe). Overall, 323 (96.7%) of 334 included children had ≥1 parameter abnormal on admission; 70 (21%) children had ≥1 parameter abnormal at discharge. For the 4 combined measures, median TCS decreased with increasing age. Increasing TCS was associated with both longer length of stay and increasing disease severity. The simplest combined measure incorporating only respiratory rate and need for supplemental oxygen performed similarly to more complex measures including additional parameters. Our study demonstrates that longitudinal TCS measures may be useful in children with pneumonia, both in clinical settings to assess recovery and readiness for discharge, and as an outcome measure in research and quality assessments. Additional study is needed to further validate our findings.

Heart rates in hospitalized children by age and body temperature

BACKGROUND AND OBJECTIVES

Heart rate (HR) is frequently used by clinicians in the hospital to assess a patient's severity of illness and make treatment decisions. We sought to develop percentiles that characterize the relationship of expected HR by age and body temperature in hospitalized children and to compare these percentiles with published references in both primary care and emergency department (ED) settings.

METHODS

Vital sign data were extracted from electronic health records of inpatients <18 years of age at 2 large freestanding children's hospitals from July 2011 to June 2012. We selected up to 10 HR-temperature measurement pairs from each admission. Measurements from 60% of patients were used to derive the percentile curves, with the remainder used for validation. We compared our upper percentiles with published references in primary care and ED settings.

RESULTS

We used 60,863 observations to derive the percentiles. Overall, an increase in body temperature of 1°C was associated with an increase of ∼ 10 beats per minute in HR, although there were variations across age and temperature ranges. For infants and young children, our upper percentiles were lower than in primary care and ED settings. For school-age children, our upper percentiles were higher.

CONCLUSIONS

We characterized expected HR by age and body temperature in hospitalized children. These percentiles differed from references in primary care and ED settings. Additional research is needed to evaluate the performance of these percentiles for the identification of children who would benefit from further evaluation or intervention for tachycardia.

Prospective validation of pediatric disease severity scores to predict mortality in Ugandan children presenting with malaria and non-malaria febrile illness

Introduction

The development of simple clinical tools to identify children at risk of death would enable rapid and rational implementation of lifesaving measures to reduce childhood mortality globally.

Methods

We evaluated the ability of three clinical scoring systems to predict in-hospital mortality in a prospective observational study of Ugandan children with fever. We computed the Lambaréné Organ Dysfunction Score (LODS), Signs of Inflammation in Children that Kill (SICK), and the Pediatric Early Death Index for Africa (PEDIA). Model discrimination was evaluated by comparing areas under receiver operating characteristic curves (AUCs) and calibration was assessed using the Hosmer-Lemeshow goodness-of-fit test. Sub-analyses were performed in malaria versus non-malaria febrile illness (NMFI), and in early (≤48 hours) versus late (>48 hours) deaths.

Results

In total, 2089 children with known outcomes were included in the study (99 deaths, 4.7% mortality). All three scoring systems yielded good discrimination (AUCs, 95% confidence interval (CI): LODS, 0.90, 0.88 to 0.91; SICK, 0.85, 0.83 to 0.86; PEDIA, 0.90, 0.88 to 0.91). Using the Youden index to identify the best cut-offs, LODS had the highest positive likelihood ratio (+LR, 95% CI: LODS, 6.5, 5.6 to 7.6; SICK, 4.4, 3.9 to 5.0; PEDIA, 4.4, 3.9 to 5.0), whereas PEDIA had the lowest negative likelihood ratio (−LR, 95% CI: LODS, 0.21, 0.1 to 0.3; SICK, 0.22, 0.1 to 0.3; PEDIA, 0.16, 0.1 to 0.3), LODS and PEDIA were well calibrated (P = 0.79 and P = 0.21 respectively), and had higher AUCs than SICK in discriminating between survivors and non-survivors in malaria (AUCs, 95% CI: LODS, 0.92, 0.90 to 0.93; SICK, 0.86, 0.84 to 0.87; PEDIA, 0.92, 0.90 to 0.93), but comparable AUCs in NMFI (AUCs, 95% CI: LODS, 0.86, 0.83 to 0.89; SICK, 0.82, 0.79 to 0.86; PEDIA, 0.87, 0.83 to 0.893). The majority of deaths in the study occurred early (n = 85, 85.9%) where LODS and PEDIA had good discrimination.

Conclusions

All three scoring systems predicted outcome, but LODS holds the most promise as a clinical prognostic score based on its simplicity to compute, requirement for no equipment, and good discrimination.

Electronic supplementary material

The online version of this article (doi:10.1186/s13054-015-0773-4) contains supplementary material, which is available to authorized users.

PCR testing for paediatric acute respiratory tract infections

Acute respiratory tract infection (ARI) is a frequently occurring disease in children. It is a clinical diagnosis for which no internationally accepted diagnostic test is available. The majority of ARI is viral in origin, though diagnostic tests for viruses were rarely performed in the past. In the past 2 decades, new molecular techniques have been introduced in many hospitals. They are capable of generating a high yield of viral and bacterial diagnoses, but their impact upon clinical practices is still questionable. In this paper, we discuss the difficulties of diagnosing ARI in children, the indications for conventional and new diagnostics and their implications.

Evaluating the Pediatric Early Warning Score (PEWS) system for admitted patients in the pediatric emergency department

OBJECTIVES

The Pediatric Early Warning Score (PEWS) systems were developed to provide a reproducible assessment of a child's clinical status while hospitalized. Most studies investigating the PEWS evaluate its usefulness in the inpatient setting. Limited studies evaluate the effectiveness and integration of PEWS in the pediatric emergency department (ED). The goal of this study was to explore the test characteristics of an ED-assigned PEWS score for intensive care unit (ICU) admission or clinical deterioration in admitted patients.

METHODS

This was a prospective 12-month observational study of patients, aged 0 to 21 years, admitted from the ED of an urban, tertiary care children's hospital. ED nurses were instructed in PEWS assignment and electronic medical record (EMR) documentation. Interrater reliability between nurses was evaluated. PEWS scores were measured at initial assessment (P0) and time of admission (P1). Patients were stratified into outcome groups: those admitted to the ICU either from the ED or as transfers from the floor and those admitted to the floor only. Clinical deterioration was defined as transfer to the ICU within 6 hours or within 6 to 24 hours of admission. PEWS scores and receiver operating characteristic (ROC) curves were compared for patients admitted to the floor, ICU, and with clinical deterioration.

RESULTS

The authors evaluated 12,306 consecutively admitted patients, with 99% having a PEWS documented in the EMR. Interrater reliability was excellent (intraclass coefficient = 0.91). A total of 1,300 (10.6%) patients were admitted to the ICU and 11,066 (89.4%) were admitted to the floor. PEWS scores were higher for patients in the ICU group (P0 = 2.8, SD ± 2.4; P1 = 3.2, SD ± 2.4; p < 0.0001) versus floor patients (P0 = 0.7, SD ± 1.2; P1 = 0.5, SD ± 0.9; p < 0.0001). To predict the need for ICU admission, the optimal cutoff points on the ROC are P0 = 1 and P1 = 2, with areas under the ROC curve (AUCs) of 0.79 and 0.86, respectively. The likelihood ratios (LRs) for these optimal cutoff points were as follows: P0 +LR = 2.5 (95% confidence interval [CI] = 2.4 to 2.6, p < 0.05), -LR = 0.32 (95% CI = 0.28 to 0.36, p < 0.05); and P1 +LR = 6.2 (95% CI = 5.8 to 6.6, p < 0.05), -LR = 0.32 (95% CI = 0.29 to 0.35, p < 0.05). For every unit increase in P0 and P1 , the odds of admission to the ICU were 1.9 times greater (95% CI = 1.8 to 1.9, p < 0.0001) and 2.9 times greater (95% CI = 2.7 to 3.1, p < 0.0001) than to the floor. There were 89 patients in the clinical deterioration group, with 36 (0.3%) patients transferred to the ICU within 6 hours of admission and 53 (0.4%) patients transferred within 6 to 24 hours. In this group, an elevated P0 and P1 were statistically associated with an increased risk of transfer with optimal cutoff points similar to above; however, there were poorer AUCs and test characteristics.

CONCLUSIONS

A PEWS system was implemented in this pediatric ED with excellent data capture and nurse interrater reliability. The study found that an elevated PEWS is associated with need for ICU admission directly from the ED and as a transfer, but lacks the necessary test characteristics to be used independently in the ED environment.

Implementation of the Bedside Paediatric Early Warning System (BedsidePEWS) for nurse identification of deteriorating patients

PURPOSE

The purpose was to explore nurses' and physicians' recognition of signs of deterioration and management of symptoms.

DESIGN AND METHODS

This descriptive, cross-sectional study used an electronic survey with 35 nurses and 17 physicians.

RESULTS

Nurses using the Bedside Paediatric Early Warning System (BedsidePEWS) were significantly more likely to recognize risk for deterioration and respond with appropriate interventions. Physicians incorporating BedsidePEWS were more likely to choose reliable indicators of deterioration and reported significantly more effective communication from nurses to identify deterioration.

PRACTICE IMPLICATIONS

BedsidePEWS may improve nurses' and physicians' abilities to recognize early signs of patient deterioration, communicate findings to providers, and plan interventions.

Developing and evaluating a machine learning based algorithm to predict the need of pediatric intensive care unit transfer for newly hospitalized children

BACKGROUND

Early warning scores (EWS) are designed to identify early clinical deterioration by combining physiologic and/or laboratory measures to generate a quantified score. Current EWS leverage only a small fraction of Electronic Health Record (EHR) content. The planned widespread implementation of EHRs brings the promise of abundant data resources for prediction purposes. The three specific aims of our research are: (1) to develop an EHR-based automated algorithm to predict the need for Pediatric Intensive Care Unit (PICU) transfer in the first 24h of admission; (2) to evaluate the performance of the new algorithm on a held-out test data set; and (3) to compare the effectiveness of the new algorithm's with those of two published Pediatric Early Warning Scores (PEWS).

METHODS

The cases were comprised of 526 encounters with 24-h Pediatric Intensive Care Unit (PICU) transfer. In addition to the cases, we randomly selected 6772 control encounters from 62516 inpatient admissions that were never transferred to the PICU. We used 29 variables in a logistic regression and compared our algorithm against two published PEWS on a held-out test data set.

RESULTS

The logistic regression algorithm achieved 0.849 (95% CI 0.753-0.945) sensitivity, 0.859 (95% CI 0.850-0.868) specificity and 0.912 (95% CI 0.905-0.919) area under the curve (AUC) in the test set. Our algorithm's AUC was significantly higher, by 11.8 and 22.6% in the test set, than two published PEWS.

CONCLUSION

The novel algorithm achieved higher sensitivity, specificity, and AUC than the two PEWS reported in the literature.

Characteristics and outcomes of pediatric rapid response teams before and after mandatory triggering by an elevated Pediatric Early Warning System (PEWS) score

BACKGROUND

The Pediatric Early Warning System (PEWS) was created to identify unstable patients before their deterioration. Rapid response teams (RRTs) were developed to assist with management of such patients. In 2009, our institution mandated the activation of RRTs if a PEWS score was elevated (ie, ≥5).

OBJECTIVES

The goal of this study was to examine changes in characteristics of RRT calls before and after the implementation of a mandatory hospital policy requiring RRT activation due to an elevated PEWS score.

METHODS

This study was a retrospective database review, with RRT data from June 2007 to December 2010 examined. A total of 44 RRTs were recorded before mandatory triggering and 69 RRTs afterward in the study period (P = .32).

RESULTS

Compared with the premandatory group, the mandatory triggering group found that tachycardia was a more frequent trigger for RRTs, with an increase of 26.1% (P = .004). RRTs triggered by a change in mental status/agitation decreased by 22.9% (P = .009). An increase of 15.1% of RRTs required no interventions with mandatory triggering. Nighttime RRTs increased by17.5% (P = .07). There was a trend toward decreased PICU transfers in the mandatory triggering group, with no significant change in code blue calls.

CONCLUSIONS

A hospital policy of mandating RRT activation based on PEWS scores increased nighttime calls and altered the primary reasons for RRT activation in our center, with no evidence of improvements in patient care. These findings should be interpreted with caution given the relatively rare outcomes the policy is intended to prevent; however, our findings highlight the difficulties inherent in evaluating methods to improve pediatric patient safety.

Clinical profile of children requiring early unplanned admission to the PICU

OBJECTIVE

The goal of this study was to describe the frequency, characteristics, and outcomes of children who require early unplanned admission to the PICU within 24 hours of hospitalization from the emergency department.

METHODS

This study was a retrospective audit of 24 months of prospectively collected medical emergency team records at a tertiary pediatric hospital in Canada. Our review identified 39 hospitalized children who had an activation that resulted in unplanned admission to the PICU within 24 hours of admission from the ED.

RESULTS

Forty-six percent of the study subjects were infants aged < 1 year, and 64% were male. Respiratory complaints were the most common reason for hospitalization (59%). Preexisting medical conditions (51%), abnormal respiratory rates (46%), abnormal heart rates (33%), abnormal blood gas values (49%), high supplemental oxygen requirement (23%), and treatment with nebulized medications (46%), intravenous fluids (33%), and antibiotics (33%) were common. The median time to medical emergency team activation was 9.4 hours (interquartile range: 4.4-14.5). Nearly one-half (49%) of the patients required a significant intervention after admission to the PICU, with a mean length of stay of 3.4 days and a mortality rate of 50/%.

CONCLUSIONS

Male subjects, infants aged < 1 year, and children with respiratory complaints accounted for a large proportion of children requiring early unplanned admission to the PICU within 24 hours of hospitalization from the ED. Further studies are required to determine which factors are associated with deterioration after hospitalization.

Pediatric early warning score at time of emergency department disposition is associated with level of care

OBJECTIVE

The objective of this study was to determine the association between the Pediatric Early Warning Score (PEWS) at time of emergency department (ED) disposition and level of care.

METHODS

We conducted a prospective observational study with a convenience sample of patients aged 0 to 21 years in the ED of an urban, tertiary care children's hospital between November 2010 and July 2011. Pediatric Early Warning Score data were obtained at time of ED disposition, and the disposition decision was collected from the electronic medical record. Multinomial logistic regression was used to determine the association between PEWS and disposition.

RESULTS

The sample of 383 patients included 239 (62%) who were discharged, 126 (33%) admitted to acute care, and 18 (5%) admitted to intensive care. Assigned scores ranged from 0 to 9. Adjusting for triage level, a 1-point increase in PEWS increased the odds of acute care admission 48% relative to the odds of discharge (odds ratio, 1.48; 95% confidence interval, 1.25-1.76) and increased the odds of intensive care admission 41% relative to the odds of acute care admission (odds ratio, 1.41; 95% confidence interval, 1.13-1.76). Pediatric Early Warning Score of 1 or more had maximum discriminant ability for admission, and PEWS of 3 or greater had maximum discriminant ability for intensive care. Area under the receiver operator characteristic curve was 0.68 to detect need for admission for the entire sample and 0.80 among the 97 patients with respiratory complaints.

CONCLUSIONS

Pediatric Early Warning Score is associated with the level of care at ED disposition but does not provide adequate sensitivity and specificity to be used in isolation. Performance characteristics are better for patients with respiratory complaints.

Automated analysis of electronic medical record data reflects the pathophysiology of operative complications

PURPOSE

We hypothesized that a novel algorithm that uses data from the electronic medical record (EMR) from multiple clinical and biometric sources could provide early warning of organ dysfunction in patients with high risk for postoperative complications and sepsis. Operative patients undergoing colorectal procedures were evaluated.

METHODS

The Rothman Index (RI) is a predictive model based on heuristic equations derived from 26 variables related to inpatient care. The RI integrates clinical nursing observations, bedside biometrics, and laboratory data into a continuously updated, numeric physiologic assessment, ranging from 100 (unimpaired) to -91. The RI can be displayed within the EMR as a graphic trend, with a decreasing trend reflecting physiologic dysfunction. Patients undergoing colorectal procedures between June and October 2011 were evaluated to determine correlation of initial RI, average inpatient RI, and lowest RI to incidence of complications and/or postoperative sepsis. Patients were stratified by color-coded RI risk group (100-65, blue; 64-40, yellow; <40 red). One-way or repeated-measures analysis of variance was used to compare groups by age, number of complications, and presence of sepsis defined by discharge International Classification of Diseases, 9(th) Revision, codes. Mean direct cost of care and duration of stay also was calculated for each group.

RESULTS

The overall incidence of perioperative complications in the 124 patient cohort was 51% (n = 64 patients). The 261 complications sustained by this group represented 82 distinct diagnoses. The 10 patients with sepsis (8%) experienced a 40% mortality. Analysis of initial RI for the population stratified by number of complications and/or sepsis demonstrated a risk-related difference. With progressive onset of complications, the RI decreased, suggesting worsening physiologic dysfunction and linear increase in direct cost of care.

CONCLUSION

These findings demonstrate that EMR data can be automatically compiled into an objective metric that reflects patient risk and changing physiologic state. The automated process of continuous update reflects a physiologic trajectory associated with evolving organ system dysfunction indicative of postoperative complications. Early intervention based on these trends may guide preoperative counseling, enhance pre-emptive management of adverse occurrences, and improve cost-efficiency of care.

The Texas Children's Hospital Pediatric Advanced Warning Score as a predictor of clinical deterioration in hospitalized infants and children: a modification of the PEWS tool

The purpose of this study was to examine the psychometric properties of the Texas Children's Hospital Pediatric Advanced Warning Score (PAWS) instrument as an indicator of clinical deterioration in infants and children.

DESIGN AND METHODS

A retrospective chart review of 150 infants and children was performed.

RESULTS

The overall Cronbach's alpha score was 0.75. The estimate of interrater reliability was 0.740.

IMPLICATIONS

The Texas Children's Hospital Pediatric Advanced Warning Score instrument was found to be reliable and valid.

Comparison of three acute care pediatric early warning scoring tools

Pediatric Early Warning (PEW) scoring tools effectively identify hospitalized children at risk for clinical deterioration. The study compared the predictability of three previously validated PEW scoring tools. A retrospective case-control design was used that identified the PEW System Score (H. Duncan, J. Hutchison, & C. Parshuram, 2006) as a stronger predictor of cardiopulmonary arrest (CPA) than either the PEW Tool (C. Haines, M. Perrott, & P. Weir, 2006) or the Bedside PEW System Score (C. Parshuram, J. Hutchison, & K. Middaugh, 2009). The PEW System Score (H. Duncan, J. Hutchison, & C. Parshuram, 2006) demonstrated a greater sensitivity (86.6%) and specificity (72.9%) at a score of five. The PEW System Score (H. Duncan, J. Hutchison, & C. Parshuram, 2006) could benefit healthcare providers in potentially averting CPA.

Development of a severity of illness scoring system (inpatient triage, assessment and treatment) for resource-constrained hospitals in developing countries

OBJECTIVE

To develop a new paediatric illness severity score, called inpatient triage, assessment and treatment (ITAT), for resource-limited settings to identify hospitalised patients at highest risk of death and facilitate urgent clinical re-evaluation.

METHODS

We performed a nested case-control study at a Malawian referral hospital. The ITAT score was derived from four equally weighted variables, yielding a cumulative score between 0 and 8. Variables included oxygen saturation, temperature, and age-adjusted heart and respiratory rates. We compared the ITAT score between cases (deaths) and controls (discharges) in predicting death within 2 days. Our analysis includes predictive statistics, bivariable and multivariable logistic regression, and calculation of data-driven scores.

RESULTS

A total of 54 cases and 161 controls were included in the analysis. The area under the receiver operating characteristic curve was 0.76. At an ITAT cut-off of 4, the sensitivity, specificity and likelihood ratio were 0.44, 0.86 and 1.70, respectively. A cumulative ITAT score of 4 or higher was associated with increased odds of death (OR 4.80; 95% CI 2.39-9.64). A score of 2 for all individual vital signs was a statistically significant independent predictor of death.

CONCLUSIONS

We developed an inpatient triage tool (ITAT) appropriate for resource-constrained hospitals that identifies high-risk children after hospital admission. Further research is needed to study how best to operationalise ITAT in developing countries.

Development and validation of a continuous measure of patient condition using the Electronic Medical Record

Patient condition is a key element in communication between clinicians. However, there is no generally accepted definition of patient condition that is independent of diagnosis and that spans acuity levels. We report the development and validation of a continuous measure of general patient condition that is independent of diagnosis, and that can be used for medical-surgical as well as critical care patients. A survey of Electronic Medical Record data identified common, frequently collected non-static candidate variables as the basis for a general, continuously updated patient condition score. We used a new methodology to estimate in-hospital risk associated with each of these variables. A risk function for each candidate input was computed by comparing the final pre-discharge measurements with 1-year post-discharge mortality. Step-wise logistic regression of the variables against 1-year mortality was used to determine the importance of each variable. The final set of selected variables consisted of 26 clinical measurements from four categories: nursing assessments, vital signs, laboratory results and cardiac rhythms. We then constructed a heuristic model quantifying patient condition (overall risk) by summing the single-variable risks. The model's validity was assessed against outcomes from 170,000 medical-surgical and critical care patients, using data from three US hospitals. Outcome validation across hospitals yields an area under the receiver operating characteristic curve(AUC) of ≥0.92 when separating hospice/deceased from all other discharge categories, an AUC of ≥0.93 when predicting 24-h mortality and an AUC of 0.62 when predicting 30-day readmissions. Correspondence with outcomes reflective of patient condition across the acuity spectrum indicates utility in both medical-surgical units and critical care units. The model output, which we call the Rothman Index, may provide clinicians with a longitudinal view of patient condition to help address known challenges in caregiver communication, continuity of care, and earlier detection of acuity trends.

Deployment and validation of a smart system for screening of language disorders in primary care

Neuro-evolutive development from birth until the age of six years is a decisive factor in a child's quality of life. Early detection of development disorders in early childhood can facilitate necessary diagnosis and/or treatment. Primary-care pediatricians play a key role in its detection as they can undertake the preventive and therapeutic actions requested to promote a child's optimal development. However, the lack of time and little specific knowledge at primary-care avoid to applying continuous early-detection anomalies procedures. This research paper focuses on the deployment and evaluation of a smart system that enhances the screening of language disorders in primary care. Pediatricians get support to proceed with early referral of language disorders. The proposed model provides them with a decision-support tool for referral actions to trigger essential diagnostic and/or therapeutic actions for a comprehensive individual development. The research was conducted by starting from a sample of 60 cases of children with language disorders. Validation was carried out through two complementary steps: first, by including a team of seven experts from the fields of neonatology, pediatrics, neurology and language therapy, and, second, through the evaluation of 21 more previously diagnosed cases. The results obtained show that therapist positively accepted the system proposal in 18 cases (86%) and suggested system redesign for single referral to a speech therapist in three remaining cases.

Placing clinical variables on a common linear scale of empirically based risk as a step towards construction of a general patient acuity score from the electronic health record: a modelling study

OBJECTIVE

To explore the hypothesis that placing clinical variables of differing metrics on a common linear scale of all-cause postdischarge mortality provides risk functions that are directly correlated with in-hospital mortality risk.

DESIGN

Modelling study.

SETTING

An 805-bed community hospital in the southeastern USA.

PARTICIPANTS

42302 inpatients admitted for any reason, excluding obstetrics, paediatric and psychiatric patients.

OUTCOME MEASURES

All-cause in-hospital and postdischarge mortalities, and associated correlations.

RESULTS

Pearson correlation coefficients comparing in-hospital risks with postdischarge risks for creatinine, heart rate and a set of 12 nursing assessments are 0.920, 0.922 and 0.892, respectively. Correlation between postdischarge risk heart rate and the Modified Early Warning System (MEWS) component for heart rate is 0.855. The minimal excess risk values for creatinine and heart rate roughly correspond to the normal reference ranges. We also provide the risks for values outside that range, independent of expert opinion or a regression model. By summing risk functions, a first-approximation patient risk score is created, which correctly ranks 6 discharge categories by average mortality with p<0.001 for differences in category means, and Tukey's Honestly Significant Difference Test confirmed that the means were all different at the 95% confidence level.

CONCLUSIONS

Quantitative or categorical clinical variables can be transformed into risk functions that correlate well with in-hospital risk. This methodology provides an empirical way to assess inpatient risk from data available in the Electronic Health Record. With just the variables in this paper, we achieve a risk score that correlates with discharge disposition. This is the first step towards creation of a universal measure of patient condition that reflects a generally applicable set of health-related risks. More importantly, we believe that our approach opens the door to a way of exploring and resolving many issues in patient assessment.

Beyond statistical prediction: qualitative evaluation of the mechanisms by which pediatric early warning scores impact patient safety

BACKGROUND

Early warning scores (EWSs) assign points to clinical observations and generate scores to help clinicians identify deteriorating patients. Despite marginal predictive accuracy in retrospective datasets and a paucity of studies prospectively evaluating their clinical effectiveness, pediatric EWSs are commonly used.

OBJECTIVE

To identify mechanisms beyond their statistical ability to predict deterioration by which physicians and nurses use EWSs to support their decision making.

DESIGN

Qualitative study.

SETTING

A children's hospital with a rapid response system.

PARTICIPANTS

Physicians and nurses who recently cared for patients with false-positive and false-negative EWSs (score failures).

INTERVENTION

Semistructured interviews.

MEASUREMENTS

Themes identified through grounded theory analysis.

RESULTS

Four themes emerged among the 57 subjects interviewed: (1) The EWS facilitates safety by alerting physicians and nurses to concerning changes and prompting them to think critically about deterioration. (2) The EWS provides less-experienced nurses with vital sign reference ranges. (3) The EWS serves as evidence that empowers nurses to overcome barriers to escalating care. (4) In stable patients, those with baseline abnormal physiology, and those experiencing neurologic deterioration, the EWS may not be helpful.

CONCLUSIONS

Although pediatric EWSs have marginal performance when applied to datasets, clinicians who recently experienced score failures still considered them valuable to identify deterioration and transcend hierarchical barriers. Combining an EWS with a clinician's judgment may result in a system better equipped to respond to deterioration than retrospective data analyses alone would suggest. Future research should seek to evaluate the clinical effectiveness of EWSs in real-world settings.

Task shifting an inpatient triage, assessment and treatment programme improves the quality of care for hospitalised Malawian children

OBJECTIVE

We aimed to improve paediatric inpatient surveillance at a busy referral hospital in Malawi with two new programmes: (i) the provision of vital sign equipment and implementation of an inpatient triage programme (ITAT) that includes a simplified paediatric severity-of-illness score, and (ii) task shifting ITAT to a new cadre of healthcare workers called 'vital sign assistants' (VSAs).

METHODS

This study, conducted on the paediatric inpatient ward of a large referral hospital in Malawi, was divided into three phases, each lasting 4 weeks. In Phase A, we collected baseline data. In Phase B, we provided three new automated vital sign poles and implemented ITAT with current hospital staff. In Phase C, VSAs were introduced and performed ITAT. Our primary outcome measures were the number of vital sign assessments performed and clinician notifications to reassess patients with high ITAT scores.

RESULTS

We enrolled 3994 patients who received 5155 vital sign assessments. Assessment frequency was equal between Phases A (0.67 assessments/patient) and B (0.61 assessments/patient), but increased 3.6-fold in Phase C (2.44 assessments/patient, P < 0.001). Clinician notifications increased from Phases A (84) and B (113) to Phase C (161, P = 0.002). Inpatient mortality fell from Phase A (9.3%) to Phases B (5.7) and C (6.9%).

CONCLUSION

ITAT with VSAs improved vital sign assessments and nearly doubled clinician notifications of patients needing further assessment due to high ITAT scores, while equipment alone made no difference. Task shifting ITAT to VSAs may improve outcomes in paediatric hospitals in the developing world.

Development of heart and respiratory rate percentile curves for hospitalized children

OBJECTIVE

To develop and validate heart and respiratory rate percentile curves for hospitalized children and compare their vital sign distributions to textbook reference ranges and pediatric early warning score (EWS) parameters.

METHODS

For this cross-sectional study, we used 6 months of nurse-documented heart and respiratory rates from the electronic records of 14,014 children on general medical and surgical wards at 2 tertiary-care children's hospitals. We developed percentile curves using generalized additive models for location, scale, and shape with 67% of the patients and validated the curves with the remaining 33%. We then determined the proportion of observations that deviated from textbook reference ranges and EWS parameters.

RESULTS

We used 116,383 heart rate and 116,383 respiratory rate values to develop and validate the percentile curves. Up to 54% of heart rate observations and up to 40% of respiratory rate observations in our sample were outside textbook reference ranges. Up to 38% of heart rate observations and up to 30% of respiratory rate observations in our sample would have resulted in increased EWSs.

CONCLUSIONS

A high proportion of vital signs among hospitalized children would be considered out of range according to existing reference ranges and pediatric EWSs. The percentiles we derived may serve as useful references for clinicians and could be used to inform the development of evidence-based vital sign parameters for physiologic monitor alarms, inpatient electronic health record vital sign alerts, medical emergency team calling criteria, and EWSs.

Early identification of children at risk for critical care: standardizing communication for inter-emergency department transfers

BACKGROUND

Interfacility transfers occur frequently and often involve critically ill patients. Clear communication at the time of patient referral is essential for patient safety.

OBJECTIVES

The objective of this work was to study whether a standardized inter-emergency department (ED) transfer communication template for pediatric patients with respiratory complaints identifies patients that require intensive care unit (ICU) admission.

METHODS

We created a template to structure the communication between referring and receiving providers involved in inter-ED transfers of children with respiratory complaints. The template was designed for use by nonphysicians to prompt specific questions that would trigger notification of the ED attending based on signs of critical illness. The template was retrospectively applied to determine whether it would have properly triggered attending physician notification of a child ultimately requiring ICU admission.

RESULTS

Of 285 transferred children, 61 (21%) were admitted to an ICU from the receiving ED. The sensitivity of the communication template in predicting the need for ICU admission was 84% (95% confidence interval [CI], 72%-92%), negative predictive value of 95% (95% CI, 90%-97%), specificity of 77% (95% CI, 71%-82%), positive predictive value of 50% (95% CI, 40%-60%). Of the 10 patients admitted to an ICU who were not identified by the tool, none were critically ill upon arrival. Of the individual communication elements, the sensitivity and negative predictive value ranged from 3% to 38% and from 79% to 86%, respectively.

CONCLUSIONS

A standardized communication template for inter-ED transfers can identify children with respiratory complaints who require ICU admission. Next steps include real-time application to judge screening performance compared with current nonstandardized intake protocols.

The Cardiac Children's Hospital Early Warning Score (C-CHEWS)

Inpatient pediatric cardiovascular patients have higher rates of cardiopulmonary arrests than other hospitalized children. Pediatric early warning scoring tools have helped to provide early identification and treatment to hospitalized children experiencing deterioration thus preventing arrests from occurring. However, the tools have rarely been used and have not been validated in the pediatric cardiac population. This paper describes the modification of a pediatric early warning scoring system for cardiovascular patients, the implementation of the tool, and its companion escalation of care algorithm on an inpatient pediatric cardiovascular unit.

Retrospective evaluation of a new neonatal trigger score

OBJECTIVES

To design and validate an objective clinical scoring system to identify unwell neonates, by using routinely collected bedside observations.

METHODS

A Neonatal Trigger Score (NTS) was designed by using local expert consensus and incorporated into a new observation chart. All neonates >35 weeks' gestation admitted to the NICU over an 18-month period, and an age-matched "well" cohort, were retrospectively scored by using the newly constructed NTS and all established pediatric early warning system (PEWS) scores.

RESULTS

Scores were calculated for 485 neonates. The NTS score area under the receiver operating characteristic curve was 0.924 with a score of 2 or more predicting need for admission to the NICU with 77% sensitivity and 97% specificity. Neonates scoring ≥2 had increased odds of needing intensive care (odds ratio [OR] 48.7, 95% confidence interval [CI] 27.5-86.3), intravenous fluids (OR 48.1, 95% CI 23.9-96.9), and continuous positive airway pressure (OR 29.5, 95% CI 6.9-125.8). The NTS was more sensitive than currently established PEWS scores.

CONCLUSIONS

The NTS observation chart acts as an adjunct to clinical assessment, highlighting unwell neonates. Its simplicity allows successful and safe use by nonpediatric specialists. NTS out-performed PEWS, with significantly better sensitivity, particularly in neonates who deteriorated within the first 12 hours after birth (P < .001) or in neonates with sepsis or respiratory symptoms (P < .001). Neonates with a score of 1 should be reviewed and those scoring ≥2 should be considered for NICU admission for further management.