External Validation of Six Pediatric Fever and Neutropenia Clinical Decision Rules

Evaluating the accuracy and efficacy of an electronic medical record alert to identify paediatric patients with low-risk febrile neutropenia

BACKGROUND

Point-of-care decision support, embedded into electronic medical record (EMR) workflows, has the potential to improve efficiency, reduce unwarranted variation and improve patient outcomes. A clinical-facing best practice advisory (BPA) in the Epic EMR system was developed to identify children admitted with low-risk febrile neutropenia (FN) who should be considered for treatment at home after a brief inpatient stay. We evaluated the accuracy and impact of this BPA and identify areas for improvement.

METHODS

The low-risk FN BPA was co-designed with key-stakeholders and implemented after a one-month testing phase. Mixed methodology was used to collect and analyse data. The sensitivity and positive predictive value of the BPA was calculated using FN episodes captured in a prospectively collected database. Overall effectiveness was defined as the proportion of alerts resulting in completion of a FN risk assessment flowsheet.

RESULTS

Over the 12-month period 176 FN episodes were admitted. Overall, the alert had poor sensitivity (58%) and positive predictive value (75%), failing to trigger in 62 (35%) episodes. In the episodes where the alert did trigger, the alert was frequently dismissed by clinicians (76%) and the overall effectiveness was extremely low (3%). Manual review of each FN episode without a BPA identified important design limitations and incorrect workflow assumptions.

DISCUSSION

Given the poor sensitivity and limited impact on clinician behaviour the low-risk BPA, in its current form, has not been an effective intervention at this site. While work is ongoing to enhance the accuracy of the BPA, alternative EMR workflows are likely required to improve the clinical impact.

Outcome prediction in pediatric fever in neutropenia: Development of clinical decision rules and external validation of published rules based on data from the prospective multicenter SPOG 2015 FN definition study

BACKGROUND

Fever in neutropenia (FN) remains a serious complication of childhood cancer therapy. Clinical decision rules (CDRs) are recommended to help distinguish between children at high and low risk of severe infection. The aim of this analysis was to develop new CDRs for three different outcomes and to externally validate published CDRs.

PROCEDURE

Children undergoing chemotherapy for cancer were observed in a prospective multicenter study. CDRs predicting low from high risk infection regarding three outcomes (bacteremia, serious medical complications (SMC), safety relevant events (SRE)) were developed from multivariable regression models. Their predictive performance was assessed by internal cross-validation. Published CDRs suitable for validation were identified by literature search. Parameters of predictive performance were compared to assess reproducibility.

RESULTS

In 158 patients recruited between April 2016 and August 2018, 360 FN episodes were recorded, including 56 (16%) with bacteremia, 30 (8%) with SMC and 72 (20%) with SRE. The CDRs for bacteremia and SRE used four characteristics (type of malignancy, severely reduced general condition, leucocyte count <0.3 G/L, bone marrow involvement), the CDR for SMC two characteristics (severely reduced general condition and platelet count <50 G/L). Eleven published CDRs were analyzed. Six CDRs showed reproducibility, but only one in both sensitivity and specificity.

CONCLUSIONS

This analysis developed CDRs predicting bacteremia, SMC or SRE at presentation with FN. In addition, it identified six published CDRs that show some reproducibility. Validation of CDRs is fundamental to find the best balance between sensitivity and specificity, and will help to further improve management of FN.

Optimising Antimicrobial Selection and Duration in the Treatment of Febrile Neutropenia in Children

Febrile neutropenia (FN) is a frequent complication of cancer treatment in children. Owing to the potential for overwhelming bacterial sepsis, the recognition and management of FN requires rapid implementation of evidenced-based management protocols. Treatment paradigms have progressed from hospitalisation with broad spectrum antibiotics for all patients, through to risk adapted approaches to management. Such risk adapted approaches aim to provide safe care through incorporating antimicrobial stewardship (AMS) principles such as implementation of comprehensive clinical pathways incorporating de-escalation strategies with the imperative to reduce hospital stay and antibiotic exposure where possible in order to improve patient experience, reduce costs and diminish the risk of nosocomial infection. This review summarises the principles of risk stratification in FN, the current key considerations for optimising empiric antimicrobial selection including knowledge of antimicrobial resistance patterns and emerging technologies for rapid diagnosis of specific infections and summarises existing evidence on time to treatment, investigations required and duration of treatment. To aid treating physicians we suggest the key features based on current evidence that should be part of any FN management guideline and highlight areas for future research. The focus is on treatment of bacterial infections although fungal and viral infections are also important in this patient group.

Aminoglycoside use in paediatric febrile neutropenia - Outcomes from a nationwide prospective cohort study

Aminoglycosides are commonly prescribed to children with febrile neutropenia (FN) but their impact on clinical outcomes is uncertain and extent of guideline compliance is unknown. We aimed to review aminoglycoside prescription and additional antibiotic prescribing, guideline compliance and outcomes for children with FN. We analysed data from the Australian Predicting Infectious ComplicatioNs in Children with Cancer (PICNICC) prospective multicentre cohort study, in children <18 years with FN between November 2016 and January 2018. Impact of aminoglycoside use in the first 12 hours of FN on composite unfavourable outcome of death, ICU admission, relapse of infection or late-onset sepsis was assessed using multivariable Cox regression. The study was conducted in Australia where antimicrobial resistance among gram negative organisms is relatively low. Data from 858 episodes of FN in 462 children from 8 centres were assessed, median age 5.8 years (IQR 3.5-10.8 years). Early empiric aminoglycosides were prescribed in 255 episodes (29.7%). Guideline non-compliance was common: in 46% (184/400) of eligible episodes, patients did not receive aminoglycosides, while aminoglycosides were prescribed in 9% (39/458) of guideline-ineligible episodes. Adjusted hazard of the composite unfavourable outcome was 3.81 times higher among patients prescribed empiric aminoglycosides than among those who weren't (95% confidence interval, 1.89-7.67), with no increased risk of unfavourable outcome in eligible patients who did not receive aminoglycosides. In a large paediatric FN cohort, aminoglycoside prescription was common and was often non-compliant with guidelines. There was no evidence for improved outcome with aminoglycosides, even in those who met guideline criteria, within a low-resistance setting. Empiric aminoglycoside prescription for children with FN requires urgent review in guidelines and in national practice.

Home-based care of low-risk febrile neutropenia in children-an implementation study in a tertiary paediatric hospital

BACKGROUND

Home-based management of low-risk febrile neutropenia (FN) is safe, improves quality of life and reduces healthcare expenditure. A formal low-risk paediatric program has not been implemented in Australia. We aimed to describe the implementation process and evaluate the clinical impact.

METHOD

This prospective study incorporated three phases: implementation, intervention and evaluation. A low-risk FN implementation toolkit was developed, including a care-pathway, patient information, home-based assessment and educational resources. The program had executive-level endorsement, a multidisciplinary committee and a nurse specialist. Children with cancer and low-risk FN were eligible to be transferred home with a nurse visiting daily after an overnight period of observation for intravenous antibiotics. Low-risk patients were identified using a validated decision rule, and suitability for home-based care was determined using disease, chemotherapy and patient-level criteria. Plan-Do-Study-Act methodology was used to evaluate clinical impact and safety.

RESULTS

Over 18 months, 292 children with FN were screened: 132 (45%) were low-risk and 63 (22%) were transferred to home-based care. Compared with pre-implementation there was a significant reduction in in-hospital median LOS (4.0 to 1.5 days, p < 0.001) and 291 in-hospital bed days were saved. Eight (13%) patients needed readmission and there were no adverse outcomes. A key barrier was timely screening of all patients and program improvements, including utilising the electronic medical record for patient identification, are planned.

CONCLUSION

This program significantly reduces in-hospital LOS for children with low-risk FN. Ongoing evaluation will inform sustainability, identify areas for improvement and support national scale-up of the program.

Re-evaluating and recalibrating predictors of bacterial infection in children with cancer and febrile neutropenia

BACKGROUND

Numerous paediatric febrile neutropenia (FN) clinical decision rules (CDRs) have been derived. Validation studies show reduced performance in external settings. We evaluated the association between variables common across published FN CDRs and bacterial infection and recalibrated existing CDRs using these data.

METHODS

Prospective data from the Australian-PICNICC study which enrolled 858 FN episodes in children with cancer were used. Variables shown to be significant predictors of infection or adverse outcome in >1 CDR were analysed using multivariable logistic regression. Recalibration included re-evaluation of beta-coefficients (logistic model) or recursive-partition analysis (tree-based models).

FINDINGS

Twenty-five unique variables were identified across 17 FN CDRs. Fourteen were included in >1 CDR and 10 were analysed in our dataset. On univariate analysis, location, temperature, hypotension, rigors, severely unwell and decreasing platelets, white cell count, neutrophil count and monocyte count were significantly associated with bacterial infection. On multivariable analysis, decreasing platelets, increasing temperature and the appearance of being clinically unwell remained significantly associated. Five rules were recalibrated. Across all rules, recalibration increased the AUC-ROC and low-risk yield as compared to non-recalibrated data. For the SPOG-adverse event CDR, recalibration also increased sensitivity and specificity and external validation showed reproducibility.

INTERPRETATION

Degree of marrow suppression (low platelets), features of inflammation (temperature) and clinical judgement (severely unwell) have been consistently shown to predict infection in children with FN. Recalibration of existing CDRs is a novel way to improve diagnostic performance of CDRs and maintain relevance over time.

FUNDING

National Health and Medical Research Council Grant (APP1104527).

Risk stratification in children with cancer and febrile neutropenia: A national, prospective, multicentre validation of nine clinical decision rules

BACKGROUND

Reduced intensity treatment of low-risk febrile neutropenia (FN) in children with cancer is safe and improves quality of life. Identifying children with low-risk FN using a validated risk stratification strategy is recommended. This study prospectively validated nine FN clinical decision rules (CDRs) designed to predict infection or adverse outcome.

METHODS

Data were collected on consecutive FN episodes in this multicentre, prospective validation study. The reproducibility and discriminatory ability of each CDR in the validation cohort was compared to the derivation dataset and details of missed outcomes were reported.

FINDINGS

There were 858 FN episodes in 462 patients from eight hospitals included. Bacteraemia occurred in 111 (12·9%) and a non-bacteraemia microbiological documented infection in 185 (21·6%). Eight CDRs exhibited reproducibility and sensitivity ranged from 64% to 96%. Rules that had >85% sensitivity in predicting outcomes classified few patients (<20%) as low risk. For three CDRs predicting a composite outcome of any bacterial or viral infection, the sensitivity and discriminatory ability improved for prediction of bacterial infection alone. Across all CDRs designed to be implemented at FN presentation, the sensitivity improved at day 2 assessment.

INTERPRETATION

While reproducibility was observed in eight out of the nine CDRs, no rule perfectly differentiated between children with FN at high or low risk of infection. This is in keeping with other validation studies and highlights the need for additional safeguards against missed infections or adverse outcomes before implementation can be considered.

Nonneutropenic fever in children with cancer: A scoping review of management and outcome

To date, very few studies have addressed nonneutropenic fever (NNF) in children with cancer, and there are no consensus guidelines. This scoping review aims to describe the rate of bacteremia, risk factors for infection and management, and outcomes of NNF in this population. Across 15 studies (n = 4106 episodes), the pooled-average bacteremia rate was 8.2%, and risk factors included tunneled external central venous catheter, clinical instability, and higher temperature. In two studies, antibiotics were successfully withheld in a subset of low-risk patients. Overall outcomes of NNF appear favorable; however, further research is required to determine its true clinical and economic impact.

Treatment of fever in neutropenia in pediatric oncology patients

PURPOSE OF REVIEW

Fever during neutropenia is a common occurrence in children with cancer. A number of studies have recently been performed to refine algorithms regarding initiation, modification, and termination of antimicrobial treatment and are the basis for international pediatric-specific guidelines for the treatment of fever and neutropenia in children with cancer.

RECENT FINDINGS

Although hospitalization and prompt initiation of intravenous broad-spectrum antibiotics remains the mainstay in the treatment of febrile neutropenic children with cancer, recent research has addressed a number of questions to optimize the management of these patients. Risk prediction rules have been evaluated to allow for individualized treatment intensity and to evaluate the safety of early discontinuation of empirical antibiotic therapy. In addition, the use of preemptive antifungal therapy has been evaluated to decrease the use of antifungal agents.

SUMMARY

Based on the results of studies in children, pediatric-specific guidelines have been established and are regularly updated.

Episodes of fever in neutropenia in pediatric patients with cancer in Bern, Switzerland, 1993-2012

Fever in neutropenia (FN) is the most frequent potentially life threatening complication of chemotherapy for cancer. Prediction of the risk to develop complications, integrated into clinical decision rules, would allow for risk-stratified treatment of FN. This retrospective, single center cohort study in pediatric patients diagnosed with cancer before 17 years, covered two decades, 1993 to 2012. In total, 703 FN episodes in 291 patients with chemotherapy (maximum per patient, 9) were reported here. Twenty-nine characteristics of FN were collected: 6 were patient- and cancer-related, 8 were characteristics of history, 8 of clinical examination, and 7 laboratory results in peripheral blood, all known at FN diagnosis. In total 28 FN outcomes were assessed: 8 described treatment of FN, 6 described microbiologically defined infections (MDI), 4 clinically defined infections, 4 were additional clinical composite outcomes, and 6 outcomes were related to discharge. These data can mainly be used to study FN characteristics and their association with outcomes over time and between centers, and for derivation and external validation of clinical decision rules.