Mortality Risk Using a Pediatric Quick Sequential (Sepsis-Related) Organ Failure Assessment Varies With Vital Sign Thresholds

Prediction of attributable mortality in pediatric patients with cancer admitted to the intensive care unit for suspected infection: A comprehensive evaluation of risk scores

OBJECTIVE

To evaluate the performance of existing sepsis scores for prediction of adverse outcomes in children with cancer admitted to the ICU with suspected sepsis.

DESIGN

Retrospective chart review using data available at 1, 6, 12, and 24 h after ICU admission to calculate the Pediatric Risk of Mortality 3 (PRISM-3), Pediatric Sequential Organ Failure Assessment (pSOFA), Paediatric Logistic Organ Dysfunction 2 (PELOD-2), and Quick Pediatric Sequential Organ Failure Assessment (qSOFA) scores. Area under the receiver operator characteristic curve (AUROC) was used to evaluate performance for prediction of attributable mortality. Sensitivity analyses included recalculation of scores using worst preceding values for each variable, excluding hematologic parameters, and prediction of alternative outcomes.

SETTING

St. Jude Children's Research Hospital, a pediatric comprehensive cancer center in the USA.

PATIENTS

Pediatric patients (<25 years of age) receiving conventional therapy for cancer admitted to the ICU with suspected sepsis between 2013 and 2019.

RESULTS

Of 207 included episodes of suspected sepsis, attributable mortality was 16 (7.7%) and all evaluated sepsis scores performed poorly (maximal AUROC of 0.73 for qSOFA at 1 and 24 h). Sensitivity analyses did not identify an alternative approach that significantly improved prediction.

CONCLUSIONS

Currently available sepsis scores perform poorly for prediction of attributable mortality in children with cancer who present to ICU with suspected sepsis. More research is needed to identify reliable predictors of adverse outcomes in this population.

Prognostic accuracy of SOFA and qSOFA for mortality among children with infection: a meta-analysis

BACKGROUND

Age-adjusted Sequential Organ Failure Assessment (SOFA) and age-adjusted quick SOFA (qSOFA) scores have been developed to predict poor outcomes in children with infection. We investigated the prognostic performance of age-adjusted SOFA and age-adjusted qSOFA scores and compared them with the systemic inflammatory response syndrome (SIRS) criteria for predicting mortality in children with infection.

METHODS

A bivariate random-effects regression model was used for synthesis of diagnostic test data.

RESULTS

A total of 14 studies invoing 70,194 participants were included. The pooled sensitivity for age-adjusted SOFA, age-adjusted qSOFA, and SIRS were 0.82 (95% CI, 0.74-0.88), 0.46 (95% CI, 0.22-0.71), and 0.79 (95% CI, 0.66-0.88), respectively. The pooled specificity for age-adjusted SOFA, age-adjusted qSOFA, and SIRS were 0.62 (95% CI, 0.45-0.77), 0.90 (95% CI, 0.66-0.98), and 0.39 (95% CI, 0.26-0.54), respectively. The area under the summary receiver operating characteristic curve (AUSROC) for age-adjusted SOFA, age-adjusted qSOFA, and SIRS were 0.82 (95% CI, 0.79-0.85), 0.66 (95% CI, 0.62-0.70), and 0.64 (95% CI, 0.60-0.68), respectively. Different baseline populations, different SOFA adaptation methods and different cut-offs used for age-adjusted SOFA may be potential sources of heterogeneity.

CONCLUSIONS

Age adjusted SOFA score is a useful tool for predicting mortality in children with sepsis/suspected sepsis.

IMPACT

First study to investigate the prognostic performance of age-adjusted sequential organ failure assessment (SOFA) and age adjusted quick SOFA (qSOFA) scores in comparison to the systemic inflammatory response criteria (SIRS) for the prediction of mortality in children with sepsis. The age-adjusted SOFA score predicts poor outcomes with high sensitivity in children with sepsis Low sensitivity limits the utility of age-adjusted qSOFA as a simple predictive tool for adverse outcomes. Developing another enhanced or modified bedside tool with higher sensitivity may be necessary.

Prediction Model Performance With Different Imputation Strategies: A Simulation Study Using a North American ICU Registry

OBJECTIVES

To evaluate the performance of pragmatic imputation approaches when estimating model coefficients using datasets with varying degrees of data missingness.

DESIGN

Performance in predicting observed mortality in a registry dataset was evaluated using simulations of two simple logistic regression models with age-specific criteria for abnormal vital signs (mentation, systolic blood pressure, respiratory rate, WBC count, heart rate, and temperature). Starting with a dataset with complete information, increasing degrees of biased missingness of WBC and mentation were introduced, depending on the values of temperature and systolic blood pressure, respectively. Missing data approaches evaluated included analysis of complete cases only, assuming missing data are normal, and multiple imputation by chained equations. Percent bias and root mean square error, in relation to parameter estimates obtained from the original data, were evaluated as performance indicators.

SETTING

Data were obtained from the Virtual Pediatric Systems, LLC, database (Los Angeles, CA), which provides clinical markers and outcomes in prospectively collected records from 117 PICUs in the United States and Canada.

PATIENTS

Children admitted to a participating PICU in 2017, for whom all required data were available.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Simulations demonstrated that multiple imputation by chained equations is an effective strategy and that even a naive implementation of multiple imputation by chained equations significantly outperforms traditional approaches: the root mean square error for model coefficients was lower using multiple imputation by chained equations in 90 of 99 of all simulations (91%) compared with discarding cases with missing data and lower in 97 of 99 (98%) compared with models assuming missing values are in the normal range. Assuming missing data to be abnormal was inferior to all other approaches.

CONCLUSIONS

Analyses of large observational studies are likely to encounter the issue of missing data, which are likely not missing at random. Researchers should always consider multiple imputation by chained equations (or similar imputation approaches) when encountering even only small proportions of missing data in their work.

Criteria for Pediatric Sepsis-A Systematic Review and Meta-Analysis by the Pediatric Sepsis Definition Taskforce

OBJECTIVE

To determine the associations of demographic, clinical, laboratory, organ dysfunction, and illness severity variable values with: 1) sepsis, severe sepsis, or septic shock in children with infection and 2) multiple organ dysfunction or death in children with sepsis, severe sepsis, or septic shock.

DATA SOURCES

MEDLINE, Embase, and the Cochrane Central Register of Controlled Trials were searched from January 1, 2004, and November 16, 2020.

STUDY SELECTION

Case-control studies, cohort studies, and randomized controlled trials in children greater than or equal to 37-week-old postconception to 18 years with suspected or confirmed infection, which included the terms "sepsis," "septicemia," or "septic shock" in the title or abstract.

DATA EXTRACTION

Study characteristics, patient demographics, clinical signs or interventions, laboratory values, organ dysfunction measures, and illness severity scores were extracted from eligible articles. Random-effects meta-analysis was performed.

DATA SYNTHESIS

One hundred and six studies met eligibility criteria of which 81 were included in the meta-analysis. Sixteen studies (9,629 patients) provided data for the sepsis, severe sepsis, or septic shock outcome and 71 studies (154,674 patients) for the mortality outcome. In children with infection, decreased level of consciousness and higher Pediatric Risk of Mortality scores were associated with sepsis/severe sepsis. In children with sepsis/severe sepsis/septic shock, chronic conditions, oncologic diagnosis, use of vasoactive/inotropic agents, mechanical ventilation, serum lactate, platelet count, fibrinogen, procalcitonin, multi-organ dysfunction syndrome, Pediatric Logistic Organ Dysfunction score, Pediatric Index of Mortality-3, and Pediatric Risk of Mortality score each demonstrated significant and consistent associations with mortality. Pooled mortality rates varied among high-, upper middle-, and lower middle-income countries for patients with sepsis, severe sepsis, and septic shock (p < 0.0001).

CONCLUSIONS

Strong associations of several markers of organ dysfunction with the outcomes of interest among infected and septic children support their inclusion in the data validation phase of the Pediatric Sepsis Definition Taskforce.

Age-adjusted quick Sequential Organ Failure Assessment score for predicting mortality and disease severity in children with infection: a systematic review and meta-analysis

We assessed the diagnostic accuracy of the age-adjusted quick Sequential Organ Failure Assessment score (qSOFA) for predicting mortality and disease severity in pediatric patients with suspected or confirmed infection. We conducted a systematic search of PubMed, EMBASE, the Cochrane Library, and Web of Science. Eleven studies with a total of 172,569 patients were included in the meta-analysis. The pooled sensitivity, specificity, and diagnostic odds ratio of the age-adjusted qSOFA for predicting mortality and disease severity were 0.69 (95% confidence interval [CI] 0.53–0.81), 0.71 (95% CI 0.36–0.91), and 6.57 (95% CI 4.46–9.67), respectively. The area under the summary receiver-operating characteristic curve was 0.733. The pooled sensitivity and specificity for predicting mortality were 0.73 (95% CI 0.66–0.79) and 0.63 (95% CI 0.21–0.92), respectively. The pooled sensitivity and specificity for predicting disease severity were 0.73 (95% CI 0.21–0.97) and 0.72 (95% CI 0.11–0.98), respectively. The performance of the age-adjusted qSOFA for predicting mortality and disease severity was better in emergency department patients than in intensive care unit patients. The age-adjusted qSOFA has moderate predictive power and can help in rapidly identifying at-risk children, but its utility may be limited by its insufficient sensitivity.

Introducing a Radiography-based Score in Children With Acute Respiratory Failure: A Cross-sectional Study

PURPOSE

Respiratory failure (RF) is one of the most common reasons for hospitalization in pediatric intensive care units (PICU). We propose a radiography-based severity score for the assessment of children with RF and investigate the possible associations with severity indices and outcome.

MATERIALS AND METHODS

Children with acute RF admitted in PICU were enrolled. Disease severity scores [Pediatric Risk of Mortality (PRISM) and Pediatric Logistic Organ Dysfunction (PELOD)], the ratio of partial pressure arterial oxygen and fraction of inspired oxygen (PaO2/FiO2) ratios, duration of ventilator support (DVS), length of PICU and hospital stay (LOS), and outcome were recorded. A 5-point radiography score that considered potential radiographic findings was derived through stepwise multivariable logistic regression analysis, and validated. Radiographs upon PICU admission and on the worst RF day (maximum respiratory support and worst oxygenation/ventilation parameters) were blindly reviewed and independently scored by 2 radiologists and 2 clinicians, following training.

RESULTS

We enrolled 104 children [median age 2.7 (interquartile range, 0.5 to 9.6) y, 65.4% boys]. Overall, 163 radiographs (PICU admission: 86, worst RF day: 77) were assessed. Radiography scores correlated positively with predicted mortality (PELOD, PRISM), DVS, LOS (all P<0.001) and inversely with PaO2/FiO2 (P<0.001). Scores differed among diagnostic categories (P<0.05); patients with acute respiratory distress syndrome, air-leaks, drowning, and pneumonia scored the highest (P<0.005). Radiography scoring trends indicating deterioration were associated with prolonged DVS, PICU, and hospital LOS (P<0.001). Agreement between all raters was good (κ=0.7, P<0.001).

CONCLUSIONS

This novel radiography score for children with RF, associated with clinical severity scores, mortality risk, duration of ventilatory support, and hospitalization, follows a simple structured approach and can be readily utilized by radiologists and pediatricians as a bedside tool for stratification of disease severity and prognosis.

An Artificial Neural Network-Based Pediatric Mortality Risk Score: Development and Performance Evaluation Using Data From a Large North American Registry

Background

In the pediatric intensive care unit (PICU), quantifying illness severity can be guided by risk models to enable timely identification and appropriate intervention. Logistic regression models, including the pediatric index of mortality 2 (PIM-2) and pediatric risk of mortality III (PRISM-III), produce a mortality risk score using data that are routinely available at PICU admission. Artificial neural networks (ANNs) outperform regression models in some medical fields.

Objective

In light of this potential, we aim to examine ANN performance, compared to that of logistic regression, for mortality risk estimation in the PICU.

Methods

The analyzed data set included patients from North American PICUs whose discharge diagnostic codes indicated evidence of infection and included the data used for the PIM-2 and PRISM-III calculations and their corresponding scores. We stratified the data set into training and test sets, with approximately equal mortality rates, in an effort to replicate real-world data. Data preprocessing included imputing missing data through simple substitution and normalizing data into binary variables using PRISM-III thresholds. A 2-layer ANN model was built to predict pediatric mortality, along with a simple logistic regression model for comparison. Both models used the same features required by PIM-2 and PRISM-III. Alternative ANN models using single-layer or unnormalized data were also evaluated. Model performance was compared using the area under the receiver operating characteristic curve (AUROC) and the area under the precision recall curve (AUPRC) and their empirical 95% CIs.

Results

Data from 102,945 patients (including 4068 deaths) were included in the analysis. The highest performing ANN (AUROC 0.871, 95% CI 0.862-0.880; AUPRC 0.372, 95% CI 0.345-0.396) that used normalized data performed better than PIM-2 (AUROC 0.805, 95% CI 0.801-0.816; AUPRC 0.234, 95% CI 0.213-0.255) and PRISM-III (AUROC 0.844, 95% CI 0.841-0.855; AUPRC 0.348, 95% CI 0.322-0.367). The performance of this ANN was also significantly better than that of the logistic regression model (AUROC 0.862, 95% CI 0.852-0.872; AUPRC 0.329, 95% CI 0.304-0.351). The performance of the ANN that used unnormalized data (AUROC 0.865, 95% CI 0.856-0.874) was slightly inferior to our highest performing ANN; the single-layer ANN architecture performed poorly and was not investigated further.

Conclusions

A simple ANN model performed slightly better than the benchmark PIM-2 and PRISM-III scores and a traditional logistic regression model trained on the same data set. The small performance gains achieved by this two-layer ANN model may not offer clinically significant improvement; however, further research with other or more sophisticated model designs and better imputation of missing data may be warranted.