Using Machine Learning to Predict Invasive Bacterial Infections in Young Febrile Infants Visiting the Emergency Department

Clinical Timing-Sequence Warning Models for Serious Bacterial Infections in Adults Based on Machine Learning: Retrospective Study

BACKGROUND

Serious bacterial infections (SBIs) are linked to unplanned hospital admissions and a high mortality rate. The early identification of SBIs is crucial in clinical practice.

OBJECTIVE

This study aims to establish and validate clinically applicable models designed to identify SBIs in patients with infective fever.

METHODS

Clinical data from 945 patients with infective fever, encompassing demographic and laboratory indicators, were retrospectively collected from a 2200-bed teaching hospital between January 2013 and December 2020. The data were randomly divided into training and test sets at a ratio of 7:3. Various machine learning (ML) algorithms, including Boruta, Lasso (least absolute shrinkage and selection operator), and recursive feature elimination, were utilized for feature filtering. The selected features were subsequently used to construct models predicting SBIs using logistic regression (LR), random forest (RF), and extreme gradient boosting (XGBoost) with 5-fold cross-validation. Performance metrics, including the receiver operating characteristic (ROC) curve and area under the ROC curve (AUC), accuracy, sensitivity, and other relevant parameters, were used to assess model performance. Considering both model performance and clinical needs, 2 clinical timing-sequence warning models were ultimately confirmed using LR analysis. The corresponding predictive nomograms were then plotted for clinical use. Moreover, a physician, blinded to the study, collected additional data from the same center involving 164 patients during 2021. The nomograms developed in the study were then applied in clinical practice to further validate their clinical utility.

RESULTS

In total, 69.9% (661/945) of the patients developed SBIs. Age, hemoglobin, neutrophil-to-lymphocyte ratio, fibrinogen, and C-reactive protein levels were identified as important features by at least two ML algorithms. Considering the collection sequence of these indicators and clinical demands, 2 timing-sequence models predicting the SBI risk were constructed accordingly: the early admission model (model 1) and the model within 24 hours of admission (model 2). LR demonstrated better stability than RF and XGBoost in both models and performed the best in model 2, with an AUC, accuracy, and sensitivity of 0.780 (95% CI 0.720-841), 0.754 (95% CI 0.698-804), and 0.776 (95% CI 0.711-832), respectively. XGBoost had an advantage over LR in AUC (0.708, 95% CI 0.641-775 vs 0.686, 95% CI 0.617-754), while RF achieved better accuracy (0.729, 95% CI 0.673-780) and sensitivity (0.790, 95% CI 0.728-844) than the other 2 approaches in model 1. Two SBI-risk prediction nomograms were developed for clinical use based on LR, and they exhibited good performance with an accuracy of 0.707 and 0.750 and a sensitivity of 0.729 and 0.927 in clinical application.

CONCLUSIONS

The clinical timing-sequence warning models demonstrated efficacy in predicting SBIs in patients suspected of having infective fever and in clinical application, suggesting good potential in clinical decision-making. Nevertheless, additional prospective and multicenter studies are necessary to further confirm their clinical utility.

Febrile infants risk score at triage (FIRST) for the early identification of serious bacterial infections

We aimed to derive the Febrile Infants Risk Score at Triage (FIRST) to quantify risk for serious bacterial infections (SBIs), defined as bacteremia, meningitis and urinary tract infections. We performed a prospective observational study on febrile infants < 3 months old at a tertiary hospital in Singapore between 2018 and 2021. We utilized machine learning and logistic regression to derive 2 models: FIRST, based on patient demographics, vital signs and history, and FIRST + , adding laboratory results to the same variables. SBIs were diagnosed in 224/1002 (22.4%) infants. Among 994 children with complete data, age (adjusted odds ratio [aOR] 1.01 95%CI 1.01-1.02, p < 0.001), high temperature (aOR 2.22 95%CI 1.69-2.91, p < 0.001), male sex (aOR 2.62 95%CI 1.86-3.70, p < 0.001) and fever of ≥ 2 days (aOR 1.79 95%CI 1.18-2.74, p = 0.007) were independently associated with SBIs. For FIRST + , abnormal urine leukocyte esterase (aOR 16.46 95%CI 10.00-27.11, p < 0.001) and procalcitonin (aOR 1.05 95%CI 1.01-1.09, p = 0.009) were further identified. A FIRST + threshold of ≥ 15% predicted risk had a sensitivity of 81.8% (95%CI 70.5-91.0%) and specificity of 65.6% (95%CI 57.8-72.7%). In the testing dataset, FIRST + had an area under receiver operating characteristic curve of 0.87 (95%CI 0.81-0.94). These scores can potentially guide triage and prioritization of febrile infants.

Explainable deep learning model to predict invasive bacterial infection in febrile young infants: A retrospective study

BACKGROUND

Machine learning models have demonstrated superior performance in predicting invasive bacterial infection (IBI) in febrile infants compared to commonly used risk stratification criteria in recent studies. However, the black-box nature of these models can make them difficult to apply in clinical practice. In this study, we developed and validated an explainable deep learning model that can predict IBI in febrile infants ≤ 60 days of age visiting the emergency department.

METHODS

We conducted a retrospective study of febrile infants aged ≤ 60 days who presented to the pediatric emergency department of a medical center in Taiwan between January 1, 2011 and December 31, 2019. Patients with uncertain test results and complex chronic health conditions were excluded. IBI was defined as the growth of a pathogen in the blood or cerebrospinal fluid. We used a deep neural network to develop a predictive model for IBI and compared its performance to the IBI score and step-by-step approach. The SHapley Additive Explanations (SHAP) technique was used to explain the model's predictions at different levels.

RESULTS

Our study included 1847 patients, 53 (2.7%) of whom had IBI. The deep learning model performed similarly to the IBI score and step-by-step approach in terms of sensitivity and negative predictive value, but provided better specificity (54%), positive predictive value (5%), and area under the receiver-operating characteristic curve (0.87). SHapley Additive exPlanations identified five influential predictive variables (absolute neutrophil count, body temperature, heart rate, age, and C-reactive protein).

CONCLUSION

We have developed an explainable deep learning model that can predict IBI in febrile infants aged 0-60 days. The model not only performs better than previous scoring systems, but also provides insight into how it arrives at its predictions through individual features and cases.

Predictor of Early Administration of Antibiotics and a Volume Resuscitation for Young Infants with Septic Shock

(1) Background: It is critical to administer antibiotics and fluid bolus within 1 h of recognizing sepsis in pediatric patients. This study aimed to identify the predictor of the successful completion of a 1-h sepsis bundle for infants with suspected sepsis. (2) Methods: This is an observational study using a prospective registry including febrile young infants (aged < 90 days) who visited a pediatric emergency department with a core body temperature of 38.0 °C or higher and 36.0 °C or lower. Univariate and logistic regression analyses were conducted to determine the predictor (s) of successful sepsis bundle completion. (3) Results: Of the 323 registered patients, 118 patients with suspected sepsis were analyzed, and 38 patients (32.2%) received a bundle-compliant treatment. Among potential variables, such as age, sex, and vital sign parameters, the logistic regression analysis showed that heart rate (odds ratio: OR 1.02; 95% confidence interval: 1.00-1.04) is a significant predictor of the completion of a 1-h sepsis bundle. (4) Conclusions: We found that tachycardia facilitated the sepsis recognition and promoted the successful completion of a 1-h sepsis bundle for young infants with suspected septic shock and a possible indicator for improving the quality of the team-based sepsis management.