Using Machine Learning to Predict Bacteremia in Febrile Children Presented to the Emergency Department

A Machine Learning Predictive Model of Bloodstream Infection in Hospitalized Patients

The aim of the study was to build a machine learning-based predictive model to discriminate between hospitalized patients at low risk and high risk of bloodstream infection (BSI). A Data Mart including all patients hospitalized between January 2016 and December 2019 with suspected BSI was built. Multivariate logistic regression was applied to develop a clinically interpretable machine learning predictive model. The model was trained on 2016-2018 data and tested on 2019 data. A feature selection based on a univariate logistic regression first selected candidate predictors of BSI. A multivariate logistic regression with stepwise feature selection in five-fold cross-validation was applied to express the risk of BSI. A total of 5660 hospitalizations (4026 and 1634 in the training and the validation subsets, respectively) were included. Eleven predictors of BSI were identified. The performance of the model in terms of AUROC was 0.74. Based on the interquartile predicted risk score, 508 (31.1%) patients were defined as being at low risk, 776 (47.5%) at medium risk, and 350 (21.4%) at high risk of BSI. Of them, 14.2% (72/508), 30.8% (239/776), and 64% (224/350) had a BSI, respectively. The performance of the predictive model of BSI is promising. Computational infrastructure and machine learning models can help clinicians identify people at low risk for BSI, ultimately supporting an antibiotic stewardship approach.

A comparative study of explainable ensemble learning and logistic regression for predicting in-hospital mortality in the emergency department

This study addresses the challenges associated with emergency department (ED) overcrowding and emphasizes the need for efficient risk stratification tools to identify high-risk patients for early intervention. While several scoring systems, often based on logistic regression (LR) models, have been proposed to indicate patient illness severity, this study aims to compare the predictive performance of ensemble learning (EL) models with LR for in-hospital mortality in the ED. A cross-sectional single-center study was conducted at the ED of Imam Reza Hospital in northeast Iran from March 2016 to March 2017. The study included adult patients with one to three levels of emergency severity index. EL models using Bagging, AdaBoost, random forests (RF), Stacking and extreme gradient boosting (XGB) algorithms, along with an LR model, were constructed. The training and validation visits from the ED were randomly divided into 80% and 20%, respectively. After training the proposed models using tenfold cross-validation, their predictive performance was evaluated. Model performance was compared using the Brier score (BS), The area under the receiver operating characteristics curve (AUROC), The area and precision-recall curve (AUCPR), Hosmer-Lemeshow (H-L) goodness-of-fit test, precision, sensitivity, accuracy, F1-score, and Matthews correlation coefficient (MCC). The study included 2025 unique patients admitted to the hospital's ED, with a total percentage of hospital deaths at approximately 19%. In the training group and the validation group, 274 of 1476 (18.6%) and 152 of 728 (20.8%) patients died during hospitalization, respectively. According to the evaluation of the presented framework, EL models, particularly Bagging, predicted in-hospital mortality with the highest AUROC (0.839, CI (0.802-0.875)) and AUCPR = 0.64 comparable in terms of discrimination power with LR (AUROC (0.826, CI (0.787-0.864)) and AUCPR = 0.61). XGB achieved the highest precision (0.83), sensitivity (0.831), accuracy (0.842), F1-score (0.833), and the highest MCC (0.48). Additionally, the most accurate models in the unbalanced dataset belonged to RF with the lowest BS (0.128). Although all studied models overestimate mortality risk and have insufficient calibration (P > 0.05), stacking demonstrated relatively good agreement between predicted and actual mortality. EL models are not superior to LR in predicting in-hospital mortality in the ED. Both EL and LR models can be considered as screening tools to identify patients at risk of mortality.

Machine learning model for the prediction of gram-positive and gram-negative bacterial bloodstream infection based on routine laboratory parameters

BACKGROUND

Bacterial bloodstream infection is responsible for the majority of cases of sepsis and septic shock. Early recognition of the causative pathogen is pivotal for administration of adequate empiric antibiotic therapy and for the survival of the patients. In this study, we developed a feasible machine learning (ML) model to predict gram-positive and gram-negative bacteremia based on routine laboratory parameters.

METHODS

Data for 2118 patients with bacteremia were obtained from the Medical Information Mart for Intensive Care dataset. Patients were randomly split into the training set and test set by stratified sampling, and 374 routine laboratory blood test variables were retrieved. Variables with missing values in more than 40% of the patients were excluded. Pearson correlation test was employed to eliminate redundant features. Five ML algorithms were used to build the model based on the selected features. Additionally, 132 patients with bacteremia who were treated at Qilu Hospital of Shandong University were included in an independent test cohort to evaluate the model.

RESULTS

After feature selection, 32 variables remained. All the five ML algorithms performed well in terms of discriminating between gram-positive and gram-negative bacteremia, but the performance of convolutional neural network (CNN) and random forest (RF) were better than other three algorithms. Consider of the interpretability of models, RF was chosen for further test (ROC-AUC = 0.768; 95%CI = 0.715-0.798, with a sensitivity of 75.20% and a specificity of 63.79%). To expand the application of the model, a decision tree (DT) was built utilizing the major variables, and it achieved an AUC of 0.679 (95%CI = 0.632-0.723), a sensitivity of 66%, and a specificity of 67.82% in the test cohort. When tested in the Qilu Hospital cohort, the ROC-AUC of the RF and DT models were 0.666 (95%CI = 0.579-0.746) and 0.615 (95%CI = 0.526-0.698), respectively. Finally, a software was developed to make the RF- and DT-based prediction models easily accessible.

CONCLUSION

The present ML-based models could effectively discriminate between gram-positive and gram-negative bacteremia based on routine laboratory blood test results. This simple model would be beneficial in terms of guiding timely antibiotic selection and administration in critically ill patients with bacteremia before their pathogen test results are available.

Real-time artificial intelligence system for bacteremia prediction in adult febrile emergency department patients

BACKGROUND

Artificial intelligence (AI) holds significant potential to be a valuable tool in healthcare. However, its application for predicting bacteremia among adult febrile patients in the emergency department (ED) remains unclear. Therefore, we conducted a study to provide clarity on this issue.

METHODS

Adult febrile ED patients with blood cultures at Chi Mei Medical Center were divided into derivation (January 2017 to June 2019) and validation groups (July 2019 to December 2020). The derivation group was utilized to develop AI models using twenty-one feature variables and five algorithms to predict bacteremia. The performance of these models was compared with qSOFA score. The AI model with the highest area under the receiver operating characteristics curve (AUC) was chosen to implement the AI prediction system and tested on the validation group.

RESULTS

The study included 5,647 febrile patients. In the derivation group, there were 3,369 patients with a mean age of 61.4 years, and 50.7% were female, including 508 (13.8%) with bacteremia. The model with the best AUC was built using the random forest algorithm (0.761), followed by logistic regression (0.755). All five models demonstrated better AUC than the qSOFA score (0.560). The random forest model was adopted to build a real-time AI prediction system integrated into the hospital information system, and the AUC achieved 0.709 in the validation group.

CONCLUSION

The AI model shows promise to predict bacteremia in adult febrile ED patients; however, further external validation in different hospitals and populations is necessary to verify its effectiveness.

Hemoglobin and Its Z Score Reference Intervals in Febrile Children: A Cohort Study of 98,572 Febrile Children

OBJECTIVES

Febrile disease and age of children were associated with a variation in hemoglobin (Hb) level. Both CRP and Hb serve as laboratory markers that offer valuable insights into a patient's health, particularly in relation to inflammation and specific medical conditions. Although a direct correlation between CRP and Hb levels is not established, the relationship between these markers has garnered academic attention and investigation. This study aimed to determine updated reference ranges for Hb levels for age and investigated its correlation with CRP in febrile children under the age of 18.

METHODS

This is a cohort study of in Chang Gung Memorial Hospitals conducted from January 2010 to December 2019. Blood samples were collected from 98,572 febrile children who were or had been admitted in the pediatric emergency department. The parameters of individuals were presented as the mean ± standard deviation or 2.5th and 97.5th percentiles. We also determined the variation of Hb and Z score of Hb between CRP levels in febrile children.

RESULT

We observed that the Hb levels were the highest immediately after birth and subsequently underwent a rapid decline, reaching their lowest point at around 1-2 months of age, and followed by a steady increment in Hb levels throughout childhood and adolescence. In addition, there was a significant and wide variation in Hb levels during the infant period. It revealed a significant association between higher CRP levels and lower Hb levels or a more negative Z score of Hb across all age subgroups. Moreover, in patients with bacteremia, CRP levels were higher, Hb concentrations were lower, and Z scores of Hb were also lower compared to the non-bacteremia group. Furthermore, the bacteremia group exhibited a more substantial negative correlation between CRP levels and a Z score of Hb (r = -0.41, p < 0.001) compared to the non-bacteremia group (r = -0.115, p < 0.049).

CONCLUSION

The study findings revealed that the Hb references varied depending on the age of the children and their CRP levels. In addition, we established new reference values for Hb and its Z scores and explore their relationship with CRP. It provides valuable insights into the Hb status and its potential association with inflammation in febrile pediatric patients.

Explainable deep learning model to predict invasive bacterial infection in febrile young infants: A retrospective study

BACKGROUND

Machine learning models have demonstrated superior performance in predicting invasive bacterial infection (IBI) in febrile infants compared to commonly used risk stratification criteria in recent studies. However, the black-box nature of these models can make them difficult to apply in clinical practice. In this study, we developed and validated an explainable deep learning model that can predict IBI in febrile infants ≤ 60 days of age visiting the emergency department.

METHODS

We conducted a retrospective study of febrile infants aged ≤ 60 days who presented to the pediatric emergency department of a medical center in Taiwan between January 1, 2011 and December 31, 2019. Patients with uncertain test results and complex chronic health conditions were excluded. IBI was defined as the growth of a pathogen in the blood or cerebrospinal fluid. We used a deep neural network to develop a predictive model for IBI and compared its performance to the IBI score and step-by-step approach. The SHapley Additive Explanations (SHAP) technique was used to explain the model's predictions at different levels.

RESULTS

Our study included 1847 patients, 53 (2.7%) of whom had IBI. The deep learning model performed similarly to the IBI score and step-by-step approach in terms of sensitivity and negative predictive value, but provided better specificity (54%), positive predictive value (5%), and area under the receiver-operating characteristic curve (0.87). SHapley Additive exPlanations identified five influential predictive variables (absolute neutrophil count, body temperature, heart rate, age, and C-reactive protein).

CONCLUSION

We have developed an explainable deep learning model that can predict IBI in febrile infants aged 0-60 days. The model not only performs better than previous scoring systems, but also provides insight into how it arrives at its predictions through individual features and cases.

Benchmarking emergency department prediction models with machine learning and public electronic health records

The demand for emergency department (ED) services is increasing across the globe, particularly during the current COVID-19 pandemic. Clinical triage and risk assessment have become increasingly challenging due to the shortage of medical resources and the strain on hospital infrastructure caused by the pandemic. As a result of the widespread use of electronic health records (EHRs), we now have access to a vast amount of clinical data, which allows us to develop prediction models and decision support systems to address these challenges. To date, there is no widely accepted clinical prediction benchmark related to the ED based on large-scale public EHRs. An open-source benchmark data platform would streamline research workflows by eliminating cumbersome data preprocessing, and facilitate comparisons among different studies and methodologies. Based on the Medical Information Mart for Intensive Care IV Emergency Department (MIMIC-IV-ED) database, we created a benchmark dataset and proposed three clinical prediction benchmarks. This study provides future researchers with insights, suggestions, and protocols for managing data and developing predictive tools for emergency care.

Predicting Bacteremia among Septic Patients Based on ED Information by Machine Learning Methods: A Comparative Study

Introduction: Bacteremia is a common but life-threatening infectious disease. However, a well-defined rule to assess patient risk of bacteremia and the urgency of blood culture is lacking. The aim of this study is to establish a predictive model for bacteremia in septic patients using available big data in the emergency department (ED) through logistic regression and other machine learning (ML) methods. Material and Methods: We conducted a retrospective cohort study at the ED of National Cheng Kung University Hospital in Taiwan from January 2015 to December 2019. ED adults (≥18 years old) with systemic inflammatory response syndrome and receiving blood cultures during the ED stay were included. Models I and II were established based on logistic regression, both of which were derived from support vector machine (SVM) and random forest (RF). Net reclassification index was used to determine which model was superior. Results: During the study period, 437,969 patients visited the study ED, and 40,395 patients were enrolled. Patients diagnosed with bacteremia accounted for 7.7% of the cohort. The area under the receiver operating curve (AUROC) in models I and II was 0.729 (95% CI, 0.718–0.740) and 0.731 (95% CI, 0.721–0.742), with Akaike information criterion (AIC) of 16,840 and 16,803, respectively. The performance of model II was superior to that of model I. The AUROC values of models III and IV in the validation dataset were 0.730 (95% CI, 0.713–0.747) and 0.705 (0.688–0.722), respectively. There is no statistical evidence to support that the performance of the model created with logistic regression is superior to those created by SVM and RF. Discussion: The advantage of the SVM or RF model is that the prediction model is more elastic and not limited to a linear relationship. The advantage of the LR model is that it is easy to explain the influence of the independent variable on the response variable. These models could help medical staff identify high-risk patients and prevent unnecessary antibiotic use. The performance of SVM and RF was not inferior to that of logistic regression. Conclusions: We established models that provide discrimination in predicting bacteremia among patients with sepsis. The reported results could inspire researchers to adopt ML in their development of prediction algorithms.

Bacteremia detection from complete blood count and differential leukocyte count with machine learning: complementary and competitive with C-reactive protein and procalcitonin tests

Background

Biomarkers, such as leukocyte count, C-reactive protein (CRP), and procalcitonin (PCT), have been commonly used to predict the occurrence of life-threatening bacteremia and provide prognostic information, given the need for prompt intervention. However, such diagnosis methods require much time and money. Therefore, we propose a method with a high prediction capability using machine learning (ML) models based on complete blood count (CBC) and differential leukocyte count (DC) and compare its performance with traditional CRP or PCT biomarker methods and those of models incorporating CRP or PCT biomarkers.

Methods

We collected 366,586 daily blood culture (BC) results, of which 350,775 (93.2%), 308,803 (82.1%), and 23,912 (6.4%) cases were issued CBC/DC (CBC/DC group), CRP with CBC/DC (CRP&CBC/DC group), and PCT with CBC/DC (PCT&CBC/DC group), respectively. For the ML methods, conventional logistic regression and random forest models were selected, trained, applied, and validated for each group. Fivefold validation and prediction capability were also evaluated and reported.

Results

Overall, the ML methods, such as the random forest model, demonstrated promising performances. When trained with CBC/DC data, it achieved an area under the ROC curve (AUC) of 0.802, which is superior to the prediction conventionally made with CRP/PCT levels (0.699/0.731). Upon evaluating the performance enhanced by incorporating CRP or PCT biomarkers, it reported no substantial AUC increase with the addition of either CRP or PCT to CBC/DC data, which suggests the predicting power and applicability of using only CBC/DC data. Moreover, it showed competitive prognostic capability compared to the PCT test with similar all-cause in-hospital mortality (45.10% vs. 47.40%) and overall median survival time (27 vs. 25 days).

Conclusions

The ML models using only CBC/DC data yielded more accurate bacteremia predictions compared to those by methods using CRP and PCT data and reached similar prognostic performance as by PCT data. Thus, such models are potentially complementary and competitive with traditional CRP and PCT biomarkers for conducting and guiding antibiotic usage.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12879-022-07223-7.

Predictor of Early Administration of Antibiotics and a Volume Resuscitation for Young Infants with Septic Shock

(1) Background: It is critical to administer antibiotics and fluid bolus within 1 h of recognizing sepsis in pediatric patients. This study aimed to identify the predictor of the successful completion of a 1-h sepsis bundle for infants with suspected sepsis. (2) Methods: This is an observational study using a prospective registry including febrile young infants (aged < 90 days) who visited a pediatric emergency department with a core body temperature of 38.0 °C or higher and 36.0 °C or lower. Univariate and logistic regression analyses were conducted to determine the predictor (s) of successful sepsis bundle completion. (3) Results: Of the 323 registered patients, 118 patients with suspected sepsis were analyzed, and 38 patients (32.2%) received a bundle-compliant treatment. Among potential variables, such as age, sex, and vital sign parameters, the logistic regression analysis showed that heart rate (odds ratio: OR 1.02; 95% confidence interval: 1.00-1.04) is a significant predictor of the completion of a 1-h sepsis bundle. (4) Conclusions: We found that tachycardia facilitated the sepsis recognition and promoted the successful completion of a 1-h sepsis bundle for young infants with suspected septic shock and a possible indicator for improving the quality of the team-based sepsis management.

An Artificial Intelligence Approach to Bloodstream Infections Prediction

This study aimed to develop an early prediction model for identifying patients with bloodstream infections. The data resource was taken from 2015 to 2019 at Taichung Veterans General Hospital, and a total of 1647 bloodstream infection episodes and 3552 non-bloodstream infection episodes in the intensive care unit (ICU) were included in the model development and evaluation. During the data analysis, 30 clinical variables were selected, including patients’ basic characteristics, vital signs, laboratory data, and clinical information. Five machine learning algorithms were applied to examine the prediction model performance. The findings indicated that the area under the receiver operating characteristic curve (AUROC) of the prediction performance of the XGBoost model was 0.825 for the validation dataset and 0.821 for the testing dataset. The random forest model also presented higher values for the AUROC on the validation dataset and testing dataset, which were 0.855 and 0.851, respectively. The tree-based ensemble learning model enabled high detection ability for patients with bloodstream infections in the ICU. Additionally, the analysis of importance of features revealed that alkaline phosphatase (ALKP) and the period of the central venous catheter are the most important predictors for bloodstream infections. We further explored the relationship between features and the risk of bloodstream infection by using the Shapley Additive exPlanations (SHAP) visualized method. The results showed that a higher prothrombin time is more prominent in a bloodstream infection. Additionally, the impact of a lower platelet count and albumin was more prominent in a bloodstream infection. Our results provide additional clinical information for cut-off laboratory values to assist clinical decision-making in bloodstream infection diagnostics.