Bacteremia Prediction Model for Community-acquired Pneumonia: External Validation in a Multicenter Retrospective Cohort

A Simple Bacteremia Score for Predicting Bacteremia in Patients with Suspected Infection in the Emergency Department: A Cohort Study

Bacteremia is a life-threatening condition that has increased in prevalence over the past two decades. Prompt recognition of bacteremia is important; however, identification of bacteremia requires 1 to 2 days. This retrospective cohort study, conducted from 10 November 2014 to November 2019, among patients with suspected infection who visited the emergency department (ED), aimed to develop and validate a simple tool for predicting bacteremia. The study population was randomly divided into derivation and development cohorts. Predictors of bacteremia based on the literature and logistic regression were assessed. A weighted value was assigned to predictors to develop a prediction model for bacteremia using the derivation cohort; discrimination was then assessed using the area under the receiver operating characteristic curve (AUC). Among the 22,519 patients enrolled, 18,015 were assigned to the derivation group and 4504 to the validation group. Sixteen candidate variables were selected, and all sixteen were used as significant predictors of bacteremia (model 1). Among the sixteen variables, the top five with higher odds ratio, including procalcitonin, neutrophil-lymphocyte ratio (NLR), lactate level, platelet count, and body temperature, were used for the simple bacteremia score (model 2). The proportion of bacteremia increased according to the simple bacteremia score in both cohorts. The AUC for model 1 was 0.805 (95% confidence interval [CI] 0.785-0.824) and model 2 was 0.791 (95% CI 0.772-0.810). The simple bacteremia prediction score using only five variables demonstrated a comparable performance with the model including sixteen variables using all laboratory results and vital signs. This simple score is useful for predicting bacteremia-assisted clinical decisions.

Concurrent external validation of bloodstream infection probability models

OBJECTIVE

Accurately estimating the likelihood of bloodstream infection (BSI) can help clinicians make diagnostic and therapeutic decisions. Many multivariate models predicting BSI probability have been published. This study measured the performance of BSI probability models within the same patient sample.

METHODS

We retrieved validated BSI probability models included in a recently published systematic review that returned a patient-level BSI probability for adults. Model applicability, discrimination, and accuracy was measured in a simple random sample of 4485 admitted adults having blood cultures ordered in the emergency department or the initial 48 hours of hospitalization.

RESULTS

Ten models were included (publication years 1991-2015). Common methodological threats to model performance included overfitting and continuous variable categorization. Restrictive inclusion criteria caused seven models to apply to <15% of validation patients. Model discrimination was less than originally reported in derivation groups (median c-statistic 60%, range 48-69). The observed BSI risk frequently deviated from expected (median integrated calibration index 4.0%, range 0.8-12.4). Notable disagreement in expected BSI probabilities was seen between models (median (25th-75th percentile) relative difference between expected risks 68.0% (28.6-113.6%)).

DISCUSSION

In a large randomly selected external validation population, many published BSI probability models had restricted applicability, limited discrimination and calibration, and extensive inter-model disagreement. Direct comparison of model performance is hampered by dissimilarities between model-specific validation groups.

ESTIMATING THE RISK OF BACTERAEMIA IN HOSPITALISED PATIENTS WITH PNEUMOCOCCAL PNEUMONIA

Objective To construct a prediction model for bacteraemia in patients with pneumococcal community-acquired pneumonia (P-CAP) based on variables easily obtained at hospital admission. MethodsThis prospective observational multicentre derivation-validation study was conducted in patients hospitalised with P-CAP between 2000-2020. All cases were diagnosed based on positive urinary antigen tests in the emergency department and had blood cultures taken on admission. A risk score to predict bacteraemia was developed. Results We included 1783 patients with P-CAP (1195 in the derivation and 588 in the validation cohort). A third (33.3%) of the patients had bacteraemia. In the multivariate analysis, the following were identified as independent factors associated with bacteraemia: no influenza vaccination the last year, no pneumococcal vaccination in the last 5 years, blood urea nitrogen (BUN) ≥30 mg/dL, sodium <130 mmol/L, lymphocyte count <800/µl, C-reactive protein ≥200 mg/L, respiratory failure, pleural effusion and no antibiotic treatment before admission. The score yielded good discrimination (AUC 0.732; 95% CI: 0.695-0.769) and calibration (Hosmer-Lemeshow p-value 0.801), with similar performance in the validation cohort (AUC 0.764; 95% CI:0.719-0.809). Conclusions We found nine predictive factors easily obtained on hospital admission that could help achieve early identification of bacteraemia. The prediction model provides a useful tool to guide diagnostic decisions.

Diagnosing hospital bacteraemia in the framework of predictive, preventive and personalised medicine using electronic health records and machine learning classifiers

Background

The bacteraemia prediction is relevant because sepsis is one of the most important causes of morbidity and mortality. Bacteraemia prognosis primarily depends on a rapid diagnosis. The bacteraemia prediction would shorten up to 6 days the diagnosis, and, in conjunction with individual patient variables, should be considered to start the early administration of personalised antibiotic treatment and medical services, the election of specific diagnostic techniques and the determination of additional treatments, such as surgery, that would prevent subsequent complications. Machine learning techniques could help physicians make these informed decisions by predicting bacteraemia using the data already available in electronic hospital records.

Objective

This study presents the application of machine learning techniques to these records to predict the blood culture's outcome, which would reduce the lag in starting a personalised antibiotic treatment and the medical costs associated with erroneous treatments due to conservative assumptions about blood culture outcomes.

Methods

Six supervised classifiers were created using three machine learning techniques, Support Vector Machine, Random Forest and K-Nearest Neighbours, on the electronic health records of hospital patients. The best approach to handle missing data was chosen and, for each machine learning technique, two classification models were created: the first uses the features known at the time of blood extraction, whereas the second uses four extra features revealed during the blood culture.

Results

The six classifiers were trained and tested using a dataset of 4357 patients with 117 features per patient. The models obtain predictions that, for the best case, are up to a state-of-the-art accuracy of 85.9%, a sensitivity of 87.4% and an AUC of 0.93.

Conclusions

Our results provide cutting-edge metrics of interest in predictive medical models with values that exceed the medical practice threshold and previous results in the literature using classical modelling techniques in specific types of bacteraemia. Additionally, the consistency of results is reasserted because the three classifiers' importance ranking shows similar features that coincide with those that physicians use in their manual heuristics. Therefore, the efficacy of these machine learning techniques confirms their viability to assist in the aims of predictive and personalised medicine once the disease presents bacteraemia-compatible symptoms and to assist in improving the healthcare economy.

The Development and Validation of a Machine Learning Model to Predict Bacteremia and Fungemia in Hospitalized Patients Using Electronic Health Record Data

OBJECTIVES

Bacteremia and fungemia can cause life-threatening illness with high mortality rates, which increase with delays in antimicrobial therapy. The objective of this study is to develop machine learning models to predict blood culture results at the time of the blood culture order using routine data in the electronic health record.

DESIGN

Retrospective analysis of a large, multicenter inpatient data.

SETTING

Two academic tertiary medical centers between the years 2007 and 2018.

SUBJECTS

All hospitalized patients who received a blood culture during hospitalization.

INTERVENTIONS

The dataset was partitioned temporally into development and validation cohorts: the logistic regression and gradient boosting machine models were trained on the earliest 80% of hospital admissions and validated on the most recent 20%.

MEASUREMENTS AND MAIN RESULTS

There were 252,569 blood culture days-defined as nonoverlapping 24-hour periods in which one or more blood cultures were ordered. In the validation cohort, there were 50,514 blood culture days, with 3,762 cases of bacteremia (7.5%) and 370 cases of fungemia (0.7%). The gradient boosting machine model for bacteremia had significantly higher area under the receiver operating characteristic curve (0.78 [95% CI 0.77-0.78]) than the logistic regression model (0.73 [0.72-0.74]) (p < 0.001). The model identified a high-risk group with over 30 times the occurrence rate of bacteremia in the low-risk group (27.4% vs 0.9%; p < 0.001). Using the low-risk cut-off, the model identifies bacteremia with 98.7% sensitivity. The gradient boosting machine model for fungemia had high discrimination (area under the receiver operating characteristic curve 0.88 [95% CI 0.86-0.90]). The high-risk fungemia group had 252 fungemic cultures compared with one fungemic culture in the low-risk group (5.0% vs 0.02%; p < 0.001). Further, the high-risk group had a mortality rate 60 times higher than the low-risk group (28.2% vs 0.4%; p < 0.001).

CONCLUSIONS

Our novel models identified patients at low and high-risk for bacteremia and fungemia using routinely collected electronic health record data. Further research is needed to evaluate the cost-effectiveness and impact of model implementation in clinical practice.

Predicting bacteraemia in maternity patients using full blood count parameters: A supervised machine learning algorithm approach

INTRODUCTION

Bacteraemia in pregnancy and the post-partum period can lead to maternal and newborn morbidly. The purpose of this study was to use machine learning tools to identify if bacteraemia in pregnant or post-partum women could be predicted by full blood count (FBC) parameters other than the white cell count.

METHODS

The study was performed on 129 women with a positive blood culture (BC) for a clinically significant organism, who had a FBC taken at the same time. They were matched with controls who had a negative BC taken at the same time as a FBC. The data were split in to a training (70%) and test (30%) data set. Machine learning techniques such as recursive partitioning and classification and regression trees were used.

RESULTS

A neutrophil/lymphocyte ratio (NLR) of >20 was found to be the most clinically relevant and interpretable construct of the FBC result to predict bacteraemia. The diagnostic accuracy of NLR >20 to predict bacteraemia was then examined. Thirty-six of the 129 bacteraemia patients had a NLR >20, while only 223 of the 3830 controls had a NLR >20. This gave a sensitivity of 27.9% (95% CI 20.3-36.4), specificity of 94.1% (93.3-94.8), positive predictive value of 13.9% (10.6-17.9) and a negative predictive value (NPV) of 97.4% (97.2-97.7) when the prevalence of bacteraemia was 3%.

CONCLUSION

The NLR should be considered for use in routine clinical practice when assessing the FBC result in patients with suspected bacteraemia during pregnancy or in the post-partum period.

The Shapiro-Procalcitonin algorithm (SPA) as a decision tool for blood culture sampling: validation in a prospective cohort study

PURPOSE

Blood cultures (BC) are the gold standard for bacteremia detection despite a relatively low diagnostic yield and high costs. A retrospective study reported high predictive values for BC positivity when combining the clinical Shapiro score with procalcitonin (PCT).

METHODS

Single-center, prospective cohort study between 01/2016 and 02/2017 to validate SPA algorithm, including a modified Shapiro score ≥ 3 points (S) PLUS admission PCT > 0.25 µg/l (P), or presence of overruling safety criteria (A) in patients with systemic inflammatory response syndrome. The diagnostic yield of SPA compared to non-standardized clinical judgment in predicting BC positivity was calculated and results presented as odds ratios (OR) with 95% confidence intervals.

RESULTS

Of 1438 patients with BC sampling, 215 (15%) had positive BC which increased to 31% (173/555) in patients fulfilling SP criteria (OR for BC positivity 9.07 [6.34-12.97]). When adding 194 patients with overruling safety criteria (i.e., SPA), OR increased to 11.12 (6.99-17.69), although BC positivity slightly decreased to 26%. With an area under the receiver operating curve of 0.742, SPA indicated better diagnostic performance than its individual components. Positive BC in 689 patients not fulfilling SPA (sampling according to non-standardized clinical judgment) were rare (3%; OR for BC positivity 0.09 [0.06-0.14]). Eight out of 21 missed pathogens were still identified by sampling the primary infection focus.

CONCLUSIONS

This study validates the high predictive value of SPA for bacteremia, increasing true BC positivity from 15 to 26%. Restricting BC sampling to SPA would have reduced BC sampling by 48%, while still detecting 194/215 organisms (90%), which makes SPA a valuable diagnostic stewardship tool.

Association of Systemic Inflammatory Response Syndrome with Bacteremia in Patients with Sepsis

The aim of this study was to evaluate the usability of systemic inflammatory response syndrome (SIRS) and commonly used biochemical parameters as predictors for positive blood culture in patients with sepsis. The study included 313 patients aged ≥18 years with severe sepsis and septic shock consecutively admitted in the Intensive Care Unit (ICU) of the University Clinic for Infectious Diseases in Skopje, Republic of North Macedonia. The study took place from January 1, 2011 to December 31, 2017. We recorded demographic variables, common laboratory tests, SIRS parameters, site of infection, comorbidities and Sequential Organ Failure Assessment (SOFA) score. Blood cultures were positive in 65 (20.8%) patients with sepsis. Gram-positive bacteria were isolated from 35 (53.8%) patients. From the evaluated variables in this study, only the presence of four SIRS parameters was associated with bacteremia, finding that will help to predict bacteremia and initiate early appropriate therapy in septic patients.

Rate and Predictors of Bacteremia in Afebrile Community-Acquired Pneumonia

BACKGROUND

Although blood cultures (BCs) are the "gold standard" for detecting bacteremia, the utility of BCs in patients with community-acquired pneumonia (CAP) is controversial. This study describes the proportion of patients with CAP and afebrile bacteremia and identifies the clinical characteristics predicting the necessity for BCs in patients who are afebrile.

METHODS

Bacteremia rates were determined in 4,349 patients with CAP enrolled in the multinational cohort study The Competence Network of Community-Acquired Pneumonia (CAPNETZ) and stratified by presence of fever at first patient contact. Independent predictors of bacteremia in patients who were afebrile were determined using logistic regression analysis.

RESULTS

Bacteremic pneumonia was present in 190 of 2,116 patients who were febrile (8.9%), 101 of 2,149 patients who were afebrile (4.7%), and one of 23 patients with hypothermia (4.3%). Bacteremia rates increased with the CURB-65 score from 3.5% in patients with CURB-65 score of 0 to 17.1% in patients with CURB-65 score of 4. Patients with afebrile bacteremia exhibited the highest 28-day mortality rate (9.9%). Positive pneumococcal urinary antigen test (adjusted OR [AOR], 4.6; 95% CI, 2.6-8.2), C-reactive protein level > 200 mg/L (AOR, 3.1; 95% CI, 1.9-5.2), and BUN level ≥ 30 mg/dL (AOR, 3.1; 95% CI, 1.9-5.3) were independent positive predictors, and antibiotic pretreatment (AOR, 0.3; 95% CI, 0.1-0.6) was an independent negative predictor of bacteremia in patients who were afebrile.

CONCLUSIONS

A relevant proportion of patients with bacteremic CAP was afebrile. These patients had an increased mortality rate compared with patients with febrile bacteremia or nonbacteremic pneumonia. Therefore, the relevance of fever as an indicator for BC necessity merits reconsideration.

Impact of bacteremia prediction rule in CAP: Before and after study

OBJECTIVE

In cases of community acquired pneumonia (CAP), it has been known that blood cultures have low yields and rarely affect clinical outcomes. Despite many studies predicting the likelihood of bacteremia in CAP patients, those results have been rarely implemented in clinical practice, and use of blood culture in CAP is still increasing. This study evaluated impact of implementing a previously derived and validated bacteremia prediction rule.

METHODS

In this registry-based before and after study, we used piecewise regression analysis to compare the blood culture rate before and after implementation of the prediction rule. We also compared 30-day mortality, emergency department (ED) length of stay, time-interval to initial antibiotics after ED arrival, and any changes to the antibiotics regimen as results of the blood cultures. In subgroup analysis, we compared two groups (with or without the use of the prediction rule) after implementation period, using propensity score matching.

RESULTS

Following the implementation, the blood culture rate declined from 85.5% to 78.1% (P=0.003) without significant changes in 30-day mortality and antibiotics regimen. The interval to initial antibiotics (231min vs. 221min, P=0.362) and length of stay (1019min vs. 954min, P=0.354) were not significantly changed. In subgroup analysis, the group that use the prediction rule showed 25min faster antibiotics initiation (P=0.002) and 48min shorter length of stay (P=0.007) than the group that did not use the rule.

CONCLUSION

Implementation of the bacteremia prediction rule in CAP patients reduced the blood culture rate without affecting the 30-day mortality and antibiotics regimen.

How to: accreditation of blood cultures' proceedings. A clinical microbiology approach for adding value to patient care

BACKGROUND

Quality assurance and quality management are driving forces for controlling blood culture best practices but should not be disconnected from the end-point target, i.e. patient value.

AIMS

This article is intended to help microbiologists implement blood culture accreditation that is actually beneficial to patient management.

SOURCES

Experience from a nationwide taskforce for promoting quality assurance and competence in clinical microbiology laboratories, guidelines on blood culture.

CONTENT

Experience in blood culture accreditation according to International standard ISO 15189 standards is provided in this review, with a particular focus on critical points that are specific to blood culture (e.g. excluding strain identification or antimicrobial susceptibility testing). Blood culture test method verification is based on risk analysis, and evaluation of the test method's performance is based on the literature review and suppliers' data. In addition, blood culture performance relies largely on the quality of its pre-analytical phase, and the test method should be monitored based on key performance indicators such as the volume of blood cultured, the contamination rate and time to transportation. Other critical key indicators include the rate of false-positive signals, the rate of positive blood cultures, the ecology associated with positive results, and the timely communication of the results to the ward during the post-analytical phase. Finally, a critical analysis of quality controls and of the tools needed to improve blood culture monitoring in the future is provided.

IMPLICATION

Appropriate quality assurance should focus on patient value rather than technical details to provide an appropriate clinical service.

Presepsin as a novel diagnostic biomarker for differentiating active pulmonary tuberculosis from bacterial community acquired pneumonia

BACKGROUND

The expression of presepsin in active pulmonary tuberculosis (APTB) is unknown. We observed the expression of presepsin in APTB, and to evaluate the value for discriminating between APTB and bacterial community acquired pneumonia (BCAP).

METHODS

Consecutive APTB patients who were accurately diagnosed by sputum culture and BCAP patients were enrolled from August 2013 to July 2015. Clinical data were collected, and plasma presepsin concentrations were tested. Receiver operating characteristic (ROC) curves were performed for diagnostic analysis.

RESULTS

In all, 133 healthy individuals, 103 APTB and 202 BCAP patients were enrolled. Presepsin concentrations in APTB group (218.0 [146.0, 368.0] pg/ml) were higher than those in the healthy control group (128.0 [101.5, 176.5] pg/ml, P<0.001), and lower than the concentrations measured in the BCAP group (532.0 [364.0, 852.3] pg/ml, P<0.001). Simple APTB and miliary tuberculosis patients showed no significant differences in presepsin concentrations. Compared with both Gram-positive and negative bacteria, Mycobacterium tuberculosis caused a limited increase of presepsin. With the cut-off value set at 401pg/ml, presepsin demonstrated high positive predictive value, allowing initial discriminating between APTB and BCAP. Presepsin combined with CURB-65 score could significantly improve the discrimination ability.

CONCLUSIONS

Presepsin concentrations in APTB patients were slightly increased, and may be helpful for initial discrimination between APTB and BCAP.