Two rules for early prediction of bacteremia: testing in a university and a community hospital

Concurrent external validation of bloodstream infection probability models

OBJECTIVE

Accurately estimating the likelihood of bloodstream infection (BSI) can help clinicians make diagnostic and therapeutic decisions. Many multivariate models predicting BSI probability have been published. This study measured the performance of BSI probability models within the same patient sample.

METHODS

We retrieved validated BSI probability models included in a recently published systematic review that returned a patient-level BSI probability for adults. Model applicability, discrimination, and accuracy was measured in a simple random sample of 4485 admitted adults having blood cultures ordered in the emergency department or the initial 48 hours of hospitalization.

RESULTS

Ten models were included (publication years 1991-2015). Common methodological threats to model performance included overfitting and continuous variable categorization. Restrictive inclusion criteria caused seven models to apply to <15% of validation patients. Model discrimination was less than originally reported in derivation groups (median c-statistic 60%, range 48-69). The observed BSI risk frequently deviated from expected (median integrated calibration index 4.0%, range 0.8-12.4). Notable disagreement in expected BSI probabilities was seen between models (median (25th-75th percentile) relative difference between expected risks 68.0% (28.6-113.6%)).

DISCUSSION

In a large randomly selected external validation population, many published BSI probability models had restricted applicability, limited discrimination and calibration, and extensive inter-model disagreement. Direct comparison of model performance is hampered by dissimilarities between model-specific validation groups.

Risk of bacteremia in patients presenting with shaking chills and vomiting - a prospective cohort study

Chills and vomiting have traditionally been associated with severe bacterial infections and bacteremia. However, few modern studies have in a prospective way evaluated the association of these signs with bacteremia, which is the aim of this prospective, multicenter study. Patients presenting to the emergency department with at least one affected vital sign (increased respiratory rate, increased heart rate, altered mental status, decreased blood pressure or decreased oxygen saturation) were included. A total of 479 patients were prospectively enrolled. Blood cultures were obtained from 197 patients. Of the 32 patients with a positive blood culture 11 patients (34%) had experienced shaking chills compared with 23 (14%) of the 165 patients with a negative blood culture, P = 0.009. A logistic regression was fitted to show the estimated odds ratio (OR) for a positive blood culture according to shaking chills. In a univariate model shaking chills had an OR of 3.23 (95% CI 1.35–7.52) and in a multivariate model the OR was 5.9 (95% CI 2.05–17.17) for those without prior antibiotics adjusted for age, sex, and prior antibiotics. The presence of vomiting was also addressed, but neither a univariate nor a multivariate logistic regression showed any association between vomiting and bacteremia. In conclusion, among patients at the emergency department with at least one affected vital sign, shaking chills but not vomiting were associated with bacteremia.

Culture if spikes? Indications and yield of blood cultures in hospitalized medical patients

BACKGROUND

Although optimal utilization of blood cultures has been studied in populations, including emergency room and intensive care patients, less is known about the use of blood cultures in populations consisting exclusively of patients on a medical service.

OBJECTIVE

To identify the physician-selected indication and yield of blood cultures ordered after hospitalization to an acute medical service and to identify populations in which blood cultures may not be necessary.

DESIGN, SETTING, AND PATIENTS

A prospective cohort study was performed at a single Veterans Affairs Medical Center from October 1, 2014 through April 15, 2015. Participants included all hospitalized patients on a medical service for whom a blood culture was ordered.

MEASUREMENTS

The main outcomes were the rate of true positive blood cultures and the predictors of true positive cultures.

RESULTS

The true positive rate was 3.6% per order. The most common physician-selected indications were fever and leukocytosis, neither of which alone was highly predictive of true positive blood cultures. The only indication significantly associated with a true positive blood culture was "follow-up previous positive" (likelihood ratio [LR]+ 3.4, 95% confidence interval [CI]: 1.8-6.5). The only clinical predictors were a working diagnosis of bacteremia/endocarditis (LR+ 3.7, 95% CI: 2.5-5.7) and absence of antibiotic exposure within 72 hours of the culture (LR+ 2.4, 95% CI: 1.2-4.9).

CONCLUSIONS

The rate of true positive blood cultures among patients on a medical service was lower than previously studied. Using objective and easily obtainable clinical characteristics, including antibiotic exposure and working diagnosis, may improve the likelihood of true positive blood cultures. Journal of Hospital Medicine 2016;11:336-340. © 2016 Society of Hospital Medicine.

An external validation study of a clinical prediction rule for medical patients with suspected bacteraemia

OBJECTIVE

The objective of this study was to externally validate a clinical prediction rule (CPR)-the 'Shapiro criteria'-to predict bacteraemia in an acute medical unit (AMU).

METHODS

Prospectively collected data, retrospectively evaluated over 11 months in an AMU in the UK. From 4810 admissions, 635 patients (13%) had blood cultures (BCs) performed. The 100 cases of true bacteraemia were compared with a randomly selected sample of 100 control cases where BCs were sterile.

RESULTS

To predict bacteraemia (at a cut-off score of two points), the Shapiro criteria had a sensitivity of 97% (95% CIs 91% to 99%), specificity 37% (28% to 47%), positive likelihood ratio 1.54 (1.3 to 1.8) and a negative likelihood ratio of 0.08 (0.03 to 0.25). The area under the receiver operating curve was 0.80 (0.74 to 0.86), and the Hosmer-Lemeshow p value was 0.45.

CONCLUSIONS

A cut-off score of two points on the Shapiro criteria had high sensitivity to predict bacteraemia in a study of acute general medical admissions. Application of the rule in patients being considered for a BC could identify those at low risk of bacteraemia. Though the model demonstrated good discrimination, the lengthy number of variables (13) and difficulty automating the CPR may limit its use.

Predicting bacteraemia in validated models--a systematic review

Bacteraemia is associated with high mortality. Although many models for predicting bacteraemia have been developed, not all have been validated, and even when they were, the validation processes varied. We identified validated models that have been developed; asked whether they were successful in defining groups with a very low or high prevalence of bacteraemia; and whether they were used in clinical practice. Electronic databases were searched to identify studies that underwent validation on prediction of bacteraemia in adults. We included only studies that were able to define groups with low or high probabilities for bacteraemia (arbitrarily defined as below 3% or above 30%). Fifteen publications fulfilled inclusion criteria, including 59 276 patients. Eleven were prospective and four retrospective. Study populations and the parameters included in the different models were heterogeneous. Ten studies underwent internal validation; the model performed well in all of them. Twelve performed external validation. Of the latter, seven models were validated in a different hospital, using a new independent database. In five of these, the model performed well. After contacting authors, we found that none of the models was implemented in clinical practice. We conclude that heterogeneous studies have been conducted in different defined groups of patients with limited external validation. Significant savings to the system and the individual patient can be gained by refraining from performing blood cultures in groups of patients in which the probability of true bacteraemia is very low, while the probability of contamination is constant. Clinical trials of existing or new models should be done to examine whether models are helpful and safe in clinical use, preferably multicentre in order to secure utility and safety in diverse clinical settings.

Can we predict which patients with community-acquired pneumonia are likely to have positive blood cultures?

BACKGROUND

Blood cultures (BC) are commonly ordered during the initial assessment of patients with community-acquired pneumonia (CAP), yet their yield remains low. Selective use of BC would allow the opportunity to save healthcare resources and avoid patient discomfort. The study was to determine what demographic and clinical factors predict a greater likelihood of a positive blood culture result in patients diagnosed with CAP.

METHODS

A structured retrospective systematic chart audit was performed to compare relevant demographic and clinical details of patients admitted with CAP, in whom blood culture results were positive, with those of age, sex, and date-matched control patients in whom blood culture results were negative.

RESULTS

On univariate analysis, eight variables were associated with a positive BC result. After logistic regression analysis, however, the only variables statistically significantly associated with a positive BC were WBC less than 4.5 × 10(9)/L [likelihood ratio (LR): 7.75, 95% CI=2.89-30.39], creatinine >106 μmol/L (LR: 3.15, 95%CI=1.71-5.80), serum glucose<6.1 mmol/L (LR: 2.46, 95%CI=1.14-5.32), and temperature > 38 °C (LR: 2.25, 95% CI =1.21-4.20). A patient with all of these variables had a LR of having a positive BC of 135.53 (95% CI=25.28-726.8) compared to patients with none of these variables.

CONCLUSIONS

Certain clinical variables in patients with CAP admitted to hospitals do appear to be associated with a higher probability of a positive yield of BC, with combinations of these variables increasing this likelihood. We have identified a subgroup of CAP patients in whom blood cultures are more likely to be useful.

A new statistical approach to predict bacteremia using electronic medical records

BACKGROUND

Previous attempts to predict bacteremia have focused on selecting significant variables. However, these approaches have had limitations such as poor reproducibility in prediction accuracy and inconsistency in predictor selection. Here we propose a Bayesian approach to predict bacteremia based on the statistical distributions of clinical variables of previous patients, which has recently become possible through the adoption of electronic medical records.

METHODS

In a derivation cohort, Bayesian prediction models were derived and their discriminative performance was compared with previous models under varying combinations of predictors. Then the Bayesian models were prospectively tested in a validation cohort. According to Bayesian probabilities of bacteremia, patients in both cohorts were grouped into bacteremia risk groups.

RESULTS

Using the same prediction variables, the Bayesian predictions were more accurate than conventional rule-based predictions. Moreover, their better discriminative performance remained consistent despite variations in clinical variables. The receiver operating characteristic (ROC) area of the Bayesian model with 20 predictors was 0.70 ± 0.007 in the derivation cohort and 0.70 ± 0.018 in the validation cohort. The prevalence of bacteremia in groups I, II, and VI (grouped according to probability ratio) were 1.9%, 3.4%, and 20.0% in the derivation cohort, and 0.4%, 3.2%, and 18.4% in the validation cohort, respectively. The overall prevalence of bacteremia was 6.9% in both cohorts.

CONCLUSIONS

In the present study, the Bayesian prediction model showed stable performance in predicting bacteremia and identifying risk groups, as the previous models did. The clinical significance of the Bayesian approach is expected to be demonstrated through a multicenter trial.

Identifying and classifying bloodstream infections among hemodialysis patients

Bloodstream infections (BSIs) are a prominent clinical problem in patients undergoing hemodialysis. These infections appear to be more common among patients who have a central line as their dialysis access and can be associated with substantial morbidity and mortality. Accurately diagnosing BSIs clearly influences patient management, but is also an important part of an infection prevention program; particularly as facility BSI rates are becoming a recognized quality measure for which dialysis facilities might be held accountable. Blood cultures remain the gold standard for diagnosing BSIs and a number of practices can affect the sensitivity and specificity of this important laboratory test. Optimizing the collection of blood cultures can assist providers with interpretation of positive blood cultures and can help minimize the impact of false-positive and false-negative cultures. This review will describe differences between BSI definitions, examine the use of blood cultures to identify these infections including the use of recommended best practices to maximize culture yield, and highlight characteristics that can assist in the clinical interpretation of positive blood cultures.

Predictive model for bacteremia in adult patients with blood cultures performed at the emergency department: a preliminary report

BACKGROUND

Useful predictive models for identifying patients at high risk of bacteremia at the emergency department (ED) are lacking. This study attempted to provide useful predictive models for identifying patients at high risk of bacteremia at the ED.

METHODS

A prospective cohort study was conducted at the ED of a tertiary care hospital from October 1 to November 30, 2004. Patients aged 15 years or older, who had at least two sets of blood culture, were recruited. Data were analyzed on selected covariates, including demographic characteristics, predisposing conditions, clinical presentations, laboratory tests, and presumptive diagnosis, at the ED. An iterative procedure was used to build up a logistic model, which was then simplified into a coefficient-based scoring system.

RESULTS

A total of 558 patients with 84 episodes of true bacteremia were enrolled. Predictors of bacteremia and their assigned scores were as follows: fever greater than or equal to 38.3°C [odds ratio (OR), 2.64], 1 point; tachycardia greater than or equal to 120/min (OR, 2.521), 1 point; lymphopenia less than 0.5×10(3)/μL (OR, 3.356), 2 points; aspartate transaminase greater than 40IU/L (OR, 2.355), 1 point; C-reactive protein greater than 10mg/dL (OR, 2.226), 1 point; procalcitonin greater than 0.5 ng/mL (OR, 3.147), 2 points; and presumptive diagnosis of respiratory tract infection (OR, 0.236), -2 points. The area under the receiver operating characteristic curves of the original logistic model and the simplified scoring model using the aforementioned seven predictors and their assigned scores were 0.854 (95% confidence interval, 0.806-0.902) and 0.845 (95% confidence interval, 0.798-0.894), respectively.

CONCLUSION

This simplified scoring system could rapidly identify high-risk patients of bacteremia at the ED.

Factors associated with positive blood cultures in outpatients with suspected bacteremia

Blood cultures are routinely taken in outpatients with fever and suspected bacterial infections. However, in the majority of cases, they are not informative and of limited value for clinical decision making. The aim of this study was therefore to investigate factors associated with positive blood cultures in outpatients presenting to an outpatient clinic and emergency room. This was a case-control study of all outpatients with positive blood cultures from January 1, 2006 to October 31, 2007 and matched control patients with negative blood cultures in the same time period. Microbiology results and medical charts were reviewed to determine factors associated with positive blood cultures. The presence of a systemic inflammation response syndrome (SIRS) (OR 2.7, 95% Cl 1.0-7.2) and increased C-reactive protein (CRP) (OR 1.1 per 10 mg/l, 95% Cl 1.0-1.2) were the most powerful predictive values for the development of positive blood cultures. In positive cases serum albumin was lower (35 mg/l versus 39 mg/l) than in controls. SIRS, increasing CRP and low albumin were associated with positive blood cultures in outpatients. With simple clinical assessment and few laboratory tests indicative of infection, it is possible to define a group at higher risk for bacteremia in outpatients.

Staphylococcus aureus bacteremia among patients with health care-associated fever

BACKGROUND

Although Staphylococcus aureus bacteremia is a common, serious infection, accurately identifying febrile patients with this diagnosis at the time of initial evaluation is difficult. The purpose of this investigation was to define clinical characteristics present at the time of the initial recognition of fever that were associated with the presence of any bloodstream infection and, in particular, with S. aureus bacteremia.

METHODS

All patients > or =18 years of age with a new episode of health care-associated fever (temperature > or =38 degrees C) and at least one blood culture drawn were eligible for enrollment into this prospective multicenter cohort study. Multivariable analyses were conducted and internally validated scoring systems were developed to categorize the risk of bacteremia.

RESULTS

Of 1015 patients enrolled, 181 patients (17.8%) had clinically significant bacteremia, including 77 patients (7.6%) with S. aureus bacteremia. Clinical characteristics associated with S. aureus bacteremia were the presence of a hemodialysis graft or shunt (odds ratio [OR] 3.22; 95% confidence interval [CI], 1.85-5.61), chills (OR 2.38; 95% CI, 1.43-3.98), and a history of S. aureus infection (OR 2.68; 95% CI, 1.38-5.20). Peripheral vascular catheters were inversely associated with S. aureus bacteremia (OR 0.42; 95% CI, 0.26-0.69). Clinical characteristics associated with any bloodstream infection were central venous access, chills, history of S. aureus infection, and hemodialysis access.

CONCLUSIONS

Among patients with health care-associated fever, the presence of easily recognizable clinical characteristics at the time of obtaining the initial blood cultures can help to identify patients at increased risk for any bloodstream infection, in particular for S. aureus bacteremia.

Who needs a blood culture? A prospectively derived and validated prediction rule

The study objective was to derive and validate a clinical decision rule for obtaining blood cultures in Emergency Department (ED) patients with suspected infection. This was a prospective, observational cohort study of consecutive adult ED patients with blood cultures obtained. The study ran from February 1, 2000 through February 1, 2001. Patients were randomly assigned to derivation (2/3) or validation (1/3) sets. The outcome was "true bacteremia." Features of the history, co-morbid illness, physical examination, and laboratory testing were used to create a clinical decision rule. Among 3901 patients, 3730 (96%) were enrolled with 305 (8.2%) episodes of true bacteremia. A decision rule was created with "major criteria" defined as: temperature > 39.5 degrees C (103.0 degrees F), indwelling vascular catheter, or clinical suspicion of endocarditis. "Minor criteria" were: temperature 38.3-39.4 degrees C (101-102.9 degrees F), age > 65 years, chills, vomiting, hypotension (systolic blood pressure < 90 mm Hg), neutrophil% > 80, white blood cell count > 18 k, bands > 5%, platelets < 150 k, and creatinine > 2.0. A blood culture is indicated by the rule if at least one major criterion or two minor criteria are present. Otherwise, patients are classified as "low risk" and cultures may be omitted. Only 4 (0.6%) low-risk patients in the derivation set and 3 (0.9%) low-risk patients in the validation set had positive cultures. The sensitivity was 98% (95% confidence interval [CI] 96-100%) (derivation) and 97% (95% CI 94-100%) (validation). We developed and validated a promising clinical decision rule for predicting bacteremia in patients with suspected infection.

Updated review of blood culture contamination

Blood culture contamination represents an ongoing source of frustration for clinicians and microbiologists alike. Ambiguous culture results often lead to diagnostic uncertainty in clinical management and are associated with increased health care costs due to unnecessary treatment and testing. A variety of strategies have been investigated and employed to decrease contamination rates. In addition, numerous approaches to increase our ability to distinguish between clinically significant bacteremia and contamination have been explored. In recent years, there has been an increase in the application of computer-based tools to support infection control activities as well as provide clinical decision support related to the management of infectious diseases. Finally, new approaches for estimating bacteremia risk which have the potential to decrease unnecessary blood culture utilization have been developed and evaluated. In this review, we provide an overview of blood culture contamination and describe the potential utility of a variety of approaches to improve both detection and prevention. While it is clear that progress is being made, fundamental challenges remain.

Prediction of bacteremia using TREAT, a computerized decision-support system

BACKGROUND

Prediction of bloodstream infection at the time of sepsis onset allows one to make appropriate and economical management decisions.

METHODS

The TREAT computerized decision-support system uses a causal probabilistic network, which is locally calibrated, to predict cases of bacteremia. We assessed the system's performance in 2 independent cohorts that included patients with suspected sepsis. Both studies were conducted in Israel, Italy, and Germany. Data were collected prospectively and were entered into the TREAT system at the time that blood samples were obtained for culture. Discriminative power was assessed using a receiver-operating characteristics curve.

RESULTS

In the first cohort, 790 patients were included. The area under the receiver-operating characteristics curve for prediction of bacteremia using the TREAT system was 0.68 (95% confidence interval [CI], 0.63-0.73). We used TREAT's prediction values to draw thresholds defining a low-, intermediate-, and high-risk groups for bacteremia, in which 3 (2.4%) of 123, 62 (12.8%) of 483, and 55 (29.9%) of 184 patients were bacteremic, respectively. In the second cohort, 1724 patients were included. The area under the receiver-operating characteristics curve was 0.70 (95% CI, 0.67-0.73). The prevalence of bacteremia observed in the low-, intermediate-, and high-risk groups defined by the first cohort were 1.3% (4 of 300 patients), 13.2% (150 of 1139 patients), and 28.1% (80 of 285 patients), respectively. The low-risk groups in the 2 cohorts comprised 15%-17% of all patients. Performance was stable in the 3 sites.

CONCLUSIONS

Using variables available at the time that blood cultures were performed, the TREAT system successfully stratified patients on the basis of the risk for bacteremia. The system's predictions were stable in 3 locations. The TREAT system can define a low-risk group of inpatients with suspected sepsis for whom blood cultures may not be needed.

Bacteriemia adquirida en la comunidad: elaboración de un modelo de predicción clínica en pacientes ingresados en un servicio de medicina interna

BACKGROUND AND OBJECTIVE

Clinical suspicion of bacteremia lacks of sensitivity, specificity or predictive values enough to be clinically useful. The aim of this study was to develop a clinical prediction rule of bacteremia for patients hospitalized in an internal medicine department, with community-acquired symptoms, who had blood cultures obtained.

PATIENTS AND METHOD

A prospective study, including all patients who had blood cultures in the first 48 h after admission, was performed. A clinical prediction rule of bacteremia was derived from a random sample of two thirds of the patients (derivation cohort) and validated in the remaining (validation cohort). After bivariate analysis, significant variables were included in a stepwise logistic regression analysis. In every patient out of the derivation and validation cohorts a score, derived from the addition of points for each of the significant predictor variables of logistic regression, was obtained; according to this score, 4 groups were formed, and the prevalence of bacteremia in each of them was calculated. Calibration and discrimination were evaluated by the Hosmer-Lemeshow test and area under the ROC curve respectively.

RESULTS

Four hundred and forty-eight blood cultures were obtained; the prevalence of bacteremia was 25.2%. Independent predictors of bacteremia in the bivariate analysis were urinary focus of infection, body temperature >= 38.3 degrees C, presence of band forms, ESR >= 70 mm, platelets < 200 * 103/microl, blood glucose >= 140 mg/dl, urea >= 50 mg/dl, C-reactive protein >= 12 mg/dl, and albumin < 3 g/dl. According to the score, in the derivation cohort, four groups with increasing prevalence of bacteremia were identified; in the group with a score between 0 and 3, the prevalence was 2.4%; between 4 and 5: 15.7%; between 6 and 7: 42.9%; and score >= 8: 65%. In the validation cohort, the prevalence was 4.1%, 22.6%, 29.3%, and 80%, respectively. The model showed good calibration (Hosmer-Lemeshow *2 = 4.91; p = 0.77). Area under the ROC curve was 0.81 (95% confidence interval, 0.76-0.86) in the derivation cohort, and 0.77 (95% confidence interval, 0.69-0.85) in validation cohort.

CONCLUSIONS

Our model, constructed with 9 variables and a simple additive point system, had good calibration and discrimination, which points at its usefulness to estimate the probability of bacteremia in patients admitted in an Internal Medicine department. Used in conjunction with clinical judgement, the model can be useful in the decision-making process, concerning blood cultures obtention, clinical monitoring, and empirical antimicrobial therapy. Before application, additional prospective validation in other settings is warranted.

Circulating inflammatory mediators in patients with fever: predicting bloodstream infection

The systemic host response to microbial infection involves clinical signs and symptoms of infection, including fever and elevated white blood cell (WBC) counts. In addition, inflammatory mediators are released, including activated complement product C3a, interleukin 6 (IL-6), and the acute-phase reactant secretory phospholipase A(2) (sPLA(2)). To compare the value of the latter with the former in predicting (the degree of) microbial infection at the bedside, we determined clinical variables and took blood samples daily for 3 consecutive days in 300 patients with a new fever (>38.0 degrees C rectally or >38.3 degrees C axillary). Microbiological culture results for 7 days after inclusion were collected. Patients were divided into clinical and microbial categories: those without and with a clinical focus of infection and those with negative cultures, with positive local cultures or specific stains for fungal (n = 13) or tuberculous infections (n = 1), and with positive blood cultures, including one patient with malaria parasitemia. The area under the curve (AUC) of the receiver operating characteristic (ROC) for prediction of positive cultures was 0.60 (P < 0.005) for peak temperature and 0.59 (P < 0.01) for peak WBC count, 0.60 (P < 0.005) for peak C3a, 0.63 (P < 0.001) for peak IL-6, and 0.61 (P < 0.001) for peak sPLA(2). The AUC under the ROC curve for prediction of positive blood cultures was 0.68 (P < 0.001) for peak temperature and 0.56 for peak WBC count (P < 0.05). The AUC for peak C3a was 0.69, that for peak IL-6 was 0.70, and that for sPLA(2) was 0.67 (for all, P < 0.001). The degree of microbial invasion is thus a major determinant of the clinical and inflammatory host response in patients with fever. Moreover, circulating inflammatory mediators such as C3a and IL-6 may help to predict positive blood cultures, together with clinical signs and symptoms of the host response to microbial infection, even before culture results are available. This may help in the designing of entry criteria for therapeutic intervention studies.