Concordance of endotoxemia with gram-negative bacteremia. A meta-analysis using receiver operating characteristic curves

Visualizing and diagnosing spillover within randomized concurrent controlled trials through the application of diagnostic test assessment methods

BACKGROUND

Spillover of effect, whether positive or negative, from intervention to control group patients invalidates the Stable Unit Treatment Variable Assumption (SUTVA). SUTVA is critical to valid causal inference from randomized concurrent controlled trials (RCCT). Spillover of infection prevention is an important population level effect mediating herd immunity. This herd effect, being additional to any individual level effect, is subsumed within the overall effect size (ES) estimate derived by contrast-based techniques from RCCT's. This herd effect would manifest only as increased dispersion among the control group infection incidence rates above background.

METHODS AND RESULTS

The objective here is to explore aspects of spillover and how this might be visualized and diagnosed. I use, for illustration, data from 190 RCCT's abstracted in 13 Cochrane reviews of various antimicrobial versus non-antimicrobial based interventions to prevent pneumonia in ICU patients. Spillover has long been postulated in this context. Arm-based techniques enable three approaches to identify increased dispersion, not available from contrast-based techniques, which enable the diagnosis of spillover within antimicrobial versus non-antimicrobial based infection prevention RCCT's. These three approaches are benchmarking the pneumonia incidence rates versus a clinically relevant range, comparing the dispersion in pneumonia incidence among the control versus the intervention groups and thirdly, visualizing the incidence dispersion within summary receiver operator characteristic (SROC) plots. By these criteria there is harmful spillover effects to concurrent control group patients.

CONCLUSIONS

Arm-based versus contrast-based techniques lead to contrary inferences from the aggregated RCCT's of antimicrobial based interventions despite similar summary ES estimates. Moreover, the inferred relationship between underlying control group risk and ES is 'flipped'.

Establishing the safety of selective digestive decontamination within the ICU population: a bridge too far?

BACKGROUND

Infection prevention interventions within the intensive care unit (ICU) setting, whether studied within quality improvement projects or cluster randomized trials (CRT), are seen as low risk and grounded in an ethical imperative. Selective digestive decontamination (SDD) appears highly effective at preventing ICU infections within randomized concurrent control trials (RCCTs) prompting mega-CRTs with mortality as the primary endpoint.

FINDINGS

Surprisingly, the summary results of RCCTs versus CRTs differ strikingly, being respectively, a 15-percentage-point versus a zero-percentage-point ICU mortality difference between control versus SDD intervention groups. Multiple other discrepancies are equally puzzling and contrary to both prior expectations and the experience within population-based studies of infection prevention interventions using vaccines. Could spillover effects from SDD conflate the RCCT control group event rate differences and represent population harm? Evidence that SDD is fundamentally safe to concurrent non-recipients in ICU populations is absent. A postulated CRT to realize this, the SDD Herd Effects Estimation Trial (SHEET), would require > 100 ICUs to achieve sufficient statistical power to find a two-percentage-point mortality spillover effect. Moreover, as a potentially harmful population-based intervention, SHEET would pose novel and insurmountable ethical issues including who is the research subject; whether informed consent is required and from whom; whether there is equipoise; the benefit versus the risk; considerations of vulnerable groups; and who should be the gatekeeper?

CONCLUSION

The basis for the mortality difference between control and intervention groups of SDD studies remains unclear. Several paradoxical results are consistent with a spillover effect that would conflate the inference of benefit originating from RCCTs. Moreover, this spillover effect would constitute to herd peril.

Structural equation modeling the "control of gut overgrowth" in the prevention of ICU-acquired Gram-negative infection

BACKGROUND

Conceptually, the "control of gut overgrowth" (COGO) is key in mediating prevention against infection with Gram-negative bacilli by topical antibiotic prophylaxis, a common constituent of selective digestive decontamination (SDD) regimens. However, the relative importance of the other SDD components, enteral and protocolized parenteral antibiotic prophylaxis, versus other methods of infection prevention and versus other contextual exposures cannot be resolved within individual studies.

METHODS

Seven candidate generalized structural equation models founded on COGO concepts were confronted with Pseudomonas and Acinetobacter bacteremia as well as ventilator-associated pneumonia data derived from > 200 infection prevention studies. The following group-level exposures were included in the models: use and mode of antibiotic prophylaxis, anti-septic and non-decontamination methods of infection prevention; proportion receiving mechanical ventilation; trauma ICU; mean length of ICU stay; and concurrency versus non-concurrency of topical antibiotic prophylaxis study control groups.

RESULTS

In modeling Pseudomonas and Acinetobacter gut overgrowth as latent variables, anti-septic interventions had the strongest negative effect against Pseudomonas gut overgrowth but no intervention was significantly negative against Acinetobacter gut overgrowth. Strikingly, protocolized parenteral antibiotic prophylaxis and concurrency each have positive effects in the model, enteral antibiotic prophylaxis is neutral, and Acinetobacter bacteremia incidences are high within topical antibiotic prophylaxis studies, moreso with protocolized parenteral antibiotic prophylaxis exposure. Paradoxically, topical antibiotic prophylaxis (moreso with protocolized parenteral antibiotic prophylaxis) appears to provide the strongest summary prevention effects against overall bacteremia and overall VAP.

CONCLUSIONS

Structural equation modeling of published Gram-negative bacillus infection data enables a test of the COGO concept. Paradoxically, Acinetobacter and Pseudomonas bacteremia incidences are unusually high among studies of topical antibiotic prophylaxis.

Invited review: Detoxifying endotoxin: time, place and person

Animals that cannot sense endotoxin may die if they are infected by Gram-negative bacteria. Animals that sense endotoxin and respond too vigorously may also die, victims of their own inflammatory reactions. The outcome of Gram-negative bacterial infection is thus determined not only by an individual's ability to sense endotoxin and respond to its presence, but also by numerous phenomena that inactivate endotoxin and/or prevent harmful reactions to it. Endotoxin sensing requires the MD-2/TLR4 recognition complex and occurs principally in local tissues and the liver. This review highlights the known detoxification mechanisms, which include: (i) proteins that facilitate LPS sequestration by plasma lipoproteins, prevent interactions between the bioactive lipid A moiety and MD-2/TLR4, or promote cellular uptake via non-signaling pathway(s); (ii) enzymes that deacylate or dephosphorylate lipid A; (iii) mechanisms that remove LPS and Gram-negative bacteria from the bloodstream; and (iv) neuroendocrine adaptations that modulate LPS-induced mediator production or neutralize pro-inflammatory molecules in the circulation. In general, the mechanisms for sensing and detoxifying endotoxin seem to be compartmentalized (local versus systemic), dynamic, and variable between individuals. They may have evolved to confine infection and inflammation to extravascular sites of infection while preventing harmful systemic reactions. Integration of endotoxin sensing and detoxification is essential for successful host defense.

The clinical relevance of endotoxin in human sepsis: a critical analysis

Correlations between circulating endotoxin levels from the blood of septic patients with clinical outcome measures have proven to be rather difficult. The clinical impact of endotoxin in septic patients depends upon the kinetics of LPS release, the concentration of LPS binding proteins, the cellular responsiveness to endotoxin, and numerous other immunogenetic, microbiological and physiological variables. Failure to account for these variables adequately has limited the clinical utility of endotoxin measurement. Several clinical studies have associated high levels of LPS with an excess risk of morbidity and mortality in sepsis that is largely independent of the nature of the micro-organism responsible for the septic episode.

Endotoxemia and Gram-negative bacteremia as predictors of outcome in sepsis: a meta-analysis using ROC curves

The prognostic value of the detection of endotoxin in plasma of patients with suspected Gram-negative sepsis is unclear despite numerous studies. Sixteen studies have reported endotoxemia detection and blood culture results with outcome data for 1937 patients with suspected Gram-negative sepsis. Comparisons of the fatality rates for the subgroups of patients with, respectively, either endotoxemia (group 3), or Gram-negative bacteremia (group 2), or both detected (group 1), versus the fatality rate for the subgroup of patients with these factors absent (group 4), were made based on summary odds ratios (and 95% CI). Summary odds ratios for the comparison of group 2 versus group 4 and group 3 versus group 4 were non-significant (2.2; 0.9-5.8 and 2.0; 0.8-4.8, respectively). The summary odds ratio was significant for group 1 versus group 4 (3.6; 2.1-6.3), and more so among studies in which non-Enterobacteriaceae were common among the Gram-negative blood culture isolates (4.8; 2.1-10.7). Endotoxemia, like Gram-negative bacteremia, is a weak prognostic indicator. Endotoxemia appears to have most prognostic significance when it is detected in the co-presence of Gram-negative bacteremia. The differing interpretations among published studies could be attributed to the variability in types of Gram-negative bacteremia observed.

Towards clinical applications of anti-endotoxin antibodies; a re-appraisal of the disconnect

Endotoxin is a potent mediator of a broad range of patho-physiological effects in humans. It is present in all Gram negative (GN) bacteria. It would be expected that anti-endotoxin therapies, whether antibody based or not, would have an important adjuvant therapeutic role along with antibiotics and other supportive therapies for GN infections. Indeed there is an extensive literature relating to both pre-clinical and clinical studies of anti-endotoxin antibodies. However, the extent of disconnect between the generally successful pre-clinical studies versus the failures of the numerous large clinical trials of antibody based and other anti-endotoxin therapies is under-appreciated and unexplained. Seeking a reconciliation of this disconnect is not an abstract academic question as clinical trials of interventions to reduce levels of endotoxemia levels are ongoing. The aim of this review is to examine new insights into the complex relationship between endotoxemia and sepsis in an attempt to bridge this disconnect. Several new factors to consider in this reappraisal include the frequency and types of GN bacteremia and the underlying mortality risk in the various study populations. For a range of reasons, endotoxemia can no longer be considered as a single entity. There are old clinical trials which warrant a re-appraisal in light of these recent advances in the understanding of the structure-function relationship of endotoxin. Fundamentally however, the disconnect not only remains, it has enlarged.

ISG12 is a critical modulator of innate immune responses in murine models of sepsis

Sepsis is still a major burden for our society with high incidence of morbidity and mortality each year. Molecular mechanisms underlying the systemic inflammatory response syndrome (SIRS) associated with sepsis are still ill defined and most therapies developed to target the acute inflammatory component of the disease are insufficient. Recently the role of nuclear receptors (NRs) became a major topic of interest in transcriptional regulation of inflammatory processes. Nuclear receptors, such as the peroxisome proliferators-activated receptors (PPARs), have been demonstrated to exert anti-inflammatory properties by interfering with the NFκB pathway. We identified the nuclear envelope protein, interferon stimulated gene 12 (ISG12), which directly interacts with NRs. ISG12 is a co-factor stimulating nuclear export of NRs, thereby reducing the anti-inflammatory potential of NRs such as NR4A1. To examine the role of ISG12 in acute inflammatory processes we used recently generated ISG12 deficient mice. We can clearly demonstrate that lack of ISG12 prolongs survival in experimental sepsis and endotoxemia. Furthermore we can show that several acute inflammatory parameters, such as systemic IL6 cytokine levels, are downregulated in septic ISG12-/- animals. Consistently, similar results were obtained in in vitro experiments in peritoneal macrophages derived from ISG12 deficient mice. In contrast, mice deficient for the nuclear receptor NR4A1 exhibited an exacerbated innate immune response, and showed a significantly higher mortality after lethal endotoxemic challenge. This dramatic phenotype could be restored in ISG12/NR4A1 double deficient mice. We conclude from our data in vitro and in vivo that ISG12 is a novel modulator of innate immune responses regulating anti-inflammatory nuclear receptors such as NR4A1.

Variation of a test's sensitivity and specificity with disease prevalence

BACKGROUND

Anecdotal evidence suggests that the sensitivity and specificity of a diagnostic test may vary with disease prevalence. Our objective was to investigate the associations between disease prevalence and test sensitivity and specificity using studies of diagnostic accuracy.

METHODS

We used data from 23 meta-analyses, each of which included 10-39 studies (416 total). The median prevalence per review ranged from 1% to 77%. We evaluated the effects of prevalence on sensitivity and specificity using a bivariate random-effects model for each meta-analysis, with prevalence as a covariate. We estimated the overall effect of prevalence by pooling the effects using the inverse variance method.

RESULTS

Within a given review, a change in prevalence from the lowest to highest value resulted in a corresponding change in sensitivity or specificity from 0 to 40 percentage points. This effect was statistically significant (p < 0.05) for either sensitivity or specificity in 8 meta-analyses (35%). Overall, specificity tended to be lower with higher disease prevalence; there was no such systematic effect for sensitivity.

INTERPRETATION

The sensitivity and specificity of a test often vary with disease prevalence; this effect is likely to be the result of mechanisms, such as patient spectrum, that affect prevalence, sensitivity and specificity. Because it may be difficult to identify such mechanisms, clinicians should use prevalence as a guide when selecting studies that most closely match their situation.

Meta-analysis of clinical studies of diagnostic tests: developments in how the receiver operating characteristic "works"

Meta-analytic summaries are needed on clinical studies of diagnostic tests. Meta-analyses on clinical studies of diagnostic tests commonly use the receiver operating characteristic method, which differs conceptually and computationally from the more widely known meta-analytic methods applicable in other contexts, such as in studies of randomized controlled trials. Important conceptual differences for clinical studies of diagnostic tests versus randomized controlled trials are that the study subpopulations are not defined by random allocation and the test threshold typically varies across studies to accommodate "rule in" versus "rule out" testing strategies. The receiver operating characteristic method has evolved substantially in the past decade, and the most recent approaches use multilevel regression methods that require iterative computational solutions to estimate the influence of the study-level variables. Using current methodology, a meta-analysis on clinical studies of diagnostic tests can address questions relevant to the clinical application of a diagnostic test that cannot be answered at the level of the individual study.

Does gram-negative bacteraemia occur without endotoxaemia? A meta-analysis using hierarchical summary ROC curves

The limulus assay for endotoxin has been studied as a method for the rapid identification of gram-negative (GN) bacteraemia. The chromogenic (C-limulus) version is 100-fold more sensitive to an internal endotoxin standard than the earlier gelation version (G-limulus). The objective of this analysis is to compare the concordance between GN bacteraemia and endotoxaemia as determined in clinical studies using either version of the limulus assay. The summary results for the diagnostic odds ratio (DOR), sensitivity and specificity were derived using a hierarchical summary receiver operating characteristic (HSROC) method of meta-analysis. Fifty-eight studies (25 G-limulus and 33 C-limulus) were included. Surprisingly, the mean DOR (4.9; 3-7.9 versus 10.7; 5.2-21.8) was inferior for studies using the C-limulus versus the original G-limulus version of the assay. Moreover, among studies limited to sepsis syndrome patients, the mean DOR remains poor at 4.2 (1.8-9.5). The proportion of GN bacteraemic patients for whom endotoxaemia is not detectable with either version of the limulus assay is >20% among the 58 studies overall, but >30% after the exclusion of studies with <25 patients and >20% among studies of patients with sepsis syndrome. These findings help to reconcile seemingly disparate study results.

Diagnosis of endotoxemia with gram-negative bacteremia is bacterial species dependent: a meta-analysis of clinical studies

Endotoxemia is undetectable for up to 60% of cases of bacteremia caused by gram-negative (GN) species, a discordance attributed to the limitations of the Limulus assay for endotoxemia. The lipid A structure of the endotoxin molecule is critical for the sensing of GN bacteria by the host immune system although not so for sensing by the Limulus assay. The lipid A structure of commensal Enterobacteriaceae is hexa-acyl, whereas non-Enterobacteriaceae have a broader range of structures. By using a previously published classification of lipid A structures (R. S. Munford, Infect. Immun. 76:454-465, 2008), the association of endotoxemia with bacteremia caused by GN organisms is reexamined for 580 GN bacteremic patients from 46 studies. Endotoxemia was less commonly detected for cases of bacteremia caused by Salmonella enterica serovar Typhi (four studies; 15 of 55 cases of bacteremia [27%]) than for cases of bacteremia caused by Neisseria meningitidis (five studies; 69 of 84 cases [82%]) and Pseudomonas pseudomallei (one study; 38 of 41 cases [93%]) among studies restricted to those with specified cases of bacteremia caused by GN organisms. Among 23 unrestricted studies, endotoxemia was less commonly detected for cases of bacteremia with a commensal member of the Enterobacteriaceae (104 of 240 cases [43%]) than with non-Enterobacteriaceae (59 of 100 cases [59%]) (summary odds ratio, 0.53 [90% confidence interval, 0.33 to 0.85]). This finding is consistent across all the unrestricted studies, even including studies with seemingly contrary results for endotoxemia diagnosis among cases of bacteremia caused by GN bacteria overall. Surprisingly, with bacteremia caused by commensal Enterobacteriaceae, the diagnosis of endotoxemia appears to be unrelated to the Limulus assay sensitivity. Across these 45 studies, the association of endotoxemia with GN bacteremia is variable but consistent for different types of GN bacteremia.

Use of methodological search filters to identify diagnostic accuracy studies can lead to the omission of relevant studies

OBJECTIVE

To determine the usefulness of methodological filters in search strategies for diagnostic studies in systematic reviews.

STUDY DESIGN AND SETTING

We made an inventory of existing methodological search filters for diagnostic accuracy studies and applied them in PubMed to a reference set derived from 27 published systematic reviews in a broad range of clinical fields. Outcome measures were the fraction of not identified relevant studies and the reduction in the number of studies to read.

RESULTS

We tested 12 search filters. Of the studies included in the systematic reviews, 2%-28% did not pass the sensitive search filters, 4%-24% did not pass the accurate filters, and 39%-42% did not pass the specific filters. Decrease in number-needed-to-read when a search filter was used in a search strategy for a diagnostic systematic review varied from 0% to 77%.

CONCLUSION

The use of methodological filters to identify diagnostic accuracy studies can lead to omission of a considerable number of relevant studies that would otherwise be included. When preparing a systematic review, it may be preferable to avoid using methodological filters.

Evidence of bias and variation in diagnostic accuracy studies

BACKGROUND

Studies with methodologic shortcomings can overestimate the accuracy of a medical test. We sought to determine and compare the direction and magnitude of the effects of a number of potential sources of bias and variation in studies on estimates of diagnostic accuracy.

METHODS

We identified meta-analyses of the diagnostic accuracy of tests through an electronic search of the databases MEDLINE, EMBASE, DARE and MEDION (1999-2002). We included meta-analyses with at least 10 primary studies without preselection based on design features. Pairs of reviewers independently extracted study characteristics and original data from the primary studies. We used a multivariable meta-epidemiologic regression model to investigate the direction and strength of the association between 15 study features on estimates of diagnostic accuracy.

RESULTS

We selected 31 meta-analyses with 487 primary studies of test evaluations. Only 1 study had no design deficiencies. The quality of reporting was poor in most of the studies. We found significantly higher estimates of diagnostic accuracy in studies with nonconsecutive inclusion of patients (relative diagnostic odds ratio [RDOR] 1.5, 95% confidence interval [CI] 1.0-2.1) and retrospective data collection (RDOR 1.6, 95% CI 1.1-2.2). The estimates were highest in studies that had severe cases and healthy controls (RDOR 4.9, 95% CI 0.6-37.3). Studies that selected patients based on whether they had been referred for the index test, rather than on clinical symptoms, produced significantly lower estimates of diagnostic accuracy (RDOR 0.5, 95% CI 0.3-0.9). The variance between meta-analyses of the effect of design features was large to moderate for type of design (cohort v. case-control), the use of composite reference standards and the use of differential verification; the variance was close to zero for the other design features.

INTERPRETATION

Shortcomings in study design can affect estimates of diagnostic accuracy, but the magnitude of the effect may vary from one situation to another. Design features and clinical characteristics of patient groups should be carefully considered by researchers when designing new studies and by readers when appraising the results of such studies. Unfortunately, incomplete reporting hampers the evaluation of potential sources of bias in diagnostic accuracy studies.

Clinical laboratory differentiation of infectious versus non-infectious systemic inflammatory response syndrome

OBJECTIVE

To evaluate the accuracy of C-reactive protein (CRP), procalcitonin (PCT), neopterin, and endotoxin in the differential diagnosis of sepsis and non-infectious systemic inflammatory response syndrome (SIRS).

METHODS

A Medline database and references from identified articles were used to perform a literature search relating to the differential diagnosis of sepsis versus non-infectious SIRS.

RESULTS

CRP, PCT, and neopterin are released both in sepsis and in non-infectious inflammatory disease. CRP and PCT are equally effective, although not perfect, in differentiating between sepsis and non-infectious SIRS. However, CRP and PCT have different kinetics and profiles. The kinetics of CRP is slower than that of PCT, and CRP levels may not further increase during more severe stages of sepsis. On the contrary, PCT rises in proportion to the severity of sepsis and reaches its highest levels in septic shock. PCT tends to be higher in nonsurvivor than in survivor. Therefore, PCT demonstrated a closer correlation with the severity of sepsis and outcome than CRP. Unlike CRP and PCT, neopterin is increased in viral infection as well as bacterial infection, and neopterin is also a useful indicator of sepsis. Endotoxemia was detected in no more than half of patients with Gram-negative bacteremia, and Gram-negative bacteremia was detected in half of patients with endotoxemia.

CONCLUSIONS

The diagnostic capacity of PCT is superior to that of CRP due to the close correlation between PCT levels and the severity of sepsis and outcome. Neopterin is very useful in the diagnosis of viral infection. The endotoxin assay in combination with CRP, PCT, or neopterin may help as a diagnostic marker for Gram-negative bacterial infection.

Septic shock

Septic shock, the most severe complication of sepsis, is a deadly disease. In recent years, exciting advances have been made in the understanding of its pathophysiology and treatment. Pathogens, via their microbial-associated molecular patterns, trigger sequential intracellular events in immune cells, epithelium, endothelium, and the neuroendocrine system. Proinflammatory mediators that contribute to eradication of invading microorganisms are produced, and anti-inflammatory mediators control this response. The inflammatory response leads to damage to host tissue, and the anti-inflammatory response causes leucocyte reprogramming and changes in immune status. The time-window for interventions is short, and treatment must promptly control the source of infection and restore haemodynamic homoeostasis. Further research is needed to establish which fluids and vasopressors are best. Some patients with septic shock might benefit from drugs such as corticosteroids or activated protein C. Other therapeutic strategies are under investigation, including those that target late proinflammatory mediators, endothelium, or the neuroendocrine system.

[Pathophysiological role of endotoxins, a common denominator to various diseases]

A growing number of investigations point to endotoxin or lipopolysaccharide as a central player in many pathophysiological states and diseases. Endotoxins are one of the most toxic biological contaminants continuously shed by both dead and live Gram negative bacteria. Endotoxins induce the primitive form of defense called innate immunity. Endotoxins have been related to inflammatory reactions observed in patients suffering from respiratory distress syndrome, multiorgan failure and septic shock, hepatic diseases, or in subjects affected by graft versus host disease after allogeneic transplantation. As our understanding of the molecular mechanisms underlying pathologies progresses, more diseases involving endotoxins emerge. Although these illnesses are multifactorial, the objective of this article is to review some of the common and distinct processes involving endotoxins in various disease states.