Objective Sepsis Surveillance Using Electronic Clinical Data

Electronic surveillance and using administrative data to identify healthcare associated infections

PURPOSE OF REVIEW

Traditional surveillance of healthcare associated infections (HCAI) is time consuming and error-prone. We have analysed literature of the past year to look at new developments in this field. It is divided into three parts: new algorithms for electronic surveillance, the use of administrative data for surveillance of HCAI, and the definition of new endpoints of surveillance, in accordance with an automatic surveillance approach.

RECENT FINDINGS

Most studies investigating electronic surveillance of HCAI have concentrated on bloodstream infection or surgical site infection. However, the lack of important parameters in hospital databases can lead to misleading results. The accuracy of administrative coding data was poor at identifying HCAI. New endpoints should be defined for automatic detection, with the most crucial step being to win clinicians' acceptance.

SUMMARY

Electronic surveillance with conventional endpoints is a successful method when hospital information systems implemented key changes and enhancements. One requirement is the access to systems for hospital administration and clinical databases.Although the primary source of data for HCAI surveillance is not administrative coding data, these are important components of a hospital-wide programme of automated surveillance. The implementation of new endpoints for surveillance is an approach which needs to be discussed further.

Incidence and Trends of Sepsis in US Hospitals Using Clinical vs Claims Data, 2009-2014

IMPORTANCE

Estimates from claims-based analyses suggest that the incidence of sepsis is increasing and mortality rates from sepsis are decreasing. However, estimates from claims data may lack clinical fidelity and can be affected by changing diagnosis and coding practices over time.

OBJECTIVE

To estimate the US national incidence of sepsis and trends using detailed clinical data from the electronic health record (EHR) systems of diverse hospitals.

DESIGN, SETTING, AND POPULATION

Retrospective cohort study of adult patients admitted to 409 academic, community, and federal hospitals from 2009-2014.

EXPOSURES

Sepsis was identified using clinical indicators of presumed infection and concurrent acute organ dysfunction, adapting Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) criteria for objective and consistent EHR-based surveillance.

MAIN OUTCOMES AND MEASURES

Sepsis incidence, outcomes, and trends from 2009-2014 were calculated using regression models and compared with claims-based estimates using International Classification of Diseases, Ninth Revision, Clinical Modification codes for severe sepsis or septic shock. Case-finding criteria were validated against Sepsis-3 criteria using medical record reviews.

RESULTS

A total of 173 690 sepsis cases (mean age, 66.5 [SD, 15.5] y; 77 660 [42.4%] women) were identified using clinical criteria among 2 901 019 adults admitted to study hospitals in 2014 (6.0% incidence). Of these, 26 061 (15.0%) died in the hospital and 10 731 (6.2%) were discharged to hospice. From 2009-2014, sepsis incidence using clinical criteria was stable (+0.6% relative change/y [95% CI, -2.3% to 3.5%], P = .67) whereas incidence per claims increased (+10.3%/y [95% CI, 7.2% to 13.3%], P < .001). In-hospital mortality using clinical criteria declined (-3.3%/y [95% CI, -5.6% to -1.0%], P = .004), but there was no significant change in the combined outcome of death or discharge to hospice (-1.3%/y [95% CI, -3.2% to 0.6%], P = .19). In contrast, mortality using claims declined significantly (-7.0%/y [95% CI, -8.8% to -5.2%], P < .001), as did death or discharge to hospice (-4.5%/y [95% CI, -6.1% to -2.8%], P < .001). Clinical criteria were more sensitive in identifying sepsis than claims (69.7% [95% CI, 52.9% to 92.0%] vs 32.3% [95% CI, 24.4% to 43.0%], P < .001), with comparable positive predictive value (70.4% [95% CI, 64.0% to 76.8%] vs 75.2% [95% CI, 69.8% to 80.6%], P = .23).

CONCLUSIONS AND RELEVANCE

In clinical data from 409 hospitals, sepsis was present in 6% of adult hospitalizations, and in contrast to claims-based analyses, neither the incidence of sepsis nor the combined outcome of death or discharge to hospice changed significantly between 2009-2014. The findings also suggest that EHR-based clinical data provide more objective estimates than claims-based data for sepsis surveillance.

Quick Sequential [Sepsis-Related] Organ Failure Assessment (qSOFA) and St. John Sepsis Surveillance Agent to Detect Patients at Risk of Sepsis: An Observational Cohort Study

The 2016 Sepsis-3 guidelines included the Quick Sequential [Sepsis-related] Organ Failure Assessment (qSOFA) tool to identify patients at risk of sepsis. The objective was to compare the utility of qSOFA to the St. John Sepsis Surveillance Agent among patients with suspected infection. The primary outcomes were in-hospital mortality or admission to the intensive care unit. A multiple center observational cohort study design was used. The study population comprised 17 044 hospitalized patients between January and March 2016. For the primary analysis, receiver operator characteristic curves were constructed for patient outcomes using qSOFA and the St. John Sepsis Surveillance Agent, and the areas under the curve were compared against a baseline risk model. Time-to-event clinical process modeling also was applied. The St. John Sepsis Surveillance Agent, when compared to qSOFA, activated earlier and was more accurate in predicting patient outcomes; in this regard, qSOFA fell far behind on both objectives.

Sepsis surveillance from administrative data in the absence of a perfect verification

PURPOSE

Past studies of sepsis epidemiology did not address misclassification bias due to imperfect verification of sepsis detection methods to estimate the true prevalence.

METHODS

We examined 273,126 hospitalizations from 2008 to 2012 at a tertiary-care center to develop surveillance-aimed sepsis detection criteria, based on the presence of the sepsis-explicit International Classification of Diseases, Ninth Revision, Clinical Modification codes (995.92 or 785.52), blood culture orders, and antibiotics administration. We used Bayesian multinomial latent class models to estimate the true prevalence of sepsis, while adjusting for the imperfect sensitivity and specificity and the conditional dependence among the individual criteria.

RESULTS

The apparent annual prevalence of sepsis hospitalizations based on explicit International Classification of Diseases, Ninth Revision, Clinical Modification codes were 1.5%, 1.4%, 1.6%, 2.2%, and 2.5% for the years 2008 to 2012. Bayesian posterior estimates for the true prevalence of sepsis suggested that it remained stable from 2008, 19.2% (95% credible interval [CI]: 17.9%, 22.9%), to 2012, 17.8% (95% CI: 16.8%, 20.2%). The sensitivity of sepsis-explicit codes, however, increased from 7.6% (95% CI: 6.4%, 8.4%) in 2008 to 13.8% (95% CI: 12.2%, 14.9%) in 2012.

CONCLUSIONS

The true prevalence of sepsis remained high, but stable despite an increase in the sensitivity of sepsis-explicit codes in administrative data.

Estimating Ten-Year Trends in Septic Shock Incidence and Mortality in United States Academic Medical Centers Using Clinical Data

BACKGROUND

Reports that septic shock incidence is rising and mortality rates declining may be confounded by improving recognition of sepsis and changing coding practices. We compared trends in septic shock incidence and mortality in academic hospitals using clinical vs claims data.

METHODS

We identified all patients with concurrent blood cultures, antibiotics, and vasopressors for ≥ two consecutive days, and all patients with International Classification of Diseases, 9th edition (ICD-9) codes for septic shock, at 27 academic hospitals from 2005 to 2014. We compared annual incidence and mortality trends. We reviewed 967 records from three hospitals to estimate the accuracy of each method.

RESULTS

Of 6.5 million adult hospitalizations, 99,312 (1.5%) were flagged by clinical criteria, 82,350 (1.3%) by ICD-9 codes, and 44,651 (0.7%) by both. Sensitivity for clinical criteria was higher than claims (74.8% vs 48.3%; P < .01), whereas positive predictive value was comparable (83% vs 89%; P = .23). Septic shock incidence, based on clinical criteria, rose from 12.8 to 18.6 cases per 1,000 hospitalizations (average, 4.9% increase/y; 95% CI, 4.0%-5.9%), while mortality declined from 54.9% to 50.7% (average, 0.6% decline/y; 95% CI, 0.4%-0.8%). In contrast, septic shock incidence, based on ICD-9 codes, increased from 6.7 to 19.3 per 1,000 hospitalizations (19.8% increase/y; 95% CI, 16.6%-20.9%), while mortality decreased from 48.3% to 39.3% (1.2% decline/y; 95% CI, 0.9%-1.6%).

CONCLUSIONS

A clinical surveillance definition based on concurrent vasopressors, blood cultures, and antibiotics accurately identifies septic shock hospitalizations and suggests that the incidence of patients receiving treatment for septic shock has risen and mortality rates have fallen, but less dramatically than estimated on the basis of ICD-9 codes.

Diagnosing sepsis is subjective and highly variable: a survey of intensivists using case vignettes

BACKGROUND

Sepsis is the focus of national quality improvement programs and a recent public reporting measure from the Centers for Medicare and Medicaid Services. However, diagnosing sepsis requires interpreting nonspecific signs and can therefore be subjective. We sought to quantify interobserver variability in diagnosing sepsis.

METHODS

We distributed five case vignettes of patients with suspected or confirmed infection and organ dysfunction to a sample of practicing intensivists. Respondents classified cases as systemic inflammatory response syndrome, sepsis, severe sepsis, septic shock, or none of the above. Interobserver variability was calculated using Fleiss' κ for the five-level classification, and for answers dichotomized as severe sepsis/septic shock versus not-severe sepsis/septic shock and any sepsis category (sepsis, severe sepsis, or septic shock) versus not-sepsis.

RESULTS

Ninety-four physicians completed the survey. Most respondents (88%) identified as critical care specialists; other specialties included pulmonology (39%), anesthesia (19%), surgery (9%), and emergency medicine (9%). Respondents had been in practice for a median of 8 years, and 90% practiced at academic hospitals. Almost all respondents (83%) felt strongly or somewhat confident in their ability to apply the traditional consensus sepsis definitions. However, overall interrater agreement in sepsis diagnoses was poor (Fleiss' κ 0.29). When responses were dichotomized into severe sepsis/septic shock versus not-severe sepsis/septic shock or any sepsis category versus not-sepsis, agreement was still poor (Fleiss' κ 0.23 and 0.18, respectively). Seventeen percent of respondents classified one of the five cases as severe sepsis/septic shock, 27.7% rated two cases, 33.0% respondents rated three cases, 19.2% rated four cases, and 3.2% rated all five cases as severe sepsis/septic shock. Among respondents who felt strongly confident in their ability to use sepsis definitions (n = 45), agreement was no better (Fleiss' κ 0.28 for the five-category classification, and Fleiss' κ 0.21 for the dichotomized severe sepsis/septic shock classification). Cases were felt to be extremely or very realistic in 74% of responses; only 3% were deemed unrealistic.

CONCLUSIONS

Diagnosing sepsis is extremely subjective and variable. Objective criteria and standardized methodology are needed to enhance consistency and comparability in sepsis research, surveillance, benchmarking, and reporting.