Mortality Measures to Profile Hospital Performance for Patients With Septic Shock

Accounting for clustering in automated variable selection using hospital data: a comparison of different LASSO approaches

BACKGROUND

Automated feature selection methods such as the Least Absolute Shrinkage and Selection Operator (LASSO) have recently gained importance in the prediction of quality-related outcomes as well as the risk-adjustment of quality indicators in healthcare. The methods that have been used so far, however, do not account for the fact that patient data are typically nested within hospitals.

METHODS

Therefore, we aimed to demonstrate how to account for the multilevel structure of hospital data with LASSO and compare the results of this procedure with a LASSO variant that ignores the multilevel structure of the data. We used three different data sets (from acute myocardial infarcation, COPD, and stroke patients) with two dependent variables (one numeric and one binary), on which different LASSO variants with and without consideration of the nested data structure were applied. Using a 20-fold sub-sampling procedure, we tested the predictive performance of the different LASSO variants and examined differences in variable importance.

RESULTS

For the metric dependent variable Duration Stay, we found that inserting hospitals led to better predictions, whereas for the binary variable Mortality, all methods performed equally well. However, in some instances, the variable importances differed greatly between the methods.

CONCLUSION

We showed that it is possible to take the multilevel structure of data into account in automated predictor selection and that this leads, at least partly, to better predictive performance. From the perspective of variable importance, including the multilevel structure is crucial to select predictors in an unbiased way under consideration of the structural differences between hospitals.

Intravenous to Oral Antibiotic Switch Therapy Among Patients Hospitalized With Community-Acquired Pneumonia

BACKGROUND

Community-acquired pneumonia (CAP) is a leading cause of hospital admissions and antimicrobial use. Clinical practice guidelines recommend switching from intravenous (IV) to oral antibiotics once patients are clinically stable.

METHODS

We conducted a retrospective cohort study of adults admitted with CAP and initially treated with IV antibiotics at 642 US hospitals from 2010 through 2015. Switching was defined as discontinuation of IV and initiation of oral antibiotics without interrupting therapy. Patients switched by hospital day 3 were considered early switchers. We compared length of stay (LOS), in-hospital 14-day mortality, late deterioration (intensive care unit [ICU] transfer), and hospital costs between early switchers and others, controlling for hospital characteristics, patient demographics, comorbidities, initial treatments, and predicted mortality.

RESULTS

Of 378 041 CAP patients, 21 784 (6%) were switched early, most frequently to fluoroquinolones. Patients switched early had fewer days on IV antibiotics, shorter duration of inpatient antibiotic treatment, shorter LOS, and lower hospitalization costs, but no significant excesses in 14-day in-hospital mortality or late ICU admission. Patients at a higher mortality risk were less likely to be switched. However, even in hospitals with relatively high switch rates, <15% of very low-risk patients were switched early.

CONCLUSIONS

Although early switching was not associated with worse outcomes and was associated with shorter LOS and fewer days on antibiotics, it occurred infrequently. Even in hospitals with high switch rates, <15% of very low-risk patients were switched early. Our findings suggest that many more patients could be switched early without compromising outcomes.

Development and validation of risk-adjusted quality indicators for the long-term outcome of acute sepsis care in German hospitals based on health claims data

Background

Methods for assessing long-term outcome quality of acute care for sepsis are lacking. We investigated a method for measuring long-term outcome quality based on health claims data in Germany.

Materials and methods

Analyses were based on data of the largest German health insurer, covering 32% of the population. Cases (aged 15 years and older) with ICD-10-codes for severe sepsis or septic shock according to sepsis-1-definitions hospitalized in 2014 were included. Short-term outcome was assessed by 90-day mortality; long-term outcome was assessed by a composite endpoint defined by 1-year mortality or increased dependency on chronic care. Risk factors were identified by logistic regressions with backward selection. Hierarchical generalized linear models were used to correct for clustering of cases in hospitals. Predictive validity of the models was assessed by internal validation using bootstrap-sampling. Risk-standardized mortality rates (RSMR) were calculated with and without reliability adjustment and their univariate and bivariate distributions were described.

Results

Among 35,552 included patients, 53.2% died within 90 days after admission; 39.8% of 90-day survivors died within the first year or had an increased dependency on chronic care. Both risk-models showed a sufficient predictive validity regarding discrimination [AUC = 0.748 (95% CI: 0.742; 0.752) for 90-day mortality; AUC = 0.675 (95% CI: 0.665; 0.685) for the 1-year composite outcome, respectively], calibration (Brier Score of 0.203 and 0.220; calibration slope of 1.094 and 0.978), and explained variance (R ² = 0.242 and R ² = 0.111). Because of a small case-volume per hospital, applying reliability adjustment to the RSMR led to a great decrease in variability across hospitals [from median (1st quartile, 3rd quartile) 54.2% (44.3%, 65.5%) to 53.2% (50.7%, 55.9%) for 90-day mortality; from 39.2% (27.8%, 51.1%) to 39.9% (39.5%, 40.4%) for the 1-year composite endpoint]. There was no substantial correlation between the two endpoints at hospital level (observed rates: ρ = 0, p = 0.99; RSMR: ρ = 0.017, p = 0.56; reliability-adjusted RSMR: ρ = 0.067; p = 0.026).

Conclusion

Quality assurance and epidemiological surveillance of sepsis care should include indicators of long-term mortality and morbidity. Claims-based risk-adjustment models for quality indicators of acute sepsis care showed satisfactory predictive validity. To increase reliability of measurement, data sources should cover the full population and hospitals need to improve ICD-10-coding of sepsis.

Early treatment with thiamine and mortality among patients with alcohol use disorder who are hospitalized for pneumonia

BACKGROUND

The paucity of research linking thiamine treatment with improved outcomes may be driving its underutilization among patients at risk for Wernicke encephalopathy.

OBJECTIVE

To assess relationships of thiamine usage to outcomes of patients hospitalized with alcohol use disorder and pneumonia. DESIGN, SETTING AND PARTICIPANTS: This is a retrospective cohort study of adult patients hospitalized with pneumonia who also have alcohol use disorder and were treated with benzodiazepines during the initial two hospital days, between 2010 and 2015 at hospitals participating in the Premier Healthcare Database.

EXPOSURE

Any thiamine treatment, and, among those treated, high-dose thiamine treatment, during the initial two hospital days.

MAIN OUTCOME AND MEASURES

Death on days 3-14 of hospitalization (primary); discharge home; transfer to intensive care unit; length of stay (LOS). We used propensity-weighted models to estimate treatment effects.

RESULTS

Among 36,732 patients from 625 hospitals, 26,520 (72.2%) patients received thiamine, with mortality of 6.5% and 8.1% among recipients and nonrecipients, respectively. With propensity score adjustment, thiamine was associated with reduced mortality (odds ratio [OR]: 0.80, 95% confidence interval [CI]: 0.75-0.85) and more frequent discharges to home (OR: 1.10, 95% CI: 1.06-1.14). Other outcomes were similar. Relative to low-dose thiamine, high-dose thiamine was not associated with mortality (adjusted OR: 0.99, 95% CI: 0.89-1.10), but LOS was longer (ratio of means: 1.06, 95% CI: 1.04-1.08), and discharges to home were less frequent (OR: 0.92, 95% CI: 0.87-0.97).

CONCLUSION

Thiamine is not reliably given to patients with pneumonia and alcohol use disorder receiving benzodiazepines. Improving thiamine administration may represent an opportunity to save lives in this high-risk group of inpatients.

Sepsis prediction, early detection, and identification using clinical text for machine learning: a systematic review

OBJECTIVE

To determine the effects of using unstructured clinical text in machine learning (ML) for prediction, early detection, and identification of sepsis.

MATERIALS AND METHODS

PubMed, Scopus, ACM DL, dblp, and IEEE Xplore databases were searched. Articles utilizing clinical text for ML or natural language processing (NLP) to detect, identify, recognize, diagnose, or predict the onset, development, progress, or prognosis of systemic inflammatory response syndrome, sepsis, severe sepsis, or septic shock were included. Sepsis definition, dataset, types of data, ML models, NLP techniques, and evaluation metrics were extracted.

RESULTS

The clinical text used in models include narrative notes written by nurses, physicians, and specialists in varying situations. This is often combined with common structured data such as demographics, vital signs, laboratory data, and medications. Area under the receiver operating characteristic curve (AUC) comparison of ML methods showed that utilizing both text and structured data predicts sepsis earlier and more accurately than structured data alone. No meta-analysis was performed because of incomparable measurements among the 9 included studies.

DISCUSSION

Studies focused on sepsis identification or early detection before onset; no studies used patient histories beyond the current episode of care to predict sepsis. Sepsis definition affects reporting methods, outcomes, and results. Many methods rely on continuous vital sign measurements in intensive care, making them not easily transferable to general ward units.

CONCLUSIONS

Approaches were heterogeneous, but studies showed that utilizing both unstructured text and structured data in ML can improve identification and early detection of sepsis.

Risk-standardized sepsis mortality map of the United States

Objective

Sepsis is the leading cause of in-hospital mortality in the United States (US). Quality improvement initiatives for improving sepsis care depend on accurate estimates of sepsis mortality. While hospital 30-day risk-standardized mortality rates have been published for patients hospitalized with acute myocardial infarction, heart failure, and pneumonia, risk-standardized mortality rates for sepsis have not been well characterized. We aimed to construct a sepsis risk-standardized mortality rate map for the United States, to illustrate disparities in sepsis care across the country.

Methods

This cross-sectional study included adults from the US Nationwide Inpatient Sample who were hospitalized with sepsis between 1 January 2010 and 30 December 2011. Hospital-level risk-standardized mortality rates were calculated using hierarchical logistic modelling, and were risk-adjusted with predicted mortality derived from (1) the Sepsis Risk Prediction Score, a logistic regression model, and (2) gradient-boosted decision trees, a supervised machine learning (ML) algorithm.

Results

Among 1,739,033 adults hospitalized with sepsis, 50% were female, and the median age was 71 years (interquartile range: 58–81). The national median risk-standardized mortality rate for sepsis was 18.4% (interquartile range: 17.0, 21.0) by the boosted tree model, which had better discrimination than the Sepsis Risk Prediction Score model (C-statistic 0.87 and 0.78, respectively). The highest risk-standardized mortality rates were found in Wyoming, North Dakota, and Mississippi, while the lowest were found in Arizona, Colorado, and Michigan.

Conclusions

Wide variation exists in sepsis risk-standardized mortality rates across states, representing opportunities for improvement in sepsis care. This represents the first map of state-level variation of risk-standardized mortality rates in sepsis.

Surveillance Strategies for Tracking Sepsis Incidence and Outcomes

Sepsis is a leading cause of death and the target of intense efforts to improve recognition, management and outcomes. Accurate sepsis surveillance is essential to properly interpreting the impact of quality improvement initiatives, making meaningful comparisons across hospitals and geographic regions, and guiding future research and resource investments. However, it is challenging to reliably track sepsis incidence and outcomes because sepsis is a heterogeneous clinical syndrome without a pathologic reference standard, allowing for subjectivity and broad discretion in assigning diagnoses. Most epidemiologic studies of sepsis to date have used hospital discharge codes and have suggested dramatic increases in sepsis incidence and decreases in mortality rates over time. However, diagnosis and coding practices vary widely between hospitals and are changing over time, complicating the interpretation of absolute rates and trends. Other surveillance approaches include death records, prospective clinical registries, retrospective medical record reviews, and analyses of the usual care arms of randomized controlled trials. Each of these strategies, however, has substantial limitations. Recently, the US Centers for Disease Control and Prevention released an "Adult Sepsis Event" definition that uses objective clinical indicators of infection and organ dysfunction that can be extracted from most hospitals' electronic health record systems. Emerging data suggest that electronic health record-based clinical surveillance, such as surveillance of Adult Sepsis Event, is accurate, can be applied uniformly across diverse hospitals, and generates more credible estimates of sepsis trends than administrative data. In this review, we discuss the advantages and limitations of different sepsis surveillance strategies and consider future directions.

U.S. hospital performance methodologies: a scoping review to identify opportunities for crossing the quality chasm

Background

Hospital performance quality assessments inform patients, providers, payers, and purchasers in making healthcare decisions. These assessments have been developed by government, private and non-profit organizations, and academic institutions. Given the number and variability in available assessments, a knowledge gap exists regarding what assessments are available and how each assessment measures quality to identify top performing hospitals. This study aims to: (a) comprehensively identify current hospital performance assessments, (b) compare quality measures from each methodology in the context of the Institute of Medicine’s (IOM) six domains of STEEEP (safety, timeliness, effectiveness, efficiency, equitable, and patient-centeredness), and (c) formulate policy recommendations that improve value-based, patient-centered care to address identified gaps.

Methods

A scoping review was conducted using a systematic search of MEDLINE and the grey literature along with handsearching to identify studies that provide assessments of US-based hospital performance whereby the study cohort examined a minimum of 250 hospitals in the last two years (2017–2019).

Results

From 3058 unique records screened, 19 hospital performance assessments met inclusion criteria. Methodologies were analyzed across each assessment and measures were mapped to STEEEP. While safety and effectiveness were commonly identified measures across assessments, efficiency, and patient-centeredness were less frequently represented. Equity measures were also limited to risk- and severity-adjustment methods to balance patient characteristics across populations, rather than stand-alone indicators to evaluate health disparities that may contribute to community-level inequities.

Conclusions

To further improve health and healthcare value-based decision-making, there remains a need for methodological transparency across assessments and the standardization of consensus-based measures that reflect the IOM’s quality framework. Additionally, a large opportunity exists to improve the assessment of health equity in the communities that hospitals serve.

Impact of Risk Adjustment Using Clinical vs Administrative Data on Hospital Sepsis Mortality Comparisons

Background

A reliable risk-adjusted sepsis outcome measure could complement current national process metrics by identifying outlier hospitals and catalyzing additional improvements in care. However, it is unclear whether integrating clinical data into risk adjustment models identifies similar high- and low-performing hospitals compared with administrative data alone, which are simpler to acquire and analyze.

Methods

We ranked 200 US hospitals by their Centers for Disease Control and Prevention Adult Sepsis Event (ASE) mortality rates and assessed how rankings changed after applying (1) an administrative risk adjustment model incorporating demographics, comorbidities, and codes for severe illness and (2) an integrated clinical and administrative model replacing severity-of-illness codes with laboratory results, vasopressors, and mechanical ventilation. We assessed agreement between hospitals’ risk-adjusted ASE mortality rates when ranked into quartiles using weighted kappa statistics (к).

Results

The cohort included 4 009 631 hospitalizations, of which 245 808 met ASE criteria. Risk-adjustment had a large effect on rankings: 22/50 hospitals (44%) in the worst quartile using crude mortality rates shifted into better quartiles after administrative risk adjustment, and a further 21/50 (42%) of hospitals in the worst quartile using administrative risk adjustment shifted to better quartiles after incorporating clinical data. Conversely, 14/50 (28%) hospitals in the best quartile using administrative risk adjustment shifted to worse quartiles with clinical data. Overall agreement between hospital quartile rankings when risk-adjusted using administrative vs clinical data was moderate (к = 0.55).

Conclusions

Incorporating clinical data into risk adjustment substantially changes rankings of hospitals’ sepsis mortality rates compared with using administrative data alone. Comprehensive risk adjustment using both administrative and clinical data is necessary before comparing hospitals by sepsis mortality rates.

Patient Outcomes and Cost-Effectiveness of a Sepsis Care Quality Improvement Program in a Health System

OBJECTIVES

Assess patient outcomes in patients with suspected infection and the cost-effectiveness of implementing a quality improvement program.

DESIGN, SETTING, AND PARTICIPANTS

We conducted an observational single-center study of 13,877 adults with suspected infection between March 1, 2014, and July 31, 2017. The 18-month period before and after the effective date for mandated reporting of the sepsis bundle was examined. The Sequential Organ Failure Assessment score and culture and antibiotic orders were used to identify patients meeting Sepsis-3 criteria from the electronic health record.

INTERVENTIONS

The following interventions were performed as follows: 1) multidisciplinary sepsis committee with sepsis coordinator and data abstractor; 2) education campaign; 3) electronic health record tools; and 4) a Modified Early Warning System.

MAIN OUTCOMES AND MEASURES

Primary health outcomes were in-hospital death and length of stay. The incremental cost-effectiveness ratio was calculated and the empirical 95% CI for the incremental cost-effectiveness ratio was estimated from 5,000 bootstrap samples.

RESULTS

In multivariable analysis, the odds ratio for in-hospital death in the post- versus pre-implementation periods was 0.70 (95% CI, 0.57-0.86) in those with suspected infection, and the hazard ratio for time to discharge was 1.25 (95% CI, 1.20-1.29). Similarly, a decrease in the odds for in-hospital death and an increase in the speed to discharge was observed for the subset that met Sepsis-3 criteria. The program was cost saving in patients with suspected infection (-$272,645.7; 95% CI, -$757,970.3 to -$79,667.7). Cost savings were also observed in the Sepsis-3 group.

CONCLUSIONS AND RELEVANCE

Our health system's program designed to adhere to the sepsis bundle metrics led to decreased mortality and length of stay in a cost-effective manner in a much larger catchment than just the cohort meeting the Centers for Medicare and Medicaid Services measures. Our single-center model of interventions may serve as a practice-based benchmark for hospitalized patients with suspected infection.

Variation in Identifying Sepsis and Organ Dysfunction Using Administrative Versus Electronic Clinical Data and Impact on Hospital Outcome Comparisons

OBJECTIVES

Administrative claims data are commonly used for sepsis surveillance, research, and quality improvement. However, variations in diagnosis, documentation, and coding practices for sepsis and organ dysfunction may confound efforts to estimate sepsis rates, compare outcomes, and perform risk adjustment. We evaluated hospital variation in the sensitivity of claims data relative to clinical data from electronic health records and its impact on outcome comparisons.

DESIGN, SETTING, AND PATIENTS

Retrospective cohort study of 4.3 million adult encounters at 193 U.S. hospitals in 2013-2014.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Sepsis was defined using electronic health record-derived clinical indicators of presumed infection (blood culture draws and antibiotic administrations) and concurrent organ dysfunction (vasopressors, mechanical ventilation, doubling in creatinine, doubling in bilirubin to ≥ 2.0 mg/dL, decrease in platelets to < 100 cells/µL, or lactate ≥ 2.0 mmol/L). We compared claims for sepsis prevalence and mortality rates between both methods. All estimates were reliability adjusted to account for random variation using hierarchical logistic regression modeling. The sensitivity of hospitals' claims data was low and variable: median 30% (range, 5-54%) for sepsis, 66% (range, 26-84%) for acute kidney injury, 39% (range, 16-60%) for thrombocytopenia, 36% (range, 29-44%) for hepatic injury, and 66% (range, 29-84%) for shock. Correlation between claims and clinical data was moderate for sepsis prevalence (Pearson coefficient, 0.64) and mortality (0.61). Among hospitals in the lowest sepsis mortality quartile by claims, 46% shifted to higher mortality quartiles using clinical data. Using implicit sepsis criteria based on infection and organ dysfunction codes also yielded major differences versus clinical data.

CONCLUSIONS

Variation in the accuracy of claims data for identifying sepsis and organ dysfunction limits their use for comparing hospitals' sepsis rates and outcomes. Using objective clinical data may facilitate more meaningful hospital comparisons.

Risk Adjustment for Sepsis Mortality to Facilitate Hospital Comparisons Using Centers for Disease Control and Prevention's Adult Sepsis Event Criteria and Routine Electronic Clinical Data

Variability in hospital-level sepsis mortality rates may be due to differences in case mix, quality of care, or diagnosis and coding practices. Centers for Disease Control and Prevention's Adult Sepsis Event definition could facilitate objective comparisons of sepsis mortality rates between hospitals but requires rigorous risk-adjustment tools. We developed risk-adjustment models for Adult Sepsis Events using administrative and electronic health record data.

DESIGN

Retrospective cohort study.

SETTING

One hundred thirty-six U.S. hospitals in Cerner HealthFacts (derivation dataset) and 137 HCA Healthcare hospitals (validation dataset).

PATIENTS

A total of 95,154 hospitalized adult patients (derivation) and 201,997 patients (validation) meeting Centers for Disease Control and Prevention Adult Sepsis Event criteria.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We created logistic regression models of increasing complexity using administrative and electronic health record data to predict in-hospital mortality. An administrative model using demographics, comorbidities, and coded markers of severity of illness at admission achieved an area under the receiver operating curve of 0.776 (95% CI, 0.770-0.783) in the Cerner cohort, with diminishing calibration at higher baseline risk deciles. An electronic health record-based model that integrated administrative data with laboratory results, vasopressors, and mechanical ventilation achieved an area under the receiver operating curve of 0.826 (95% CI, 0.820-0.831) in the derivation cohort and 0.827 (95% CI, 0.824-0.829) in the validation cohort, with better calibration than the administrative model. Adding vital signs and Glasgow Coma Score minimally improved performance.

CONCLUSIONS

Models incorporating electronic health record data accurately predict hospital mortality for patients with Adult Sepsis Events and outperform models using administrative data alone. Utilizing laboratory test results, vasopressors, and mechanical ventilation without vital signs may achieve a good balance between data collection needs and model performance, but electronic health record-based models must be attentive to potential variability in data quality and availability. With ongoing testing and refinement of these risk-adjustment models, Adult Sepsis Event surveillance may enable more meaningful comparisons of hospital sepsis outcomes and provide an important window into quality of care.

An administrative model for benchmarking hospitals on their 30-day sepsis mortality

Background

Given the increased attention to sepsis at the population level there is a need to assess hospital performance in the care of sepsis patients using widely-available administrative data. The goal of this study was to develop an administrative risk-adjustment model suitable for profiling hospitals on their 30-day mortality rates for patients with sepsis.

Methods

We conducted a retrospective cohort study using hospital discharge data from general acute care hospitals in Pennsylvania in 2012 and 2013. We identified adult patients with sepsis as determined by validated diagnosis and procedure codes. We developed an administrative risk-adjustment model in 2012 data. We then validated this model in two ways: by examining the stability of performance assessments over time between 2012 and 2013, and by examining the stability of performance assessments in 2012 after the addition of laboratory variables measured on day one of hospital admission.

Results

In 2012 there were 115,213 sepsis encounters in 152 hospitals. The overall unadjusted mortality rate was 18.5%. The final risk-adjustment model had good discrimination (C-statistic = 0.78) and calibration (slope and intercept of the calibration curve = 0.960 and 0.007, respectively). Based on this model, hospital-specific risk-standardized mortality rates ranged from 12.2 to 24.5%. Comparing performance assessments between years, correlation in risk-adjusted mortality rates was good (Pearson’s correlation = 0.53) and only 19.7% of hospitals changed by more than one quintile in performance rankings. Comparing performance assessments after the addition of laboratory variables, correlation in risk-adjusted mortality rates was excellent (Pearson’s correlation = 0.93) and only 2.6% of hospitals changed by more than one quintile in performance rankings.

Conclusions

A novel claims-based risk-adjustment model demonstrated wide variation in risk-standardized 30-day sepsis mortality rates across hospitals. Individual hospitals’ performance rankings were stable across years and after the addition of laboratory data. This model provides a robust way to rank hospitals on sepsis mortality while adjusting for patient risk.

Electronic supplementary material

The online version of this article (10.1186/s12913-019-4037-x) contains supplementary material, which is available to authorized users.