The Relationship Between Acuity of Organ Failure and Predictive Validity of Sepsis-3 Criteria

The Impact of Common Variations in Sequential Organ Failure Assessment Score Calculation on Sepsis Measurement Using Sepsis-3 Criteria: A Retrospective Analysis Using Electronic Health Record Data

OBJECTIVES

To assess the impact of different methods of calculating Sequential Organ Failure Assessment (SOFA) scores using electronic health record data on the incidence, outcomes, agreement, and predictive validity of Sepsis-3 criteria.

DESIGN

Retrospective observational study.

SETTING

Five Massachusetts hospitals.

PATIENTS

Hospitalized adults, 2015 to 2022.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We defined sepsis as a suspected infection (culture obtained and antibiotic administered) with a concurrent increase in SOFA score by greater than or equal to 2 points (Sepsis-3 criteria). Our reference SOFA implementation strategy imputed normal values for missing data, used Pao2/Fio2 ratios for respiratory scores, and assumed normal baseline SOFA scores for community-onset sepsis. We then implemented SOFA scores using different missing data imputation strategies (averaging worst values from preceding and following days vs. carrying forward nonmissing values), imputing respiratory scores using Spo2/Fio2 ratios, and incorporating comorbidities and prehospital laboratory data into baseline SOFA scores. Among 1,064,459 hospitalizations, 297,512 (27.9%) had suspected infection and 141,052 (13.3%) had sepsis with an in-hospital mortality rate of 10.3% using the reference SOFA method. The percentage of patients missing SOFA components for at least 1 day in the infection window was highest for Pao2/Fio2 ratios (98.6%), followed by Spo2/Fio2 ratios (73.5%), bilirubin (68.5%), and Glasgow Coma Scale scores (57.2%). Different missing data imputation strategies yielded near-perfect agreement in identifying sepsis (kappa 0.99). However, using Spo2/Fio2 imputations yielded higher sepsis incidence (18.3%), lower mortality (8.1%), and slightly lower predictive validity for mortality (area under the receiver operating curves [AUROC] 0.76 vs. 0.78). For community-onset sepsis, incorporating comorbidities and historical laboratory data into baseline SOFA score estimates yielded lower sepsis incidence (6.9% vs. 11.6%), higher mortality (13.4% vs. 9.6%), and higher predictive validity (AUROC 0.79 vs. 0.75) relative to the reference SOFA implementation.

CONCLUSIONS

Common variations in calculating respiratory and baseline SOFA scores, but not in handling missing data, lead to substantial differences in observed incidence, mortality, agreement, and predictive validity of Sepsis-3 criteria.

Performance of bedside tools for predicting infection-related mortality and administrative data for sepsis surveillance: An observational cohort study

BACKGROUND

Measuring sepsis incidence and associated mortality at scale using administrative data is hampered by variation in diagnostic coding. This study aimed first to compare how well bedside severity scores predict 30-day mortality in hospitalised patients with infection, then to assess the ability of combinations of administrative data items to identify patients with sepsis.

METHODS

This retrospective case note review examined 958 adult hospital admissions between October 2015 and March 2016. Admissions with blood culture sampling were matched 1:1 to admissions without a blood culture. Case note review data were linked to discharge coding and mortality. For patients with infection the performance characteristics of Sequential Organ Failure Assessment (SOFA), National Early Warning System (NEWS), quick SOFA (qSOFA), and Systemic Inflammatory Response Syndrome (SIRS) were calculated for predicting 30-day mortality. Next, the performance characteristics of administrative data (blood cultures and discharge codes) for identifying patients with sepsis, defined as SOFA ≥2 because of infection, were calculated.

RESULTS

Infection was documented in 630 (65.8%) admissions and 347 (55.1%) patients with infection had sepsis. NEWS (Area Under the Receiver Operating Characteristic, AUROC 0.78 95%CI 0.72-0.83) and SOFA (AUROC 0.77, 95%CI 0.72-0.83), performed similarly well for prediction of 30-day mortality. Having an infection and/or sepsis International Classification of Diseases, Tenth Revision (ICD-10) code (AUROC 0.68, 95%CI 0.64-0.71) performed as well in identifying patients with sepsis as having at least one of: an infection code; sepsis code, or; blood culture (AUROC 0.68, 95%CI 0.65-0.71), Sepsis codes (AUROC 0.53, 95%CI 0.49-0.57) and positive blood cultures (AUROC 0.52, 95%CI 0.49-0.56) performed least well.

CONCLUSIONS

SOFA and NEWS best predicted 30-day mortality in patients with infection. Sepsis ICD-10 codes lack sensitivity. For health systems without suitable electronic health records, blood culture sampling has potential utility as a clinical component of a proxy marker for sepsis surveillance.

Mortality and Sequential Organ Failure Assessment Score in Patients With Suspected Sepsis: The Impact of Acute and Preexisting Organ Failures and Infection Likelihood

The Sequential Organ Failure Assessment (SOFA) was chosen in the definition of sepsis due to superior validity in predicting mortality. However, few studies have assessed the contributions of acute versus chronic organ failures to SOFA for mortality prediction.

OBJECTIVES

The main objective in this study was to assess the relative importance of chronic and acute organ failures in mortality prediction in patients with suspected sepsis at hospital admission. We also evaluated how the presence of infection influenced the ability of SOFA to predict 30-day mortality.

DESIGN SETTING AND PARTICIPANTS

Single-center prospective cohort study including 1,313 adult patients with suspected sepsis in rapid response teams in the emergency department.

MAIN OUTCOMES AND MEASURES

The main outcome was 30-day mortality. We measured the maximum total SOFA score during admission (SOFATotal), whereas preexisting chronic organ failure SOFA (SOFAChronic) score was assessed by chart review, allowing calculation of the corresponding acute SOFA (SOFAAcute) score. Likelihood of infection was determined post hoc as "No infection" or "Infection."

RESULTS

SOFAAcute and SOFAChronic were both associated with 30-day mortality, adjusted for age and sex (adjusted odds ratios [AORs], 1.3; 95% CI, 1.3-14 and 1.3; 1.2-1.7), respectively. Presence of infection was associated with lower 30-day mortality (AOR, 0.4; 95% CI, 0.2-0.6), even when corrected for SOFA. In "No infection" patients, SOFAAcute was not associated with mortality (AOR, 1.1; 95% CI, 1.0-1.2), and in this subgroup, neither SOFAAcute greater than or equal to 2 (relative risk [RR], 1.1; 95% CI, 0.6-1.8) nor SOFATotal greater than or equal to 2 (RR, 3.6; 95% CI, 0.9-14.1) was associated with higher mortality.

CONCLUSIONS AND RELEVANCE

Chronic and acute organ failures were equally associated with 30-day mortality in suspected sepsis. A substantial part of the total SOFA score was due to chronic organ failure, calling for caution when using total SOFA in defining sepsis and as an outcome in intervention studies. SOFA's mortality prediction ability was highly dependent on actual presence of infection.

Overt and Occult Hypoxemia in Patients Hospitalized With COVID-19

Progressive hypoxemia is the predominant mode of deterioration in COVID-19. Among hypoxemia measures, the ratio of the Pao₂ to the Fio₂ (P/F ratio) has optimal construct validity but poor availability because it requires arterial blood sampling. Pulse oximetry reports oxygenation continuously (ratio of the Spo₂ to the Fio₂ [S/F ratio]), but it is affected by skin color and occult hypoxemia can occur in Black patients. Oxygen dissociation curves allow noninvasive estimation of P/F ratios (ePFRs) but remain unproven.

OBJECTIVES

Measure overt and occult hypoxemia using ePFR.

DESIGN SETTING AND PARTICIPANTS

We retrospectively studied COVID-19 hospital encounters (n = 5,319) at two academic centers (University of Virginia [UVA] and Emory University).

MAIN OUTCOMES AND MEASURES

We measured primary outcomes (death or ICU transfer within 24 hr), ePFR, conventional hypoxemia measures, baseline predictors (age, sex, race, comorbidity), and acute predictors (National Early Warning Score [NEWS] and Sequential Organ Failure Assessment [SOFA]). We updated predictors every 15 minutes. We assessed predictive validity using adjusted odds ratios (AORs) and area under the receiver operating characteristic curves (AUROCs). We quantified disparities (Black vs non-Black) in empirical cumulative distributions using the Kolmogorov-Smirnov (K-S) two-sample test.

RESULTS

Overt hypoxemia (low ePFR) predicted bad outcomes (AOR for a 100-point ePFR drop: 2.7 [UVA]; 1.7 [Emory]; p < 0.01) with better discrimination (AUROC: 0.76 [UVA]; 0.71 [Emory]) than NEWS (0.70 [both sites]) or SOFA (0.68 [UVA]; 0.65 [Emory]) and similar to S/F ratio (0.76 [UVA]; 0.70 [Emory]). We found racial differences consistent with occult hypoxemia. Black patients had better apparent oxygenation (K-S distance: 0.17 [both sites]; p < 0.01) but, for comparable ePFRs, worse outcomes than other patients (AOR: 2.2 [UVA]; 1.2 [Emory]; p < 0.01).

CONCLUSIONS AND RELEVANCE

The ePFR was a valid measure of overt hypoxemia. In COVID-19, it may outperform multi-organ dysfunction models. By accounting for biased oximetry as well as clinicians' real-time responses to it (supplemental oxygen adjustment), ePFRs may reveal racial disparities attributable to occult hypoxemia.

Overt and occult hypoxemia in patients hospitalized with novel coronavirus disease 2019

Background

Progressive hypoxemia is the predominant mode of deterioration in COVID-19. Among hypoxemia measures, the ratio of the partial pressure of arterial oxygen to the fraction of inspired oxygen (P/F ratio) has optimal construct validity but poor availability because it requires arterial blood sampling. Pulse oximetry reports oxygenation continuously, but occult hypoxemia can occur in Black patients because the technique is affected by skin color. Oxygen dissociation curves allow non-invasive estimation of P/F ratios (ePFR) but this approach remains unproven.

Research Question

Can ePFRs measure overt and occult hypoxemia?

Study Design and methods

We retrospectively studied COVID-19 hospital encounters (n=5319) at two academic centers (University of Virginia [UVA] and Emory University). We measured primary outcomes (death or ICU transfer within 24 hours), ePFR, conventional hypoxemia measures, baseline predictors (age, sex, race, comorbidity), and acute predictors (National Early Warning Score (NEWS) and Sepsis-3). We updated predictors every 15 minutes. We assessed predictive validity using adjusted odds ratios (AOR) and area under receiver operating characteristics curves (AUROC). We quantified disparities (Black vs non-Black) in empirical cumulative distributions using the Kolmogorov-Smirnov (K-S) two-sample test.

Results

Overt hypoxemia (low ePFR) predicted bad outcomes (AOR for a 100-point ePFR drop: 2.7 [UVA]; 1.7 [Emory]; p<0.01) with better discrimination (AUROC: 0.76 [UVA]; 0.71 [Emory]) than NEWS (AUROC: 0.70 [UVA]; 0.70 [Emory]) or Sepsis-3 (AUROC: 0.68 [UVA]; 0.65 [Emory]). We found racial differences consistent with occult hypoxemia. Black patients had better apparent oxygenation (K-S distance: 0.17 [both sites]; p<0.01) but, for comparable ePFRs, worse outcomes than other patients (AOR: 2.2 [UVA]; 1.2 [Emory], p<0.01).

Interpretation

The ePFR was a valid measure of overt hypoxemia. In COVID-19, it may outperform multi-organ dysfunction models like NEWS and Sepsis-3. By accounting for biased oximetry as well as clinicians’ real-time responses to it (supplemental oxygen adjustment), ePFRs may enable statistical modelling of racial disparities in outcomes attributable to occult hypoxemia.