Comparison of Sepsis Definitions as Automated Criteria

Use of artificial intelligence in critical care: opportunities and obstacles

BACKGROUND

Perhaps nowhere else in the healthcare system than in the intensive care unit environment are the challenges to create useful models with direct time-critical clinical applications more relevant and the obstacles to achieving those goals more massive. Machine learning-based artificial intelligence (AI) techniques to define states and predict future events are commonplace activities of modern life. However, their penetration into acute care medicine has been slow, stuttering and uneven. Major obstacles to widespread effective application of AI approaches to the real-time care of the critically ill patient exist and need to be addressed.

MAIN BODY

Clinical decision support systems (CDSSs) in acute and critical care environments support clinicians, not replace them at the bedside. As will be discussed in this review, the reasons are many and include the immaturity of AI-based systems to have situational awareness, the fundamental bias in many large databases that do not reflect the target population of patient being treated making fairness an important issue to address and technical barriers to the timely access to valid data and its display in a fashion useful for clinical workflow. The inherent "black-box" nature of many predictive algorithms and CDSS makes trustworthiness and acceptance by the medical community difficult. Logistically, collating and curating in real-time multidimensional data streams of various sources needed to inform the algorithms and ultimately display relevant clinical decisions support format that adapt to individual patient responses and signatures represent the efferent limb of these systems and is often ignored during initial validation efforts. Similarly, legal and commercial barriers to the access to many existing clinical databases limit studies to address fairness and generalizability of predictive models and management tools.

CONCLUSIONS

AI-based CDSS are evolving and are here to stay. It is our obligation to be good shepherds of their use and further development.

Association of Time of Day with Delays in Antimicrobial Initiation among Ward Patients with Hospital-Onset Sepsis

Rationale: Although the mainstay of sepsis treatment is timely initiation of broad-spectrum antimicrobials, treatment delays are common, especially among patients who develop hospital-onset sepsis. The time of day has been associated with suboptimal clinical care in several contexts, but its association with treatment initiation among patients with hospital-onset sepsis is unknown. Objectives: Assess the association of time of day with antimicrobial initiation among ward patients with hospital-onset sepsis. Methods: This retrospective cohort study included ward patients who developed hospital-onset sepsis while admitted to five acute care hospitals in a single health system from July 2017 through December 2019. Hospital-onset sepsis was defined by the Centers for Disease Control and Prevention Adult Sepsis Event criteria. We estimated the association between the hour of day and antimicrobial initiation among patients with hospital-onset sepsis using a discrete-time time-to-event model, accounting for time elapsed from sepsis onset. In a secondary analysis, we fit a quantile regression model to estimate the association between the hour of day of sepsis onset and time to antimicrobial initiation. Results: Among 1,672 patients with hospital-onset sepsis, the probability of antimicrobial initiation at any given hour varied nearly fivefold throughout the day, ranging from 3.0% (95% confidence interval [CI], 1.8-4.1%) at 7 a.m. to 13.9% (95% CI, 11.3-16.5%) at 6 p.m., with nadirs at 7 a.m. and 7 p.m. and progressive decline throughout the night shift (13.4% [95% CI, 10.7-16.0%] at 9 p.m. to 3.2% [95% CI, 2.0-4.0] at 6 a.m.). The standardized predicted median time to antimicrobial initiation was 3.2 hours (interquartile range [IQR], 2.5-3.8 h) for sepsis onset during the day shift (7 a.m.-7 p.m.) and 12.9 hours (IQR, 10.9-14.9 h) during the night shift (7 p.m.-7 a.m.). Conclusions: The probability of antimicrobial initiation among patients with new hospital-onset sepsis declined at shift changes and overnight. Time to antimicrobial initiation for patients with sepsis onset overnight was four times longer than for patients with onset during the day. These findings indicate that time of day is associated with important care processes for ward patients with hospital-onset sepsis. Future work should validate these findings in other settings and elucidate underlying mechanisms to inform quality-enhancing interventions.

OpenSep: a generalizable open source pipeline for SOFA score calculation and Sepsis-3 classification

EHR-based sepsis research often uses heterogeneous definitions of sepsis leading to poor generalizability and difficulty in comparing studies to each other. We have developed OpenSep, an open-source pipeline for sepsis phenotyping according to the Sepsis-3 definition, as well as determination of time of sepsis onset and SOFA scores. The Minimal Sepsis Data Model was developed alongside the pipeline to enable the execution of the pipeline to diverse sources of electronic health record data. The pipeline's accuracy was validated by applying it to the MIMIC-IV version 1.0 data and comparing sepsis onset and SOFA scores to those produced by the pipeline developed by the curators of MIMIC. We demonstrated high reliability between both the sepsis onsets and SOFA scores, however the use of the Minimal Sepsis Data model developed for this work allows our pipeline to be applied to more broadly to data sources beyond MIMIC.

Multistate Modeling of Clinical Trajectories and Outcomes in the ICU: A Proof-of-Concept Evaluation of Acute Kidney Injury Among Critically Ill Patients With COVID-19

Multistate models yield high-fidelity analyses of the dynamic state transition and temporal dimensions of a clinical condition's natural history, offering superiority over aggregate modeling techniques for addressing these types of problems.

OBJECTIVES

To demonstrate the utility of these models in critical care, we examined acute kidney injury (AKI) development, progression, and outcomes in COVID-19 critical illness through multistate analyses.

DESIGN SETTING AND PARTICIPANTS

Retrospective cohort study at an urban tertiary-care academic hospital in the United States. All patients greater than or equal to 18 years in an ICU with COVID-19 in 2020, excluding patients with preexisting end-stage renal disease.

MAIN OUTCOMES AND MEASURES

Using electronic health record data, we determined AKI presence/stage in discrete 12-hour time windows and fit multistate models to determine longitudinal transitions and outcomes.

RESULTS

Of 367 encounters, 241 (66%) experienced AKI (maximal stages: 88 stage-1, 49 stage-2, 104 stage-3 AKI [51 received renal replacement therapy (RRT), 53 did not]). Patients receiving RRT overwhelmingly received invasive mechanical ventilation (IMV) (n = 60, 95%) compared with the AKI-without-RRT (n = 98, 53%) and no-AKI groups (n = 39, 32%; p < 0.001), with similar mortality patterns (RRT: n = 36, 57%; AKI: n = 74, 40%; non-AKI: n = 23, 19%; p < 0.001). After 24 hours in the ICU, almost half the cohort had AKI (44.9%; 95% CI, 41.6-48.2%). At 7 days after stage-1 AKI, 74.0% (63.6-84.4) were AKI-free or discharged. By contrast, fewer patients experiencing stage-3 AKI were recovered (30.0% [24.1-35.8%]) or discharged (7.9% [5.2-10.7%]) after 7 days. Early AKI occurred with similar frequency in patients receiving and not receiving IMV: after 24 hours in the ICU, 20.9% of patients (18.3-23.6%) had AKI and IMV, while 23.4% (20.6-26.2%) had AKI without IMV.

CONCLUSIONS AND RELEVANCE

In a multistate analysis of critically ill patients with COVID-19, AKI occurred early and heterogeneously in the course of critical illness. Multistate methods are useful and underused in ICU care delivery science as tools for understanding trajectories, prognoses, and resource needs.

Hospital trajectories and early predictors of clinical outcomes differ between SARS-CoV-2 and influenza pneumonia

BACKGROUND

A comparison of pneumonias due to SARS-CoV-2 and influenza, in terms of clinical course and predictors of outcomes, might inform prognosis and resource management. We aimed to compare clinical course and outcome predictors in SARS-CoV-2 and influenza pneumonia using multi-state modelling and supervised machine learning on clinical data among hospitalised patients.

METHODS

This multicenter retrospective cohort study of patients hospitalised with SARS-CoV-2 (March-December 2020) or influenza (Jan 2015-March 2020) pneumonia had the composite of hospital mortality and hospice discharge as the primary outcome. Multi-state models compared differences in oxygenation/ventilatory utilisation between pneumonias longitudinally throughout hospitalisation. Differences in predictors of outcome were modelled using supervised machine learning classifiers.

FINDINGS

Among 2,529 hospitalisations with SARS-CoV-2 and 2,256 with influenza pneumonia, the primary outcome occurred in 21% and 9%, respectively. Multi-state models differentiated oxygen requirement progression between viruses, with SARS-CoV-2 manifesting rapidly-escalating early hypoxemia. Highly contributory classifier variables for the primary outcome differed substantially between viruses.

INTERPRETATION

SARS-CoV-2 and influenza pneumonia differ in presentation, hospital course, and outcome predictors. These pathogen-specific differential responses in viral pneumonias suggest distinct management approaches should be investigated.

FUNDING

This project was supported by NIH/NCATS UL1 TR002345, NIH/NCATS KL2 TR002346 (PGL), the Doris Duke Charitable Foundation grant 2015215 (PGL), NIH/NHLBI R35 HL140026 (CSC), and a Big Ideas Award from the BJC HealthCare and Washington University School of Medicine Healthcare Innovation Lab and NIH/NIGMS R35 GM142992 (PS).

Investigating hypozincemia and validity of plasma zinc measurements in infected patients

Hypozincemia is a well-known phenomenon in patients with infection caused by the activation of the acute phase response (APR). Zn status is still based upon plasma Zn levels in venous blood samples. Recent trials have questioned the validity of this measurement in infected patients. The aim of this study was to assess plasma levels of Zn, albumin and Zinc-binding capacity in patients during and following infection. Furthermore, to assess if an assay for albumin-corrected Zn could potentially replace or add knowledge to existing tools for assessment of Zinc-status. A prospective clinical observational trial was conducted. Associations between P-Zn, -Albumin, -Albumin-corrected Zn and Zn binding capacity were analyzed. Analyzes were based upon two venous blood samples drawn during and following infection, respectively. Twenty-three patients admitted to a medical ward showing paraclinical signs of infection were included in the study. Significantly lower levels of Zn and albumin were found during infection compared with the levels post-infection. These findings corresponded to the changes found in Zn binding capacity. About 52% of patients were deemed Zn deficient by plasma Zn levels during infection but after applying the correction for P-Albumin, all patients were found to be within normal ranges of Zn levels. Furthermore, we found no statistically significant difference between albumin-corrected Zn during infection and P-Zn post-infection. The new assay was found to accurately estimate the 'true' Zn levels in infected patients. Based on our findings, we propose albumin-corrected P-Zn as a promising new tool, which may result in more precise diagnostics and treatment.

Machine learning to predict vasopressin responsiveness in patients with septic shock

STUDY OBJECTIVES

The objective of this study was to develop and externally validate a model to predict adjunctive vasopressin response in patients with septic shock being treated with norepinephrine for bedside use in the intensive care unit.

DESIGN

This was a retrospective analysis of two adult tertiary intensive care unit septic shock populations.

SETTING

Barnes-Jewish Hospital (BJH) from 2010 to 2017 and Beth Israel Deaconess Medical Center (BIDMC) from 2001 to 2012.

PATIENTS

Two septic shock populations (548 BJH patients and 464 BIDMC patients) that received vasopressin as second-line vasopressor.

INTERVENTION

Patients who were vasopressin responsive were compared with those who were nonresponsive. Vasopressin response was defined as survival with at least a 20% decrease in maximum daily norepinephrine requirements by one calendar day after vasopressin initiation, without a third-line vasopressor.

MEASUREMENTS

Two supervised machine learning models (gradient-boosting machine [XGBoost] and elastic net penalized logistic regression [EN]) were trained in 1000 bootstrap replications of the BJH data and externally validated in the BIDMC data to predict vasopressin responsiveness.

MAIN RESULTS

Vasopressin responsiveness was similar among each cohort (BJH 45% and BIDMC 39%). Mortality was lower for vasopressin responders compared with nonresponders in the BJH (51% vs. 73%) and BIDMC (45% vs. 83%) cohorts, respectively. Both models demonstrated modest discrimination in the training (XGBoost area under receiver operator curve [AUROC] 0.61 [95% confidence interval (CI) 0.61-0.61], EN 0.59 [95% CI 0.58-0.59]) and external validation (XGBoost 0.68 [95% CI 0.63-0.73], EN 0.64 [95% CI 0.59-0.69]) datasets.

CONCLUSION

Vasopressin nonresponsiveness is common and associated with increased mortality. The models' modest performances highlight the complexity of septic shock and indicate that more research will be required before clinical decision support tools can aid in anticipating patient-specific responsiveness to vasopressin.

Identification of Clinical Phenotypes in Septic Patients Presenting With Hypotension or Elevated Lactate

Introduction

Targeted therapies for sepsis have failed to show benefit due to high variability among subjects. We sought to demonstrate different phenotypes of septic shock based solely on clinical features and show that these relate to outcome.

Methods

A retrospective analysis was performed of a 1,023-subject cohort with early septic shock from the ProCESS trial. Twenty-three clinical variables at baseline were analyzed using hierarchical clustering, with consensus clustering used to identify and validate the ideal number of clusters in a derivation cohort of 642 subjects from 20 hospitals. Clusters were visualized using heatmaps over 0, 6, 24, and 72 h. Clinical outcomes were 14-day all-cause mortality and organ failure pattern. Cluster robustness was confirmed in a validation cohort of 381 subjects from 11 hospitals.

Results

Five phenotypes were identified, each with unique organ failure patterns that persisted in time. By enrollment criteria, all patients had shock. The two high-risk phenotypes were characterized by distinct multi-organ failure patterns and cytokine signatures, with the highest mortality group characterized most notably by liver dysfunction and coagulopathy while the other group exhibited primarily respiratory failure, neurologic dysfunction, and renal dysfunction. The moderate risk phenotype was that of respiratory failure, while low-risk phenotypes did not have a high degree of additional organ failure.

Conclusions

Sepsis phenotypes with distinct biochemical abnormalities may be identified by clinical characteristics alone and likely provide an opportunity for early clinical actionability and prognosis.

Sepsis Prediction for the General Ward Setting

Objective

To develop and evaluate a sepsis prediction model for the general ward setting and extend the evaluation through a novel pseudo-prospective trial design.

Design

Retrospective analysis of data extracted from electronic health records (EHR).

Setting

Single, tertiary-care academic medical center in St. Louis, MO, USA.

Patients

Adult, non-surgical inpatients admitted between January 1, 2012 and June 1, 2019.

Interventions

None.

Measurements and Main Results

Of the 70,034 included patient encounters, 3.1% were septic based on the Sepsis-3 criteria. Features were generated from the EHR data and were used to develop a machine learning model to predict sepsis 6-h ahead of onset. The best performing model had an Area Under the Receiver Operating Characteristic curve (AUROC or c-statistic) of 0.862 ± 0.011 and Area Under the Precision-Recall Curve (AUPRC) of 0.294 ± 0.021 compared to that of Logistic Regression (0.857 ± 0.008 and 0.256 ± 0.024) and NEWS 2 (0.699 ± 0.012 and 0.092 ± 0.009). In the pseudo-prospective trial, 388 (69.7%) septic patients were alerted on with a specificity of 81.4%. Within 24 h of crossing the alert threshold, 20.9% had a sepsis-related event occur.

Conclusions

A machine learning model capable of predicting sepsis in the general ward setting was developed using the EHR data. The pseudo-prospective trial provided a more realistic estimation of implemented performance and demonstrated a 29.1% Positive Predictive Value (PPV) for sepsis-related intervention or outcome within 48 h.

Sepsis prediction, early detection, and identification using clinical text for machine learning: a systematic review

OBJECTIVE

To determine the effects of using unstructured clinical text in machine learning (ML) for prediction, early detection, and identification of sepsis.

MATERIALS AND METHODS

PubMed, Scopus, ACM DL, dblp, and IEEE Xplore databases were searched. Articles utilizing clinical text for ML or natural language processing (NLP) to detect, identify, recognize, diagnose, or predict the onset, development, progress, or prognosis of systemic inflammatory response syndrome, sepsis, severe sepsis, or septic shock were included. Sepsis definition, dataset, types of data, ML models, NLP techniques, and evaluation metrics were extracted.

RESULTS

The clinical text used in models include narrative notes written by nurses, physicians, and specialists in varying situations. This is often combined with common structured data such as demographics, vital signs, laboratory data, and medications. Area under the receiver operating characteristic curve (AUC) comparison of ML methods showed that utilizing both text and structured data predicts sepsis earlier and more accurately than structured data alone. No meta-analysis was performed because of incomparable measurements among the 9 included studies.

DISCUSSION

Studies focused on sepsis identification or early detection before onset; no studies used patient histories beyond the current episode of care to predict sepsis. Sepsis definition affects reporting methods, outcomes, and results. Many methods rely on continuous vital sign measurements in intensive care, making them not easily transferable to general ward units.

CONCLUSIONS

Approaches were heterogeneous, but studies showed that utilizing both unstructured text and structured data in ML can improve identification and early detection of sepsis.

Ground truth labels challenge the validity of sepsis consensus definitions in critical illness

BACKGROUND

Sepsis is the leading cause of death in the intensive care unit (ICU). Expediting its diagnosis, largely determined by clinical assessment, improves survival. Predictive and explanatory modelling of sepsis in the critically ill commonly bases both outcome definition and predictions on clinical criteria for consensus definitions of sepsis, leading to circularity. As a remedy, we collected ground truth labels for sepsis.

METHODS

In the Ground Truth for Sepsis Questionnaire (GTSQ), senior attending physicians in the ICU documented daily their opinion on each patient's condition regarding sepsis as a five-category working diagnosis and nine related items. Working diagnosis groups were described and compared and their SOFA-scores analyzed with a generalized linear mixed model. Agreement and discriminatory performance measures for clinical criteria of sepsis and GTSQ labels as reference class were derived.

RESULTS

We analyzed 7291 questionnaires and 761 complete encounters from the first survey year. Editing rates for all items were > 90%, and responses were consistent with current understanding of critical illness pathophysiology, including sepsis pathogenesis. Interrater agreement for presence and absence of sepsis was almost perfect but only slight for suspected infection. ICU mortality was 19.5% in encounters with SIRS as the "worst" working diagnosis compared to 5.9% with sepsis and 5.9% with severe sepsis without differences in admission and maximum SOFA. Compared to sepsis, proportions of GTSQs with SIRS plus acute organ dysfunction were equal and macrocirculatory abnormalities higher (p < 0.0001). SIRS proportionally ranked above sepsis in daily assessment of illness severity (p < 0.0001). Separate analyses of neurosurgical referrals revealed similar differences. Discriminatory performance of Sepsis-1/2 and Sepsis-3 compared to GTSQ labels was similar with sensitivities around 70% and specificities 92%. Essentially no difference between the prevalence of SIRS and SOFA ≥ 2 yielded sensitivities and specificities for detecting sepsis onset close to 55% and 83%, respectively.

CONCLUSIONS

GTSQ labels are a valid measure of sepsis in the ICU. They reveal suspicion of infection as an unclear clinical concept and refute an illness severity hierarchy in the SIRS-sepsis-severe sepsis spectrum. Ground truth challenges the accuracy of Sepsis-1/2 and Sepsis-3 in detecting sepsis onset. It is an indispensable intermediate step towards advancing diagnosis and therapy in the ICU and, potentially, other health care settings.

Comparison of early warning scores for sepsis early identification and prediction in the general ward setting

The objective of this study was to directly compare the ability of commonly used early warning scores (EWS) for early identification and prediction of sepsis in the general ward setting. For general ward patients at a large, academic medical center between early-2012 and mid-2018, common EWS and patient acuity scoring systems were calculated from electronic health records (EHR) data for patients that both met and did not meet Sepsis-3 criteria. For identification of sepsis at index time, National Early Warning Score 2 (NEWS 2) had the highest performance (area under the receiver operating characteristic curve: 0.803 [95% confidence interval [CI]: 0.795-0.811], area under the precision recall curves: 0.130 [95% CI: 0.121-0.140]) followed NEWS, Modified Early Warning Score, and quick Sequential Organ Failure Assessment (qSOFA). Using validated thresholds, NEWS 2 also had the highest recall (0.758 [95% CI: 0.736-0.778]) but qSOFA had the highest specificity (0.950 [95% CI: 0.948-0.952]), positive predictive value (0.184 [95% CI: 0.169-0.198]), and F1 score (0.236 [95% CI: 0.220-0.253]). While NEWS 2 outperformed all other compared EWS and patient acuity scores, due to the low prevalence of sepsis, all scoring systems were prone to false positives (low positive predictive value without drastic sacrifices in sensitivity), thus leaving room for more computationally advanced approaches.

Timing of antibiotic therapy in the ICU

Severe or life threatening infections are common among patients in the intensive care unit (ICU). Most infections in the ICU are bacterial or fungal in origin and require antimicrobial therapy for clinical resolution. Antibiotics are the cornerstone of therapy for infected critically ill patients. However, antibiotics are often not optimally administered resulting in less favorable patient outcomes including greater mortality. The timing of antibiotics in patients with life threatening infections including sepsis and septic shock is now recognized as one of the most important determinants of survival for this population. Individuals who have a delay in the administration of antibiotic therapy for serious infections can have a doubling or more in their mortality. Additionally, the timing of an appropriate antibiotic regimen, one that is active against the offending pathogens based on in vitro susceptibility, also influences survival. Thus not only is early empiric antibiotic administration important but the selection of those agents is crucial as well. The duration of antibiotic infusions, especially for β-lactams, can also influence antibiotic efficacy by increasing antimicrobial drug exposure for the offending pathogen. However, due to mounting antibiotic resistance, aggressive antimicrobial de-escalation based on microbiology results is necessary to counterbalance the pressures of early broad-spectrum antibiotic therapy. In this review, we examine time related variables impacting antibiotic optimization as it relates to the treatment of life threatening infections in the ICU. In addition to highlighting the importance of antibiotic timing in the ICU we hope to provide an approach to antimicrobials that also minimizes the unnecessary use of these agents. Such approaches will increasingly be linked to advances in molecular microbiology testing and artificial intelligence/machine learning. Such advances should help identify patients needing empiric antibiotic therapy at an earlier time point as well as the specific antibiotics required in order to avoid unnecessary administration of broad-spectrum antibiotics.

Efficacy of intravenous vitamin C intervention for septic patients: A systematic review and meta-analysis based on randomized controlled trials

BACKGROUND

The role of vitamin C in sepsis is still controversial, we aimed to systematically review the efficacy of intravenous vitamin C supplementation in the treatment of sepsis.

METHODS

MEDLINE, EmBase, Web of Science, WanFang Data and CNKI were comprehensively searched to collect randomized controlled trails (RCTs) of vitamin C supplementation for patients with sepsis or sepsis shock from January 2000 to March 2021. Two researchers independently screened the literature, extracted the data and accessed the risk of bias in the included studies; meta-analysis was then performed by using Revman 5.4 software.

RESULTS

A total of 10 RCTs involving 1400 participants were included. The results of meta-analysis showed that intravenous vitamin C supplementation can improve SOFA (ΔSOFA) within 72 h [RR = 1.32,95% CI(0.80,1.85), P < 0.0001] of septic patients. There were no difference on short term mortality (28-30d)[RR = 0.83,95% CI(0.65,1.05), P = 0.11], long term mortality (90d) [RR = 1.16,95% CI(0.82,1.66), P = 0.40], hospital LOS[RR = 0.15,95% CI(-0.73,1.03), P = 0.55], ventilator-free days[RR = 0.09,95% CI(-0.24,0.42), P = 0.60], ICU-LOS[RR = 0.22,95% CI(-0.13,0.57), P = 0.22], between two groups. The results of Subgroup analysis showed that intravenous vitamin C alone can reduce the risk of short term mortality (28-30d) [RR = 0.61,95% CI(0.47,0.79), P = 0.0002]of sepsis patients.

CONCLUSION

Based on current RCTs, our work indicated that mono-intravenous vitamin C therapy may reduce short-term mortality of sepsis patients, and it may protect organ functions. Due to the limitation of the quantity and quality of included studies, the above conclusions need to be verified by more large scale and high quality randomized control trials.