Serial Daily Organ Failure Assessment Beyond ICU Day 5 Does Not Independently Add Precision to ICU Risk-of-Death Prediction

Assessing SOFA score trajectories in sepsis using machine learning: A pragmatic approach to improve the accuracy of mortality prediction

INTRODUCTION

An increasing amount of longitudinal health data is available on critically ill septic patients in the age of digital medicine, including daily sequential organ failure assessment (SOFA) score measurements. Thus, the assessment in sepsis focuses increasingly on the evaluation of the individual disease's trajectory. Machine learning (ML) algorithms may provide a promising approach here to improve the evaluation of daily SOFA score dynamics. We tested whether ML algorithms can outperform the conventional ΔSOFA score regarding the accuracy of 30-day mortality prediction.

METHODS

We used the multicentric SepsisDataNet.NRW study cohort that prospectively enrolled 252 sepsis patients between 03/2018 and 09/2019 for training ML algorithms, i.e. support vector machine (SVM) with polynomial kernel and artificial neural network (aNN). We used the Amsterdam UMC database covering 1,790 sepsis patients for external and independent validation.

RESULTS

Both SVM (AUC 0.84; 95% CI: 0.71-0.96) and aNN (AUC 0.82; 95% CI: 0.69-0.95) assessing the SOFA scores of the first seven days led to a more accurate prognosis of 30-day mortality compared to the ΔSOFA score between day 1 and 7 (AUC 0.73; 95% CI: 0.65-0.80; p = 0.02 and p = 0.05, respectively). These differences were even more prominent the shorter the time interval considered. Using the SOFA scores of day 1 to 3 SVM (AUC 0.82; 95% CI: 0.68 0.95) and aNN (AUC 0.80; 95% CI: 0.660.93) led to a more accurate prognosis of 30-day mortality compared to the ΔSOFA score (AUC 0.66; 95% CI: 0.58-0.74; p < 0.01 and p < 0.01, respectively). Strikingly, all these findings could be confirmed in the independent external validation cohort.

CONCLUSIONS

The ML-based algorithms using daily SOFA scores markedly improved the accuracy of mortality compared to the conventional ΔSOFA score. Therefore, this approach could provide a promising and automated approach to assess the individual disease trajectory in sepsis. These findings reflect the potential of incorporating ML algorithms as robust and generalizable support tools on intensive care units.

A time-incorporated SOFA score-based machine learning model for predicting mortality in critically ill patients: A multicenter, real-world study

BACKGROUND

Organ dysfunction (OD) assessment is essential in intensive care units (ICUs). However, current OD assessment scores merely describe the number and the severity of each OD, without evaluating the duration of organ injury. The objective of this study is to develop and validate a machine learning model based on the Sequential Organ Failure Assessment (SOFA) score for the prediction of mortality in critically ill patients.

MATERIAL AND METHODS

Data from the eICU Collaborative Research Database and Medical Information Mart for Intensive Care (MIMIC) -III were mixed for model development. The MIMIC-IV and Nanjing Jinling Hospital Surgical ICU database were used as external test set A and set B, respectively. The outcome of interest was in-ICU mortality. A modified SOFA model incorporating time-dimension (T-SOFA) was stepwise developed to predict ICU mortality using extreme gradient boosting (XGBoost), support vector machine, random forest and logistic regression algorithms. Time-dimensional features were calculated based on six consecutive SOFA scores collected every 12 h within the first three days of admission. The predictive performance was assessed with the area under the receiver operating characteristic curves (AUROC) and calibration plot.

RESULTS

A total of 82,132 patients from the real-world datasets were included in this study, and 7,494 patients (9.12%) died during their ICU stay. The T-SOFA M3 that incorporated the time-dimension features and age, using the XGBoost algorithm, significantly outperformed the original SOFA score in the validation set (AUROC 0.800 95% CI [0.787-0.813] vs. 0.693 95% CI [0.678-0.709], p < 0.01). Good discrimination and calibration were maintained in the test set A and B, with AUROC of 0.803, 95% CI [0.791-0.815] and 0.830, 95% CI [0.789-0.870], respectively.

CONCLUSIONS

The time-incorporated T-SOFA model could significantly improve the prediction performance of the original SOFA score and is of potential for identifying high-risk patients in future clinical application.

African Americans Struggle With the Current COVID-19

OBJECTIVES

Our study aims to explore the differential impact of this pandemic on clinical presentations and outcomes in African Americans (AAs) compared to white patients.

BACKGROUND

AAs have worse outcomes compared to whites while facing heart diseases, stroke, cancer, asthma, influenza and pneumonia, diabetes, and HIV/AIDS. However, there is no current study to show the impact of COVID-19 pandemic on the AA communities.

METHODS

This is a retrospective study that included patients with laboratory-confirmed COVID-19 from 2 tertiary centers in New Orleans, LA. Clinical and laboratory data were collected. Multivariate analyses were performed to identify the risk factors associated with adverse events.

RESULTS

A total of 157 patients were identified. Of these, 134 (77%) were AAs, whereas 23.4% of patients were Whites. Interestingly, AA were younger, with a mean age of 63 ± 13.4 compared to 75.7 ± 23 years in Whites (P < 0.001). Thirty-seven patients presented with no insurance, and 34 of them were AA. SOFA Score was significantly higher in AA (2.57 ± 2.1) compared to White patients (1.69 ± 1.7), P = 0.041. Elevated SOFA score was associated with higher odds for intubation (odds ratio = 1.6, 95% confidence interval = 1.32-1.93, P < 0.001). AA had more prolonged length of hospital stays (11.1 ± 13.4 days vs 7.7 ± 23 days) than in Whites, P = 0.01.

CONCLUSION

AAs present with more advanced disease and eventually have worse outcomes from COVID-19 infection. Future studies are warranted for further investigations that should impact the need for providing additional resources to the AA communities.

Prognostic Performance of SOFA, qSOFA, and SIRS in Kidney Transplant Recipients Suffering from Infection: A Retrospective Observational Study

INTRODUCTION

The prognostic performance of scoring systems for illness severity in infectious kidney transplant recipients (KTRs) is rarely reported. We investigated the ability of the scores for the quick Sequential Organ Failure Assessment (qSOFA), Sequential Organ Failure Assessment (SOFA) and Systemic Inflammatory Response Syndrome (SIRS) to predict in-hospital mortality, intensive care unit (ICU) admission and mechanical ventilation (MV) requirement.

METHODS

This was a second analysis of a retrospective observational study. Scores for SIRS, SOFA and qSOFA were calculated upon hospitalization (infection onset was before hospitalization) or on the day of infection onset (infection episodes were during hospitalization). The primary outcome was in-hospital mortality. The secondary outcomes were ICU admission and MV requirement. Binary logistic regression and area under the receiver operating characteristic curve (AUC) were employed to assess prognostic performance.

RESULTS

A total of 161 infectious episodes occurred in 97 KTRs. Forty patients (41%) experienced more than one episode. The SOFA score was available in 161 infections, and scores for qSOFA and SIRS were available in 160 infections. The SIRS score was not different between KTRs with opposite outcomes. The qSOFA score was higher in infections necessitating MV. The SOFA score was significantly higher in the deceased, those needing ICU admission, MV, and for those with positive etiology results. The SOFA score was the only independent predictor of in-hospital mortality, ICU admission, and MV requirement, and the AUCs were 0.879, 0.815, and 0.784, respectively. The optimum cutoff value of predicting the three outcomes was SOFA score ≥ 3.

CONCLUSIONS

The SOFA score (but not those for SIRS and qSOFA) independently predicted in-hospital mortality, ICU admission, and MV requirement in infectious KTRs.

Unsupervised Clustering Analysis Based on MODS Severity Identifies Four Distinct Organ Dysfunction Patterns in Severely Injured Blunt Trauma Patients

Purpose: We sought to identify a MODS score parameter that highly correlates with adverse outcomes and then use this parameter to test the hypothesis that multiple severity-based MODS clusters could be identified after blunt trauma. Methods: MOD score across days (D) 2-5 was subjected to Fuzzy C-means Clustering Analysis (FCM) followed by eight Clustering Validity Indices (CVI) to derive organ dysfunction patterns among 376 blunt trauma patients admitted to the intensive care unit (ICU) who survived to discharge. Thirty-one inflammation biomarkers were assayed (Luminex™) in serial blood samples (3 samples within the first 24 h and then daily up to D 5) and were analyzed using Two-Way ANOVA and Dynamic Network analysis (DyNA). Results: The FCM followed by CVI suggested four distinct clusters based on MOD score magnitude between D2 and D5. Distinct patterns of organ dysfunction emerged in each of the four clusters and exhibited statistically significant differences with regards to in-hospital outcomes. Interleukin (IL)-6, MCP-1, IL-10, IL-8, IP-10, sST2, and MIG were elevated differentially over time across the four clusters. DyNA identified remarkable differences in inflammatory network interconnectivity. Conclusion: These results suggest the existence of four distinct organ failure patterns based on MOD score magnitude in blunt trauma patients admitted to the ICU who survive to discharge.

Evaluation of ICU Risk Models Adapted for Use as Continuous Markers of Severity of Illness Throughout the ICU Stay

OBJECTIVES

Evaluate the accuracy of different ICU risk models repurposed as continuous markers of severity of illness.

DESIGN

Nonintervention cohort study.

SETTING

eICU Research Institute ICUs using tele-ICU software calculating continuous ICU Discharge Readiness Scores between January 2013 and March 2016.

PATIENTS

Five hundred sixty-one thousand four hundred seventy-eight adult ICU patients with an ICU length of stay between 4 hours and 30 days.

INTERVENTIONS

Not available.

MEASUREMENTS AND MAIN RESULTS

Hourly Acute Physiology and Chronic Health Evaluation IV, Sequential Organ Failure Assessment, and Discharge Readiness Scores were calculated beginning hour 4 of the ICU stay. Primary outcome was the area under the receiver operating characteristic curve for the mean score with ICU mortality. Secondary outcomes included area under the receiver operating characteristic curves for ICU mortality with admission, median, maximum and last scores, and for death within 24 hours. The trajectories of each score were visualized by plotting the hourly averages against time in the ICU, stratified by mortality and length of stay. The area under the receiver operating characteristic curves for mean Acute Physiology and Chronic Health Evaluation, Sequential Organ Failure Assessment, and Discharge Readiness Scores were 0.90 (0.89-0.90), 0.86 (0.86-0.86), and 0.94 (0.94-0.94), respectively. The area under the receiver operating characteristic curves for hourly Acute Physiology and Chronic Health Evaluation, Sequential Organ Failure Assessment, and Discharge Readiness Scores predicting 24-hour mortality were 0.81 (0.81-0.81), 0.76 (0.76-0.76), and 0.86 (0.86-0.86). Discharge Readiness Scores had a higher area under the receiver operating characteristic curve than both Acute Physiology and Chronic Health Evaluation and Sequential Organ Failure Assessment for each metric. Acute Physiology and Chronic Health Evaluation and Sequential Organ Failure Assessment scores increased throughout the first 24 hours in both survivors and nonsurvivors; Discharge Readiness Scores continuously decreased in survivors and temporarily decreased before increasing by hour 36 in nonsurvivors with longer length of stays.

CONCLUSIONS

Acute Physiology and Chronic Health Evaluation, Sequential Organ Failure Assessment, and Discharge Readiness Scores all have relatively high discrimination for ICU mortality when used continuously; Discharge Readiness Scores tended to have slightly higher area under the receiver operating characteristic curves for each endpoint. These findings validate the use of these models on a population level for continuous risk adjustment in the ICU, although Acute Physiology and Chronic Health Evaluation and Sequential Organ Failure Assessment appear slower to respond to improvements in patient status than Discharge Readiness Scores, and Discharge Readiness Scores may reflect physiologic improvement from interventions, potentially underestimating risk.

Describing organ dysfunction in the intensive care unit: a cohort study of 20,000 patients

BACKGROUND

Multiple organ dysfunction is a common cause of morbidity and mortality in intensive care units (ICUs). Original development of the Sequential Organ Failure Assessment (SOFA) score was not to predict outcome, but to describe temporal changes in organ dysfunction in critically ill patients. Organ dysfunction scoring may be a reasonable surrogate outcome in clinical trials but further exploration of the impact of case mix on the temporal sequence of organ dysfunction is required. Our aim was to compare temporal changes in SOFA scores between hospital survivors and non-survivors.

METHODS

We performed a population-based observational retrospective cohort study of critically ill patients admitted from January 1, 2004, to December 31, 2013, to 4 multisystem adult intensive care units (ICUs) in Calgary, Canada. The primary outcome was temporal changes in daily SOFA scores during the first 14 days of ICU admission. SOFA scores were modeled between hospital survivors and non-survivors using generalized estimating equations (GEE) and were also stratified by admission SOFA (≤ 11 versus > 11).

RESULTS

The cohort consisted of 20,007 patients with at least one SOFA score and was mostly male (58.2%) with a median age of 59 (interquartile range [IQR] 44-72). Median ICU length of stay was 3.5 (IQR 1.7-7.5) days. ICU and hospital mortality were 18.5% and 25.5%, respectively. Temporal change in SOFA scores varied by survival and admission SOFA score in a complicated relationship. Area under the receiver operating characteristic (ROC) curve using admission SOFA as a predictor of hospital mortality was 0.77. The hospital mortality rate was 5.6% for patients with an admission SOFA of 0-2 and 94.4% with an admission SOFA of 20-24. There was an approximately linear increase in hospital mortality for SOFA scores of 3-19 (range 8.7-84.7%).

CONCLUSIONS

Examining the clinical course of organ dysfunction in a large non-selective cohort of patients provides insight into the utility of SOFA. We have demonstrated that hospital outcome is associated with both admission SOFA and the temporal rate of change in SOFA after admission. It is necessary to further explore the impact of additional clinical factors on the clinical course of SOFA with large datasets.

Using the Shapes of Clinical Data Trajectories to Predict Mortality in ICUs

Supplemental Digital Content is available in the text.

1) To show how to exploit the information contained in the trajectories of time-varying patient clinical data for dynamic predictions of mortality in the ICU; and 2) to demonstrate the additional predictive value that can be achieved by incorporating this trajectory information.