External validation of the ISARIC 4C Mortality Score to predict in-hospital mortality among patients with COVID-19 in a Canadian intensive care unit: a single-centre historical cohort study

PURPOSE

With uncertain prognostic utility of existing predictive scoring systems for COVID-19-related illness, the International Severe Acute Respiratory and Emerging Infection Consortium (ISARIC) 4C Mortality Score was developed by the International Severe Acute Respiratory and Emerging Infection Consortium as a COVID-19 mortality prediction tool. We sought to externally validate this score among critically ill patients admitted to an intensive care unit (ICU) with COVID-19 and compare its discrimination characteristics to that of the Acute Physiology and Chronic Health Evaluation (APACHE) II and Sequential Organ Failure Assessment (SOFA) scores.

METHODS

We enrolled all consecutive patients admitted with COVID-19-associated respiratory failure between 5 March 2020 and 5 March 2022 to our university-affiliated and intensivist-staffed ICU (Jewish General Hospital, Montreal, QC, Canada). After data abstraction, our primary outcome of in-hospital mortality was evaluated with an objective of determining the discriminative properties of the ISARIC 4C Mortality Score, using the area under the curve of a logistic regression model.

RESULTS

A total of 429 patients were included, 102 (23.8%) of whom died in hospital. The receiver operator curve of the ISARIC 4C Mortality Score had an area under the curve of 0.762 (95% confidence interval [CI], 0.717 to 0.811), whereas those of the SOFA and APACHE II scores were 0.705 (95% CI, 0.648 to 0.761) and 0.722 (95% CI, 0.667 to 0.777), respectively.

CONCLUSIONS

The ISARIC 4C Mortality Score is a tool that had a good predictive performance for in-hospital mortality in a cohort of patients with COVID-19 admitted to an ICU for respiratory failure. Our results suggest a good external validity of the 4C score when applied to a more severely ill population.

Serum Krebs von den Lungen-6 for Predicting the Severity of COVID-19: A Systematic Review, Meta-Analysis, and Trial Sequence Analysis

OBJECTIVE

This systematic review and meta-analysis aimed to find the association between serum Krebs von den Lungen-6 (KL-6) and the severity of Coronavirus disease 2019 (COVID-19) infection.

DATA SOURCES

Databases of Embase, PubMed, Web of Science, Science Direct, and Google Scholar were searched for studies reporting KL-6 levels in COVID-19 patients, published between January 2020 and September 30 2022.

DATA SYNTHESIS

For comparison between the groups, standard mean difference (SMD) and 95% confidence intervals (CI) were computed as the effect sizes. Sensitivity, specificity, positive likelihood ratio (PLR), and negative likelihood ratio (NLR) were measured to assess the diagnostic power of KL-6. In addition, the summary receiver operating characteristics curve (sROC) was constructed to summarize the true positive (TP), and false positive (FP) rates. To validate the findings of meta-analysis, Trial Sequential Analysis (TSA) was conducted.

RESULTS

Altogether 497 severe COVID-19 patients and 934 non-severe (mild to moderate) COVID-19 patients were included. Pooling of 12 studies indicated that the serum KL-6 level had significant association with severity of COVID-19 infection: standard mean difference = 1.18 (95% CI: 0.93-1.43), p = 0.01; I²: 58.56%]. Pooled diagnostic parameters calculated from eight studies were: sensitivity 0.53 (95% CI: 0.47-0.59); specificity 0.90 (95% CI: 0.88-0.93); positive likelihood ratio 4.80 (95% CI: 3.53-6.53); negative likelihood ratio 0.46 (95% CI: 0.32-0.68); and area under curve: 0.8841. Additionally, TSA verified the adequacy of sample size and robustness of the meta-analysis.

CONCLUSION

Serum KL-6 level has a moderate degree of correlation with the severity of COVID-19 infection but has low sensitivity. So, it is not recommended as a screening test for severe COVID-19 infection.

Comparison Between Physiological Scores SIPF, CURB-65, and APACHE II as Predictors of Prognosis and Mortality in Hospitalized Patients with COVID-19 Pneumonia: A Multicenter Study, Saudi Arabia

Background

A coronavirus pandemic (COVID-19) is associated with catastrophic effects on the world with high morbidity and mortality. We aimed to evaluate the accuracy of physiological shock index (SIPF) (shock index and hypoxemia), CURB -65, acute physiology, and chronic health assessment II (APACHE II) as predictors of prognosis and in-hospital mortality in patients with COVID-19 pneumonia.

Methods

In Saudi Arabia, a multicenter retrospective study was conducted on hospitalized adult patients confirmed to have COVID-19 pneumonia. Information needed to calculate SIPF, CURB-65, and APACHE II scores were obtained from medical records within 24 hours of admission.

Results

The study included 1131 COVID-19 patients who met the inclusion criteria. They were divided into two groups: (A) the ICU group (n=340; 30.1%) and (B) the ward group (n=791; 69.9%). The most common concomitant diseases of patients at initial ICU admission were hypertension (71.5%) and diabetes (62.4%), and most of them were men (63.8%). The overall mortality was 18.7%, and the mortality rate was higher in the ICU group than in the ward group (39.4% vs 9.6%; p < 0.001). The SIPF score showed a significantly higher ability to predict both ICU admission and mortality in patients with COVID-19 pneumonia compared with APACHE II and CURB -65; (AUC 0.89 vs 0.87; p < 0.001) and (AUC 0.89 vs 0.84; p < 0.001) for ICU admission and (AUC 0.90 vs 0.65; p < 0.001) and (AUC 0.90 vs 0.80; p < 0.001) for mortality, respectively.

Conclusion

The ability of the SIPF score to predict ICU admission and mortality in COVID-19 pneumonia is higher than that of APACHE II and CURB-65. The overall mortality was 18.7%, and the mortality rate was higher in the ICU group than in the ward group (39.4% vs 9.6%; p < 0.001).

A Multimodal Approach for the Risk Prediction of Intensive Care and Mortality in Patients with COVID-19

BACKGROUND

Although several studies have been launched towards the prediction of risk factors for mortality and admission in the intensive care unit (ICU) in COVID-19, none of them focuses on the development of explainable AI models to define an ICU scoring index using dynamically associated biological markers.

METHODS

We propose a multimodal approach which combines explainable AI models with dynamic modeling methods to shed light into the clinical features of COVID-19. Dynamic Bayesian networks were used to seek associations among cytokines across four time intervals after hospitalization. Explainable gradient boosting trees were trained to predict the risk for ICU admission and mortality towards the development of an ICU scoring index.

RESULTS

Our results highlight LDH, IL-6, IL-8, Cr, number of monocytes, lymphocyte count, TNF as risk predictors for ICU admission and survival along with LDH, age, CRP, Cr, WBC, lymphocyte count for mortality in the ICU, with prediction accuracy 0.79 and 0.81, respectively. These risk factors were combined with dynamically associated biological markers to develop an ICU scoring index with accuracy 0.9.

CONCLUSIONS

to our knowledge, this is the first multimodal and explainable AI model which quantifies the risk of intensive care with accuracy up to 0.9 across multiple timepoints.