Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: an observational cohort study

Overt and occult hypoxemia in patients hospitalized with novel coronavirus disease 2019

Background

Progressive hypoxemia is the predominant mode of deterioration in COVID-19. Among hypoxemia measures, the ratio of the partial pressure of arterial oxygen to the fraction of inspired oxygen (P/F ratio) has optimal construct validity but poor availability because it requires arterial blood sampling. Pulse oximetry reports oxygenation continuously, but occult hypoxemia can occur in Black patients because the technique is affected by skin color. Oxygen dissociation curves allow non-invasive estimation of P/F ratios (ePFR) but this approach remains unproven.

Research Question

Can ePFRs measure overt and occult hypoxemia?

Study Design and methods

We retrospectively studied COVID-19 hospital encounters (n=5319) at two academic centers (University of Virginia [UVA] and Emory University). We measured primary outcomes (death or ICU transfer within 24 hours), ePFR, conventional hypoxemia measures, baseline predictors (age, sex, race, comorbidity), and acute predictors (National Early Warning Score (NEWS) and Sepsis-3). We updated predictors every 15 minutes. We assessed predictive validity using adjusted odds ratios (AOR) and area under receiver operating characteristics curves (AUROC). We quantified disparities (Black vs non-Black) in empirical cumulative distributions using the Kolmogorov-Smirnov (K-S) two-sample test.

Results

Overt hypoxemia (low ePFR) predicted bad outcomes (AOR for a 100-point ePFR drop: 2.7 [UVA]; 1.7 [Emory]; p<0.01) with better discrimination (AUROC: 0.76 [UVA]; 0.71 [Emory]) than NEWS (AUROC: 0.70 [UVA]; 0.70 [Emory]) or Sepsis-3 (AUROC: 0.68 [UVA]; 0.65 [Emory]). We found racial differences consistent with occult hypoxemia. Black patients had better apparent oxygenation (K-S distance: 0.17 [both sites]; p<0.01) but, for comparable ePFRs, worse outcomes than other patients (AOR: 2.2 [UVA]; 1.2 [Emory], p<0.01).

Interpretation

The ePFR was a valid measure of overt hypoxemia. In COVID-19, it may outperform multi-organ dysfunction models like NEWS and Sepsis-3. By accounting for biased oximetry as well as clinicians’ real-time responses to it (supplemental oxygen adjustment), ePFRs may enable statistical modelling of racial disparities in outcomes attributable to occult hypoxemia.

Antibiotic Prescription and In-Hospital Mortality in COVID-19: A Prospective Multicentre Cohort Study

BACKGROUND

Since the beginning of the COVID-19 pandemic, empiric antibiotics (ATBs) have been prescribed on a large scale in both in- and outpatients. We aimed to assess the impact of antibiotic treatment on the outcomes of hospitalised patients with moderate and severe coronavirus disease 2019 (COVID-19).

METHODS

We conducted a prospective multicentre cohort study in six clinical hospitals, between January 2021 and May 2021.

RESULTS

We included 553 hospitalised COVID-19 patients, of whom 58% (311/553) were prescribed antibiotics, while bacteriological tests were performed in 57% (178/311) of them. Death was the outcome in 48 patients-39 from the ATBs group and 9 from the non-ATBs group. The patients who received antibiotics during hospitalisation had a higher mortality (RR = 3.37, CI 95%: 1.7-6.8), and this association was stronger in the subgroup of patients without reasons for antimicrobial treatment (RR = 6.1, CI 95%: 1.9-19.1), while in the subgroup with reasons for antimicrobial therapy the association was not statistically significant (OR = 2.33, CI 95%: 0.76-7.17). After adjusting for the confounders, receiving antibiotics remained associated with a higher mortality only in the subgroup of patients without criteria for antibiotic prescription (OR = 10.3, CI 95%: 2-52).

CONCLUSIONS

In our study, antibiotic treatment did not decrease the risk of death in the patients with mild and severe COVID-19, but was associated with a higher risk of death in the subgroup of patients without reasons for it.

Risk stratification of patients with SARS-CoV-2 by tissue factor expression in circulating extracellular vesicles

Inflammatory response following SARS-CoV-2 infection results in substantial increase of amounts of intravascular pro-coagulant extracellular vesicles (EVs) expressing tissue factor (CD142) on their surface. CD142-EV turned out to be useful as diagnostic biomarker in COVID-19 patients. Here we aimed at studying the prognostic capacity of CD142-EV in SARS-CoV-2 infection.

Expression of CD142-EV was evaluated in 261 subjects admitted to hospital for pneumonia and with a positive molecular test for SARS-CoV-2. The study population consisted of a discovery cohort of selected patients (n = 60) and an independent validation cohort including unselected consecutive enrolled patients (n = 201). CD142-EV levels were correlated with post-hospitalization course of the disease and compared to the clinically available 4C Mortality Score as referral.

CD142-EV showed a reliable performance to predict patient prognosis in the discovery cohort (AUC = 0.906) with an accuracy of 81.7%, that was confirmed in the validation cohort (AUC = 0.736). Kaplan-Meier curves highlighted a high discrimination power in unselected subjects with CD142-EV being able to stratify the majority of patients according to their prognosis. We obtained a comparable accuracy, being not inferior in terms of prediction of patients' prognosis and risk of mortality, with 4C Mortality Score. The expression of surface vesicular CD142 and its reliability as prognostic marker was technically validated using different immunocapture strategies and assays.

The detection of CD142 on EV surface gains considerable interest as risk stratification tool to support clinical decision making in COVID-19.

Clinical prediction models for diagnosis of COVID-19 among adult patients: a validation and agreement study

BACKGROUND

Since the beginning of the pandemic, hospitals have been constantly overcrowded, with several observed waves of infected cases and hospitalisations. To avoid as much as possible this situation, efficient tools to facilitate the diagnosis of COVID-19 are needed.

OBJECTIVE

To evaluate and compare prediction models to diagnose COVID-19 identified in a systematic review published recently using performance indicators such as discrimination and calibration measures.

METHODS

A total of 1618 adult patients present at two Emergency Department triage centers and for whom qRT-PCR tests had been performed were included in this study. Six previously published models were reconstructed and assessed using diagnostic tests as sensitivity (Se) and negative predictive value (NPV), discrimination (Area Under the Roc Curve (AUROC)) and calibration measures. Agreement was also measured between them using Kappa's coefficient and IntraClass Correlation Coefficient (ICC). A sensitivity analysis has been conducted by waves of patients.

RESULTS

Among the 6 selected models, those based only on symptoms and/or risk exposure were found to be less efficient than those based on biological parameters and/or radiological examination with smallest AUROC values (< 0.80). However, all models showed good calibration and values above > 0.75 for Se and NPV but poor agreement (Kappa and ICC < 0.5) between them. The results of the first wave were similar to those of the second wave.

CONCLUSION

Although quite acceptable and similar results were found between all models, the importance of radiological examination was also emphasized, making it difficult to find an appropriate triage system to classify patients at risk for COVID-19.

External Validation of the Modified 4C Deterioration Model and 4C Mortality Score for COVID-19 Patients in a Swiss Tertiary Hospital

Prognostic models to predict the deterioration and mortality risk in COVID-19 patients are utterly needed to assist in informed decision making. Most of these models, however, are at high risk of bias, model overfitting, and unclear reporting. Here, we aimed to externally validate the modified (urea was omitted) 4C Deterioration Model and 4C Mortality Score in a cohort of Swiss COVID-19 patients and, second, to evaluate whether the inclusion of the neutrophil-to-lymphocyte ratio (NLR) improves the predictive performance of the models. We conducted a retrospective single-centre study with adult patients hospitalized with COVID-19. Both prediction models were updated by including the NLR. Model performance was assessed via the models' discriminatory performance (area under the curve, AUC), calibration (intercept and slope), and their performance overall (Brier score). For the validation of the 4C Deterioration Model and Mortality Score, 546 and 527 patients were included, respectively. In total, 133 (24.4%) patients met the definition of in-hospital deterioration. Discrimination of the 4C Deterioration Model was AUC = 0.78 (95% CI 0.73-0.82). A total of 55 (10.44%) patients died in hospital. Discrimination of the 4C Mortality Score was AUC = 0.85 (95% CI 0.79-0.89). There was no evidence for an incremental value of the NLR. Our data confirm the role of the modified 4C Deterioration Model and Mortality Score as reliable prediction tools for the risk of deterioration and mortality. There was no evidence that the inclusion of NLR improved model performance.

Early Recognition of Low-Risk SARS-CoV-2 Pneumonia

Background

A shortage of beds in ICUs and conventional wards during the COVID-19 pandemic led to a collapse of health care resources.

Research Question

Can admission data and minor criteria by the Infectious Diseases Society of America (IDSA) and the American Thoracic Society (ATS) help identify patients with low-risk SARS-CoV-2 pneumonia?

Study Design and Methods

This multicenter cohort study included 1,274 patients in a derivation cohort and 830 (first wave) and 754 (second wave) patients in two validation cohorts. A multinomial regression analysis was performed on the derivation cohort to compare the following patients: those admitted to the ward (assessed as low risk); those admitted to the ICU directly; those transferred to the ICU after general ward admission; and those who died. A regression analysis identified independent factors for low-risk pneumonia. The model was subsequently validated.

Results

In the derivation cohort, similarities existed among those either directly admitted or transferred to the ICU and those who died. These patients could, therefore, be merged into one group. Five independently associated factors were identified as being predictors of low risk (not dying and/or requiring ICU admission) (ORs, with 95% CIs): peripheral blood oxygen saturation/Fio₂ > 450 (0.233; 0.149-0.364); < 3 IDSA/ATS minor criteria (0.231; 0.146-0.365); lymphocyte count > 723 cells/mL (0.539; 0.360-0.806); urea level < 40 mg/dL (0.651; 0.426-0.996); and C-reactive protein level < 60 mg/L (0.454; 0.285-0.724). The areas under the curve were 0.802 (0.769-0.835) in the derivation cohort, and 0.779 (0.742-0.816) and 0.801 (0.757-0.845) for the validation cohorts (first and second waves, respectively).

Interpretation

Initial biochemical findings and the application of < 3 IDSA/ATS minor criteria make early identification of low-risk SARS-CoV-2 pneumonia (approximately 80% of hospitalized patients) feasible. This scenario could facilitate and streamline health care resource allocation for patients with COVID-19.

Clinical Effectiveness of Regdanvimab Treatment for Mild to Moderate COVID-19: A Retrospective Cohort Study

Background

In a Phase III study, regdanvimab (CT-P59) reduced the risk of hospitalization or death versus placebo in patients with mild-to-moderate coronavirus disease 2019 (COVID-19).

Purpose

We performed a retrospective cohort study of patients with COVID-19 to examine the effect of regdanvimab versus standard of care (SoC) on oxygen saturation.

Methods

We reviewed patients with mild-to-moderate COVID-19 confirmed by reverse transcription-polymerase chain reaction at a single hospital in the Republic of Korea. The primary efficacy end point was the proportion of patients deteriorating with peripheral capillary oxygen saturation <94% on room air up to day 28.

Results

A total of 127 patients were treated for COVID-19 with regdanvimab, 190 with SoC. The proportion of patients deteriorating with peripheral capillary oxygen saturation <94% on room air up to day 28 was 13.4% with regdanvimab and 39.5% with SoC (P < 0.0001); median time (range) until sustained recovery of fever was 2.0 (0.2–14.8) and 4.2 (0.1–17.1) days, respectively. Supplemental oxygen was required by 23.6% of patients with regdanvimab and 52.1% with SoC (P<0.0001) for a mean of 6.3 and 8.7 days, respectively (P = 0.0113); no patients needed mechanical ventilation. Compared with SoC, hospitalization was shorter with regdanvimab (mean = 11.1 vs 13.6 days; 63.8% vs 31.6% discharged within 11 days; both P values < 0.0001). Fewer regdanvimab-treated patients required remdesivir (14.2% vs 43.2%; P < 0.0001). There were no deaths. Two patients had adverse reactions with regdanvimab.

Conclusions

This real-world study indicates that regdanvimab can prevent deterioration in patients with mild-to-moderate COVID-19. (Curr Ther Res Clin Exp. 2022; 83:XXX–XXX)

Development and Validation of the RCOS Prognostic Index: A Bedside Multivariable Logistic Regression Model to Predict Hypoxaemia or Death in Patients with SARS-CoV-2 Infection

Introduction

Previous COVID-19 prognostic models have been developed in hospital settings and are not applicable to COVID-19 cases in the general population. There is an urgent need for prognostic scores aimed to identify patients at high risk of complications at the time of COVID-19 diagnosis.

Methods

The RDT COVID-19 Observational Study (RCOS) collected clinical data from patients with COVID-19 admitted regardless of the severity of their symptoms in a general hospital in India. We aimed to develop and validate a simple bedside prognostic score to predict the risk of hypoxaemia or death.

Results

4035 patients were included in the development cohort and 2046 in the validation cohort. The primary outcome occurred in 961 (23.8%) and 548 (26.8%) patients in the development and validation cohorts, respectively. The final model included 12 variables: age, systolic blood pressure, heart rate, respiratory rate, aspartate transaminase, lactate dehydrogenase, urea, C-reactive protein, sodium, lymphocyte count, neutrophil count, and neutrophil/lymphocyte ratio. In the validation cohort, the area under the receiver operating characteristic curve (AUROCC) was 0.907 (95% CI, 0.892–0.922), and the Brier Score was 0.098. The decision curve analysis showed good clinical utility in hypothetical scenarios where the admission of patients was decided according to the prognostic index. When the prognostic index was used to predict mortality in the validation cohort, the AUROCC was 0.947 (95% CI, 0.925–0.97) and the Brier score was 0.0188.

Conclusions

The RCOS prognostic index could help improve the decision making in the current COVID-19 pandemic, especially in resource-limited settings with poor healthcare infrastructure such as India. However, implementation in other settings is needed to cross-validate and verify our findings.

Development and Validation of a Two-Step Predictive Risk Stratification Model for Coronavirus Disease 2019 In-hospital Mortality: A Multicenter Retrospective Cohort Study

Objectives

An accurate prognostic score to predict mortality for adults with COVID-19 infection is needed to understand who would benefit most from hospitalizations and more intensive support and care. We aimed to develop and validate a two-step score system for patient triage, and to identify patients at a relatively low level of mortality risk using easy-to-collect individual information.

Design

Multicenter retrospective observational cohort study.

Setting

Four health centers from Virginia Commonwealth University, Georgetown University, the University of Florida, and the University of California, Los Angeles.

Patients

Coronavirus Disease 2019-confirmed and hospitalized adult patients.

Measurements and Main Results

We included 1,673 participants from Virginia Commonwealth University (VCU) as the derivation cohort. Risk factors for in-hospital death were identified using a multivariable logistic model with variable selection procedures after repeated missing data imputation. A two-step risk score was developed to identify patients at lower, moderate, and higher mortality risk. The first step selected increasing age, more than one pre-existing comorbidities, heart rate >100 beats/min, respiratory rate ≥30 breaths/min, and SpO₂ <93% into the predictive model. Besides age and SpO₂, the second step used blood urea nitrogen, absolute neutrophil count, C-reactive protein, platelet count, and neutrophil-to-lymphocyte ratio as predictors. C-statistics reflected very good discrimination with internal validation at VCU (0.83, 95% CI 0.79-0.88) and external validation at the other three health systems (range, 0.79-0.85). A one-step model was also derived for comparison. Overall, the two-step risk score had better performance than the one-step score.

Conclusions

The two-step scoring system used widely available, point-of-care data for triage of COVID-19 patients and is a potentially time- and cost-saving tool in practice.

External validation of the 4C Mortality Score for hospitalised patients with COVID-19 in the RECOVER network

OBJECTIVES

Estimating mortality risk in hospitalised SARS-CoV-2+ patients may help with choosing level of care and discussions with patients. The Coronavirus Clinical Characterisation Consortium Mortality Score (4C Score) is a promising COVID-19 mortality risk model. We examined the association of risk factors with 30-day mortality in hospitalised, full-code SARS-CoV-2+ patients and investigated the discrimination and calibration of the 4C Score. This was a retrospective cohort study of SARS-CoV-2+ hospitalised patients within the RECOVER (REgistry of suspected COVID-19 in EmeRgency care) network.

SETTING

99 emergency departments (EDs) across the USA.

PARTICIPANTS

Patients ≥18 years old, positive for SARS-CoV-2 in the ED, and hospitalised.

PRIMARY OUTCOME

Death within 30 days of the index visit. We performed logistic regression analysis, reporting multivariable risk ratios (MVRRs) and calculated the area under the ROC curve (AUROC) and mean prediction error for the original 4C Score and after dropping the C reactive protein (CRP) component.

RESULTS

Of 6802 hospitalised patients with COVID-19, 1149 (16.9%) died within 30 days. The 30-day mortality was increased with age 80+ years (MVRR=5.79, 95% CI 4.23 to 7.34); male sex (MVRR=1.17, 1.05 to 1.28); and nursing home/assisted living facility residence (MVRR=1.29, 1.1 to 1.48). The 4C Score had comparable discrimination in the RECOVER dataset compared with the original 4C validation dataset (AUROC: RECOVER 0.786 (95% CI 0.773 to 0.799), 4C validation 0.763 (95% CI 0.757 to 0.769). Score-specific mortalities in our sample were lower than in the 4C validation sample (mean prediction error 6.0%). Dropping the CRP component from the 4C Score did not substantially affect discrimination and 4C risk estimates were now close (mean prediction error 0.7%).

CONCLUSIONS

We independently validated 4C Score as predicting risk of 30-day mortality in hospitalised SARS-CoV-2+ patients. We recommend dropping the CRP component of the score and using our recalibrated mortality risk estimates.

Clinical update on COVID-19 for the emergency clinician: Presentation and evaluation

INTRODUCTION

Coronavirus disease of 2019 (COVID-19) has resulted in millions of cases worldwide. As the pandemic has progressed, the understanding of this disease has evolved.

OBJECTIVE

This first in a two-part series on COVID-19 updates provides a focused overview of the presentation and evaluation of COVID-19 for emergency clinicians.

DISCUSSION

COVID-19, caused by Severe Acute Respiratory Syndrome coronavirus 2 (SARS-CoV-2), has resulted in significant morbidity and mortality worldwide. Several variants exist, including a variant of concern known as Delta (B.1.617.2 lineage) and the Omicron variant (B.1.1.529 lineage). The Delta variant is associated with higher infectivity and poor patient outcomes, and the Omicron variant has resulted in a significant increase in infections. While over 80% of patients experience mild symptoms, a significant proportion can be critically ill, including those who are older and those with comorbidities. Upper respiratory symptoms, fever, and changes in taste/smell remain the most common presenting symptoms. Extrapulmonary complications are numerous and may be severe, including the cardiovascular, neurologic, gastrointestinal, and dermatologic systems. Emergency department evaluation includes focused testing for COVID-19 and assessment of end-organ injury. Imaging may include chest radiography, computed tomography, or ultrasound. Several risk scores may assist in prognostication, including the 4C (Coronavirus Clinical Characterisation Consortium) score, quick COVID Severity Index (qCSI), NEWS2, and the PRIEST score, but these should only supplement and not replace clinical judgment.

CONCLUSION

This review provides a focused update of the presentation and evaluation of COVID-19 for emergency clinicians.

Can we reliably automate clinical prognostic modelling? A retrospective cohort study for ICU triage prediction of in-hospital mortality of COVID-19 patients in the Netherlands

BACKGROUND

Building Machine Learning (ML) models in healthcare may suffer from time-consuming and potentially biased pre-selection of predictors by hand that can result in limited or trivial selection of suitable models. We aimed to assess the predictive performance of automating the process of building ML models (AutoML) in-hospital mortality prediction modelling of triage COVID-19 patients at ICU admission versus expert-based predictor pre-selection followed by logistic regression.

METHODS

We conducted an observational study of all COVID-19 patients admitted to Dutch ICUs between February and July 2020. We included 2,690 COVID-19 patients from 70 ICUs participating in the Dutch National Intensive Care Evaluation (NICE) registry. The main outcome measure was in-hospital mortality. We asessed model performance (at admission and after 24h, respectively) of AutoML compared to the more traditional approach of predictor pre-selection and logistic regression.

FINDINGS

Predictive performance of the autoML models with variables available at admission shows fair discrimination (average AUROC = 0·75-0·76 (sdev = 0·03), PPV = 0·70-0·76 (sdev = 0·1) at cut-off = 0·3 (the observed mortality rate), and good calibration. This performance is on par with a logistic regression model with selection of patient variables by three experts (average AUROC = 0·78 (sdev = 0·03) and PPV = 0·79 (sdev = 0·2)). Extending the models with variables that are available at 24h after admission resulted in models with higher predictive performance (average AUROC = 0·77-0·79 (sdev = 0·03) and PPV = 0·79-0·80 (sdev = 0·10-0·17)).

CONCLUSIONS

AutoML delivers prediction models with fair discriminatory performance, and good calibration and accuracy, which is as good as regression models with expert-based predictor pre-selection. In the context of the restricted availability of data in an ICU quality registry, extending the models with variables that are available at 24h after admission showed small (but significantly) performance increase.

Examining the utility of extended laboratory panel testing in the emergency department for risk stratification of patients with COVID-19: a single-centre retrospective service evaluation

BACKGROUND

The role of specific blood tests to predict poor prognosis in patients admitted with infection from SARS-CoV-2 remains uncertain. During the first wave of the global pandemic, an extended laboratory testing panel was integrated into the local pathway to guide triage and healthcare resource utilisation for emergency admissions. We conducted a retrospective service evaluation to determine the utility of extended tests (D-dimer, ferritin, high-sensitivity troponin I, lactate dehydrogenase and procalcitonin) compared with the core panel (full blood count, urea and electrolytes, liver function tests and C reactive protein).

METHODS

Clinical outcomes for adult patients with laboratory-confirmed COVID-19 admitted between 17 March and 30 June 2020 were extracted, alongside costs estimates for individual tests. Prognostic performance was assessed using multivariable logistic regression analysis with 28-day mortality used as the primary endpoint and a composite of 28-day intensive care escalation or mortality for secondary analysis.

RESULTS

From 13 500 emergency attendances, we identified 391 unique adults admitted with COVID-19. Of these, 113 died (29%) and 151 (39%) reached the composite endpoint. 'Core' test variables adjusted for age, gender and index of deprivation had a prognostic area under the curve of 0.79 (95% CI 0.67 to 0.91) for mortality and 0.70 (95% CI 0.56 to 0.84) for the composite endpoint. Addition of 'extended' test components did not improve on this.

CONCLUSION

Our findings suggest use of the extended laboratory testing panel to risk stratify community-acquired COVID-19 positive patients on admission adds limited prognostic value. We suggest laboratory requesting should be targeted to patients with specific clinical indications.

[Usage of Artificial Intelligence in the Combat against the COVID-19 Pandemic]

The COVID-19 pandemic is a global health emergency of historic dimension. In this situation, researchers worldwide wanted to help manage the pandemic by using artificial intelligence (AI). This narrative review aims to describe the usage of AI in the combat against COVID-19. The addressed aspects encompass AI algorithms for analysis of thoracic X-rays or CTs, prediction models for severity and outcome of the disease, AI applications in development of new drugs and vaccines as well as forecasting models for spread of the virus. The review shows, which approaches were pursued, and which were successful.

Development and validation of a population-based risk stratification model for severe COVID-19 in the general population

The shortage of recently approved vaccines against the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has highlighted the need for evidence-based tools to prioritize healthcare resources for people at higher risk of severe coronavirus disease (COVID-19). Although age has been identified as the most important risk factor (particularly for mortality), the contribution of underlying comorbidities is often assessed using a pre-defined list of chronic conditions. Furthermore, the count of individual risk factors has limited applicability to population-based "stratify-and-shield" strategies. We aimed to develop and validate a COVID-19 risk stratification system that allows allocating individuals of the general population into four mutually-exclusive risk categories based on multivariate models for severe COVID-19, a composite of hospital admission, transfer to intensive care unit (ICU), and mortality among the general population. The model was developed using clinical, hospital, and epidemiological data from all individuals among the entire population of Catalonia (North-East Spain; 7.5 million people) who experienced a COVID-19 event (i.e., hospitalization, ICU admission, or death due to COVID-19) between March 1 and September 15, 2020, and validated using an independent dataset of 218,329 individuals with COVID-19 confirmed by reverse transcription-polymerase chain reaction (RT-PCR), who were infected after developing the model. No exclusion criteria were defined. The final model included age, sex, a summary measure of the comorbidity burden, the socioeconomic status, and the presence of specific diagnoses potentially associated with severe COVID-19. The validation showed high discrimination capacity, with an area under the curve of the receiving operating characteristics of 0.85 (95% CI 0.85-0.85) for hospital admissions, 0.86 (0.86-0.97) for ICU transfers, and 0.96 (0.96-0.96) for deaths. Our results provide clinicians and policymakers with an evidence-based tool for prioritizing COVID-19 healthcare resources in other population groups aside from those with higher exposure to SARS-CoV-2 and frontline workers.

The CCEDRRN COVID-19 Mortality Score to predict death among nonpalliative patients with COVID-19 presenting to emergency departments: a derivation and validation study

BACKGROUND

Predicting mortality from COVID-19 using information available when patients present to the emergency department can inform goals-of-care decisions and assist with ethical allocation of critical care resources. The study objective was to develop and validate a clinical score to predict emergency department and in-hospital mortality among consecutive nonpalliative patients with COVID-19; in this study, we define palliative patients as those who do not want resuscitative measures, such as intubation, intensive care unit care or cardiopulmonary resuscitation.

METHODS

This derivation and validation study used observational cohort data recruited from 46 hospitals in 8 Canadian provinces participating in the Canadian COVID-19 Emergency Department Rapid Response Network (CCEDRRN). We included adult (age ≥ 18 yr) nonpalliative patients with confirmed COVID-19 who presented to the emergency department of a participating site between Mar. 1, 2020, and Jan. 31, 2021. We randomly assigned hospitals to derivation or validation, and prespecified clinical variables as candidate predictors. We used logistic regression to develop the score in a derivation cohort and examined its performance in predicting emergency department and in-hospital mortality in a validation cohort.

RESULTS

Of 8761 eligible patients, 618 (7.0%) died. The CCEDRRN COVID-19 Mortality Score included age, sex, type of residence, arrival mode, chest pain, severe liver disease, respiratory rate and level of respiratory support. The area under the curve was 0.92 (95% confidence interval [CI] 0.90-0.93) in derivation and 0.92 (95% CI 0.90-0.93) in validation. The score had excellent calibration. These results suggest that scores of 6 or less would categorize patients as being at low risk for in-hospital death, with a negative predictive value of 99.9%. Patients in the low-risk group had an in-hospital mortality rate of 0.1%. Patients with a score of 15 or higher had an observed mortality rate of 81.0%.

INTERPRETATION

The CCEDRRN COVID-19 Mortality Score is a simple score that can be used for level-of-care discussions with patients and in situations of critical care resource constraints to accurately predict death using variables available on emergency department arrival. The score was derived and validated mostly in unvaccinated patients, and before variants of concern were circulating widely and newer treatment regimens implemented in Canada.

STUDY REGISTRATION

ClinicalTrials.gov, no. NCT04702945.

Symptom-Based Predictive Model of COVID-19 Disease in Children

BACKGROUND

Testing for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection is neither always accessible nor easy to perform in children. We aimed to propose a machine learning model to assess the need for a SARS-CoV-2 test in children (<16 years old), depending on their clinical symptoms.

METHODS

Epidemiological and clinical data were obtained from the REDCap® registry. Overall, 4434 SARS-CoV-2 tests were performed in symptomatic children between 1 November 2020 and 31 March 2021, 784 were positive (17.68%). We pre-processed the data to be suitable for a machine learning (ML) algorithm, balancing the positive-negative rate and preparing subsets of data by age. We trained several models and chose those with the best performance for each subset.

RESULTS

The use of ML demonstrated an AUROC of 0.65 to predict a COVID-19 diagnosis in children. The absence of high-grade fever was the major predictor of COVID-19 in younger children, whereas loss of taste or smell was the most determinant symptom in older children.

CONCLUSIONS

Although the accuracy of the models was lower than expected, they can be used to provide a diagnosis when epidemiological data on the risk of exposure to COVID-19 is unknown.

COVID-19 Prognostic Models: A Pro-con Debate for Machine Learning vs. Traditional Statistics

The SARS-CoV-2 virus, which causes the COVID-19 pandemic, has had an unprecedented impact on healthcare requiring multidisciplinary innovation and novel thinking to minimize impact and improve outcomes. Wide-ranging disciplines have collaborated including diverse clinicians (radiology, microbiology, and critical care), who are working increasingly closely with data-science. This has been leveraged through the democratization of data-science with the increasing availability of easy to access open datasets, tutorials, programming languages, and hardware which makes it significantly easier to create mathematical models. To address the COVID-19 pandemic, such data-science has enabled modeling of the impact of the virus on the population and individuals for diagnostic, prognostic, and epidemiological ends. This has led to two large systematic reviews on this topic that have highlighted the two different ways in which this feat has been attempted: one using classical statistics and the other using more novel machine learning techniques. In this review, we debate the relative strengths and weaknesses of each method toward the specific task of predicting COVID-19 outcomes.

ISARIC-4C Mortality Score overestimates risk of death due to COVID-19 in Australian ICU patients: a validation cohort study

Objective: To assess the performance of the UK International Severe Acute Respiratory and Emerging Infections Consortium (ISARIC) Coronavirus Clinical Characterisation Consortium (4C) Mortality Score for predicting mortality in Australian patients with coronavirus disease 2019 (COVID-19) requiring intensive care unit (ICU) admission. Design: Multicentre, prospective, observational cohort study. Setting: 78 Australian ICUs participating in the SPRINT-SARI (Short Period Incidence Study of Severe Acute Respiratory Infection) Australia study of COVID-19. Participants: Patients aged 16 years or older admitted to participating Australian ICUs with polymerase chain reaction (PCR)-confirmed COVID-19 between 27 February and 10 October 2020. Main outcome measures: ISARIC-4C Mortality Score, calculated at the time of ICU admission. The primary outcome was observed versus predicted in-hospital mortality (by 4C Mortality and APACHE II). Results: 461 patients admitted to a participating ICU were included. 149 (32%) had complete data to calculate a 4C Mortality Score without imputation. Overall, 61/461 patients (13.2%) died, 16.9% lower than the comparable ISARIC-4C cohort in the United Kingdom. In patients with complete data, the median (interquartile range [IQR]) 4C Mortality Score was 10.0 (IQR, 8.0-13.0) and the observed mortality was 16.1% (24/149) versus 22.9% median predicted risk of death. The 4C Mortality Score discriminatory performance measured by the area under the receiver operating characteristic curve (AUROC) was 0.79 (95% CI, 0.68-0.90), similar to its performance in the original ISARIC-4C UK cohort (0.77) and not superior to APACHE II (AUROC, 0.81; 95% CI, 0.75-0.87). Conclusions: When calculated at the time of ICU admission, the 4C Mortality Score consistently overestimated the risk of death for Australian ICU patients with COVID-19. The 4C Mortality Score may need to be individually recalibrated for use outside the UK and in different hospital settings.

Pandemic strategies with computational and structural biology against COVID-19: A retrospective

The emergence of the Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), which is the etiologic agent of the coronavirus disease 2019 (COVID-19) pandemic, has dominated all aspects of life since of 2020. Research studies on the virus and exploration of therapeutic and preventive strategies has been moving at rapid rates to control the pandemic. In the field of bioinformatics or computational and structural biology, recent research strategies have used multiple disciplines to compile large datasets to uncover statistical correlations and significance, visualize and model proteins, perform molecular dynamics simulations, and employ the help of artificial intelligence and machine learning to harness computational processing power to further the research on COVID-19, including drug screening, drug design, vaccine development, prognosis prediction, and outbreak prediction. These recent developments should help us better understand the viral disease and develop the much-needed therapies and strategies for the management of COVID-19.

Telemedicine Readiness Across Medical Conditions in a US National Representative Sample of Older Adults

Telemedicine has provided older adults the ability to seek care remotely during the coronavirus disease (COVID-19) pandemic. However, it is unclear how diverse medical conditions play a role in telemedicine uptake. A total of 3379 participants (≥65 years) were interviewed in 2018 as part of the National Health and Aging Trends Study. We assessed telemedicine readiness across multiple medical conditions. Most chronic medical conditions and mood symptoms were significantly associated with telemedicine unreadiness, for physical or technical reasons or both, while cancer, hypertension, and arthritis were significantly associated with telemedicine readiness. Our findings suggest that multiple medical conditions play a substantial role in telemedicine uptake among older adults in the US. Therefore, comorbidities should be taken into consideration when promoting and adopting telemedicine technologies among older adults.

Clinical Management of Adult Patients with COVID-19 Outside Intensive Care Units: Guidelines from the Italian Society of Anti-Infective Therapy (SITA) and the Italian Society of Pulmonology (SIP)

INTRODUCTION

The Italian Society of Anti-Infective Therapy (SITA) and the Italian Society of Pulmonology (SIP) constituted an expert panel for developing evidence-based guidance for the clinical management of adult patients with coronavirus disease 2019 (COVID-19) outside intensive care units.

METHODS

Ten systematic literature searches were performed to answer ten different key questions. The retrieved evidence was graded according to the Grading of Recommendations Assessment, Development, and Evaluation methodology (GRADE).

RESULTS AND CONCLUSION

The literature searches mostly assessed the available evidence on the management of COVID-19 patients in terms of antiviral, anticoagulant, anti-inflammatory, immunomodulatory, and continuous positive airway pressure (CPAP)/non-invasive ventilation (NIV) treatment. Most evidence was deemed as of low certainty, and in some cases, recommendations could not be developed according to the GRADE system (best practice recommendations were provided in similar situations). The use of neutralizing monoclonal antibodies may be considered for outpatients at risk of disease progression. For inpatients, favorable recommendations were provided for anticoagulant prophylaxis and systemic steroids administration, although with low certainty of evidence. Favorable recommendations, with very low/low certainty of evidence, were also provided for, in specific situations, remdesivir, alone or in combination with baricitinib, and tocilizumab. The presence of many best practice recommendations testified to the need for further investigations by means of randomized controlled trials, whenever possible, with some possible future research directions stemming from the results of the ten systematic reviews.

Predicting critical illness on initial diagnosis of COVID-19 based on easily obtained clinical variables: development and validation of the PRIORITY model

OBJECTIVES

We aimed to develop and validate a prediction model, based on clinical history and examination findings on initial diagnosis of coronavirus disease 2019 (COVID-19), to identify patients at risk of critical outcomes.

METHODS

We used data from the SEMI-COVID-19 Registry, a cohort of consecutive patients hospitalized for COVID-19 from 132 centres in Spain (23rd March to 21st May 2020). For the development cohort, tertiary referral hospitals were selected, while the validation cohort included smaller hospitals. The primary outcome was a composite of in-hospital death, mechanical ventilation, or admission to intensive care unit. Clinical signs and symptoms, demographics, and medical history ascertained at presentation were screened using least absolute shrinkage and selection operator, and logistic regression was used to construct the predictive model.

RESULTS

There were 10 433 patients, 7850 in the development cohort (primary outcome 25.1%, 1967/7850) and 2583 in the validation cohort (outcome 27.0%, 698/2583). The PRIORITY model included: age, dependency, cardiovascular disease, chronic kidney disease, dyspnoea, tachypnoea, confusion, systolic blood pressure, and SpO2 ≤93% or oxygen requirement. The model showed high discrimination for critical illness in both the development (C-statistic 0.823; 95% confidence interval (CI) 0.813, 0.834) and validation (C-statistic 0.794; 95%CI 0.775, 0.813) cohorts. A freely available web-based calculator was developed based on this model (https://www.evidencio.com/models/show/2344).

CONCLUSIONS

The PRIORITY model, based on easily obtained clinical information, had good discrimination and generalizability for identifying COVID-19 patients at risk of critical outcomes.

Predictive Value of an Age-Based Modification of the National Early Warning System in Hospitalized Patients With COVID-19

BACKGROUND

Early recognition of high-risk patients with coronavirus disease 2019 (COVID-19) may improve outcomes. Although many predictive scoring systems exist, their complexity may limit utility in COVID-19. We assessed the prognostic performance of the National Early Warning Score (NEWS) and an age-based modification (NEWS+age) among hospitalized COVID-19 patients enrolled in a prospective, multicenter US Military Health System (MHS) observational cohort study.

METHODS

Hospitalized adults with confirmed COVID-19 not requiring invasive mechanical ventilation at admission and with a baseline NEWS were included. We analyzed each scoring system's ability to predict key clinical outcomes, including progression to invasive ventilation or death, stratified by baseline severity (low [0-3], medium [4-6], and high [≥7]).

RESULTS

Among 184 included participants, those with low baseline NEWS had significantly shorter hospitalizations (P < .01) and lower maximum illness severity (P < .001). Most (80.2%) of low NEWS vs 15.8% of high NEWS participants required no or at most low-flow oxygen supplementation. Low NEWS (≤3) had a negative predictive value of 97.2% for progression to invasive ventilation or death; a high NEWS (≥7) had high specificity (93.1%) but low positive predictive value (42.1%) for such progression. NEWS+age performed similarly to NEWS at predicting invasive ventilation or death (NEWS+age: area under the receiver operating characteristics curve [AUROC], 0.69; 95% CI, 0.65-0.73; NEWS: AUROC, 0.70; 95% CI, 0.66-0.75).

CONCLUSIONS

NEWS and NEWS+age showed similar test characteristics in an MHS COVID-19 cohort. Notably, low baseline scores had an excellent negative predictive value. Given their easy applicability, these scoring systems may be useful in resource-limited settings to identify COVID-19 patients who are unlikely to progress to critical illness.

Stratification of the risk of developing severe or lethal Covid-19 using a new score from a large Italian population: a population-based cohort study

OBJECTIVES

To develop a population-based risk stratification model (COVID-19 Vulnerability Score) for predicting severe/fatal clinical manifestations of SARS-CoV-2 infection, using the multiple source information provided by the healthcare utilisation databases of the Italian National Health Service.

DESIGN

Retrospective observational cohort study.

SETTING

Population-based study using the healthcare utilisation database from five Italian regions.

PARTICIPANTS

Beneficiaries of the National Health Service, aged 18-79 years, who had the residentship in the five participating regions. Residents in a nursing home were not included. The model was built from the 7 655 502 residents of Lombardy region.

MAIN OUTCOME MEASURE

The score included gender, age and 29 conditions/diseases selected from a list of 61 conditions which independently predicted the primary outcome, that is, severe (intensive care unit admission) or fatal manifestation of COVID-19 experienced during the first epidemic wave (until June 2020). The score performance was validated by applying the model to several validation sets, that is, Lombardy population (second epidemic wave), and the other four Italian regions (entire 2020) for a total of about 15.4 million individuals and 7031 outcomes. Predictive performance was assessed by discrimination (areas under the receiver operating characteristic curve) and calibration (plot of observed vs predicted outcomes).

RESULTS

We observed a clear positive trend towards increasing outcome incidence as the score increased. The areas under the receiver operating characteristic curve of the COVID-19 Vulnerability Score ranged from 0.85 to 0.88, which compared favourably with the areas of generic scores such as the Charlson Comorbidity Score (0.60). A remarkable performance of the score on the calibration of observed and predicted outcome probability was also observed.

CONCLUSIONS

A score based on data used for public health management accurately predicted the occurrence of severe/fatal manifestations of COVID-19. Use of this score may help health decision-makers to more accurately identify high-risk citizens who need early preventive or treatment interventions.

Contrastive Learning Improves Critical Event Prediction in COVID-19 Patients

Deep Learning (DL) models typically require large-scale, balanced training data to be robust, generalizable, and effective in the context of healthcare. This has been a major issue for developing DL models for the coronavirus-disease 2019 (COVID-19) pandemic where data are highly class imbalanced. Conventional approaches in DL use cross-entropy loss (CEL) which often suffers from poor margin classification. We show that contrastive loss (CL) improves the performance of CEL especially in imbalanced electronic health records (EHR) data for COVID-19 analyses. We use a diverse EHR data set to predict three outcomes: mortality, intubation, and intensive care unit (ICU) transfer in hospitalized COVID-19 patients over multiple time windows. To compare the performance of CEL and CL, models are tested on the full data set and a restricted data set. CL models consistently outperform CEL models with differences ranging from 0.04 to 0.15 for AUPRC and 0.05 to 0.1 for AUROC.

Predictors and clinical outcomes of silent hypoxia in COVID-19 patients, a single-center retrospective cohort study

Background

Patients with COVID-19 usually present with fever and respiratory symptoms such as cough, sputum production, and dyspnea. However, they may suffer from severe hypoxemia without a clinical correlation with the respiratory symptoms, also known as silent or apathetic hypoxia. The aim of the study was to assess the predictors and clinical outcomes of COVID-19 patients without dyspnea.

Methods

A single-center retrospective cohort study, based on data extracted from the electronic hospital information system, with COVID-19 patients over a 10-month period in Riyadh, Saudi Arabia.

Results

Of the COVID-19 patients presenting at the Emergency Department with a SpO₂ < 90%, 13% had silent hypoxia. The majority of the patients required BiPAP, 34% were intubated and 60% were admitted to an intensive care unit. There was no association between dyspnea and gender, age group, body mass index, or comorbidity. Cough, fever, and chronic cardiac diseases were predictive for dyspnea in a regression analysis. There was no difference in the clinical outcome between patients with silent dyspnea or dyspnea. Age and obesity were significantly associated with a decrease in survival, and an increase in the initial SpO₂ increased survival.

Conclusion

Patients with cardiac disease are more likely to present with silent hypoxia. The SpO₂ saturation in COVID-19 may be an independent predictor of survival. Silent hypoxia in COVID-19 patients does not appear to have an association with increase in mortality.

Disease progression of 213 patients hospitalized with Covid-19 in the Czech Republic in March-October 2020: An exploratory analysis

We collected a multi-centric retrospective dataset of patients (N = 213) who were admitted to ten hospitals in Czech Republic and tested positive for SARS-CoV-2 during the early phases of the pandemic in March-October 2020. The dataset contains baseline patient characteristics, breathing support required, pharmacological treatment received and multiple markers on daily resolution. Patients in the dataset were treated with hydroxychloroquine (N = 108), azithromycin (N = 72), favipiravir (N = 9), convalescent plasma (N = 7), dexamethasone (N = 4) and remdesivir (N = 3), often in combination. To explore association between treatments and patient outcomes we performed multiverse analysis, observing how the conclusions change between defensible choices of statistical model, predictors included in the model and other analytical degrees of freedom. Weak evidence to constrain the potential efficacy of azithromycin and favipiravir can be extracted from the data. Additionally, we performed external validation of several proposed prognostic models for Covid-19 severity showing that they mostly perform unsatisfactorily on our dataset.

Clinical significance of prognostic nutrition index in hospitalized patients with COVID-19: Results from single-center experience with systematic review and meta-analysis

BACKGROUND

We aimed to ascertain risk indicators of in-hospital mortality and severity as well as to provide a comprehensive systematic review and meta-analysis to investigate the prognostic significance of the prognostic nutrition index (PNI) as a predictor of adverse outcomes in hospitalized coronavirus disease 2019 (COVID-19) patients.

METHODS

In this cross-sectional study, we studied patients with COVID-19 who were referred to our hospital from February 16 to November 1, 2020. Patients with either a real-time reverse-transcriptase polymerase chain reaction test that was positive for COVID-19 or high clinical suspicion based on the World Health Organization (WHO) interim guidance were enrolled. A parallel systematic review/meta-analysis (in PubMed, Embase, and Web of Science) was performed.

RESULTS

A total of 504 hospitalized COVID-19 patients were included in this study, among which 101 (20.04%) patients died during hospitalization, and 372 (73.81%) patients were categorized as severe cases. At a multivariable level, lower PNI, higher lactate dehydrogenase (LDH), and higher D-dimer levels were independent risk indicators of in-hospital mortality. Additionally, patients with a history of diabetes, lower PNI, and higher LDH levels had a higher tendency to develop severe disease. The meta-analysis indicated the PNI as an independent predictor of in-hospital mortality (odds ratio [OR] = 0.80; P < .001) and disease severity (OR = 0.78; P = .009).

CONCLUSION

Our results emphasized the predictive value of the PNI in the prognosis of patients with COVID-19, necessitating the implementation of a risk stratification index based on PNI values in hospitalized patients with COVID-19.

Circulating microRNA profiles predict the severity of COVID-19 in hospitalized patients

We aimed to examine the circulating microRNA (miRNA) profile of hospitalized COVID-19 patients and evaluate its potential as a source of biomarkers for the management of the disease. This was an observational and multicenter study that included 84 patients with a positive nasopharyngeal swab Polymerase chain reaction (PCR) test for SARS-CoV-2 recruited during the first pandemic wave in Spain (March-June 2020). Patients were stratified according to disease severity: hospitalized patients admitted to the clinical wards without requiring critical care and patients admitted to the intensive care unit (ICU). An additional study was completed including ICU nonsurvivors and survivors. Plasma miRNA profiling was performed using reverse transcription polymerase quantitative chain reaction (RT-qPCR). Predictive models were constructed using least absolute shrinkage and selection operator (LASSO) regression. Ten circulating miRNAs were dysregulated in ICU patients compared to ward patients. LASSO analysis identified a signature of three miRNAs (miR-148a-3p, miR-451a and miR-486-5p) that distinguishes between ICU and ward patients [AUC (95% CI) = 0.89 (0.81-0.97)]. Among critically ill patients, six miRNAs were downregulated between nonsurvivors and survivors. A signature based on two miRNAs (miR-192-5p and miR-323a-3p) differentiated ICU nonsurvivors from survivors [AUC (95% CI) = 0.80 (0.64-0.96)]. The discriminatory potential of the signature was higher than that observed for laboratory parameters such as leukocyte counts, C-reactive protein (CRP) or D-dimer [maximum AUC (95% CI) for these variables = 0.73 (0.55-0.92)]. miRNA levels were correlated with the duration of ICU stay. Specific circulating miRNA profiles are associated with the severity of COVID-19. Plasma miRNA signatures emerge as a novel tool to assist in the early prediction of vital status deterioration among ICU patients.

Immune and cellular damage biomarkers to predict COVID-19 mortality in hospitalized patients

Early prediction of COVID-19 in-hospital mortality relies usually on patients' preexisting comorbidities and is rarely reproducible in independent cohorts. We wanted to compare the role of routinely measured biomarkers of immunity, inflammation, and cellular damage with preexisting comorbidities in eight different machine-learning models to predict mortality, and evaluate their performance in an independent population. We recruited and followed-up consecutive adult patients with SARS-Cov-2 infection in two different Italian hospitals. We predicted 60-day mortality in one cohort (development dataset, n = 299 patients, of which 80% was allocated to the development dataset and 20% to the training set) and retested the models in the second cohort (external validation dataset, n = 402). Demographic, clinical, and laboratory features at admission, treatments and disease outcomes were significantly different between the two cohorts. Notably, significant differences were observed for %lymphocytes (p < 0.05), international-normalized-ratio (p < 0.01), platelets, alanine-aminotransferase, creatinine (all p < 0.001). The primary outcome (60-day mortality) was 29.10% (n = 87) in the development dataset, and 39.55% (n = 159) in the external validation dataset. The performance of the 8 tested models on the external validation dataset were similar to that of the holdout test dataset, indicating that the models capture the key predictors of mortality. The shap analysis in both datasets showed that age, immune features (%lymphocytes, platelets) and LDH substantially impacted on all models' predictions, while creatinine and CRP varied among the different models. The model with the better performance was model 8 (60-day mortality AUROC 0.83 ± 0.06 in holdout test set, 0.79 ± 0.02 in external validation dataset). The features that had the greatest impact on this model's prediction were age, LDH, platelets, and %lymphocytes, more than comorbidities or inflammation markers, and these findings were highly consistent in both datasets, likely reflecting the virus effect at the very beginning of the disease.

External validation of the 4C mortality score among COVID-19 patients admitted to hospital in Ontario, Canada: a retrospective study

Risk prediction scores are important tools to support clinical decision-making for patients with coronavirus disease (COVID-19). The objective of this paper was to validate the 4C mortality score, originally developed in the United Kingdom, for a Canadian population, and to examine its performance over time. We conducted an external validation study within a registry of COVID-19 positive hospital admissions in the Kitchener-Waterloo and Hamilton regions of southern Ontario between March 4, 2020 and June 13, 2021. We examined the validity of the 4C score to prognosticate in-hospital mortality using the area under the receiver operating characteristic curve (AUC) with 95% confidence intervals calculated via bootstrapping. The study included 959 individuals, of whom 224 (23.4%) died in-hospital. Median age was 72 years and 524 individuals (55%) were male. The AUC of the 4C score was 0.77, 95% confidence interval 0.79-0.87. Overall mortality rates across the pre-defined risk groups were 0% (Low), 8.0% (Intermediate), 27.2% (High), and 54.2% (Very High). Wave 1, 2 and 3 values of the AUC were 0.81 (0.76, 0.86), 0.74 (0.69, 0.80), and 0.76 (0.69, 0.83) respectively. The 4C score is a valid tool to prognosticate mortality from COVID-19 in Canadian hospitals and can be used to prioritize care and resources for patients at greatest risk of death.

Evaluating risk stratification scoring systems to predict mortality in patients with COVID-19

Background

The COVID-19 pandemic has necessitated efficient and accurate triaging of patients for more effective allocation of resources and treatment.

Objectives

The objectives are to investigate parameters and risk stratification tools that can be applied to predict mortality within 90 days of hospital admission in patients with COVID-19.

Methods

A literature search of original studies assessing systems and parameters predicting mortality of patients with COVID-19 was conducted using MEDLINE and EMBASE.

Results

589 titles were screened, and 76 studies were found investigating the prognostic ability of 16 existing scoring systems (area under the receiving operator curve (AUROC) range: 0.550–0.966), 38 newly developed COVID-19-specific prognostic systems (AUROC range: 0.6400–0.9940), 15 artificial intelligence (AI) models (AUROC range: 0.840–0.955) and 16 studies on novel blood parameters and imaging.

Discussion

Current scoring systems generally underestimate mortality, with the highest AUROC values found for APACHE II and the lowest for SMART-COP. Systems featuring heavier weighting on respiratory parameters were more predictive than those assessing other systems. Cardiac biomarkers and CT chest scans were the most commonly studied novel parameters and were independently associated with mortality, suggesting potential for implementation into model development. All types of AI modelling systems showed high abilities to predict mortality, although none had notably higher AUROC values than COVID-19-specific prediction models. All models were found to have bias, including lack of prospective studies, small sample sizes, single-centre data collection and lack of external validation.

Conclusion

The single parameters established within this review would be useful to look at in future prognostic models in terms of the predictive capacity their combined effect may harness.

Simple demographic characteristics and laboratory findings on admission may predict in-hospital mortality in patients with SARS-CoV-2 infection: development and validation of the covid-19 score

Background

Coronavirus disease 2019 (COVID-19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) constitutes a major health burden worldwide due to high mortality rates and hospital bed shortages. SARS-CoV-2 infection is associated with several laboratory abnormalities. We aimed to develop and validate a risk score based on simple demographic and laboratory data that could be used on admission in patients with SARS-CoV-2 infection to predict in-hospital mortality.

Methods

Three cohorts of patients from different hospitals were studied consecutively (developing, validation, and prospective cohorts). The following demographic and laboratory data were obtained from medical records: sex, age, hemoglobin, mean corpuscular volume (MCV), platelets, leukocytes, sodium, potassium, creatinine, and C-reactive protein (CRP). For each variable, classification and regression tree analysis were used to establish the cut-off point(s) associated with in-hospital mortality outcome based on data from developing cohort and before they were used for analysis in the validation and prospective cohort. The covid-19 score was calculated as a sum of cut-off points associated with mortality outcome.

Results

The developing, validation, and prospective cohorts included 129, 239, and 497 patients, respectively (median age, 71, 67, and 70 years, respectively). The following cut of points associated with in-hospital mortality: age > 56 years, male sex, hemoglobin < 10.55 g/dL, MCV > 92.9 fL, leukocyte count > 9.635 or < 2.64 10³/µL, platelet count, < 81.49 or > 315.5 10³/µL, CRP > 51.14 mg/dL, creatinine > 1.115 mg/dL, sodium < 134.7 or > 145.4 mEq/L, and potassium < 3.65 or > 6.255 mEq/L. The AUC of the covid-19 score for predicting in-hospital mortality was 0.89 (0.84–0.95), 0.850 (0.75–0.88), and 0.773 (0.731–0.816) in the developing, validation, and prospective cohorts, respectively (P < 0.001The mortality of the prospective cohort stratified on the basis of the covid-19 score was as follows: 0–2 points,4.2%; 3 points, 15%; 4 points, 29%; 5 points, 38.2%; 6 and more points, 60%.

Conclusion

The covid-19 score based on simple demographic and laboratory parameters may become an easy-to-use, widely accessible, and objective tool for predicting mortality in hospitalized patients with SARS-CoV-2 infection.

External validation of the 4C Mortality Score for patients with COVID-19 and pre-existing cardiovascular diseases/risk factors

OBJECTIVES

Predictive algorithms to inform risk management decisions are needed for patients with COVID-19, although the traditional risk scores have not been adequately assessed in Asian patients. We aimed to evaluate the performance of a COVID-19-specific prediction model, the 4C (Coronavirus Clinical Characterisation Consortium) Mortality Score, along with other conventional critical care risk models in Japanese nationwide registry data.

DESIGN

Retrospective cohort study.

SETTING AND PARTICIPANTS

Hospitalised patients with COVID-19 and cardiovascular disease or coronary risk factors from January to May 2020 in 49 hospitals in Japan.

MAIN OUTCOME MEASURES

Two different types of outcomes, in-hospital mortality and a composite outcome, defined as the need for invasive mechanical ventilation and mortality.

RESULTS

The risk scores for 693 patients were tested by predicting in-hospital mortality for all patients and composite endpoint among those not intubated at baseline (n=659). The number of events was 108 (15.6%) for mortality and 178 (27.0%) for composite endpoints. After missing values were multiply imputed, the performance of the 4C Mortality Score was assessed and compared with three prediction models that have shown good discriminatory ability (RISE UP score, A-DROP score and the Rapid Emergency Medicine Score (REMS)). The area under the receiver operating characteristic curve (AUC) for the 4C Mortality Score was 0.84 (95% CI 0.80 to 0.88) for in-hospital mortality and 0.78 (95% CI 0.74 to 0.81) for the composite endpoint. It showed greater discriminatory ability compared with other scores, except for the RISE UP score, for predicting in-hospital mortality (AUC: 0.82, 95% CI 0.78 to 0.86). Similarly, the 4C Mortality Score showed a positive net reclassification improvement index over the A-DROP and REMS for mortality and over all three scores for the composite endpoint. The 4C Mortality Score model showed good calibration, regardless of outcome.

CONCLUSIONS

The 4C Mortality Score performed well in an independent external COVID-19 cohort and may enable appropriate disposition of patients and allocation of medical resources.Trial registration number UMIN000040598.

Age-Adjusted Endothelial Activation and Stress Index for Coronavirus Disease 2019 at Admission Is a Reliable Predictor for 28-Day Mortality in Hospitalized Patients With Coronavirus Disease 2019

Background: Endothelial Activation and Stress Index (EASIX) predict death in patients undergoing allogeneic hematopoietic stem cell transplantation who develop endothelial complications. Because coronavirus disease 2019 (COVID-19) patients also have coagulopathy and endotheliitis, we aimed to assess whether EASIX predicts death within 28 days in hospitalized COVID-19 patients. Methods: We performed a retrospective study on COVID-19 patients from two different cohorts [derivation (n = 1,200 patients) and validation (n = 1,830 patients)]. The endpoint was death within 28 days. The main factors were EASIX [(lactate dehydrogenase * creatinine)/thrombocytes] and aEASIX-COVID (EASIX * age), which were log2-transformed for analysis. Results: Log2-EASIX and log2-aEASIX-COVID were independently associated with an increased risk of death in both cohorts (p < 0.001). Log2-aEASIX-COVID showed a good predictive performance for 28-day mortality both in the derivation cohort (area under the receiver-operating characteristic = 0.827) and in the validation cohort (area under the receiver-operating characteristic = 0.820), with better predictive performance than log2-EASIX (p < 0.001). For log2 aEASIX-COVID, patients with low/moderate risk (<6) had a 28-day mortality probability of 5.3% [95% confidence interval (95% CI) = 4-6.5%], high (6-7) of 17.2% (95% CI = 14.7-19.6%), and very high (>7) of 47.6% (95% CI = 44.2-50.9%). The cutoff of log2 aEASIX-COVID = 6 showed a positive predictive value of 31.7% and negative predictive value of 94.7%, and log2 aEASIX-COVID = 7 showed a positive predictive value of 47.6% and negative predictive value of 89.8%. Conclusion: Both EASIX and aEASIX-COVID were associated with death within 28 days in hospitalized COVID-19 patients. However, aEASIX-COVID had significantly better predictive performance than EASIX, particularly for discarding death. Thus, aEASIX-COVID could be a reliable predictor of death that could help to manage COVID-19 patients.

Machine learning to advance the prediction, prevention and treatment of eating disorders

Machine learning approaches are just emerging in eating disorders research. Promising early results suggest that such approaches may be a particularly promising and fruitful future direction. However, there are several challenges related to the nature of eating disorders in building robust, reliable and clinically meaningful prediction models. This article aims to provide a brief introduction to machine learning and to discuss several such challenges, including issues of sample size, measurement, imbalanced data and bias; I also provide concrete steps and recommendations for each of these issues. Finally, I outline key outstanding questions and directions for future research in building, testing and implementing machine learning models to advance our prediction, prevention, and treatment of eating disorders.

Time to hospitalisation, CT pulmonary involvement and in-hospital death in COVID-19 patients in an Emergency Medicine Unit

BACKGROUND

Patients with coronavirus disease 2019 (COVID-19) are often treated at home given the limited healthcare resources. Many patients may have sudden clinical worsening and may be already compromised at hospitalisation. We investigated the burden of lung involvement according to the time to hospitalisation.

METHODS

In this observational cohort study, 55 consecutive COVID-19-related pneumonia patients were admitted to the Emergency Medicine Unit. Groups of lung involvement at computed tomography were classified as follows: 0 (<5%), 1 (5%-25%), 2 (26%-50%), 3 (51%-75%) and 4 (>75%). We also investigated in-hospital death and the predictive value of Yan-XGBoost model and PREDI-CO scores for death.

RESULTS

The median age was 74 years and 34 were men. Time to admission increased from 2 days in group 0 to 8.5-9 days in groups 3 and 4. A progressive increase in LDH, CRP and d-dimer was found across groups, while a decrease of lymphocytes paO₂ /FiO₂ ratio and SpO₂ was found. Ten (18.2%) patients died during the in-hospital staying. Patients who died were older, with a trend to lower lymphocytes, a higher d-dimer, creatine phosphokinase and troponin T. The Yan-XGBoost model did not accurately predict in-hospital death with an AUC of 0.57 (95% confidence interval [CI] 0.37-0.76), which improved after the addition of the lung involvement groups (AUC 0.68, 95%CI 0.45-0.90). Conversely, a good predictive value was found for the original PREDI-CO score with an AUC of 0.76 (95% CI 0.58-0.93) which remained similar after the addition of the lung involvement (AUC 0.76, 95% CI 0.57-0.94).

CONCLUSION

We found that delayed hospital admission is associated with higher lung involvement. Hence, our data suggest that patients at risk for more severe disease, such as those with high LDH, CRP and d-dimer, should be promptly referred to hospital care.

Fibrosis-4 Predicts the Need for Mechanical Ventilation in a National Multiethnic Cohort of Corona Virus Disease 2019

Simple tests of routine data are needed for those with severe acute respiratory syndrome coronavirus 2, which causes corona virus disease 2019 (COVID-19), to help identify those who may need mechanical ventilation (MV). In this study, we aimed to determine if fibrosis-4 (FIB-4) is associated with the need for MV in patients with COVID-19 and if there is an association to determine the optimal FIB-4 cutoff. This was a retrospective, national, multiethnic cohort study of adults seen in an ambulatory or emergency department setting who were diagnosed with COVID-19. We used the TriNetX platform for analysis. Measures included demographics, comorbid diseases, and routine laboratory tests. A total of 4,901 patients with COVID-19 were included. Patients had a mean age of 56, 48% were women, 42% were obese, 38% were white, 40% were black, 15% had cardiac disease, 39% had diabetes mellitus, 20% had liver disease, and 50% had respiratory disease. The need for MV was 6%. The optimal FIB-4 cutoff for the need for MV was 3.04 (area under the curve, 0.735), which had sensitivity, specificity, and positive and negative predictive values of 42%, 77%, 11%, and 95%, respectively, with 93% accuracy. When stratified by race, increased FIB-4 remained associated with the need for MV in both white and black patients. Conclusion: FIB-4 can be used by frontline providers to identify patients that may require MV.

COVID-19 in young and middle-aged adults: predictors of poor outcome and clinical differences

Introduction

Young and middle-aged adults are the largest group of patients infected with SARS-CoV-2 and some of them develop severe disease.

Objective

To investigate clinical manifestations in adults aged 18–65 years hospitalized for COVID-19 and identify predictors of poor outcome. Secondary objectives: to explore differences compared to the disease in elderly patients and the suitability of the commonly used community-acquired pneumonia prognostic scales in younger populations.

Methods

Multicenter prospective registry of consecutive patients hospitalized for COVID-19 pneumonia aged 18–65 years between March and May 2020. We considered a composite outcome of “poor outcome” including intensive care unit admission and/or use of noninvasive ventilation, continuous positive airway pressure or high flow nasal cannula oxygen and/or death.

Results

We identified 513 patients < 65 years of age, from a cohort of 993 patients. 102 had poor outcomes (19.8%) and 3.9% died. 78% and 55% of patients with poor outcomes were classified as low risk based on CURB and PSI scores, respectively.

A multivariate Cox regression model identified six independent factors associated with poor outcome: heart disease, absence of chest pain or anosmia, low oxygen saturation, high LDH and lymphocyte count < 800/mL.

Conclusions

COVID-19 in younger patients carries significant morbidity and differs in some respects from this disease in the elderly. Baseline heart disease is a relevant risk factor, while anosmia and pleuritic pain are associated to better prognosis. Hypoxemia, LDH and lymphocyte count are predictors of poor outcome. We consider that CURB and PSI scores are not suitable criteria for deciding admission in this population.

Regional performance variation in external validation of four prediction models for severity of COVID-19 at hospital admission: An observational multi-centre cohort study

BACKGROUND

Prediction models should be externally validated to assess their performance before implementation. Several prediction models for coronavirus disease-19 (COVID-19) have been published. This observational cohort study aimed to validate published models of severity for hospitalized patients with COVID-19 using clinical and laboratory predictors.

METHODS

Prediction models fitting relevant inclusion criteria were chosen for validation. The outcome was either mortality or a composite outcome of mortality and ICU admission (severe disease). 1295 patients admitted with symptoms of COVID-19 at Kings Cross Hospital (KCH) in London, United Kingdom, and 307 patients at Oslo University Hospital (OUH) in Oslo, Norway were included. The performance of the models was assessed in terms of discrimination and calibration.

RESULTS

We identified two models for prediction of mortality (referred to as Xie and Zhang1) and two models for prediction of severe disease (Allenbach and Zhang2). The performance of the models was variable. For prediction of mortality Xie had good discrimination at OUH with an area under the receiver-operating characteristic (AUROC) 0.87 [95% confidence interval (CI) 0.79-0.95] and acceptable discrimination at KCH, AUROC 0.79 [0.76-0.82]. In prediction of severe disease, Allenbach had acceptable discrimination (OUH AUROC 0.81 [0.74-0.88] and KCH AUROC 0.72 [0.68-0.75]). The Zhang models had moderate to poor discrimination. Initial calibration was poor for all models but improved with recalibration.

CONCLUSIONS

The performance of the four prediction models was variable. The Xie model had the best discrimination for mortality, while the Allenbach model had acceptable results for prediction of severe disease.

Early Measurement of Blood sST2 Is a Good Predictor of Death and Poor Outcomes in Patients Admitted for COVID-19 Infection

Although several biomarkers have shown correlation to prognosis in COVID-19 patients, their clinical value is limited because of lack of specificity, suboptimal sensibility or poor dynamic behavior. We hypothesized that circulating soluble ST2 (sST2) could be associated to a worse outcome in COVID-19. In total, 152 patients admitted for confirmed COVID-19 were included in a prospective non-interventional, observational study. Blood samples were drawn at admission, 48-72 h later and at discharge. sST2 concentrations and routine blood laboratory were analyzed. Primary endpoints were admission at intensive care unit (ICU) and mortality. Median age was 57.5 years [Standard Deviation (SD: 12.8)], 60.4% males. 10% of patients (n = 15) were derived to ICU and/or died during admission. Median (IQR) sST2 serum concentration (ng/mL) rose to 53.1 (30.9) at admission, peaked at 48-72 h (79.5(64)) and returned to admission levels at discharge (44.9[36.7]). A concentration of sST2 above 58.9 ng/mL was identified patients progressing to ICU admission or death. Results remained significant after multivariable analysis. The area under the receiver operating characteristics curve (AUC) of sST2 for endpoints was 0.776 (p = 0.001). In patients admitted for COVID-19 infection, early measurement of sST2 was able to identify patients at risk of severe complications or death.

COVID-19 admission risk tools should include multiethnic age structures, multimorbidity and deprivation metrics for air pollution, household overcrowding, housing quality and adult skills

BACKGROUND

Ethnic minorities account for 34% of critically ill patients with COVID-19 despite constituting 14% of the UK population. Internationally, researchers have called for studies to understand deterioration risk factors to inform clinical risk tool development.

METHODS

Multicentre cohort study of hospitalised patients with COVID-19 (n=3671) exploring determinants of health, including Index of Multiple Deprivation (IMD) subdomains, as risk factors for presentation, deterioration and mortality by ethnicity. Receiver operator characteristics were plotted for CURB65 and ISARIC4C by ethnicity and area under the curve (AUC) calculated.

RESULTS

Ethnic minorities were hospitalised with higher Charlson Comorbidity Scores than age, sex and deprivation matched controls and from the most deprived quintile of at least one IMD subdomain: indoor living environment (LE), outdoor LE, adult skills, wider barriers to housing and services. Admission from the most deprived quintile of these deprivation forms was associated with multilobar pneumonia on presentation and ICU admission. AUC did not exceed 0.7 for CURB65 or ISARIC4C among any ethnicity except ISARIC4C among Indian patients (0.83, 95% CI 0.73 to 0.93). Ethnic minorities presenting with pneumonia and low CURB65 (0-1) had higher mortality than White patients (22.6% vs 9.4%; p<0.001); Africans were at highest risk (38.5%; p=0.006), followed by Caribbean (26.7%; p=0.008), Indian (23.1%; p=0.007) and Pakistani (21.2%; p=0.004).

CONCLUSIONS

Ethnic minorities exhibit higher multimorbidity despite younger age structures and disproportionate exposure to unscored risk factors including obesity and deprivation. Household overcrowding, air pollution, housing quality and adult skills deprivation are associated with multilobar pneumonia on presentation and ICU admission which are mortality risk factors. Risk tools need to reflect risks predominantly affecting ethnic minorities.

Innate and Adaptive Immune Assessment at Admission to Predict Clinical Outcome in COVID-19 Patients

During the COVID-19 pandemic, many studies have been carried out to evaluate different immune system components to search for prognostic biomarkers of the disease. A broad multiparametric antibody panel of cellular and humoral components of the innate and the adaptative immune response in patients with active SARS-CoV-2 infection has been evaluated in this study. A total of 155 patients were studied at admission into our center and were categorized according to the requirement of oxygen therapy as mild or severe (the latter being those with the requirement). The patients with severe disease were older and had high ferritin, D-dimer, C-reactive protein, troponin, interleukin-6 (IL-6) levels, and neutrophilia with lymphopenia at admission. Moreover, the patients with mild symptoms had significantly increased circulating non-classical monocytes, innate lymphoid cells, and regulatory NK cells. In contrast, severe patients had a low frequency of Th1 and regulatory T cells with increased activated and exhausted CD8 phenotype (CD8⁺CD38⁺HLADR⁺ and CD8⁺CD27^-CD28^-, respectively). The predictive model included age, ferritin, D-dimer, lymph counts, C4, CD8⁺CD27^-CD28^-, and non-classical monocytes in the logistic regression analysis. The model predicted severity with an area under the curve of 78%. Both innate and adaptive immune parameters could be considered potential predictive biomarkers of the prognosis of COVID-19 disease.

Predicting clinical outcomes among hospitalized COVID-19 patients using both local and published models

BACKGROUND

Many models are published which predict outcomes in hospitalized COVID-19 patients. The generalizability of many is unknown. We evaluated the performance of selected models from the literature and our own models to predict outcomes in patients at our institution.

METHODS

We searched the literature for models predicting outcomes in inpatients with COVID-19. We produced models of mortality or criticality (mortality or ICU admission) in a development cohort. We tested external models which provided sufficient information and our models using a test cohort of our most recent patients. The performance of models was compared using the area under the receiver operator curve (AUC).

RESULTS

Our literature review yielded 41 papers. Of those, 8 were found to have sufficient documentation and concordance with features available in our cohort to implement in our test cohort. All models were from Chinese patients. One model predicted criticality and seven mortality. Tested against the test cohort, internal models had an AUC of 0.84 (0.74-0.94) for mortality and 0.83 (0.76-0.90) for criticality. The best external model had an AUC of 0.89 (0.82-0.96) using three variables, another an AUC of 0.84 (0.78-0.91) using ten variables. AUC's ranged from 0.68 to 0.89. On average, models tested were unable to produce predictions in 27% of patients due to missing lab data.

CONCLUSION

Despite differences in pandemic timeline, race, and socio-cultural healthcare context some models derived in China performed well. For healthcare organizations considering implementation of an external model, concordance between the features used in the model and features available in their own patients may be important. Analysis of both local and external models should be done to help decide on what prediction method is used to provide clinical decision support to clinicians treating COVID-19 patients as well as what lab tests should be included in order sets.

Prediction of COVID-19 deterioration in high-risk patients at diagnosis: an early warning score for advanced COVID-19 developed by machine learning

PURPOSE

While more advanced COVID-19 necessitates medical interventions and hospitalization, patients with mild COVID-19 do not require this. Identifying patients at risk of progressing to advanced COVID-19 might guide treatment decisions, particularly for better prioritizing patients in need for hospitalization.

METHODS

We developed a machine learning-based predictor for deriving a clinical score identifying patients with asymptomatic/mild COVID-19 at risk of progressing to advanced COVID-19. Clinical data from SARS-CoV-2 positive patients from the multicenter Lean European Open Survey on SARS-CoV-2 Infected Patients (LEOSS) were used for discovery (2020-03-16 to 2020-07-14) and validation (data from 2020-07-15 to 2021-02-16).

RESULTS

The LEOSS dataset contains 473 baseline patient parameters measured at the first patient contact. After training the predictor model on a training dataset comprising 1233 patients, 20 of the 473 parameters were selected for the predictor model. From the predictor model, we delineated a composite predictive score (SACOV-19, Score for the prediction of an Advanced stage of COVID-19) with eleven variables. In the validation cohort (n = 2264 patients), we observed good prediction performance with an area under the curve (AUC) of 0.73 ± 0.01. Besides temperature, age, body mass index and smoking habit, variables indicating pulmonary involvement (respiration rate, oxygen saturation, dyspnea), inflammation (CRP, LDH, lymphocyte counts), and acute kidney injury at diagnosis were identified. For better interpretability, the predictor was translated into a web interface.

CONCLUSION

We present a machine learning-based predictor model and a clinical score for identifying patients at risk of developing advanced COVID-19.