Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: an observational cohort study

Clinically meaningful phenotypes among SARS-CoV-2 reinfections: Informing prevention strategies for future pandemics

OBJECTIVE

Rapidly phenotyping patients can inform public health action plans in new pandemics. This study aimed to derive meaningful SARS-CoV-2 reinfected patients' phenotypes based on easily-available patient data and explore key epidemiological factors of reinfections.

METHODS

We conducted a retrospective study of a cohort of SARS-CoV-2 reinfected adults from the Basque Country between January 1, 2021 and January 9, 2022. Phenotypes were defined in an unsupervised manner with clustering algorithms, incorporating variables like age, Charlson score, vaccination status and pre-existing treatments and comorbidities. Subsequently, clinical characteristics of phenotypes were compared, and their behavioral differences were evaluated through generalized additive models. Finally, their association with clinical outcomes was assessed.

RESULTS

Four phenotypes were identified, which subsequently had a direct relationship with the risk levels for severe COVID-19 outcomes. The highest-risk group, phenotype 4, consisted of older adults -76 years, [62-85] (Median, [Interquartile range])- with multiple comorbidities and extensive baseline medication use. Phenotype 3 was slightly younger -64 years, [58-77]- but presented very low Charlson scores and few comorbidities, representing an intermediate-risk group. Phenotypes 1 and 2 were younger and healthier adults with similar clinical profiles. However, phenotype 1 showed a less protective attitude, with a higher rate of unvaccinated patients and shorter time intervals between infections.

CONCLUSIONS

We were able to classify reinfected patients into four distinct groups based on easily available variables, and these phenotypes had a direct relationship with COVID-19 clinical outcomes. Thus, rapidly phenotyping infected individuals can serve as a preventive public health strategy during new pandemics.

The influence of the dynamic context of the pandemic on the predictive performance of mortality predictions over time in older patients hospitalized for COVID-19

OBJECTIVES

During the COVID-19 pandemic, dynamic factors, such as governmental policies, improved treatment, prevention options, and viral mutations changed the incidence of outcomes and possibly changed the relation between predictors and outcomes. The aim of the present study was to assess whether the dynamic context of the pandemic influenced the predictive performance of mortality predictions over time in older patients hospitalized for COVID-19.

STUDY DESIGN AND SETTING

The COVID-19 Ouderen Landelijke Database study, a multicentre cohort study in the Netherlands, included COVID-19 patients aged 70 years and older hospitalized during the first (early 2020), second (late 2020), third (late 2021), or fourth wave (early 2022). We developed a prediction model for in-hospital mortality that included variables commonly collected at the emergency department with least absolute shrinkage and selection operator (LASSO) regression on patients admitted in the first pandemic wave and temporally validated this model in patients admitted in the second, third, or fourth wave.

RESULTS

In total, 3067 patients (median age 79 years, 60% men) were included. The final model included demographics, frailty, and indicators of disease severity that were generally available within 3 hours after admission. The model differentiated between death and alive after hospitalization for COVID-19 with an area under the curve (AUC) of 0.80 (95% CI: 0.76-0.84) in the internal validation cohort. In terms of discrimination and calibration, predictive performance of the model decreased over time with an AUC of 0.76 (0.73-0.79) and calibration slope of 0.81 (0.68-0.96) in the second wave, an AUC of 0.77 (0.72-0.82) and calibration slope of 0.85 (0.65-1.10) in the third wave, and an AUC of 0.59 (0.48-0.70) and calibration slope of 0.35 (-0.05, 0.72) in the fourth wave.

CONCLUSION

Compared to the moderate model performance in the first wave, we observed a slight decrease in terms of discrimination and calibration in the second and third wave with a much larger decrease in the fourth wave. This highlights the importance of ongoing data collection, monitoring of model performance, and model updates during a pandemic.

Adaptable graph neural networks design to support generalizability for clinical event prediction

OBJECTIVE

While many machine learning and deep learning-based models for clinical event prediction leverage various data elements from electronic healthcare records such as patient demographics and billing codes, such models face severe challenges when tested outside of their institution of training. These challenges are rooted not only in differences in patient population characteristics, but medical practice patterns of different institutions.

METHOD

We propose a solution to this problem through systematically adaptable design of graph-based convolutional neural networks (GCNN) for clinical event prediction. Our solution relies on the unique property of GCNN where data encoded as graph edges is only implicitly used during the prediction process and can be adapted after model training without requiring model re-training.

RESULTS

Our adaptable GCNN-based prediction models outperformed all comparative models during external validation for two different clinical problems, while supporting multimodal data integration. For prediction of hospital discharge and mortality, the comparative fusion baseline model achieved 0.58 [0.52-0.59] and 0.81[0.80-0.82] AUROC on the external dataset while the GCNN achieved 0.70 [0.68-0.70] and 0.91 [0.90-0.92] respectively. For prediction of future unplanned transfusion, we observed even more gaps in performance due to missing/incomplete data in the external dataset - late fusion achieved 0.44[0.31-0.56] while the GCNN model achieved 0.70 [0.62-0.84].

CONCLUSION

These results support our hypothesis that carefully designed GCNN-based models can overcome generalization challenges faced by prediction models.

Validation of prognostic models predicting mortality or ICU admission in patients with COVID-19 in low- and middle-income countries: a global individual participant data meta-analysis

BACKGROUND

We evaluated the performance of prognostic models for predicting mortality or ICU admission in hospitalized patients with COVID-19 in the World Health Organization (WHO) Global Clinical Platform, a repository of individual-level clinical data of patients hospitalized with COVID-19, including in low- and middle-income countries (LMICs).

METHODS

We identified eligible multivariable prognostic models for predicting overall mortality and ICU admission during hospital stay in patients with confirmed or suspected COVID-19 from a living review of COVID-19 prediction models. These models were evaluated using data contributed to the WHO Global Clinical Platform for COVID-19 from nine LMICs (Burkina Faso, Cameroon, Democratic Republic of Congo, Guinea, India, Niger, Nigeria, Zambia, and Zimbabwe). Model performance was assessed in terms of discrimination and calibration.

RESULTS

Out of 144 eligible models, 140 were excluded due to a high risk of bias, predictors unavailable in LIMCs, or insufficient model description. Among 11,338 participants, the remaining models showed good discrimination for predicting in-hospital mortality (3 models), with areas under the curve (AUCs) ranging between 0.76 (95% CI 0.71-0.81) and 0.84 (95% CI 0.77-0.89). An AUC of 0.74 (95% CI 0.70-0.78) was found for predicting ICU admission risk (one model). All models showed signs of miscalibration and overfitting, with extensive heterogeneity between countries.

CONCLUSIONS

Among the available COVID-19 prognostic models, only a few could be validated on data collected from LMICs, mainly due to limited predictor availability. Despite their discriminative ability, selected models for mortality prediction or ICU admission showed varying and suboptimal calibration.

Prospective clinical surveillance for severe acute respiratory illness and COVID-19 vaccine effectiveness in Kenyan hospitals during the COVID-19 pandemic

BACKGROUND

There are limited data from sub-Saharan Africa describing the demographic characteristics, clinical features and outcome of patients admitted to public hospitals with severe acute respiratory infections during the COVID-19 pandemic.

METHODS

We conducted a prospective longitudinal hospital-based sentinel surveillance between May 2020 and December 2022 at 16 public hospitals in Kenya. All patients aged above 18 years admitted to adult medical wards in the participating hospitals were included. We collected data on demographic and clinical characteristics, SARS-CoV-2 infection and COVID-19 vaccination status and, admission episode outcomes. We determined COVID-19 vaccine effectiveness (VE) against admission with SARS-CoV-2 positive severe acute respiratory illness (SARI) (i.e., COVID-19) and progression to inpatient mortality among patients admitted with SARI, using a test-negative case control design.

RESULTS

Of the 52,636 patients included in the study, 17,950 (34.1%) were admitted with SARI. The median age was 50 years. Patients were equally distributed across sexes. Pneumonia was the most common diagnosis at discharge. Hypertension, Human Immunodeficiency Virus (HIV) infection and Diabetes Mellitus were the most common chronic comorbidities. SARS-CoV-2 test results were positive in 2,364 (27.9%) of the 8,471 patients that underwent testing. After adjusting for age, sex and presence of a chronic comorbidity, SARI patients were more likely to progress to inpatient mortality compared to non-SARI patients regardless of their SARS-CoV-2 infection status (adjusted odds ratio (aOR) for SARI and SARS-CoV-2 negative patients 1.22, 95% CI 1.10-1.37; and aOR for SARI and SARS-CoV-2 positive patients 1.32, 95% CI 1.24-1.40). After adjusting for age, sex and presence of a chronic comorbidity, COVID-19 VE against progression to inpatient mortality following admission with SARI for those with a confirmed vaccination status was 0.59 (95% CI 0.27-0.77).

CONCLUSION

We have provided a comprehensive description of the demographic and clinical pattern of admissions with SARI in Kenyan hospitals during the COVID-19 pandemic period as well as the COVID-19 VE for these patients. These data were useful in providing situational awareness during the first three years of the pandemic in Kenya and informing national response measures.

Phenotyping COVID-19 respiratory failure in spontaneously breathing patients with AI on lung CT-scan

BACKGROUND

Automated analysis of lung computed tomography (CT) scans may help characterize subphenotypes of acute respiratory illness. We integrated lung CT features measured via deep learning with clinical and laboratory data in spontaneously breathing subjects to enhance the identification of COVID-19 subphenotypes.

METHODS

This is a multicenter observational cohort study in spontaneously breathing patients with COVID-19 respiratory failure exposed to early lung CT within 7 days of admission. We explored lung CT images using deep learning approaches to quantitative and qualitative analyses; latent class analysis (LCA) by using clinical, laboratory and lung CT variables; regional differences between subphenotypes following 3D spatial trajectories.

RESULTS

Complete datasets were available in 559 patients. LCA identified two subphenotypes (subphenotype 1 and 2). As compared with subphenotype 2 (n = 403), subphenotype 1 patients (n = 156) were older, had higher inflammatory biomarkers, and were more hypoxemic. Lungs in subphenotype 1 had a higher density gravitational gradient with a greater proportion of consolidated lungs as compared with subphenotype 2. In contrast, subphenotype 2 had a higher density submantellar-hilar gradient with a greater proportion of ground glass opacities as compared with subphenotype 1. Subphenotype 1 showed higher prevalence of comorbidities associated with endothelial dysfunction and higher 90-day mortality than subphenotype 2, even after adjustment for clinically meaningful variables.

CONCLUSIONS

Integrating lung-CT data in a LCA allowed us to identify two subphenotypes of COVID-19, with different clinical trajectories. These exploratory findings suggest a role of automated imaging characterization guided by machine learning in subphenotyping patients with respiratory failure.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT04395482. Registration date: 19/05/2020.

Evolution of serious and life-threatening COVID-19 pneumonia as the SARS-CoV-2 pandemic progressed: an observational study of mortality to 60 days after admission to a 15-hospital US health system

OBJECTIVE

In order to predict at hospital admission the prognosis of patients with serious and life-threatening COVID-19 pneumonia, we sought to understand the clinical characteristics of hospitalised patients at admission as the SARS-CoV-2 pandemic progressed, document their changing response to the virus and its variants over time, and identify factors most importantly associated with mortality after hospital admission.

DESIGN

Observational study using a prospective hospital systemwide COVID-19 database.

SETTING

15-hospital US health system.

PARTICIPANTS

26 872 patients admitted with COVID-19 to our Northeast Ohio and Florida hospitals from 1 March 2020 to 1 June 2022.

MAIN OUTCOME MEASURES

60-day mortality (highest risk period) after hospital admission analysed by random survival forests machine learning using demographics, medical history, and COVID-19 vaccination status, and viral variant, symptoms, and routine laboratory test results obtained at hospital admission.

RESULTS

Hospital mortality fell from 11% in March 2020 to 3.7% in March 2022, a 66% decrease (p<0.0001); 60-day mortality fell from 17% in May 2020 to 4.7% in May 2022, a 72% decrease (p<0.0001). Advanced age was the strongest predictor of 60-day mortality, followed by admission laboratory test results. Risk-adjusted 60-day mortality had all patients been admitted in March 2020 was 15% (CI 3.0% to 28%), and had they all been admitted in May 2022, 12% (CI 2.2% to 23%), a 20% decrease (p<0.0001). Dissociation between observed and predicted decrease in mortality was related to temporal change in admission patient profile, particularly in laboratory test results, but not vaccination status or viral variant.

CONCLUSIONS

Hospital mortality from COVID-19 decreased substantially as the pandemic evolved but persisted after hospital discharge, eclipsing hospital mortality by 50% or more. However, after accounting for the many, even subtle, changes across the pandemic in patients' demographics, medical history and particularly admission laboratory results, a patient admitted early in the pandemic and predicted to be at high risk would remain at high risk of mortality if admitted tomorrow.

Basic host response parameters to classify mortality risk in COVID-19 and community-acquired pneumonia

Improved phenotyping in pneumonia is necessary to strengthen risk assessment. Via a feasible and multidimensional approach with basic parameters, we aimed to evaluate the effect of host response at admission on severity stratification in COVID-19 and community-acquired pneumonia (CAP). Three COVID-19 and one CAP multicenter cohorts including hospitalized patients were recruited. Three easily available variables reflecting different pathophysiologic mechanisms-immune, inflammation, and respiratory-were selected (absolute lymphocyte count [ALC], C-reactive protein [CRP] and, SpO2/FiO2). In-hospital mortality and intensive care unit (ICU) admission were analyzed as outcomes. A multivariable, penalized maximum likelihood logistic regression was performed with ALC (< 724 lymphocytes/mm3), CRP (> 60 mg/L), and, SpO2/FiO2 (< 450). A total of 1452, 1222 and 462 patients were included in the three COVID-19 and 1292 in the CAP cohort for the analysis. Mortality ranged between 4 and 32% (0 to 3 abnormal biomarkers) and 0-9% in SARS-CoV-2 pneumonia and CAP, respectively. In the first COVID-19 cohort, adjusted for age and sex, we observed an increased odds ratio for in-hospital mortality in COVID-19 with elevated biomarkers altered (OR 1.8, 3, and 6.3 with 1, 2, and 3 abnormal biomarkers, respectively). The model had an AUROC of 0.83. Comparable findings were found for ICU admission, with an AUROC of 0.76. These results were confirmed in the other COVID-19 cohorts Similar OR trends were reported in the CAP cohort; however, results were not statistically significant. Assessing the host response via accessible biomarkers is a simple and rapidly applicable approach for pneumonia.

COVID-19 patient profiles over four waves in Barcelona metropolitan area: A clustering approach

OBJECTIVES

Identifying profiles of hospitalized COVID-19 patients and explore their association with different degrees of severity of COVID-19 outcomes (i.e. in-hospital mortality, ICU assistance, and invasive mechanical ventilation). The findings of this study could inform the development of multiple care intervention strategies to improve patient outcomes.

METHODS

Prospective multicentre cohort study during four different waves of COVID-19 from March 1st, 2020 to August 31st, 2021 in four health consortiums within the southern Barcelona metropolitan region. From a starting point of over 292 demographic characteristics, comorbidities, vital signs, severity scores, and clinical analytics at hospital admission, we used both clinical judgment and supervised statistical methods to reduce to the 36 most informative completed covariates according to the disease outcomes for each wave. Patients were then grouped using an unsupervised semiparametric method (KAMILA). Results were interpreted by clinical and statistician team consensus to identify clinically-meaningful patient profiles.

RESULTS

The analysis included nw1 = 1657, nw2 = 697, nw3 = 677, and nw4 = 787 hospitalized-COVID-19 patients for each of the four waves. Clustering analysis identified 2 patient profiles for waves 1 and 3, while 3 profiles were determined for waves 2 and 4. Patients allocated in those groups showed a different percentage of disease outcomes (e.g., wave 1: 15.9% (Cluster 1) vs. 31.8% (Cluster 2) for in-hospital mortality rate). The main factors to determine groups were the patient's age and number of obese patients, number of comorbidities, oxygen support requirement, and various severity scores. The last wave is also influenced by the massive incorporation of COVID-19 vaccines.

CONCLUSION

Our study suggests that a single care model at hospital admission may not meet the needs of hospitalized-COVID-19 adults. A clustering approach appears to be appropriate for helping physicians to differentiate patients and, thus, apply multiple care intervention strategies, as another way of responding to new outbreaks of this or future diseases.

Predicting outcomes of the acute phase of COVID-19. High sensitive prognostic model, based on the results of the international registry "analysis of chronic non-infectious diseases dynamics after COVID-19 infection in adult patients" (ACTIV)

The aim of this study is to investigate the course of the acute period of COVID-19 and devise a prognostic scale for patients hospitalized.

Materials and methods

The ACTIV registry encompassed both male and female patients aged 18 years and above, who were diagnosed with COVID-19 and subsequently hospitalized. Between June 2020 and March 2021, a total of 9364 patients were enrolled across 26 medical centers in seven countries. Data collected during the patients' hospital stay were subjected to multivariate analysis within the R computational environment. A predictive mathematical model, utilizing the "Random Forest" machine learning algorithm, was established to assess the risk of reaching the endpoint (defined as in-hospital death from any cause). This model was constructed using a training subsample (70% of patients), and subsequently tested using a control subsample (30% of patients).

Results

Out of the 9364 hospitalized COVID-19 patients, 545 (5.8%) died. Multivariate analysis resulted in the selection of eleven variables for the final model: minimum oxygen saturation, glomerular filtration rate, age, hemoglobin level, lymphocyte percentage, white blood cell count, platelet count, aspartate aminotransferase, glucose, heart rate, and respiratory rate. Receiver operating characteristic analysis yielded an area under the curve of 89.2%, a sensitivity of 86.2%, and a specificity of 76.0%. Utilizing the final model, a predictive equation and nomogram (termed the ACTIV scale) were devised for estimating in-hospital mortality amongst COVID-19 patients.

Conclusion

The ACTIV scale provides a valuable tool for practicing clinicians to predict the risk of in-hospital death in patients hospitalized with COVID-19.

A Systematic Review of Predictor Composition, Outcomes, Risk of Bias, and Validation of COVID-19 Prognostic Scores

BACKGROUND

Numerous prognostic scores have been published to support risk stratification for patients with coronavirus disease 2019 (COVID-19).

METHODS

We performed a systematic review to identify the scores for confirmed or clinically assumed COVID-19 cases. An in-depth assessment and risk of bias (ROB) analysis (Prediction model Risk Of Bias ASsessment Tool [PROBAST]) was conducted for scores fulfilling predefined criteria ([I] area under the curve [AUC)] ≥ 0.75; [II] a separate validation cohort present; [III] training data from a multicenter setting [≥2 centers]; [IV] point-scale scoring system).

RESULTS

Out of 1522 studies extracted from MEDLINE/Web of Science (20/02/2023), we identified 242 scores for COVID-19 outcome prognosis (mortality 109, severity 116, hospitalization 14, long-term sequelae 3). Most scores were developed using retrospective (75.2%) or single-center (57.1%) cohorts. Predictor analysis revealed the primary use of laboratory data and sociodemographic information in mortality and severity scores. Forty-nine scores were included in the in-depth analysis. The results indicated heterogeneous quality and predictor selection, with only five scores featuring low ROB. Among those, based on the number and heterogeneity of validation studies, only the 4C Mortality Score can be recommended for clinical application so far.

CONCLUSIONS

The application and translation of most existing COVID scores appear unreliable. Guided development and predictor selection would have improved the generalizability of the scores and may enhance pandemic preparedness in the future.

External validation of six COVID-19 prognostic models for predicting mortality risk in older populations in a hospital, primary care, and nursing home setting

OBJECTIVES

To systematically evaluate the performance of COVID-19 prognostic models and scores for mortality risk in older populations across three health-care settings: hospitals, primary care, and nursing homes.

STUDY DESIGN AND SETTING

This retrospective external validation study included 14,092 older individuals of ≥70 years of age with a clinical or polymerase chain reaction-confirmed COVID-19 diagnosis from March 2020 to December 2020. The six validation cohorts include three hospital-based (CliniCo, COVID-OLD, COVID-PREDICT), two primary care-based (Julius General Practitioners Network/Academisch network huisartsgeneeskunde/Network of Academic general Practitioners, PHARMO), and one nursing home cohort (YSIS) in the Netherlands. Based on a living systematic review of COVID-19 prediction models using Prediction model Risk Of Bias ASsessment Tool for quality and risk of bias assessment and considering predictor availability in validation cohorts, we selected six prognostic models predicting mortality risk in adults with COVID-19 infection (GAL-COVID-19 mortality, 4C Mortality Score, National Early Warning Score 2-extended model, Xie model, Wang clinical model, and CURB65 score). All six prognostic models were validated in the hospital cohorts and the GAL-COVID-19 mortality model was validated in all three healthcare settings. The primary outcome was in-hospital mortality for hospitals and 28-day mortality for primary care and nursing home settings. Model performance was evaluated in each validation cohort separately in terms of discrimination, calibration, and decision curves. An intercept update was performed in models indicating miscalibration followed by predictive performance re-evaluation.

MAIN OUTCOME MEASURE

In-hospital mortality for hospitals and 28-day mortality for primary care and nursing home setting.

RESULTS

All six prognostic models performed poorly and showed miscalibration in the older population cohorts. In the hospital settings, model performance ranged from calibration-in-the-large -1.45 to 7.46, calibration slopes 0.24-0.81, and C-statistic 0.55-0.71 with 4C Mortality Score performing as the most discriminative and well-calibrated model. Performance across health-care settings was similar for the GAL-COVID-19 model, with a calibration-in-the-large in the range of -2.35 to -0.15 indicating overestimation, calibration slopes of 0.24-0.81 indicating signs of overfitting, and C-statistic of 0.55-0.71.

CONCLUSION

Our results show that most prognostic models for predicting mortality risk performed poorly in the older population with COVID-19, in each health-care setting: hospital, primary care, and nursing home settings. Insights into factors influencing predictive model performance in the older population are needed for pandemic preparedness and reliable prognostication of health-related outcomes in this demographic.

Identification of Predictors for Clinical Deterioration in Patients With COVID-19 via Electronic Nursing Records: Retrospective Observational Study

BACKGROUND

Few studies have used standardized nursing records with Systematized Nomenclature of Medicine-Clinical Terms (SNOMED CT) to identify predictors of clinical deterioration.

OBJECTIVE

This study aims to standardize the nursing documentation records of patients with COVID-19 using SNOMED CT and identify predictive factors of clinical deterioration in patients with COVID-19 via standardized nursing records.

METHODS

In this study, 57,558 nursing statements from 226 patients with COVID-19 were analyzed. Among these, 45,852 statements were from 207 patients in the stable (control) group and 11,706 from 19 patients in the exacerbated (case) group who were transferred to the intensive care unit within 7 days. The data were collected between December 2019 and June 2022. These nursing statements were standardized using the SNOMED CT International Edition released on November 30, 2022. The 260 unique nursing statements that accounted for the top 90% of 57,558 statements were selected as the mapping source and mapped into SNOMED CT concepts based on their meaning by 2 experts with more than 5 years of SNOMED CT mapping experience. To identify the main features of nursing statements associated with the exacerbation of patient condition, random forest algorithms were used, and optimal hyperparameters were selected for nursing problems or outcomes and nursing procedure-related statements. Additionally, logistic regression analysis was conducted to identify features that determine clinical deterioration in patients with COVID-19.

RESULTS

All nursing statements were semantically mapped to SNOMED CT concepts for "clinical finding," "situation with explicit context," and "procedure" hierarchies. The interrater reliability of the mapping results was 87.7%. The most important features calculated by random forest were "oxygen saturation below reference range," "dyspnea," "tachypnea," and "cough" in "clinical finding," and "oxygen therapy," "pulse oximetry monitoring," "temperature taking," "notification of physician," and "education about isolation for infection control" in "procedure." Among these, "dyspnea" and "inadequate food diet" in "clinical finding" increased clinical deterioration risk (dyspnea: odds ratio [OR] 5.99, 95% CI 2.25-20.29; inadequate food diet: OR 10.0, 95% CI 2.71-40.84), and "oxygen therapy" and "notification of physician" in "procedure" also increased the risk of clinical deterioration in patients with COVID-19 (oxygen therapy: OR 1.89, 95% CI 1.25-3.05; notification of physician: OR 1.72, 95% CI 1.02-2.97).

CONCLUSIONS

The study used SNOMED CT to express and standardize nursing statements. Further, it revealed the importance of standardized nursing records as predictive variables for clinical deterioration in patients.

The reporting completeness and transparency of systematic reviews of prognostic prediction models for COVID-19 was poor: a methodological overview of systematic reviews

OBJECTIVES

To conduct a methodological overview of reviews to evaluate the reporting completeness and transparency of systematic reviews (SRs) of prognostic prediction models (PPMs) for COVID-19.

STUDY DESIGN AND SETTING

MEDLINE, Scopus, Cochrane Database of Systematic Reviews, and Epistemonikos (epistemonikos.org) were searched for SRs of PPMs for COVID-19 until December 31, 2022. The risk of bias in systematic reviews tool was used to assess the risk of bias. The protocol for this overview was uploaded in the Open Science Framework (https://osf.io/7y94c).

RESULTS

Ten SRs were retrieved; none of them synthesized the results in a meta-analysis. For most of the studies, there was absence of a predefined protocol and missing information on study selection, data collection process, and reporting of primary studies and models included, while only one SR had its data publicly available. In addition, for the majority of the SRs, the overall risk of bias was judged as being high. The overall corrected covered area was 6.3% showing a small amount of overlapping among the SRs.

CONCLUSION

The reporting completeness and transparency of SRs of PPMs for COVID-19 was poor. Guidance is urgently required, with increased awareness and education of minimum reporting standards and quality criteria. Specific focus is needed in predefined protocol, information on study selection and data collection process, and in the reporting of findings to improve the quality of SRs of PPMs for COVID-19.

Mitigating the COVID-19 Pandemic through Data-Driven Resource Sharing

COVID-19 outbreaks in local communities can result in a drastic surge in demand for scarce resources such as mechanical ventilators. To deal with such demand surges, many hospitals (1) purchased large quantities of mechanical ventilators, and (2) canceled/postponed elective procedures to preserve care capacity for COVID-19 patients. These measures resulted in a substantial financial burden to the hospitals and poor outcomes for non-COVID-19 patients. Given that COVID-19 transmits at different rates across various regions, there is an opportunity to share portable healthcare resources to mitigate capacity shortages triggered by local outbreaks with fewer total resources. This paper develops a novel data-driven adaptive robust simulation-based optimization (DARSO) methodology for optimal allocation and relocation of mechanical ventilators over different states and regions. Our main methodological contributions lie in a new policy-guided approach and an efficient algorithmic framework that mitigates critical limitations of current robust and stochastic models and make resource-sharing decisions implementable in real-time. In collaboration with epidemiologists and infectious disease doctors, we give proof of concept for the DARSO methodology through a case study of sharing ventilators among regions in Ohio and Michigan. The results suggest that our optimal policy could satisfy ventilator demand during the first pandemic's peak in Ohio and Michigan with 14% (limited sharing) to 63% (full sharing) fewer ventilators compared to a no sharing strategy (status quo), thereby allowing hospitals to preserve more elective procedures. Furthermore, we demonstrate that sharing unused ventilators (rather than purchasing new machines) can result in 5% (limited sharing) to 44% (full sharing) lower expenditure, compared to no sharing, considering the transshipment and new ventilator costs.

Development and Validation of a Robust and Interpretable Early Triaging Support System for Patients Hospitalized With COVID-19: Predictive Algorithm Modeling and Interpretation Study

BACKGROUND

Robust and accurate prediction of severity for patients with COVID-19 is crucial for patient triaging decisions. Many proposed models were prone to either high bias risk or low-to-moderate discrimination. Some also suffered from a lack of clinical interpretability and were developed based on early pandemic period data. Hence, there has been a compelling need for advancements in prediction models for better clinical applicability.

OBJECTIVE

The primary objective of this study was to develop and validate a machine learning-based Robust and Interpretable Early Triaging Support (RIETS) system that predicts severity progression (involving any of the following events: intensive care unit admission, in-hospital death, mechanical ventilation required, or extracorporeal membrane oxygenation required) within 15 days upon hospitalization based on routinely available clinical and laboratory biomarkers.

METHODS

We included data from 5945 hospitalized patients with COVID-19 from 19 hospitals in South Korea collected between January 2020 and August 2022. For model development and external validation, the whole data set was partitioned into 2 independent cohorts by stratified random cluster sampling according to hospital type (general and tertiary care) and geographical location (metropolitan and nonmetropolitan). Machine learning models were trained and internally validated through a cross-validation technique on the development cohort. They were externally validated using a bootstrapped sampling technique on the external validation cohort. The best-performing model was selected primarily based on the area under the receiver operating characteristic curve (AUROC), and its robustness was evaluated using bias risk assessment. For model interpretability, we used Shapley and patient clustering methods.

RESULTS

Our final model, RIETS, was developed based on a deep neural network of 11 clinical and laboratory biomarkers that are readily available within the first day of hospitalization. The features predictive of severity included lactate dehydrogenase, age, absolute lymphocyte count, dyspnea, respiratory rate, diabetes mellitus, c-reactive protein, absolute neutrophil count, platelet count, white blood cell count, and saturation of peripheral oxygen. RIETS demonstrated excellent discrimination (AUROC=0.937; 95% CI 0.935-0.938) with high calibration (integrated calibration index=0.041), satisfied all the criteria of low bias risk in a risk assessment tool, and provided detailed interpretations of model parameters and patient clusters. In addition, RIETS showed potential for transportability across variant periods with its sustainable prediction on Omicron cases (AUROC=0.903, 95% CI 0.897-0.910).

CONCLUSIONS

RIETS was developed and validated to assist early triaging by promptly predicting the severity of hospitalized patients with COVID-19. Its high performance with low bias risk ensures considerably reliable prediction. The use of a nationwide multicenter cohort in the model development and validation implicates generalizability. The use of routinely collected features may enable wide adaptability. Interpretations of model parameters and patients can promote clinical applicability. Together, we anticipate that RIETS will facilitate the patient triaging workflow and efficient resource allocation when incorporated into a routine clinical practice.

The National Early Warning Score 2 with Age and Body Mass Index (NEWS2 Plus) to Determine Patients with Severe COVID-19 Pneumonia

(1) Background: Early identification of severe coronavirus disease 2019 (COVID-19) pneumonia at the initial phase of hospitalization is very crucial. To address this, we validated and updated the National Early Warning Score 2 (NEWS2) for this purpose. (2) Methods: We conducted a study on adult patients with COVID-19 infection in Chiang Mai, Thailand, between May 2021 and October 2021. (3) Results: From a total of 725 COVID-19 adult patients, 350 (48.3%) patients suffered severe COVID-19 pneumonia. In determining severe COVID-19 pneumonia, NEWS2 and NEWS2 + Age + BMI (NEWS2 Plus) showed the C-statistic values of 0.798 (95% CI, 0.767-0.830) and 0.821 (95% CI, 0.791-0.850), respectively. The C-statistic values of NEWS2 Plus were significantly improved compared to those of NEWS2 alone (p = 0.012). Utilizing a cut-off point of five, NEWS2 Plus exhibited better sensitivity and negative predictive value than the traditional NEWS2, with values of 99.7% vs. 83.7% and 98.9% vs. 80.7%, respectively. (4) Conclusions: The incorporation of age and BMI into the traditional NEWS2 score enhanced the efficacy of determining severe COVID-19 pneumonia. Physicians can rely on NEWS2 Plus (NEWS2 + Age + BMI) as a more effective decision-making tool for triaging COVID-19 patients during early hospitalization.

Validation of oxygen saturations measured in the community by emergency medical services as a marker of clinical deterioration in patients with confirmed COVID-19: a retrospective cohort study

OBJECTIVES

To evaluate oxygen saturation and vital signs measured in the community by emergency medical services (EMS) as clinical markers of COVID-19-positive patient deterioration.

DESIGN

A retrospective data analysis.

SETTING

Patients were conveyed by EMS to two hospitals in Hampshire, UK, between 1 March 2020 and 31 July 2020.

PARTICIPANTS

A total of 1080 patients aged ≥18 years with a COVID-19 diagnosis were conveyed by EMS to the hospital.

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary study outcome was admission to the intensive care unit (ICU) within 30 days of conveyance, with a secondary outcome representing mortality within 30 days of conveyance. Receiver operating characteristic (ROC) analysis was performed to evaluate, in a retrospective fashion, the efficacy of different variables in predicting patient outcomes.

RESULTS

Vital signs measured by EMS staff at the first point of contact in the community correlated with patient 30-day ICU admission and mortality. Oxygen saturation was comparably predictive of 30-day ICU admission (area under ROC (AUROC) 0.753; 95% CI 0.668 to 0.826) to the National Early Warning Score 2 (AUROC 0.731; 95% CI 0.655 to 0.800), followed by temperature (AUROC 0.720; 95% CI 0.640 to 0.793) and respiration rate (AUROC 0.672; 95% CI 0.586 to 0.756).

CONCLUSIONS

Initial oxygen saturation measurements (on air) for confirmed COVID-19 patients conveyed by EMS correlated with short-term patient outcomes, demonstrating an AUROC of 0.753 (95% CI 0.668 to 0.826) in predicting 30-day ICU admission. We found that the threshold of 93% oxygen saturation is prognostic of adverse events and of value for clinician decision-making with sensitivity (74.2% CI 0.642 to 0.840) and specificity (70.6% CI 0.678 to 0.734).

Development and external validation of a prediction model for the transition from mild to moderate or severe form of COVID-19

OBJECTIVES

COVID-19 pandemic seems to be under control. However, despite the vaccines, 5 to 10% of the patients with mild disease develop moderate to critical forms with potential lethal evolution. In addition to assess lung infection spread, chest CT helps to detect complications. Developing a prediction model to identify at-risk patients of worsening from mild COVID-19 combining simple clinical and biological parameters with qualitative or quantitative data using CT would be relevant to organizing optimal patient management.

METHODS

Four French hospitals were used for model training and internal validation. External validation was conducted in two independent hospitals. We used easy-to-obtain clinical (age, gender, smoking, symptoms' onset, cardiovascular comorbidities, diabetes, chronic respiratory diseases, immunosuppression) and biological parameters (lymphocytes, CRP) with qualitative or quantitative data (including radiomics) from the initial CT in mild COVID-19 patients.

RESULTS

Qualitative CT scan with clinical and biological parameters can predict which patients with an initial mild presentation would develop a moderate to critical form of COVID-19, with a c-index of 0.70 (95% CI 0.63; 0.77). CT scan quantification improved the performance of the prediction up to 0.73 (95% CI 0.67; 0.79) and radiomics up to 0.77 (95% CI 0.71; 0.83). Results were similar in both validation cohorts, considering CT scans with or without injection.

CONCLUSION

Adding CT scan quantification or radiomics to simple clinical and biological parameters can better predict which patients with an initial mild COVID-19 would worsen than qualitative analyses alone. This tool could help to the fair use of healthcare resources and to screen patients for potential new drugs to prevent a pejorative evolution of COVID-19.

CLINICAL TRIAL REGISTRATION

NCT04481620.

CLINICAL RELEVANCE STATEMENT

CT scan quantification or radiomics analysis is superior to qualitative analysis, when used with simple clinical and biological parameters, to determine which patients with an initial mild presentation of COVID-19 would worsen to a moderate to critical form.

KEY POINTS

• Qualitative CT scan analyses with simple clinical and biological parameters can predict which patients with an initial mild COVID-19 and respiratory symptoms would worsen with a c-index of 0.70. • Adding CT scan quantification improves the performance of the clinical prediction model to an AUC of 0.73. • Radiomics analyses slightly improve the performance of the model to a c-index of 0.77.

Predicting COVID-19 severity: Challenges in reproducibility and deployment of machine learning methods

The increasing use of electronic health records (EHR) based computable phenotypes in clinical research is providing new opportunities for development of data-driven medical applications. Adopted widely in the United States and globally, EHRs facilitate systematic collection of patients' longitudinal information, which serves as one of the important foundations for artificial intelligence applications in medicine. Harmonization of input variables and outcome definitions is critically important for wider clinical applicability of artificial intelligence (AI) methodologies. In this review, we focused on Coronavirus Disease 2019 (COVID-19) severity machine learning prediction models and explored the pipeline for standardizing future disease severity model development using EHR information. We identified 2,967 studies published between 01/01/2020 and 02/15/2022 and selected 135 independent studies that had built machine learning prediction models to predict severity related outcomes of COVID-19 patients based on EHR data for the final review. These 135 studies spanning across 27 counties covered a broad range of severity related prediction outcomes. We observed substantial inconsistency in COVID-19 severity phenotype definitions among models in these studies. Moreover, there was a gap between the outcome of these models and clinician-recognized clinical concepts. Accordingly, we recommend that robust clinical input metrics, with outcome definitions which eliminate ambiguity in interpretation, to reduce algorithmic bias, mitigate model brittleness and improve generalizability of a universal model for COVID-19 severity. This framework can potentially be extended to broader clinical application.

Prior physical illness predicts death better than acute physiological derangement on intensive care unit admission in COVID-19: A Swedish registry study

COVID-19 is associated with prolonged intensive care unit (ICU) stay and considerable mortality. The onset of persistent critical illness, defined as when prior illness predicts death better than acute physiological derangement, has not been studied in COVID-19. This national cohort study based on the Swedish Intensive Care Registry (SIR) included all patients admitted to a Swedish ICU due to COVID-19 from 6 March 2020 to 9 November 2021. Simplified Acute Physiology Score-3 (SAPS3) Box 1 was used as a measure of prior illness and Box 3 as a measure of acute derangement to evaluate the onset and importance of persistent critical illness in COVID-19. To compare predictive capacity, the area under receiver operating characteristic (AUC) of SAPS3 and its constituent Box 1 and 3 was calculated for 30-day mortality. In 7 969 patients, of which 1 878 (23.6%) died within 30 days of ICU admission, the complete SAPS3 score had acceptable discrimination: AUC 0.75 (95% CI 0.74 to 0.76) but showed under prediction in low-risk patients and over prediction in high-risk patients. SAPS3 Box 1 showed markedly better discrimination than Box 3 (AUC 0.74 vs 0.65, P<0,0001). Using custom logistic models, the difference in predictive performance of prior and acute illness was validated, AUC 0.76 vs AUC 0.69, p<0.0001. Prior physical illness predicts death in COVID-19 better than acute physiological derangement during ICU stay, and the whole SAPS3 score is not significantly better than just prior illness. The results suggests that COVID-19 may exhibit similarities to persistent critical illness immediately from ICU admission, potentially because of long median ICU length-of-stay. Alternatively, the variables in the acute physiological derangement model may not adequately capture the severity of illness in COVID-19.

Validity of prognostic models of critical COVID-19 is variable. A systematic review with external validation

OBJECTIVES

To identify prognostic models which estimate the risk of critical COVID-19 in hospitalized patients and to assess their validation properties.

STUDY DESIGN AND SETTING

We conducted a systematic review in Medline (up to January 2021) of studies developing or updating a model that estimated the risk of critical COVID-19, defined as death, admission to intensive care unit, and/or use of mechanical ventilation during admission. Models were validated in two datasets with different backgrounds (HM [private Spanish hospital network], n = 1,753, and ICS [public Catalan health system], n = 1,104), by assessing discrimination (area under the curve [AUC]) and calibration (plots).

RESULTS

We validated 18 prognostic models. Discrimination was good in nine of them (AUCs ≥ 80%) and higher in those predicting mortality (AUCs 65%-87%) than those predicting intensive care unit admission or a composite outcome (AUCs 53%-78%). Calibration was poor in all models providing outcome's probabilities and good in four models providing a point-based score. These four models used mortality as outcome and included age, oxygen saturation, and C-reactive protein among their predictors.

CONCLUSION

The validity of models predicting critical COVID-19 by using only routinely collected predictors is variable. Four models showed good discrimination and calibration when externally validated and are recommended for their use.

Is Age the Most Important Risk Factor in COVID-19 Patients? The Relevance of Comorbidity Burden: A Retrospective Analysis of 10,551 Hospitalizations

Purpose

To assess the contribution of age and comorbidity to the risk of critical illness in hospitalized COVID-19 patients using increasingly exhaustive tools for measuring comorbidity burden.

Patients and Methods

We assessed the effect of age and comorbidity burden in a retrospective, multicenter cohort of patients hospitalized due to COVID-19 in Catalonia (North-East Spain) between March 1, 2020, and January 31, 2022. Vaccinated individuals and those admitted within the first of the six COVID-19 epidemic waves were excluded from the primary analysis but were included in secondary analyses. The primary outcome was critical illness, defined as the need for invasive mechanical ventilation, transfer to the intensive care unit (ICU), or in-hospital death. Explanatory variables included age, sex, and four summary measures of comorbidity burden on admission extracted from three indices: the Charlson index (17 diagnostic group codes), the Elixhauser index and count (31 diagnostic group codes), and the Queralt DxS index (3145 diagnostic group codes). All models were adjusted by wave and center. The proportion of the effect of age attributable to comorbidity burden was assessed using a causal mediation analysis.

Results

The primary analysis included 10,551 hospitalizations due to COVID-19; of them, 3632 (34.4%) experienced critical illness. The frequency of critical illness increased with age and comorbidity burden on admission, irrespective of the measure used. In multivariate analyses, the effect size of age decreased with the number of diagnoses considered to estimate comorbidity burden. When adjusting for the Queralt DxS index, age showed a minimal contribution to critical illness; according to the causal mediation analysis, comorbidity burden on admission explained the 98.2% (95% CI 84.1-117.1%) of the observed effect of age on critical illness.

Conclusion

Comorbidity burden (when measured exhaustively) explains better than chronological age the increased risk of critical illness observed in patients hospitalized with COVID-19.

Validation of the qSOFA and CRB-65 in SARS-CoV-2-infected community-acquired pneumonia

Rationale

Prognostic accuracy of the quick sequential organ failure assessment (qSOFA) and CRB-65 (confusion, respiratory rate, blood pressure and age (≥65 years)) risk scores have not been widely evaluated in patients with SARS-CoV-2-positive compared to SARS-CoV-2-negative community-acquired pneumonia (CAP). The aim of the present study was to validate the qSOFA(-65) and CRB-65 scores in a large cohort of SARS-CoV-2-positive and SARS-CoV-2-negative CAP patients.

Methods

We included all cases with CAP hospitalised in 2020 from the German nationwide mandatory quality assurance programme and compared cases with SARS-CoV-2 infection to cases without. We excluded cases with unclear SARS-CoV-2 infection state, transferred to another hospital or on mechanical ventilation during admission. Predefined outcomes were hospital mortality and need for mechanical ventilation.

Results

Among 68 594 SARS-CoV-2-positive patients, hospital mortality (22.7%) and mechanical ventilation (14.9%) were significantly higher when compared to 167 880 SARS-CoV-2-negative patients (15.7% and 9.2%, respectively). All CRB-65 and qSOFA criteria were associated with both outcomes, and age dominated mortality prediction in SARS-CoV-2 (risk ratio >9). Scores including the age criterion had higher area under the curve (AUCs) for mortality in SARS-CoV-2-positive patients (e.g. CRB-65 AUC 0.76) compared to SARS-CoV-2 negative patients (AUC 0.68), and negative predictive value was highest for qSOFA-65=0 (98.2%). Sensitivity for mechanical ventilation prediction was poor with all scores (AUCs 0.59-0.62), and negative predictive values were insufficient (qSOFA-65=0 missed 1490 out of 10 198 patients (∼15%) with mechanical ventilation). Results were similar when excluding frail and palliative patients.

Conclusions

Hospital mortality and mechanical ventilation rates were higher in SARS-CoV-2-positive than SARS-CoV-2-negative CAP. For SARS-CoV-2-positive CAP, the CRB-65 and qSOFA-65 scores showed adequate prediction of mortality but not of mechanical ventilation.

A study protocol of external validation of eight COVID-19 prognostic models for predicting mortality risk in older populations in a hospital, primary care, and nursing home setting

BACKGROUND

The COVID-19 pandemic has a large impact worldwide and is known to particularly affect the older population. This paper outlines the protocol for external validation of prognostic models predicting mortality risk after presentation with COVID-19 in the older population. These prognostic models were originally developed in an adult population and will be validated in an older population (≥ 70 years of age) in three healthcare settings: the hospital setting, the primary care setting, and the nursing home setting.

METHODS

Based on a living systematic review of COVID-19 prediction models, we identified eight prognostic models predicting the risk of mortality in adults with a COVID-19 infection (five COVID-19 specific models: GAL-COVID-19 mortality, 4C Mortality Score, NEWS2 + model, Xie model, and Wang clinical model and three pre-existing prognostic scores: APACHE-II, CURB65, SOFA). These eight models will be validated in six different cohorts of the Dutch older population (three hospital cohorts, two primary care cohorts, and a nursing home cohort). All prognostic models will be validated in a hospital setting while the GAL-COVID-19 mortality model will be validated in hospital, primary care, and nursing home settings. The study will include individuals ≥ 70 years of age with a highly suspected or PCR-confirmed COVID-19 infection from March 2020 to December 2020 (and up to December 2021 in a sensitivity analysis). The predictive performance will be evaluated in terms of discrimination, calibration, and decision curves for each of the prognostic models in each cohort individually. For prognostic models with indications of miscalibration, an intercept update will be performed after which predictive performance will be re-evaluated.

DISCUSSION

Insight into the performance of existing prognostic models in one of the most vulnerable populations clarifies the extent to which tailoring of COVID-19 prognostic models is needed when models are applied to the older population. Such insight will be important for possible future waves of the COVID-19 pandemic or future pandemics.

Prognostic models in COVID-19 infection that predict severity: a systematic review

Current evidence on COVID-19 prognostic models is inconsistent and clinical applicability remains controversial. We performed a systematic review to summarize and critically appraise the available studies that have developed, assessed and/or validated prognostic models of COVID-19 predicting health outcomes. We searched six bibliographic databases to identify published articles that investigated univariable and multivariable prognostic models predicting adverse outcomes in adult COVID-19 patients, including intensive care unit (ICU) admission, intubation, high-flow nasal therapy (HFNT), extracorporeal membrane oxygenation (ECMO) and mortality. We identified and assessed 314 eligible articles from more than 40 countries, with 152 of these studies presenting mortality, 66 progression to severe or critical illness, 35 mortality and ICU admission combined, 17 ICU admission only, while the remaining 44 studies reported prediction models for mechanical ventilation (MV) or a combination of multiple outcomes. The sample size of included studies varied from 11 to 7,704,171 participants, with a mean age ranging from 18 to 93 years. There were 353 prognostic models investigated, with area under the curve (AUC) ranging from 0.44 to 0.99. A great proportion of studies (61.5%, 193 out of 314) performed internal or external validation or replication. In 312 (99.4%) studies, prognostic models were reported to be at high risk of bias due to uncertainties and challenges surrounding methodological rigor, sampling, handling of missing data, failure to deal with overfitting and heterogeneous definitions of COVID-19 and severity outcomes. While several clinical prognostic models for COVID-19 have been described in the literature, they are limited in generalizability and/or applicability due to deficiencies in addressing fundamental statistical and methodological concerns. Future large, multi-centric and well-designed prognostic prospective studies are needed to clarify remaining uncertainties.

Potential and limitations of machine meta-learning (ensemble) methods for predicting COVID-19 mortality in a large inhospital Brazilian dataset

The majority of early prediction scores and methods to predict COVID-19 mortality are bound by methodological flaws and technological limitations (e.g., the use of a single prediction model). Our aim is to provide a thorough comparative study that tackles those methodological issues, considering multiple techniques to build mortality prediction models, including modern machine learning (neural) algorithms and traditional statistical techniques, as well as meta-learning (ensemble) approaches. This study used a dataset from a multicenter cohort of 10,897 adult Brazilian COVID-19 patients, admitted from March/2020 to November/2021, including patients [median age 60 (interquartile range 48-71), 46% women]. We also proposed new original population-based meta-features that have not been devised in the literature. Stacking has shown to achieve the best results reported in the literature for the death prediction task, improving over previous state-of-the-art by more than 46% in Recall for predicting death, with AUROC 0.826 and MacroF1 of 65.4%. The newly proposed meta-features were highly discriminative of death, but fell short in producing large improvements in final prediction performance, demonstrating that we are possibly on the limits of the prediction capabilities that can be achieved with the current set of ML techniques and (meta-)features. Finally, we investigated how the trained models perform on different hospitals, showing that there are indeed large differences in classifier performance between different hospitals, further making the case that errors are produced by factors that cannot be modeled with the current predictors.

Cardiovascular and Renal Comorbidities Included into Neural Networks Predict the Outcome in COVID-19 Patients Admitted to an Intensive Care Unit: Three-Center, Cross-Validation, Age- and Sex-Matched Study

Here, we performed a multicenter, age- and sex-matched study to compare the efficiency of various machine learning algorithms in the prediction of COVID-19 fatal outcomes and to develop sensitive, specific, and robust artificial intelligence tools for the prompt triage of patients with severe COVID-19 in the intensive care unit setting. In a challenge against other established machine learning algorithms (decision trees, random forests, extra trees, neural networks, k-nearest neighbors, and gradient boosting: XGBoost, LightGBM, and CatBoost) and multivariate logistic regression as a reference, neural networks demonstrated the highest sensitivity, sufficient specificity, and excellent robustness. Further, neural networks based on coronary artery disease/chronic heart failure, stage 3-5 chronic kidney disease, blood urea nitrogen, and C-reactive protein as the predictors exceeded 90% sensitivity and 80% specificity, reaching AUROC of 0.866 at primary cross-validation and 0.849 at secondary cross-validation on virtual samples generated by the bootstrapping procedure. These results underscore the impact of cardiovascular and renal comorbidities in the context of thrombotic complications characteristic of severe COVID-19. As aforementioned predictors can be obtained from the case histories or are inexpensive to be measured at admission to the intensive care unit, we suggest this predictor composition is useful for the triage of critically ill COVID-19 patients.

Rationing scarce healthcare capacity: A study of the ventilator allocation guidelines during the COVID-19 pandemic

In the United States, even though national guidelines for allocating scarce healthcare resources are lacking, 26 states have specific ventilator allocation guidelines to be invoked in case of a shortage. While several states developed their guidelines in response to the recent COVID-19 pandemic, New York State developed these guidelines in 2015 as "pandemic influenza is a foreseeable threat, one that we cannot ignore." The primary objective of this study is to assess the existing procedures and priority rules in place for allocating/rationing scarce ventilator capacity and propose alternative (and improved) priority schemes. We first build machine learning models using inpatient records of COVID-19 patients admitted to New York-Presbyterian/Columbia University Irving Medical Center and an affiliated community health center to predict survival probabilities as well as ventilator length-of-use. Then, we use the resulting point estimators and their uncertainties as inputs for a multiclass priority queueing model with abandonments to assess three priority schemes: (i) SOFA-P (Sequential Organ Failure Assessment based prioritization), which most closely mimics the existing practice by prioritizing patients with sufficiently low SOFA scores; (ii) ISP (incremental survival probability), which assigns priority based on patient-level survival predictions; and (iii) ISP-LU (incremental survival probability per length-of-use), which takes into account survival predictions and resource use duration. Our findings highlight that our proposed priority scheme, ISP-LU, achieves a demonstrable improvement over the other two alternatives. Specifically, the expected number of survivals increases and death risk while waiting for ventilator use decreases. We also show that ISP-LU is a robust priority scheme whose implementation yields a Pareto-improvement over both SOFA-P and ISP in terms of maximizing saved lives after mechanical ventilation while limiting racial disparity in access to the priority queue.

Utility of the 4C ISARIC mortality score in hospitalized COVID-19 patients at a large tertiary Saudi Arabian center

Background

The International Severe Acute Respiratory and Emerging Infections Consortium (ISARIC) 4C mortality score has been used before as a valuable tool for predicting mortality in COVID-19 patients. We aimed to address the utility of the 4C score in a well-defined Saudi population with COVID-19 admitted to a large tertiary referral hospital in Saudi Arabia.

Methods

A retrospective study was conducted that included all adults COVID‑19 patients admitted to the Armed Forces Hospital Southern Region (AFHSR), between January 2021 and September 2022. The receiver operating characteristic (ROC) curve depicted the diagnostic performance of the 4C Score for mortality prediction.

Results

A total of 1,853 patients were enrolled. The ROC curve of the 4C score had an area under the curve of 0.73 (95% CI: 0.702-0.758), p<0.001. The sensitivity and specificity with scores >8 were 80% and 58%, respectively, the positive and negative predictive values were 28% and 93%, respectively. Three hundred and sixteen (17.1%), 638 (34.4%), 814 (43.9%), and 85 (4.6%) patients had low, intermediate, high, and very high values, respectively. There were significant differences between survivors and non-survivors with regard to all variables used in the calculation of the 4C score. Multivariable logistic regression analysis revealed that all components of the 4C score, except gender and O2 saturation, were independent significant predictors of mortality.

Conclusions

Our data support previous international and Saudi studies that the 4C mortality score is a reliable tool with good sensitivity and specificity in the mortality prediction of COVID-19 patients. All components of the 4C score, except gender and O2 saturation, were independent significant predictors of mortality. Within the 4C score, odds ratios increased proportionately with an increase in the score value. Future multi-center prospective studies are warranted.

Overt and Occult Hypoxemia in Patients Hospitalized With COVID-19

Progressive hypoxemia is the predominant mode of deterioration in COVID-19. Among hypoxemia measures, the ratio of the Pao2 to the Fio2 (P/F ratio) has optimal construct validity but poor availability because it requires arterial blood sampling. Pulse oximetry reports oxygenation continuously (ratio of the Spo2 to the Fio2 [S/F ratio]), but it is affected by skin color and occult hypoxemia can occur in Black patients. Oxygen dissociation curves allow noninvasive estimation of P/F ratios (ePFRs) but remain unproven.

OBJECTIVES

Measure overt and occult hypoxemia using ePFR.

DESIGN SETTING AND PARTICIPANTS

We retrospectively studied COVID-19 hospital encounters (n = 5,319) at two academic centers (University of Virginia [UVA] and Emory University).

MAIN OUTCOMES AND MEASURES

We measured primary outcomes (death or ICU transfer within 24 hr), ePFR, conventional hypoxemia measures, baseline predictors (age, sex, race, comorbidity), and acute predictors (National Early Warning Score [NEWS] and Sequential Organ Failure Assessment [SOFA]). We updated predictors every 15 minutes. We assessed predictive validity using adjusted odds ratios (AORs) and area under the receiver operating characteristic curves (AUROCs). We quantified disparities (Black vs non-Black) in empirical cumulative distributions using the Kolmogorov-Smirnov (K-S) two-sample test.

RESULTS

Overt hypoxemia (low ePFR) predicted bad outcomes (AOR for a 100-point ePFR drop: 2.7 [UVA]; 1.7 [Emory]; p < 0.01) with better discrimination (AUROC: 0.76 [UVA]; 0.71 [Emory]) than NEWS (0.70 [both sites]) or SOFA (0.68 [UVA]; 0.65 [Emory]) and similar to S/F ratio (0.76 [UVA]; 0.70 [Emory]). We found racial differences consistent with occult hypoxemia. Black patients had better apparent oxygenation (K-S distance: 0.17 [both sites]; p < 0.01) but, for comparable ePFRs, worse outcomes than other patients (AOR: 2.2 [UVA]; 1.2 [Emory]; p < 0.01).

CONCLUSIONS AND RELEVANCE

The ePFR was a valid measure of overt hypoxemia. In COVID-19, it may outperform multi-organ dysfunction models. By accounting for biased oximetry as well as clinicians' real-time responses to it (supplemental oxygen adjustment), ePFRs may reveal racial disparities attributable to occult hypoxemia.

Relevance of prediction scores derived from the SARS-CoV-2 first wave, in the evolving UK COVID-19 second wave, for safe early discharge and mortality: a PREDICT COVID-19 UK prospective observational cohort study

OBJECTIVE

Prospectively validate prognostication scores, SOARS and 4C Mortality Score, derived from the COVID-19 first wave, for mortality and safe early discharge in the evolving pandemic with SARS-CoV-2 variants (B.1.1.7 replacing D614) and healthcare responses altering patient demographic and mortality.

DESIGN

Protocol-based prospective observational cohort study.

SETTING

Single site PREDICT and multisite ISARIC (International Severe Acute Respiratory and Emerging Infections Consortium) cohorts in UK COVID-19 second wave, October 2020 to January 2021.

PARTICIPANTS

1383 PREDICT and 20 595 ISARIC SARS-CoV-2 patients.

PRIMARY OUTCOME MEASURES

Relevance of SOARS and 4C Mortality Score determining in-hospital mortality and safe early discharge in the evolving UK COVID-19 second wave.

RESULTS

1383 (median age 67 years, IQR 52-82; mortality 24.7%) PREDICT and 20 595 (mortality 19.4%) ISARIC patient cohorts showed SOARS had area under the curve (AUC) of 0.8 and 0.74, while 4C Mortality Score had AUC of 0.83 and 0.91 for hospital mortality, in the PREDICT and ISARIC cohorts respectively, therefore, effective in evaluating safe discharge and in-hospital mortality. 19.3% (231/1195, PREDICT cohort) and 16.7% (2550/14992, ISARIC cohort) with SOARS of 0-1 were candidates for safe discharge to a virtual hospital (VH) model. SOARS implementation in the VH pathway resulted in low readmission, 11.8% (27/229) and low mortality, 0.9% (2/229). Use to prevent admission is still suboptimal, as 8.1% in the PREDICT cohort and 9.5% in the ISARIC cohort were admitted despite SOARS score of 0-1.

CONCLUSIONS

SOARS and 4C Mortality Score remains valid, transforming complex clinical presentations into tangible numbers, aiding objective decision making, despite SARS-CoV-2 variants and healthcare responses altering patient demographic and mortality. Both scores, easily implemented within urgent care pathways for safe early discharge, allocate hospital resources appropriately to the pandemic's needs while enabling normal healthcare services resumption.

Artificial intelligence-based model for COVID-19 prognosis incorporating chest radiographs and clinical data; a retrospective model development and validation study

OBJECTIVES

The purpose of this study was to develop an artificial intelligence-based model to prognosticate COVID-19 patients at admission by combining clinical data and chest radiographs.

METHODS

This retrospective study used the Stony Brook University COVID-19 dataset of 1384 inpatients. After exclusions, 1356 patients were randomly divided into training (1083) and test datasets (273). We implemented three artificial intelligence models, which classified mortality, ICU admission, or ventilation risk. Each model had three submodels with different inputs: clinical data, chest radiographs, and both. We showed the importance of the variables using SHapley Additive exPlanations (SHAP) values.

RESULTS

The mortality prediction model was best overall with area under the curve, sensitivity, specificity, and accuracy of 0.79 (0.72-0.86), 0.74 (0.68-0.79), 0.77 (0.61-0.88), and 0.74 (0.69-0.79) for the clinical data-based model; 0.77 (0.69-0.85), 0.67 (0.61-0.73), 0.81 (0.67-0.92), 0.70 (0.64-0.75) for the image-based model, and 0.86 (0.81-0.91), 0.76 (0.70-0.81), 0.77 (0.61-0.88), 0.76 (0.70-0.81) for the mixed model. The mixed model had the best performance (p value < 0.05). The radiographs ranked fourth for prognostication overall, and first of the inpatient tests assessed.

CONCLUSIONS

These results suggest that prognosis models become more accurate if AI-derived chest radiograph features and clinical data are used together.

ADVANCES IN KNOWLEDGE

This AI model evaluates chest radiographs together with clinical data in order to classify patients as having high or low mortality risk. This work shows that chest radiographs taken at admission have significant COVID-19 prognostic information compared to clinical data other than age and sex.

Predicting COVID-19 prognosis in the ICU remained challenging: external validation in a multinational regional cohort

OBJECTIVES

Many prediction models for coronavirus disease 2019 (COVID-19) have been developed. External validation is mandatory before implementation in the intensive care unit (ICU). We selected and validated prognostic models in the Euregio Intensive Care COVID (EICC) cohort.

STUDY DESIGN AND SETTING

In this multinational cohort study, routine data from COVID-19 patients admitted to ICUs within the Euregio Meuse-Rhine were collected from March to August 2020. COVID-19 models were selected based on model type, predictors, outcomes, and reporting. Furthermore, general ICU scores were assessed. Discrimination was assessed by area under the receiver operating characteristic curves (AUCs) and calibration by calibration-in-the-large and calibration plots. A random-effects meta-analysis was used to pool results.

RESULTS

551 patients were admitted. Mean age was 65.4 ± 11.2 years, 29% were female, and ICU mortality was 36%. Nine out of 238 published models were externally validated. Pooled AUCs were between 0.53 and 0.70 and calibration-in-the-large between -9% and 6%. Calibration plots showed generally poor but, for the 4C Mortality score and Spanish Society of Infectious Diseases and Clinical Microbiology (SEIMC) score, moderate calibration.

CONCLUSION

Of the nine prognostic models that were externally validated in the EICC cohort, only two showed reasonable discrimination and moderate calibration. For future pandemics, better models based on routine data are needed to support admission decision-making.

Association of miR-144 levels in the peripheral blood with COVID-19 severity and mortality

Coronavirus disease-2019 (COVID-19) can be asymptomatic or lead to a wide symptom spectrum, including multi-organ damage and death. Here, we explored the potential of microRNAs in delineating patient condition and predicting clinical outcome. Plasma microRNA profiling of hospitalized COVID-19 patients showed that miR-144-3p was dynamically regulated in response to COVID-19. Thus, we further investigated the biomarker potential of miR-144-3p measured at admission in 179 COVID-19 patients and 29 healthy controls recruited in three centers. In hospitalized patients, circulating miR-144-3p levels discriminated between non-critical and critical illness (AUC_miR-144-3p = 0.71; p = 0.0006), acting also as mortality predictor (AUC_miR-144-3p = 0.67; p = 0.004). In non-hospitalized patients, plasma miR-144-3p levels discriminated mild from moderate disease (AUC_miR-144-3p = 0.67; p = 0.03). Uncontrolled release of pro-inflammatory cytokines can lead to clinical deterioration. Thus, we explored the added value of a miR-144/cytokine combined analysis in the assessment of hospitalized COVID-19 patients. A miR-144-3p/Epidermal Growth Factor (EGF) combined score discriminated between non-critical and critical hospitalized patients (AUC_{miR-144-3p/EGF} = 0.81; p < 0.0001); moreover, a miR-144-3p/Interleukin-10 (IL-10) score discriminated survivors from nonsurvivors (AUC_{miR-144-3p/IL-10} = 0.83; p < 0.0001). In conclusion, circulating miR-144-3p, possibly in combination with IL-10 or EGF, emerges as a noninvasive tool for early risk-based stratification and mortality prediction in COVID-19.

Development and validation of self-monitoring auto-updating prognostic models of survival for hospitalized COVID-19 patients

Clinical prognostic models can assist patient care decisions. However, their performance can drift over time and location, necessitating model monitoring and updating. Despite rapid and significant changes during the pandemic, prognostic models for COVID-19 patients do not currently account for these drifts. We develop a framework for continuously monitoring and updating prognostic models and apply it to predict 28-day survival in COVID-19 patients. We use demographic, laboratory, and clinical data from electronic health records of 34912 hospitalized COVID-19 patients from March 2020 until May 2022 and compare three modeling methods. Model calibration performance drift is immediately detected with minor fluctuations in discrimination. The overall calibration on the prospective validation cohort is significantly improved when comparing the dynamically updated models against their static counterparts. Our findings suggest that, using this framework, models remain accurate and well-calibrated across various waves, variants, race and sex and yield positive net-benefits.

Physiologic Scoring Systems in Predicting the COVID-19 Patients' one-month Mortality; a Prognostic Accuracy Study

Introduction : It is critical to quickly and easily identify severe coronavirus disease 2019 (COVID-19) patients and predict their mortality. This study aimed to determine the accuracy of the physiologic scoring systems in predicting the mortality of COVID-19 patients. Methods: This prospective cross-sectional study was performed on COVID-19 patients admitted to the emergency department (ED). The clinical characteristics of the participants were collected by the emergency physicians and the accuracy of the Quick Sequential Failure Assessment (qSOFA), Coronavirus Clinical Characterization Consortium (4C) Mortality, National Early Warning Score-2 (NEWS2), and Pandemic Respiratory Infection Emergency System Triage (PRIEST) scores for mortality prediction was evaluated. Results: Nine hundred and twenty-one subjects were included. Of whom, 745 (80.9%) patients survived after 30 days of admission. The mean age of patients was 59.13 ± 17.52 years, and 550 (61.6%) subjects were male. Non-Survived patients were significantly older (66.02 ± 17.80 vs. 57.45 ± 17.07, P< 0.001) and had more comorbidities (diabetes mellitus, respiratory, cardiovascular, and cerebrovascular disease) in comparison with survived patients. For COVID-19 mortality prediction, the AUROCs of PRIEST, qSOFA, NEWS2, and 4C Mortality score were 0.846 (95% CI [0.821-0.868]), 0.788 (95% CI [0.760-0.814]), 0.843 (95% CI [0.818-0.866]), and 0.804 (95% CI [0.776-0.829]), respectively. All scores were good predictors of COVID-19 mortality. Conclusion: All studied physiologic scores were good predictors of COVID-19 mortality and could be a useful screening tool for identifying high-risk patients. The NEWS2 and PRIEST scores predicted mortality in COVID-19 patients significantly better than qSOFA.

Prevalence and Outcomes of COVID -19 Patients with Happy Hypoxia: A Systematic Review

Background

In Coronavirus disease 2019 (COVID-19), some patients have low oxygen saturation without any dyspnea. This has been termed “happy hypoxia.” No worldwide prevalence survey of this phenomenon has been conducted. This review aimed to summarize information on the prevalence, risk factors, and outcomes of patients with happy hypoxia to improve their management.

Methods

We conducted a systematic search of electronic databases for all studies published up to April 30, 2022. We included high-quality studies using the Newcastle-Ottawa Scale (NOS) tool for qualitative assessment of searches. The prevalence of happy hypoxia, as well as the mortality rate of patients with happy hypoxia, were estimated by pooled analysis and heterogeneity by I².

Results

Of the 25,086 COVID-19 patients from the 7 studies, the prevalence of happy hypoxia ranged from 4.8 to 65%. The pooled prevalence was 6%. Happy hypoxia was associated with age > 65 years, male sex, body mass index (BMI)> 25 kg/m2, smoking, chronic obstructive pulmonary disease, diabetes mellitus, high respiratory rate, and high d-dimer. Mortality ranged from 01 to 45.4%. The pooled mortality was 2%. In 2 studies, patients with dyspnea were admitted to intensive care more often than those with happy hypoxia. One study reported that the length of stay in intensive care did not differ between patients with dyspnea and those with happy hypoxia at admission. One study reported the need for extracorporeal membrane oxygenation (ECMO) in patients with happy hypoxia.

Conclusion

The pooled prevalence and mortality of patients with happy hypoxia were not very high. Happy hypoxia was associated with advanced age and comorbidities. Some patients were admitted to the intensive care unit, although fewer than dyspneic patients. Its early detection and management should improve the prognosis.

Optimising precision and power by machine learning in randomised trials with ordinal and time-to-event outcomes with an application to COVID-19

The rapid finding of effective therapeutics requires efficient use of available resources in clinical trials. Covariate adjustment can yield statistical estimates with improved precision, resulting in a reduction in the number of participants required to draw futility or efficacy conclusions. We focus on time-to-event and ordinal outcomes. When more than a few baseline covariates are available, a key question for covariate adjustment in randomised studies is how to fit a model relating the outcome and the baseline covariates to maximise precision. We present a novel theoretical result establishing conditions for asymptotic normality of a variety of covariate-adjusted estimators that rely on machine learning (e.g.,ℓ 1 -regularisation, Random Forests, XGBoost, and Multivariate Adaptive Regression Splines [MARS]), under the assumption that outcome data are missing completely at random. We further present a consistent estimator of the asymptotic variance. Importantly, the conditions do not require the machine learning methods to converge to the true outcome distribution conditional on baseline variables, as long as they converge to some (possibly incorrect) limit. We conducted a simulation study to evaluate the performance of the aforementioned prediction methods in COVID-19 trials. Our simulation is based on resampling longitudinal data from over 1500 patients hospitalised with COVID-19 at Weill Cornell Medicine New York Presbyterian Hospital. We found that usingℓ 1 -regularisation led to estimators and corresponding hypothesis tests that control type 1 error and are more precise than an unadjusted estimator across all sample sizes tested. We also show that when covariates are not prognostic of the outcome,ℓ 1 -regularisation remains as precise as the unadjusted estimator, even at small sample sizes (n = 100 ). We give an R package adjrct that performs model-robust covariate adjustment for ordinal and time-to-event outcomes.

Assess and validate predictive performance of models for in-hospital mortality in COVID-19 patients: A retrospective cohort study in the Netherlands comparing the value of registry data with high-granular electronic health records

PURPOSE

To assess, validate and compare the predictive performance of models for in-hospital mortality of COVID-19 patients admitted to the intensive care unit (ICU) over two different waves of infections. Our models were built with high-granular Electronic Health Records (EHR) data versus less-granular registry data.

METHODS

Observational study of all COVID-19 patients admitted to 19 Dutch ICUs participating in both the national quality registry National Intensive Care Evaluation (NICE) and the EHR-based Dutch Data Warehouse (hereafter EHR). Multiple models were developed on data from the first 24 h of ICU admissions from February to June 2020 (first COVID-19 wave) and validated on prospective patients admitted to the same ICUs between July and December 2020 (second COVID-19 wave). We assessed model discrimination, calibration, and the degree of relatedness between development and validation population. Coefficients were used to identify relevant risk factors.

RESULTS

A total of 1533 patients from the EHR and 1563 from the registry were included. With high granular EHR data, the average AUROC was 0.69 (standard deviation of 0.05) for the internal validation, and the AUROC was 0.75 for the temporal validation. The registry model achieved an average AUROC of 0.76 (standard deviation of 0.05) in the internal validation and 0.77 in the temporal validation. In the EHR data, age, and respiratory-system related variables were the most important risk factors identified. In the NICE registry data, age and chronic respiratory insufficiency were the most important risk factors.

CONCLUSION

In our study, prognostic models built on less-granular but readily-available registry data had similar performance to models built on high-granular EHR data and showed similar transportability to a prospective COVID-19 population. Future research is needed to verify whether this finding can be confirmed for upcoming waves.

Development and validation of a dynamic 48-hour in-hospital mortality risk stratification for COVID-19 in a UK teaching hospital: a retrospective cohort study

OBJECTIVES

To develop a disease stratification model for COVID-19 that updates according to changes in a patient's condition while in hospital to facilitate patient management and resource allocation.

DESIGN

In this retrospective cohort study, we adopted a landmarking approach to dynamic prediction of all-cause in-hospital mortality over the next 48 hours. We accounted for informative predictor missingness and selected predictors using penalised regression.

SETTING

All data used in this study were obtained from a single UK teaching hospital.

PARTICIPANTS

We developed the model using 473 consecutive patients with COVID-19 presenting to a UK hospital between 1 March 2020 and 12 September 2020; and temporally validated using data on 1119 patients presenting between 13 September 2020 and 17 March 2021.

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcome is all-cause in-hospital mortality within 48 hours of the prediction time. We accounted for the competing risks of discharge from hospital alive and transfer to a tertiary intensive care unit for extracorporeal membrane oxygenation.

RESULTS

Our final model includes age, Clinical Frailty Scale score, heart rate, respiratory rate, oxygen saturation/fractional inspired oxygen ratio, white cell count, presence of acidosis (pH <7.35) and interleukin-6. Internal validation achieved an area under the receiver operating characteristic (AUROC) of 0.90 (95% CI 0.87 to 0.93) and temporal validation gave an AUROC of 0.86 (95% CI 0.83 to 0.88).

CONCLUSIONS

Our model incorporates both static risk factors (eg, age) and evolving clinical and laboratory data, to provide a dynamic risk prediction model that adapts to both sudden and gradual changes in an individual patient's clinical condition. On successful external validation, the model has the potential to be a powerful clinical risk assessment tool.

TRIAL REGISTRATION

The study is registered as 'researchregistry5464' on the Research Registry (www.researchregistry.com).

Site of care and factors associated with mortality in unvaccinated Australian aged care residents during COVID-19 outbreaks

BACKGROUND

Residential InReach presents an alternative to hospital admission for aged care residents swabbed for COVID-19, though relative outcomes remain unknown.

AIMS

To compare rates and predictors of 28-day mortality for aged care residents seen by InReach with COVID-19, or 'suspected COVID-19' ('sCOVID'), including hospital vs InReach-based care.

METHODS

Prospective observational study of consecutive patients referred to a Victorian InReach service meeting COVID-19 testing criteria between April-October 2020 (pre-vaccine availability). COVID-19 was determined by positive polymerase chain reaction testing on nasopharyngeal swab. sCOVID-19 was defined as meeting symptomatic Victorian Government testing criteria but persistently swab negative.

RESULTS

There were no significant differences in age, gender, Clinical Frailty Score (CFS) or Charlson Comorbidity Index (CCI) between 152 patients with COVID-19 and 118 patients with sCOVID. 28-day mortality was similar between patients with COVID-19 (35/152, 23%) and sCOVID (32/118, 27%) (p=0.4). For the combined cohort, 28-day mortality was associated with initial oxygen saturation (p<0.001), delirium (p<0.001), hospital transfer for acuity (p=0.02; but not public health/facility reasons), CFS (p=0.04), prior ischaemic heart disease (p=0.01) and dementia (p=0.02). For COVID-19 patients, 28-day mortality was associated with initial oxygen saturation (p=0.02), delirium (p<0.001), and hospital transfer for acuity (p=0.01), but not public health/facility reasons.

CONCLUSION

Unvaccinated aged care residents meeting COVID-19 testing criteria seen by InReach during a pandemic experience high mortality rates, including with negative swab result. Residents remaining within-facility (with InReach) experienced similar adjusted mortality odds to residents transferred to hospital for public health/facility-based reasons, and lower than those transferred for clinical acuity. This article is protected by copyright. All rights reserved.

Clinical prediction models in hospitalized patients with COVID-19: A multicenter cohort study

Background

Clinical spectrum of novel coronavirus disease (COVID-19) ranges from asymptomatic infection to severe respiratory failure that may result in death. We aimed at validating and potentially improve existing clinical models to predict prognosis in hospitalized patients with acute COVID-19.

Methods

Consecutive patients with acute confirmed COVID-19 pneumonia hospitalized at 5 Italian non-intensive care unit centers during the 2020 outbreak were included in the study. Twelve validated prognostic scores for pneumonia and/or sepsis and specific COVID-19 scores were calculated for each study patient and their accuracy was compared in predicting in-hospital death at 30 days and the composite of death and orotracheal intubation.

Results

During hospital stay, 302 of 1044 included patients presented critical illness (28.9%), and 226 died (21.6%). Nine out of 34 items included in different prognostic scores were independent predictors of all-cause-death. The discrimination was acceptable for the majority of scores (APACHE II, COVID-GRAM, REMS, CURB-65, NEWS II, ROX-index, 4C, SOFA) to predict in-hospital death at 30 days and poor for the rest. A high negative predictive value was observed for REMS (100.0%) and 4C (98.7%) scores; the positive predictive value was poor overall, ROX-index having the best value (75.0%).

Conclusions

Despite the growing interest in prognostic models, their performance in patients with COVID-19 is modest. The 4C, REMS and ROX-index may have a role to select high and low risk patients at admission. However, simple predictors as age and PaO2/FiO2 ratio can also be useful as standalone predictors to inform decision making.

Using patient biomarker time series to determine mortality risk in hospitalised COVID-19 patients: A comparative analysis across two New York hospitals

A large range of prognostic models for determining the risk of COVID-19 patient mortality exist, but these typically restrict the set of biomarkers considered to measurements available at patient admission. Additionally, many of these models are trained and tested on patient cohorts from a single hospital, raising questions about the generalisability of results. We used a Bayesian Markov model to analyse time series data of biomarker measurements taken throughout the duration of a COVID-19 patient's hospitalisation for n = 1540 patients from two hospitals in New York: State University of New York (SUNY) Downstate Health Sciences University and Maimonides Medical Center. Our main focus was to quantify the mortality risk associated with both static (e.g. demographic and patient history variables) and dynamic factors (e.g. changes in biomarkers) throughout hospitalisation, by so doing, to explain the observed patterns of mortality. By using our model to make predictions across the hospitals, we assessed how predictive factors generalised between the two cohorts. The individual dynamics of the measurements and their associated mortality risk were remarkably consistent across the hospitals. The model accuracy in predicting patient outcome (death or discharge) was 72.3% (predicting SUNY; posterior median accuracy) and 71.3% (predicting Maimonides) respectively. Model sensitivity was higher for detecting patients who would go on to be discharged (78.7%) versus those who died (61.8%). Our results indicate the utility of including dynamic clinical measurements when assessing patient mortality risk but also highlight the difficulty of identifying high risk patients.

Validation of a deep learning, value-based care model to predict mortality and comorbidities from chest radiographs in COVID-19

We validate a deep learning model predicting comorbidities from frontal chest radiographs (CXRs) in patients with coronavirus disease 2019 (COVID-19) and compare the model's performance with hierarchical condition category (HCC) and mortality outcomes in COVID-19. The model was trained and tested on 14,121 ambulatory frontal CXRs from 2010 to 2019 at a single institution, modeling select comorbidities using the value-based Medicare Advantage HCC Risk Adjustment Model. Sex, age, HCC codes, and risk adjustment factor (RAF) score were used. The model was validated on frontal CXRs from 413 ambulatory patients with COVID-19 (internal cohort) and on initial frontal CXRs from 487 COVID-19 hospitalized patients (external cohort). The discriminatory ability of the model was assessed using receiver operating characteristic (ROC) curves compared to the HCC data from electronic health records, and predicted age and RAF score were compared using correlation coefficient and absolute mean error. The model predictions were used as covariables in logistic regression models to evaluate the prediction of mortality in the external cohort. Predicted comorbidities from frontal CXRs, including diabetes with chronic complications, obesity, congestive heart failure, arrhythmias, vascular disease, and chronic obstructive pulmonary disease, had a total area under ROC curve (AUC) of 0.85 (95% CI: 0.85-0.86). The ROC AUC of predicted mortality for the model was 0.84 (95% CI,0.79-0.88) for the combined cohorts. This model using only frontal CXRs predicted select comorbidities and RAF score in both internal ambulatory and external hospitalized COVID-19 cohorts and was discriminatory of mortality, supporting its potential use in clinical decision making.

A multiplex protein panel assay for severity prediction and outcome prognosis in patients with COVID-19: An observational multi-cohort study

Background

Global healthcare systems continue to be challenged by the COVID-19 pandemic, and there is a need for clinical assays that can help optimise resource allocation, support treatment decisions, and accelerate the development and evaluation of new therapies.

Methods

We developed a multiplexed proteomics assay for determining disease severity and prognosis in COVID-19. The assay quantifies up to 50 peptides, derived from 30 known and newly introduced COVID-19-related protein markers, in a single measurement using routine-lab compatible analytical flow rate liquid chromatography and multiple reaction monitoring (LC-MRM). We conducted two observational studies in patients with COVID-19 hospitalised at Charité - Universitätsmedizin Berlin, Germany before (from March 1 to 26, 2020, n=30) and after (from April 4 to November 19, 2020, n=164) dexamethasone became standard of care. The study is registered in the German and the WHO International Clinical Trials Registry (DRKS00021688).

Findings

The assay produces reproducible (median inter-batch CV of 10.9%) absolute quantification of 47 peptides with high sensitivity (median LLOQ of 143 ng/ml) and accuracy (median 96.8%). In both studies, the assay reproducibly captured hallmarks of COVID-19 infection and severity, as it distinguished healthy individuals, mild, moderate, and severe COVID-19. In the post-dexamethasone cohort, the assay predicted survival with an accuracy of 0.83 (108/130), and death with an accuracy of 0.76 (26/34) in the median 2.5 weeks before the outcome, thereby outperforming compound clinical risk assessments such as SOFA, APACHE II, and ABCS scores.

Interpretation

Disease severity and clinical outcomes of patients with COVID-19 can be stratified and predicted by the routine-applicable panel assay that combines known and novel COVID-19 biomarkers. The prognostic value of this assay should be prospectively assessed in larger patient cohorts for future support of clinical decisions, including evaluation of sample flow in routine setting. The possibility to objectively classify COVID-19 severity can be helpful for monitoring of novel therapies, especially in early clinical trials.

Funding

This research was funded in part by the European Research Council (ERC) under grant agreement ERC-SyG-2020 951475 (to M.R) and by the Wellcome Trust (IA 200829/Z/16/Z to M.R.). The work was further supported by the Ministry of Education and Research (BMBF) as part of the National Research Node 'Mass Spectrometry in Systems Medicine (MSCoresys)', under grant agreements 031L0220 and 161L0221. J.H. was supported by a Swiss National Science Foundation (SNSF) Postdoc Mobility fellowship (project number 191052). This study was further supported by the BMBF grant NaFoUniMedCOVID-19 - NUM-NAPKON, FKZ: 01KX2021. The study was co-funded by the UK's innovation agency, Innovate UK, under project numbers 75594 and 56328.

A machine learning-based approach to determine infection status in recipients of BBV152 (Covaxin) whole-virion inactivated SARS-CoV-2 vaccine for serological surveys

Data science has been an invaluable part of the COVID-19 pandemic response with multiple applications, ranging from tracking viral evolution to understanding the vaccine effectiveness. Asymptomatic breakthrough infections have been a major problem in assessing vaccine effectiveness in populations globally. Serological discrimination of vaccine response from infection has so far been limited to Spike protein vaccines since whole virion vaccines generate antibodies against all the viral proteins. Here, we show how a statistical and machine learning (ML) based approach can be used to discriminate between SARS-CoV-2 infection and immune response to an inactivated whole virion vaccine (BBV152, Covaxin). For this, we assessed serial data on antibodies against Spike and Nucleocapsid antigens, along with age, sex, number of doses taken, and days since last dose, for 1823 Covaxin recipients. An ensemble ML model, incorporating a consensus clustering approach alongside the support vector machine model, was built on 1063 samples where reliable qualifying data existed, and then applied to the entire dataset. Of 1448 self-reported negative subjects, our ensemble ML model classified 724 to be infected. For method validation, we determined the relative ability of a random subset of samples to neutralize Delta versus wild-type strain using a surrogate neutralization assay. We worked on the premise that antibodies generated by a whole virion vaccine would neutralize wild type more efficiently than delta strain. In 100 of 156 samples, where ML prediction differed from self-reported uninfected status, neutralization against Delta strain was more effective, indicating infection. We found 71.8% subjects predicted to be infected during the surge, which is concordant with the percentage of sequences classified as Delta (75.6%-80.2%) over the same period. Our approach will help in real-world vaccine effectiveness assessments where whole virion vaccines are commonly used.

Exposing and Overcoming Limitations of Clinical Laboratory Tests in COVID-19 by Adding Immunological Parameters; A Retrospective Cohort Analysis and Pilot Study

BACKGROUND

Two years since the onset of the COVID-19 pandemic no predictive algorithm has been generally adopted for clinical management and in most algorithms the contribution of laboratory variables is limited.

OBJECTIVES

To measure the predictive performance of currently used clinical laboratory tests alone or combined with clinical variables and explore the predictive power of immunological tests adequate for clinical laboratories. Methods: Data from 2,600 COVID-19 patients of the first wave of the pandemic in the Barcelona area (exploratory cohort of 1,579, validation cohorts of 598 and 423 patients) including clinical parameters and laboratory tests were retrospectively collected. 28-day survival and maximal severity were the main outcomes considered in the multiparametric classical and machine learning statistical analysis. A pilot study was conducted in two subgroups (n=74 and n=41) measuring 17 cytokines and 27 lymphocyte phenotypes respectively.

FINDINGS

1) Despite a strong association of clinical and laboratory variables with the outcomes in classical pairwise analysis, the contribution of laboratory tests to the combined prediction power was limited by redundancy. Laboratory variables reflected only two types of processes: inflammation and organ damage but none reflected the immune response, one major determinant of prognosis. 2) Eight of the thirty variables: age, comorbidity index, oxygen saturation to fraction of inspired oxygen ratio, neutrophil-lymphocyte ratio, C-reactive protein, aspartate aminotransferase/alanine aminotransferase ratio, fibrinogen, and glomerular filtration rate captured most of the combined statistical predictive power. 3) The interpretation of clinical and laboratory variables was moderately improved by grouping them in two categories i.e., inflammation related biomarkers and organ damage related biomarkers; Age and organ damage-related biomarker tests were the best predictors of survival, and inflammatory-related ones were the best predictors of severity. 4) The pilot study identified immunological tests (CXCL10, IL-6, IL-1RA and CCL2), that performed better than most currently used laboratory tests.

CONCLUSIONS

Laboratory tests for clinical management of COVID 19 patients are valuable but limited predictors due to redundancy; this limitation could be overcome by adding immunological tests with independent predictive power. Understanding the limitations of tests in use would improve their interpretation and simplify clinical management but a systematic search for better immunological biomarkers is urgent and feasible.