Predictive Models of Mortality for Hospitalized Patients With COVID-19: Retrospective Cohort Study

Accuracy of automated computer-aided risk scoring systems to estimate the risk of COVID-19: a retrospective cohort study

BACKGROUND

In the UK National Health Service (NHS), the patient's vital signs are monitored and summarised into a National Early Warning Score (NEWS) score. A set of computer-aided risk scoring systems (CARSS) was developed and validated for predicting in-hospital mortality and sepsis in unplanned admission to hospital using NEWS and routine blood tests results. We sought to assess the accuracy of these models to predict the risk of COVID-19 in unplanned admissions during the first phase of the pandemic.

METHODS

Adult ( > = 18 years) non-elective admissions discharged (alive/deceased) between 11-March-2020 to 13-June-2020 from two acute hospitals with an index NEWS electronically recorded within ± 24 h of admission. We identified COVID-19 admission based on ICD-10 code 'U071' which was determined by COVID-19 swab test results (hospital or community). We assessed the performance of CARSS (CARS_N, CARS_NB, CARM_N, CARM_NB) for predicting the risk of COVID-19 in terms of discrimination (c-statistic) and calibration (graphically).

RESULTS

The risk of in-hospital mortality following emergency medical admission was 8.4% (500/6444) and 9.6% (620/6444) had a diagnosis of COVID-19. For predicting COVID-19 admissions, the CARS_N model had the highest discrimination 0.73 (0.71 to 0.75) and calibration slope 0.81 (0.72 to 0.89) compared to other CARSS models: CARM_N (discrimination:0.68 (0.66 to 0.70) and calibration slope 0.47 (0.41 to 0.54)), CARM_NB (discrimination:0.68 (0.65 to 0.70) and calibration slope 0.37 (0.31 to 0.43)), and CARS_NB (discrimination:0.68 (0.66 to 0.70) and calibration slope 0.56 (0.47 to 0.64)).

CONCLUSIONS

The CARS_N model is reasonably accurate for predicting the risk of COVID-19. It may be clinically useful as an early warning system at the time of admission especially to triage large numbers of unplanned admissions because it requires no additional data collection and is readily automated.

Predictive Model for Mortality in Severe COVID-19 Patients across the Six Pandemic Waves

The impact of SARS-CoV-2 infection remains substantial on a global scale, despite widespread vaccination efforts, early therapeutic interventions, and an enhanced understanding of the disease's underlying mechanisms. At the same time, a significant number of patients continue to develop severe COVID-19, necessitating admission to intensive care units (ICUs). This study aimed to provide evidence concerning the most influential predictors of mortality among critically ill patients with severe COVID-19, employing machine learning (ML) techniques. To accomplish this, we conducted a retrospective multicenter investigation involving 684 patients with severe COVID-19, spanning from 1 June 2020 to 31 March 2023, wherein we scrutinized sociodemographic, clinical, and analytical data. These data were extracted from electronic health records. Out of the six supervised ML methods scrutinized, the extreme gradient boosting (XGB) method exhibited the highest balanced accuracy at 96.61%. The variables that exerted the greatest influence on mortality prediction encompassed ferritin, fibrinogen, D-dimer, platelet count, C-reactive protein (CRP), prothrombin time (PT), invasive mechanical ventilation (IMV), PaFi (PaO2/FiO2), lactate dehydrogenase (LDH), lymphocyte levels, activated partial thromboplastin time (aPTT), body mass index (BMI), creatinine, and age. These findings underscore XGB as a robust candidate for accurately classifying patients with COVID-19.

Artificial intelligence-driven prediction of COVID-19-related hospitalization and death: a systematic review

Aim

To perform a systematic review on the use of Artificial Intelligence (AI) techniques for predicting COVID-19 hospitalization and mortality using primary and secondary data sources.

Study eligibility criteria

Cohort, clinical trials, meta-analyses, and observational studies investigating COVID-19 hospitalization or mortality using artificial intelligence techniques were eligible. Articles without a full text available in the English language were excluded.

Data sources

Articles recorded in Ovid MEDLINE from 01/01/2019 to 22/08/2022 were screened.

Data extraction

We extracted information on data sources, AI models, and epidemiological aspects of retrieved studies.

Bias assessment

A bias assessment of AI models was done using PROBAST.

Participants

Patients tested positive for COVID-19.

Results

We included 39 studies related to AI-based prediction of hospitalization and death related to COVID-19. The articles were published in the period 2019-2022, and mostly used Random Forest as the model with the best performance. AI models were trained using cohorts of individuals sampled from populations of European and non-European countries, mostly with cohort sample size <5,000. Data collection generally included information on demographics, clinical records, laboratory results, and pharmacological treatments (i.e., high-dimensional datasets). In most studies, the models were internally validated with cross-validation, but the majority of studies lacked external validation and calibration. Covariates were not prioritized using ensemble approaches in most of the studies, however, models still showed moderately good performances with Area under the Receiver operating characteristic Curve (AUC) values >0.7. According to the assessment with PROBAST, all models had a high risk of bias and/or concern regarding applicability.

Conclusions

A broad range of AI techniques have been used to predict COVID-19 hospitalization and mortality. The studies reported good prediction performance of AI models, however, high risk of bias and/or concern regarding applicability were detected.

Interpretable generalized neural additive models for mortality prediction of COVID-19 hospitalized patients in Hamadan, Iran

BACKGROUND

The high number of COVID-19 deaths is a serious threat to the world. Demographic and clinical biomarkers are significantly associated with the mortality risk of this disease. This study aimed to implement Generalized Neural Additive Model (GNAM) as an interpretable machine learning method to predict the COVID-19 mortality of patients.

METHODS

This cohort study included 2181 COVID-19 patients admitted from February 2020 to July 2021 in Sina and Besat hospitals in Hamadan, west of Iran. A total of 22 baseline features including patients' demographic information and clinical biomarkers were collected. Four strategies including removing missing values, mean, K-Nearest Neighbor (KNN), and Multivariate Imputation by Chained Equations (MICE) imputation methods were used to deal with missing data. Firstly, the important features for predicting binary outcome (1: death, 0: recovery) were selected using the Random Forest (RF) method. Also, synthetic minority over-sampling technique (SMOTE) method was used for handling imbalanced data. Next, considering the selected features, the predictive performance of GNAM for predicting mortality outcome was compared with logistic regression, RF, generalized additive model (GAMs), gradient boosting decision tree (GBDT), and deep neural networks (DNNs) classification models. Each model trained on fifty different subsets of a train-test dataset to ensure a model performance. The average accuracy, F1-score and area under the curve (AUC) evaluation indices were used for comparison of the predictive performance of the models.

RESULTS

Out of the 2181 COVID-19 patients, 624 died during hospitalization and 1557 recovered. The missing rate was 3 percent for each patient. The mean age of dead patients (71.17 ± 14.44 years) was statistically significant higher than recovered patients (58.25 ± 16.52 years). Based on RF, 10 features with the highest relative importance were selected as the best influential features; including blood urea nitrogen (BUN), lymphocytes (Lym), age, blood sugar (BS), serum glutamic-oxaloacetic transaminase (SGOT), monocytes (Mono), blood creatinine (CR), neutrophils (NUT), alkaline phosphatase (ALP) and hematocrit (HCT). The results of predictive performance comparisons showed GNAM with the mean accuracy, F1-score, and mean AUC in the test dataset of 0.847, 0.691, and 0.774, respectively, had the best performance. The smooth function graphs learned from the GNAM were descending for the Lym and ascending for the other important features.

CONCLUSIONS

Interpretable GNAM can perform well in predicting the mortality of COVID-19 patients. Therefore, the use of such a reliable model can help physicians to prioritize some important demographic and clinical biomarkers by identifying the effective features and the type of predictive trend in disease progression.

A machine learning approach for predicting high risk hospitalized patients with COVID-19 SARS-Cov-2

BACKGROUND

This study aimed to explore whether explainable Artificial Intelligence methods can be fruitfully used to improve the medical management of patients suffering from complex diseases, and in particular to predict the death risk in hospitalized patients with SARS-Cov-2 based on admission data.

METHODS

This work is based on an observational ambispective study that comprised patients older than 18 years with a positive SARS-Cov-2 diagnosis that were admitted to the hospital Azienda Ospedaliera "SS Antonio e Biagio e Cesare Arrigo", Alessandria, Italy from February, 24 2020 to May, 31 2021, and that completed the disease treatment inside this structure. The patients'medical history, demographic, epidemiologic and clinical data were collected from the electronic medical records system and paper based medical records, entered and managed by the Clinical Study Coordinators using the REDCap electronic data capture tool patient chart. The dataset was used to train and to evaluate predictive ML models.

RESULTS

We overall trained, analysed and evaluated 19 predictive models (both supervised and unsupervised) on data from 824 patients described by 43 features. We focused our attention on models that provide an explanation that is understandable and directly usable by domain experts, and compared the results against other classical machine learning approaches. Among the former, JRIP showed the best performance in 10-fold cross validation, and the best average performance in a further validation test using a different patient dataset from the beginning of the third COVID-19 wave. Moreover, JRIP showed comparable performances with other approaches that do not provide a clear and/or understandable explanation.

CONCLUSIONS

The ML supervised models showed to correctly discern between low-risk and high-risk patients, even when the medical disease context is complex and the list of features is limited to information available at admission time. Furthermore, the models demonstrated to reasonably perform on a dataset from the third COVID-19 wave that was not used in the training phase. Overall, these results are remarkable: (i) from a medical point of view, these models evaluate good predictions despite the possible differences entitled with different care protocols and the possible influence of other viral variants (i.e. delta variant); (ii) from the organizational point of view, they could be used to optimize the management of health-care path at the admission time.

eXtreme Gradient Boosting-based method to classify patients with COVID-19

Different demographic, clinical and laboratory variables have been related to the severity and mortality following SARS-CoV-2 infection. Most studies applied traditional statistical methods and in some cases combined with a machine learning (ML) method. This is the first study to date to comparatively analyze five ML methods to select the one that most closely predicts mortality in patients admitted with COVID-19. The aim of this single-center observational study is to classify, based on different types of variables, adult patients with COVID-19 at increased risk of mortality. SARS-CoV-2 infection was defined by a positive reverse transcriptase PCR. A total of 203 patients were admitted between March 15 and June 15, 2020 to a tertiary hospital. Data were extracted from the electronic medical record. Four supervised ML algorithms (k-nearest neighbors (KNN), decision tree (DT), Gaussian naïve Bayes (GNB) and support vector machine (SVM)) were compared with the eXtreme Gradient Boosting (XGB) method proposed to have excellent scalability and high running speed, among other qualities. The results indicate that the XGB method has the best prediction accuracy (92%), high precision (>0.92) and high recall (>0.92). The KNN, SVM and DT approaches present moderate prediction accuracy (>80%), moderate recall (>0.80) and moderate precision (>0.80). The GNB algorithm shows relatively low classification performance. The variables with the greatest weight in predicting mortality were C reactive protein, procalcitonin, glutamyl oxaloacetic transaminase, glutamyl pyruvic transaminase, neutrophils, D-dimer, creatinine, lactic acid, ferritin, days of non-invasive ventilation, septic shock and age. Based on these results, XGB is a solid candidate for correct classification of patients with COVID-19.

Automated System for Identifying COVID-19 Infections in Computed Tomography Images Using Deep Learning Models

Coronavirus disease 2019 (COVID-19) is a novel disease that affects healthcare on a global scale and cannot be ignored because of its high fatality rate. Computed tomography (CT) images are presently being employed to assist doctors in detecting COVID-19 in its early stages. In several scenarios, a combination of epidemiological criteria (contact during the incubation period), the existence of clinical symptoms, laboratory tests (nucleic acid amplification tests), and clinical imaging-based tests are used to diagnose COVID-19. This method can miss patients and cause more complications. Deep learning is one of the techniques that has been proven to be prominent and reliable in several diagnostic domains involving medical imaging. This study utilizes a convolutional neural network (CNN), stacked autoencoder, and deep neural network to develop a COVID-19 diagnostic system. In this system, classification undergoes some modification before applying the three CT image techniques to determine normal and COVID-19 cases. A large-scale and challenging CT image dataset was used in the training process of the employed deep learning model and reporting their final performance. Experimental outcomes show that the highest accuracy rate was achieved using the CNN model with an accuracy of 88.30%, a sensitivity of 87.65%, and a specificity of 87.97%. Furthermore, the proposed system has outperformed the current existing state-of-the-art models in detecting the COVID-19 virus using CT images.

Predictive models of pregnancy based on data from a preconception cohort study

STUDY QUESTION

Can we derive adequate models to predict the probability of conception among couples actively trying to conceive?

SUMMARY ANSWER

Leveraging data collected from female participants in a North American preconception cohort study, we developed models to predict pregnancy with performance of ∼70% in the area under the receiver operating characteristic curve (AUC).

WHAT IS KNOWN ALREADY

Earlier work has focused primarily on identifying individual risk factors for infertility. Several predictive models have been developed in subfertile populations, with relatively low discrimination (AUC: 59-64%).

STUDY DESIGN, SIZE, DURATION

Study participants were female, aged 21-45 years, residents of the USA or Canada, not using fertility treatment, and actively trying to conceive at enrollment (2013-2019). Participants completed a baseline questionnaire at enrollment and follow-up questionnaires every 2 months for up to 12 months or until conception. We used data from 4133 participants with no more than one menstrual cycle of pregnancy attempt at study entry.

PARTICIPANTS/MATERIALS, SETTING, METHODS

On the baseline questionnaire, participants reported data on sociodemographic factors, lifestyle and behavioral factors, diet quality, medical history and selected male partner characteristics. A total of 163 predictors were considered in this study. We implemented regularized logistic regression, support vector machines, neural networks and gradient boosted decision trees to derive models predicting the probability of pregnancy: (i) within fewer than 12 menstrual cycles of pregnancy attempt time (Model I), and (ii) within 6 menstrual cycles of pregnancy attempt time (Model II). Cox models were used to predict the probability of pregnancy within each menstrual cycle for up to 12 cycles of follow-up (Model III). We assessed model performance using the AUC and the weighted-F1 score for Models I and II, and the concordance index for Model III.

MAIN RESULTS AND THE ROLE OF CHANCE

Model I and II AUCs were 70% and 66%, respectively, in parsimonious models, and the concordance index for Model III was 63%. The predictors that were positively associated with pregnancy in all models were: having previously breastfed an infant and using multivitamins or folic acid supplements. The predictors that were inversely associated with pregnancy in all models were: female age, female BMI and history of infertility. Among nulligravid women with no history of infertility, the most important predictors were: female age, female BMI, male BMI, use of a fertility app, attempt time at study entry and perceived stress.

LIMITATIONS, REASONS FOR CAUTION

Reliance on self-reported predictor data could have introduced misclassification, which would likely be non-differential with respect to the pregnancy outcome given the prospective design. In addition, we cannot be certain that all relevant predictor variables were considered. Finally, though we validated the models using split-sample replication techniques, we did not conduct an external validation study.

WIDER IMPLICATIONS OF THE FINDINGS

Given a wide range of predictor data, machine learning algorithms can be leveraged to analyze epidemiologic data and predict the probability of conception with discrimination that exceeds earlier work.

STUDY FUNDING/COMPETING INTEREST(S)

The research was partially supported by the U.S. National Science Foundation (under grants DMS-1664644, CNS-1645681 and IIS-1914792) and the National Institutes for Health (under grants R01 GM135930 and UL54 TR004130). In the last 3 years, L.A.W. has received in-kind donations for primary data collection in PRESTO from FertilityFriend.com, Kindara.com, Sandstone Diagnostics and Swiss Precision Diagnostics. L.A.W. also serves as a fibroid consultant to AbbVie, Inc. The other authors declare no competing interests.

TRIAL REGISTRATION NUMBER

N/A.

Artificial intelligence in clinical care amidst COVID-19 pandemic: A systematic review

The worldwide health crisis caused by the SARS-Cov-2 virus has resulted in>3 million deaths so far. Improving early screening, diagnosis and prognosis of the disease are critical steps in assisting healthcare professionals to save lives during this pandemic. Since WHO declared the COVID-19 outbreak as a pandemic, several studies have been conducted using Artificial Intelligence techniques to optimize these steps on clinical settings in terms of quality, accuracy and most importantly time. The objective of this study is to conduct a systematic literature review on published and preprint reports of Artificial Intelligence models developed and validated for screening, diagnosis and prognosis of the coronavirus disease 2019. We included 101 studies, published from January 1st, 2020 to December 30th, 2020, that developed AI prediction models which can be applied in the clinical setting. We identified in total 14 models for screening, 38 diagnostic models for detecting COVID-19 and 50 prognostic models for predicting ICU need, ventilator need, mortality risk, severity assessment or hospital length stay. Moreover, 43 studies were based on medical imaging and 58 studies on the use of clinical parameters, laboratory results or demographic features. Several heterogeneous predictors derived from multimodal data were identified. Analysis of these multimodal data, captured from various sources, in terms of prominence for each category of the included studies, was performed. Finally, Risk of Bias (RoB) analysis was also conducted to examine the applicability of the included studies in the clinical setting and assist healthcare providers, guideline developers, and policymakers.

Prediction of COVID-19 mortality among hospitalized patients in Sudan.. [DOI: 10.1101/2021.03.09.21253179] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Background

COVID-19 was primarily reported in China. The mortality rate across countries had ranged from 1% up to more than 10% and it is underestimated in some countries. Advanced age is the most frequently reported factor associated to mortality. Other factors were the presence of comorbidities such as diabetes mellitus, hypertension and obesity. Several models for mortality prediction had been developed to assist in improving the prognosis. The aim of our study was to assess the factors related to mortality among COVID-19 patients and develop a prediction model based on these factors.

Methods

A retrospective cohort study assessed the factors related to the mortality among COVID-19 patients who attended Imperial Hospital isolation centre on November-December, 2020, Khartoum, Sudan. Statistical tests performed were chi-square test, odds ratio and regression to develop the prediction model. Tests were considered statistically significant whenp< 0.05.

Results

105 patients were studied. 29% of the patients were deceased, while, 71% were discharged alive. A statistically significant association was found between the age and severity with regards to mortality rate (p=0.034, 0.018 respectively). The model equation for mortality prediction: Mortality = −14.724+ (1.387* Age) + (−0.323* Gender) + (1.814* Admission) + (0.193* Ischemic Heart Disease) + (−0.369* Fever) + (1.595* Cough) + (1.953* Complications) + (0.149* Duration of hospitalization) + (0.999* Enoxaparin dose).

Conclusions

Age, admission ward, cough and enoxaparin dose were statistically significant predictors for COVID-19 mortality (p= 0.014, 0.011, 0.015, 0.006 respectively).

Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal

OBJECTIVE

To review and appraise the validity and usefulness of published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at increased risk of covid-19 infection or being admitted to hospital with the disease.

DESIGN

Living systematic review and critical appraisal by the COVID-PRECISE (Precise Risk Estimation to optimise covid-19 Care for Infected or Suspected patients in diverse sEttings) group.

DATA SOURCES

PubMed and Embase through Ovid, up to 1 July 2020, supplemented with arXiv, medRxiv, and bioRxiv up to 5 May 2020.

STUDY SELECTION

Studies that developed or validated a multivariable covid-19 related prediction model.

DATA EXTRACTION

At least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool).

RESULTS

37 421 titles were screened, and 169 studies describing 232 prediction models were included. The review identified seven models for identifying people at risk in the general population; 118 diagnostic models for detecting covid-19 (75 were based on medical imaging, 10 to diagnose disease severity); and 107 prognostic models for predicting mortality risk, progression to severe disease, intensive care unit admission, ventilation, intubation, or length of hospital stay. The most frequent types of predictors included in the covid-19 prediction models are vital signs, age, comorbidities, and image features. Flu-like symptoms are frequently predictive in diagnostic models, while sex, C reactive protein, and lymphocyte counts are frequent prognostic factors. Reported C index estimates from the strongest form of validation available per model ranged from 0.71 to 0.99 in prediction models for the general population, from 0.65 to more than 0.99 in diagnostic models, and from 0.54 to 0.99 in prognostic models. All models were rated at high or unclear risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, high risk of model overfitting, and unclear reporting. Many models did not include a description of the target population (n=27, 12%) or care setting (n=75, 32%), and only 11 (5%) were externally validated by a calibration plot. The Jehi diagnostic model and the 4C mortality score were identified as promising models.

CONCLUSION

Prediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that almost all pubished prediction models are poorly reported, and at high risk of bias such that their reported predictive performance is probably optimistic. However, we have identified two (one diagnostic and one prognostic) promising models that should soon be validated in multiple cohorts, preferably through collaborative efforts and data sharing to also allow an investigation of the stability and heterogeneity in their performance across populations and settings. Details on all reviewed models are publicly available at https://www.covprecise.org/. Methodological guidance as provided in this paper should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, prediction model authors should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline.

SYSTEMATIC REVIEW REGISTRATION

Protocol https://osf.io/ehc47/, registration https://osf.io/wy245.

READERS' NOTE

This article is a living systematic review that will be updated to reflect emerging evidence. Updates may occur for up to two years from the date of original publication. This version is update 3 of the original article published on 7 April 2020 (BMJ 2020;369:m1328). Previous updates can be found as data supplements (https://www.bmj.com/content/369/bmj.m1328/related#datasupp). When citing this paper please consider adding the update number and date of access for clarity.