Construction and validation of a machine learning-based nomogram: A tool to predict the risk of getting severe coronavirus disease 2019 (COVID-19)

Development and validation of a nomogram for predicting in-hospital survival rates of patients with COVID-19

Objective

Our aim was to develop and validate a nomogram for predicting the in-hospital 14-day (14 d) and 28-day (28 d) survival rates of patients with coronavirus disease 2019 (COVID-19).

Methods

Clinical data of patients with COVID-19 admitted to the Renmin Hospital of Wuhan University from December 2022 to February 2023 and the north campus of Shanghai Ninth People's Hospital from April 2022 to June 2022 were collected. A total of 408 patients from Renmin Hospital of Wuhan University were selected as the training cohort, and 151 patients from Shanghai Ninth People's Hospital were selected as the verification cohort. Independent variables were screened using Cox regression analysis, and a nomogram was constructed using R software. The prediction accuracy of the nomogram was evaluated using the receiver operating characteristic (ROC) curve, C-index, and calibration curve. Decision curve analysis was used to evaluate the clinical application value of the model. The nomogram was externally validated using a validation cohort.

Result

In total, 559 patients with severe/critical COVID-19 were included in this study, of whom 179 (32.02 %) died. Multivariate Cox regression analysis showed that age >80 years [hazard ratio (HR) = 1.539, 95 % confidence interval (CI): 1.027-2.306, P = 0.037], history of diabetes (HR = 1.741, 95 % CI: 1.253-2.420, P = 0.001), high APACHE II score (HR = 1.083, 95 % CI: 1.042-1.126, P < 0.001), sepsis (HR = 2.387, 95 % CI: 1.707-3.338, P < 0.001), high neutrophil-to-lymphocyte ratio (NLR) (HR = 1.010, 95 % CI: 1.003-1.017, P = 0.007), and high D-dimer level (HR = 1.005, 95 % CI: 1.001-1.009, P = 0.028) were independent risk factors for 14 d and 28 d survival rates, whereas COVID-19 vaccination (HR = 0.625, 95 % CI: 0.440-0.886, P = 0.008) was a protective factor affecting prognosis. ROC curve analysis showed that the area under the curve (AUC) of the 14 d and 28 d hospital survival rates in the training cohort was 0.765 (95 % CI: 0.641-0.923) and 0.814 (95 % CI: 0.702-0.938), respectively, and the AUC of the 14 d and 28 d hospital survival rates in the verification cohort was 0.898 (95 % CI: 0.765-0.962) and 0.875 (95 % CI: 0.741-0.945), respectively. The calibration curves of 14 d and 28 d hospital survival showed that the predicted probability of the model agreed well with the actual probability. Decision curve analysis (DCA) showed that the nomogram has high clinical application value.

Conclusion

In-hospital survival rates of patients with COVID-19 were predicted using a nomogram, which will help clinicians in make appropriate clinical decisions.

HB-EGF Plasmatic Level Contributes to the Development of Early Risk Prediction Nomogram for Severe COVID-19 Cases

(1) Background: Heparin-Binding Epidermal Growth Factor-like Growth Factor (HB-EGF) is involved in wound healing, cardiac hypertrophy, and heart development processes. Recently, circulant HB-EGF was reported upregulated in severely hospitalized COVID-19 patients. However, the clinical correlations of HB-EGF plasma levels with COVID-19 patients' characteristics have not been defined yet. In this study, we assessed the plasma HB-EGF correlations with the clinical and paraclinical patients' data, evaluated its predictive clinical value, and built a risk prediction model for severe COVID-19 cases based on the resulting significant prognostic markers. (2) Methods: Our retrospective study enrolled 75 COVID-19 patients and 17 control cases from May 2020 to September 2020. We quantified plasma HB-EGF levels using the sandwich ELISA technique. Correlations between HB-EGF plasma levels with clinical and paraclinical patients' data were calculated using two-tailed Spearman and Point-Biserial tests. Significantly upregulated parameters for severe COVID-19 cases were identified and selected to build a multivariate logistic regression prediction model. The clinical significance of the prediction model was assessed by risk prediction nomogram and decision curve analyses. (3) Results: HB-EGF plasma levels were significantly higher in the severe COVID-19 subgroup compared to the controls (p = 0.004) and moderate cases (p = 0.037). In the severe COVID-19 group, HB-EGF correlated with age (p = 0.028), pulse (p = 0.016), dyspnea (p = 0.014) and prothrombin time (PT) (p = 0.04). The multivariate risk prediction model built on seven identified risk parameters (age p = 0.043, HB-EGF p = 0.0374, Fibrinogen p = 0.009, PT p = 0.008, Creatinine p = 0.026, D-Dimers p = 0.024 and delta miR-195 p < 0.0001) identifies severe COVID-19 with AUC = 0.9556 (p < 0.0001). The decision curve analysis revealed that the nomogram model is clinically relevant throughout a wide threshold probability range. (4) Conclusions: Upregulated HB-EGF plasma levels might serve as a prognostic factor for severe COVID-19 and help build a reliable risk prediction nomogram that improves the identification of high-risk patients at an early stage of COVID-19.

Evaluation of an adjusted MEWS (Modified Early Warning Score) for COVID-19 patients to identify risk of ICU admission or death in the Kingdom of Bahrain

BACKGROUND

While most COVID-19 cases have uncomplicated infection, a small proportion has the potential to develop life-threatening disease, as such development of a prediction tool using patients baseline characteristics at the time of diagnosis should aid in early identification of high-risk groups and devise pertinent management. Hence, we set up this retrospective study to determine preadmission triaging tool to predict the development of severe COVID-19 in the Kingdom of Bahrain MATERIALS AND METHODS: A retrospective study was conducted from 1 September 2020 to 30 November 2020 with enrolment of all SARS-CoV-2 PCR-confirmed persons aged ≥ 14 years who attended Al-Shamil Field Hospital (SFH) in the Kingdom of Bahrain for triaging and assessment with recording of the following parameters: systolic blood pressure, heart rate, respiratory rate, temperature, the alert, verbal, pain, unresponsive neurological score, age, oxygen saturation, comorbidities, Body Mass Index (BMI), duration of symptoms and living with immunocompromised populations to develop our local adjusted MEWS as predictor for ICU admission & for consideration of suitable isolation at home. Follow up data of all patients was obtained from the electronic medical records system including CXR findings, treatments/medications received, need of oxygen supplements /intubation, needs of ICU care, and the outcome (death /discharged alive) IBM SPSS statistic version 21 program was used for data analysis.

RESULTS

Our study showed that using the locally developed adjusted MEWS score, there was an significant association between high value of this adjusted MEWS score and abnormal radiographic finding (49.7 % Vs. 17 % for patients with high score Vs. those with low score respectively). Out of the 181 patients with high scores on adjusted MEWS; 38.7 % required oxygen via nasal cannula, 14.4 % required face mask and 8.3 % non-rebreather mask; this proportion was significantly higher than their counterpart patients who score low on adjusted MEWS (20.9 %, 7.7 %, 4.8 %respectively) with statistically significance difference between the two groups (p value of 0.00, 0.00,.004 respectively) Requirement of ICU admission was significantly higher among patients with high score in comparison to those with low score (14.4 % vs. 3 %) with significant p value (0.00) But higher score value was not associated significantly with increase mortality rate among COVID patients.

CONCLUSION

Development of our new Adjusted MEWS score system by adding the additional elements of age, oxygen saturation, comorbidities, Body Mass Index (BMI) and duration of symptoms found to be very useful predictor tool for preadmission triaging of COVID patients based on their risk assessment to help clinician to decide on the appropriate placement to different level of isolation facilities.

Prognostic models in COVID-19 infection that predict severity: a systematic review

Current evidence on COVID-19 prognostic models is inconsistent and clinical applicability remains controversial. We performed a systematic review to summarize and critically appraise the available studies that have developed, assessed and/or validated prognostic models of COVID-19 predicting health outcomes. We searched six bibliographic databases to identify published articles that investigated univariable and multivariable prognostic models predicting adverse outcomes in adult COVID-19 patients, including intensive care unit (ICU) admission, intubation, high-flow nasal therapy (HFNT), extracorporeal membrane oxygenation (ECMO) and mortality. We identified and assessed 314 eligible articles from more than 40 countries, with 152 of these studies presenting mortality, 66 progression to severe or critical illness, 35 mortality and ICU admission combined, 17 ICU admission only, while the remaining 44 studies reported prediction models for mechanical ventilation (MV) or a combination of multiple outcomes. The sample size of included studies varied from 11 to 7,704,171 participants, with a mean age ranging from 18 to 93 years. There were 353 prognostic models investigated, with area under the curve (AUC) ranging from 0.44 to 0.99. A great proportion of studies (61.5%, 193 out of 314) performed internal or external validation or replication. In 312 (99.4%) studies, prognostic models were reported to be at high risk of bias due to uncertainties and challenges surrounding methodological rigor, sampling, handling of missing data, failure to deal with overfitting and heterogeneous definitions of COVID-19 and severity outcomes. While several clinical prognostic models for COVID-19 have been described in the literature, they are limited in generalizability and/or applicability due to deficiencies in addressing fundamental statistical and methodological concerns. Future large, multi-centric and well-designed prognostic prospective studies are needed to clarify remaining uncertainties.

Independent predictors of in-hospital mortality and the need for intensive care in hospitalized non-critical COVID-19 patients: a prospective cohort study

One of the most helpful strategies to deal with ongoing coronavirus pandemics is to use some prudence when treating patients infected with SARS-CoV-2. We aimed to evaluate the clinical, demographic, and laboratory parameters that might have predictive value for in-hospital mortality and the need for intensive care and build a model based on them. This study was a prospective, observational, single-center study including non-critical patients admitted to COVID-19 wards. Besides classical clinic-demographic features, basic laboratory parameters obtained on admission were tested, and then new models for each outcome were developed built on the most significant variables. Receiver operating characteristics (ROC) analyses were performed by calculating each model's probability. A total of 368 non-critical hospitalized patients were recruited, the need for ICU care was observed in 70 patients (19%). The total number of patients who died in either ICU or wards was 39 (10.6%). The first two models (based on clinical features and demographics) were developed to predict ICU and death, respectively; older age, male sex, active cancer, and low baseline saturation were noted to be independent predictors. The area under the curve values of the first two models were noted 0.878 and 0.882 (p < .001; confidence interval [CI] 95% [0.837-0.919], p < .001; CI 95% [0.844-0.922]). Following two models, the third and fourth were based on laboratory parameters with clinic-demographic features. Initial lower sodium and lower albumin levels were determined as independent factors in predicting the need for ICU care; higher blood urea nitrogen and lower albumin were independent factors in predicting in-hospital mortality. The area under the curve values of the third and fourth model was noted 0.938 and 0.929, respectively (p < .001; CI 95% [0.912-0.965], p < .001; CI 95% [0.895-962]). By integrating the widely available blood tests results with simple clinic demographic data, non-critical patients can be stratified according to their risk level. Such stratification is essential to filter the patients' non-critical underlying diseases and conditions that can obfuscate the physician's predictive capacity.

Molecular and Clinical Prognostic Biomarkers of COVID-19 Severity and Persistence

The coronavirus disease 2019 (COVID-19), caused by the severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2), poses several challenges to clinicians, due to its unpredictable clinical course. The identification of laboratory biomarkers, specific cellular, and molecular mediators of immune response could contribute to the prognosis and management of COVID-19 patients. Of utmost importance is also the detection of differentially expressed genes, which can serve as transcriptomic signatures, providing information valuable to stratify patients into groups, based on the severity of the disease. The role of biomarkers such as IL-6, procalcitonin, neutrophil–lymphocyte ratio, white blood cell counts, etc. has already been highlighted in recently published studies; however, there is a notable amount of new evidence that has not been summarized yet, especially regarding transcriptomic signatures. Hence, in this review, we assess the latest cellular and molecular data and determine the significance of abnormalities in potential biomarkers for COVID-19 severity and persistence. Furthermore, we applied Gene Ontology (GO) enrichment analysis using the genes reported as differentially expressed in the literature in order to investigate which biological pathways are significantly enriched. The analysis revealed a number of processes, such as inflammatory response, and monocyte and neutrophil chemotaxis, which occur as part of the complex immune response to SARS-CoV-2.

A novel method for handling pre-existing conditions in multivariate prediction model development for COVID-19 death in the Department of Veterans Affairs

Many mathematical models have been proposed to predict death following the Coronavirus Disease 2019 (COVID-19); all started with comorbidity subsets for this still-little understood disease. Thus, we derived a novel predicted probability of death model (PDeathDx) upon all diagnostic codes documented in the Department of Veterans Affairs. We present the conceptual underpinnings and analytic approach in estimating the independent contribution of pre-existing conditions. This is the largest study to-date following patients with COVID-19 to predict mortality. Cases were identified with at least one positive nucleic acid amplification test. Starting in 1997, we use diagnoses from the first time a patient sought care until 14 days before a positive nucleic acid amplification test. We demonstrate the clear advantage of using an unrestricted set of pre-existing conditions to model COVID-19 mortality, as models using conventional comorbidity indices often assign little weight or usually do not include some of the highest risk conditions; the same is true of conditions associated with COVID-19 severity. Our findings suggest that it is risky to pick comorbidities for analysis without a systematic review of all those experienced by the cohort. Unlike conventional approaches, our comprehensive methodology provides the flexibility that has been advocated for comorbidity indices since 1993; such an approach can be readily adapted for other diseases and outcomes. With our comorbidity risk adjustment approach outperforming conventional indices for predicting COVID-19 mortality, it shows promise for predicting outcomes for other conditions of interest.

Development and validation of prognostic scoring system for COVID-19 severity in South India

Background

Development of a prediction model using baseline characteristics of COVID-19 patients at the time of diagnosis will aid us in early identification of the high-risk groups and devise pertinent strategies accordingly. Hence, we did this study to develop a prognostic-scoring system for predicting the COVID-19 severity in South India.

Methods

We undertook this retrospective cohort study among COVID-19 patients reporting to Hindu Mission Hospital, India. Multivariable logistic regression using the LASSO procedure was used to select variables for the model building, and the nomogram scoring system was developed with the final selected model. Model discrimination, calibration, and decision curve analysis (DCA) was performed.

Results

In total, 35.1% of the patients in the training set developed severe COVID-19 during their follow-up period. In the basic model, nine variables (age group, sex, education, chronic kidney disease, tobacco, cough, dyspnea, olfactory-gustatory dysfunction [OGD], and gastrointestinal symptoms) were selected and a nomogram was built using these variables. In the advanced model, in addition to these variables (except OGD), C-reactive protein, lactate dehydrogenase, ferritin, d-dimer, and CT severity score were selected. The discriminatory power (c-index) for basic model was 0.78 (95%CI: 0.74–0.82) and advanced model was 0.83 (95%CI: 0.79–0.87). DCA showed that both the models are beneficial at a threshold probability around 10–95% than treat-none or treat-all strategies.

Conclusion

The present study has developed two separate prognostic-scoring systems to predict the COVID-19 severity. This scoring system could help the clinicians and policymakers to devise targeted interventions and in turn reduce the COVID-19 mortality in India.

Supplementary information

The online version contains supplementary material available at 10.1007/s11845-021-02876-w.

A Retrospective Cohort Study on the Clinical Course of Patients With Moderate-Type COVID-19

Background: A large number of people contracted moderate-type COVID-19 around the world. However, to our knowledge no studies have covered the clinical course of patients with moderate-type COVID-19. This study describes the clinical course of moderate-type patients with COVID-19 from Wuhan City and Yiyang City, and explores factors relevant to the length of hospitalization and symptoms relief. Methods: The study analyzed the clinical course of 107 moderate-type patients with COVID-19 from the outbreak area (Wuhan) and the imported area (Yiyang), and used automatic linear modeling and multivariate linear regression analysis to explore the factors relevant to the length of hospitalization and symptoms relief. Furthermore, we created a scoring system to value the length of hospitalization and symptoms relief. Results: Lymphopenia, elevated C-reactive protein, increased LDH, bilateral lung GGO (ground glass opacity), and lung consolidation were more likely to appear in ordinary inpatients with moderate-type COVID-19 from Wuhan (P < 0.05), compared to infected medical staff from Wuhan and ordinary inpatients with moderate-type COVID-19 from Yiyang. Meanwhile, the length of hospitalization and symptoms relief was longer in ordinary patients with moderate-type COVID-19 from Wuhan (P < 0.05). Onset of symptoms to admission, ESR, leucocytes count, and bilateral lung GGO were linearly related to the length of hospitalization (P < 0.05); onset of symptoms to admission, leucocytes count, bilateral lung GGO, and lung consolidation were linearly related to the length of symptoms relief (P < 0.05). By using the scoring system, we found that the time of hospitalization and symptoms relief lengthened as the scores increased. Conclusions: This study described the clinical course of patients with moderate-type COVID-19, and found that ordinary patients with moderate-type COVID-19 in outbreak areas were more serious and needed stronger treatment and longer treatment time. Onset of symptoms to admission, ESR, leucocytes count, and bilateral lung GGO can be effective predictors of the length of hospitalization. And onset of symptoms to admission, leucocytes count, bilateral lung GGO, and lung consolidation can be effective predictors of the amount of time until symptoms relief. Most importantly, we have created a scoring system, which could contribute to the diagnosis and treatment of COVID-19.

Construction and validation of a machine learning-based nomogram: A tool to predict the risk of getting severe coronavirus disease 2019 (COVID-19)

Background

Identifying patients who may develop severe coronavirus disease 2019 (COVID‐19) will facilitate personalized treatment and optimize the distribution of medical resources.

Methods

In this study, 590 COVID‐19 patients during hospitalization were enrolled (Training set: n = 285; Internal validation set: n = 127; Prospective set: n = 178). After filtered by two machine learning methods in the training set, 5 out of 31 clinical features were selected into the model building to predict the risk of developing severe COVID‐19 disease. Multivariate logistic regression was applied to build the prediction nomogram and validated in two different sets. Receiver operating characteristic (ROC) analysis and decision curve analysis (DCA) were used to evaluate its performance.

Results

From 31 potential predictors in the training set, 5 independent predictive factors were identified and included in the risk score: C‐reactive protein (CRP), lactate dehydrogenase (LDH), Age, Charlson/Deyo comorbidity score (CDCS), and erythrocyte sedimentation rate (ESR). Subsequently, we generated the nomogram based on the above features for predicting severe COVID‐19. In the training cohort, the area under curves (AUCs) were 0.822 (95% CI, 0.765–0.875) and the internal validation cohort was 0.762 (95% CI, 0.768–0.844). Further, we validated it in a prospective cohort with the AUCs of 0.705 (95% CI, 0.627–0.778). The internally bootstrapped calibration curve showed favorable consistency between prediction by nomogram and the actual situation. And DCA analysis also conferred high clinical net benefit.

Conclusion

In this study, our predicting model based on five clinical characteristics of COVID‐19 patients will enable clinicians to predict the potential risk of developing critical illness and thus optimize medical management.