1
|
Faisal M, Mohammed MA, Richardson D, Fiori M, Beatson K. Accuracy of automated computer-aided risk scoring systems to estimate the risk of COVID-19: a retrospective cohort study. BMC Res Notes 2024; 17:109. [PMID: 38637897 PMCID: PMC11027522 DOI: 10.1186/s13104-024-06773-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2023] [Accepted: 04/15/2024] [Indexed: 04/20/2024] Open
Abstract
BACKGROUND In the UK National Health Service (NHS), the patient's vital signs are monitored and summarised into a National Early Warning Score (NEWS) score. A set of computer-aided risk scoring systems (CARSS) was developed and validated for predicting in-hospital mortality and sepsis in unplanned admission to hospital using NEWS and routine blood tests results. We sought to assess the accuracy of these models to predict the risk of COVID-19 in unplanned admissions during the first phase of the pandemic. METHODS Adult ( > = 18 years) non-elective admissions discharged (alive/deceased) between 11-March-2020 to 13-June-2020 from two acute hospitals with an index NEWS electronically recorded within ± 24 h of admission. We identified COVID-19 admission based on ICD-10 code 'U071' which was determined by COVID-19 swab test results (hospital or community). We assessed the performance of CARSS (CARS_N, CARS_NB, CARM_N, CARM_NB) for predicting the risk of COVID-19 in terms of discrimination (c-statistic) and calibration (graphically). RESULTS The risk of in-hospital mortality following emergency medical admission was 8.4% (500/6444) and 9.6% (620/6444) had a diagnosis of COVID-19. For predicting COVID-19 admissions, the CARS_N model had the highest discrimination 0.73 (0.71 to 0.75) and calibration slope 0.81 (0.72 to 0.89) compared to other CARSS models: CARM_N (discrimination:0.68 (0.66 to 0.70) and calibration slope 0.47 (0.41 to 0.54)), CARM_NB (discrimination:0.68 (0.65 to 0.70) and calibration slope 0.37 (0.31 to 0.43)), and CARS_NB (discrimination:0.68 (0.66 to 0.70) and calibration slope 0.56 (0.47 to 0.64)). CONCLUSIONS The CARS_N model is reasonably accurate for predicting the risk of COVID-19. It may be clinically useful as an early warning system at the time of admission especially to triage large numbers of unplanned admissions because it requires no additional data collection and is readily automated.
Collapse
Affiliation(s)
- Muhammad Faisal
- Centre for Digital Innovations in Health & Social Care, Faculty of Health Studies, University of Bradford, Bradford, UK
- Wolfson Centre for Applied Health Research, Bradford, UK
| | - Mohammed Amin Mohammed
- Faculty of Health Studies, University of Bradford, Richmond Road, BD7 1DP, Bradford, UK.
- NHS Midlands and Lancashire Commissioning Support Unit, The Strategy Unit, Kingston House, B70 9LD, West Bromwich, UK.
| | - Donald Richardson
- Consultant Renal Physician York & Scarborough Teaching Hospitals NHS Foundation Trust, York, UK
| | - Massimo Fiori
- York & Scarborough Teaching Hospitals NHS Foundation Trust, York, UK
| | - Kevin Beatson
- York & Scarborough Teaching Hospitals NHS Foundation Trust, York, UK
| |
Collapse
|
2
|
Casillas N, Ramón A, Torres AM, Blasco P, Mateo J. Predictive Model for Mortality in Severe COVID-19 Patients across the Six Pandemic Waves. Viruses 2023; 15:2184. [PMID: 38005862 PMCID: PMC10675561 DOI: 10.3390/v15112184] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2023] [Revised: 10/21/2023] [Accepted: 10/28/2023] [Indexed: 11/26/2023] Open
Abstract
The impact of SARS-CoV-2 infection remains substantial on a global scale, despite widespread vaccination efforts, early therapeutic interventions, and an enhanced understanding of the disease's underlying mechanisms. At the same time, a significant number of patients continue to develop severe COVID-19, necessitating admission to intensive care units (ICUs). This study aimed to provide evidence concerning the most influential predictors of mortality among critically ill patients with severe COVID-19, employing machine learning (ML) techniques. To accomplish this, we conducted a retrospective multicenter investigation involving 684 patients with severe COVID-19, spanning from 1 June 2020 to 31 March 2023, wherein we scrutinized sociodemographic, clinical, and analytical data. These data were extracted from electronic health records. Out of the six supervised ML methods scrutinized, the extreme gradient boosting (XGB) method exhibited the highest balanced accuracy at 96.61%. The variables that exerted the greatest influence on mortality prediction encompassed ferritin, fibrinogen, D-dimer, platelet count, C-reactive protein (CRP), prothrombin time (PT), invasive mechanical ventilation (IMV), PaFi (PaO2/FiO2), lactate dehydrogenase (LDH), lymphocyte levels, activated partial thromboplastin time (aPTT), body mass index (BMI), creatinine, and age. These findings underscore XGB as a robust candidate for accurately classifying patients with COVID-19.
Collapse
Affiliation(s)
- Nazaret Casillas
- Department of Internal Medicine, Hospital Virgen De La Luz, 16002 Cuenca, Spain
- Medical Analysis Expert Group, Institute of Technology, University of Castilla-La Mancha, 16002 Cuenca, Spain
| | - Antonio Ramón
- Department of Pharmacy, General University Hospital, 46014 Valencia, Spain
| | - Ana María Torres
- Medical Analysis Expert Group, Institute of Technology, University of Castilla-La Mancha, 16002 Cuenca, Spain
- Instituto de Investigación Sanitaria de Castilla-La Mancha (IDISCAM), 45071 Toledo, Spain
| | - Pilar Blasco
- Department of Pharmacy, General University Hospital, 46014 Valencia, Spain
| | - Jorge Mateo
- Medical Analysis Expert Group, Institute of Technology, University of Castilla-La Mancha, 16002 Cuenca, Spain
- Instituto de Investigación Sanitaria de Castilla-La Mancha (IDISCAM), 45071 Toledo, Spain
| |
Collapse
|
3
|
Shakibfar S, Nyberg F, Li H, Zhao J, Nordeng HME, Sandve GKF, Pavlovic M, Hajiebrahimi M, Andersen M, Sessa M. Artificial intelligence-driven prediction of COVID-19-related hospitalization and death: a systematic review. Front Public Health 2023; 11:1183725. [PMID: 37408750 PMCID: PMC10319067 DOI: 10.3389/fpubh.2023.1183725] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Accepted: 05/31/2023] [Indexed: 07/07/2023] Open
Abstract
Aim To perform a systematic review on the use of Artificial Intelligence (AI) techniques for predicting COVID-19 hospitalization and mortality using primary and secondary data sources. Study eligibility criteria Cohort, clinical trials, meta-analyses, and observational studies investigating COVID-19 hospitalization or mortality using artificial intelligence techniques were eligible. Articles without a full text available in the English language were excluded. Data sources Articles recorded in Ovid MEDLINE from 01/01/2019 to 22/08/2022 were screened. Data extraction We extracted information on data sources, AI models, and epidemiological aspects of retrieved studies. Bias assessment A bias assessment of AI models was done using PROBAST. Participants Patients tested positive for COVID-19. Results We included 39 studies related to AI-based prediction of hospitalization and death related to COVID-19. The articles were published in the period 2019-2022, and mostly used Random Forest as the model with the best performance. AI models were trained using cohorts of individuals sampled from populations of European and non-European countries, mostly with cohort sample size <5,000. Data collection generally included information on demographics, clinical records, laboratory results, and pharmacological treatments (i.e., high-dimensional datasets). In most studies, the models were internally validated with cross-validation, but the majority of studies lacked external validation and calibration. Covariates were not prioritized using ensemble approaches in most of the studies, however, models still showed moderately good performances with Area under the Receiver operating characteristic Curve (AUC) values >0.7. According to the assessment with PROBAST, all models had a high risk of bias and/or concern regarding applicability. Conclusions A broad range of AI techniques have been used to predict COVID-19 hospitalization and mortality. The studies reported good prediction performance of AI models, however, high risk of bias and/or concern regarding applicability were detected.
Collapse
Affiliation(s)
- Saeed Shakibfar
- Pharmacovigilance Research Center, Department of Drug Design and Pharmacology, University of Copenhagen, Copenhagen, Denmark
| | - Fredrik Nyberg
- School of Public Health and Community Medicine, Institute of Medicine, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
| | - Huiqi Li
- School of Public Health and Community Medicine, Institute of Medicine, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
| | - Jing Zhao
- Pharmacoepidemiology and Drug Safety Research Group, Department of Pharmacy, Faculty of Mathematics and Natural Sciences, University of Oslo, Oslo, Norway
- UiORealArt Convergence Environment, University of Oslo, Oslo, Norway
| | - Hedvig Marie Egeland Nordeng
- Pharmacoepidemiology and Drug Safety Research Group, Department of Pharmacy, Faculty of Mathematics and Natural Sciences, University of Oslo, Oslo, Norway
- UiORealArt Convergence Environment, University of Oslo, Oslo, Norway
| | - Geir Kjetil Ferkingstad Sandve
- UiORealArt Convergence Environment, University of Oslo, Oslo, Norway
- Department of Informatics, Faculty of Mathematics and Natural Sciences, University of Oslo, Oslo, Norway
| | - Milena Pavlovic
- UiORealArt Convergence Environment, University of Oslo, Oslo, Norway
- Department of Informatics, Faculty of Mathematics and Natural Sciences, University of Oslo, Oslo, Norway
| | | | - Morten Andersen
- Pharmacovigilance Research Center, Department of Drug Design and Pharmacology, University of Copenhagen, Copenhagen, Denmark
| | - Maurizio Sessa
- Pharmacovigilance Research Center, Department of Drug Design and Pharmacology, University of Copenhagen, Copenhagen, Denmark
| |
Collapse
|
4
|
Moslehi S, Mahjub H, Farhadian M, Soltanian AR, Mamani M. Interpretable generalized neural additive models for mortality prediction of COVID-19 hospitalized patients in Hamadan, Iran. BMC Med Res Methodol 2022; 22:339. [PMID: 36585627 PMCID: PMC9803600 DOI: 10.1186/s12874-022-01827-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2022] [Accepted: 12/21/2022] [Indexed: 01/01/2023] Open
Abstract
BACKGROUND The high number of COVID-19 deaths is a serious threat to the world. Demographic and clinical biomarkers are significantly associated with the mortality risk of this disease. This study aimed to implement Generalized Neural Additive Model (GNAM) as an interpretable machine learning method to predict the COVID-19 mortality of patients. METHODS This cohort study included 2181 COVID-19 patients admitted from February 2020 to July 2021 in Sina and Besat hospitals in Hamadan, west of Iran. A total of 22 baseline features including patients' demographic information and clinical biomarkers were collected. Four strategies including removing missing values, mean, K-Nearest Neighbor (KNN), and Multivariate Imputation by Chained Equations (MICE) imputation methods were used to deal with missing data. Firstly, the important features for predicting binary outcome (1: death, 0: recovery) were selected using the Random Forest (RF) method. Also, synthetic minority over-sampling technique (SMOTE) method was used for handling imbalanced data. Next, considering the selected features, the predictive performance of GNAM for predicting mortality outcome was compared with logistic regression, RF, generalized additive model (GAMs), gradient boosting decision tree (GBDT), and deep neural networks (DNNs) classification models. Each model trained on fifty different subsets of a train-test dataset to ensure a model performance. The average accuracy, F1-score and area under the curve (AUC) evaluation indices were used for comparison of the predictive performance of the models. RESULTS Out of the 2181 COVID-19 patients, 624 died during hospitalization and 1557 recovered. The missing rate was 3 percent for each patient. The mean age of dead patients (71.17 ± 14.44 years) was statistically significant higher than recovered patients (58.25 ± 16.52 years). Based on RF, 10 features with the highest relative importance were selected as the best influential features; including blood urea nitrogen (BUN), lymphocytes (Lym), age, blood sugar (BS), serum glutamic-oxaloacetic transaminase (SGOT), monocytes (Mono), blood creatinine (CR), neutrophils (NUT), alkaline phosphatase (ALP) and hematocrit (HCT). The results of predictive performance comparisons showed GNAM with the mean accuracy, F1-score, and mean AUC in the test dataset of 0.847, 0.691, and 0.774, respectively, had the best performance. The smooth function graphs learned from the GNAM were descending for the Lym and ascending for the other important features. CONCLUSIONS Interpretable GNAM can perform well in predicting the mortality of COVID-19 patients. Therefore, the use of such a reliable model can help physicians to prioritize some important demographic and clinical biomarkers by identifying the effective features and the type of predictive trend in disease progression.
Collapse
Affiliation(s)
- Samad Moslehi
- grid.411950.80000 0004 0611 9280Department of Biostatistics, School of Public Health, Hamadan University of Medical Sciences, Hamadan, Iran
| | - Hossein Mahjub
- grid.411950.80000 0004 0611 9280Department of Biostatistics, School of Public Health, Research Center for Health Sciences, Hamadan University of Medical Sciences, Hamadan, Iran
| | - Maryam Farhadian
- grid.411950.80000 0004 0611 9280Department of Biostatistics, School of Public Health, Research Center for Health Sciences, Hamadan University of Medical Sciences, Hamadan, Iran
| | - Ali Reza Soltanian
- grid.411950.80000 0004 0611 9280Department of Biostatistics, School of Public Health, Modeling of Noncommunicable Diseases Research Center, Hamadan University of Medical Sciences, Hamadan, Iran
| | - Mojgan Mamani
- grid.411950.80000 0004 0611 9280Brucellosis Research Center, Hamadan University of Medical Sciences, Hamadan, Iran
| |
Collapse
|
5
|
Bottrighi A, Pennisi M, Roveta A, Massarino C, Cassinari A, Betti M, Bolgeo T, Bertolotti M, Rava E, Maconi A. A machine learning approach for predicting high risk hospitalized patients with COVID-19 SARS-Cov-2. BMC Med Inform Decis Mak 2022; 22:340. [PMID: 36578017 PMCID: PMC9795955 DOI: 10.1186/s12911-022-02076-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2022] [Accepted: 12/06/2022] [Indexed: 12/29/2022] Open
Abstract
BACKGROUND This study aimed to explore whether explainable Artificial Intelligence methods can be fruitfully used to improve the medical management of patients suffering from complex diseases, and in particular to predict the death risk in hospitalized patients with SARS-Cov-2 based on admission data. METHODS This work is based on an observational ambispective study that comprised patients older than 18 years with a positive SARS-Cov-2 diagnosis that were admitted to the hospital Azienda Ospedaliera "SS Antonio e Biagio e Cesare Arrigo", Alessandria, Italy from February, 24 2020 to May, 31 2021, and that completed the disease treatment inside this structure. The patients'medical history, demographic, epidemiologic and clinical data were collected from the electronic medical records system and paper based medical records, entered and managed by the Clinical Study Coordinators using the REDCap electronic data capture tool patient chart. The dataset was used to train and to evaluate predictive ML models. RESULTS We overall trained, analysed and evaluated 19 predictive models (both supervised and unsupervised) on data from 824 patients described by 43 features. We focused our attention on models that provide an explanation that is understandable and directly usable by domain experts, and compared the results against other classical machine learning approaches. Among the former, JRIP showed the best performance in 10-fold cross validation, and the best average performance in a further validation test using a different patient dataset from the beginning of the third COVID-19 wave. Moreover, JRIP showed comparable performances with other approaches that do not provide a clear and/or understandable explanation. CONCLUSIONS The ML supervised models showed to correctly discern between low-risk and high-risk patients, even when the medical disease context is complex and the list of features is limited to information available at admission time. Furthermore, the models demonstrated to reasonably perform on a dataset from the third COVID-19 wave that was not used in the training phase. Overall, these results are remarkable: (i) from a medical point of view, these models evaluate good predictions despite the possible differences entitled with different care protocols and the possible influence of other viral variants (i.e. delta variant); (ii) from the organizational point of view, they could be used to optimize the management of health-care path at the admission time.
Collapse
Affiliation(s)
- Alessio Bottrighi
- grid.16563.370000000121663741DISIT, Computer Science Institute, Università del Piemonte Orientale, Viale T. Michel, 11, 15121 Alessandria, Italy ,grid.16563.370000000121663741AI@UPO, Università del Piemonte Orientale, Vercelli, Italy
| | - Marzio Pennisi
- grid.16563.370000000121663741DISIT, Computer Science Institute, Università del Piemonte Orientale, Viale T. Michel, 11, 15121 Alessandria, Italy ,grid.16563.370000000121663741AI@UPO, Università del Piemonte Orientale, Vercelli, Italy
| | - Annalisa Roveta
- Research Laboratory Facility, Research and Innovation Department, Azienda Ospedaliera “SS Antonio e Biagio e Cesare Arrigo”, Alessandria, Italy
| | - Costanza Massarino
- Research Training Innovation Infrastructure, Research and Innovation Department, Azienda Ospedaliera “SS Antonio e Biagio e Cesare Arrigo”, Alessandria, Italy
| | - Antonella Cassinari
- Research Training Innovation Infrastructure, Research and Innovation Department, Azienda Ospedaliera “SS Antonio e Biagio e Cesare Arrigo”, Alessandria, Italy
| | - Marta Betti
- Research Training Innovation Infrastructure, Research and Innovation Department, Azienda Ospedaliera “SS Antonio e Biagio e Cesare Arrigo”, Alessandria, Italy
| | - Tatiana Bolgeo
- Research Training Innovation Infrastructure, Research and Innovation Department, Azienda Ospedaliera “SS Antonio e Biagio e Cesare Arrigo”, Alessandria, Italy
| | - Marinella Bertolotti
- Research Training Innovation Infrastructure, Research and Innovation Department, Azienda Ospedaliera “SS Antonio e Biagio e Cesare Arrigo”, Alessandria, Italy
| | - Emanuele Rava
- grid.16563.370000000121663741DISIT, Università del Piemonte Orientale, Viale T. Michel, 11, 15121 Alessandria, Italy
| | - Antonio Maconi
- Research Laboratory Facility, Research and Innovation Department, Azienda Ospedaliera “SS Antonio e Biagio e Cesare Arrigo”, Alessandria, Italy
| |
Collapse
|
6
|
Ramón A, Torres AM, Milara J, Cascón J, Blasco P, Mateo J. eXtreme Gradient Boosting-based method to classify patients with COVID-19. J Investig Med 2022; 70:jim-2021-002278. [PMID: 35850970 DOI: 10.1136/jim-2021-002278] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/15/2022] [Indexed: 01/08/2023]
Abstract
Different demographic, clinical and laboratory variables have been related to the severity and mortality following SARS-CoV-2 infection. Most studies applied traditional statistical methods and in some cases combined with a machine learning (ML) method. This is the first study to date to comparatively analyze five ML methods to select the one that most closely predicts mortality in patients admitted with COVID-19. The aim of this single-center observational study is to classify, based on different types of variables, adult patients with COVID-19 at increased risk of mortality. SARS-CoV-2 infection was defined by a positive reverse transcriptase PCR. A total of 203 patients were admitted between March 15 and June 15, 2020 to a tertiary hospital. Data were extracted from the electronic medical record. Four supervised ML algorithms (k-nearest neighbors (KNN), decision tree (DT), Gaussian naïve Bayes (GNB) and support vector machine (SVM)) were compared with the eXtreme Gradient Boosting (XGB) method proposed to have excellent scalability and high running speed, among other qualities. The results indicate that the XGB method has the best prediction accuracy (92%), high precision (>0.92) and high recall (>0.92). The KNN, SVM and DT approaches present moderate prediction accuracy (>80%), moderate recall (>0.80) and moderate precision (>0.80). The GNB algorithm shows relatively low classification performance. The variables with the greatest weight in predicting mortality were C reactive protein, procalcitonin, glutamyl oxaloacetic transaminase, glutamyl pyruvic transaminase, neutrophils, D-dimer, creatinine, lactic acid, ferritin, days of non-invasive ventilation, septic shock and age. Based on these results, XGB is a solid candidate for correct classification of patients with COVID-19.
Collapse
Affiliation(s)
- Antonio Ramón
- Pharmacy Department, General University Hospital Consortium of Valencia, Valencia, Spain
| | - Ana Maria Torres
- Institute of Technology, Universidad de Castilla-La Mancha, Cuenca, Spain
| | - Javier Milara
- Pharmacy Department, General University Hospital Consortium of Valencia, Valencia, Spain
- Pharmacy Department, University of Valencia, Valencia, Spain
| | - Joaquín Cascón
- Institute of Technology, Universidad de Castilla-La Mancha, Cuenca, Spain
| | - Pilar Blasco
- Pharmacy Department, General University Hospital Consortium of Valencia, Valencia, Spain
| | - Jorge Mateo
- Institute of Technology, Universidad de Castilla-La Mancha, Cuenca, Spain
| |
Collapse
|
7
|
Abdulkareem KH, Mostafa SA, Al-Qudsy ZN, Mohammed MA, Al-Waisy AS, Kadry S, Lee J, Nam Y. Automated System for Identifying COVID-19 Infections in Computed Tomography Images Using Deep Learning Models. JOURNAL OF HEALTHCARE ENGINEERING 2022; 2022:5329014. [PMID: 35368962 PMCID: PMC8968354 DOI: 10.1155/2022/5329014] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/31/2021] [Revised: 01/29/2022] [Accepted: 02/18/2022] [Indexed: 12/24/2022]
Abstract
Coronavirus disease 2019 (COVID-19) is a novel disease that affects healthcare on a global scale and cannot be ignored because of its high fatality rate. Computed tomography (CT) images are presently being employed to assist doctors in detecting COVID-19 in its early stages. In several scenarios, a combination of epidemiological criteria (contact during the incubation period), the existence of clinical symptoms, laboratory tests (nucleic acid amplification tests), and clinical imaging-based tests are used to diagnose COVID-19. This method can miss patients and cause more complications. Deep learning is one of the techniques that has been proven to be prominent and reliable in several diagnostic domains involving medical imaging. This study utilizes a convolutional neural network (CNN), stacked autoencoder, and deep neural network to develop a COVID-19 diagnostic system. In this system, classification undergoes some modification before applying the three CT image techniques to determine normal and COVID-19 cases. A large-scale and challenging CT image dataset was used in the training process of the employed deep learning model and reporting their final performance. Experimental outcomes show that the highest accuracy rate was achieved using the CNN model with an accuracy of 88.30%, a sensitivity of 87.65%, and a specificity of 87.97%. Furthermore, the proposed system has outperformed the current existing state-of-the-art models in detecting the COVID-19 virus using CT images.
Collapse
Affiliation(s)
| | - Salama A. Mostafa
- Faculty of Computer Science and Information Technology, Universiti Tun Hussein Onn Malaysia, 86400 Batu Pahat, Johor, Malaysia
| | - Zainab N. Al-Qudsy
- Computer Sciences Department, Baghdad College of Economic Sciences University, Baghdad, Iraq
| | - Mazin Abed Mohammed
- College of Computer Science and Information Technology, University of Anbar, 11 Ramadi, Anbar, Iraq
| | - Alaa S. Al-Waisy
- Communications Engineering Techniques Department Information Technology Collage, Imam Ja'afar Al-Sadiq University, Baghdad, Iraq
| | - Seifedine Kadry
- Faculty of Applied Computing and Technology, Noroff University College, Kristiansand, Norway
| | - Jinseok Lee
- Department of Biomedical Engineering, College of Electronics and Information, Kyung Hee University, Yongin-si, Gyeonggi-do 17104, Republic of Korea
| | - Yunyoung Nam
- Department of Computer Science and Engineering, Soonchunhyang University, Asan 31538, Republic of Korea
| |
Collapse
|
8
|
Yland JJ, Wang T, Zad Z, Willis SK, Wang TR, Wesselink AK, Jiang T, Hatch EE, Wise LA, Paschalidis IC. Predictive models of pregnancy based on data from a preconception cohort study. Hum Reprod 2022; 37:565-576. [PMID: 35024824 PMCID: PMC8888990 DOI: 10.1093/humrep/deab280] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2021] [Revised: 11/30/2021] [Indexed: 01/16/2023] Open
Abstract
STUDY QUESTION Can we derive adequate models to predict the probability of conception among couples actively trying to conceive? SUMMARY ANSWER Leveraging data collected from female participants in a North American preconception cohort study, we developed models to predict pregnancy with performance of ∼70% in the area under the receiver operating characteristic curve (AUC). WHAT IS KNOWN ALREADY Earlier work has focused primarily on identifying individual risk factors for infertility. Several predictive models have been developed in subfertile populations, with relatively low discrimination (AUC: 59-64%). STUDY DESIGN, SIZE, DURATION Study participants were female, aged 21-45 years, residents of the USA or Canada, not using fertility treatment, and actively trying to conceive at enrollment (2013-2019). Participants completed a baseline questionnaire at enrollment and follow-up questionnaires every 2 months for up to 12 months or until conception. We used data from 4133 participants with no more than one menstrual cycle of pregnancy attempt at study entry. PARTICIPANTS/MATERIALS, SETTING, METHODS On the baseline questionnaire, participants reported data on sociodemographic factors, lifestyle and behavioral factors, diet quality, medical history and selected male partner characteristics. A total of 163 predictors were considered in this study. We implemented regularized logistic regression, support vector machines, neural networks and gradient boosted decision trees to derive models predicting the probability of pregnancy: (i) within fewer than 12 menstrual cycles of pregnancy attempt time (Model I), and (ii) within 6 menstrual cycles of pregnancy attempt time (Model II). Cox models were used to predict the probability of pregnancy within each menstrual cycle for up to 12 cycles of follow-up (Model III). We assessed model performance using the AUC and the weighted-F1 score for Models I and II, and the concordance index for Model III. MAIN RESULTS AND THE ROLE OF CHANCE Model I and II AUCs were 70% and 66%, respectively, in parsimonious models, and the concordance index for Model III was 63%. The predictors that were positively associated with pregnancy in all models were: having previously breastfed an infant and using multivitamins or folic acid supplements. The predictors that were inversely associated with pregnancy in all models were: female age, female BMI and history of infertility. Among nulligravid women with no history of infertility, the most important predictors were: female age, female BMI, male BMI, use of a fertility app, attempt time at study entry and perceived stress. LIMITATIONS, REASONS FOR CAUTION Reliance on self-reported predictor data could have introduced misclassification, which would likely be non-differential with respect to the pregnancy outcome given the prospective design. In addition, we cannot be certain that all relevant predictor variables were considered. Finally, though we validated the models using split-sample replication techniques, we did not conduct an external validation study. WIDER IMPLICATIONS OF THE FINDINGS Given a wide range of predictor data, machine learning algorithms can be leveraged to analyze epidemiologic data and predict the probability of conception with discrimination that exceeds earlier work. STUDY FUNDING/COMPETING INTEREST(S) The research was partially supported by the U.S. National Science Foundation (under grants DMS-1664644, CNS-1645681 and IIS-1914792) and the National Institutes for Health (under grants R01 GM135930 and UL54 TR004130). In the last 3 years, L.A.W. has received in-kind donations for primary data collection in PRESTO from FertilityFriend.com, Kindara.com, Sandstone Diagnostics and Swiss Precision Diagnostics. L.A.W. also serves as a fibroid consultant to AbbVie, Inc. The other authors declare no competing interests. TRIAL REGISTRATION NUMBER N/A.
Collapse
Affiliation(s)
- Jennifer J Yland
- Department of Epidemiology, Boston University School of Public Health, Boston, MA, USA,Correspondence address. Department of Epidemiology, Boston University School of Public Health, 715 Albany Street, Boston, MA 02118, USA. E-mail:
| | - Taiyao Wang
- Center for Information and Systems Engineering, Boston University, Boston, MA, USA,Philips Research North America, Cambridge, MA, USA
| | - Zahra Zad
- Center for Information and Systems Engineering, Boston University, Boston, MA, USA,Division of Systems Engineering, Department of Electrical and Computer Engineering, Boston University, Boston, MA, USA
| | - Sydney K Willis
- Department of Epidemiology, Boston University School of Public Health, Boston, MA, USA
| | - Tanran R Wang
- Department of Epidemiology, Boston University School of Public Health, Boston, MA, USA
| | - Amelia K Wesselink
- Department of Epidemiology, Boston University School of Public Health, Boston, MA, USA
| | - Tammy Jiang
- Department of Epidemiology, Boston University School of Public Health, Boston, MA, USA
| | - Elizabeth E Hatch
- Department of Epidemiology, Boston University School of Public Health, Boston, MA, USA
| | - Lauren A Wise
- Department of Epidemiology, Boston University School of Public Health, Boston, MA, USA
| | - Ioannis Ch Paschalidis
- Center for Information and Systems Engineering, Boston University, Boston, MA, USA,Division of Systems Engineering, Department of Electrical and Computer Engineering, Boston University, Boston, MA, USA,Department of Biomedical Engineering, Boston University, Boston, MA, USA
| |
Collapse
|
9
|
Adamidi ES, Mitsis K, Nikita KS. Artificial intelligence in clinical care amidst COVID-19 pandemic: A systematic review. Comput Struct Biotechnol J 2021; 19:2833-2850. [PMID: 34025952 PMCID: PMC8123783 DOI: 10.1016/j.csbj.2021.05.010] [Citation(s) in RCA: 46] [Impact Index Per Article: 15.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2021] [Revised: 05/01/2021] [Accepted: 05/02/2021] [Indexed: 12/23/2022] Open
Abstract
The worldwide health crisis caused by the SARS-Cov-2 virus has resulted in>3 million deaths so far. Improving early screening, diagnosis and prognosis of the disease are critical steps in assisting healthcare professionals to save lives during this pandemic. Since WHO declared the COVID-19 outbreak as a pandemic, several studies have been conducted using Artificial Intelligence techniques to optimize these steps on clinical settings in terms of quality, accuracy and most importantly time. The objective of this study is to conduct a systematic literature review on published and preprint reports of Artificial Intelligence models developed and validated for screening, diagnosis and prognosis of the coronavirus disease 2019. We included 101 studies, published from January 1st, 2020 to December 30th, 2020, that developed AI prediction models which can be applied in the clinical setting. We identified in total 14 models for screening, 38 diagnostic models for detecting COVID-19 and 50 prognostic models for predicting ICU need, ventilator need, mortality risk, severity assessment or hospital length stay. Moreover, 43 studies were based on medical imaging and 58 studies on the use of clinical parameters, laboratory results or demographic features. Several heterogeneous predictors derived from multimodal data were identified. Analysis of these multimodal data, captured from various sources, in terms of prominence for each category of the included studies, was performed. Finally, Risk of Bias (RoB) analysis was also conducted to examine the applicability of the included studies in the clinical setting and assist healthcare providers, guideline developers, and policymakers.
Collapse
Key Words
- ABG, Arterial Blood Gas
- ADA, Adenosine Deaminase
- AI, Artificial Intelligence
- ANN, Artificial Neural Networks
- APTT, Activated Partial Thromboplastin Time
- ARMED, Attribute Reduction with Multi-objective Decomposition Ensemble optimizer
- AUC, Area Under the Curve
- Acc, Accuracy
- Adaboost, Adaptive Boosting
- Apol AI, Apolipoprotein AI
- Apol B, Apolipoprotein B
- Artificial intelligence
- BNB, Bernoulli Naïve Bayes
- BUN, Blood Urea Nitrogen
- CI, Confidence Interval
- CK-MB, Creatine Kinase isoenzyme
- CNN, Convolutional Neural Networks
- COVID-19
- CPP, COVID-19 Positive Patients
- CRP, C-Reactive Protein
- CRT, Classification and Regression Decision Tree
- CoxPH, Cox Proportional Hazards
- DCNN, Deep Convolutional Neural Networks
- DL, Deep Learning
- DLC, Density Lipoprotein Cholesterol
- DNN, Deep Neural Networks
- DT, Decision Tree
- Diagnosis
- ED, Emergency Department
- ESR, Erythrocyte Sedimentation Rate
- ET, Extra Trees
- FCV, Fold Cross Validation
- FL, Federated Learning
- FiO2, Fraction of Inspiration O2
- GBDT, Gradient Boost Decision Tree
- GBM light, Gradient Boosting Machine light
- GDCNN, Genetic Deep Learning Convolutional Neural Network
- GFR, Glomerular Filtration Rate
- GFS, Gradient boosted feature selection
- GGT, Glutamyl Transpeptidase
- GNB, Gaussian Naïve Bayes
- HDLC, High Density Lipoprotein Cholesterol
- INR, International Normalized Ratio
- Inception Resnet, Inception Residual Neural Network
- L1LR, L1 Regularized Logistic Regression
- LASSO, Least Absolute Shrinkage and Selection Operator
- LDA, Linear Discriminant Analysis
- LDH, Lactate Dehydrogenase
- LDLC, Low Density Lipoprotein Cholesterol
- LR, Logistic Regression
- LSTM, Long-Short Term Memory
- MCHC, Mean Corpuscular Hemoglobin Concentration
- MCV, Mean corpuscular volume
- ML, Machine Learning
- MLP, MultiLayer Perceptron
- MPV, Mean Platelet Volume
- MRMR, Maximum Relevance Minimum Redundancy
- Multimodal data
- NB, Naïve Bayes
- NLP, Natural Language Processing
- NPV, Negative Predictive Values
- Nadam optimizer, Nesterov Accelerated Adaptive Moment optimizer
- OB, Occult Blood test
- PCT, Thrombocytocrit
- PPV, Positive Predictive Values
- PWD, Platelet Distribution Width
- PaO2, Arterial Oxygen Tension
- Paco2, Arterial Carbondioxide Tension
- Prognosis
- RBC, Red Blood Cell
- RBF, Radial Basis Function
- RBP, Retinol Binding Protein
- RDW, Red blood cell Distribution Width
- RF, Random Forest
- RFE, Recursive Feature Elimination
- RSV, Respiratory Syncytial Virus
- SEN, Sensitivity
- SG, Specific Gravity
- SMOTE, Synthetic Minority Oversampling Technique
- SPE, Specificity
- SRLSR, Sparse Rescaled Linear Square Regression
- SVM, Support Vector Machine
- SaO2, Arterial Oxygen saturation
- Screening
- TBA, Total Bile Acid
- TTS, Training Test Split
- WBC, White Blood Cell count
- XGB, eXtreme Gradient Boost
- k-NN, K-Nearest Neighbor
Collapse
Affiliation(s)
- Eleni S. Adamidi
- Biomedical Simulations and Imaging Lab, School of Electrical and Computer Engineering, National Technical University of Athens, Greece
| | - Konstantinos Mitsis
- Biomedical Simulations and Imaging Lab, School of Electrical and Computer Engineering, National Technical University of Athens, Greece
| | - Konstantina S. Nikita
- Biomedical Simulations and Imaging Lab, School of Electrical and Computer Engineering, National Technical University of Athens, Greece
| |
Collapse
|
10
|
Abd El-raheem GOH, Awad Yousif MA, Ibrahim Mohamed DS. Prediction of COVID-19 mortality among hospitalized patients in Sudan.. [DOI: 10.1101/2021.03.09.21253179] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/01/2023]
Abstract
AbstractBackgroundCOVID-19 was primarily reported in China. The mortality rate across countries had ranged from 1% up to more than 10% and it is underestimated in some countries. Advanced age is the most frequently reported factor associated to mortality. Other factors were the presence of comorbidities such as diabetes mellitus, hypertension and obesity. Several models for mortality prediction had been developed to assist in improving the prognosis. The aim of our study was to assess the factors related to mortality among COVID-19 patients and develop a prediction model based on these factors.MethodsA retrospective cohort study assessed the factors related to the mortality among COVID-19 patients who attended Imperial Hospital isolation centre on November-December, 2020, Khartoum, Sudan. Statistical tests performed were chi-square test, odds ratio and regression to develop the prediction model. Tests were considered statistically significant whenp< 0.05.Results105 patients were studied. 29% of the patients were deceased, while, 71% were discharged alive. A statistically significant association was found between the age and severity with regards to mortality rate (p=0.034, 0.018 respectively). The model equation for mortality prediction: Mortality = −14.724+ (1.387* Age) + (−0.323* Gender) + (1.814* Admission) + (0.193* Ischemic Heart Disease) + (−0.369* Fever) + (1.595* Cough) + (1.953* Complications) + (0.149* Duration of hospitalization) + (0.999* Enoxaparin dose).ConclusionsAge, admission ward, cough and enoxaparin dose were statistically significant predictors for COVID-19 mortality (p= 0.014, 0.011, 0.015, 0.006 respectively).
Collapse
|
11
|
Wynants L, Van Calster B, Collins GS, Riley RD, Heinze G, Schuit E, Bonten MMJ, Dahly DL, Damen JAA, Debray TPA, de Jong VMT, De Vos M, Dhiman P, Haller MC, Harhay MO, Henckaerts L, Heus P, Kammer M, Kreuzberger N, Lohmann A, Luijken K, Ma J, Martin GP, McLernon DJ, Andaur Navarro CL, Reitsma JB, Sergeant JC, Shi C, Skoetz N, Smits LJM, Snell KIE, Sperrin M, Spijker R, Steyerberg EW, Takada T, Tzoulaki I, van Kuijk SMJ, van Bussel B, van der Horst ICC, van Royen FS, Verbakel JY, Wallisch C, Wilkinson J, Wolff R, Hooft L, Moons KGM, van Smeden M. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ 2020; 369:m1328. [PMID: 32265220 PMCID: PMC7222643 DOI: 10.1136/bmj.m1328] [Citation(s) in RCA: 1640] [Impact Index Per Article: 410.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 03/31/2020] [Indexed: 12/12/2022]
Abstract
OBJECTIVE To review and appraise the validity and usefulness of published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at increased risk of covid-19 infection or being admitted to hospital with the disease. DESIGN Living systematic review and critical appraisal by the COVID-PRECISE (Precise Risk Estimation to optimise covid-19 Care for Infected or Suspected patients in diverse sEttings) group. DATA SOURCES PubMed and Embase through Ovid, up to 1 July 2020, supplemented with arXiv, medRxiv, and bioRxiv up to 5 May 2020. STUDY SELECTION Studies that developed or validated a multivariable covid-19 related prediction model. DATA EXTRACTION At least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool). RESULTS 37 421 titles were screened, and 169 studies describing 232 prediction models were included. The review identified seven models for identifying people at risk in the general population; 118 diagnostic models for detecting covid-19 (75 were based on medical imaging, 10 to diagnose disease severity); and 107 prognostic models for predicting mortality risk, progression to severe disease, intensive care unit admission, ventilation, intubation, or length of hospital stay. The most frequent types of predictors included in the covid-19 prediction models are vital signs, age, comorbidities, and image features. Flu-like symptoms are frequently predictive in diagnostic models, while sex, C reactive protein, and lymphocyte counts are frequent prognostic factors. Reported C index estimates from the strongest form of validation available per model ranged from 0.71 to 0.99 in prediction models for the general population, from 0.65 to more than 0.99 in diagnostic models, and from 0.54 to 0.99 in prognostic models. All models were rated at high or unclear risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, high risk of model overfitting, and unclear reporting. Many models did not include a description of the target population (n=27, 12%) or care setting (n=75, 32%), and only 11 (5%) were externally validated by a calibration plot. The Jehi diagnostic model and the 4C mortality score were identified as promising models. CONCLUSION Prediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that almost all pubished prediction models are poorly reported, and at high risk of bias such that their reported predictive performance is probably optimistic. However, we have identified two (one diagnostic and one prognostic) promising models that should soon be validated in multiple cohorts, preferably through collaborative efforts and data sharing to also allow an investigation of the stability and heterogeneity in their performance across populations and settings. Details on all reviewed models are publicly available at https://www.covprecise.org/. Methodological guidance as provided in this paper should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, prediction model authors should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline. SYSTEMATIC REVIEW REGISTRATION Protocol https://osf.io/ehc47/, registration https://osf.io/wy245. READERS' NOTE This article is a living systematic review that will be updated to reflect emerging evidence. Updates may occur for up to two years from the date of original publication. This version is update 3 of the original article published on 7 April 2020 (BMJ 2020;369:m1328). Previous updates can be found as data supplements (https://www.bmj.com/content/369/bmj.m1328/related#datasupp). When citing this paper please consider adding the update number and date of access for clarity.
Collapse
Affiliation(s)
- Laure Wynants
- Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Peter Debyeplein 1, 6229 HA Maastricht, Netherlands
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium
| | - Ben Van Calster
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, Netherlands
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Musculoskeletal Sciences, University of Oxford, Oxford, UK
- NIHR Oxford Biomedical Research Centre, John Radcliffe Hospital, Oxford, UK
| | - Richard D Riley
- Centre for Prognosis Research, School of Primary, Community and Social Care, Keele University, Keele, UK
| | - Georg Heinze
- Section for Clinical Biometrics, Centre for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Vienna, Austria
| | - Ewoud Schuit
- Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Cochrane Netherlands, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Marc M J Bonten
- Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Department of Medical Microbiology, University Medical Centre Utrecht, Utrecht, Netherlands
| | - Darren L Dahly
- HRB Clinical Research Facility, Cork, Ireland
- School of Public Health, University College Cork, Cork, Ireland
| | - Johanna A A Damen
- Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Cochrane Netherlands, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Thomas P A Debray
- Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Cochrane Netherlands, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Valentijn M T de Jong
- Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Cochrane Netherlands, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Maarten De Vos
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium
- Department of Electrical Engineering, ESAT Stadius, KU Leuven, Leuven, Belgium
| | - Paul Dhiman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Musculoskeletal Sciences, University of Oxford, Oxford, UK
- NIHR Oxford Biomedical Research Centre, John Radcliffe Hospital, Oxford, UK
| | - Maria C Haller
- Section for Clinical Biometrics, Centre for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Vienna, Austria
- Ordensklinikum Linz, Hospital Elisabethinen, Department of Nephrology, Linz, Austria
| | - Michael O Harhay
- Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Palliative and Advanced Illness Research Center and Division of Pulmonary and Critical Care Medicine, Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Liesbet Henckaerts
- Department of Microbiology, Immunology and Transplantation, KU Leuven-University of Leuven, Leuven, Belgium
- Department of General Internal Medicine, KU Leuven-University Hospitals Leuven, Leuven, Belgium
| | - Pauline Heus
- Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Cochrane Netherlands, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Michael Kammer
- Section for Clinical Biometrics, Centre for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Vienna, Austria
- Department of Nephrology, Medical University of Vienna, Vienna, Austria
| | - Nina Kreuzberger
- Evidence-Based Oncology, Department I of Internal Medicine and Centre for Integrated Oncology Aachen Bonn Cologne Dusseldorf, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany
| | - Anna Lohmann
- Department of Clinical Epidemiology, Leiden University Medical Centre, Leiden, Netherlands
| | - Kim Luijken
- Department of Clinical Epidemiology, Leiden University Medical Centre, Leiden, Netherlands
| | - Jie Ma
- NIHR Oxford Biomedical Research Centre, John Radcliffe Hospital, Oxford, UK
| | - Glen P Martin
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK
| | - David J McLernon
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
| | - Constanza L Andaur Navarro
- Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Cochrane Netherlands, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Johannes B Reitsma
- Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Cochrane Netherlands, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Jamie C Sergeant
- Centre for Biostatistics, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
- Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Chunhu Shi
- Division of Nursing, Midwifery and Social Work, School of Health Sciences, University of Manchester, Manchester, UK
| | - Nicole Skoetz
- Department of Nephrology, Medical University of Vienna, Vienna, Austria
| | - Luc J M Smits
- Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Peter Debyeplein 1, 6229 HA Maastricht, Netherlands
| | - Kym I E Snell
- Centre for Prognosis Research, School of Primary, Community and Social Care, Keele University, Keele, UK
| | - Matthew Sperrin
- Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK
| | - René Spijker
- Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Cochrane Netherlands, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Medical Library, Netherlands
| | - Ewout W Steyerberg
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, Netherlands
| | - Toshihiko Takada
- Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Ioanna Tzoulaki
- Department of Epidemiology and Biostatistics, Imperial College London School of Public Health, London, UK
- Department of Hygiene and Epidemiology, University of Ioannina Medical School, Ioannina, Greece
| | - Sander M J van Kuijk
- Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre+, Maastricht, Netherlands
| | - Bas van Bussel
- Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Peter Debyeplein 1, 6229 HA Maastricht, Netherlands
- Department of Intensive Care, Maastricht University Medical Centre+, Maastricht University, Maastricht, Netherlands
| | - Iwan C C van der Horst
- Department of Intensive Care, Maastricht University Medical Centre+, Maastricht University, Maastricht, Netherlands
| | - Florien S van Royen
- Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Jan Y Verbakel
- EPI-Centre, Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Christine Wallisch
- Section for Clinical Biometrics, Centre for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Vienna, Austria
- Charité Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin, Germany
- Berlin Institute of Health, Berlin, Germany
| | - Jack Wilkinson
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK
| | | | - Lotty Hooft
- Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Cochrane Netherlands, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Cochrane Netherlands, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Maarten van Smeden
- Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| |
Collapse
|