1
|
Razavi SR, Szun T, Zaremba AC, Shah AH, Moussavi Z. 1-Year Mortality Prediction through Artificial Intelligence Using Hemodynamic Trace Analysis among Patients with ST Elevation Myocardial Infarction. MEDICINA (KAUNAS, LITHUANIA) 2024; 60:558. [PMID: 38674204 PMCID: PMC11052412 DOI: 10.3390/medicina60040558] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/02/2024] [Revised: 03/23/2024] [Accepted: 03/26/2024] [Indexed: 04/28/2024]
Abstract
Background and Objectives: Patients presenting with ST Elevation Myocardial Infarction (STEMI) due to occlusive coronary arteries remain at a higher risk of excess morbidity and mortality despite being treated with primary percutaneous coronary intervention (PPCI). Identifying high-risk patients is prudent so that close monitoring and timely interventions can improve outcomes. Materials and Methods: A cohort of 605 STEMI patients [64.2 ± 13.2 years, 432 (71.41%) males] treated with PPCI were recruited. Their arterial pressure (AP) wave recorded throughout the PPCI procedure was analyzed to extract features to predict 1-year mortality. After denoising and extracting features, we developed two distinct feature selection strategies. The first strategy uses linear discriminant analysis (LDA), and the second employs principal component analysis (PCA), with each method selecting the top five features. Then, three machine learning algorithms were employed: LDA, K-nearest neighbor (KNN), and support vector machine (SVM). Results: The performance of these algorithms, measured by the area under the curve (AUC), ranged from 0.73 to 0.77, with accuracy, specificity, and sensitivity ranging between 68% and 73%. Moreover, we extended the analysis by incorporating demographics, risk factors, and catheterization information. This significantly improved the overall accuracy and specificity to more than 76% while maintaining the same level of sensitivity. This resulted in an AUC greater than 0.80 for most models. Conclusions: Machine learning algorithms analyzing hemodynamic traces in STEMI patients identify high-risk patients at risk of mortality.
Collapse
Affiliation(s)
- Seyed Reza Razavi
- Biomedical Engineering Program, University of Manitoba, Winnipeg, MB R3T 5V6, Canada;
| | - Tyler Szun
- Department of Medicine, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB R3E 3P5, Canada; (T.S.); (A.C.Z.); (A.H.S.)
| | - Alexander C. Zaremba
- Department of Medicine, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB R3E 3P5, Canada; (T.S.); (A.C.Z.); (A.H.S.)
| | - Ashish H. Shah
- Department of Medicine, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB R3E 3P5, Canada; (T.S.); (A.C.Z.); (A.H.S.)
| | - Zahra Moussavi
- Biomedical Engineering Program, University of Manitoba, Winnipeg, MB R3T 5V6, Canada;
| |
Collapse
|
2
|
Zhang X, Wang X, Xu L, Liu J, Ren P, Wu H. The predictive value of machine learning for mortality risk in patients with acute coronary syndromes: a systematic review and meta-analysis. Eur J Med Res 2023; 28:451. [PMID: 37864271 PMCID: PMC10588162 DOI: 10.1186/s40001-023-01027-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Accepted: 01/20/2023] [Indexed: 10/22/2023] Open
Abstract
BACKGROUND Acute coronary syndromes (ACS) are the leading cause of global death. Optimizing mortality risk prediction and early identification of high-risk patients is essential for developing targeted prevention strategies. Many researchers have built machine learning (ML) models to predict the mortality risk in ACS patients. Our meta-analysis aimed to evaluate the predictive value of various ML models in predicting death in ACS patients at different times. METHODS PubMed, Embase, Web of Science, and Cochrane Library were searched systematically from database establishment to March 12, 2022 for studies developing or validating at least one ML predictive model for death in ACS patients. We used PROBAST to assess the risk of bias in the reported predictive models and a random-effects model to assess the pooled C-index and accuracy of these models. RESULTS Fifty papers were included, involving 216 ML prediction models, 119 of which were externally validated. The combined C-index of the ML models in the validation cohort predicting the in-hospital mortality, 30-day mortality, 3- or 6-month mortality, and 1 year or above mortality in ACS patients were 0.8633 (95% CI 0.8467-0.8802), 0.8296 (95% CI 0.8134-0.8462), 0.8205 (95% CI 0.7881-0.8541), and 0.8197 (95% CI 0.8042-0.8354), respectively, with the corresponding combined accuracy of 0.8569 (95% CI 0.8411-0.8715), 0.8282 (95% CI 0.7922-0.8591), 0.7303 (95% CI 0.7184-0.7418), and 0.7837 (95% CI 0.7455-0.8175), indicating that the ML models were relatively excellent in predicting ACS mortality at different times. Furthermore, common predictors of death in ML models included age, sex, systolic blood pressure, serum creatinine, Killip class, heart rate, diastolic blood pressure, blood glucose, and hemoglobin. CONCLUSIONS The ML models had excellent predictive power for mortality in ACS, and the methodologies may need to be addressed before they can be used in clinical practice.
Collapse
Affiliation(s)
- Xiaoxiao Zhang
- Dongzhimen Hospital, Beijing University of Chinese Medicine, Beijing, China
| | - Xi Wang
- Dongzhimen Hospital, Beijing University of Chinese Medicine, Beijing, China
| | - Luxin Xu
- Dongzhimen Hospital, Beijing University of Chinese Medicine, Beijing, China
| | - Jia Liu
- Dongzhimen Hospital, Beijing University of Chinese Medicine, Beijing, China
| | - Peng Ren
- School of Life Science and Engineering, Southwest University of Science and Technology, Mianyang, China
| | - Huanlin Wu
- Dongzhimen Hospital, Beijing University of Chinese Medicine, Beijing, China.
| |
Collapse
|
3
|
Chen W, Tan X, Du X, Li Q, Yuan M, Ni H, Wang Y, Du J. Prediction models for major adverse cardiovascular events following ST-segment elevation myocardial infarction and subgroup-specific performance. Front Cardiovasc Med 2023; 10:1181424. [PMID: 37180806 PMCID: PMC10167292 DOI: 10.3389/fcvm.2023.1181424] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Accepted: 03/27/2023] [Indexed: 05/16/2023] Open
Abstract
Background ST-segment elevation myocardial infarction (STEMI) patients are at a high residual risk of major adverse cardiovascular events (MACEs) after revascularization. Risk factors modify prognostic risk in distinct ways in different STEMI subpopulations. We developed a MACEs prediction model in patients with STEMI and examined its performance across subgroups. Methods Machine-learning models based on 63 clinical features were trained in patients with STEMI who underwent PCI. The best-performing model (the iPROMPT score) was further validated in an external cohort. Its predictive value and variable contribution were studied in the entire population and subgroups. Results Over 2.56 and 2.84 years, 5.0% and 8.33% of patients experienced MACEs in the derivation and external validation cohorts, respectively. The iPROMPT score predictors were ST-segment deviation, brain natriuretic peptide (BNP), low-density lipoprotein cholesterol (LDL-C), estimated glomerular filtration rate (eGFR), age, hemoglobin, and white blood cell (WBC) count. The iPROMPT score improved the predictive value of the existing risk score, with an increase in the area under the curve to 0.837 [95% confidence interval (CI): 0.784-0.889] in the derivation cohort and 0.730 (95% CI: 0.293-1.162) in the external validation cohort. Comparable performance was observed between subgroups. The ST-segment deviation was the most important predictor, followed by LDL-C in hypertensive patients, BNP in males, WBC count in females with diabetes mellitus, and eGFR in patients without diabetes mellitus. Hemoglobin was the top predictor in non-hypertensive patients. Conclusion The iPROMPT score predicts long-term MACEs following STEMI and provides insights into the pathophysiological mechanisms for subgroup differences.
Collapse
Affiliation(s)
- Weiyao Chen
- Department of Physiology and Pathophysiology, School of Basic Medical Sciences, Tianjin Medical University, Tianjin, China
- Beijing Institute of Heart Lung and Blood Vessel Diseases, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
- The Key Laboratory of Remodeling-Related Cardiovascular Diseases, Ministry of Education, Beijing Collaborative Innovation Centre for Cardiovascular Disorders, Beijing, China
| | - Xin Tan
- Beijing Institute of Heart Lung and Blood Vessel Diseases, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
- The Key Laboratory of Remodeling-Related Cardiovascular Diseases, Ministry of Education, Beijing Collaborative Innovation Centre for Cardiovascular Disorders, Beijing, China
| | - Xiaoyu Du
- Beijing Institute of Heart Lung and Blood Vessel Diseases, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
- The Key Laboratory of Remodeling-Related Cardiovascular Diseases, Ministry of Education, Beijing Collaborative Innovation Centre for Cardiovascular Disorders, Beijing, China
- Center for Cardiovascular Medicine, The First Hospital of Jilin University, Changchun, China
| | - Qin Li
- Beijing Institute of Heart Lung and Blood Vessel Diseases, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
- The Key Laboratory of Remodeling-Related Cardiovascular Diseases, Ministry of Education, Beijing Collaborative Innovation Centre for Cardiovascular Disorders, Beijing, China
| | - Meng Yuan
- Beijing Institute of Heart Lung and Blood Vessel Diseases, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
- The Key Laboratory of Remodeling-Related Cardiovascular Diseases, Ministry of Education, Beijing Collaborative Innovation Centre for Cardiovascular Disorders, Beijing, China
| | - Hui Ni
- Beijing Institute of Heart Lung and Blood Vessel Diseases, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
- The Key Laboratory of Remodeling-Related Cardiovascular Diseases, Ministry of Education, Beijing Collaborative Innovation Centre for Cardiovascular Disorders, Beijing, China
| | - Yuan Wang
- Beijing Institute of Heart Lung and Blood Vessel Diseases, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
- The Key Laboratory of Remodeling-Related Cardiovascular Diseases, Ministry of Education, Beijing Collaborative Innovation Centre for Cardiovascular Disorders, Beijing, China
| | - Jie Du
- Department of Physiology and Pathophysiology, School of Basic Medical Sciences, Tianjin Medical University, Tianjin, China
- Beijing Institute of Heart Lung and Blood Vessel Diseases, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
- The Key Laboratory of Remodeling-Related Cardiovascular Diseases, Ministry of Education, Beijing Collaborative Innovation Centre for Cardiovascular Disorders, Beijing, China
| |
Collapse
|
4
|
Oliveira M, Seringa J, Pinto FJ, Henriques R, Magalhães T. Machine learning prediction of mortality in Acute Myocardial Infarction. BMC Med Inform Decis Mak 2023; 23:70. [PMID: 37072766 PMCID: PMC10111317 DOI: 10.1186/s12911-023-02168-6] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2022] [Accepted: 04/04/2023] [Indexed: 04/20/2023] Open
Abstract
BACKGROUND Acute Myocardial Infarction (AMI) is the leading cause of death in Portugal and globally. The present investigation created a model based on machine learning for predictive analysis of mortality in patients with AMI upon admission, using different variables to analyse their impact on predictive models. METHODS Three experiments were built for mortality in AMI in a Portuguese hospital between 2013 and 2015 using various machine learning techniques. The three experiments differed in the number and type of variables used. We used a discharged patients' episodes database, including administrative data, laboratory data, and cardiac and physiologic test results, whose primary diagnosis was AMI. RESULTS Results show that for Experiment 1, Stochastic Gradient Descent was more suitable than the other classification models, with a classification accuracy of 80%, a recall of 77%, and a discriminatory capacity with an AUC of 79%. Adding new variables to the models increased AUC in Experiment 2 to 81% for the Support Vector Machine method. In Experiment 3, we obtained an AUC, in Stochastic Gradient Descent, of 88% and a recall of 80%. These results were obtained when applying feature selection and the SMOTE technique to overcome imbalanced data. CONCLUSIONS Our results show that the introduction of new variables, namely laboratory data, impacts the performance of the methods, reinforcing the premise that no single approach is adapted to all situations regarding AMI mortality prediction. Instead, they must be selected, considering the context and the information available. Integrating Artificial Intelligence (AI) and machine learning with clinical decision-making can transform care, making clinical practice more efficient, faster, personalised, and effective. AI emerges as an alternative to traditional models since it has the potential to explore large amounts of information automatically and systematically.
Collapse
Affiliation(s)
- Mariana Oliveira
- NOVA National School of Public Health, Universidade NOVA Lisboa, Lisbon, Portugal
| | - Joana Seringa
- NOVA National School of Public Health, Universidade NOVA Lisboa, Lisbon, Portugal
| | - Fausto José Pinto
- Serviço de Cardiologia, Centro Hospitalar Universitário de Lisboa Norte (CHULN), CAML, CCUL, Faculdade de Medicina, Universidade de Lisboa, Lisbon, Portugal
| | - Roberto Henriques
- NOVA Information Management School (NOVA IMS), Universidade Nova de Lisboa, 1070-312, Lisbon, Portugal.
| | - Teresa Magalhães
- NOVA National School of Public Health, Public Health Research Centre, Comprehensive Health Research Center, CHRC, Nova University of Lisbon, Lisbon, Portugal
| |
Collapse
|
5
|
Kasim S, Malek S, Cheen S, Safiruz MS, Ahmad WAW, Ibrahim KS, Aziz F, Negishi K, Ibrahim N. In-hospital risk stratification algorithm of Asian elderly patients. Sci Rep 2022; 12:17592. [PMID: 36266376 PMCID: PMC9584943 DOI: 10.1038/s41598-022-18839-9] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2022] [Accepted: 08/22/2022] [Indexed: 01/13/2023] Open
Abstract
Limited research has been conducted in Asian elderly patients (aged 65 years and above) for in-hospital mortality prediction after an ST-segment elevation myocardial infarction (STEMI) using Deep Learning (DL) and Machine Learning (ML). We used DL and ML to predict in-hospital mortality in Asian elderly STEMI patients and compared it to a conventional risk score for myocardial infraction outcomes. Malaysia's National Cardiovascular Disease Registry comprises an ethnically diverse Asian elderly population (3991 patients). 50 variables helped in establishing the in-hospital death prediction model. The TIMI score was used to predict mortality using DL and feature selection methods from ML algorithms. The main performance metric was the area under the receiver operating characteristic curve (AUC). The DL and ML model constructed using ML feature selection outperforms the conventional risk scoring score, TIMI (AUC 0.75). DL built from ML features (AUC ranging from 0.93 to 0.95) outscored DL built from all features (AUC 0.93). The TIMI score underestimates mortality in the elderly. TIMI predicts 18.4% higher mortality than the DL algorithm (44.7%). All ML feature selection algorithms identify age, fasting blood glucose, heart rate, Killip class, oral hypoglycemic agent, systolic blood pressure, and total cholesterol as common predictors of mortality in the elderly. In a multi-ethnic population, DL outperformed the TIMI risk score in classifying elderly STEMI patients. ML improves death prediction by identifying separate characteristics in older Asian populations. Continuous testing and validation will improve future risk classification, management, and results.
Collapse
Affiliation(s)
- Sazzli Kasim
- grid.412259.90000 0001 2161 1343Cardiology Department, Faculty of Medicine, Universiti Teknologi MARA (UiTM), Shah Alam, Malaysia ,grid.412259.90000 0001 2161 1343Cardiac Vascular and Lung Research Institute, Universiti Teknologi MARA (UiTM), Shah Alam, Malaysia ,National Heart Association of Malaysia, Heart House, Kuala Lumpur, Malaysia ,grid.412259.90000 0001 2161 1343Faculty of Medicine, Universiti Teknologi MARA (UiTM), Sungai Buloh Campus, Sungai Buloh, Malaysia
| | - Sorayya Malek
- grid.10347.310000 0001 2308 5949Bioinformatics Division, Institute of Biological Sciences, Faculty of Science, University of Malaya, Kuala Lumpur, Malaysia
| | - Song Cheen
- grid.10347.310000 0001 2308 5949Bioinformatics Division, Institute of Biological Sciences, Faculty of Science, University of Malaya, Kuala Lumpur, Malaysia
| | - Muhammad Shahreeza Safiruz
- grid.10347.310000 0001 2308 5949Department of Artificial Intelligence, Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, Malaysia
| | - Wan Azman Wan Ahmad
- National Heart Association of Malaysia, Heart House, Kuala Lumpur, Malaysia ,grid.413018.f0000 0000 8963 3111Division of Cardiology, University Malaya Medical Centre, Kuala Lumpur, Malaysia
| | - Khairul Shafiq Ibrahim
- grid.412259.90000 0001 2161 1343Cardiology Department, Faculty of Medicine, Universiti Teknologi MARA (UiTM), Shah Alam, Malaysia ,grid.412259.90000 0001 2161 1343Cardiac Vascular and Lung Research Institute, Universiti Teknologi MARA (UiTM), Shah Alam, Malaysia ,National Heart Association of Malaysia, Heart House, Kuala Lumpur, Malaysia
| | - Firdaus Aziz
- grid.10347.310000 0001 2308 5949Bioinformatics Division, Institute of Biological Sciences, Faculty of Science, University of Malaya, Kuala Lumpur, Malaysia
| | - Kazuaki Negishi
- grid.1013.30000 0004 1936 834XSydney Medical School Nepean, Faculty of Medicine and Health, Charles Perkins Centre Nepean, The University of Sydney, Sydney, NSW Australia ,grid.413243.30000 0004 0453 1183Nepean Hospital, Sydney, NSW Australia
| | - Nurulain Ibrahim
- grid.412259.90000 0001 2161 1343Faculty of Medicine, Universiti Teknologi MARA (UiTM), Sungai Buloh Campus, Sungai Buloh, Malaysia
| |
Collapse
|
6
|
Zhong Z, Sun S, Weng J, Zhang H, Lin H, Sun J, Pan M, Guo H, Chi J. Machine learning algorithms identifying the risk of new-onset ACS in patients with type 2 diabetes mellitus: A retrospective cohort study. Front Public Health 2022; 10:947204. [PMID: 36148336 PMCID: PMC9486471 DOI: 10.3389/fpubh.2022.947204] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2022] [Accepted: 08/08/2022] [Indexed: 01/21/2023] Open
Abstract
Background In recent years, the prevalence of type 2 diabetes mellitus (T2DM) has increased annually. The major complication of T2DM is cardiovascular disease (CVD). CVD is the main cause of death in T2DM patients, particularly those with comorbid acute coronary syndrome (ACS). Although risk prediction models using multivariate logistic regression are available to assess the probability of new-onset ACS development in T2DM patients, none have been established using machine learning (ML). Methods Between January 2019 and January 2020, we enrolled 521 T2DM patients with new-onset ACS or no ACS from our institution's medical information recording system and divided them into a training dataset and a testing dataset. Seven ML algorithms were used to establish models to assess the probability of ACS coupled with 5-cross validation. Results We established a nomogram to assess the probability of newly diagnosed ACS in T2DM patients with an area under the curve (AUC) of 0.80 in the testing dataset and identified some key features: family history of CVD, history of smoking and drinking, aspartate aminotransferase level, age, neutrophil count, and Killip grade, which accelerated the development of ACS in patients with T2DM. The AUC values of the seven ML models were 0.70-0.96, and random forest model had the best performance (accuracy, 0.89; AUC, 0.96; recall, 0.83; precision, 0.91; F1 score, 0.87). Conclusion ML algorithms, especially random forest model (AUC, 0.961), had higher performance than conventional logistic regression (AUC, 0.801) for assessing new-onset ACS probability in T2DM patients with excellent clinical and diagnostic value.
Collapse
Affiliation(s)
- Zuoquan Zhong
- Department of Cardiology, Shaoxing People's Hospital, Shaoxing Hospital of Zhejiang University, Shaoxing, China
| | - Shiming Sun
- The First Clinical Medical College, Wenzhou Medical University, Wenzhou, China
| | - Jingfan Weng
- Department of Cardiology, Zhejiang University School of Medicine, Hangzhou, China
| | - Hanlin Zhang
- The First Clinical Medical College, Wenzhou Medical University, Wenzhou, China
| | - Hui Lin
- Department of Cardiology, Shaoxing People's Hospital, Shaoxing Hospital of Zhejiang University, Shaoxing, China
| | - Jing Sun
- The First Clinical Medical College, Wenzhou Medical University, Wenzhou, China
| | - Miaohong Pan
- College of Medicine, Shaoxing University, Shaoxing, China
| | - Hangyuan Guo
- College of Medicine, Shaoxing University, Shaoxing, China,*Correspondence: Hangyuan Guo
| | - Jufang Chi
- Department of Cardiology, Shaoxing People's Hospital, Shaoxing Hospital of Zhejiang University, Shaoxing, China,Jufang Chi
| |
Collapse
|
7
|
Chopannejad S, Sadoughi F, Bagherzadeh R, Shekarchi S. Predicting major adverse cardiovascular events in acute coronary syndrome: A scoping review of machine learning approaches. Appl Clin Inform 2022; 13:720-740. [PMID: 35617971 PMCID: PMC9329142 DOI: 10.1055/a-1863-1589] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022] Open
Abstract
BACKGROUND Acute coronary syndrome is the topmost cause of death worldwide; therefore, it is necessary to predict major adverse cardiovascular events and cardiovascular deaths in patients with acute coronary syndrome to make correct and timely clinical decisions. OBJECTIVE The current review aimed to highlight algorithms and important predictor variables through examining those studies which used machine learning algorithms for predicting major adverse cardiovascular events in patients with acute coronary syndrome. METHODS In order to predict major adverse cardiovascular events in patients with acute coronary syndrome, the preferred reporting items for scoping reviews guidelines were used. PubMed, Embase, Web of Science, Scopus, Springer, and IEEE Xplore databases were searched for articles published between 2005 and 2021. The findings of the studies are presented in the form of a narrative synthesis of evidence. RESULTS According to the results, 14 (63.64%) studies did not perform external validation and only used registry data. The algorithms used in this study comprised, inter alia, Regression Logistic, Random Forest, Boosting Ensemble, Non-Boosting Ensemble, Decision Trees, and Naive Bayes. Multiple studies (N=20) achieved a high Area under the ROC Curve between 0.8 to 0.99 in predicting mortality and major adverse cardiovascular events. The predictor variables used in these studies were divided into demographic, clinical, and therapeutic features. However, no study reported the integration of machine learning model into clinical practice. CONCLUSION Machine learning algorithms rendered acceptable results to predict major adverse cardiovascular events and mortality outcomes in patients with acute coronary syndrome. However, these approaches have never been integrated into clinical practice. Further research is required to develop feasible and effective machine learning prediction models to measure their potentially important implications for optimizing the quality of care in patients with acute coronary syndrome.
Collapse
Affiliation(s)
- Sara Chopannejad
- Student Research Committee, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran (the Islamic Republic of)
| | - Farahnaz Sadoughi
- School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran (the Islamic Republic of)
| | - Rafat Bagherzadeh
- English Language Department, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran (the Islamic Republic of)
| | - Sakineh Shekarchi
- School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran (the Islamic Republic of)
| |
Collapse
|
8
|
Wang X, Zhu T, Xia M, Liu Y, Wang Y, Wang X, Zhuang L, Zhong D, Zhu J, He H, Weng S, Zhu J, Lai D. Predicting the Prognosis of Patients in the Coronary Care Unit: A Novel Multi-Category Machine Learning Model Using XGBoost. Front Cardiovasc Med 2022; 9:764629. [PMID: 35647052 PMCID: PMC9133425 DOI: 10.3389/fcvm.2022.764629] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Accepted: 04/20/2022] [Indexed: 11/13/2022] Open
Abstract
Background Early prediction and classification of prognosis is essential for patients in the coronary care unit (CCU). We applied a machine learning (ML) model using the eXtreme Gradient Boosting (XGBoost) algorithm to prognosticate CCU patients and compared XGBoost with traditional classification models. Methods CCU patients' data were extracted from the MIMIC-III v1.4 clinical database, and divided into four groups based on the time to death: <30 days, 30 days−1 year, 1–5 years, and ≥5 years. Four classification models, including XGBoost, naïve Bayes (NB), logistic regression (LR), and support vector machine (SVM) were constructed using the Python software. These four models were tested and compared for accuracy, F1 score, Matthews correlation coefficient (MCC), and area under the curve (AUC) of the receiver operating characteristic curves. Subsequently, Local Interpretable Model-Agnostic Explanations method was performed to improve XGBoost model interpretability. We also constructed sub-models of each model based on the different categories of death time and compared the differences by decision curve analysis. The optimal model was further analyzed using a clinical impact curve. At last, feature ablation curves of the XGBoost model were conducted to obtain the simplified model. Results Overall, 5360 CCU patients were included. Compared to NB, LR, and SVM, the XGBoost model showed better accuracy (0.663, 0.605, 0.632, and 0.622), micro-AUCs (0.873, 0.811, 0.841, and 0.818), and MCC (0.337, 0.317, 0.250, and 0.182). In subgroup analysis, the XGBoost model had a better predictive performance in acute myocardial infarction subgroup. The decision curve and clinical impact curve analyses verified the clinical utility of the XGBoost model for different categories of patients. Finally, we obtained a simplified model with thirty features. Conclusions For CCU physicians, the ML technique by XGBoost is a potential predictive tool in patients with different conditions, and it may contribute to improvements in prognosis.
Collapse
Affiliation(s)
- Xingchen Wang
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Tianqi Zhu
- College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, China
| | - Minghong Xia
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Yu Liu
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Yao Wang
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Xizhi Wang
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Lenan Zhuang
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Danfeng Zhong
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Jun Zhu
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Hong He
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Shaoxiang Weng
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Junhui Zhu
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
- Junhui Zhu
| | - Dongwu Lai
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
- *Correspondence: Dongwu Lai
| |
Collapse
|
9
|
Machine learning based model for risk prediction after ST-Elevation myocardial infarction: Insights from the North India ST elevation myocardial infarction (NORIN-STEMI) Registry. Int J Cardiol 2022; 362:6-13. [DOI: 10.1016/j.ijcard.2022.05.023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/22/2022] [Revised: 04/20/2022] [Accepted: 05/11/2022] [Indexed: 11/18/2022]
|
10
|
Rousset A, Dellamonica D, Menuet R, Lira Pineda A, Sabatine MS, Giugliano RP, Trichelair P, Zaslavskiy M, Ricci L. Can machine learning bring cardiovascular risk assessment to the next level? A methodological study using FOURIER trial data. EUROPEAN HEART JOURNAL. DIGITAL HEALTH 2022; 3:38-48. [PMID: 36713994 PMCID: PMC9707897 DOI: 10.1093/ehjdh/ztab093] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/12/2021] [Revised: 09/16/2021] [Accepted: 10/26/2021] [Indexed: 02/01/2023]
Abstract
Aims Through this proof of concept, we studied the potential added value of machine learning (ML) methods in building cardiovascular risk scores from structured data and the conditions under which they outperform linear statistical models. Methods and results Relying on extensive cardiovascular clinical data from FOURIER, a randomized clinical trial to test for evolocumab efficacy, we compared linear models, neural networks, random forest, and gradient boosting machines for predicting the risk of major adverse cardiovascular events. To study the relative strengths of each method, we extended the comparison to restricted subsets of the full FOURIER dataset, limiting either the number of available patients or the number of their characteristics. When using all the 428 covariates available in the dataset, ML methods significantly (c-index 0.67, P-value 2e-5) outperformed linear models built from the same variables (c-index 0.62), as well as a reference cardiovascular risk score based on only 10 variables (c-index 0.60). We showed that gradient boosting-the best performing model in our setting-requires fewer patients and significantly outperforms linear models when using large numbers of variables. On the other hand, we illustrate how linear models suffer from being trained on too many variables, thus requiring a more careful prior selection. These ML methods proved to consistently improve risk assessment, to be interpretable despite their complexity and to help identify the minimal set of covariates necessary to achieve top performance. Conclusion In the field of secondary cardiovascular events prevention, given the increased availability of extensive electronic health records, ML methods could open the door to more powerful tools for patient risk stratification and treatment allocation strategies.
Collapse
Affiliation(s)
- Adrien Rousset
- AMGEN Europe GmbH, Suurstoffi 22, 6343 Rotkreuz ZG, Switzerland
| | | | - Romuald Menuet
- OWKIN Inc, 831 Broadway, Unit 3R NY 10003 New York City, USA
| | | | - Marc S Sabatine
- TIMI Study Group, Division of Cardiovascular Medicine, Brigham and Women's Hospital, 350 Longwood Ave, Boston, MA 02115, USA
| | - Robert P Giugliano
- TIMI Study Group, Division of Cardiovascular Medicine, Brigham and Women's Hospital, 350 Longwood Ave, Boston, MA 02115, USA
| | - Paul Trichelair
- OWKIN Inc, 831 Broadway, Unit 3R NY 10003 New York City, USA
| | | | - Lea Ricci
- AMGEN Europe GmbH, Suurstoffi 22, 6343 Rotkreuz ZG, Switzerland
| |
Collapse
|
11
|
Artificial Intelligence: A Shifting Paradigm in Cardio-Cerebrovascular Medicine. J Clin Med 2021; 10:jcm10235710. [PMID: 34884412 PMCID: PMC8658222 DOI: 10.3390/jcm10235710] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2021] [Accepted: 12/02/2021] [Indexed: 12/21/2022] Open
Abstract
The future of healthcare is an organic blend of technology, innovation, and human connection. As artificial intelligence (AI) is gradually becoming a go-to technology in healthcare to improve efficiency and outcomes, we must understand our limitations. We should realize that our goal is not only to provide faster and more efficient care, but also to deliver an integrated solution to ensure that the care is fair and not biased to a group of sub-population. In this context, the field of cardio-cerebrovascular diseases, which encompasses a wide range of conditions-from heart failure to stroke-has made some advances to provide assistive tools to care providers. This article aimed to provide an overall thematic review of recent development focusing on various AI applications in cardio-cerebrovascular diseases to identify gaps and potential areas of improvement. If well designed, technological engines have the potential to improve healthcare access and equitability while reducing overall costs, diagnostic errors, and disparity in a system that affects patients and providers and strives for efficiency.
Collapse
|
12
|
Wang S, Li J, Sun L, Cai J, Wang S, Zeng L, Sun S. Application of machine learning to predict the occurrence of arrhythmia after acute myocardial infarction. BMC Med Inform Decis Mak 2021; 21:301. [PMID: 34724938 PMCID: PMC8560220 DOI: 10.1186/s12911-021-01667-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2021] [Accepted: 10/22/2021] [Indexed: 12/23/2022] Open
Abstract
Background Early identification of the occurrence of arrhythmia in patients with acute myocardial infarction plays an essential role in clinical decision-making. The present study attempted to use machine learning (ML) methods to build predictive models of arrhythmia after acute myocardial infarction (AMI). Methods A total of 2084 patients with acute myocardial infarction were enrolled in this study. (All data is available on Github: https://github.com/wangsuhuai/AMI-database1.git). The primary outcome is whether tachyarrhythmia occurred during admission containing atrial arrhythmia, ventricular arrhythmia, and supraventricular tachycardia. All data is randomly divided into a training set (80%) and an internal testing set (20%). Apply three machine learning algorithms: decision tree, random forest (RF), and artificial neural network (ANN) to learn the training set to build a model, then use the testing set to evaluate the prediction performance, and compare it with the model built by the Global Registry of Acute Coronary Events (GRACE) risk variable set. Results Three ML models predict the occurrence of tachyarrhythmias after AMI. After variable selection, the artificial neural network (ANN) model has reached the highest accuracy rate, which is better than the model constructed using the Grace variable set. After applying SHapley Additive exPlanations (SHAP) to make the model interpretable, the most important features are abnormal wall motion, lesion location, bundle branch block, age, and heart rate. Among them, RBBB (odds ratio [OR]: 4.21; 95% confidence interval [CI]: 2.42–7.02), ≥ 2 ventricular walls motion abnormal (OR: 3.26; 95% CI: 2.01–4.36) and right coronary artery occlusion (OR: 3.00; 95% CI: 1.98–4.56) are significant factors related to arrhythmia after AMI. Conclusions We used advanced machine learning methods to build prediction models for tachyarrhythmia after AMI for the first time (especially the ANN model that has the best performance). The current study can supplement the current AMI risk score, provide a reliable evaluation method for the clinic, and broaden the new horizons of ML and clinical research. Trial registration Clinical Trial Registry No.: ChiCTR2100041960. Supplementary Information The online version contains supplementary material available at 10.1186/s12911-021-01667-8.
Collapse
Affiliation(s)
- Suhuai Wang
- Department of Cardiology, The First Affiliated Hospital of Harbin Medical University, 122 Postal Street, Nangang District, Harbin City, Heilongjiang Province, China
| | - Jingjie Li
- Department of Cardiology, The First Affiliated Hospital of Harbin Medical University, 122 Postal Street, Nangang District, Harbin City, Heilongjiang Province, China.
| | - Lin Sun
- Department of Cardiology, The First Affiliated Hospital of Harbin Medical University, 122 Postal Street, Nangang District, Harbin City, Heilongjiang Province, China.
| | - Jianing Cai
- Department of Cardiology, The First Affiliated Hospital of Harbin Medical University, 122 Postal Street, Nangang District, Harbin City, Heilongjiang Province, China
| | - Shihui Wang
- Department of Cardiology, The First Affiliated Hospital of Harbin Medical University, 122 Postal Street, Nangang District, Harbin City, Heilongjiang Province, China
| | - Linwen Zeng
- Department of Cardiology, The First Affiliated Hospital of Harbin Medical University, 122 Postal Street, Nangang District, Harbin City, Heilongjiang Province, China
| | - Shaoqing Sun
- Department of Cardiology, The First Affiliated Hospital of Harbin Medical University, 122 Postal Street, Nangang District, Harbin City, Heilongjiang Province, China
| |
Collapse
|
13
|
Zhao J, Zhao P, Li C, Hou Y. Optimized Machine Learning Models to Predict In-Hospital Mortality for Patients with ST-Segment Elevation Myocardial Infarction. Ther Clin Risk Manag 2021; 17:951-961. [PMID: 34511920 PMCID: PMC8427294 DOI: 10.2147/tcrm.s321799] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2021] [Accepted: 08/26/2021] [Indexed: 12/22/2022] Open
Abstract
PURPOSE This study aimed to optimize machine learning (ML) models for predicting in-hospital mortality in patients with ST-segment elevation acute myocardial infarction (STEMI). PATIENTS AND METHODS A total of 5708 STEMI patients were enrolled and divided into two groups according to patients' hospital outcomes. Both groups were randomly split into a training set (75%) and a testing set (25%). Four ML models were trained with data, which applied random under-sampling (RUS). The performance of optimized ML models was evaluated with respect to accuracy, sensitivity, specificity, G-mean and AUC. Two sets of features in chronological order were considered: a full set that included all variables during hospitalization and a simplified set that only included variables prior to reperfusion therapy, and the performance of the prediction models trained with these two sets of features was compared. RESULTS For the comprehensive metric - G-mean, the models trained with RUS outperformed those without, 80.54% vs 23.31% on average in the full set and 75.72% vs 35.76% on average in the simplified set. For models trained with the full set, the SVM achieved the best performance with 85.62% accuracy, 84.21% sensitivity, 85.66% specificity, 84.93% G-mean and 0.919 AUC. For models trained with the simplified set, the SVM achieved 83.48% G-mean, which was comparable to the models trained using the full set. For the most critical metric - sensitivity, the SVM trained using the simplified set achieved 89.47%, which even exceed the SVM (84.21%), DT (81.58%) and RF (81.58%) trained using the full set. CONCLUSION Applying RUS can improve the performance of prediction models, and the models trained with simplified set, which only included variables prior to reperfusion therapy can accurately predict high-risk patients.
Collapse
Affiliation(s)
- Jia Zhao
- Graduate School, Tianjin Medical University, Tianjin, 300070, People’s Republic of China
- Department of Cardiology, Tianjin Chest Hospital, Tianjin, 300222, People’s Republic of China
| | - Pengyu Zhao
- School of Electrical and Information Engineering, Tianjin University, Tianjin, 300072, People’s Republic of China
| | - Chunjie Li
- Department of Cardiology, Tianjin Chest Hospital, Tianjin, 300222, People’s Republic of China
| | - Yonghong Hou
- School of Electrical and Information Engineering, Tianjin University, Tianjin, 300072, People’s Republic of China
| |
Collapse
|
14
|
Exploring and predicting mortality among patients with end-stage liver disease without cancer: a machine learning approach. Eur J Gastroenterol Hepatol 2021; 33:1117-1123. [PMID: 33905216 DOI: 10.1097/meg.0000000000002169] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
OBJECTIVE End-stage liver disease is a global public health problem with a high mortality rate. Early identification of people at risk of poor prognosis is fundamental for decision-making in clinical settings. This study created a machine learning prediction system that provides several related models with visualized graphs, including decision trees, ensemble learning and clustering, to predict mortality in patients with end-stage liver disease. METHODS A retrospective cohort study was conducted: the training data were from patients enrolled from January 2009 to December 2010 and followed up to December 2014; validation data were from patients enrolled from January 2015 to December 2016 and followed up to January 2019. Hospitalized patients with noncancer-related chronic liver disease were identified from the hospital's electrical medical records. RESULTS In traditional multivariable logistic regression and Cox proportional hazard model, prothrombin time of international normalized ratio, which was significant with P value = 0.002, odds ratio = 2.790 and hazard ratio 1.363. Besides, blood urea nitrogen and C-reactive protein were also significant, with P value <0.001 and 0.026. The area under the curve was 0.771 in the receiver operating characteristic curve. In machine learning, blood urea nitrogen and age were regarded as the primary factors for predicting mortality. Creatinine, prothrombin time of international normalized ratio and bilirubin were also significant mortality predictors. The area under the curve of the random forest and AdaBoost was 0.838 and 0.792. CONCLUSION The machine learning techniques provided a comprehensive assessment of patient conditions; it could help physicians make an accurate diagnosis of chronic liver disease and improve healthcare management.
Collapse
|
15
|
Clinical Feature-Based Machine Learning Model for 1-Year Mortality Risk Prediction of ST-Segment Elevation Myocardial Infarction in Patients with Hyperuricemia: A Retrospective Study. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2021; 2021:7252280. [PMID: 34285708 PMCID: PMC8275420 DOI: 10.1155/2021/7252280] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/14/2021] [Accepted: 06/16/2021] [Indexed: 12/22/2022]
Abstract
Accurate risk assessment of high-risk patients is essential in clinical practice. However, there is no practical method to predict or monitor the prognosis of patients with ST-segment elevation myocardial infarction (STEMI) complicated by hyperuricemia. We aimed to evaluate the performance of different machine learning models for the prediction of 1-year mortality in STEMI patients with hyperuricemia. We compared five machine learning models (logistic regression, k-nearest neighbor, CatBoost, random forest, and XGBoost) with the traditional global (GRACE) risk score for acute coronary event registrations. We registered patients aged >18 years diagnosed with STEMI and hyperuricemia at the Affiliated Hospital of Zunyi Medical University between January 2016 and January 2020. Overall, 656 patients were enrolled (average age, 62.5 ± 13.6 years; 83.6%, male). All patients underwent emergency percutaneous coronary intervention. We evaluated the performance of five machine learning classifiers and the GRACE risk model in predicting 1-year mortality. The area under the curve (AUC) of the six models, including the GRACE risk model, ranged from 0.75 to 0.88. Among all the models, CatBoost had the highest predictive accuracy (0.89), AUC (0.87), precision (0.84), and F1 value (0.44). After hybrid sampling technique optimization, CatBoost had the highest accuracy (0.96), AUC (0.99), precision (0.95), and F1 value (0.97). Machine learning algorithms, especially the CatBoost model, can accurately predict the mortality associated with STEMI complicated by hyperuricemia after a 1-year follow-up.
Collapse
|
16
|
Lee W, Lee J, Woo SI, Choi SH, Bae JW, Jung S, Jeong MH, Lee WK. Machine learning enhances the performance of short and long-term mortality prediction model in non-ST-segment elevation myocardial infarction. Sci Rep 2021; 11:12886. [PMID: 34145358 PMCID: PMC8213755 DOI: 10.1038/s41598-021-92362-1] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2021] [Accepted: 06/07/2021] [Indexed: 11/09/2022] Open
Abstract
Machine learning (ML) has been suggested to improve the performance of prediction models. Nevertheless, research on predicting the risk in patients with acute myocardial infarction (AMI) has been limited and showed inconsistency in the performance of ML models versus traditional models (TMs). This study developed ML-based models (logistic regression with regularization, random forest, support vector machine, and extreme gradient boosting) and compared their performance in predicting the short- and long-term mortality of patients with AMI with those of TMs with comparable predictors. The endpoints were the in-hospital mortality of 14,183 participants and the three- and 12-month mortality in patients who survived at discharge. The performance of the ML models in predicting the mortality of patients with an ST-segment elevation myocardial infarction (STEMI) was comparable to the TMs. In contrast, the areas under the curves (AUC) of the ML models for non-STEMI (NSTEMI) in predicting the in-hospital, 3-month, and 12-month mortality were 0.889, 0.849, and 0.860, respectively, which were superior to the TMs, which had corresponding AUCs of 0.873, 0.795, and 0.808. Overall, the performance of the predictive model could be improved, particularly for long-term mortality in NSTEMI, from the ML algorithm rather than using more clinical predictors.
Collapse
Affiliation(s)
- Woojoo Lee
- Department of Public Health Sciences, Graduate School of Public Health, Seoul National University, Seoul, Republic of Korea
| | - Joongyub Lee
- Department of Preventive Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Seoung-Il Woo
- Department of Cardiology, School of Medicine, Inha University Hospital, Inha University, Incheon, Republic of Korea
| | - Seong Huan Choi
- Department of Cardiology, School of Medicine, Inha University Hospital, Inha University, Incheon, Republic of Korea
| | - Jang-Whan Bae
- Division of Cardiology, Department of Internal Medicine, Chungbuk National University College of Medicine, Cheongju, Republic of Korea
| | - Seungpil Jung
- Department of Public Health Sciences, Graduate School of Public Health, Seoul National University, Seoul, Republic of Korea
| | - Myung Ho Jeong
- Chonnam National University Hospital, Gwangju, Republic of Korea
| | - Won Kyung Lee
- Department of Prevention and Management, School of Medicine, Inha University Hospital, Inha University, 27 Inhang-Ro, Jung-Gu, Incheon, Republic of Korea.
| |
Collapse
|