1
|
Nong X, Lai C, Chen L, Wei J. A novel coupling interpretable machine learning framework for water quality prediction and environmental effect understanding in different flow discharge regulations of hydro-projects. THE SCIENCE OF THE TOTAL ENVIRONMENT 2024; 950:175281. [PMID: 39117235 DOI: 10.1016/j.scitotenv.2024.175281] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/21/2024] [Revised: 08/01/2024] [Accepted: 08/02/2024] [Indexed: 08/10/2024]
Abstract
Machine learning models (MLMs) have been increasingly used to forecast water pollution. However, the "black box" characteristic for understanding mechanism processes still limits the applicability of MLMs for water quality management in hydro-projects under complex and frequently artificial regulation. This study proposes an interpretable machine learning framework for water quality prediction coupled with a hydrodynamic (flow discharge) scenario-based Random Forest (RF) model with multiple model-agnostic techniques and quantifies global, local, and joint interpretations (i.e., partial dependence, individual conditional expectation, and accumulated local effects) of environmental factor implications. The framework was applied and verified to predict the permanganate index (CODMn) under different flow discharge regulation scenarios in the Middle Route of the South-to-North Water Diversion Project of China (MRSNWDPC). A total of 4664 sampling cases data matrices, including water quality, meteorological, and hydrological indicators from eight national stations along the main canal of the MRSNWDPC, were collected from May 2019 to December 2020. The results showed that the RF models were effective in forecasting CODMn in all flow discharge scenarios, with a mean square error, coefficient of determination, and mean absolute error of 0.006-0.026, 0.481-0.792, and 0.069-0.104, respectively, in the testing dataset. A global interpretation indicated that dissolved oxygen, flow discharge, and surface pressure are the three most important variables of CODMn. Local and joint interpretations indicated that the RF-based prediction model provides a basic understanding of the physical mechanisms of environmental systems. The proposed framework can effectively learn the fundamental environmental implications of water quality variations and provide reliable prediction performance, highlighting the importance of model interpretability for trustworthy machine learning applications in water management projects. This study provides scientific references for applying advanced data-driven MLMs to water quality forecasting and a reliable methodological framework for water quality management and similar hydro-projects.
Collapse
Affiliation(s)
- Xizhi Nong
- College of Civil Engineering and Architecture, Guangxi University, Nanning 530004, China; State Key Laboratory of Hydroscience and Engineering, Tsinghua University, Beijing 100084, China; Centre for Urban Sustainability and Resilience, Department of Civil, Environmental and Geomatic Engineering, University College London, London WC1E 6BT, UK; School of Computing and Engineering, University of West London, London W5 5RF, UK
| | - Cheng Lai
- College of Civil Engineering and Architecture, Guangxi University, Nanning 530004, China
| | - Lihua Chen
- College of Civil Engineering and Architecture, Guangxi University, Nanning 530004, China.
| | - Jiahua Wei
- State Key Laboratory of Hydroscience and Engineering, Tsinghua University, Beijing 100084, China
| |
Collapse
|
2
|
Feng Z, Chen Y, Guo Y, Lyu J. Deciphering the environmental chemical basis of muscle quality decline by interpretable machine learning models. Am J Clin Nutr 2024; 120:407-418. [PMID: 38825185 DOI: 10.1016/j.ajcnut.2024.05.022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Revised: 05/07/2024] [Accepted: 05/28/2024] [Indexed: 06/04/2024] Open
Abstract
BACKGROUND Sarcopenia is known as a decline in skeletal muscle quality and function that is associated with age. Sarcopenia is linked to diverse health problems, including endocrine-related diseases. Environmental chemicals (ECs), a broad class of chemicals released from industry, may influence muscle quality decline. OBJECTIVES In this work, we aimed to simultaneously elucidate the associations between muscle quality decline and diverse EC exposures based on the data from the 2011-2012 and 2013-2014 survey cycles in the National Health and Nutrition Examination Survey (NHANES) project using machine learning models. METHODS Six machine learning models were trained based on the EC and non-EC exposures from NHANES to distinguish low from normal muscle quality index status. Different machine learning metrics were evaluated for these models. The Shapley additive explanations (SHAP) approach was used to provide explainability for machine learning models. RESULTS Random forest (RF) performed best on the independent testing data set. Based on the testing data set, ECs can independently predict the binary muscle quality status with good performance by RF (area under the receiver operating characteristic curve = 0.793; area under the precision-recall curve = 0.808). The SHAP ranked the importance of ECs for the RF model. As a result, several metals and chemicals in urine, including 3-phenoxybenzoic acid and cobalt, were more associated with the muscle quality decline. CONCLUSIONS Altogether, our analyses suggest that ECs can independently predict muscle quality decline with a good performance by RF, and the SHAP-identified ECs can be closely related to muscle quality decline and sarcopenia. Our analyses may provide valuable insights into ECs that may be the important basis of sarcopenia and endocrine-related diseases in United States populations.
Collapse
Affiliation(s)
- Zhen Feng
- Joint Centre of Translational Medicine, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, People's Republic of China; Joint Centre of Translational Medicine, Wenzhou Institute, University of Chinese Academy of Sciences, Wenzhou, Zhejiang, People's Republic of China; College of Information and Engineering, Wenzhou Medical University, Wenzhou, Zhejiang, People's Republic of China
| | - Ying'ao Chen
- Postgraduate Training Base Alliance of Wenzhou Medical University, Wenzhou, Zhejiang, People's Republic of China; Wenzhou Institute, University of Chinese Academy of Sciences, Wenzhou, Zhejiang, People's Republic of China
| | - Yuxin Guo
- College of Information and Engineering, Wenzhou Medical University, Wenzhou, Zhejiang, People's Republic of China
| | - Jie Lyu
- Joint Centre of Translational Medicine, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, People's Republic of China; Joint Centre of Translational Medicine, Wenzhou Institute, University of Chinese Academy of Sciences, Wenzhou, Zhejiang, People's Republic of China; Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Wenzhou, Zhejiang, People's Republic of China.
| |
Collapse
|
3
|
Bu ZJ, Jiang N, Li KC, Lu ZL, Zhang N, Yan SS, Chen ZL, Hao YH, Zhang YH, Xu RB, Chi HW, Chen ZY, Liu JP, Wang D, Xu F, Liu ZL. Development and Validation of an Interpretable Machine Learning Model for Early Prognosis Prediction in ICU Patients with Malignant Tumors and Hyperkalemia. Medicine (Baltimore) 2024; 103:e38747. [PMID: 39058887 PMCID: PMC11272258 DOI: 10.1097/md.0000000000038747] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/18/2024] [Accepted: 06/07/2024] [Indexed: 07/28/2024] Open
Abstract
This study aims to develop and validate a machine learning (ML) predictive model for assessing mortality in patients with malignant tumors and hyperkalemia (MTH). We extracted data on patients with MTH from the Medical Information Mart for Intensive Care-IV, version 2.2 (MIMIC-IV v2.2) database. The dataset was split into a training set (75%) and a validation set (25%). We used the Least Absolute Shrinkage and Selection Operator (LASSO) regression to identify potential predictors, which included clinical laboratory indicators and vital signs. Pearson correlation analysis tested the correlation between predictors. In-hospital death was the prediction target. The Area Under the Curve (AUC) and accuracy of the training and validation sets of 7 ML algorithms were compared, and the optimal 1 was selected to develop the model. The calibration curve was used to evaluate the prediction accuracy of the model further. SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME) enhanced model interpretability. 496 patients with MTH in the Intensive Care Unit (ICU) were included. After screening, 17 clinical features were included in the construction of the ML model, and the Pearson correlation coefficient was <0.8, indicating that the correlation between the clinical features was small. eXtreme Gradient Boosting (XGBoost) outperformed other algorithms, achieving perfect scores in the training set (accuracy: 1.000, AUC: 1.000) and high scores in the validation set (accuracy: 0.734, AUC: 0.733). The calibration curves indicated good predictive calibration of the model. SHAP analysis identified the top 8 predictive factors: urine output, mean heart rate, maximum urea nitrogen, minimum oxygen saturation, minimum mean blood pressure, maximum total bilirubin, mean respiratory rate, and minimum pH. In addition, SHAP and LIME performed in-depth individual case analyses. This study demonstrates the effectiveness of ML methods in predicting mortality risk in ICU patients with MTH. It highlights the importance of predictors like urine output and mean heart rate. SHAP and LIME significantly enhanced the model's interpretability.
Collapse
Affiliation(s)
- Zhi-Jun Bu
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- School of Traditional Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
| | - Nan Jiang
- School of Traditional Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- The Third Affiliated Hospital, Beijing University of Chinese Medicine, Beijing, China
| | - Ke-Cheng Li
- School of Traditional Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Department of Andrology, Dongzhimen Hospital, Beijing University of Chinese Medicine, Beijing, China
| | - Zhi-Lin Lu
- First Clinical College, Hubei University of Chinese Medicine, Wuhan, China
| | - Nan Zhang
- School of International Studies, University of International Business and Economics, Beijing, China
| | - Shao-Shuai Yan
- Department of Thyropathy, Dongzhimen Hospital, Beijing University of Chinese Medicine, Beijing, China
| | - Zhi-Lin Chen
- School of Traditional Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
| | - Yu-Han Hao
- School of Traditional Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
| | - Yu-Huan Zhang
- School of Acupuncture and Orthopedics, Hubei University of Chinese Medicine, Wuhan, China
| | - Run-Bing Xu
- School of Traditional Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Department of Hematology and Oncology, Dongzhimen Hospital, Beijing University of Chinese Medicine, Beijing, China
| | - Han-Wei Chi
- School of Traditional Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
| | - Zu-Yi Chen
- School of Traditional Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
| | - Jian-Ping Liu
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- School of Traditional Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
| | - Dan Wang
- Surgery of Thyroid Gland and Breast, Hubei Provincial Hospital of Traditional Chinese Medicine, Wuhan, China
- Hubei Shizhen Laboratory, Wuhan, China
| | - Feng Xu
- The Third Affiliated Hospital, Beijing University of Chinese Medicine, Beijing, China
| | - Zhao-Lan Liu
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- School of Traditional Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
| |
Collapse
|
4
|
Zuo W, Yang X. A machine learning model predicts stroke associated with blood cadmium level. Sci Rep 2024; 14:14739. [PMID: 38926494 PMCID: PMC11208606 DOI: 10.1038/s41598-024-65633-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2024] [Accepted: 06/21/2024] [Indexed: 06/28/2024] Open
Abstract
Stroke is the leading cause of death and disability worldwide. Cadmium is a prevalent environmental toxicant that may contribute to cardiovascular disease, including stroke. We aimed to build an effective and interpretable machine learning (ML) model that links blood cadmium to the identification of stroke. Our data exploring the association between blood cadmium and stroke came from the National Health and Nutrition Examination Survey (NHANES, 2013-2014). In total, 2664 participants were eligible for this study. We divided these data into a training set (80%) and a test set (20%). To analyze the relationship between blood cadmium and stroke, a multivariate logistic regression analysis was performed. We constructed and tested five ML algorithms including K-nearest neighbor (KNN), decision tree (DT), logistic regression (LR), multilayer perceptron (MLP), and random forest (RF). The best-performing model was selected to identify stroke in US adults. Finally, the features were interpreted using the Shapley Additive exPlanations (SHAP) tool. In the total population, participants in the second, third, and fourth quartiles had an odds ratio of 1.32 (95% CI 0.55, 3.14), 1.65 (95% CI 0.71, 3.83), and 2.67 (95% CI 1.10, 6.49) for stroke compared with the lowest reference group for blood cadmium, respectively. This blood cadmium-based LR approach demonstrated the greatest performance in identifying stroke (area under the operator curve: 0.800, accuracy: 0.966). Employing interpretable methods, we found blood cadmium to be a notable contributor to the predictive model. We found that blood cadmium was positively correlated with stroke risk and that stroke risk from cadmium exposure could be effectively predicted by using ML modeling.
Collapse
Affiliation(s)
- Wenwei Zuo
- School of Gongli Hospital Medical Technology, University of Shanghai for Science and Technology, No. 516, Jungong Road, Yangpu Area, Shanghai, 200093, China
| | - Xuelian Yang
- Department of Neurology, Shanghai Pudong New Area Gongli Hospital, No. 219 Miaopu Road, Pudong New Area, Shanghai, 200135, China.
| |
Collapse
|
5
|
Jiang X, Zhou R, Jiang F, Yan Y, Zhang Z, Wang J. Construction of diagnostic models for the progression of hepatocellular carcinoma using machine learning. Front Oncol 2024; 14:1401496. [PMID: 38812780 PMCID: PMC11133637 DOI: 10.3389/fonc.2024.1401496] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2024] [Accepted: 04/29/2024] [Indexed: 05/31/2024] Open
Abstract
Liver cancer is one of the most prevalent forms of cancer worldwide. A significant proportion of patients with hepatocellular carcinoma (HCC) are diagnosed at advanced stages, leading to unfavorable treatment outcomes. Generally, the development of HCC occurs in distinct stages. However, the diagnostic and intervention markers for each stage remain unclear. Therefore, there is an urgent need to explore precise grading methods for HCC. Machine learning has emerged as an effective technique for studying precise tumor diagnosis. In this research, we employed random forest and LightGBM machine learning algorithms for the first time to construct diagnostic models for HCC at various stages of progression. We categorized 118 samples from GSE114564 into three groups: normal liver, precancerous lesion (including chronic hepatitis, liver cirrhosis, dysplastic nodule), and HCC (including early stage HCC and advanced HCC). The LightGBM model exhibited outstanding performance (accuracy = 0.96, precision = 0.96, recall = 0.96, F1-score = 0.95). Similarly, the random forest model also demonstrated good performance (accuracy = 0.83, precision = 0.83, recall = 0.83, F1-score = 0.83). When the progression of HCC was categorized into the most refined six stages: normal liver, chronic hepatitis, liver cirrhosis, dysplastic nodule, early stage HCC, and advanced HCC, the diagnostic model still exhibited high efficacy. Among them, the LightGBM model exhibited good performance (accuracy = 0.71, precision = 0.71, recall = 0.71, F1-score = 0.72). Also, performance of the LightGBM model was superior to that of the random forest model. Overall, we have constructed a diagnostic model for the progression of HCC and identified potential diagnostic characteristic gene for the progression of HCC.
Collapse
Affiliation(s)
- Xin Jiang
- Innovation Center for Cancer Research, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, China
- Fujian Key Laboratory of Advanced Technology for Cancer Screening and Early Diagnosis, Fuzhou, China
| | - Ruilong Zhou
- Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong, Hong Kong SAR, China
| | - Fengle Jiang
- Innovation Center for Cancer Research, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, China
- Fujian Key Laboratory of Advanced Technology for Cancer Screening and Early Diagnosis, Fuzhou, China
| | - Yanan Yan
- Innovation Center for Cancer Research, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, China
- Fujian Key Laboratory of Advanced Technology for Cancer Screening and Early Diagnosis, Fuzhou, China
| | - Zheting Zhang
- Innovation Center for Cancer Research, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, China
- Fujian Key Laboratory of Advanced Technology for Cancer Screening and Early Diagnosis, Fuzhou, China
| | - Jianmin Wang
- Innovation Center for Cancer Research, Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fuzhou, China
- Fujian Key Laboratory of Advanced Technology for Cancer Screening and Early Diagnosis, Fuzhou, China
| |
Collapse
|
6
|
Zhu G, Wen Y, Cao K, He S, Wang T. A review of common statistical methods for dealing with multiple pollutant mixtures and multiple exposures. Front Public Health 2024; 12:1377685. [PMID: 38784575 PMCID: PMC11113012 DOI: 10.3389/fpubh.2024.1377685] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2024] [Accepted: 04/15/2024] [Indexed: 05/25/2024] Open
Abstract
Traditional environmental epidemiology has consistently focused on studying the impact of single exposures on specific health outcomes, considering concurrent exposures as variables to be controlled. However, with the continuous changes in environment, humans are increasingly facing more complex exposures to multi-pollutant mixtures. In this context, accurately assessing the impact of multi-pollutant mixtures on health has become a central concern in current environmental research. Simultaneously, the continuous development and optimization of statistical methods offer robust support for handling large datasets, strengthening the capability to conduct in-depth research on the effects of multiple exposures on health. In order to examine complicated exposure mixtures, we introduce commonly used statistical methods and their developments, such as weighted quantile sum, bayesian kernel machine regression, toxic equivalency analysis, and others. Delineating their applications, advantages, weaknesses, and interpretability of results. It also provides guidance for researchers involved in studying multi-pollutant mixtures, aiding them in selecting appropriate statistical methods and utilizing R software for more accurate and comprehensive assessments of the impact of multi-pollutant mixtures on human health.
Collapse
Affiliation(s)
- Guiming Zhu
- Department of Health Statistics, School of Public Health, Shanxi Medical University, Taiyuan, China
- Key Laboratory of Coal Environmental Pathogenicity and Prevention (Shanxi Medical University), Ministry of Education, Taiyuan, China
| | - Yanchao Wen
- Department of Health Statistics, School of Public Health, Shanxi Medical University, Taiyuan, China
- Key Laboratory of Coal Environmental Pathogenicity and Prevention (Shanxi Medical University), Ministry of Education, Taiyuan, China
| | - Kexin Cao
- Department of Health Statistics, School of Public Health, Shanxi Medical University, Taiyuan, China
- Key Laboratory of Coal Environmental Pathogenicity and Prevention (Shanxi Medical University), Ministry of Education, Taiyuan, China
| | - Simin He
- Department of Health Statistics, School of Public Health, Shanxi Medical University, Taiyuan, China
- Key Laboratory of Coal Environmental Pathogenicity and Prevention (Shanxi Medical University), Ministry of Education, Taiyuan, China
| | - Tong Wang
- Department of Health Statistics, School of Public Health, Shanxi Medical University, Taiyuan, China
- Key Laboratory of Coal Environmental Pathogenicity and Prevention (Shanxi Medical University), Ministry of Education, Taiyuan, China
| |
Collapse
|
7
|
Li J, Zou L, Ma H, Zhao J, Wang C, Li J, Hu G, Yang H, Wang B, Xu D, Xia Y, Jiang Y, Jiang X, Li N. Interpretable machine learning based on CT-derived extracellular volume fraction to predict pathological grading of hepatocellular carcinoma. Abdom Radiol (NY) 2024:10.1007/s00261-024-04313-9. [PMID: 38703190 DOI: 10.1007/s00261-024-04313-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2024] [Revised: 03/23/2024] [Accepted: 03/25/2024] [Indexed: 05/06/2024]
Abstract
PURPOSE To develop a non-invasive auxiliary assessment method based on CT-derived extracellular volume (ECV) to predict the pathological grading (PG) of hepatocellular carcinoma (HCC). METHODS The study retrospectively analyzed 238 patients who underwent HCC resection surgery between January 2013 and April 2023. Six machine learning algorithms were employed to construct predictive models for HCC PG: logistic regression, extreme gradient boosting, Light Gradient Boosting Machine (LightGBM), random forest, adaptive boosting, and Gaussian naive Bayes. Model performance was evaluated using receiver operating characteristic curve analysis, including area under the curve (AUC), sensitivity, specificity, accuracy, positive predictive value, negative predictive value, and F1 score. Calibration plots were used for visual evaluation of model calibration. Clinical decision curve analysis was performed to assess potential clinical utility by calculating net benefit. RESULTS 166 patients from Hospital A were allocated to the training set, while 72 patients from Hospital B (constituting 30.25% of the total sample) were assigned to the test set. The model achieved an AUC of 1.000 (95%CI: 1.000-1.000) in the training set and 0.927 (95%CI: 0.837-0.999) in the validation set, respectively. Ultimately, the model achieved an AUC of 0.909 (95%CI: 0.837-0.980) in the test set, with an accuracy of 0.778, sensitivity of 0.906, specificity of 0.789, negative predictive value of 0.556, and F1 score of 0.908. CONCLUSION This study successfully developed and validated a non-invasive auxiliary assessment method based on CT-derived ECV to predict the HCC PG, providing important supplementary information for clinical decision-making.
Collapse
Affiliation(s)
- Jie Li
- Department of Radiology, Binzhou Medical University Hospital, No. 661 Huanghe 2nd Road, Bincheng District, Binzhou, 256600, China
| | - Linxuan Zou
- Department of Radiology, Binzhou Medical University Hospital, No. 661 Huanghe 2nd Road, Bincheng District, Binzhou, 256600, China
| | - Heng Ma
- Department of Radiology, Yantai Yuhuangding Hospital, Qingdao University, Yantai, 264000, China
| | - Jifu Zhao
- Department of Radiology, Binzhou Medical University Hospital, No. 661 Huanghe 2nd Road, Bincheng District, Binzhou, 256600, China
| | - Chengyan Wang
- Department of Radiology, Binzhou Medical University Hospital, No. 661 Huanghe 2nd Road, Bincheng District, Binzhou, 256600, China
| | - Jun Li
- Department of Radiology, Yantai Affiliated Hospital of Binzhou Medical University, Yantai, 264000, China
| | - Guangchao Hu
- School of Medical Imaging, Binzhou Medical University, No. 346 Guanhai Road, Laishan District, Yantai, 264003, China
| | - Haoran Yang
- School of Medical Imaging, Binzhou Medical University, No. 346 Guanhai Road, Laishan District, Yantai, 264003, China
| | - Beizhong Wang
- Department of Radiology, Binzhou Medical University Hospital, No. 661 Huanghe 2nd Road, Bincheng District, Binzhou, 256600, China
| | - Donghao Xu
- School of Medical Imaging, Binzhou Medical University, No. 346 Guanhai Road, Laishan District, Yantai, 264003, China
| | - Yuanhao Xia
- Department of Radiology, Yantai Yuhuangding Hospital, Qingdao University, Yantai, 264000, China
- School of Medical Imaging, Binzhou Medical University, No. 346 Guanhai Road, Laishan District, Yantai, 264003, China
| | - Yi Jiang
- Department of Vascular Interventional Surgery, Yantai Affiliated Hospital of Binzhou Medical University, Yantai, 264000, China
| | - Xingyue Jiang
- Department of Radiology, Binzhou Medical University Hospital, No. 661 Huanghe 2nd Road, Bincheng District, Binzhou, 256600, China.
| | - Naixuan Li
- Department of Vascular Interventional Surgery, Yantai Affiliated Hospital of Binzhou Medical University, Yantai, 264000, China.
| |
Collapse
|
8
|
Abstract
Heavy metals are harmful environmental pollutants that have attracted widespread attention due to their health hazards to human cardiovascular disease. Heavy metals, including lead, cadmium, mercury, arsenic, and chromium, are found in various sources such as air, water, soil, food, and industrial products. Recent research strongly suggests a connection between cardiovascular disease and exposure to toxic heavy metals. Epidemiological, basic, and clinical studies have revealed that heavy metals can promote the production of reactive oxygen species, which can then exacerbate reactive oxygen species generation and induce inflammation, resulting in endothelial dysfunction, lipid metabolism distribution, disruption of ion homeostasis, and epigenetic changes. Over time, heavy metal exposure eventually results in an increased risk of hypertension, arrhythmia, and atherosclerosis. Strengthening public health prevention and the application of chelation or antioxidants, such as vitamins and beta-carotene, along with minerals, such as selenium and zinc, can diminish the burden of cardiovascular disease attributable to metal exposure.
Collapse
Affiliation(s)
- Ziwei Pan
- Key Laboratory of Combined Multi Organ Transplantation, Ministry of Public Health, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China (Z.P., P.L.)
- Institute of Translational Medicine, Zhejiang University, Hangzhou, China (Z.P., P.L.)
| | - Tingyu Gong
- Shulan International Medical College, Zhejiang Shuren University, Hangzhou, China (T.G.)
| | - Ping Liang
- Key Laboratory of Combined Multi Organ Transplantation, Ministry of Public Health, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China (Z.P., P.L.)
- Institute of Translational Medicine, Zhejiang University, Hangzhou, China (Z.P., P.L.)
| |
Collapse
|
9
|
Liu J, Li X, Zhu P. Effects of Various Heavy Metal Exposures on Insulin Resistance in Non-diabetic Populations: Interpretability Analysis from Machine Learning Modeling Perspective. Biol Trace Elem Res 2024:10.1007/s12011-024-04126-3. [PMID: 38409445 DOI: 10.1007/s12011-024-04126-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/27/2023] [Accepted: 02/22/2024] [Indexed: 02/28/2024]
Abstract
Increasing and compelling evidence has been proved that heavy metal exposure is involved in the development of insulin resistance (IR). We trained an interpretable predictive machine learning (ML) model for IR in the non-diabetic populations based on levels of heavy metal exposure. A total of 4354 participants from the NHANES (2003-2020) with complete information were randomly divided into a training set and a test set. Twelve ML algorithms, including random forest (RF), XGBoost (XGB), logistic regression (LR), GaussianNB (GNB), ridge regression (RR), support vector machine (SVM), multilayer perceptron (MLP), decision tree (DT), AdaBoost (AB), Gradient Boosting Decision Tree (GBDT), Voting Classifier (VC), and K-Nearest Neighbour (KNN), were constructed for IR prediction using the training set. Among these models, the RF algorithm had the best predictive performance, showing an accuracy of 80.14%, an AUC of 0.856, and an F1 score of 0.74 in the test set. We embedded three interpretable methods, the permutation feature importance analysis, partial dependence plot (PDP), and Shapley additive explanations (SHAP) in RF model for model interpretation. Urinary Ba, urinary Mo, blood Pb, and blood Cd levels were identified as the main influencers of IR. Within a specific range, urinary Ba (0.56-3.56 µg/L) and urinary Mo (1.06-20.25 µg/L) levels exhibited the most pronounced upwards trend with the risk of IR, while blood Pb (0.05-2.81 µg/dL) and blood Cd (0.24-0.65 µg/L) levels showed a declining trend with IR. The findings on the synergistic effects demonstrated that controlling urinary Ba levels might be more crucial for the management of IR. The SHAP decision plot offered personalized care for IR based on heavy metal control. In conclusion, by utilizing interpretable ML approaches, we emphasize the predictive value of heavy metals for IR, especially Ba, Mo, Pb, and Cd.
Collapse
Affiliation(s)
- Jun Liu
- Department of Gastrointestinal Surgery, The Second Affiliated Hospital of Chongqing Medical University, 74 Linjiang Road, Yuzhong District, Chongqing, 400010, China
| | - Xingyu Li
- Cardiovascular Medicine, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, China
| | - Peng Zhu
- Department of Gastrointestinal Surgery, The Second Affiliated Hospital of Chongqing Medical University, 74 Linjiang Road, Yuzhong District, Chongqing, 400010, China.
| |
Collapse
|
10
|
Song W, Wu F, Yan Y, Li Y, Wang Q, Hu X, Li Y. Gut microbiota landscape and potential biomarker identification in female patients with systemic lupus erythematosus using machine learning. Front Cell Infect Microbiol 2023; 13:1289124. [PMID: 38169617 PMCID: PMC10758415 DOI: 10.3389/fcimb.2023.1289124] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Accepted: 11/28/2023] [Indexed: 01/05/2024] Open
Abstract
Objectives Systemic Lupus Erythematosus (SLE) is a complex autoimmune disease that disproportionately affects women. Early diagnosis and prevention are crucial for women's health, and the gut microbiota has been found to be strongly associated with SLE. This study aimed to identify potential biomarkers for SLE by characterizing the gut microbiota landscape using feature selection and exploring the use of machine learning (ML) algorithms with significantly dysregulated microbiotas (SDMs) for early identification of SLE patients. Additionally, we used the SHapley Additive exPlanations (SHAP) interpretability framework to visualize the impact of SDMs on the risk of developing SLE in females. Methods Stool samples were collected from 54 SLE patients and 55 Negative Controls (NC) for microbiota analysis using 16S rRNA sequencing. Feature selection was performed using Elastic Net and Boruta on species-level taxonomy. Subsequently, four ML algorithms, namely logistic regression (LR), Adaptive Boosting (AdaBoost), Random Forest (RF), and eXtreme gradient boosting (XGBoost), were used to achieve early identification of SLE with SDMs. Finally, the best-performing algorithm was combined with SHAP to explore how SDMs affect the risk of developing SLE in females. Results Both alpha and beta diversity were found to be different in SLE group. Following feature selection, 68 and 21 microbiota were retained in Elastic Net and Boruta, respectively, with 16 microbiota overlapping between the two, i.e., SDMs for SLE. The four ML algorithms with SDMs could effectively identify SLE patients, with XGBoost performing the best, achieving Accuracy, Sensitivity, Specificity, Positive Predictive Value, Negative Predictive Value, and AUC values of 0.844, 0.750, 0.938, 0.923, 0.790, and 0.930, respectively. The SHAP interpretability framework showed a complex non-linear relationship between the relative abundance of SDMs and the risk of SLE, with Escherichia_fergusonii having the largest SHAP value. Conclusions This study revealed dysbiosis in the gut microbiota of female SLE patients. ML classifiers combined with SDMs can facilitate early identification of female patients with SLE, particularly XGBoost. The SHAP interpretability framework provides insight into the impact of SDMs on the risk of SLE and may inform future scientific treatment for SLE.
Collapse
Affiliation(s)
- Wenzhu Song
- School of Public Health, Shanxi Medical University, Taiyuan, Shanxi, China
| | - Feng Wu
- Department of Nephrology, Shanxi Provincial People's Hospital (Fifth Hospital) of Shanxi Medical University, Taiyuan, China
| | - Yan Yan
- Department of Nephrology, Shanxi Provincial People's Hospital (Fifth Hospital) of Shanxi Medical University, Taiyuan, China
| | - Yaheng Li
- Shanxi Provincial Key Laboratory of Kidney Disease, Taiyuan, Shanxiuan, China
| | - Qian Wang
- Shanxi Provincial Key Laboratory of Kidney Disease, Taiyuan, Shanxiuan, China
| | - Xueli Hu
- Department of Nephrology, Hejin People’s Hospital, Yuncheng, Shanxi, China
| | - Yafeng Li
- Department of Nephrology, Shanxi Provincial People's Hospital (Fifth Hospital) of Shanxi Medical University, Taiyuan, China
- Shanxi Provincial Key Laboratory of Kidney Disease, Taiyuan, Shanxiuan, China
- Core Laboratory, Shanxi Provincial People's Hospital (Fifth Hospital) of Shanxi Medical University, Taiyuan, China
- Academy of Microbial Ecology, Shanxi Medical University, Taiyuan, China
| |
Collapse
|
11
|
Li Q, Zheng JX, Jia TW, Feng XY, Lv C, Zhang LJ, Yang GJ, Xu J, Zhou XN. Optimized strategy for schistosomiasis elimination: results from marginal benefit modeling. Parasit Vectors 2023; 16:419. [PMID: 37968661 PMCID: PMC10652544 DOI: 10.1186/s13071-023-06001-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Accepted: 10/06/2023] [Indexed: 11/17/2023] Open
Abstract
BACKGROUND Poverty contributes to the transmission of schistosomiasis via multiple pathways, with the insufficiency of appropriate interventions being a crucial factor. The aim of this article is to provide more economical and feasible intervention measures for endemic areas with varying levels of poverty. METHODS We collected and analyzed the prevalence patterns along with the cost of control measures in 11 counties over the last 20 years in China. Seven machine learning models, including XGBoost, support vector machine, generalized linear model, regression tree, random forest, gradient boosting machine and neural network, were used for developing model and calculate marginal benefits. RESULTS The XGBoost model had the highest prediction accuracy with an R2 of 0.7308. Results showed that risk surveillance, snail control with molluscicides and treatment were the most effective interventions in controlling schistosomiasis prevalence. The best combination of interventions was interlacing seven interventions, including risk surveillance, treatment, toilet construction, health education, snail control with molluscicides, cattle slaughter and animal chemotherapy. The marginal benefit of risk surveillance is the most effective intervention among nine interventions, which was influenced by the prevalence of schistosomiasis and cost. CONCLUSIONS In the elimination phase of the national schistosomiasis program, emphasizing risk surveillance holds significant importance in terms of cost-saving.
Collapse
Affiliation(s)
- Qin Li
- National Institute of Parasitic Diseases, Chinese Center for Disease Control and Prevention (Chinese Center for Tropical Diseases Research), National Health Commission Key Laboratory of Parasite and Vector Biology, WHO Collaborating Centre for Tropical Diseases, National Center for International Research on Tropical Diseases, Shanghai, 200025, China
| | - Jin-Xin Zheng
- Ruijin Hospital Affiliated to The Shanghai Jiao Tong University Medical School, Shanghai, 200025, China
| | - Tie-Wu Jia
- National Institute of Parasitic Diseases, Chinese Center for Disease Control and Prevention (Chinese Center for Tropical Diseases Research), National Health Commission Key Laboratory of Parasite and Vector Biology, WHO Collaborating Centre for Tropical Diseases, National Center for International Research on Tropical Diseases, Shanghai, 200025, China
| | - Xin-Yu Feng
- National Institute of Parasitic Diseases, Chinese Center for Disease Control and Prevention (Chinese Center for Tropical Diseases Research), National Health Commission Key Laboratory of Parasite and Vector Biology, WHO Collaborating Centre for Tropical Diseases, National Center for International Research on Tropical Diseases, Shanghai, 200025, China
| | - Chao Lv
- National Institute of Parasitic Diseases, Chinese Center for Disease Control and Prevention (Chinese Center for Tropical Diseases Research), National Health Commission Key Laboratory of Parasite and Vector Biology, WHO Collaborating Centre for Tropical Diseases, National Center for International Research on Tropical Diseases, Shanghai, 200025, China
- School of Global Health, Chinese Center for Tropical Diseases Research and Shanghai Jiao Tong University School of Medicine, One Health Center, Shanghai Jiao Tong University and The Edinburgh University, Shanghai, 200025, China
| | - Li-Juan Zhang
- National Institute of Parasitic Diseases, Chinese Center for Disease Control and Prevention (Chinese Center for Tropical Diseases Research), National Health Commission Key Laboratory of Parasite and Vector Biology, WHO Collaborating Centre for Tropical Diseases, National Center for International Research on Tropical Diseases, Shanghai, 200025, China
| | - Guo-Jing Yang
- School of Tropical Medicine, Hainan Medical University, Haikou, 571199, China
| | - Jing Xu
- National Institute of Parasitic Diseases, Chinese Center for Disease Control and Prevention (Chinese Center for Tropical Diseases Research), National Health Commission Key Laboratory of Parasite and Vector Biology, WHO Collaborating Centre for Tropical Diseases, National Center for International Research on Tropical Diseases, Shanghai, 200025, China
| | - Xiao-Nong Zhou
- National Institute of Parasitic Diseases, Chinese Center for Disease Control and Prevention (Chinese Center for Tropical Diseases Research), National Health Commission Key Laboratory of Parasite and Vector Biology, WHO Collaborating Centre for Tropical Diseases, National Center for International Research on Tropical Diseases, Shanghai, 200025, China.
- School of Global Health, Chinese Center for Tropical Diseases Research and Shanghai Jiao Tong University School of Medicine, One Health Center, Shanghai Jiao Tong University and The Edinburgh University, Shanghai, 200025, China.
| |
Collapse
|
12
|
Li X, Zhang D, Zhao Y, Kuang L, Huang H, Chen W, Fu X, Wu Y, Li T, Zhang J, Yuan L, Hu H, Liu Y, Hu F, Zhang M, Sun X, Hu D. Correlation of heavy metals' exposure with the prevalence of coronary heart disease among US adults: findings of the US NHANES from 2003 to 2018. ENVIRONMENTAL GEOCHEMISTRY AND HEALTH 2023; 45:6745-6759. [PMID: 37378736 DOI: 10.1007/s10653-023-01670-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/11/2023] [Accepted: 06/19/2023] [Indexed: 06/29/2023]
Abstract
We sought to explore the association between heavy metal exposure and coronary heart disease (CHD) based on data from the US National Health and Nutrition Examination Survey (NHANES, 2003-2018). In the analyses, participants were all aged > 20 and had participated in heavy metal sub-tests with valid CHD status. The Mann-Kendall test was employed to assess the trends in heavy metals' exposure and the trends in CHD prevalence over 16 years. Spearman's rank correlation coefficient and a logistics regression (LR) model were used to estimate the association between heavy metals and CHD prevalence. 42,749 participants were included in our analyses, 1802 of whom had a CHD diagnosis. Total arsenic, dimethylarsonic acid, monomethylarsonic acid, barium, cadmium, lead, and antimony in urine, and cadmium, lead, and total mercury in blood all showed a substantial decreasing exposure level tendency over the 16 years (all Pfor trend < 0.05). CHD prevalence varied from 3.53 to 5.23% between 2003 and 2018. The correlation between 15 heavy metals and CHD ranges from - 0.238 to 0.910. There was also a significant positive correlation between total arsenic, monomethylarsonic acid, and thallium in urine and CHD by data release cycles (all P < 0.05). The cesium in urine showed a negative correlation with CHD (P < 0.05). We found that exposure trends of total arsenic, dimethylarsonic acid, monomethylarsonic acid, barium, cadmium, lead, and antimony in urine and blood decreased. CHD prevalence fluctuated, however. Moreover, total arsenic, monomethylarsonic acid, and thallium in urine all showed positive relationships with CHD, while cesium in urine showed a negative relationship with CHD.
Collapse
Affiliation(s)
- Xi Li
- Department of Epidemiology and Health Statistics, College of Public Health, Zhengzhou University, 100 Kexue Avenue, Zhengzhou, 450001, Henan, People's Republic of China
| | - Dongdong Zhang
- Department of General Practice, The Affiliated Luohu Hospital of Shenzhen University Medical School, Shenzhen, People's Republic of China
| | - Yang Zhao
- Department of Epidemiology and Health Statistics, College of Public Health, Zhengzhou University, 100 Kexue Avenue, Zhengzhou, 450001, Henan, People's Republic of China
| | - Lei Kuang
- Department of Biostatistics and Epidemiology, School of Public Health, Shenzhen University Medical School, Shenzhen, Guangdong, People's Republic of China
| | - Hao Huang
- Department of General Practice, The Affiliated Luohu Hospital of Shenzhen University Medical School, Shenzhen, People's Republic of China
| | - Weiling Chen
- Department of Biostatistics and Epidemiology, School of Public Health, Shenzhen University Medical School, Shenzhen, Guangdong, People's Republic of China
| | - Xueru Fu
- Department of Epidemiology and Health Statistics, College of Public Health, Zhengzhou University, 100 Kexue Avenue, Zhengzhou, 450001, Henan, People's Republic of China
| | - Yuying Wu
- Department of Epidemiology and Health Statistics, College of Public Health, Zhengzhou University, 100 Kexue Avenue, Zhengzhou, 450001, Henan, People's Republic of China
| | - Tianze Li
- Department of Epidemiology and Health Statistics, College of Public Health, Zhengzhou University, 100 Kexue Avenue, Zhengzhou, 450001, Henan, People's Republic of China
| | - Jinli Zhang
- Department of Epidemiology and Health Statistics, College of Public Health, Zhengzhou University, 100 Kexue Avenue, Zhengzhou, 450001, Henan, People's Republic of China
| | - Lijun Yuan
- Department of Epidemiology and Health Statistics, College of Public Health, Zhengzhou University, 100 Kexue Avenue, Zhengzhou, 450001, Henan, People's Republic of China
| | - Huifang Hu
- Department of Epidemiology and Health Statistics, College of Public Health, Zhengzhou University, 100 Kexue Avenue, Zhengzhou, 450001, Henan, People's Republic of China
| | - Yu Liu
- Department of General Practice, The Affiliated Luohu Hospital of Shenzhen University Medical School, Shenzhen, People's Republic of China
| | - Fulan Hu
- Department of Biostatistics and Epidemiology, School of Public Health, Shenzhen University Medical School, Shenzhen, Guangdong, People's Republic of China
| | - Ming Zhang
- Department of Biostatistics and Epidemiology, School of Public Health, Shenzhen University Medical School, Shenzhen, Guangdong, People's Republic of China
| | - Xizhuo Sun
- Department of General Practice, The Affiliated Luohu Hospital of Shenzhen University Medical School, Shenzhen, People's Republic of China
| | - Dongsheng Hu
- Department of Epidemiology and Health Statistics, College of Public Health, Zhengzhou University, 100 Kexue Avenue, Zhengzhou, 450001, Henan, People's Republic of China.
| |
Collapse
|
13
|
Li W, Huang G, Tang N, Lu P, Jiang L, Lv J, Qin Y, Lin Y, Xu F, Lei D. Effects of heavy metal exposure on hypertension: A machine learning modeling approach. CHEMOSPHERE 2023; 337:139435. [PMID: 37422210 DOI: 10.1016/j.chemosphere.2023.139435] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/11/2023] [Revised: 07/04/2023] [Accepted: 07/05/2023] [Indexed: 07/10/2023]
Abstract
Heavy metal exposure is a common risk factor for hypertension. To develop an interpretable predictive machine learning (ML) model for hypertension based on levels of heavy metal exposure, data from the NHANES (2003-2016) were employed. Random forest (RF), support vector machine (SVM), decision tree (DT), multilayer perceptron (MLP), ridge regression (RR), AdaBoost (AB), gradient boosting decision tree (GBDT), voting classifier (VC), and K-nearest neighbour (KNN) algorithms were utilized to generate an optimal predictive model for hypertension. Three interpretable methods, the permutation feature importance analysis, partial dependence plot (PDP), and Shapley additive explanations (SHAP) methods, were integrated into a pipeline and embedded in ML for model interpretation. A total of 9005 eligible individuals were randomly allocated into two distinct sets for predictive model training and validation. The results showed that among the predictive models, the RF model demonstrated the highest performance, achieving an accuracy rate of 77.40% in the validation set. The AUC and F1 score for the model were 0.84 and 0.76, respectively. Blood Pb, urinary Cd, urinary Tl, and urinary Co levels were identified as the main influencers of hypertension, and their contribution weights were 0.0504 ± 0.0482, 0.0389 ± 0.0256, 0.0307 ± 0.0179, and 0.0296 ± 0.0162, respectively. Blood Pb (0.55-2.93 μg/dL) and urinary Cd (0.06-0.15 μg/L) levels exhibited the most pronounced upwards trend with the risk of hypertension within a specific value range, while urinary Tl (0.06-0.26 μg/L) and urinary Co (0.02-0.32 μg/L) levels demonstrated a declining trend with hypertension. The findings on the synergistic effects indicated that Pb and Cd were the primary determinants of hypertension. Our findings underscore the predictive value of heavy metals for hypertension. By utilizing interpretable methods, we discerned that Pb, Cd, Tl, and Co emerged as noteworthy contributors within the predictive model.
Collapse
Affiliation(s)
- Wenxiang Li
- Department of Ophthalmology, the People's Hospital of Guangxi Zhuang Autonomous Region & Institute of Ophthalmic Diseases, Guangxi Academy of Medical Sciences & Guangxi Key Laboratory of Eye Health & Guangxi Health Commission Key Laboratory of Ophthalmology and Related Systemic Diseases Artificial Intelligence Screening Technology, Nanning, 530021, China.
| | - Guangyi Huang
- Department of Ophthalmology, the People's Hospital of Guangxi Zhuang Autonomous Region & Institute of Ophthalmic Diseases, Guangxi Academy of Medical Sciences & Guangxi Key Laboratory of Eye Health & Guangxi Health Commission Key Laboratory of Ophthalmology and Related Systemic Diseases Artificial Intelligence Screening Technology, Nanning, 530021, China
| | - Ningning Tang
- Department of Ophthalmology, the People's Hospital of Guangxi Zhuang Autonomous Region & Institute of Ophthalmic Diseases, Guangxi Academy of Medical Sciences & Guangxi Key Laboratory of Eye Health & Guangxi Health Commission Key Laboratory of Ophthalmology and Related Systemic Diseases Artificial Intelligence Screening Technology, Nanning, 530021, China
| | - Peng Lu
- Department of Ophthalmology, the People's Hospital of Guangxi Zhuang Autonomous Region & Institute of Ophthalmic Diseases, Guangxi Academy of Medical Sciences & Guangxi Key Laboratory of Eye Health & Guangxi Health Commission Key Laboratory of Ophthalmology and Related Systemic Diseases Artificial Intelligence Screening Technology, Nanning, 530021, China
| | - Li Jiang
- Department of Ophthalmology, the People's Hospital of Guangxi Zhuang Autonomous Region & Institute of Ophthalmic Diseases, Guangxi Academy of Medical Sciences & Guangxi Key Laboratory of Eye Health & Guangxi Health Commission Key Laboratory of Ophthalmology and Related Systemic Diseases Artificial Intelligence Screening Technology, Nanning, 530021, China
| | - Jian Lv
- Department of Ophthalmology, the People's Hospital of Guangxi Zhuang Autonomous Region & Institute of Ophthalmic Diseases, Guangxi Academy of Medical Sciences & Guangxi Key Laboratory of Eye Health & Guangxi Health Commission Key Laboratory of Ophthalmology and Related Systemic Diseases Artificial Intelligence Screening Technology, Nanning, 530021, China
| | - Yuanjun Qin
- Department of Ophthalmology, the People's Hospital of Guangxi Zhuang Autonomous Region & Institute of Ophthalmic Diseases, Guangxi Academy of Medical Sciences & Guangxi Key Laboratory of Eye Health & Guangxi Health Commission Key Laboratory of Ophthalmology and Related Systemic Diseases Artificial Intelligence Screening Technology, Nanning, 530021, China
| | - Yunru Lin
- Department of Ophthalmology, the People's Hospital of Guangxi Zhuang Autonomous Region & Institute of Ophthalmic Diseases, Guangxi Academy of Medical Sciences & Guangxi Key Laboratory of Eye Health & Guangxi Health Commission Key Laboratory of Ophthalmology and Related Systemic Diseases Artificial Intelligence Screening Technology, Nanning, 530021, China
| | - Fan Xu
- Department of Ophthalmology, the People's Hospital of Guangxi Zhuang Autonomous Region & Institute of Ophthalmic Diseases, Guangxi Academy of Medical Sciences & Guangxi Key Laboratory of Eye Health & Guangxi Health Commission Key Laboratory of Ophthalmology and Related Systemic Diseases Artificial Intelligence Screening Technology, Nanning, 530021, China.
| | - Daizai Lei
- Department of Ophthalmology, the People's Hospital of Guangxi Zhuang Autonomous Region & Institute of Ophthalmic Diseases, Guangxi Academy of Medical Sciences & Guangxi Key Laboratory of Eye Health & Guangxi Health Commission Key Laboratory of Ophthalmology and Related Systemic Diseases Artificial Intelligence Screening Technology, Nanning, 530021, China.
| |
Collapse
|
14
|
Huang AA, Huang SY. Increasing transparency in machine learning through bootstrap simulation and shapely additive explanations. PLoS One 2023; 18:e0281922. [PMID: 36821544 PMCID: PMC9949629 DOI: 10.1371/journal.pone.0281922] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2022] [Accepted: 02/05/2023] [Indexed: 02/24/2023] Open
Abstract
Machine learning methods are widely used within the medical field. However, the reliability and efficacy of these models is difficult to assess, making it difficult for researchers to identify which machine-learning model to apply to their dataset. We assessed whether variance calculations of model metrics (e.g., AUROC, Sensitivity, Specificity) through bootstrap simulation and SHapely Additive exPlanations (SHAP) could increase model transparency and improve model selection. Data from the England National Health Services Heart Disease Prediction Cohort was used. After comparison of model metrics for XGBoost, Random Forest, Artificial Neural Network, and Adaptive Boosting, XGBoost was used as the machine-learning model of choice in this study. Boost-strap simulation (N = 10,000) was used to empirically derive the distribution of model metrics and covariate Gain statistics. SHapely Additive exPlanations (SHAP) to provide explanations to machine-learning output and simulation to evaluate the variance of model accuracy metrics. For the XGBoost modeling method, we observed (through 10,000 completed simulations) that the AUROC ranged from 0.771 to 0.947, a difference of 0.176, the balanced accuracy ranged from 0.688 to 0.894, a 0.205 difference, the sensitivity ranged from 0.632 to 0.939, a 0.307 difference, and the specificity ranged from 0.595 to 0.944, a 0.394 difference. Among 10,000 simulations completed, we observed that the gain for Angina ranged from 0.225 to 0.456, a difference of 0.231, for Cholesterol ranged from 0.148 to 0.326, a difference of 0.178, for maximum heart rate (MaxHR) ranged from 0.081 to 0.200, a range of 0.119, and for Age ranged from 0.059 to 0.157, difference of 0.098. Use of simulations to empirically evaluate the variability of model metrics and explanatory algorithms to observe if covariates match the literature are necessary for increased transparency, reliability, and utility of machine learning methods. These variance statistics, combined with model accuracy statistics can help researchers identify the best model for a given dataset.
Collapse
Affiliation(s)
- Alexander A. Huang
- Department of Statistics and Data Science, Cornell University, Ithaca, New York, United States of America
- Department of MD Education, Northwestern University Feinberg School of Medicine, Chicago, Illinois, United States of America
| | - Samuel Y. Huang
- Department of Statistics and Data Science, Cornell University, Ithaca, New York, United States of America
- Department of Internal Medicine, Virginia Commonwealth University School of Medicine, Richmond, Virginia, United States of America
| |
Collapse
|