1
|
Salih AM, Galazzo IB, Gkontra P, Rauseo E, Lee AM, Lekadir K, Radeva P, Petersen SE, Menegaz G. A review of evaluation approaches for explainable AI with applications in cardiology. Artif Intell Rev 2024; 57:240. [PMID: 39132011 PMCID: PMC11315784 DOI: 10.1007/s10462-024-10852-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/03/2024] [Indexed: 08/13/2024]
Abstract
Explainable artificial intelligence (XAI) elucidates the decision-making process of complex AI models and is important in building trust in model predictions. XAI explanations themselves require evaluation as to accuracy and reasonableness and in the context of use of the underlying AI model. This review details the evaluation of XAI in cardiac AI applications and has found that, of the studies examined, 37% evaluated XAI quality using literature results, 11% used clinicians as domain-experts, 11% used proxies or statistical analysis, with the remaining 43% not assessing the XAI used at all. We aim to inspire additional studies within healthcare, urging researchers not only to apply XAI methods but to systematically assess the resulting explanations, as a step towards developing trustworthy and safe models. Supplementary Information The online version contains supplementary material available at 10.1007/s10462-024-10852-w.
Collapse
Affiliation(s)
- Ahmed M. Salih
- William Harvey Research Institute, NIHR Barts Biomedical Research Centre, Queen Mary University of London, Charterhouse Square, London, EC1M 6BQ UK
- Department of Population Health Sciences, University of Leicester, University Rd, Leicester, LE1 7RH UK
- Department of Computer Science, University of Zakho, Duhok road, Zakho, Kurdistan Iraq
| | - Ilaria Boscolo Galazzo
- Department of Engineering for Innovative Medicine, University of Verona, S. Francesco, 22, 37129 Verona, Italy
| | - Polyxeni Gkontra
- Artificial Intelligence in Medicine Lab (BCN-AIM), Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Gran Via de les Corts Catalanes, 585, 08007 Barcelona, Spain
| | - Elisa Rauseo
- William Harvey Research Institute, NIHR Barts Biomedical Research Centre, Queen Mary University of London, Charterhouse Square, London, EC1M 6BQ UK
| | - Aaron Mark Lee
- William Harvey Research Institute, NIHR Barts Biomedical Research Centre, Queen Mary University of London, Charterhouse Square, London, EC1M 6BQ UK
| | - Karim Lekadir
- Artificial Intelligence in Medicine Lab (BCN-AIM), Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Gran Via de les Corts Catalanes, 585, 08007 Barcelona, Spain
- Institució Catalana de Recerca i Estudis Avançats (ICREA), Passeig Lluís Companys 23, Barcelona, Spain
| | - Petia Radeva
- Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Gran Via de les Corts Catalanes, 585, 08007 Barcelona, Spain
| | - Steffen E. Petersen
- William Harvey Research Institute, NIHR Barts Biomedical Research Centre, Queen Mary University of London, Charterhouse Square, London, EC1M 6BQ UK
- Barts Heart Centre, St Bartholomew’s Hospital, Barts Health NHS Trust, West Smithfield, London, UK
- Health Data Research, London, UK
- Alan Turing Institute, London, UK
| | - Gloria Menegaz
- Department of Engineering for Innovative Medicine, University of Verona, S. Francesco, 22, 37129 Verona, Italy
| |
Collapse
|
2
|
Zhang C, Xu B, Zhu C, Pu K, Bian L. Risk factors of malnutrition in children with congenital heart disease: a meta-analysis. Front Pediatr 2024; 12:1258725. [PMID: 39135858 PMCID: PMC11317268 DOI: 10.3389/fped.2024.1258725] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Accepted: 07/08/2024] [Indexed: 08/15/2024] Open
Abstract
Background The associated factors of malnutrition in children with congenital heart disease (CHD) must be evaluated to provide evidence for the treatment and care of such children. Methods Two investigators searched the PubMed database until 25 June 2023 for literature about the associated factors of malnutrition in children with CHD. A meta-analysis of associated factors of malnutrition was performed by RevMan 5.3 software. Results Thirteen studies involving 8,031 children with CHD were included. Pulmonary hypertension (OR = 3.81, 95% CI: 2.46-4.12), low birth weight (OR = 2.69, 95% CI: 1.25-5.77) and parents' height (OR = 2.15, 95% CI: 1.89-2.92) were the associated factors of growth retardation (all P < 0.05). Pulmonary hypertension (OR = 3.77, 95% CI: 3.13-4.24), low birth weight (OR = 3.04, 95% CI: 2.61-4.18) and pneumonia (OR = 2.35, 95% CI: 2.08-2.83) were the associated factors of low body weight of children with CHD (all P < 0.05). Conclusions Medical staff should fully understand the risk factors, strengthen nutritional support and enhance nursing care for children with CHD to reduce malnutrition.
Collapse
Affiliation(s)
| | | | | | - Kai Pu
- Department of Cardiothoracic Surgery, Children’s Hospital of Nanjing Medical University, Nanjing, China
| | - Lanzheng Bian
- Department of Cardiothoracic Surgery, Children’s Hospital of Nanjing Medical University, Nanjing, China
| |
Collapse
|
3
|
Wang X, Liu Y, Rong Z, Wang W, Han M, Chen M, Fu J, Chong Y, Long X, Tang Y, Chen W. Development and evaluation of a deep learning framework for the diagnosis of malnutrition using a 3D facial points cloud: A cross-sectional study. JPEN J Parenter Enteral Nutr 2024; 48:554-561. [PMID: 38796717 DOI: 10.1002/jpen.2643] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Revised: 04/25/2024] [Accepted: 04/30/2024] [Indexed: 05/28/2024]
Abstract
BACKGROUND The feasibility of diagnosing malnutrition using facial features has been validated. A tool to integrate all facial features associated with malnutrition for disease screening is still demanded. This work aims to develop and evaluate a deep learning (DL) framework to accurately determine malnutrition based on a 3D facial points cloud. METHODS A group of 482 patients were studied in this perspective work. The 3D facial data were obtained using a 3D camera and represented as a 3D facial points cloud. A DL model, PointNet++, was trained and evaluated using the points cloud as inputs and classified the malnutrition states. The performance was evaluated with the area under the receiver operating characteristic curve, accuracy, specificity, sensitivity, and F1 score. RESULTS Among the 482 patients, 150 patients (31.1%) were diagnosed as having moderate malnutrition and 54 patients (11.2%) as having severe malnutrition. The DL model achieved the performance with an area under the receiver operating characteristic curve of 0.7240 ± 0.0416. CONCLUSION The DL model achieved encouraging performance in accurately classifying nutrition states based on a points cloud of 3D facial information of patients with malnutrition.
Collapse
Affiliation(s)
- Xue Wang
- Department of Clinical Nutrition, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yan Liu
- Department of Clinical Nutrition, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Zhiqin Rong
- Genesis Artificial Intelligence Laboratory, Futong Technology, Chengdu, China
| | - Weijia Wang
- Genesis Artificial Intelligence Laboratory, Futong Technology, Chengdu, China
| | - Meifen Han
- Department of Pharmacy, Peking Union Medical College Hospital, Chinese Academy of Medical Science & Peking Union Medical College, Beijing, China
- School of Basic Medicine and Clinical Pharmacy, China Pharmaceutical University, Nanjing, China
| | - Moxi Chen
- Department of Clinical Nutrition, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Jin Fu
- Department of Clinical Nutrition, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yuming Chong
- Department of Plastic Surgery, Peking Union Medical College Hospital, Beijing, China
| | - Xiao Long
- Department of Plastic Surgery, Peking Union Medical College Hospital, Beijing, China
| | - Yong Tang
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Wei Chen
- Department of Clinical Nutrition, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
4
|
Shafiabady N, Hadjinicolaou N, Hettikankanamage N, MohammadiSavadkoohi E, Wu RMX, Vakilian J. eXplainable Artificial Intelligence (XAI) for improving organisational regility. PLoS One 2024; 19:e0301429. [PMID: 38656983 PMCID: PMC11042710 DOI: 10.1371/journal.pone.0301429] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2023] [Accepted: 03/15/2024] [Indexed: 04/26/2024] Open
Abstract
Since the pandemic started, organisations have been actively seeking ways to improve their organisational agility and resilience (regility) and turn to Artificial Intelligence (AI) to gain a deeper understanding and further enhance their agility and regility. Organisations are turning to AI as a critical enabler to achieve these goals. AI empowers organisations by analysing large data sets quickly and accurately, enabling faster decision-making and building agility and resilience. This strategic use of AI gives businesses a competitive advantage and allows them to adapt to rapidly changing environments. Failure to prioritise agility and responsiveness can result in increased costs, missed opportunities, competition and reputational damage, and ultimately, loss of customers, revenue, profitability, and market share. Prioritising can be achieved by utilising eXplainable Artificial Intelligence (XAI) techniques, illuminating how AI models make decisions and making them transparent, interpretable, and understandable. Based on previous research on using AI to predict organisational agility, this study focuses on integrating XAI techniques, such as Shapley Additive Explanations (SHAP), in organisational agility and resilience. By identifying the importance of different features that affect organisational agility prediction, this study aims to demystify the decision-making processes of the prediction model using XAI. This is essential for the ethical deployment of AI, fostering trust and transparency in these systems. Recognising key features in organisational agility prediction can guide companies in determining which areas to concentrate on in order to improve their agility and resilience.
Collapse
Affiliation(s)
- Niusha Shafiabady
- Faculty of Science and Technology, Charles Darwin University, Haymarket, New South Wales, Australia
| | - Nick Hadjinicolaou
- Adelaide Institute of Higher Education, Adelaide, South Australia, Australia
| | | | | | - Robert M. X. Wu
- Faculty of Engineering and Information Technology, University of Technology Sydney, Broadway, New South Wales, Australia
| | - James Vakilian
- Faculty of Science and Technology, Charles Darwin University, Haymarket, New South Wales, Australia
| |
Collapse
|
5
|
Xie X, Li J, Zhong Y, Fang Z, Feng Y, Chen C, Zou J, Si Y. A risk prediction model based on machine learning for postoperative cognitive dysfunction in elderly patients with non-cardiac surgery. Aging Clin Exp Res 2023; 35:2951-2960. [PMID: 37864763 DOI: 10.1007/s40520-023-02573-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2023] [Accepted: 09/20/2023] [Indexed: 10/23/2023]
Abstract
BACKGROUND Early identification of elderly patients undergoing non-cardiac surgery who may be at high risk for postoperative cognitive dysfunction (POCD) can increase the chances of prevention for them, as extra attention and limited resources can be allocated more to these patients. AIM We performed this analysis with the aim of developing a simple, clinically useful machine learning (ML) model to predict the probability of POCD at 3 months in elderly patients after non-cardiac surgery. METHODS We collected information on patients who received surgical treatment at Nanjing First Hospital from May 2020 to May 2021. We used LASSO regression to select key features and built 5 ML models to assess the risk of POCD at 3 months in elderly patients after non-cardiac surgery. The Shapley Additive exPlanations (SHAP) and methods were introduced to interpret the best model. RESULTS A total of 415 patients with non-cardiac surgery were included. The support vector machine (SVM) was the best-performing model of the five ML models. The model showed excellent performance compared to the other four models. The SHAP results showed that VAS score, age, intraoperative hypotension, and preoperative hemoglobin were the four most important features, indicating that the SVM model had good interpretability and reliability. The website of the web-based calculator was https://modricreagan-non-3-pocd-9w2q78.streamlit.app/ . CONCLUSION Based on six important perioperative variables, we successfully established a series of ML models for predicting POCD occurrence at 3 months after surgery in elderly non-cardiac patients, with SVM model being the best-performing model. Our models are expected to serve as decision aids for clinicians to monitor screened high-risk patients more closely or to consider further interventions.
Collapse
Affiliation(s)
- Xianhai Xie
- School of Basic Medicine and Clinical Pharmacy, China Pharmaceutical University, Nanjing, China
- Department of Clinical Pharmacology, Nanjing First Hospital, Nanjing Medical University, Nanjing, China
| | - Junlin Li
- School of Basic Medicine and Clinical Pharmacy, China Pharmaceutical University, Nanjing, China
- Department of Clinical Pharmacology, Nanjing First Hospital, Nanjing Medical University, Nanjing, China
| | - Yi Zhong
- Department of Anesthesiology, Nanjing First Hospital, Nanjing Medical University, Nanjing, China
| | - Zhaojing Fang
- Department of Anesthesiology, Nanjing First Hospital, Nanjing Medical University, Nanjing, China
| | - Yue Feng
- Department of Anesthesiology, Nanjing First Hospital, Nanjing Medical University, Nanjing, China
| | - Chen Chen
- Department of Clinical Pharmacology, Nanjing First Hospital, Nanjing Medical University, Nanjing, China
- Department of Pharmacy, Nanjing First Hospital, China Pharmaceutical University, Nanjing, China
| | - Jianjun Zou
- Department of Clinical Pharmacology, Nanjing First Hospital, Nanjing Medical University, Nanjing, China.
- Department of Pharmacy, Nanjing First Hospital, China Pharmaceutical University, Nanjing, China.
| | - Yanna Si
- Department of Anesthesiology, Nanjing First Hospital, Nanjing Medical University, Nanjing, China.
| |
Collapse
|
6
|
Hsu W, Ko A, Weng C, Chang C, Jan Y, Lin J, Chien H, Lin W, Sun F, Wu K, Lee J. Explainable machine learning model for predicting skeletal muscle loss during surgery and adjuvant chemotherapy in ovarian cancer. J Cachexia Sarcopenia Muscle 2023; 14:2044-2053. [PMID: 37435785 PMCID: PMC10570082 DOI: 10.1002/jcsm.13282] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Revised: 03/30/2023] [Accepted: 05/22/2023] [Indexed: 07/13/2023] Open
Abstract
BACKGROUND Skeletal muscle loss during treatment is associated with poor survival outcomes in patients with ovarian cancer. Although changes in muscle mass can be assessed on computed tomography (CT) scans, this labour-intensive process can impair its utility in clinical practice. This study aimed to develop a machine learning (ML) model to predict muscle loss based on clinical data and to interpret the ML model by applying SHapley Additive exPlanations (SHAP) method. METHODS This study included the data of 617 patients with ovarian cancer who underwent primary debulking surgery and platinum-based chemotherapy at a tertiary centre between 2010 and 2019. The cohort data were split into training and test sets based on the treatment time. External validation was performed using 140 patients from a different tertiary centre. The skeletal muscle index (SMI) was measured from pre- and post-treatment CT scans, and a decrease in SMI ≥ 5% was defined as muscle loss. We evaluated five ML models to predict muscle loss, and their performance was determined using the area under the receiver operating characteristic curve (AUC) and F1 score. The features for analysis included demographic and disease-specific characteristics and relative changes in body mass index (BMI), albumin, neutrophil-to-lymphocyte ratio (NLR), and platelet-to-lymphocyte ratio (PLR). The SHAP method was applied to determine the importance of the features and interpret the ML models. RESULTS The median (inter-quartile range) age of the cohort was 52 (46-59) years. After treatment, 204 patients (33.1%) experienced muscle loss in the training and test datasets, while 44 (31.4%) patients experienced muscle loss in the external validation dataset. Among the five evaluated ML models, the random forest model achieved the highest AUC (0.856, 95% confidence interval: 0.854-0.859) and F1 score (0.726, 95% confidence interval: 0.722-0.730). In the external validation, the random forest model outperformed all ML models with an AUC of 0.874 and an F1 score of 0.741. The results of the SHAP method showed that the albumin change, BMI change, malignant ascites, NLR change, and PLR change were the most important factors in muscle loss. At the patient level, SHAP force plots demonstrated insightful interpretation of our random forest model to predict muscle loss. CONCLUSIONS Explainable ML model was developed using clinical data to identify patients experiencing muscle loss after treatment and provide information of feature contribution. Using the SHAP method, clinicians may better understand the contributors to muscle loss and target interventions to counteract muscle loss.
Collapse
Affiliation(s)
- Wen‐Han Hsu
- Institute of Biomedical InformaticsNational Yang Ming Chiao Tung UniversityTaipeiTaiwan
| | - Ai‐Tung Ko
- Institute of Biomedical InformaticsNational Yang Ming Chiao Tung UniversityTaipeiTaiwan
| | - Chia‐Sui Weng
- Department of Obstetrics and GynecologyMacKay Memorial HospitalTaipeiTaiwan
- Department of MedicineMacKay Medical CollegeNew Taipei CityTaiwan
| | - Chih‐Long Chang
- Department of Obstetrics and GynecologyMacKay Memorial HospitalTaipeiTaiwan
- Department of MedicineMacKay Medical CollegeNew Taipei CityTaiwan
| | - Ya‐Ting Jan
- Department of RadiologyMacKay Memorial HospitalTaipeiTaiwan
| | - Jhen‐Bin Lin
- Department of Radiation OncologyChanghua Christian HospitalChanghuaTaiwan
| | - Hung‐Ju Chien
- Department of Obstetrics and GynecologyChanghua Christian HospitalTaipeiTaiwan
| | - Wan‐Chun Lin
- Institute of Biomedical InformaticsNational Yang Ming Chiao Tung UniversityTaipeiTaiwan
| | - Fang‐Ju Sun
- Institute of Biomedical InformaticsNational Yang Ming Chiao Tung UniversityTaipeiTaiwan
- Department of Medical ResearchMacKay Memorial HospitalTaipeiTaiwan
| | - Kun‐Pin Wu
- Institute of Biomedical InformaticsNational Yang Ming Chiao Tung UniversityTaipeiTaiwan
| | - Jie Lee
- Department of MedicineMacKay Medical CollegeNew Taipei CityTaiwan
- Department of Radiation OncologyMacKay Memorial HospitalTaipeiTaiwan
| |
Collapse
|
7
|
Huang L, Niannian F, Zhang Y, Shi Y, Gao X, Zhang L, Wu Y, Dai C, Huang Y, Cao Q, Zhou W, Jiang H, Jing X, Zhu W, Wang X. Rapid assessment of malnutrition based on GLIM diagnosis in Crohn's disease. Front Nutr 2023; 10:1236036. [PMID: 37736137 PMCID: PMC10510406 DOI: 10.3389/fnut.2023.1236036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2023] [Accepted: 08/24/2023] [Indexed: 09/23/2023] Open
Abstract
Background and aims Malnutrition is strongly linked to adverse outcomes in patients with Crohn's disease (CD). In this study, our objective was to validate the Global Leadership Initiative on Malnutrition (GLIM) criteria and develop a fast and accurate diagnostic approach for identifying malnutrition in CD patients. Methods This study assessed 177 CD patients from four general hospitals. The efficacy of the GLIM criteria for the diagnosis of CD malnutrition was compared. By analyzing the independent factors, a nomogram model was derived and internally validated to predict the diagnosis of malnutrition in patients with CD. Model performance was assessed using discrimination and calibration, decision curves, and net benefit analyses. Results Compared with the SGA criteria, the GLIM criteria was consistent in sensitivity (88.89%) and specificity (78.43%) [AUC = 0.84; 95% Confidence Interval (CI): 0.77-0.89]. The Harvey-Bradshaw index(HBI) score (OR: 1.58; 95% CI: 1.15-2.18), body mass index (OR: 0.41; 95% CI: 0.27-0.64), and mid-upper arm circumference (OR: 0.68; 95% CI: 0.47-0.9) were independent factors associated with malnutrition. The nomogram was developed based on these indicators showing good discrimination in malnutrition diagnosis (AUC = 0.953; 95% CI: 0.922-0.984), with agreement after calibration curve and decision curve analysis. Conclusion The GLIM criteria are appropriate for diagnosing malnutrition in CD patients. The HBI score may be used to diagnose malnutrition in patients with CD and become a possible selection for the GLIM etiologic criteria of inflammation. The HBM nomogram could be a simple, rapid, and efficient method for diagnosing malnutrition in CD patients.
Collapse
Affiliation(s)
- Longchang Huang
- Department of General Surgery, Nanjing Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China
| | - Fu Niannian
- Department of General Surgery, Nanjing Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China
| | - Yupeng Zhang
- Department of General Surgery, Nanjing Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China
| | - Yifan Shi
- Department of General Surgery, Nanjing Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China
| | - Xuejin Gao
- Department of General Surgery, Nanjing Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China
| | - Li Zhang
- Department of General Surgery, Nanjing Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China
| | - Yan Wu
- Department of Pharmacy, Sir Run Run Shaw Hospital, College of Medicine, Zhejiang University, Hangzhou, China
| | - Cong Dai
- Department of Gastroenterology, First Affiliated Hospital, China Medical University, Shenyang, China
| | - Yuhong Huang
- Department of Gastroenterology, First Affiliated Hospital, China Medical University, Shenyang, China
| | - Qian Cao
- Inflammatory Bowel Disease Center, Sir Run Run Shaw Hospital, College of Medicine, Zhejiang University, Hangzhou, China
| | - Wei Zhou
- Department of General Surgery, Sir Run Run Shaw Hospital, College of Medicine, Zhejiang University, Hangzhou, China
| | - Haitao Jiang
- Department of Gastrointestinal Surgery, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Xue Jing
- Department of Gastroenterology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Weiming Zhu
- Department of General Surgery, Nanjing Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China
| | - Xinying Wang
- Department of General Surgery, Nanjing Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China
| |
Collapse
|
8
|
Wang X, Yang F, Zhu M, Cui H, Wei J, Li J, Chen W. Development and Assessment of Assisted Diagnosis Models Using Machine Learning for Identifying Elderly Patients With Malnutrition: Cohort Study. J Med Internet Res 2023; 25:e42435. [PMID: 36917167 PMCID: PMC10131894 DOI: 10.2196/42435] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2022] [Revised: 12/05/2022] [Accepted: 01/10/2023] [Indexed: 03/16/2023] Open
Abstract
BACKGROUND Older patients are at an increased risk of malnutrition due to many factors related to poor clinical outcomes. OBJECTIVE This study aims to develop an assisted diagnosis model using machine learning (ML) for identifying older patients with malnutrition and providing the focus of individualized treatment. METHODS We reanalyzed a multicenter, observational cohort study including 2660 older patients. Baseline malnutrition was defined using the global leadership initiative on malnutrition (GLIM) criteria, and the study population was randomly divided into a derivation group (2128/2660, 80%) and a validation group (532/2660, 20%). We applied 5 ML algorithms and further explored the relationship between features and the risk of malnutrition by using the Shapley additive explanations visualization method. RESULTS The proposed ML models were capable to identify older patients with malnutrition. In the external validation cohort, the top 3 models by the area under the receiver operating characteristic curve were light gradient boosting machine (92.1%), extreme gradient boosting (91.9%), and the random forest model (91.5%). Additionally, the analysis of the importance of features revealed that BMI, weight loss, and calf circumference were the strongest predictors to affect GLIM. A BMI of below 21 kg/m2 was associated with a higher risk of GLIM in older people. CONCLUSIONS We developed ML models for assisting diagnosis of malnutrition based on the GLIM criteria. The cutoff values of laboratory tests generated by Shapley additive explanations could provide references for the identification of malnutrition. TRIAL REGISTRATION Chinese Clinical Trial Registry ChiCTR-EPC-14005253; https://www.chictr.org.cn/showproj.aspx?proj=9542.
Collapse
Affiliation(s)
- Xue Wang
- Department of Clinical Nutrition, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Fengchun Yang
- Institute of Medical Information, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Mingwei Zhu
- Department of General Surgery, Beijing Hospital, National Center of Gerontology, Beijing, China.,Institute of Geriatric Medicine, Chinese Academy of Medical Sciences, Beijing, China.,Department of Gastrointestinal Surgery, Beijing Hospital, National Center of Gerontology, Beijing, China
| | - Hongyuan Cui
- Department of General Surgery, Beijing Hospital, National Center of Gerontology, Beijing, China.,Institute of Geriatric Medicine, Chinese Academy of Medical Sciences, Beijing, China.,Department of Gastrointestinal Surgery, Beijing Hospital, National Center of Gerontology, Beijing, China
| | - Junmin Wei
- Department of General Surgery, Beijing Hospital, National Center of Gerontology, Beijing, China.,Institute of Geriatric Medicine, Chinese Academy of Medical Sciences, Beijing, China.,Department of Gastrointestinal Surgery, Beijing Hospital, National Center of Gerontology, Beijing, China
| | - Jiao Li
- Institute of Medical Information, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Wei Chen
- Department of Clinical Nutrition, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
9
|
Huang J, Li Z, Zhang W, Lv Z, Dong S, Feng Y, Liu R, Zhao Y. Explainable machine learning-assisted origin identification: Chemical profiling of five lotus (Nelumbo nucifera Gaertn.) parts. Food Chem 2023; 404:134517. [DOI: 10.1016/j.foodchem.2022.134517] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2022] [Revised: 09/24/2022] [Accepted: 10/02/2022] [Indexed: 11/22/2022]
|
10
|
Huang AA, Huang SY. Increasing transparency in machine learning through bootstrap simulation and shapely additive explanations. PLoS One 2023; 18:e0281922. [PMID: 36821544 PMCID: PMC9949629 DOI: 10.1371/journal.pone.0281922] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2022] [Accepted: 02/05/2023] [Indexed: 02/24/2023] Open
Abstract
Machine learning methods are widely used within the medical field. However, the reliability and efficacy of these models is difficult to assess, making it difficult for researchers to identify which machine-learning model to apply to their dataset. We assessed whether variance calculations of model metrics (e.g., AUROC, Sensitivity, Specificity) through bootstrap simulation and SHapely Additive exPlanations (SHAP) could increase model transparency and improve model selection. Data from the England National Health Services Heart Disease Prediction Cohort was used. After comparison of model metrics for XGBoost, Random Forest, Artificial Neural Network, and Adaptive Boosting, XGBoost was used as the machine-learning model of choice in this study. Boost-strap simulation (N = 10,000) was used to empirically derive the distribution of model metrics and covariate Gain statistics. SHapely Additive exPlanations (SHAP) to provide explanations to machine-learning output and simulation to evaluate the variance of model accuracy metrics. For the XGBoost modeling method, we observed (through 10,000 completed simulations) that the AUROC ranged from 0.771 to 0.947, a difference of 0.176, the balanced accuracy ranged from 0.688 to 0.894, a 0.205 difference, the sensitivity ranged from 0.632 to 0.939, a 0.307 difference, and the specificity ranged from 0.595 to 0.944, a 0.394 difference. Among 10,000 simulations completed, we observed that the gain for Angina ranged from 0.225 to 0.456, a difference of 0.231, for Cholesterol ranged from 0.148 to 0.326, a difference of 0.178, for maximum heart rate (MaxHR) ranged from 0.081 to 0.200, a range of 0.119, and for Age ranged from 0.059 to 0.157, difference of 0.098. Use of simulations to empirically evaluate the variability of model metrics and explanatory algorithms to observe if covariates match the literature are necessary for increased transparency, reliability, and utility of machine learning methods. These variance statistics, combined with model accuracy statistics can help researchers identify the best model for a given dataset.
Collapse
Affiliation(s)
- Alexander A. Huang
- Department of Statistics and Data Science, Cornell University, Ithaca, New York, United States of America
- Department of MD Education, Northwestern University Feinberg School of Medicine, Chicago, Illinois, United States of America
| | - Samuel Y. Huang
- Department of Statistics and Data Science, Cornell University, Ithaca, New York, United States of America
- Department of Internal Medicine, Virginia Commonwealth University School of Medicine, Richmond, Virginia, United States of America
| |
Collapse
|
11
|
Sethi Y, Patel N, Kaka N, Desai A, Kaiwan O, Sheth M, Sharma R, Huang H, Chopra H, Khandaker MU, Lashin MMA, Hamd ZY, Emran TB. Artificial Intelligence in Pediatric Cardiology: A Scoping Review. J Clin Med 2022; 11:jcm11237072. [PMID: 36498651 PMCID: PMC9738645 DOI: 10.3390/jcm11237072] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2022] [Revised: 11/22/2022] [Accepted: 11/26/2022] [Indexed: 12/05/2022] Open
Abstract
The evolution of AI and data science has aided in mechanizing several aspects of medical care requiring critical thinking: diagnosis, risk stratification, and management, thus mitigating the burden of physicians and reducing the likelihood of human error. AI modalities have expanded feet to the specialty of pediatric cardiology as well. We conducted a scoping review searching the Scopus, Embase, and PubMed databases covering the recent literature between 2002-2022. We found that the use of neural networks and machine learning has significantly improved the diagnostic value of cardiac magnetic resonance imaging, echocardiograms, computer tomography scans, and electrocardiographs, thus augmenting the clinicians' diagnostic accuracy of pediatric heart diseases. The use of AI-based prediction algorithms in pediatric cardiac surgeries improves postoperative outcomes and prognosis to a great extent. Risk stratification and the prediction of treatment outcomes are feasible using the key clinical findings of each CHD with appropriate computational algorithms. Notably, AI can revolutionize prenatal prediction as well as the diagnosis of CHD using the EMR (electronic medical records) data on maternal risk factors. The use of AI in the diagnostics, risk stratification, and management of CHD in the near future is a promising possibility with current advancements in machine learning and neural networks. However, the challenges posed by the dearth of appropriate algorithms and their nascent nature, limited physician training, fear of over-mechanization, and apprehension of missing the 'human touch' limit the acceptability. Still, AI proposes to aid the clinician tomorrow with precision cardiology, paving a way for extremely efficient human-error-free health care.
Collapse
Affiliation(s)
- Yashendra Sethi
- PearResearch, Dehradun 248001, India
- Department of Medicine, Government Doon Medical College, Dehradun 248001, India
| | - Neil Patel
- PearResearch, Dehradun 248001, India
- Department of Medicine, GMERS Medical College, Himmatnagar 383001, India
| | - Nirja Kaka
- PearResearch, Dehradun 248001, India
- Department of Medicine, GMERS Medical College, Himmatnagar 383001, India
| | - Ami Desai
- Department of Medicine, SMIMER Medical College, Surat 395010, India
| | - Oroshay Kaiwan
- PearResearch, Dehradun 248001, India
- Department of Medicine, Northeast Ohio Medical University, Rootstown, OH 44272, USA
- Correspondence: (O.K.); (Z.Y.H.); (T.B.E.)
| | - Mili Sheth
- Department of Medicine, GMERS Gandhinagar, Gandhinagar 382012, India
| | - Rupal Sharma
- Department of Medicine, Government Medical College, Nagpur 440003, India
| | - Helen Huang
- Faculty of Medicine and Health Science, Royal College of Surgeons in Ireland, D02 YN77 Dublin, Ireland
| | - Hitesh Chopra
- Chitkara College of Pharmacy, Chitkara University, Rajpura 140401, India
| | - Mayeen Uddin Khandaker
- Centre for Applied Physics and Radiation Technologies, School of Engineering and Technology, Sunway University, Bandar Sunway 47500, Malaysia
| | - Maha M. A. Lashin
- Department of Biomedical Engineering, College of Engineering, Princess Nourah bint Abdulrahman University, P.O. 84428, Riyadh 11671, Saudi Arabia
| | - Zuhal Y. Hamd
- Department of Radiological Sciences, College of Health and Rehabilitation Sciences, Princess Nourah bint Abdulrahman University, P.O. 84428, Riyadh 11671, Saudi Arabia
- Correspondence: (O.K.); (Z.Y.H.); (T.B.E.)
| | - Talha Bin Emran
- Department of Pharmacy, BGC Trust University Bangladesh, Chittagong 4381, Bangladesh
- Department of Pharmacy, Faculty of Allied Health Sciences, Daffodil International University, Dhaka 1207, Bangladesh
- Correspondence: (O.K.); (Z.Y.H.); (T.B.E.)
| |
Collapse
|
12
|
Loh HW, Ooi CP, Seoni S, Barua PD, Molinari F, Acharya UR. Application of explainable artificial intelligence for healthcare: A systematic review of the last decade (2011-2022). COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 226:107161. [PMID: 36228495 DOI: 10.1016/j.cmpb.2022.107161] [Citation(s) in RCA: 88] [Impact Index Per Article: 44.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Revised: 09/16/2022] [Accepted: 09/25/2022] [Indexed: 06/16/2023]
Abstract
BACKGROUND AND OBJECTIVES Artificial intelligence (AI) has branched out to various applications in healthcare, such as health services management, predictive medicine, clinical decision-making, and patient data and diagnostics. Although AI models have achieved human-like performance, their use is still limited because they are seen as a black box. This lack of trust remains the main reason for their low use in practice, especially in healthcare. Hence, explainable artificial intelligence (XAI) has been introduced as a technique that can provide confidence in the model's prediction by explaining how the prediction is derived, thereby encouraging the use of AI systems in healthcare. The primary goal of this review is to provide areas of healthcare that require more attention from the XAI research community. METHODS Multiple journal databases were thoroughly searched using PRISMA guidelines 2020. Studies that do not appear in Q1 journals, which are highly credible, were excluded. RESULTS In this review, we surveyed 99 Q1 articles covering the following XAI techniques: SHAP, LIME, GradCAM, LRP, Fuzzy classifier, EBM, CBR, rule-based systems, and others. CONCLUSION We discovered that detecting abnormalities in 1D biosignals and identifying key text in clinical notes are areas that require more attention from the XAI research community. We hope this is review will encourage the development of a holistic cloud system for a smart city.
Collapse
Affiliation(s)
- Hui Wen Loh
- School of Science and Technology, Singapore University of Social Sciences, Singapore
| | - Chui Ping Ooi
- School of Science and Technology, Singapore University of Social Sciences, Singapore
| | - Silvia Seoni
- Department of Electronics and Telecommunications, Biolab, Politecnico di Torino, Torino 10129, Italy
| | - Prabal Datta Barua
- Faculty of Engineering and Information Technology, University of Technology Sydney, Australia; School of Business (Information Systems), Faculty of Business, Education, Law & Arts, University of Southern Queensland, Australia
| | - Filippo Molinari
- Department of Electronics and Telecommunications, Biolab, Politecnico di Torino, Torino 10129, Italy
| | - U Rajendra Acharya
- School of Science and Technology, Singapore University of Social Sciences, Singapore; School of Business (Information Systems), Faculty of Business, Education, Law & Arts, University of Southern Queensland, Australia; School of Engineering, Ngee Ann Polytechnic, Singapore; Department of Bioinformatics and Medical Engineering, Asia University, Taiwan; Research Organization for Advanced Science and Technology (IROAST), Kumamoto University, Kumamoto, Japan.
| |
Collapse
|
13
|
Koletzko B, Shamir R. Editorial: Disease-associated malnutrition in paediatric patients must be diagnosed and treated. Curr Opin Clin Nutr Metab Care 2022; 25:186-187. [PMID: 35762172 DOI: 10.1097/mco.0000000000000823] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Affiliation(s)
- Berthold Koletzko
- LMU - Ludwig-Maximilians-Universität Munich, Division of Metabolic and Nutritional Medicine, Department of Pediatrics, Dr von Hauner Children's Hospital, LMU University Hospitals, Munich, Germany
| | - Raanan Shamir
- Institute for Gastroenterology, Nutrition and Liver Diseases, Schneider Children's Medical Center, Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel
| |
Collapse
|