2
|
Champendal M, Müller H, Prior JO, Dos Reis CS. A scoping review of interpretability and explainability concerning artificial intelligence methods in medical imaging. Eur J Radiol 2023; 169:111159. [PMID: 37976760 DOI: 10.1016/j.ejrad.2023.111159] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 09/26/2023] [Accepted: 10/19/2023] [Indexed: 11/19/2023]
Abstract
PURPOSE To review eXplainable Artificial Intelligence/(XAI) methods available for medical imaging/(MI). METHOD A scoping review was conducted following the Joanna Briggs Institute's methodology. The search was performed on Pubmed, Embase, Cinhal, Web of Science, BioRxiv, MedRxiv, and Google Scholar. Studies published in French and English after 2017 were included. Keyword combinations and descriptors related to explainability, and MI modalities were employed. Two independent reviewers screened abstracts, titles and full text, resolving differences through discussion. RESULTS 228 studies met the criteria. XAI publications are increasing, targeting MRI (n = 73), radiography (n = 47), CT (n = 46). Lung (n = 82) and brain (n = 74) pathologies, Covid-19 (n = 48), Alzheimer's disease (n = 25), brain tumors (n = 15) are the main pathologies explained. Explanations are presented visually (n = 186), numerically (n = 67), rule-based (n = 11), textually (n = 11), and example-based (n = 6). Commonly explained tasks include classification (n = 89), prediction (n = 47), diagnosis (n = 39), detection (n = 29), segmentation (n = 13), and image quality improvement (n = 6). The most frequently provided explanations were local (78.1 %), 5.7 % were global, and 16.2 % combined both local and global approaches. Post-hoc approaches were predominantly employed. The used terminology varied, sometimes indistinctively using explainable (n = 207), interpretable (n = 187), understandable (n = 112), transparent (n = 61), reliable (n = 31), and intelligible (n = 3). CONCLUSION The number of XAI publications in medical imaging is increasing, primarily focusing on applying XAI techniques to MRI, CT, and radiography for classifying and predicting lung and brain pathologies. Visual and numerical output formats are predominantly used. Terminology standardisation remains a challenge, as terms like "explainable" and "interpretable" are sometimes being used indistinctively. Future XAI development should consider user needs and perspectives.
Collapse
Affiliation(s)
- Mélanie Champendal
- School of Health Sciences HESAV, HES-SO, University of Applied Sciences Western Switzerland, Lausanne, CH, Switzerland; Faculty of Biology and Medicine, University of Lausanne, Lausanne, CH, Switzerland.
| | - Henning Müller
- Informatics Institute, University of Applied Sciences Western Switzerland (HES-SO Valais) Sierre, CH, Switzerland; Medical faculty, University of Geneva, CH, Switzerland.
| | - John O Prior
- Faculty of Biology and Medicine, University of Lausanne, Lausanne, CH, Switzerland; Nuclear Medicine and Molecular Imaging Department, Lausanne University Hospital (CHUV), Lausanne, CH, Switzerland.
| | - Cláudia Sá Dos Reis
- School of Health Sciences HESAV, HES-SO, University of Applied Sciences Western Switzerland, Lausanne, CH, Switzerland.
| |
Collapse
|
3
|
Prelaj A, Galli EG, Miskovic V, Pesenti M, Viscardi G, Pedica B, Mazzeo L, Bottiglieri A, Provenzano L, Spagnoletti A, Marinacci R, De Toma A, Proto C, Ferrara R, Brambilla M, Occhipinti M, Manglaviti S, Galli G, Signorelli D, Giani C, Beninato T, Pircher CC, Rametta A, Kosta S, Zanitti M, Di Mauro MR, Rinaldi A, Di Gregorio S, Antonia M, Garassino MC, de Braud FGM, Restelli M, Lo Russo G, Ganzinelli M, Trovò F, Pedrocchi ALG. Real-world data to build explainable trustworthy artificial intelligence models for prediction of immunotherapy efficacy in NSCLC patients. Front Oncol 2023; 12:1078822. [PMID: 36755856 PMCID: PMC9899835 DOI: 10.3389/fonc.2022.1078822] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Accepted: 12/14/2022] [Indexed: 01/24/2023] Open
Abstract
Introduction Artificial Intelligence (AI) methods are being increasingly investigated as a means to generate predictive models applicable in the clinical practice. In this study, we developed a model to predict the efficacy of immunotherapy (IO) in patients with advanced non-small cell lung cancer (NSCLC) using eXplainable AI (XAI) Machine Learning (ML) methods. Methods We prospectively collected real-world data from patients with an advanced NSCLC condition receiving immune-checkpoint inhibitors (ICIs) either as a single agent or in combination with chemotherapy. With regards to six different outcomes - Disease Control Rate (DCR), Objective Response Rate (ORR), 6 and 24-month Overall Survival (OS6 and OS24), 3-months Progression-Free Survival (PFS3) and Time to Treatment Failure (TTF3) - we evaluated five different classification ML models: CatBoost (CB), Logistic Regression (LR), Neural Network (NN), Random Forest (RF) and Support Vector Machine (SVM). We used the Shapley Additive Explanation (SHAP) values to explain model predictions. Results Of 480 patients included in the study 407 received immunotherapy and 73 chemo- and immunotherapy. From all the ML models, CB performed the best for OS6 and TTF3, (accuracy 0.83 and 0.81, respectively). CB and LR reached accuracy of 0.75 and 0.73 for the outcome DCR. SHAP for CB demonstrated that the feature that strongly influences models' prediction for all three outcomes was Neutrophil to Lymphocyte Ratio (NLR). Performance Status (ECOG-PS) was an important feature for the outcomes OS6 and TTF3, while PD-L1, Line of IO and chemo-immunotherapy appeared to be more important in predicting DCR. Conclusions In this study we developed a ML algorithm based on real-world data, explained by SHAP techniques, and able to accurately predict the efficacy of immunotherapy in sets of NSCLC patients.
Collapse
Affiliation(s)
- Arsela Prelaj
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy,Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy,*Correspondence: Arsela Prelaj,
| | - Edoardo Gregorio Galli
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy,Niguarda Cancer Center, Grande Ospedale Metropolitano Niguarda, Milan, Italy,Oncology Department, University of Milan, Milan, Italy
| | - Vanja Miskovic
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy
| | - Mattia Pesenti
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy
| | - Giuseppe Viscardi
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy,Medical Oncology Unit, Department of Precision Medicine, University of Campania “Luigi Vanvitelli”, Naples, Italy
| | - Benedetta Pedica
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy
| | - Laura Mazzeo
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy,Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy,Oncology Department, University of Milan, Milan, Italy
| | - Achille Bottiglieri
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy,Oncology Department, University of Milan, Milan, Italy
| | - Leonardo Provenzano
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy,Oncology Department, University of Milan, Milan, Italy
| | - Andrea Spagnoletti
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy,Oncology Department, University of Milan, Milan, Italy
| | - Roberto Marinacci
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy
| | - Alessandro De Toma
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy
| | - Claudia Proto
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy
| | - Roberto Ferrara
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy
| | - Marta Brambilla
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy
| | - Mario Occhipinti
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy
| | - Sara Manglaviti
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy
| | - Giulia Galli
- Medical Oncology Unit, Policlinico San Matteo Fondazione IRCCS, Pavia, Italy
| | - Diego Signorelli
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy,Niguarda Cancer Center, Grande Ospedale Metropolitano Niguarda, Milan, Italy
| | - Claudia Giani
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy,Oncology Department, University of Milan, Milan, Italy
| | - Teresa Beninato
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy,Oncology Department, University of Milan, Milan, Italy
| | - Chiara Carlotta Pircher
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy,Oncology Department, University of Milan, Milan, Italy
| | - Alessandro Rametta
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy,Oncology Department, University of Milan, Milan, Italy
| | - Sokol Kosta
- Department of Electronic System, Aalborg University, Copenhagen, Aalborg, Denmark
| | - Michele Zanitti
- Department of Electronic System, Aalborg University, Copenhagen, Aalborg, Denmark
| | - Maria Rosa Di Mauro
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy
| | - Arturo Rinaldi
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy
| | - Settimio Di Gregorio
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy
| | - Martinetti Antonia
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy
| | - Marina Chiara Garassino
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy,Thoracic Oncology Program, Section of Hematology/Oncology, University of Chicago, Chicago, IL, United States
| | - Filippo G. M. de Braud
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy,Oncology Department, University of Milan, Milan, Italy
| | - Marcello Restelli
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy
| | - Giuseppe Lo Russo
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy
| | - Monica Ganzinelli
- Medical Oncology Department, Fondazione IRCCS Istituto Nazionale Tumori, Milan, Italy
| | - Francesco Trovò
- Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy
| | | |
Collapse
|
5
|
Zhang Y, Weng Y, Lund J. Applications of Explainable Artificial Intelligence in Diagnosis and Surgery. Diagnostics (Basel) 2022; 12:diagnostics12020237. [PMID: 35204328 PMCID: PMC8870992 DOI: 10.3390/diagnostics12020237] [Citation(s) in RCA: 48] [Impact Index Per Article: 24.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2021] [Revised: 01/11/2022] [Accepted: 01/13/2022] [Indexed: 02/06/2023] Open
Abstract
In recent years, artificial intelligence (AI) has shown great promise in medicine. However, explainability issues make AI applications in clinical usages difficult. Some research has been conducted into explainable artificial intelligence (XAI) to overcome the limitation of the black-box nature of AI methods. Compared with AI techniques such as deep learning, XAI can provide both decision-making and explanations of the model. In this review, we conducted a survey of the recent trends in medical diagnosis and surgical applications using XAI. We have searched articles published between 2019 and 2021 from PubMed, IEEE Xplore, Association for Computing Machinery, and Google Scholar. We included articles which met the selection criteria in the review and then extracted and analyzed relevant information from the studies. Additionally, we provide an experimental showcase on breast cancer diagnosis, and illustrate how XAI can be applied in medical XAI applications. Finally, we summarize the XAI methods utilized in the medical XAI applications, the challenges that the researchers have met, and discuss the future research directions. The survey result indicates that medical XAI is a promising research direction, and this study aims to serve as a reference to medical experts and AI scientists when designing medical XAI applications.
Collapse
Affiliation(s)
- Yiming Zhang
- School of Computer Science, Faculty of Science and Engineering, University of Nottingham Ningbo China, Ningbo 315100, China;
- School of Medicine, University of Nottingham, Nottingham NG7 2RD, UK;
| | - Ying Weng
- School of Computer Science, Faculty of Science and Engineering, University of Nottingham Ningbo China, Ningbo 315100, China;
- Correspondence:
| | - Jonathan Lund
- School of Medicine, University of Nottingham, Nottingham NG7 2RD, UK;
| |
Collapse
|
8
|
Zhan X, Long H, Gou F, Duan X, Kong G, Wu J. A Convolutional Neural Network-Based Intelligent Medical System with Sensors for Assistive Diagnosis and Decision-Making in Non-Small Cell Lung Cancer. SENSORS 2021; 21:s21237996. [PMID: 34884000 PMCID: PMC8659811 DOI: 10.3390/s21237996] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Revised: 11/26/2021] [Accepted: 11/28/2021] [Indexed: 12/15/2022]
Abstract
In many regions of the world, early diagnosis of non-small cell lung cancer (NSCLC) is a major challenge due to the large population and lack of medical resources, which is difficult toeffectively address via limited physician manpower alone. Therefore, we developed a convolutional neural network (CNN)-based assisted diagnosis and decision-making intelligent medical system with sensors. This system analyzes NSCLC patients' medical records using sensors to assist staging a diagnosis and provides recommended treatment plans to physicians. To address the problem of unbalanced case samples across pathological stages, we used transfer learning and dynamic sampling techniques to reconstruct and iteratively train the model to improve the accuracy of the prediction system. In this paper, all data for training and testing the system were obtained from the medical records of 2,789,675 patients with NSCLC, which were recorded in three hospitals in China over a five-year period. When the number of case samples reached 8000, the system achieved an accuracy rate of 0.84, which is already close to that of the doctors (accuracy: 0.86). The experimental results proved that the system can quickly and accurately analyze patient data and provide decision information support for physicians.
Collapse
Affiliation(s)
- Xiangbing Zhan
- State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang 550025, China; (X.Z.); (X.D.); (G.K.)
| | - Huiyun Long
- State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang 550025, China; (X.Z.); (X.D.); (G.K.)
- Correspondence: (H.L.); (J.W.)
| | - Fangfang Gou
- School of Computer Science and Engineering, Central South University, Changsha 410083, China;
| | - Xun Duan
- State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang 550025, China; (X.Z.); (X.D.); (G.K.)
| | - Guangqian Kong
- State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang 550025, China; (X.Z.); (X.D.); (G.K.)
| | - Jia Wu
- School of Computer Science and Engineering, Central South University, Changsha 410083, China;
- Research Center for Artificial Intelligence, Monash University, Clayton, VIC 3800, Australia
- Correspondence: (H.L.); (J.W.)
| |
Collapse
|