1
|
Lamba K, Rani S. A novel approach of brain-computer interfacing (BCI) and Grad-CAM based explainable artificial intelligence: Use case scenario for smart healthcare. J Neurosci Methods 2024; 408:110159. [PMID: 38723868 DOI: 10.1016/j.jneumeth.2024.110159] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2024] [Revised: 04/02/2024] [Accepted: 04/29/2024] [Indexed: 05/19/2024]
Abstract
BACKGROUND In order to push the frontiers of brain-computer interface (BCI) and neuron-electronics, this research presents a novel framework that combines cutting-edge technologies for improved brain-related diagnostics in smart healthcare. This research offers a ground-breaking application of transparent strategies to BCI, promoting openness and confidence in brain-computer interactions and taking inspiration from Grad-CAM (Gradient-weighted Class Activation Mapping) based Explainable Artificial Intelligence (XAI) methodology. The landscape of healthcare diagnostics is about to be redefined by the integration of various technologies, especially when it comes to illnesses related to the brain. NEW METHOD A novel approach has been proposed in this study comprising of Xception architecture which is trained on imagenet database following transfer learning process for extraction of significant features from magnetic resonance imaging dataset acquired from publicly available distinct sources as an input and linear support vector machine has been used for distinguishing distinct classes.Afterwards, gradient-weighted class activation mapping has been deployed as the foundation for explainable artificial intelligence (XAI) for generating informative heatmaps, representing spatial localization of features which were focused to achieve model's predictions. RESULTS Thus, the proposed model not only provides accurate outcomes but also provides transparency for the predictions generated by the Xception network to diagnose presence of abnormal tissues and avoids overfitting issues. Hyperparameters along with performance-metrics are also obtained while validating the proposed network on unseen brain MRI scans to ensure effectiveness of the proposed network. COMPARISON WITH EXISTING METHODS AND CONCLUSIONS The integration of Grad-CAM based explainable artificial intelligence with deep neural network namely Xception offers a significant impact in diagnosing brain tumor disease while highlighting the specific regions of input brain MRI images responsible for making predictions. In this study, the proposed network results in 98.92% accuracy, 98.15% precision, 99.09% sensitivity, 98.18% specificity and 98.91% dice-coefficient while identifying presence of abnormal tissues in the brain. Thus, Xception model trained on distinct dataset following transfer learning process offers remarkable diagnostic accuracy and linear support vector act as a classifier to provide efficient classification among distinct classes. In addition, the deployed explainable artificial intelligence approach helps in revealing the reasoning behind predictions made by deep neural network having black-box nature and provides a clear perspective to assist medical experts in achieving trustworthiness and transparency while diagnosing brain tumor disease in the smart healthcare.
Collapse
Affiliation(s)
- Kamini Lamba
- Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura, Punjab 140401, India.
| | - Shalli Rani
- Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura, Punjab 140401, India.
| |
Collapse
|
2
|
T R M, Gupta M, T A A, Kumar V V, Geman O, Kumar V D. An XAI-Enhanced EfficientNetB0 Framework for Precision Brain Tumor Detection in MRI Imaging. J Neurosci Methods 2024:110227. [PMID: 39038716 DOI: 10.1016/j.jneumeth.2024.110227] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2024] [Revised: 06/25/2024] [Accepted: 07/19/2024] [Indexed: 07/24/2024]
Abstract
BACKGROUND Accurately diagnosing brain tumors from MRI scans is crucial for effective treatment planning. While traditional methods heavily rely on radiologist expertise, the integration of AI, particularly Convolutional Neural Networks (CNNs), has shown promise in improving accuracy. However, the lack of transparency in AI decision-making processes presents a challenge for clinical adoption. METHODS Recent advancements in deep learning, particularly the utilization of CNNs, have facilitated the development of models for medical image analysis. In this study, we employed the EfficientNetB0 architecture and integrated explainable AI techniques to enhance both accuracy and interpretability. Grad-CAM visualization was utilized to highlight significant areas in MRI scans influencing classification decisions. RESULTS Our model achieved a classification accuracy of 98.72% across four categories of brain tumors (Glioma, Meningioma, No Tumor, Pituitary), with precision and recall exceeding 97% for all categories. The incorporation of explainable AI techniques was validated through visual inspection of Grad-CAM heatmaps, which aligned well with established diagnostic markers in MRI scans. CONCLUSION The AI-enhanced EfficientNetB0 framework with explainable AI techniques significantly improves brain tumor classification accuracy to 98.72%, offering clear visual insights into the decision-making process. This method enhances diagnostic reliability and trust, demonstrating substantial potential for clinical adoption in medical diagnostics.
Collapse
Affiliation(s)
- Mahesh T R
- Department of Computer Science and Engineering, Faculty of Engineering and Technology, JAIN (Deemed-to-be University), Bangalore, 562112, India.
| | - Muskan Gupta
- Department of Computer Science and Engineering, Faculty of Engineering and Technology, JAIN (Deemed-to-be University), Bangalore, 562112, India; Department of Computer Science and Engineering, Siddaganga Institute of Technology, Tumakuru - 572103, India; School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, India; Stefan Cel Mare University of Suceava, Suceava, Romania; Vel Tech Rangarajan Dr.Sagunthala R & D Instiute of Science and Technology, Chennai, India.
| | - Anupama T A
- Department of Computer Science and Engineering, Siddaganga Institute of Technology, Tumakuru - 572103, India.
| | - Vinoth Kumar V
- School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, India.
| | - Oana Geman
- Stefan Cel Mare University of Suceava, Suceava, Romania.
| | - Dhilip Kumar V
- Vel Tech Rangarajan Dr.Sagunthala R & D Instiute of Science and Technology, Chennai, India.
| |
Collapse
|
3
|
Appiah R, Pulletikurthi V, Esquivel-Puentes HA, Cabrera C, Hasan NI, Dharmarathne S, Gomez LJ, Castillo L. Brain tumor detection using proper orthogonal decomposition integrated with deep learning networks. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 250:108167. [PMID: 38669717 DOI: 10.1016/j.cmpb.2024.108167] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/09/2023] [Revised: 03/11/2024] [Accepted: 04/06/2024] [Indexed: 04/28/2024]
Abstract
BACKGROUND AND OBJECTIVE The central organ of the human nervous system is the brain, which receives and sends stimuli to the various parts of the body to engage in daily activities. Uncontrolled growth of brain cells can result in tumors which affect the normal functions of healthy brain cells. An automatic reliable technique for detecting tumors is imperative to assist medical practitioners in the timely diagnosis of patients. Although machine learning models are being used, with minimal data availability to train, development of low-order based models integrated with machine learning are a tool for reliable detection. METHODS In this study, we focus on comparing a low-order model such as proper orthogonal decomposition (POD) coupled with convolutional neural network (CNN) on 2D images from magnetic resonance imaging (MRI) scans to effectively identify brain tumors. The explainability of the coupled POD-CNN prediction output as well as the state-of-the-art pre-trained transfer learning models such as MobileNetV2, Inception-v3, ResNet101, and VGG-19 were explored. RESULTS The results showed that CNN predicted tumors with an accuracy of 99.21% whereas POD-CNN performed better with about 1/3rd of computational time at an accuracy of 95.88%. Explainable AI with SHAP showed MobileNetV2 has better prediction in identifying the tumor boundaries. CONCLUSIONS Integration of POD with CNN is carried for the first time to detect brain tumor detection with minimal MRI scan data. This study facilitates low-model approaches in machine learning to improve the accuracy and performance of tumor detection.
Collapse
Affiliation(s)
- Rita Appiah
- School of Nuclear Engineering, Purdue University, West Lafayette, IN 47906, USA.
| | | | | | - Cristiano Cabrera
- School of Mechanical Engineering, Purdue University, West Lafayette, IN 47906, USA
| | - Nahian I Hasan
- School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47906, USA
| | - Suranga Dharmarathne
- R.B. Annis School of Engineering, University of Indianapolis, Indianapolis, IN 46227, USA
| | - Luis J Gomez
- School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47906, USA
| | - Luciano Castillo
- School of Mechanical Engineering, Purdue University, West Lafayette, IN 47906, USA
| |
Collapse
|
4
|
Alshuhail A, Thakur A, Chandramma R, Mahesh TR, Almusharraf A, Vinoth Kumar V, Khan SB. Refining neural network algorithms for accurate brain tumor classification in MRI imagery. BMC Med Imaging 2024; 24:118. [PMID: 38773391 PMCID: PMC11110259 DOI: 10.1186/s12880-024-01285-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2024] [Accepted: 04/29/2024] [Indexed: 05/23/2024] Open
Abstract
Brain tumor diagnosis using MRI scans poses significant challenges due to the complex nature of tumor appearances and variations. Traditional methods often require extensive manual intervention and are prone to human error, leading to misdiagnosis and delayed treatment. Current approaches primarily include manual examination by radiologists and conventional machine learning techniques. These methods rely heavily on feature extraction and classification algorithms, which may not capture the intricate patterns present in brain MRI images. Conventional techniques often suffer from limited accuracy and generalizability, mainly due to the high variability in tumor appearance and the subjective nature of manual interpretation. Additionally, traditional machine learning models may struggle with the high-dimensional data inherent in MRI images. To address these limitations, our research introduces a deep learning-based model utilizing convolutional neural networks (CNNs).Our model employs a sequential CNN architecture with multiple convolutional, max-pooling, and dropout layers, followed by dense layers for classification. The proposed model demonstrates a significant improvement in diagnostic accuracy, achieving an overall accuracy of 98% on the test dataset. The proposed model demonstrates a significant improvement in diagnostic accuracy, achieving an overall accuracy of 98% on the test dataset. The precision, recall, and F1-scores ranging from 97 to 98% with a roc-auc ranging from 99 to 100% for each tumor category further substantiate the model's effectiveness. Additionally, the utilization of Grad-CAM visualizations provides insights into the model's decision-making process, enhancing interpretability. This research addresses the pressing need for enhanced diagnostic accuracy in identifying brain tumors through MRI imaging, tackling challenges such as variability in tumor appearance and the need for rapid, reliable diagnostic tools.
Collapse
Affiliation(s)
- Asma Alshuhail
- Department of Information Systems, College of Computer Sciences and Information Technology, King Faisal University, Hofuf, Saudi Arabia
| | - Arastu Thakur
- Department of Computer Science and Engineering, Faculty of Engineering and Technology, JAIN (Deemed-to-be University), Bangalore, 562112, India
| | - R Chandramma
- Department of Computer Science & Engineering (AI & ML), Global Academy of Technology, Bangalore, India
| | - T R Mahesh
- Department of Computer Science and Engineering, Faculty of Engineering and Technology, JAIN (Deemed-to-be University), Bangalore, 562112, India
| | - Ahlam Almusharraf
- Department of Management, College of Business Administration, Princess Nourah Bint Abdulrahman University, P.O. Box 84428, Riyadh, 11671, Saudi Arabia.
| | - V Vinoth Kumar
- School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, 632001, India
| | - Surbhi Bhatia Khan
- School of Science, Engineering and Environment, University of Salford, Manchester, UK
- Department of Electrical and Computer Engineering, Lebanese American University, Byblos, Lebanon
| |
Collapse
|
5
|
Albalawi E, T R M, Thakur A, Kumar VV, Gupta M, Khan SB, Almusharraf A. Integrated approach of federated learning with transfer learning for classification and diagnosis of brain tumor. BMC Med Imaging 2024; 24:110. [PMID: 38750436 PMCID: PMC11097560 DOI: 10.1186/s12880-024-01261-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2023] [Accepted: 03/27/2024] [Indexed: 05/18/2024] Open
Abstract
Brain tumor classification using MRI images is a crucial yet challenging task in medical imaging. Accurate diagnosis is vital for effective treatment planning but is often hindered by the complex nature of tumor morphology and variations in imaging. Traditional methodologies primarily rely on manual interpretation of MRI images, supplemented by conventional machine learning techniques. These approaches often lack the robustness and scalability needed for precise and automated tumor classification. The major limitations include a high degree of manual intervention, potential for human error, limited ability to handle large datasets, and lack of generalizability to diverse tumor types and imaging conditions.To address these challenges, we propose a federated learning-based deep learning model that leverages the power of Convolutional Neural Networks (CNN) for automated and accurate brain tumor classification. This innovative approach not only emphasizes the use of a modified VGG16 architecture optimized for brain MRI images but also highlights the significance of federated learning and transfer learning in the medical imaging domain. Federated learning enables decentralized model training across multiple clients without compromising data privacy, addressing the critical need for confidentiality in medical data handling. This model architecture benefits from the transfer learning technique by utilizing a pre-trained CNN, which significantly enhances its ability to classify brain tumors accurately by leveraging knowledge gained from vast and diverse datasets.Our model is trained on a diverse dataset combining figshare, SARTAJ, and Br35H datasets, employing a federated learning approach for decentralized, privacy-preserving model training. The adoption of transfer learning further bolsters the model's performance, making it adept at handling the intricate variations in MRI images associated with different types of brain tumors. The model demonstrates high precision (0.99 for glioma, 0.95 for meningioma, 1.00 for no tumor, and 0.98 for pituitary), recall, and F1-scores in classification, outperforming existing methods. The overall accuracy stands at 98%, showcasing the model's efficacy in classifying various tumor types accurately, thus highlighting the transformative potential of federated learning and transfer learning in enhancing brain tumor classification using MRI images.
Collapse
Affiliation(s)
- Eid Albalawi
- Department of Computer science, College of Computer Science and Information Technology, King faisal University, 31982, Hofuf, Saudi Arabia
| | - Mahesh T R
- Department of Computer Science and Engineering, Faculty of Engineering and Technology, JAIN (Deemed-to-be University), 562112, Bangalore, India
| | - Arastu Thakur
- Department of Computer Science and Engineering, Faculty of Engineering and Technology, JAIN (Deemed-to-be University), 562112, Bangalore, India
| | - V Vinoth Kumar
- School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, 632014, Vellore, India
| | - Muskan Gupta
- Department of Computer Science and Engineering, Faculty of Engineering and Technology, JAIN (Deemed-to-be University), 562112, Bangalore, India
| | - Surbhi Bhatia Khan
- School of Science, Engineering and environment, University of Salford, M5 4WT, Manchester, UK.
- , Department of Electrical and Computer Engineering, Lebanese American University, Byblos, Lebanon, Lebanon.
| | - Ahlam Almusharraf
- Department of Business Administration, College of Business and Administration, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh, 11671, Riyadh, Saudi Arabia
| |
Collapse
|
6
|
Xie Y, Zaccagna F, Rundo L, Testa C, Zhu R, Tonon C, Lodi R, Manners DN. IMPA-Net: Interpretable Multi-Part Attention Network for Trustworthy Brain Tumor Classification from MRI. Diagnostics (Basel) 2024; 14:997. [PMID: 38786294 PMCID: PMC11119919 DOI: 10.3390/diagnostics14100997] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2024] [Revised: 05/08/2024] [Accepted: 05/09/2024] [Indexed: 05/25/2024] Open
Abstract
Deep learning (DL) networks have shown attractive performance in medical image processing tasks such as brain tumor classification. However, they are often criticized as mysterious "black boxes". The opaqueness of the model and the reasoning process make it difficult for health workers to decide whether to trust the prediction outcomes. In this study, we develop an interpretable multi-part attention network (IMPA-Net) for brain tumor classification to enhance the interpretability and trustworthiness of classification outcomes. The proposed model not only predicts the tumor grade but also provides a global explanation for the model interpretability and a local explanation as justification for the proffered prediction. Global explanation is represented as a group of feature patterns that the model learns to distinguish high-grade glioma (HGG) and low-grade glioma (LGG) classes. Local explanation interprets the reasoning process of an individual prediction by calculating the similarity between the prototypical parts of the image and a group of pre-learned task-related features. Experiments conducted on the BraTS2017 dataset demonstrate that IMPA-Net is a verifiable model for the classification task. A percentage of 86% of feature patterns were assessed by two radiologists to be valid for representing task-relevant medical features. The model shows a classification accuracy of 92.12%, of which 81.17% were evaluated as trustworthy based on local explanations. Our interpretable model is a trustworthy model that can be used for decision aids for glioma classification. Compared with black-box CNNs, it allows health workers and patients to understand the reasoning process and trust the prediction outcomes.
Collapse
Affiliation(s)
- Yuting Xie
- Department of Biomedical and Neuromotor Sciences, University of Bologna, 40126 Bologna, Italy; (Y.X.); (C.T.); (R.L.)
- Functional and Molecular Neuroimaging Unit, IRCCS Istituto delle Scienze Neurologiche di Bologna, Bellaria Hospital, 40139 Bologna, Italy
| | - Fulvio Zaccagna
- Department of Imaging, Cambridge University Hospitals NHS Foundation Trust, Cambridge Biomedical Campus, Cambridge CB2 0SL, UK;
- Department of Radiology, University of Cambridge, Cambridge CB2 0QQ, UK
| | - Leonardo Rundo
- Department of Information and Electrical Engineering and Applied Mathematics, University of Salerno, 84084 Fisciano, Italy;
| | - Claudia Testa
- INFN Bologna Division, Viale C. Berti Pichat, 6/2, 40127 Bologna, Italy
- Department of Physics and Astronomy, University of Bologna, 40127 Bologna, Italy
| | - Ruifeng Zhu
- Department of Engineering “Enzo Ferrari”, University of Modena and Reggio Emilia, 41125 Modena, Italy;
| | - Caterina Tonon
- Department of Biomedical and Neuromotor Sciences, University of Bologna, 40126 Bologna, Italy; (Y.X.); (C.T.); (R.L.)
- Functional and Molecular Neuroimaging Unit, IRCCS Istituto delle Scienze Neurologiche di Bologna, Bellaria Hospital, 40139 Bologna, Italy
| | - Raffaele Lodi
- Department of Biomedical and Neuromotor Sciences, University of Bologna, 40126 Bologna, Italy; (Y.X.); (C.T.); (R.L.)
- Functional and Molecular Neuroimaging Unit, IRCCS Istituto delle Scienze Neurologiche di Bologna, Bellaria Hospital, 40139 Bologna, Italy
| | - David Neil Manners
- Functional and Molecular Neuroimaging Unit, IRCCS Istituto delle Scienze Neurologiche di Bologna, Bellaria Hospital, 40139 Bologna, Italy
- Department for Life Quality Studies, University of Bologna, 40126 Bologna, Italy
| |
Collapse
|
7
|
Gaur L, Siarry P, Abraham A, Castillo O. Editorial: Advancements of deep learning in medical imaging for neurodegenerative diseases. Front Neurosci 2024; 18:1361055. [PMID: 38312932 PMCID: PMC10834769 DOI: 10.3389/fnins.2024.1361055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2023] [Accepted: 01/10/2024] [Indexed: 02/06/2024] Open
Affiliation(s)
- Loveleen Gaur
- Department of Computer Science, Taylor's University, Subang Jaya, Malaysia
| | - Patrick Siarry
- Department of Computer Science, Universite Paris 12, Creteil, France
| | - Ajith Abraham
- Department of Computer Science, Machine Intelligence Research Labs, Auburn, AL, United States
| | | |
Collapse
|
8
|
Pitarch C, Ungan G, Julià-Sapé M, Vellido A. Advances in the Use of Deep Learning for the Analysis of Magnetic Resonance Image in Neuro-Oncology. Cancers (Basel) 2024; 16:300. [PMID: 38254790 PMCID: PMC10814384 DOI: 10.3390/cancers16020300] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Revised: 12/28/2023] [Accepted: 01/08/2024] [Indexed: 01/24/2024] Open
Abstract
Machine Learning is entering a phase of maturity, but its medical applications still lag behind in terms of practical use. The field of oncological radiology (and neuro-oncology in particular) is at the forefront of these developments, now boosted by the success of Deep-Learning methods for the analysis of medical images. This paper reviews in detail some of the most recent advances in the use of Deep Learning in this field, from the broader topic of the development of Machine-Learning-based analytical pipelines to specific instantiations of the use of Deep Learning in neuro-oncology; the latter including its use in the groundbreaking field of ultra-low field magnetic resonance imaging.
Collapse
Affiliation(s)
- Carla Pitarch
- Department of Computer Science, Universitat Politècnica de Catalunya (UPC BarcelonaTech) and Intelligent Data Science and Artificial Intelligence (IDEAI-UPC) Research Center, 08034 Barcelona, Spain;
- Eurecat, Digital Health Unit, Technology Centre of Catalonia, 08005 Barcelona, Spain
| | - Gulnur Ungan
- Departament de Bioquímica i Biologia Molecular and Institut de Biotecnologia i Biomedicina (IBB), Universitat Autònoma de Barcelona (UAB), 08193 Barcelona, Spain; (G.U.); (M.J.-S.)
- Centro de Investigación Biomédica en Red (CIBER), 28029 Madrid, Spain
| | - Margarida Julià-Sapé
- Departament de Bioquímica i Biologia Molecular and Institut de Biotecnologia i Biomedicina (IBB), Universitat Autònoma de Barcelona (UAB), 08193 Barcelona, Spain; (G.U.); (M.J.-S.)
- Centro de Investigación Biomédica en Red (CIBER), 28029 Madrid, Spain
| | - Alfredo Vellido
- Department of Computer Science, Universitat Politècnica de Catalunya (UPC BarcelonaTech) and Intelligent Data Science and Artificial Intelligence (IDEAI-UPC) Research Center, 08034 Barcelona, Spain;
- Centro de Investigación Biomédica en Red (CIBER), 28029 Madrid, Spain
| |
Collapse
|
9
|
Champendal M, Müller H, Prior JO, Dos Reis CS. A scoping review of interpretability and explainability concerning artificial intelligence methods in medical imaging. Eur J Radiol 2023; 169:111159. [PMID: 37976760 DOI: 10.1016/j.ejrad.2023.111159] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 09/26/2023] [Accepted: 10/19/2023] [Indexed: 11/19/2023]
Abstract
PURPOSE To review eXplainable Artificial Intelligence/(XAI) methods available for medical imaging/(MI). METHOD A scoping review was conducted following the Joanna Briggs Institute's methodology. The search was performed on Pubmed, Embase, Cinhal, Web of Science, BioRxiv, MedRxiv, and Google Scholar. Studies published in French and English after 2017 were included. Keyword combinations and descriptors related to explainability, and MI modalities were employed. Two independent reviewers screened abstracts, titles and full text, resolving differences through discussion. RESULTS 228 studies met the criteria. XAI publications are increasing, targeting MRI (n = 73), radiography (n = 47), CT (n = 46). Lung (n = 82) and brain (n = 74) pathologies, Covid-19 (n = 48), Alzheimer's disease (n = 25), brain tumors (n = 15) are the main pathologies explained. Explanations are presented visually (n = 186), numerically (n = 67), rule-based (n = 11), textually (n = 11), and example-based (n = 6). Commonly explained tasks include classification (n = 89), prediction (n = 47), diagnosis (n = 39), detection (n = 29), segmentation (n = 13), and image quality improvement (n = 6). The most frequently provided explanations were local (78.1 %), 5.7 % were global, and 16.2 % combined both local and global approaches. Post-hoc approaches were predominantly employed. The used terminology varied, sometimes indistinctively using explainable (n = 207), interpretable (n = 187), understandable (n = 112), transparent (n = 61), reliable (n = 31), and intelligible (n = 3). CONCLUSION The number of XAI publications in medical imaging is increasing, primarily focusing on applying XAI techniques to MRI, CT, and radiography for classifying and predicting lung and brain pathologies. Visual and numerical output formats are predominantly used. Terminology standardisation remains a challenge, as terms like "explainable" and "interpretable" are sometimes being used indistinctively. Future XAI development should consider user needs and perspectives.
Collapse
Affiliation(s)
- Mélanie Champendal
- School of Health Sciences HESAV, HES-SO, University of Applied Sciences Western Switzerland, Lausanne, CH, Switzerland; Faculty of Biology and Medicine, University of Lausanne, Lausanne, CH, Switzerland.
| | - Henning Müller
- Informatics Institute, University of Applied Sciences Western Switzerland (HES-SO Valais) Sierre, CH, Switzerland; Medical faculty, University of Geneva, CH, Switzerland.
| | - John O Prior
- Faculty of Biology and Medicine, University of Lausanne, Lausanne, CH, Switzerland; Nuclear Medicine and Molecular Imaging Department, Lausanne University Hospital (CHUV), Lausanne, CH, Switzerland.
| | - Cláudia Sá Dos Reis
- School of Health Sciences HESAV, HES-SO, University of Applied Sciences Western Switzerland, Lausanne, CH, Switzerland.
| |
Collapse
|
10
|
Syed SR, M. A. SD. A diagnosis model for brain atrophy using deep learning and MRI of type 2 diabetes mellitus. Front Neurosci 2023; 17:1291753. [PMID: 37965222 PMCID: PMC10642919 DOI: 10.3389/fnins.2023.1291753] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2023] [Accepted: 10/09/2023] [Indexed: 11/16/2023] Open
Abstract
Objective Type 2 Diabetes Mellitus (T2DM) is linked to cognitive deterioration and anatomical brain abnormalities like cerebral brain atrophy and cerebral diseases. We aim to develop an automatic deep learning-based brain atrophy diagnosis model to detect, segment, classify, and predict the survival rate. Methods Two hundred thirty-five MRI images affected with brain atrophy due to prolonged T2DM were acquired. The dataset was divided into training and testing (80:20%; 188, 47, respectively). Pre-processing is done through a novel convolutional median filter, followed by segmentation of atrophy regions, i.e., the brain shrinkage, white and gray matter is done through the proposed TRAU-Net model (Transfer Residual Attention U-Net), classification with the proposed Multinomial Logistic regression with Attention Swin Transformer (MLAST), and prediction of chronological age is determined through Multivariate CoX Regression model (MCR). The classification of Brain Atrophy (BA) types is determined based on the features extracted from the segmented region. Performance measures like confusion matrix, specificity, sensitivity, accuracy, F1-score, and ROC-AUC curve are used to measure classification model performance, whereas, for the segmentation model, pixel accuracy and dice similarity coefficient are applied. Results The pixel accuracy and dice coefficient for segmentation were 98.25 and 96.41, respectively. Brain atrophy multi-class classification achieved overall training accuracy is 0.9632 ± 1.325, 0.9677 ± 1.912, 0.9682 ± 1.715, and 0.9521 ± 1.877 for FA, PA, R-MTA, and L-MTA, respectively. The overall AUC-ROC curve for the classification model is 0.9856. The testing and validation accuracy obtained for the proposed model are 0.9379 and 0.9694, respectively. The prediction model's performance is measured using correlation coefficient (r), coefficient determination r2, and Mean Square Error (MSE) and recorded 0.951, 0.904, and 0.5172, respectively. Conclusion The brain atrophy diagnosis model consists of sub-models to detect, segment, and classify the atrophy regions using novel deep learning and multivariate mathematical models. The proposed model has outperformed the existing models regarding multi-classification and segmentation; therefore, the automated diagnosis model can be deployed in healthcare centers to assist physicians.
Collapse
Affiliation(s)
| | - Saleem Durai M. A.
- Vellore Institute of Technology, School of Computer Science and Engineering, Vellore, Tamilnadu, India
| |
Collapse
|
11
|
Azam H, Tariq H, Shehzad D, Akbar S, Shah H, Khan ZA. Fully Automated Skull Stripping from Brain Magnetic Resonance Images Using Mask RCNN-Based Deep Learning Neural Networks. Brain Sci 2023; 13:1255. [PMID: 37759856 PMCID: PMC10526767 DOI: 10.3390/brainsci13091255] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2023] [Revised: 08/09/2023] [Accepted: 08/21/2023] [Indexed: 09/29/2023] Open
Abstract
This research comprises experiments with a deep learning framework for fully automating the skull stripping from brain magnetic resonance (MR) images. Conventional techniques for segmentation have progressed to the extent of Convolutional Neural Networks (CNN). We proposed and experimented with a contemporary variant of the deep learning framework based on mask region convolutional neural network (Mask-RCNN) for all anatomical orientations of brain MR images. We trained the system from scratch to build a model for classification, detection, and segmentation. It is validated by images taken from three different datasets: BrainWeb; NAMIC, and a local hospital. We opted for purposive sampling to select 2000 images of T1 modality from data volumes followed by a multi-stage random sampling technique to segregate the dataset into three batches for training (75%), validation (15%), and testing (10%) respectively. We utilized a robust backbone architecture, namely ResNet-101 and Functional Pyramid Network (FPN), to achieve optimal performance with higher accuracy. We subjected the same data to two traditional methods, namely Brain Extraction Tools (BET) and Brain Surface Extraction (BSE), to compare their performance results. Our proposed method had higher mean average precision (mAP) = 93% and content validity index (CVI) = 0.95%, which were better than comparable methods. We contributed by training Mask-RCNN from scratch for generating reusable learning weights known as transfer learning. We contributed to methodological novelty by applying a pragmatic research lens, and used a mixed method triangulation technique to validate results on all anatomical modalities of brain MR images. Our proposed method improved the accuracy and precision of skull stripping by fully automating it and reducing its processing time and operational cost and reliance on technicians. This research study has also provided grounds for extending the work to the scale of explainable artificial intelligence (XAI).
Collapse
Affiliation(s)
- Humera Azam
- Department of Computer Science, University of Karachi, Karachi 75270, Pakistan
| | - Humera Tariq
- Department of Radiology, University of Michigan, Ann Arbor, MI 48109, USA;
| | - Danish Shehzad
- Department of Computer Science, The Superior University, Lahore 54590, Pakistan
| | - Saad Akbar
- College of Computing and Information Sciences, Karachi Institute of Economics and Technology, Karachi 75190, Pakistan;
| | - Habib Shah
- Department of Computer Science, College of Computer Science, King Khalid University, Abha 61421, Saudi Arabia;
| | - Zamin Ali Khan
- Department of Computer Science, IQRA University, Karachi 71500, Pakistan;
| |
Collapse
|
12
|
McFall GP, Bohn L, Gee M, Drouin SM, Fah H, Han W, Li L, Camicioli R, Dixon RA. Identifying key multi-modal predictors of incipient dementia in Parkinson's disease: a machine learning analysis and Tree SHAP interpretation. Front Aging Neurosci 2023; 15:1124232. [PMID: 37455938 PMCID: PMC10347530 DOI: 10.3389/fnagi.2023.1124232] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2022] [Accepted: 06/13/2023] [Indexed: 07/18/2023] Open
Abstract
Background Persons with Parkinson's disease (PD) differentially progress to cognitive impairment and dementia. With a 3-year longitudinal sample of initially non-demented PD patients measured on multiple dementia risk factors, we demonstrate that machine learning classifier algorithms can be combined with explainable artificial intelligence methods to identify and interpret leading predictors that discriminate those who later converted to dementia from those who did not. Method Participants were 48 well-characterized PD patients (Mbaseline age = 71.6; SD = 4.8; 44% female). We tested 38 multi-modal predictors from 10 domains (e.g., motor, cognitive) in a computationally competitive context to identify those that best discriminated two unobserved baseline groups, PD No Dementia (PDND), and PD Incipient Dementia (PDID). We used Random Forest (RF) classifier models for the discrimination goal and Tree SHapley Additive exPlanation (Tree SHAP) values for deep interpretation. Results An excellent RF model discriminated baseline PDID from PDND (AUC = 0.84; normalized Matthews Correlation Coefficient = 0.76). Tree SHAP showed that ten leading predictors of PDID accounted for 62.5% of the model, as well as their relative importance, direction, and magnitude (risk threshold). These predictors represented the motor (e.g., poorer gait), cognitive (e.g., slower Trail A), molecular (up-regulated metabolite panel), demographic (age), imaging (ventricular volume), and lifestyle (activities of daily living) domains. Conclusion Our data-driven protocol integrated RF classifier models and Tree SHAP applications to selectively identify and interpret early dementia risk factors in a well-characterized sample of initially non-demented persons with PD. Results indicate that leading dementia predictors derive from multiple complementary risk domains.
Collapse
Affiliation(s)
- G. Peggy McFall
- Department of Psychology, University of Alberta, Edmonton, AB, Canada
- Neuroscience and Mental Health Institute, University of Alberta, Edmonton, AB, Canada
| | - Linzy Bohn
- Department of Psychology, University of Alberta, Edmonton, AB, Canada
- Neuroscience and Mental Health Institute, University of Alberta, Edmonton, AB, Canada
| | - Myrlene Gee
- Department of Medicine (Neurology), University of Alberta, Edmonton, AB, Canada
| | - Shannon M. Drouin
- Department of Psychology, University of Alberta, Edmonton, AB, Canada
- Neuroscience and Mental Health Institute, University of Alberta, Edmonton, AB, Canada
| | - Harrison Fah
- Neuroscience and Mental Health Institute, University of Alberta, Edmonton, AB, Canada
- Department of Computing Science, University of Alberta, Edmonton, AB, Canada
| | - Wei Han
- Department of Chemistry, University of Alberta, Edmonton, AB, Canada
| | - Liang Li
- Department of Chemistry, University of Alberta, Edmonton, AB, Canada
| | - Richard Camicioli
- Neuroscience and Mental Health Institute, University of Alberta, Edmonton, AB, Canada
- Department of Medicine (Neurology), University of Alberta, Edmonton, AB, Canada
| | - Roger A. Dixon
- Department of Psychology, University of Alberta, Edmonton, AB, Canada
- Neuroscience and Mental Health Institute, University of Alberta, Edmonton, AB, Canada
| |
Collapse
|
13
|
de Vries BM, Zwezerijnen GJC, Burchell GL, van Velden FHP, Menke-van der Houven van Oordt CW, Boellaard R. Explainable artificial intelligence (XAI) in radiology and nuclear medicine: a literature review. Front Med (Lausanne) 2023; 10:1180773. [PMID: 37250654 PMCID: PMC10213317 DOI: 10.3389/fmed.2023.1180773] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2023] [Accepted: 04/17/2023] [Indexed: 05/31/2023] Open
Abstract
Rational Deep learning (DL) has demonstrated a remarkable performance in diagnostic imaging for various diseases and modalities and therefore has a high potential to be used as a clinical tool. However, current practice shows low deployment of these algorithms in clinical practice, because DL algorithms lack transparency and trust due to their underlying black-box mechanism. For successful employment, explainable artificial intelligence (XAI) could be introduced to close the gap between the medical professionals and the DL algorithms. In this literature review, XAI methods available for magnetic resonance (MR), computed tomography (CT), and positron emission tomography (PET) imaging are discussed and future suggestions are made. Methods PubMed, Embase.com and Clarivate Analytics/Web of Science Core Collection were screened. Articles were considered eligible for inclusion if XAI was used (and well described) to describe the behavior of a DL model used in MR, CT and PET imaging. Results A total of 75 articles were included of which 54 and 17 articles described post and ad hoc XAI methods, respectively, and 4 articles described both XAI methods. Major variations in performance is seen between the methods. Overall, post hoc XAI lacks the ability to provide class-discriminative and target-specific explanation. Ad hoc XAI seems to tackle this because of its intrinsic ability to explain. However, quality control of the XAI methods is rarely applied and therefore systematic comparison between the methods is difficult. Conclusion There is currently no clear consensus on how XAI should be deployed in order to close the gap between medical professionals and DL algorithms for clinical implementation. We advocate for systematic technical and clinical quality assessment of XAI methods. Also, to ensure end-to-end unbiased and safe integration of XAI in clinical workflow, (anatomical) data minimization and quality control methods should be included.
Collapse
Affiliation(s)
- Bart M. de Vries
- Department of Radiology and Nuclear Medicine, Cancer Center Amsterdam, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | - Gerben J. C. Zwezerijnen
- Department of Radiology and Nuclear Medicine, Cancer Center Amsterdam, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | | | | | | | - Ronald Boellaard
- Department of Radiology and Nuclear Medicine, Cancer Center Amsterdam, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| |
Collapse
|
14
|
Pham NT, Bunruangses M, Youplao P, Garhwal A, Ray K, Roy A, Boonkirdram S, Yupapin P, Jalil MA, Ali J, Kaiser S, Mahmud M, Mallik S, Zhao Z. An exploratory simulation study and prediction model on human brain behavior and activity using an integration of deep neural network and biosensor Rabi antenna. Heliyon 2023; 9:e15749. [PMID: 37305516 PMCID: PMC10256856 DOI: 10.1016/j.heliyon.2023.e15749] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2022] [Revised: 04/20/2023] [Accepted: 04/20/2023] [Indexed: 06/13/2023] Open
Abstract
The plasmonic antenna probe is constructed using a silver rod embedded in a modified Mach-Zehnder interferometer (MZI) ad-drop filter. Rabi antennas are formed when space-time control reaches two levels of system oscillation and can be used as human brain sensor probes. Photonic neural networks are designed using brain-Rabi antenna communication, and transmissions are connected via neurons. Communication signals are carried by electron spin (up and down) and adjustable Rabi frequency. Hidden variables and deep brain signals can be obtained by external detection. A Rabi antenna has been developed by simulation using computer simulation technology (CST) software. Additionally, a communication device has been developed that uses the Optiwave program with Finite-Difference Time-Domain (OptiFDTD). The output signal is plotted using the MATLAB program with the parameters of the OptiFDTD simulation results. The proposed antenna oscillates in the frequency range of 192 THz to 202 THz with a maximum gain of 22.4 dBi. The sensitivity of the sensor is calculated along with the result of electron spin and applied to form a human brain connection. Moreover, intelligent machine learning algorithms are proposed to identify high-quality transmissions and predict the behavior of transmissions in the near future. During the process, a root mean square error (RMSE) of 2.3332(±0.2338) was obtained. Finally, it can be said that our proposed model can efficiently predict human mind, thoughts, behavior as well as action/reaction, which can be greatly helpful in the diagnosis of various neuro-degenerative/psychological diseases (such as Alzheimer's, dementia, etc.) and for security purposes.
Collapse
Affiliation(s)
- Nhat Truong Pham
- Computational Biology and Bioinformatics Laboratory, Department of Integrative Biotechnology, College of Biotechnology and Bioengineering, Sungkyunkwan University, Suwon 16419, Gyeonggi-do, Republic of Korea
| | - Montree Bunruangses
- Department of Computer Engineering, Faculty of Industrial Education, Rajamangala University of Technology Phra Nakhon, Bangkok 10300, Thailand
| | - Phichai Youplao
- Department of Electrical Engineering, Faculty of Industry and Technology, Rajamangala University of Technology Isan Sakon Nakhon Campus, 199 Village no. 3, Phungkon, Sakon Nakhon 47160, Thailand
| | - Anita Garhwal
- Asia Metropolitan University, 6, Jalan Lembah, Bandar Baru Seri Alam 81750, Masai, Johor, Malaysia
| | - Kanad Ray
- Amity School of Applied Sciences, Amity University Rajasthan, Jaipur, India
- Facultad de CienciasFisico-Matematicas, Benemérita Universidad Autónoma de Puebla, Av. San Claudio y AV. 18 sur, Col. San Manuel Ciudad Universitaria, Pueble Pue. 72570, Mexico
- Faubert Lab, Ecole d'optométrie, Université de Montréal, Montréal, QC H3T1P1, Canada
| | - Arup Roy
- School of Computing and Information Technology, Reva University, Bengaluru, Karnataka 560064, India
| | - Sarawoot Boonkirdram
- Program of Electrical and Electronics, Faculty of Industrial Technology, Sakon Nakhon Rajabhat University, 680 Nittayo, Mueang, Sakon Nakhon 47000, Thailand
| | - Preecha Yupapin
- Department of Electrical Technology, School of Industrial Technology, Sakonnakhon Technical College, Institute of Vocational Education Northeastern 2, Sakonnakhon 47000, Thailand
| | - Muhammad Arif Jalil
- Department of Physics, Faculty of Science, Unversiti Teknologi Malaysia, 81310 Skudai, Johor, Malaysia
| | - Jalil Ali
- Department of Electrical Engineering, Faculty of Industry and Technology, Rajamangala University of Technology Isan Sakon Nakhon Campus, 199 Village no. 3, Phungkon, Sakon Nakhon 47160, Thailand
| | - Shamim Kaiser
- Institute of Information Technology, Jahangirnagar University, Savar, Dhaka 1342, Bangladesh
| | - Mufti Mahmud
- Nottingham Trent University, Clifton Lane, NG11 8NS, Nottingham, United Kingdom
| | - Saurav Mallik
- Department of Environmental Health, Harvard T H Chan School of Public Health, Boston, MA 02115, USA
| | - Zhongming Zhao
- Center for Precision Health, School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Human Genetics Center, School of Public Health, The University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| |
Collapse
|
15
|
Mallik S, Sarkar A, Nath S, Maulik U, Das S, Pati SK, Ghosh S, Zhao Z. 3PNMF-MKL: A non-negative matrix factorization-based multiple kernel learning method for multi-modal data integration and its application to gene signature detection. Front Genet 2023; 14:1095330. [PMID: 36865387 PMCID: PMC9971618 DOI: 10.3389/fgene.2023.1095330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2022] [Accepted: 01/30/2023] [Indexed: 02/16/2023] Open
Abstract
In this current era, biomedical big data handling is a challenging task. Interestingly, the integration of multi-modal data, followed by significant feature mining (gene signature detection), becomes a daunting task. Remembering this, here, we proposed a novel framework, namely, three-factor penalized, non-negative matrix factorization-based multiple kernel learning with soft margin hinge loss (3PNMF-MKL) for multi-modal data integration, followed by gene signature detection. In brief, limma, employing the empirical Bayes statistics, was initially applied to each individual molecular profile, and the statistically significant features were extracted, which was followed by the three-factor penalized non-negative matrix factorization method used for data/matrix fusion using the reduced feature sets. Multiple kernel learning models with soft margin hinge loss had been deployed to estimate average accuracy scores and the area under the curve (AUC). Gene modules had been identified by the consecutive analysis of average linkage clustering and dynamic tree cut. The best module containing the highest correlation was considered the potential gene signature. We utilized an acute myeloid leukemia cancer dataset from The Cancer Genome Atlas (TCGA) repository containing five molecular profiles. Our algorithm generated a 50-gene signature that achieved a high classification AUC score (viz., 0.827). We explored the functions of signature genes using pathway and Gene Ontology (GO) databases. Our method outperformed the state-of-the-art methods in terms of computing AUC. Furthermore, we included some comparative studies with other related methods to enhance the acceptability of our method. Finally, it can be notified that our algorithm can be applied to any multi-modal dataset for data integration, followed by gene module discovery.
Collapse
Affiliation(s)
- Saurav Mallik
- Department of Environmental Health, Harvard T H Chan School of public Health, Boston, MA, United States,*Correspondence: Saurav Mallik, , ; Zhongming Zhao,
| | - Anasua Sarkar
- Department of Computer Science & Engineering, Jadavpur University, Kolkata, India
| | - Sagnik Nath
- Department of Computer Science & Engineering, Jadavpur University, Kolkata, India
| | - Ujjwal Maulik
- Department of Computer Science & Engineering, Jadavpur University, Kolkata, India
| | - Supantha Das
- Department of Information Technology, Academy of Technology, Hooghly, West Bengal, India
| | - Soumen Kumar Pati
- Department of Bioinformatics, Maulana Abul Kalam Azad University, Kolkata, West Bengal, India
| | - Soumadip Ghosh
- Department of Computer Science & Engineering, Sister Nivedita University, New Town, West Bengal, India
| | - Zhongming Zhao
- Human Genetics Center, School of Public Health, The University of Texas Health Science Center at Houston, Houston, TX, United States,Center for Precision Health, School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX, United States,*Correspondence: Saurav Mallik, , ; Zhongming Zhao,
| |
Collapse
|
16
|
Bhandari M, Neupane A, Mallik S, Gaur L, Qin H. Auguring Fake Face Images Using Dual Input Convolution Neural Network. J Imaging 2022; 9:3. [PMID: 36662101 PMCID: PMC9861767 DOI: 10.3390/jimaging9010003] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2022] [Revised: 12/06/2022] [Accepted: 12/13/2022] [Indexed: 12/24/2022] Open
Abstract
Deepfake technology uses auto-encoders and generative adversarial networks to replace or artificially construct fine-tuned faces, emotions, and sounds. Although there have been significant advancements in the identification of particular fake images, a reliable counterfeit face detector is still lacking, making it difficult to identify fake photos in situations with further compression, blurring, scaling, etc. Deep learning models resolve the research gap to correctly recognize phony images, whose objectionable content might encourage fraudulent activity and cause major problems. To reduce the gap and enlarge the fields of view of the network, we propose a dual input convolutional neural network (DICNN) model with ten-fold cross validation with an average training accuracy of 99.36 ± 0.62, a test accuracy of 99.08 ± 0.64, and a validation accuracy of 99.30 ± 0.94. Additionally, we used 'SHapley Additive exPlanations (SHAP) ' as explainable AI (XAI) Shapely values to explain the results and interoperability visually by imposing the model into SHAP. The proposed model holds significant importance for being accepted by forensics and security experts because of its distinctive features and considerably higher accuracy than state-of-the-art methods.
Collapse
Affiliation(s)
- Mohan Bhandari
- Department of Science and Technology, Samriddhi College, Lokanthali, Bhaktapur 44800, Nepal
| | - Arjun Neupane
- School of Engineering and Technology, Central Queensland University, Norman Gardens, Rockhampton, QLD 4701, Australia
| | - Saurav Mallik
- Department of Environmental Health, School of Public Health, Harvard University, Boston, MA 02115, USA
- Research Assistant, University of Arizona, Tucson, AZ 85721, USA
| | - Loveleen Gaur
- Amity International Business School, Amity University, Noida 201303, India
- School of Computer Science, Taylor University, Subang Jaya 47500, Malaysia
- Graduate School of Business, University of South Pacific, Suva 1168, Fiji
| | - Hong Qin
- Department of Computer Science and Engineering, University of Tennessee, Chattanooga, TN 37996, USA
| |
Collapse
|
17
|
Ladbury C, Zarinshenas R, Semwal H, Tam A, Vaidehi N, Rodin AS, Liu A, Glaser S, Salgia R, Amini A. Utilization of model-agnostic explainable artificial intelligence frameworks in oncology: a narrative review. Transl Cancer Res 2022; 11:3853-3868. [PMID: 36388027 PMCID: PMC9641128 DOI: 10.21037/tcr-22-1626] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2022] [Accepted: 09/07/2022] [Indexed: 11/25/2022]
Abstract
Background and Objective Machine learning (ML) models are increasingly being utilized in oncology research for use in the clinic. However, while more complicated models may provide improvements in predictive or prognostic power, a hurdle to their adoption are limits of model interpretability, wherein the inner workings can be perceived as a "black box". Explainable artificial intelligence (XAI) frameworks including Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP) are novel, model-agnostic approaches that aim to provide insight into the inner workings of the "black box" by producing quantitative visualizations of how model predictions are calculated. In doing so, XAI can transform complicated ML models into easily understandable charts and interpretable sets of rules, which can give providers with an intuitive understanding of the knowledge generated, thus facilitating the deployment of such models in routine clinical workflows. Methods We performed a comprehensive, non-systematic review of the latest literature to define use cases of model-agnostic XAI frameworks in oncologic research. The examined database was PubMed/MEDLINE. The last search was run on May 1, 2022. Key Content and Findings In this review, we identified several fields in oncology research where ML models and XAI were utilized to improve interpretability, including prognostication, diagnosis, radiomics, pathology, treatment selection, radiation treatment workflows, and epidemiology. Within these fields, XAI facilitates determination of feature importance in the overall model, visualization of relationships and/or interactions, evaluation of how individual predictions are produced, feature selection, identification of prognostic and/or predictive thresholds, and overall confidence in the models, among other benefits. These examples provide a basis for future work to expand on, which can facilitate adoption in the clinic when the complexity of such modeling would otherwise be prohibitive. Conclusions Model-agnostic XAI frameworks offer an intuitive and effective means of describing oncology ML models, with applications including prognostication and determination of optimal treatment regimens. Using such frameworks presents an opportunity to improve understanding of ML models, which is a critical step to their adoption in the clinic.
Collapse
Affiliation(s)
- Colton Ladbury
- Department of Radiation Oncology, City of Hope National Medical Center, Duarte, CA, USA
| | - Reza Zarinshenas
- Department of Radiation Oncology, City of Hope National Medical Center, Duarte, CA, USA
| | - Hemal Semwal
- Departments of Bioengineering and Integrated Biology and Physiology, University of California Los Angeles, Los Angeles, CA, USA
| | - Andrew Tam
- Department of Radiation Oncology, City of Hope National Medical Center, Duarte, CA, USA
| | - Nagarajan Vaidehi
- Department of Computational and Quantitative Medicine, City of Hope National Medical Center, Duarte, CA, USA
| | - Andrei S Rodin
- Department of Computational and Quantitative Medicine, City of Hope National Medical Center, Duarte, CA, USA
| | - An Liu
- Department of Radiation Oncology, City of Hope National Medical Center, Duarte, CA, USA
| | - Scott Glaser
- Department of Radiation Oncology, City of Hope National Medical Center, Duarte, CA, USA
| | - Ravi Salgia
- Department of Medical Oncology, City of Hope National Medical Center, Duarte, CA, USA
| | - Arya Amini
- Department of Radiation Oncology, City of Hope National Medical Center, Duarte, CA, USA
| |
Collapse
|
18
|
Ghose P, Alavi M, Tabassum M, Ashraf Uddin M, Biswas M, Mahbub K, Gaur L, Mallik S, Zhao Z. Detecting COVID-19 infection status from chest X-ray and CT scan via single transfer learning-driven approach. Front Genet 2022; 13:980338. [PMID: 36212141 PMCID: PMC9533058 DOI: 10.3389/fgene.2022.980338] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2022] [Accepted: 08/18/2022] [Indexed: 11/17/2022] Open
Abstract
COVID-19 has caused over 528 million infected cases and over 6.25 million deaths since its outbreak in 2019. The uncontrolled transmission of the SARS-CoV-2 virus has caused human suffering and the death of uncountable people. Despite the continuous effort by the researchers and laboratories, it has been difficult to develop reliable efficient and stable vaccines to fight against the rapidly evolving virus strains. Therefore, effectively preventing the transmission in the community and globally has remained an urgent task since its outbreak. To avoid the rapid spread of infection, we first need to identify the infected individuals and isolate them. Therefore, screening computed tomography (CT scan) and X-ray can better separate the COVID-19 infected patients from others. However, one of the main challenges is to accurately identify infection from a medical image. Even experienced radiologists often have failed to do it accurately. On the other hand, deep learning algorithms can tackle this task much easier, faster, and more accurately. In this research, we adopt the transfer learning method to identify the COVID-19 patients from normal individuals when there is an inadequacy of medical image data to save time by generating reliable results promptly. Furthermore, our model can perform both X-rays and CT scan. The experimental results found that the introduced model can achieve 99.59% accuracy for X-rays and 99.95% for CT scan images. In summary, the proposed method can effectively identify COVID-19 infected patients, could be a great way which will help to classify COVID-19 patients quickly and prevent the viral transmission in the community.
Collapse
Affiliation(s)
- Partho Ghose
- Department of Computer Science and Engineering, Bangladesh University of Business and Technology, Dhaka, Bangladesh
| | - Muhaddid Alavi
- Department of Computer Science and Engineering, Jagannath University, Dhaka, Bangladesh
| | - Mehnaz Tabassum
- Center for Health Informatics, Macquarie University, Sydney, NSW, Australia
| | - Md. Ashraf Uddin
- Department of Computer Science and Engineering, Jagannath University, Dhaka, Bangladesh
| | - Milon Biswas
- Department of Computer Science and Engineering, Bangladesh University of Business and Technology, Dhaka, Bangladesh
| | - Kawsher Mahbub
- Department of Computer Science and Engineering, Bangladesh University of Business and Technology, Dhaka, Bangladesh
| | - Loveleen Gaur
- Amity International Business School, Amity University, Noida, India
| | - Saurav Mallik
- Department of Environmental Health, Harvard T H Chan School of Public Health, Boston, MA, United States
| | - Zhongming Zhao
- Department of Environmental Health, Harvard T H Chan School of Public Health, Boston, MA, United States
- Human Genetics Center, School of Public Health, The University of Texas Health Science Center at Houston, Houston, TX, United States
- *Correspondence: Zhongming Zhao,
| |
Collapse
|
19
|
Designing optimal convolutional neural network architecture using differential evolution algorithm. PATTERNS 2022; 3:100567. [PMID: 36124301 PMCID: PMC9481963 DOI: 10.1016/j.patter.2022.100567] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/10/2022] [Revised: 06/04/2022] [Accepted: 07/13/2022] [Indexed: 01/08/2023]
Abstract
Convolutional neural networks (CNNs) are deep learning models used widely for solving various tasks like computer vision and speech recognition. CNNs are developed manually based on problem-specific domain knowledge and tricky settings, which are laborious, time consuming, and challenging. To solve these, our study develops an improved differential evolution of convolutional neural network (IDECNN) algorithm to design CNN layer architectures for image classification. Variable-length encoding is utilized to represent the flexible layer architecture of a CNN model in IDECNN. An efficient heuristic mechanism is proposed in IDECNN to evolve CNN architecture through mutation and crossover to prevent premature convergence during the evolutionary process. Eight well-known imaging datasets were utilized. The results showed that IDECNN could design suitable architecture compared with 20 existing CNN models. Finally, CNN architectures are applied to pneumonia and coronavirus disease 2019 (COVID-19) X-ray biomedical image data. The results demonstrated the usefulness of the proposed approach to generate a suitable CNN model. Introduce DE algorithm to automatically design CNN architectures Variable-length encoding strategy is proposed to encode each CNN model For the DE framework, two CNN architectures undergo a refinement difference approach Design a heuristic mechanism for mutation operation to evolve CNN architectures
Convolutional neural networks (CNNs) are a class of deep learning (DL) methods that have demonstrated improved performance in various computer vision tasks. With the growing popularity of CNNs, several CNN architectures have been introduced with a large number of design options that are problem dependent. In most situations, the constructed CNN model performs well on the dataset used to train it. There is no guarantee that the designed CNN model can achieve sufficient classification accuracy for other datasets. Designing an appropriate CNN model architecture for a particular problem requires human interaction and trial-and-error procedures, which are laborious and time consuming. This study uses an improved differential evolution of convolutional neural network (IDECNN) technique to automatically construct effective CNN architectures for several image classification problems, which mitigates the issues found with manually designed CNN models.
Collapse
|
20
|
Xie Y, Zaccagna F, Rundo L, Testa C, Agati R, Lodi R, Manners DN, Tonon C. Convolutional Neural Network Techniques for Brain Tumor Classification (from 2015 to 2022): Review, Challenges, and Future Perspectives. Diagnostics (Basel) 2022; 12:diagnostics12081850. [PMID: 36010200 PMCID: PMC9406354 DOI: 10.3390/diagnostics12081850] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Revised: 07/20/2022] [Accepted: 07/28/2022] [Indexed: 12/21/2022] Open
Abstract
Convolutional neural networks (CNNs) constitute a widely used deep learning approach that has frequently been applied to the problem of brain tumor diagnosis. Such techniques still face some critical challenges in moving towards clinic application. The main objective of this work is to present a comprehensive review of studies using CNN architectures to classify brain tumors using MR images with the aim of identifying useful strategies for and possible impediments in the development of this technology. Relevant articles were identified using a predefined, systematic procedure. For each article, data were extracted regarding training data, target problems, the network architecture, validation methods, and the reported quantitative performance criteria. The clinical relevance of the studies was then evaluated to identify limitations by considering the merits of convolutional neural networks and the remaining challenges that need to be solved to promote the clinical application and development of CNN algorithms. Finally, possible directions for future research are discussed for researchers in the biomedical and machine learning communities. A total of 83 studies were identified and reviewed. They differed in terms of the precise classification problem targeted and the strategies used to construct and train the chosen CNN. Consequently, the reported performance varied widely, with accuracies of 91.63–100% in differentiating meningiomas, gliomas, and pituitary tumors (26 articles) and of 60.0–99.46% in distinguishing low-grade from high-grade gliomas (13 articles). The review provides a survey of the state of the art in CNN-based deep learning methods for brain tumor classification. Many networks demonstrated good performance, and it is not evident that any specific methodological choice greatly outperforms the alternatives, especially given the inconsistencies in the reporting of validation methods, performance metrics, and training data encountered. Few studies have focused on clinical usability.
Collapse
Affiliation(s)
- Yuting Xie
- Department of Biomedical and Neuromotor Sciences, University of Bologna, 40126 Bologna, Italy; (Y.X.); (F.Z.); (R.L.); (C.T.)
| | - Fulvio Zaccagna
- Department of Biomedical and Neuromotor Sciences, University of Bologna, 40126 Bologna, Italy; (Y.X.); (F.Z.); (R.L.); (C.T.)
- Functional and Molecular Neuroimaging Unit, IRCCS Istituto delle Scienze Neurologiche di Bologna, Bellaria Hospital, 40139 Bologna, Italy;
| | - Leonardo Rundo
- Department of Information and Electrical Engineering and Applied Mathematics, University of Salerno, 84084 Fisciano, Italy;
| | - Claudia Testa
- Functional and Molecular Neuroimaging Unit, IRCCS Istituto delle Scienze Neurologiche di Bologna, Bellaria Hospital, 40139 Bologna, Italy;
- Department of Physics and Astronomy, University of Bologna, 40127 Bologna, Italy
| | - Raffaele Agati
- Programma Neuroradiologia con Tecniche ad elevata complessità, IRCCS Istituto delle Scienze Neurologiche di Bologna, Bellaria Hospital, 40139 Bologna, Italy;
| | - Raffaele Lodi
- Department of Biomedical and Neuromotor Sciences, University of Bologna, 40126 Bologna, Italy; (Y.X.); (F.Z.); (R.L.); (C.T.)
- IRCCS Istituto delle Scienze Neurologiche di Bologna, Bellaria Hospital, 40139 Bologna, Italy
| | - David Neil Manners
- Department of Biomedical and Neuromotor Sciences, University of Bologna, 40126 Bologna, Italy; (Y.X.); (F.Z.); (R.L.); (C.T.)
- Correspondence:
| | - Caterina Tonon
- Department of Biomedical and Neuromotor Sciences, University of Bologna, 40126 Bologna, Italy; (Y.X.); (F.Z.); (R.L.); (C.T.)
- Functional and Molecular Neuroimaging Unit, IRCCS Istituto delle Scienze Neurologiche di Bologna, Bellaria Hospital, 40139 Bologna, Italy;
| |
Collapse
|