1
|
Mattia GM, Villain E, Nemmi F, Le Lann MV, Franceries X, Péran P. Investigating the discrimination ability of 3D convolutional neural networks applied to altered brain MRI parametric maps. Artif Intell Med 2024; 153:102897. [PMID: 38810471 DOI: 10.1016/j.artmed.2024.102897] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 03/05/2024] [Accepted: 05/15/2024] [Indexed: 05/31/2024]
Abstract
Convolutional neural networks (CNNs) are gradually being recognized in the neuroimaging community as a powerful tool for image analysis. Despite their outstanding performances, some aspects of CNN functioning are still not fully understood by human operators. We postulated that the interpretability of CNNs applied to neuroimaging data could be improved by investigating their behavior when they are fed data with known characteristics. We analyzed the ability of 3D CNNs to discriminate between original and altered whole-brain parametric maps derived from diffusion-weighted magnetic resonance imaging. The alteration consisted in linearly changing the voxel intensity of either one (monoregion) or two (biregion) anatomical regions in each brain volume, but without mimicking any neuropathology. Performing ten-fold cross-validation and using a hold-out set for testing, we assessed the CNNs' discrimination ability according to the intensity of the altered regions, comparing the latter's size and relative position. Monoregion CNNs showed that the larger the modified region, the smaller the intensity increase needed to achieve good performances. Biregion CNNs systematically outperformed monoregion CNNs, but could only detect one of the two target regions when tested on the corresponding monoregion images. Exploiting prior information on training data allowed for a better understanding of CNN behavior, especially when altered regions were combined. This can inform about the complexity of CNN pattern retrieval and elucidate misclassified examples, particularly relevant for pathological data. The proposed analytical approach may serve to gain insights into CNN behavior and guide the design of enhanced detection systems exploiting our prior knowledge.
Collapse
Affiliation(s)
- Giulia Maria Mattia
- ToNIC, Toulouse NeuroImaging Center, Université de Toulouse, Inserm, UPS, Toulouse, France.
| | - Edouard Villain
- ToNIC, Toulouse NeuroImaging Center, Université de Toulouse, Inserm, UPS, Toulouse, France; LAAS CNRS, Université de Toulouse, CNRS, INSA, UPS, Toulouse, France.
| | - Federico Nemmi
- ToNIC, Toulouse NeuroImaging Center, Université de Toulouse, Inserm, UPS, Toulouse, France.
| | | | - Xavier Franceries
- CRCT, Centre de Recherche en Cancérologie de Toulouse, Inserm, UPS, Toulouse, France.
| | - Patrice Péran
- ToNIC, Toulouse NeuroImaging Center, Université de Toulouse, Inserm, UPS, Toulouse, France.
| |
Collapse
|
2
|
Chen Z, Ge R, Wang C, Elazab A, Fu X, Min W, Qin F, Jia G, Fan X. Identification of important gene signatures in schizophrenia through feature fusion and genetic algorithm. Mamm Genome 2024; 35:241-255. [PMID: 38512459 DOI: 10.1007/s00335-024-10034-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Accepted: 02/07/2024] [Indexed: 03/23/2024]
Abstract
Schizophrenia is a debilitating psychiatric disorder that can significantly affect a patient's quality of life and lead to permanent brain damage. Although medical research has identified certain genetic risk factors, the specific pathogenesis of the disorder remains unclear. Despite the prevalence of research employing magnetic resonance imaging, few studies have focused on the gene level and gene expression profile involving a large number of screened genes. However, the high dimensionality of genetic data presents a great challenge to accurately modeling the data. To tackle the current challenges, this study presents a novel feature selection strategy that utilizes heuristic feature fusion and a multi-objective optimization genetic algorithm. The goal is to improve classification performance and identify the key gene subset for schizophrenia diagnostics. Traditional gene screening techniques are inadequate for accurately determining the precise number of key genes associated with schizophrenia. Our innovative approach integrates a filter-based feature selection method to reduce data dimensionality and a multi-objective optimization genetic algorithm for improved classification tasks. By combining the filtering and wrapper methods, our strategy leverages their respective strengths in a deliberate manner, leading to superior classification accuracy and a more efficient selection of relevant genes. This approach has demonstrated significant improvements in classification results across 11 out of 14 relevant datasets. The performance on the remaining three datasets is comparable to the existing methods. Furthermore, visual and enrichment analyses have confirmed the practicality of our proposed method as a promising tool for the early detection of schizophrenia.
Collapse
Affiliation(s)
| | - Ruiquan Ge
- Hangzhou Dianzi University, Hangzhou, China.
- Hangzhou Institute of Advanced Technology, Hangzhou, China.
- Key Laboratory of Discrete Industrial Internet of Things of Zhejiang Province, Hangzhou, China.
| | - Changmiao Wang
- Shenzhen Research Institute of Big Data, Shenzhen, China
| | - Ahmed Elazab
- Computer Science Department, Misr Higher Institute for Commerce and Computers, Mansoura, Egypt
| | - Xianjun Fu
- School of Artificial Intelligence, Zhejiang College of Security Technology, Wenzhou, China
| | - Wenwen Min
- School of Information Science and Engineering, Yunnan University, Kunming, China
| | - Feiwei Qin
- Hangzhou Dianzi University, Hangzhou, China
| | | | - Xiaopeng Fan
- Hangzhou Institute of Advanced Technology, Hangzhou, China
| |
Collapse
|
3
|
Warren SL, Khan DM, Moustafa AA. Assistive tools for classifying neurological disorders using fMRI and deep learning: A guide and example. Brain Behav 2024; 14:e3554. [PMID: 38841732 PMCID: PMC11154821 DOI: 10.1002/brb3.3554] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Revised: 05/02/2024] [Accepted: 05/03/2024] [Indexed: 06/07/2024] Open
Abstract
BACKGROUND Deep-learning (DL) methods are rapidly changing the way researchers classify neurological disorders. For example, combining functional magnetic resonance imaging (fMRI) and DL has helped researchers identify functional biomarkers of neurological disorders (e.g., brain activation and connectivity) and pilot innovative diagnostic models. However, the knowledge required to perform DL analyses is often domain-specific and is not widely taught in the brain sciences (e.g., psychology, neuroscience, and cognitive science). Conversely, neurological diagnoses and neuroimaging training (e.g., fMRI) are largely restricted to the brain and medical sciences. In turn, these disciplinary knowledge barriers and distinct specializations can act as hurdles that prevent the combination of fMRI and DL pipelines. The complexity of fMRI and DL methods also hinders their clinical adoption and generalization to real-world diagnoses. For example, most current models are not designed for clinical settings or use by nonspecialized populations such as students, clinicians, and healthcare workers. Accordingly, there is a growing area of assistive tools (e.g., software and programming packages) that aim to streamline and increase the accessibility of fMRI and DL pipelines for the diagnoses of neurological disorders. OBJECTIVES AND METHODS In this study, we present an introductory guide to some popular DL and fMRI assistive tools. We also create an example autism spectrum disorder (ASD) classification model using assistive tools (e.g., Optuna, GIFT, and the ABIDE preprocessed repository), fMRI, and a convolutional neural network. RESULTS In turn, we provide researchers with a guide to assistive tools and give an example of a streamlined fMRI and DL pipeline. CONCLUSIONS We are confident that this study can help more researchers enter the field and create accessible fMRI and deep-learning diagnostic models for neurological disorders.
Collapse
Affiliation(s)
- Samuel L. Warren
- Faculty of Society and Design, School of PsychologyBond UniversityGold CoastQueenslandAustralia
| | - Danish M. Khan
- Department of Electronic EngineeringNED University of Engineering & TechnologyKarachiSindhPakistan
| | - Ahmed A. Moustafa
- Faculty of Society and Design, School of PsychologyBond UniversityGold CoastQueenslandAustralia
- The Faculty of Health Sciences, Department of Human Anatomy and PhysiologyUniversity of JohannesburgAuckland ParkSouth Africa
| |
Collapse
|
4
|
Song B, Yoshida S. Explainability of three-dimensional convolutional neural networks for functional magnetic resonance imaging of Alzheimer's disease classification based on gradient-weighted class activation mapping. PLoS One 2024; 19:e0303278. [PMID: 38771733 PMCID: PMC11108152 DOI: 10.1371/journal.pone.0303278] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Accepted: 04/22/2024] [Indexed: 05/23/2024] Open
Abstract
Currently, numerous studies focus on employing fMRI-based deep neural networks to diagnose neurological disorders such as Alzheimer's Disease (AD), yet only a handful have provided results regarding explainability. We address this gap by applying several prevalent explainability methods such as gradient-weighted class activation mapping (Grad-CAM) to an fMRI-based 3D-VGG16 network for AD diagnosis to improve the model's explainability. The aim is to explore the specific Region of Interest (ROI) of brain the model primarily focuses on when making predictions, as well as whether there are differences in these ROIs between AD and normal controls (NCs). First, we utilized multiple resting-state functional activity maps including ALFF, fALFF, ReHo, and VMHC to reduce the complexity of fMRI data, which differed from many studies that utilized raw fMRI data. Compared to methods utilizing raw fMRI data, this manual feature extraction approach may potentially alleviate the model's burden. Subsequently, 3D-VGG16 were employed for AD classification, where the final fully connected layers were replaced with a Global Average Pooling (GAP) layer, aimed at mitigating overfitting while preserving spatial information within the feature maps. The model achieved a maximum of 96.4% accuracy on the test set. Finally, several 3D CAM methods were employed to interpret the models. In the explainability results of the models with relatively high accuracy, the highlighted ROIs were primarily located in the precuneus and the hippocampus for AD subjects, while the models focused on the entire brain for NC. This supports current research on ROIs involved in AD. We believe that explaining deep learning models would not only provide support for existing research on brain disorders, but also offer important referential recommendations for the study of currently unknown etiologies.
Collapse
Affiliation(s)
- Boyue Song
- Graduate School of Engineering, Kochi University of Technology, Kami City, Kochi Prefecture, Japan
| | - Shinichi Yoshida
- School of Information, Kochi University of Technology, Kami City, Kochi Prefecture, Japan
| | | |
Collapse
|
5
|
Gao J, Qian M, Wang Z, Li Y, Luo N, Xie S, Shi W, Li P, Chen J, Chen Y, Wang H, Liu W, Li Z, Yang Y, Guo H, Wan P, Lv L, Lu L, Yan J, Song Y, Wang H, Zhang H, Wu H, Ning Y, Du Y, Cheng Y, Xu J, Xu X, Zhang D, Jiang T. Exploring Schizophrenia Classification Through Multimodal MRI and Deep Graph Neural Networks: Unveiling Brain Region-Specific Weight Discrepancies and Their Association With Cell-Type Specific Transcriptomic Features. Schizophr Bull 2024:sbae069. [PMID: 38754993 DOI: 10.1093/schbul/sbae069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/18/2024]
Abstract
BACKGROUND AND HYPOTHESIS Schizophrenia (SZ) is a prevalent mental disorder that imposes significant health burdens. Diagnostic accuracy remains challenging due to clinical subjectivity. To address this issue, we explore magnetic resonance imaging (MRI) as a tool to enhance SZ diagnosis and provide objective references and biomarkers. Using deep learning with graph convolution, we represent MRI data as graphs, aligning with brain structure, and improving feature extraction, and classification. Integration of multiple modalities is expected to enhance classification. STUDY DESIGN Our study enrolled 683 SZ patients and 606 healthy controls from 7 hospitals, collecting structural MRI and functional MRI data. Both data types were represented as graphs, processed by 2 graph attention networks, and fused for classification. Grad-CAM with graph convolution ensured interpretability, and partial least squares analyzed gene expression in brain regions. STUDY RESULTS Our method excelled in the classification task, achieving 83.32% accuracy, 83.41% sensitivity, and 83.20% specificity in 10-fold cross-validation, surpassing traditional methods. And our multimodal approach outperformed unimodal methods. Grad-CAM identified potential brain biomarkers consistent with gene analysis and prior research. CONCLUSIONS Our study demonstrates the effectiveness of deep learning with graph attention networks, surpassing previous SZ diagnostic methods. Multimodal MRI's superiority over unimodal MRI confirms our initial hypothesis. Identifying potential brain biomarkers alongside gene biomarkers holds promise for advancing objective SZ diagnosis and research in SZ.
Collapse
Affiliation(s)
- Jingjing Gao
- School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Maomin Qian
- School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Zhengning Wang
- School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Yanling Li
- School of Electrical Engineering and Electronic Information, Xihua University, Chengdu, China
| | - Na Luo
- Brainnetome Center, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Sangma Xie
- Institute of Biomedical Engineering and Instrumentation, School of Automation, Hangzhou Dianzi University, Hangzhou, China
| | - Weiyang Shi
- Brainnetome Center and National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Peng Li
- Institute of Mental Health, Peking University Sixth Hospital, Beijing, China
- Key Laboratory of Mental Health, Ministry of Health, and National Clinical Research Center for Mental Disorders (Peking University Sixth Hospital), Beijing, China
| | - Jun Chen
- Department of Radiology, Renmin Hospital of Wuhan University, Wuhan, China
| | - Yunchun Chen
- Department of Psychiatry, Xijing Hospital, The Fourth Military Medical University, Xi'an, China
| | - Huaning Wang
- Department of Psychiatry, Xijing Hospital, The Fourth Military Medical University, Xi'an, China
| | - Wenming Liu
- Department of Psychiatry, Xijing Hospital, The Fourth Military Medical University, Xi'an, China
| | - Zhigang Li
- Zhumadian Psychiatric Hospital, Zhumadian, China
| | - Yongfeng Yang
- Department of Psychiatry, Henan Mental Hospital, The Second Affiliated Hospital of Xinxiang Medical University, Xinxiang, China
- Henan Key Lab of Biological Psychiatry of Xinxiang Medical University, International Joint Research Laboratory for Psychiatry and Neuroscience of Henan, Xinxiang, China
| | - Hua Guo
- Zhumadian Psychiatric Hospital, Zhumadian, China
| | - Ping Wan
- Zhumadian Psychiatric Hospital, Zhumadian, China
| | - Luxian Lv
- Department of Psychiatry, Henan Mental Hospital, The Second Affiliated Hospital of Xinxiang Medical University, Xinxiang, China
- Henan Key Lab of Biological Psychiatry of Xinxiang Medical University, International Joint Research Laboratory for Psychiatry and Neuroscience of Henan, Xinxiang, China
| | - Lin Lu
- Institute of Mental Health, Peking University Sixth Hospital, Beijing, China
- Key Laboratory of Mental Health, Ministry of Health, and National Clinical Research Center for Mental Disorders (Peking University Sixth Hospital), Beijing, China
| | - Jun Yan
- Institute of Mental Health, Peking University Sixth Hospital, Beijing, China
- Key Laboratory of Mental Health, Ministry of Health, and National Clinical Research Center for Mental Disorders (Peking University Sixth Hospital), Beijing, China
| | - Yuqing Song
- Institute of Mental Health, Peking University Sixth Hospital, Beijing, China
- Key Laboratory of Mental Health, Ministry of Health, and National Clinical Research Center for Mental Disorders (Peking University Sixth Hospital), Beijing, China
| | - Huiling Wang
- Department of Psychiatry, Renmin Hospital of Wuhan University, Wuhan, China
| | - Hongxing Zhang
- Department of Psychiatry, Henan Mental Hospital, The Second Affiliated Hospital of Xinxiang Medical University, Xinxiang, China
- Henan Key Lab of Biological Psychiatry of Xinxiang Medical University, International Joint Research Laboratory for Psychiatry and Neuroscience of Henan, Xinxiang, China
- Department of Psychology, Xinxiang Medical University, Xinxiang, China
| | - Huawang Wu
- The Affiliated Brain Hospital of Guangzhou Medical University (Guangzhou Huiai Hospital), Guangzhou, China
| | - Yuping Ning
- The Affiliated Brain Hospital of Guangzhou Medical University (Guangzhou Huiai Hospital), Guangzhou, China
| | - Yuhui Du
- School of Computer and Information Technology, Shanxi University, Taiyuan, China
| | - Yuqi Cheng
- Department of Psychiatry, First Affiliated Hospital of Kunming Medical University, Kunming, China
| | - Jian Xu
- Department of Psychiatry, First Affiliated Hospital of Kunming Medical University, Kunming, China
| | - Xiufeng Xu
- Department of Psychiatry, First Affiliated Hospital of Kunming Medical University, Kunming, China
| | - Dai Zhang
- Institute of Mental Health, Peking University Sixth Hospital, Beijing, China
- Key Laboratory of Mental Health, Ministry of Health, and National Clinical Research Center for Mental Disorders (Peking University Sixth Hospital), Beijing, China
- Center for Life Sciences/PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing, China
| | - Tianzai Jiang
- Brainnetome Center, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- Center for Excellence in Brain Science and Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- Research Center for Augmented Intelligence, Zhejiang Lab, Hangzhou, China
- Xiaoxiang Institute for Brain Health and Yongzhou Central Hospital, Yongzhou, China
| |
Collapse
|
6
|
Huynh N, Deshpande G. A review of the applications of generative adversarial networks to structural and functional MRI based diagnostic classification of brain disorders. Front Neurosci 2024; 18:1333712. [PMID: 38686334 PMCID: PMC11057233 DOI: 10.3389/fnins.2024.1333712] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2023] [Accepted: 02/19/2024] [Indexed: 05/02/2024] Open
Abstract
Structural and functional MRI (magnetic resonance imaging) based diagnostic classification using machine learning has long held promise, but there are many roadblocks to achieving their potential. While traditional machine learning models suffered from their inability to capture the complex non-linear mapping, deep learning models tend to overfit the model. This is because there is data scarcity and imbalanced classes in neuroimaging; it is expensive to acquire data from human subjects and even more so in clinical populations. Due to their ability to augment data by learning underlying distributions, generative adversarial networks (GAN) provide a potential solution to this problem. Here, we provide a methodological primer on GANs and review the applications of GANs to classification of mental health disorders from neuroimaging data such as functional MRI and showcase the progress made thus far. We also highlight gaps in methodology as well as interpretability that are yet to be addressed. This provides directions about how the field can move forward. We suggest that since there are a range of methodological choices available to users, it is critical for users to interact with method developers so that the latter can tailor their development according to the users' needs. The field can be enriched by such synthesis between method developers and users in neuroimaging.
Collapse
Affiliation(s)
- Nguyen Huynh
- Auburn University Neuroimaging Center, Department of Electrical and Computer Engineering, Auburn University, Auburn, AL, United States
| | - Gopikrishna Deshpande
- Auburn University Neuroimaging Center, Department of Electrical and Computer Engineering, Auburn University, Auburn, AL, United States
- Department of Psychological Sciences, Auburn University, Auburn, AL, United States
- Alabama Advanced Imaging Consortium, Birmingham, AL, United States
- Center for Neuroscience, Auburn University, Auburn, AL, United States
- Department of Psychiatry, National Institute of Mental Health and Neurosciences, Bangalore, India
- Department of Heritage Science and Technology, Indian Institute of Technology, Hyderabad, India
| |
Collapse
|
7
|
Ikemitsu N, Kanazawa Y, Haga A, Hayashi H, Matsumoto Y, Harada M. Determination of Alzheimer's disease based on morphology and atrophy using machine learning combined with automated segmentation. Acta Radiol 2024; 65:359-366. [PMID: 38196180 DOI: 10.1177/02841851231218384] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2024]
Abstract
BACKGROUND To evaluate the degree of cerebral atrophy for Alzheimer's disease (AD), voxel-based morphometry has been performed with magnetic resonance imaging. Detailed morphological changes in a specific tissue area having the most evidence of atrophy were not considered by the machine-learning technique. PURPOSE To develop a machine-learning system that can capture morphology features for determination of atrophy of brain tissue in early-stage AD and classification of healthy participants or patients. MATERIAL AND METHODS Three-dimensional T1-weighted (3D-T1W) data were obtained from AD Neuroimaging Initiative (200 healthy controls and 200 patients with early-stage AD). Automated segmentation of 3D-T1W data was performed. Deep learning (DL) and support vector machine (SVM) were trained using 66-segmented volume values as input and AD diagnosis as output. DL was performed using 66 volume values or gray matter (GM) and white matter (WM) volume values. SVM learning was performed using 66 volume values and six regions with high variable importance. 3D convolutional neural network (3D-CNN) was trained using the segmented images. Accuracy and area under curve (AUC) were obtained. Variable importance was evaluated from logistic regression analysis. RESULTS DL for GM and WM volume values, accuracy 0.6; SVM for all volume values, accuracy 0.82 and AUC 0.81; DL for all volume values, accuracy 0.82 and AUC 0.8; 3D-CNN using segmental images of the whole brain, accuracy 0.5 and AUC 0.51. SVM using volume values of six regions, accuracy 0.82; image-based 3D-CNN, highest accuracy 0.69. CONCLUSION Our results show that atrophic features are more considerable than morphological features in the early detection of AD.
Collapse
Affiliation(s)
- Natsuki Ikemitsu
- Graduate school of Health Science, Tokushima University, Tokushima, Japan
| | - Yuki Kanazawa
- Institute of Biomedical Sciences, Tokushima University Graduate School, Tokushima, Japan
| | - Akihiro Haga
- Institute of Biomedical Sciences, Tokushima University Graduate School, Tokushima, Japan
| | - Hiroaki Hayashi
- Faculty of Health Sciences, Institute of Medical, Pharmaceutical and Health Sciences, Kanazawa University, Ishikawa, Japan
| | - Yuki Matsumoto
- Institute of Biomedical Sciences, Tokushima University Graduate School, Tokushima, Japan
| | - Masafumi Harada
- Institute of Biomedical Sciences, Tokushima University Graduate School, Tokushima, Japan
| |
Collapse
|
8
|
Ma H, Wang Y, Hao Z, Yu Y, Jia X, Li M, Chen L. Classification of Alzheimer's disease: application of a transfer learning deep Q-network method. Eur J Neurosci 2024; 59:2118-2127. [PMID: 38282277 DOI: 10.1111/ejn.16261] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2023] [Revised: 12/25/2023] [Accepted: 01/08/2024] [Indexed: 01/30/2024]
Abstract
Early diagnosis is crucial to slowing the progression of Alzheimer's disease (AD), so it is urgent to find an effective diagnostic method for AD. This study intended to investigate whether the transfer learning approach of deep Q-network (DQN) could effectively distinguish AD patients using local metrics of resting-state functional magnetic resonance imaging (rs-fMRI) as features. This study included 1310 subjects from the Consortium for Reliability and Reproducibility (CoRR) and 50 subjects from the Alzheimer's Disease Neuroimaging Initiative (ADNI) GO/2. The amplitude of low-frequency fluctuation (ALFF), fractional ALFF (fALFF) and percent amplitude of fluctuation (PerAF) were extracted as features using the Power 264 atlas. Based on gender bias in AD, we searched for transferable similar parts between the CoRR feature matrix and the ADNI feature matrix, resulting in the CoRR similar feature matrix served as the source domain and the ADNI similar feature matrix served as the target domain. A DQN classifier was pre-trained in the source domain and transferred to the target domain. Finally, the transferred DQN classifier was used to classify AD and healthy controls (HC). A permutation test was performed. The DQN transfer learning achieved a classification accuracy of 86.66% (p < 0.01), recall of 83.33% and precision of 83.33%. The findings suggested that the transfer learning approach using DQN could be an effective way to distinguish AD from HC. It also revealed the potential value of local brain activity in AD clinical diagnosis.
Collapse
Affiliation(s)
- Huibin Ma
- School of Information and Electronics Technology, Jiamusi University, Jiamusi, China
- Key Laboratory of Autonomous Intelligence and Information Processing in Heilongjiang Province, Jiamusi, China
| | - Yadan Wang
- School of Information and Electronics Technology, Jiamusi University, Jiamusi, China
- Key Laboratory of Autonomous Intelligence and Information Processing in Heilongjiang Province, Jiamusi, China
| | - Zeqi Hao
- School of Psychology, Zhejiang Normal University, Jinhua, China
| | - Yang Yu
- Department of Psychiatry, the second affiliated hospital of Zhejiang University school of Medicine, Zhejiang, China
| | - Xize Jia
- Department of Radiology, Changshu No. 2 People's Hospital, The Affiliated Changshu Hospital of Xuzhou Medical University, Changshu, China
| | - Mengting Li
- School of Psychology, Zhejiang Normal University, Jinhua, China
| | - Lanfen Chen
- School of Medical Imaging, Weifang Medical University, Weifang, China
| |
Collapse
|
9
|
Zhang J, Wang Q, Wang X, Qiao L, Liu M. Preserving specificity in federated graph learning for fMRI-based neurological disorder identification. Neural Netw 2024; 169:584-596. [PMID: 37956575 DOI: 10.1016/j.neunet.2023.11.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Revised: 10/22/2023] [Accepted: 11/03/2023] [Indexed: 11/15/2023]
Abstract
Resting-state functional magnetic resonance imaging (rs-fMRI) offers a non-invasive approach to examining abnormal brain connectivity associated with brain disorders. Graph neural network (GNN) gains popularity in fMRI representation learning and brain disorder analysis with powerful graph representation capabilities. Training a general GNN often necessitates a large-scale dataset from multiple imaging centers/sites, but centralizing multi-site data generally faces inherent challenges related to data privacy, security, and storage burden. Federated Learning (FL) enables collaborative model training without centralized multi-site fMRI data. Unfortunately, previous FL approaches for fMRI analysis often ignore site-specificity, including demographic factors such as age, gender, and education level. To this end, we propose a specificity-aware federated graph learning (SFGL) framework for rs-fMRI analysis and automated brain disorder identification, with a server and multiple clients/sites for federated model aggregation and prediction. At each client, our model consists of a shared and a personalized branch, where parameters of the shared branch are sent to the server while those of the personalized branch remain local. This can facilitate knowledge sharing among sites and also helps preserve site specificity. In the shared branch, we employ a spatio-temporal attention graph isomorphism network to learn dynamic fMRI representations. In the personalized branch, we integrate vectorized demographic information (i.e., age, gender, and education years) and functional connectivity networks to preserve site-specific characteristics. Representations generated by the two branches are then fused for classification. Experimental results on two fMRI datasets with a total of 1218 subjects suggest that SFGL outperforms several state-of-the-art approaches.
Collapse
Affiliation(s)
- Junhao Zhang
- School of Mathematics Science, Liaocheng University, Liaocheng, Shandong, 252000, China
| | - Qianqian Wang
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA
| | - Xiaochuan Wang
- School of Mathematics Science, Liaocheng University, Liaocheng, Shandong, 252000, China
| | - Lishan Qiao
- School of Mathematics Science, Liaocheng University, Liaocheng, Shandong, 252000, China; School of Computer Science and Technology, Shandong Jianzhu University, Jinan, Shandong, 250101, China.
| | - Mingxia Liu
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA.
| |
Collapse
|
10
|
Guo H, Jian S, Zhou Y, Chen X, Chen J, Zhou J, Huang Y, Ma G, Li X, Ning Y, Wu F, Wu K. Discriminative analysis of schizophrenia patients using an integrated model combining 3D CNN with 2D CNN: A multimodal MR image and connectomics analysis. Brain Res Bull 2024; 206:110846. [PMID: 38104672 DOI: 10.1016/j.brainresbull.2023.110846] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Revised: 11/20/2023] [Accepted: 12/12/2023] [Indexed: 12/19/2023]
Abstract
OBJECTIVE Few studies have applied deep learning to the discriminative analysis of schizophrenia (SZ) patients using the fusional features of multimodal MRI data. Here, we proposed an integrated model combining a 3D convolutional neural network (CNN) with a 2D CNN to classify SZ patients. METHOD Structural MRI (sMRI) and resting-state functional MRI (rs-fMRI) data were acquired for 140 SZ patients and 205 normal controls. We computed structural connectivity (SC) from the sMRI data as well as functional connectivity (FC), amplitude of low-frequency fluctuation (ALFF), and regional homogeneity (ReHo) from the rs-fMRI data. The 3D images of T1, ReHo, and ALFF were used as the inputs for the 3D CNN model, while the SC and FC matrices were used as the inputs for the 2D CNN model. Moreover, we added squeeze and excitation blocks (SE-blocks) to each layer of the integrated model and used a support vector machine (SVM) to replace the softmax classifier. RESULTS The integrated model proposed in this study, using the fusional features of the T1 images, and the matrices of FC, showed the best performance. The use of the SE-blocks and SVM classifiers significantly improved the performance of the integrated model, in which the accuracy, sensitivity, specificity, area under the curve, and F1-score were 89.86%, 86.21%, 92.50%, 89.35%, and 87.72%, respectively. CONCLUSIONS Our findings indicated that an integrated model combining 3D CNN with 2D CNN is a promising method to improve the classification performance of SZ patients and has potential for the clinical diagnosis of psychiatric diseases.
Collapse
Affiliation(s)
- Haiman Guo
- School of Biomedical Sciences and Engineering, South China University of Technology, Guangzhou International Campus, Guangzhou 511442, China
| | - Shuyi Jian
- School of Biomedical Sciences and Engineering, South China University of Technology, Guangzhou International Campus, Guangzhou 511442, China
| | - Yubin Zhou
- School of Biomedical Sciences and Engineering, South China University of Technology, Guangzhou International Campus, Guangzhou 511442, China
| | - Xiaoyi Chen
- School of Biomedical Sciences and Engineering, South China University of Technology, Guangzhou International Campus, Guangzhou 511442, China
| | - Jinbiao Chen
- School of Biomedical Sciences and Engineering, South China University of Technology, Guangzhou International Campus, Guangzhou 511442, China
| | - Jing Zhou
- School of Material Sciences and Engineering, South China University of Technology, Guangzhou 510610, China; Guangdong Engineering Technology Research Center for Translational Medicine of Mental Disorders, Guangzhou 510370, China; Guangdong Engineering Technology Research Center for Diagnosis and Rehabilitation of Dementia, Guangzhou 510500, China
| | - Yuanyuan Huang
- The Affiliated Brain Hospital of Guangzhou Medical University, Guangzhou 510370, China; Guangdong Engineering Technology Research Center for Translational Medicine of Mental Disorders, Guangzhou 510370, China
| | - Guolin Ma
- Department of Radiology, China-Japan Friendship Hospital, Beijing 100029, China
| | - Xiaobo Li
- Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ, USA
| | - Yuping Ning
- The Affiliated Brain Hospital of Guangzhou Medical University, Guangzhou 510370, China; Guangdong Engineering Technology Research Center for Translational Medicine of Mental Disorders, Guangzhou 510370, China
| | - Fengchun Wu
- The Affiliated Brain Hospital of Guangzhou Medical University, Guangzhou 510370, China; Guangdong Engineering Technology Research Center for Translational Medicine of Mental Disorders, Guangzhou 510370, China.
| | - Kai Wu
- School of Biomedical Sciences and Engineering, South China University of Technology, Guangzhou International Campus, Guangzhou 511442, China; National Engineering Research Center for Tissue Restoration and Reconstruction, South China University of Technology, Guangzhou 510006, China; Guangdong Province Key Laboratory of Biomedical Engineering, South China University of Technology, Guangzhou 510006, China; Department of Nuclear Medicine and Radiology, Institute of Development, Aging and Cancer, Tohoku University, Sendai 980-8575, Japan.
| |
Collapse
|
11
|
Lin C, Huang C, Chang W, Chang Y, Liu H, Ng S, Lin H, Lee TM, Lee S, Wu S. Predicting suicidality in late-life depression by 3D convolutional neural network and cross-sample entropy analysis of resting-state fMRI. Brain Behav 2024; 14:e3348. [PMID: 38376042 PMCID: PMC10790060 DOI: 10.1002/brb3.3348] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Revised: 11/19/2023] [Accepted: 11/20/2023] [Indexed: 02/21/2024] Open
Abstract
BACKGROUND Predicting suicide is a pressing issue among older adults; however, predicting its risk is difficult. Capitalizing on the recent development of machine learning, considerable progress has been made in predicting complex behavior such as suicide. As depression remained the strongest risk for suicide, we aimed to apply deep learning algorithms to identify suicidality in a group with late-life depression (LLD). METHODS We enrolled 83 patients with LLD, 35 of which were non-suicidal and 48 were suicidal, including 26 with only suicidal ideation and 22 with past suicide attempts, for resting-state functional magnetic resonance imaging (MRI). Cross-sample entropy (CSE) analysis was conducted to examine the complexity of MRI signals among brain regions. Three-dimensional (3D) convolutional neural networks (CNNs) were used, and the classification accuracy in each brain region was averaged to predict suicidality after sixfold cross-validation. RESULTS We found brain regions with a mean accuracy above 75% to predict suicidality located mostly in default mode, fronto-parietal, and cingulo-opercular resting-state networks. The models with right amygdala and left caudate provided the most reliable accuracy in all cross-validation folds, indicating their neurobiological importance in late-life suicide. CONCLUSION Combining CSE analysis and the 3D CNN, several brain regions were found to be associated with suicidality.
Collapse
Affiliation(s)
- Chemin Lin
- Department of PsychiatryKeelung Chang Gung Memorial HospitalKeelungTaiwan
- College of MedicineChang Gung UniversityTaoyuanTaiwan
- Community Medicine Research CenterChang Gung Memorial HospitalKeelungTaiwan
| | - Chih‐Mao Huang
- Department of Biological Science and TechnologyNational Yang Ming Chiao Tung UniversityHsinchuTaiwan
| | - Wei Chang
- Department of Engineering and System ScienceNational Tsing Hua UniversityHsinchuTaiwan
| | - You‐Xun Chang
- Department of Engineering and System ScienceNational Tsing Hua UniversityHsinchuTaiwan
| | - Ho‐Ling Liu
- Community Medicine Research CenterChang Gung Memorial HospitalKeelungTaiwan
- Department of Imaging PhysicsUniversity of Texas MD Anderson Cancer CenterHoustonTexasUSA
| | - Shu‐Hang Ng
- Department of Head and Neck Oncology GroupLinkou Chang Gung Memorial Hospital and Chang Gung UniversityTaoyuanTaiwan
- Department of Diagnostic RadiologyLinkou Chang Gung Memorial Hospital and Chang Gung UniversityTaoyuanTaiwan
| | - Huang‐Li Lin
- Department of PsychiatryLinkou Chang Gung Memorial HospitalTaoyuanTaiwan
| | - Tatia Mei‐Chun Lee
- Community Medicine Research CenterChang Gung Memorial HospitalKeelungTaiwan
- Laboratory of Neuropsychology and Human NeuroscienceThe University of Hong KongPok Fu LamHong Kong
- State Key Laboratory of Brain and Cognitive ScienceThe University of Hong KongPok Fu LamHong Kong
| | - Shwu‐Hua Lee
- Department of PsychiatryLinkou Chang Gung Memorial HospitalTaoyuanTaiwan
| | - Shun‐Chi Wu
- Department of Engineering and System ScienceNational Tsing Hua UniversityHsinchuTaiwan
| |
Collapse
|
12
|
Yin G, Chang Y, Zhao Y, Liu C, Yin M, Fu Y, Shi D, Wang L, Jin L, Huang J, Li D, Niu Y, Wang B, Tan S. Automatic recognition of schizophrenia from brain-network features using graph convolutional neural network. Asian J Psychiatr 2023; 87:103687. [PMID: 37418809 DOI: 10.1016/j.ajp.2023.103687] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Revised: 06/25/2023] [Accepted: 06/30/2023] [Indexed: 07/09/2023]
Abstract
Schizophrenia is a severe mental illness that imposes considerable economic burden on families and society. However, its clinical diagnosis primarily relies on scales and doctors' clinical experience and lacks an objective and accurate diagnostic approach. In recent years, graph convolutional neural networks (GCN) have been used to assist in psychiatric diagnosis owing to their ability to learn spatial-association information. Therefore, this study proposes a schizophrenia automatic recognition model based on graph convolutional neural network. Herein, the resting-state electroencephalography (EEG) data of 103 first-episode schizophrenia patients and 92 normal controls (NCs) were obtained. The automatic recognition model was trained with a nodal feature matrix that comprised the time and frequency-domain features of the EEG signals and local features of the brain network. The most significant regions that contributed to the model classification were identified, and the correlation between the node topological features of each significant region and clinical evaluation metrics was explored. Experiments were conducted to evaluate the performance of the model using 10-fold cross-validation. The best performance in the theta frequency band with a 6 s epoch length and phase-locked value. The recognition accuracy was 90.01%. The most significant region for identifying with first-episode schizophrenia patients and NCs was located in the parietal lobe. The results of this study verify the applicability of the proposed novel method for the identification and diagnosis of schizophrenia.
Collapse
Affiliation(s)
- Guimei Yin
- College of Computer Science and Technology, Taiyuan Normal University, City Jinzhong 030619 Shanxi, China
| | - Ying Chang
- Departs of Ultrasonography, Xuan Wu Hospital, Capital Medical University, Beijing 100053, China
| | - Yanli Zhao
- Peking University Huilonguan Clinical Medical School, Psychiatry Research Center, Beijing Huilongguan Hospital, Beijing 100096, China
| | - Chenxu Liu
- College of Computer Science and Technology, Taiyuan Normal University, City Jinzhong 030619 Shanxi, China
| | - Mengzhen Yin
- College of Computer Science and Technology, Taiyuan Normal University, City Jinzhong 030619 Shanxi, China
| | - Yongcan Fu
- College of Computer Science and Technology, Taiyuan Normal University, City Jinzhong 030619 Shanxi, China
| | - Dongli Shi
- College of Computer Science and Technology, Taiyuan Normal University, City Jinzhong 030619 Shanxi, China
| | - Lin Wang
- College of Computer Science and Technology, Taiyuan Normal University, City Jinzhong 030619 Shanxi, China
| | - Lizhong Jin
- Taiyuan University of Science and Technology, Taiyuan 030024 Shanxi, China
| | - Jie Huang
- Peking University Huilonguan Clinical Medical School, Psychiatry Research Center, Beijing Huilongguan Hospital, Beijing 100096, China
| | - Dandan Li
- Taiyuan University of Technology, Jinzhong 030600 Shanxi, China
| | - Yan Niu
- Taiyuan University of Technology, Jinzhong 030600 Shanxi, China
| | - Bin Wang
- Taiyuan University of Technology, Jinzhong 030600 Shanxi, China.
| | - Shuping Tan
- Peking University Huilonguan Clinical Medical School, Psychiatry Research Center, Beijing Huilongguan Hospital, Beijing 100096, China.
| |
Collapse
|
13
|
Sun J, Dong QX, Wang SW, Zheng YB, Liu XX, Lu TS, Yuan K, Shi J, Hu B, Lu L, Han Y. Artificial intelligence in psychiatry research, diagnosis, and therapy. Asian J Psychiatr 2023; 87:103705. [PMID: 37506575 DOI: 10.1016/j.ajp.2023.103705] [Citation(s) in RCA: 15] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Revised: 07/16/2023] [Accepted: 07/20/2023] [Indexed: 07/30/2023]
Abstract
Psychiatric disorders are now responsible for the largest proportion of the global burden of disease, and even more challenges have been seen during the COVID-19 pandemic. Artificial intelligence (AI) is commonly used to facilitate the early detection of disease, understand disease progression, and discover new treatments in the fields of both physical and mental health. The present review provides a broad overview of AI methodology and its applications in data acquisition and processing, feature extraction and characterization, psychiatric disorder classification, potential biomarker detection, real-time monitoring, and interventions in psychiatric disorders. We also comprehensively summarize AI applications with regard to the early warning, diagnosis, prognosis, and treatment of specific psychiatric disorders, including depression, schizophrenia, autism spectrum disorder, attention-deficit/hyperactivity disorder, addiction, sleep disorders, and Alzheimer's disease. The advantages and disadvantages of AI in psychiatry are clarified. We foresee a new wave of research opportunities to facilitate and improve AI technology and its long-term implications in psychiatry during and after the COVID-19 era.
Collapse
Affiliation(s)
- Jie Sun
- Pain Medicine Center, Peking University Third Hospital, Beijing 100191, China; Peking University Sixth Hospital, Peking University Institute of Mental Health, NHC Key Laboratory of Mental Health (Peking University), National Clinical Research Center for Mental Disorders (Peking University Sixth Hospital), Beijing 100191, China
| | - Qun-Xi Dong
- School of Medical Technology, Beijing Institute of Technology, Beijing 100081, China
| | - San-Wang Wang
- Peking University Sixth Hospital, Peking University Institute of Mental Health, NHC Key Laboratory of Mental Health (Peking University), National Clinical Research Center for Mental Disorders (Peking University Sixth Hospital), Beijing 100191, China; Department of Psychiatry, Renmin Hospital of Wuhan University, Wuhan 430060, China
| | - Yong-Bo Zheng
- Peking University Sixth Hospital, Peking University Institute of Mental Health, NHC Key Laboratory of Mental Health (Peking University), National Clinical Research Center for Mental Disorders (Peking University Sixth Hospital), Beijing 100191, China; Peking-Tsinghua Center for Life Sciences and PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing 100871, China
| | - Xiao-Xing Liu
- Peking University Sixth Hospital, Peking University Institute of Mental Health, NHC Key Laboratory of Mental Health (Peking University), National Clinical Research Center for Mental Disorders (Peking University Sixth Hospital), Beijing 100191, China
| | - Tang-Sheng Lu
- National Institute on Drug Dependence and Beijing Key Laboratory of Drug Dependence Research, Peking University, Beijing 100191, China
| | - Kai Yuan
- Peking University Sixth Hospital, Peking University Institute of Mental Health, NHC Key Laboratory of Mental Health (Peking University), National Clinical Research Center for Mental Disorders (Peking University Sixth Hospital), Beijing 100191, China
| | - Jie Shi
- National Institute on Drug Dependence and Beijing Key Laboratory of Drug Dependence Research, Peking University, Beijing 100191, China
| | - Bin Hu
- School of Medical Technology, Beijing Institute of Technology, Beijing 100081, China.
| | - Lin Lu
- Peking University Sixth Hospital, Peking University Institute of Mental Health, NHC Key Laboratory of Mental Health (Peking University), National Clinical Research Center for Mental Disorders (Peking University Sixth Hospital), Beijing 100191, China; Peking-Tsinghua Center for Life Sciences and PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing 100871, China.
| | - Ying Han
- National Institute on Drug Dependence and Beijing Key Laboratory of Drug Dependence Research, Peking University, Beijing 100191, China.
| |
Collapse
|
14
|
Li ML, Zhang F, Chen YY, Luo HY, Quan ZW, Wang YF, Huang LT, Wang JH. A state-of-the-art review of functional magnetic resonance imaging technique integrated with advanced statistical modeling and machine learning for primary headache diagnosis. Front Hum Neurosci 2023; 17:1256415. [PMID: 37746052 PMCID: PMC10513061 DOI: 10.3389/fnhum.2023.1256415] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Accepted: 08/14/2023] [Indexed: 09/26/2023] Open
Abstract
Primary headache is a very common and burdensome functional headache worldwide, which can be classified as migraine, tension-type headache (TTH), trigeminal autonomic cephalalgia (TAC), and other primary headaches. Managing and treating these different categories require distinct approaches, and accurate diagnosis is crucial. Functional magnetic resonance imaging (fMRI) has become a research hotspot to explore primary headache. By examining the interrelationships between activated brain regions and improving temporal and spatial resolution, fMRI can distinguish between primary headaches and their subtypes. Currently the most commonly used is the cortical brain mapping technique, which is based on blood oxygen level-dependent functional magnetic resonance imaging (BOLD-fMRI). This review sheds light on the state-of-the-art advancements in data analysis based on fMRI technology for primary headaches along with their subtypes. It encompasses not only the conventional analysis methodologies employed to unravel pathophysiological mechanisms, but also deep-learning approaches that integrate these techniques with advanced statistical modeling and machine learning. The aim is to highlight cutting-edge fMRI technologies and provide new insights into the diagnosis of primary headaches.
Collapse
Affiliation(s)
- Ming-Lin Li
- Department of Family Medicine, Shengjing Hospital of China Medical University, Shenyang, Liaoning, China
| | - Fei Zhang
- Department of Family Medicine, Shengjing Hospital of China Medical University, Shenyang, Liaoning, China
| | - Yi-Yang Chen
- Department of Family Medicine, Shengjing Hospital of China Medical University, Shenyang, Liaoning, China
- Department of Family Medicine, Liaoning Health Industry Group Fukuang General Hospital, Fushun, Liaoning, China
| | - Han-Yong Luo
- Department of Family Medicine, Shengjing Hospital of China Medical University, Shenyang, Liaoning, China
| | - Zi-Wei Quan
- Department of Family Medicine, Shengjing Hospital of China Medical University, Shenyang, Liaoning, China
| | - Yi-Fei Wang
- Department of Family Medicine, Shengjing Hospital of China Medical University, Shenyang, Liaoning, China
| | - Le-Tian Huang
- Department of Oncology, Shengjing Hospital of China Medical University, Shenyang, Liaoning, China
| | - Jia-He Wang
- Department of Family Medicine, Shengjing Hospital of China Medical University, Shenyang, Liaoning, China
| |
Collapse
|
15
|
Tang Y, Tong G, Xiong X, Zhang C, Zhang H, Yang Y. Multi-site diagnostic classification of Autism spectrum disorder using adversarial deep learning on resting-state fMRI. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104892] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/29/2023]
|
16
|
Porter A, Fei S, Damme KSF, Nusslock R, Gratton C, Mittal VA. A meta-analysis and systematic review of single vs. multimodal neuroimaging techniques in the classification of psychosis. Mol Psychiatry 2023; 28:3278-3292. [PMID: 37563277 PMCID: PMC10618094 DOI: 10.1038/s41380-023-02195-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Revised: 07/11/2023] [Accepted: 07/17/2023] [Indexed: 08/12/2023]
Abstract
BACKGROUND Psychotic disorders are characterized by structural and functional abnormalities in brain networks. Neuroimaging techniques map and characterize such abnormalities using unique features (e.g., structural integrity, coactivation). However, it is unclear if a specific method, or a combination of modalities, is particularly effective in identifying differences in brain networks of someone with a psychotic disorder. METHODS A systematic meta-analysis evaluated machine learning classification of schizophrenia spectrum disorders in comparison to healthy control participants using various neuroimaging modalities (i.e., T1-weighted imaging (T1), diffusion tensor imaging (DTI), resting state functional connectivity (rs-FC), or some combination (multimodal)). Criteria for manuscript inclusion included whole-brain analyses and cross-validation to provide a complete picture regarding the predictive ability of large-scale brain systems in psychosis. For this meta-analysis, we searched Ovid MEDLINE, PubMed, PsychInfo, Google Scholar, and Web of Science published between inception and March 13th 2023. Prediction results were averaged for studies using the same dataset, but parallel analyses were run that included studies with pooled sample across many datasets. We assessed bias through funnel plot asymmetry. A bivariate regression model determined whether differences in imaging modality, demographics, and preprocessing methods moderated classification. Separate models were run for studies with internal prediction (via cross-validation) and external prediction. RESULTS 93 studies were identified for quantitative review (30 T1, 9 DTI, 40 rs-FC, and 14 multimodal). As a whole, all modalities reliably differentiated those with schizophrenia spectrum disorders from controls (OR = 2.64 (95%CI = 2.33 to 2.95)). However, classification was relatively similar across modalities: no differences were seen across modalities in the classification of independent internal data, and a small advantage was seen for rs-FC studies relative to T1 studies in classification in external datasets. We found large amounts of heterogeneity across results resulting in significant signs of bias in funnel plots and Egger's tests. Results remained similar, however, when studies were restricted to those with less heterogeneity, with continued small advantages for rs-FC relative to structural measures. Notably, in all cases, no significant differences were seen between multimodal and unimodal approaches, with rs-FC and unimodal studies reporting largely overlapping classification performance. Differences in demographics and analysis or denoising were not associated with changes in classification scores. CONCLUSIONS The results of this study suggest that neuroimaging approaches have promise in the classification of psychosis. Interestingly, at present most modalities perform similarly in the classification of psychosis, with slight advantages for rs-FC relative to structural modalities in some specific cases. Notably, results differed substantially across studies, with suggestions of biased effect sizes, particularly highlighting the need for more studies using external prediction and large sample sizes. Adopting more rigorous and systematized standards will add significant value toward understanding and treating this critical population.
Collapse
Affiliation(s)
- Alexis Porter
- Department of Psychology, Northwestern University, Evanston, IL, USA.
| | - Sihan Fei
- Department of Psychology, Northwestern University, Evanston, IL, USA
| | - Katherine S F Damme
- Department of Psychology, Northwestern University, Evanston, IL, USA
- Institute for Innovations in Developmental Sciences, Northwestern University, Evanston and Chicago, IL, USA
| | - Robin Nusslock
- Department of Psychology, Northwestern University, Evanston, IL, USA
| | - Caterina Gratton
- Department of Psychology, Florida State University, Tallahassee, FL, USA
| | - Vijay A Mittal
- Department of Psychology, Northwestern University, Evanston, IL, USA
- Institute for Innovations in Developmental Sciences, Northwestern University, Evanston and Chicago, IL, USA
- Department of Psychiatry, Northwestern University, Chicago, IL, USA
- Medical Social Sciences, Northwestern University, Chicago, IL, USA
- Institute for Policy Research, Northwestern University, Chicago, IL, USA
| |
Collapse
|
17
|
Chen X, Ke P, Huang Y, Zhou J, Li H, Peng R, Huang J, Liang L, Ma G, Li X, Ning Y, Wu F, Wu K. Discriminative analysis of schizophrenia patients using graph convolutional networks: A combined multimodal MRI and connectomics analysis. Front Neurosci 2023; 17:1140801. [PMID: 37090813 PMCID: PMC10117439 DOI: 10.3389/fnins.2023.1140801] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Accepted: 03/10/2023] [Indexed: 03/31/2023] Open
Abstract
IntroductionRecent studies in human brain connectomics with multimodal magnetic resonance imaging (MRI) data have widely reported abnormalities in brain structure, function and connectivity associated with schizophrenia (SZ). However, most previous discriminative studies of SZ patients were based on MRI features of brain regions, ignoring the complex relationships within brain networks.MethodsWe applied a graph convolutional network (GCN) to discriminating SZ patients using the features of brain region and connectivity derived from a combined multimodal MRI and connectomics analysis. Structural magnetic resonance imaging (sMRI) and resting-state functional magnetic resonance imaging (rs-fMRI) data were acquired from 140 SZ patients and 205 normal controls. Eighteen types of brain graphs were constructed for each subject using 3 types of node features, 3 types of edge features, and 2 brain atlases. We investigated the performance of 18 brain graphs and used the TopK pooling layers to highlight salient brain regions (nodes in the graph).ResultsThe GCN model, which used functional connectivity as edge features and multimodal features (sMRI + fMRI) of brain regions as node features, obtained the highest average accuracy of 95.8%, and outperformed other existing classification studies in SZ patients. In the explainability analysis, we reported that the top 10 salient brain regions, predominantly distributed in the prefrontal and occipital cortices, were mainly involved in the systems of emotion and visual processing.DiscussionOur findings demonstrated that GCN with a combined multimodal MRI and connectomics analysis can effectively improve the classification of SZ at an individual level, indicating a promising direction for the diagnosis of SZ patients. The code is available at https://github.com/CXY-scut/GCN-SZ.git.
Collapse
Affiliation(s)
- Xiaoyi Chen
- Department of Biomedical Engineering, School of Biomedical Sciences and Engineering, South China University of Technology, Guangzhou International Campus, Guangzhou, China
| | - Pengfei Ke
- Department of Biomedical Engineering, School of Biomedical Sciences and Engineering, South China University of Technology, Guangzhou International Campus, Guangzhou, China
| | - Yuanyuan Huang
- Department of Emotional Disorders, The Affiliated Brain Hospital of Guangzhou Medical University, Guangzhou, China
- Guangdong Engineering Technology Research Center for Translational Medicine of Mental Disorders, Guangzhou, China
| | - Jing Zhou
- School of Material Science and Engineering, South China University of Technology, Guangzhou, China
- National Engineering Research Center for Tissue Restoration and Reconstruction, South China University of Technology, Guangzhou, China
- Guangdong Province Key Laboratory of Biomedical Engineering, South China University of Technology, Guangzhou, China
| | - Hehua Li
- Department of Emotional Disorders, The Affiliated Brain Hospital of Guangzhou Medical University, Guangzhou, China
- Guangdong Engineering Technology Research Center for Translational Medicine of Mental Disorders, Guangzhou, China
| | - Runlin Peng
- Department of Biomedical Engineering, School of Biomedical Sciences and Engineering, South China University of Technology, Guangzhou International Campus, Guangzhou, China
| | - Jiayuan Huang
- Department of Biomedical Engineering, School of Biomedical Sciences and Engineering, South China University of Technology, Guangzhou International Campus, Guangzhou, China
| | - LiQing Liang
- Department of Biomedical Engineering, School of Biomedical Sciences and Engineering, South China University of Technology, Guangzhou International Campus, Guangzhou, China
| | - Guolin Ma
- Department of Radiology, China-Japan Friendship Hospital, Beijing, China
| | - Xiaobo Li
- Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ, United States
| | - Yuping Ning
- Guangdong Engineering Technology Research Center for Translational Medicine of Mental Disorders, Guangzhou, China
- Department of Psychosomatic, The Affiliated Brain Hospital of Guangzhou Medical University, Guangzhou, China
| | - Fengchun Wu
- Department of Psychiatry, The Affiliated Brain Hospital of Guangzhou Medical University, Guangzhou, China
- *Correspondence: Fengchun Wu,
| | - Kai Wu
- Department of Biomedical Engineering, School of Biomedical Sciences and Engineering, South China University of Technology, Guangzhou International Campus, Guangzhou, China
- National Engineering Research Center for Tissue Restoration and Reconstruction, South China University of Technology, Guangzhou, China
- Guangdong Province Key Laboratory of Biomedical Engineering, South China University of Technology, Guangzhou, China
- Department of Nuclear Medicine and Radiology, Institute of Development, Aging and Cancer, Tohoku University, Sendai, Japan
- Kai Wu,
| |
Collapse
|
18
|
Sharma M, Patel RK, Garg A, SanTan R, Acharya UR. Automated detection of schizophrenia using deep learning: a review for the last decade. Physiol Meas 2023; 44. [PMID: 36630717 DOI: 10.1088/1361-6579/acb24d] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2022] [Accepted: 01/11/2023] [Indexed: 01/12/2023]
Abstract
Schizophrenia (SZ) is a devastating mental disorder that disrupts higher brain functions like thought, perception, etc., with a profound impact on the individual's life. Deep learning (DL) can detect SZ automatically by learning signal data characteristics hierarchically without the need for feature engineering associated with traditional machine learning. We performed a systematic review of DL models for SZ detection. Various deep models like long short-term memory, convolution neural networks, AlexNet, etc., and composite methods have been published based on electroencephalographic signals, and structural and/or functional magnetic resonance imaging acquired from SZ patients and healthy patients control subjects in diverse public and private datasets. The studies, the study datasets, and model methodologies are reported in detail. In addition, the challenges of DL models for SZ diagnosis and future works are discussed.
Collapse
Affiliation(s)
- Manish Sharma
- Department of Electrical and Computer Science Engineering, Institute of Infrastructure Technology Research and Management, Ahmedabad 380026, India
| | - Ruchit Kumar Patel
- Department of Electrical and Computer Science Engineering, Institute of Infrastructure Technology Research and Management, Ahmedabad 380026, India
| | - Akshat Garg
- Department of Electrical and Computer Science Engineering, Institute of Infrastructure Technology Research and Management, Ahmedabad 380026, India
| | - Ru SanTan
- Department of Cardiology, National Heart Centre Singapore, Singapore 169609, Singapore
| | - U Rajendra Acharya
- Department of Electronics and Computer Engineering, Ngee Ann Polytechnic, Singapore 639798, Singapore.,Department of Bioinformatics and Medical Engineering, Asia University, Taichung 41354, Taiwan.,Department of Biomedical Engineering, School of Science and Technology, Singapore 639798, Singapore
| |
Collapse
|
19
|
Uyulan C, Erguzel TT, Turk O, Farhad S, Metin B, Tarhan N. A Class Activation Map-Based Interpretable Transfer Learning Model for Automated Detection of ADHD from fMRI Data. Clin EEG Neurosci 2023; 54:151-159. [PMID: 36052402 DOI: 10.1177/15500594221122699] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
Automatic detection of Attention Deficit Hyperactivity Disorder (ADHD) based on the functional Magnetic Resonance Imaging (fMRI) through Deep Learning (DL) is becoming a quite useful methodology due to the curse of-dimensionality problem of the data is solved. Also, this method proposes an invasive and robust solution to the variances in data acquisition and class distribution imbalances. In this paper, a transfer learning approach, specifically ResNet-50 type pre-trained 2D-Convolutional Neural Network (CNN) was used to automatically classify ADHD and healthy children. The results demonstrated that ResNet-50 architecture with 10-k cross-validation (CV) achieves an overall classification accuracy of 93.45%. The interpretation of the results was done via the Class Activation Map (CAM) analysis which showed that children with ADHD differed from controls in a wide range of brain areas including frontal, parietal and temporal lobes.
Collapse
Affiliation(s)
- Caglar Uyulan
- Department of Mechanical Engineering, Faculty of Engineering and Architecture, İzmir Katip Çelebi University, İzmir, Turkey
| | | | - Omer Turk
- Department of Computer Programming, Vocational School, Mardin Artuklu University, Mardin, Turkey
| | - Shams Farhad
- Department of Neuroscience, 232990Uskudar University, Istanbul, Turkey
| | - Baris Metin
- Department of Neuroscience, 232990Uskudar University, Istanbul, Turkey
| | - Nevzat Tarhan
- Department of Psychiatry, NPIstanbul Brain Hospital, Istanbul, Turkey
| |
Collapse
|
20
|
Balasubramanian K, Ramya K, Gayathri Devi K. Optimized adaptive neuro-fuzzy inference system based on hybrid grey wolf-bat algorithm for schizophrenia recognition from EEG signals. Cogn Neurodyn 2023; 17:133-151. [PMID: 36704627 PMCID: PMC9871147 DOI: 10.1007/s11571-022-09817-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2021] [Revised: 03/23/2022] [Accepted: 04/27/2022] [Indexed: 02/01/2023] Open
Abstract
Schizophrenia is a chronic mental disorder that impairs a person's thinking capacity, feelings and emotions, behavioural traits, etc., Emotional distortions, delusions, hallucinations, and incoherent speech are all some of the symptoms of schizophrenia, and cause disruption of routine activities. Computer-assisted diagnosis of schizophrenia is significantly needed to give its patients a higher quality of life. Hence, an improved adaptive neuro-fuzzy inference system based on the Hybrid Grey Wolf-Bat Algorithm for accurate prediction of schizophrenia from multi-channel EEG signals is presented in this study. The EEG signals are pre-processed using a Butterworth band pass filter and wICA initially, from which statistical, time-domain, frequency-domain, and spectral features are extracted. Discriminating features are selected using the ReliefF algorithm and are then forwarded to ANFIS for classification into either schizophrenic or normal. ANFIS is optimized by the Hybrid Grey Wolf-Bat Algorithm (HWBO) for better efficiency. The method is experimented on two separate EEG datasets-1 and 2, demonstrating an accuracy of 99.54% and 99.35%, respectively, with appreciable F1-score and MCC. Further experiments reveal the efficiency of the Hybrid Wolf-Bat algorithm in optimizing the ANFIS parameters when compared with traditional ANFIS model and other proven algorithms like genetic algorithm-ANFIS, particle optimization-ANFIS, crow search optimization algorithm-ANFIS and ant colony optimization algorithm-ANFIS, showing high R2 value and low RSME value. To provide a bias free classification, tenfold cross validation is performed which produced an accuracy of 97.8% and 98.5% on the two datasets respectively. Experimental outcomes demonstrate the superiority of the Hybrid Grey Wolf-Bat Algorithm over the similar techniques in predicting schizophrenia.
Collapse
Affiliation(s)
| | - K. Ramya
- PA College of Engineering and Technology, Pollachi, India
| | | |
Collapse
|
21
|
Classification of schizophrenia patients using a graph convolutional network: A combined functional MRI and connectomics analysis. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104293] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
22
|
Deep Learning Classification of Treatment Response in Diabetic Painful Neuropathy: A Combined Machine Learning and Magnetic Resonance Neuroimaging Methodological Study. Neuroinformatics 2023; 21:35-43. [PMID: 36018533 PMCID: PMC9931783 DOI: 10.1007/s12021-022-09603-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/17/2022] [Indexed: 10/15/2022]
Abstract
Functional magnetic resonance imaging (fMRI) has been shown successfully to assess and stratify patients with painful diabetic peripheral neuropathy (pDPN). This supports the idea of using neuroimaging as a mechanism-based technique to individualise therapy for patients with painful DPN. The aim of this study was to use deep learning to predict treatment response in patients with pDPN using resting state functional imaging (rs-fMRI). We divided 43 painful pDPN patients into responders and non-responders to lidocaine treatment (responders n = 29 and non-responders n = 14). We used rs-fMRI to extract functional connectivity features, using group independent component analysis (gICA), and performed automated treatment response deep learning classification with three-dimensional convolutional neural networks (3D-CNN). Using gICA we achieved an area under the receiver operating characteristic curve (AUC) of 96.60% and F1-Score of 95% in a ten-fold cross validation (CV) experiment using our described 3D-CNN algorithm. To our knowledge, this is the first study utilising deep learning methods to classify treatment response in pDPN.
Collapse
|
23
|
Automatic diagnosis of late-life depression by 3D convolutional neural networks and cross-sample Entropy analysis from resting-state fMRI. Brain Imaging Behav 2023; 17:125-135. [PMID: 36418676 PMCID: PMC9922223 DOI: 10.1007/s11682-022-00748-0] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2022] [Revised: 10/26/2022] [Accepted: 11/12/2022] [Indexed: 11/25/2022]
Abstract
Resting-state fMRI has been widely used in investigating the pathophysiology of late-life depression (LLD). Unlike the conventional linear approach, cross-sample entropy (CSE) analysis shows the nonlinear property in fMRI signals between brain regions. Moreover, recent advances in deep learning, such as convolutional neural networks (CNNs), provide a timely application for understanding LLD. Accurate and prompt diagnosis is essential in LLD; hence, this study aimed to combine CNN and CSE analysis to discriminate LLD patients and non-depressed comparison older adults based on brain resting-state fMRI signals. Seventy-seven older adults, including 49 patients and 28 comparison older adults, were included for fMRI scans. Three-dimensional CSEs with volumes corresponding to 90 seed regions of interest of each participant were developed and fed into models for disease classification and depression severity prediction. We obtained a diagnostic accuracy > 85% in the superior frontal gyrus (left dorsolateral and right orbital parts), left insula, and right middle occipital gyrus. With a mean root-mean-square error (RMSE) of 2.41, three separate models were required to predict depressive symptoms in the severe, moderate, and mild depression groups. The CSE volumes in the left inferior parietal lobule, left parahippocampal gyrus, and left postcentral gyrus performed best in each respective model. Combined complexity analysis and deep learning algorithms can classify patients with LLD from comparison older adults and predict symptom severity based on fMRI data. Such application can be utilized in precision medicine for disease detection and symptom monitoring in LLD.
Collapse
|
24
|
SupriyaPatro P, Goel T, VaraPrasad SA, Tanveer M, Murugan R. Lightweight 3D Convolutional Neural Network for Schizophrenia Diagnosis Using MRI Images and Ensemble Bagging Classifier. Cognit Comput 2022. [DOI: 10.1007/s12559-022-10093-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
|
25
|
Avberšek LK, Repovš G. Deep learning in neuroimaging data analysis: Applications, challenges, and solutions. FRONTIERS IN NEUROIMAGING 2022; 1:981642. [PMID: 37555142 PMCID: PMC10406264 DOI: 10.3389/fnimg.2022.981642] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/29/2022] [Accepted: 10/10/2022] [Indexed: 08/10/2023]
Abstract
Methods for the analysis of neuroimaging data have advanced significantly since the beginning of neuroscience as a scientific discipline. Today, sophisticated statistical procedures allow us to examine complex multivariate patterns, however most of them are still constrained by assuming inherent linearity of neural processes. Here, we discuss a group of machine learning methods, called deep learning, which have drawn much attention in and outside the field of neuroscience in recent years and hold the potential to surpass the mentioned limitations. Firstly, we describe and explain the essential concepts in deep learning: the structure and the computational operations that allow deep models to learn. After that, we move to the most common applications of deep learning in neuroimaging data analysis: prediction of outcome, interpretation of internal representations, generation of synthetic data and segmentation. In the next section we present issues that deep learning poses, which concerns multidimensionality and multimodality of data, overfitting and computational cost, and propose possible solutions. Lastly, we discuss the current reach of DL usage in all the common applications in neuroimaging data analysis, where we consider the promise of multimodality, capability of processing raw data, and advanced visualization strategies. We identify research gaps, such as focusing on a limited number of criterion variables and the lack of a well-defined strategy for choosing architecture and hyperparameters. Furthermore, we talk about the possibility of conducting research with constructs that have been ignored so far or/and moving toward frameworks, such as RDoC, the potential of transfer learning and generation of synthetic data.
Collapse
Affiliation(s)
- Lev Kiar Avberšek
- Department of Psychology, Faculty of Arts, University of Ljubljana, Ljubljana, Slovenia
| | | |
Collapse
|
26
|
Liu F, Meamardoost S, Gunawan R, Komiyama T, Mewes C, Zhang Y, Hwang E, Wang L. Deep learning for neural decoding in motor cortex. J Neural Eng 2022; 19. [PMID: 36148535 DOI: 10.1088/1741-2552/ac8fb5] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2022] [Accepted: 09/06/2022] [Indexed: 11/12/2022]
Abstract
Objective. Neural decoding is an important tool in neural engineering and neural data analysis. Of various machine learning algorithms adopted for neural decoding, the recently introduced deep learning is promising to excel. Therefore, we sought to apply deep learning to decode movement trajectories from the activity of motor cortical neurons.Approach. In this paper, we assessed the performance of deep learning methods in three different decoding schemes, concurrent, time-delay, and spatiotemporal. In the concurrent decoding scheme where the input to the network is the neural activity coincidental to the movement, deep learning networks including artificial neural network (ANN) and long-short term memory (LSTM) were applied to decode movement and compared with traditional machine learning algorithms. Both ANN and LSTM were further evaluated in the time-delay decoding scheme in which temporal delays are allowed between neural signals and movements. Lastly, in the spatiotemporal decoding scheme, we trained convolutional neural network (CNN) to extract movement information from images representing the spatial arrangement of neurons, their activity, and connectomes (i.e. the relative strengths of connectivity between neurons) and combined CNN and ANN to develop a hybrid spatiotemporal network. To reveal the input features of the CNN in the hybrid network that deep learning discovered for movement decoding, we performed a sensitivity analysis and identified specific regions in the spatial domain.Main results. Deep learning networks (ANN and LSTM) outperformed traditional machine learning algorithms in the concurrent decoding scheme. The results of ANN and LSTM in the time-delay decoding scheme showed that including neural data from time points preceding movement enabled decoders to perform more robustly when the temporal relationship between the neural activity and movement dynamically changes over time. In the spatiotemporal decoding scheme, the hybrid spatiotemporal network containing the concurrent ANN decoder outperformed single-network concurrent decoders.Significance. Taken together, our study demonstrates that deep learning could become a robust and effective method for the neural decoding of behavior.
Collapse
Affiliation(s)
- Fangyu Liu
- Department of Civil and Environmental Engineering, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, United States of America
| | - Saber Meamardoost
- Department of Chemical and Biological Engineering, University at Buffalo, Buffalo, NY 14260, United States of America
| | - Rudiyanto Gunawan
- Department of Chemical and Biological Engineering, University at Buffalo, Buffalo, NY 14260, United States of America
| | - Takaki Komiyama
- Department of Neurobiology, Center for Neural Circuits and Behavior, and Department of Neurosciences, University of California San Diego, La Jolla, CA 92093, United States of America
| | - Claudia Mewes
- Department of Physics and Astronomy, University of Alabama, Tuscaloosa, AL 35487, United States of America
| | - Ying Zhang
- Department of Cell and Molecular Biology, University of Rhode Island, Kingston, RI 02881, United States of America
| | - EunJung Hwang
- Department of Neurobiology, Center for Neural Circuits and Behavior, and Department of Neurosciences, University of California San Diego, La Jolla, CA 92093, United States of America.,Cell Biology and Anatomy Discipline, Center for Brain Function and Repair, Chicago Medical School, Rosalind Franklin University of Medicine and Science, North Chicago, IL 60064, United States of America
| | - Linbing Wang
- Department of Civil and Environmental Engineering, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, United States of America
| |
Collapse
|
27
|
Rahaman MA, Chen J, Fu Z, Lewis N, Iraji A, van Erp TGM, Calhoun VD. Deep multimodal predictome for studying mental disorders. Hum Brain Mapp 2022; 44:509-522. [PMID: 36574598 PMCID: PMC9842924 DOI: 10.1002/hbm.26077] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2022] [Revised: 07/29/2022] [Accepted: 08/16/2022] [Indexed: 01/25/2023] Open
Abstract
Characterizing neuropsychiatric disorders is challenging due to heterogeneity in the population. We propose combining structural and functional neuroimaging and genomic data in a multimodal classification framework to leverage their complementary information. Our objectives are two-fold (i) to improve the classification of disorders and (ii) to introspect the concepts learned to explore underlying neural and biological mechanisms linked to mental disorders. Previous multimodal studies have focused on naïve neural networks, mostly perceptron, to learn modality-wise features and often assume equal contribution from each modality. Our focus is on the development of neural networks for feature learning and implementing an adaptive control unit for the fusion phase. Our mid fusion with attention model includes a multilayer feed-forward network, an autoencoder, a bi-directional long short-term memory unit with attention as the features extractor, and a linear attention module for controlling modality-specific influence. The proposed model acquired 92% (p < .0001) accuracy in schizophrenia prediction, outperforming several other state-of-the-art models applied to unimodal or multimodal data. Post hoc feature analyses uncovered critical neural features and genes/biological pathways associated with schizophrenia. The proposed model effectively combines multimodal neuroimaging and genomics data for predicting mental disorders. Interpreting salient features identified by the model may advance our understanding of their underlying etiological mechanisms.
Collapse
Affiliation(s)
- Md Abdur Rahaman
- Department of Computational Science and EngineeringGeorgia Institute of TechnologyAtlantaGeorgiaUSA,Tri‐Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS)Georgia State University, Georgia Institute of Technology, Emory UniversityAtlantaGeorgiaUSA
| | - Jiayu Chen
- Tri‐Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS)Georgia State University, Georgia Institute of Technology, Emory UniversityAtlantaGeorgiaUSA
| | - Zening Fu
- Tri‐Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS)Georgia State University, Georgia Institute of Technology, Emory UniversityAtlantaGeorgiaUSA
| | - Noah Lewis
- Department of Computational Science and EngineeringGeorgia Institute of TechnologyAtlantaGeorgiaUSA,Tri‐Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS)Georgia State University, Georgia Institute of Technology, Emory UniversityAtlantaGeorgiaUSA
| | - Armin Iraji
- Tri‐Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS)Georgia State University, Georgia Institute of Technology, Emory UniversityAtlantaGeorgiaUSA
| | - Theo G. M. van Erp
- Clinical Translational Neuroscience Laboratory, Department of Psychiatry and Human BehaviorUniversity of California IrvineIrvineCaliforniaUSA,Center for the Neurobiology of Learning and MemoryUniversity of California IrvineIrvineCaliforniaUSA
| | - Vince D. Calhoun
- Department of Computational Science and EngineeringGeorgia Institute of TechnologyAtlantaGeorgiaUSA,Tri‐Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS)Georgia State University, Georgia Institute of Technology, Emory UniversityAtlantaGeorgiaUSA
| |
Collapse
|
28
|
Multimodal MRI-Based Whole-Brain Assessment in Patients In Anoxoischemic Coma by Using 3D Convolutional Neural Networks. Neurocrit Care 2022; 37:303-312. [PMID: 35876960 PMCID: PMC9343298 DOI: 10.1007/s12028-022-01525-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2021] [Accepted: 04/20/2022] [Indexed: 11/17/2022]
Abstract
Background There is an unfulfilled need to find the best way to automatically capture, analyze, organize, and merge structural and functional brain magnetic resonance imaging (MRI) data to ultimately extract relevant signals that can assist the medical decision process at the bedside of patients in postanoxic coma. We aimed to develop and validate a deep learning model to leverage multimodal 3D MRI whole-brain times series for an early evaluation of brain damages related to anoxoischemic coma. Methods This proof-of-concept, prospective, cohort study was undertaken at the intensive care unit affiliated with the University Hospital (Toulouse, France), between March 2018 and May 2020. All patients were scanned in coma state at least 2 days (4 ± 2 days) after cardiac arrest. Over the same period, age-matched healthy volunteers were recruited and included. Brain MRI quantification encompassed both “functional data” from regions of interest (precuneus and posterior cingulate cortex) with whole-brain functional connectivity analysis and “structural data” (gray matter volume, T1-weighted, fractional anisotropy, and mean diffusivity). A specifically designed 3D convolutional neuronal network (CNN) was created to allow conscious state discrimination (coma vs. controls) by using raw MRI indices as the input. A voxel-wise visualization method based on the study of convolutional filters was applied to support CNN outcome. The Ethics Committee of the University Teaching Hospital of Toulouse, France (2018-A31) approved the study and informed consent was obtained from all participants. Results The final cohort consisted of 29 patients in postanoxic coma and 34 healthy volunteers. Coma patients were successfully discerned from controls by using 3D CNN in combination with different MR indices. The best accuracy was achieved by functional MRI data, in particular with resting-state functional MRI of the posterior cingulate cortex, with an accuracy of 0.96 (range 0.94–0.98) on the test set from 10-time repeated tenfold cross-validation. Even more satisfactory performances were achieved through the majority voting strategy, which was able to compensate for mistakes from single MR indices. Visualization maps allowed us to identify the most relevant regions for each MRI index, notably regions previously described as possibly being involved in consciousness emergence. Interestingly, a posteriori analysis of misclassified patients indicated that they may present some common functional MRI traits with controls, which suggests further favorable outcomes. Conclusions A fully automated identification of clinically relevant signals from complex multimodal neuroimaging data is a major research topic that may bring a radical paradigm shift in the neuroprognostication of patients with severe brain injury. We report for the first time a successful discrimination between patients in postanoxic coma patients from people serving as controls by using 3D CNN whole-brain structural and functional MRI data. Clinical Trial Numberhttp://ClinicalTrials.gov (No. NCT03482115). Supplementary Information The online version contains supplementary material available at 10.1007/s12028-022-01525-z.
Collapse
|
29
|
Sadeghi D, Shoeibi A, Ghassemi N, Moridian P, Khadem A, Alizadehsani R, Teshnehlab M, Gorriz JM, Khozeimeh F, Zhang YD, Nahavandi S, Acharya UR. An overview of artificial intelligence techniques for diagnosis of Schizophrenia based on magnetic resonance imaging modalities: Methods, challenges, and future works. Comput Biol Med 2022; 146:105554. [DOI: 10.1016/j.compbiomed.2022.105554] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Revised: 04/11/2022] [Accepted: 04/11/2022] [Indexed: 12/21/2022]
|
30
|
Lin QH, Niu YW, Sui J, Zhao WD, Zhuo C, Calhoun VD. SSPNet: An interpretable 3D-CNN for classification of schizophrenia using phase maps of resting-state complex-valued fMRI data. Med Image Anal 2022; 79:102430. [PMID: 35397470 DOI: 10.1016/j.media.2022.102430] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2020] [Revised: 03/16/2022] [Accepted: 03/18/2022] [Indexed: 01/05/2023]
Abstract
Convolutional neural networks (CNNs) have shown promising results in classifying individuals with mental disorders such as schizophrenia using resting-state fMRI data. However, complex-valued fMRI data is rarely used since additional phase data introduces high-level noise though it is potentially useful information for the context of classification. As such, we propose to use spatial source phase (SSP) maps derived from complex-valued fMRI data as the CNN input. The SSP maps are not only less noisy, but also more sensitive to spatial activation changes caused by mental disorders than magnitude maps. We build a 3D-CNN framework with two convolutional layers (named SSPNet) to fully explore the 3D structure and voxel-level relationships from the SSP maps. Two interpretability modules, consisting of saliency map generation and gradient-weighted class activation mapping (Grad-CAM), are incorporated into the well-trained SSPNet to provide additional information helpful for understanding the output. Experimental results from classifying schizophrenia patients (SZs) and healthy controls (HCs) show that the proposed SSPNet significantly improved accuracy and AUC compared to CNN using magnitude maps extracted from either magnitude-only (by 23.4 and 23.6% for DMN) or complex-valued fMRI data (by 10.6 and 5.8% for DMN). SSPNet captured more prominent HC-SZ differences in saliency maps, and Grad-CAM localized all contributing brain regions with opposite strengths for HCs and SZs within SSP maps. These results indicate the potential of SSPNet as a sensitive tool that may be useful for the development of brain-based biomarkers of mental disorders.
Collapse
Affiliation(s)
- Qiu-Hua Lin
- School of Information and Communication Engineering, Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian 116024, China.
| | - Yan-Wei Niu
- School of Information and Communication Engineering, Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian 116024, China
| | - Jing Sui
- State Key Laboratory of Brain Cognition and Learning, Beijing Normal University, Beijing, 100875, China
| | - Wen-Da Zhao
- School of Information and Communication Engineering, Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian 116024, China
| | - Chuanjun Zhuo
- Department of Psychiatry, The Fourth Center Hospital of Tianjin, Tianjin Medical University Affiliated Fourth Center Hospital, Tianjin 300140, China
| | - Vince D Calhoun
- Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS) Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, USA
| |
Collapse
|
31
|
Going deep into schizophrenia with artificial intelligence. Schizophr Res 2022; 245:122-140. [PMID: 34103242 DOI: 10.1016/j.schres.2021.05.018] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/14/2020] [Revised: 05/24/2021] [Accepted: 05/27/2021] [Indexed: 12/30/2022]
Abstract
Despite years of research, the mechanisms governing the onset, relapse, symptomatology, and treatment of schizophrenia (SZ) remain elusive. The lack of appropriate analytic tools to deal with the heterogeneity and complexity of SZ may be one of the reasons behind this situation. Deep learning, a subfield of artificial intelligence (AI) inspired by the nervous system, has recently provided an accessible way of modeling and analyzing complex, high-dimensional, nonlinear systems. The unprecedented accuracy of deep learning algorithms in classification and prediction tasks has revolutionized a wide range of scientific fields and is rapidly permeating SZ research. Deep learning has the potential of becoming a valuable aid for clinicians in the prediction, diagnosis, and treatment of SZ, especially in combination with principles from Bayesian statistics. Furthermore, deep learning could become a powerful tool for uncovering the mechanisms underlying SZ thanks to a growing number of techniques designed for improving model interpretability and causal reasoning. The purpose of this article is to introduce SZ researchers to the field of deep learning and review its latest applications in SZ research. In general, existing studies have yielded impressive results in classification and outcome prediction tasks. However, methodological concerns related to the assessment of model performance in several studies, the widespread use of small training datasets, and the little clinical value of some models suggest that some of these results should be taken with caution.
Collapse
|
32
|
Alzheimer disease stages identification based on correlation transfer function system using resting-state functional magnetic resonance imaging. PLoS One 2022; 17:e0264710. [PMID: 35413053 PMCID: PMC9004771 DOI: 10.1371/journal.pone.0264710] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2021] [Accepted: 02/15/2022] [Indexed: 11/21/2022] Open
Abstract
Alzheimer’s disease (AD) affects the quality of life as it causes; memory loss, difficulty in thinking, learning, and performing familiar tasks. Resting-state functional magnetic resonance imaging (rs-fMRI) has been widely used to investigate and analyze different brain regions for AD identification. This study investigates the effectiveness of using correlated transfer function (CorrTF) as a new biomarker to extract the essential features from rs-fMRI, along with support vector machine (SVM) ordered hierarchically, in order to distinguish between the different AD stages. Additionally, we explored the regions, showing significant changes based on the CorrTF extracted features’ strength among different AD stages. First, the process was initialized by applying the preprocessing on rs-fMRI data samples to reduce noise and retain the essential information. Then, the automated anatomical labeling (AAL) atlas was employed to divide the brain into 116 regions, where the intensity time series was calculated, and the CorrTF features were extracted for each region. The proposed framework employed the SVM classifier in two different methodologies, hierarchical and flat multi-classification schemes, to differentiate between the different AD stages for early detection purposes. The ADNI rs-fMRI dataset, employed in this study, consists of 167, 102, 129, and 114 normal, early, late mild cognitive impairment (MCI), and AD subjects, respectively. The proposed schemes achieved an average accuracy of 98.2% and 95.5% for hierarchical and flat multi-classification tasks, respectively, calculated using ten folds cross-validation. Therefore, CorrTF is considered a promising biomarker for AD early-stage identification. Moreover, the significant changes in the strengths of CorrTF connections among the different AD stages can help us identify and explore the affected brain regions and their latent associations during the progression of AD.
Collapse
|
33
|
A Comprehensive Survey on the Detection, Classification, and Challenges of Neurological Disorders. BIOLOGY 2022; 11:biology11030469. [PMID: 35336842 PMCID: PMC8945195 DOI: 10.3390/biology11030469] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/13/2022] [Revised: 03/12/2022] [Accepted: 03/14/2022] [Indexed: 01/19/2023]
Abstract
Simple Summary This study represents a resourceful review article that can deliver resources on neurological diseases and their implemented classification algorithms to reveal the future direction of researchers. Researchers interested in studying neurological diseases and previously implemented techniques in this field can follow this article. Various challenges occur in detecting different stages of the disorders. A limited amount of labeled and unlabeled datasets and other limitations is represented in this article to assist them in finding out the directions. The authors’ purpose for composing this article is to make a straightforward and concrete path for researchers to quickly find the way and the scope in this field for implementing future research on neurological disease detection. Abstract Neurological disorders (NDs) are becoming more common, posing a concern to pregnant women, parents, healthy infants, and children. Neurological disorders arise in a wide variety of forms, each with its own set of origins, complications, and results. In recent years, the intricacy of brain functionalities has received a better understanding due to neuroimaging modalities, such as magnetic resonance imaging (MRI), magnetoencephalography (MEG), and positron emission tomography (PET), etc. With high-performance computational tools and various machine learning (ML) and deep learning (DL) methods, these modalities have discovered exciting possibilities for identifying and diagnosing neurological disorders. This study follows a computer-aided diagnosis methodology, leading to an overview of pre-processing and feature extraction techniques. The performance of existing ML and DL approaches for detecting NDs is critically reviewed and compared in this article. A comprehensive portion of this study also shows various modalities and disease-specified datasets that detect and records images, signals, and speeches, etc. Limited related works are also summarized on NDs, as this domain has significantly fewer works focused on disease and detection criteria. Some of the standard evaluation metrics are also presented in this study for better result analysis and comparison. This research has also been outlined in a consistent workflow. At the conclusion, a mandatory discussion section has been included to elaborate on open research challenges and directions for future work in this emerging field.
Collapse
|
34
|
Chen M, Li H, Fan H, Dillman JR, Wang H, Altaye M, Zhang B, Parikh NA, He L. ConCeptCNN: A novel multi-filter convolutional neural network for the prediction of neurodevelopmental disorders using brain connectome. Med Phys 2022; 49:3171-3184. [PMID: 35246986 DOI: 10.1002/mp.15545] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2021] [Revised: 01/31/2022] [Accepted: 02/09/2022] [Indexed: 11/11/2022] Open
Abstract
BACKGROUND Deep convolutional neural network (CNN) and its derivatives have recently shown great promise in the prediction of brain disorders using brain connectome data. Existing deep CNN methods using single global row and column convolutional filters have limited ability to extract discriminative information from brain connectome for prediction tasks. PURPOSE This paper presents a novel deep Connectome-Inception CNN (ConCeptCNN) model, which is developed based on multiple convolutional filters. The proposed model is used to extract topological features from brain connectome data for neurological disorders classification and analysis. METHODS The ConCeptCNN uses multiple vector-shaped filters extract topological information from the brain connectome at different levels for complementary feature embeddings of brain connectome. The proposed model is validated using two datasets: the Neuro Bureau ADHD-200 dataset and the Cincinnati Early Prediction Study (CINEPS) dataset. RESULTS In a cross-validation experiment, the ConCeptCNN achieved a prediction accuracy of 78.7% for the detection of ADHD in adolescents and an accuracy of 81.6% for the prediction of cognitive deficits at 2-year corrected age in very preterm infants. In addition to the classification tasks, the ConCeptCNN identified several brain regions that are discriminative to neurodevelopmental disorders. CONCLUSIONS We compared the ConCeptCNN with several peer CNN methods. The results demonstrated that proposed model improves overall classification performance of neurodevelopmental disorders prediction tasks. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Ming Chen
- Imaging Research Center, Department of Radiology, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA.,Department of Electrical Engineering and Computer Science, University of Cincinnati, Cincinnati, OH, USA
| | - Hailong Li
- Imaging Research Center, Department of Radiology, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA.,Center for Prevention of Neurodevelopmental Disorders, Perinatal Institute, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
| | - Howard Fan
- Department of Electrical Engineering and Computer Science, University of Cincinnati, Cincinnati, OH, USA
| | - Jonathan R Dillman
- Imaging Research Center, Department of Radiology, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA.,Department of Radiology, University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Hui Wang
- Imaging Research Center, Department of Radiology, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA.,MR Clinical Science, Philips, Cincinnati, OH, USA
| | - Mekibib Altaye
- Division of Biostatistics and Epidemiology, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA.,Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Bin Zhang
- Division of Biostatistics and Epidemiology, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA.,Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Nehal A Parikh
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA.,Center for Prevention of Neurodevelopmental Disorders, Perinatal Institute, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
| | - Lili He
- Imaging Research Center, Department of Radiology, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA.,Department of Radiology, University of Cincinnati College of Medicine, Cincinnati, OH, USA.,Center for Prevention of Neurodevelopmental Disorders, Perinatal Institute, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
| |
Collapse
|
35
|
Yang Z, Leng L, Li M, Chu J. A computer-aid multi-task light-weight network for macroscopic feces diagnosis. MULTIMEDIA TOOLS AND APPLICATIONS 2022; 81:15671-15686. [PMID: 35250359 PMCID: PMC8884099 DOI: 10.1007/s11042-022-12565-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/18/2021] [Revised: 06/15/2021] [Accepted: 01/31/2022] [Indexed: 06/14/2023]
Abstract
The abnormal traits and colors of feces typically indicate that the patients are probably suffering from tumor or digestive-system diseases. Thus a fast, accurate and automatic health diagnosis system based on feces is urgently necessary for improving the examination speed and reducing the infection risk. The rarity of the pathological images would deteriorate the accuracy performance of the trained models. In order to alleviate this problem, we employ augmentation and over-sampling to expand the samples of the classes that have few samples in the training batch. In order to achieve an impressive recognition performance and leverage the latent correlation between the traits and colors of feces pathological samples, a multi-task network is developed to recognize colors and traits of the macroscopic feces images. The parameter number of a single multi-task network is generally much smaller than the total parameter number of multiple single-task networks, so the storage cost is reduced. The loss function of the multi-task network is the weighted sum of the losses of the two tasks. In this paper, the weights of the tasks are determined according to their difficulty levels that are measured by the fitted linear functions. The sufficient experiments confirm that the proposed method can yield higher accuracies, and the efficiency is also improved.
Collapse
Affiliation(s)
- Ziyuan Yang
- School of Software, Nanchang Hangkong University, Nanchang, 330063 People’s Republic of China
- College of Computer Science, Sichuan University, Chengdu, 610065 People’s Republic of China
| | - Lu Leng
- School of Software, Nanchang Hangkong University, Nanchang, 330063 People’s Republic of China
- School of Electrical and Electronic Engineering, College of Engineering, Yonsei University, Seoul, 120749 Republic of Korea
| | - Ming Li
- School of Information Engineering, Nanchang Hangkong University, Nanchang, 330063 People’s Republic of China
| | - Jun Chu
- School of Software, Nanchang Hangkong University, Nanchang, 330063 People’s Republic of China
| |
Collapse
|
36
|
Wang J, Ke P, Zang J, Wu F, Wu K. Discriminative Analysis of Schizophrenia Patients Using Topological Properties of Structural and Functional Brain Networks: A Multimodal Magnetic Resonance Imaging Study. Front Neurosci 2022; 15:785595. [PMID: 35087373 PMCID: PMC8787107 DOI: 10.3389/fnins.2021.785595] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2021] [Accepted: 12/01/2021] [Indexed: 12/12/2022] Open
Abstract
Interest in the application of machine learning (ML) techniques to multimodal magnetic resonance imaging (MRI) data for the diagnosis of schizophrenia (SZ) at the individual level is growing. However, a few studies have applied the features of structural and functional brain networks derived from multimodal MRI data to the discriminative analysis of SZ patients at different clinical stages. In this study, 205 normal controls (NCs), 61 first-episode drug-naive SZ (FESZ) patients, and 79 chronic SZ (CSZ) patients were recruited. We acquired their structural MRI, diffusion tensor imaging, and resting-state functional MRI data and constructed brain networks for each participant, including the gray matter network (GMN), white matter network (WMN), and functional brain network (FBN). We then calculated 3 nodal properties for each brain network, including degree centrality, nodal efficiency, and betweenness centrality. Two classifications (SZ vs. NC and FESZ vs. CSZ) were performed using five ML algorithms. We found that the SVM classifier with the input features of the combination of nodal properties of both the GMN and FBN achieved the best performance to discriminate SZ patients from NCs [accuracy, 81.2%; area under the receiver operating characteristic curve (AUC), 85.2%; p < 0.05]. Moreover, the SVM classifier with the input features of the combination of the nodal properties of both the GMN and WMN achieved the best performance to discriminate FESZ from CSZ patients (accuracy, 86.2%; AUC, 92.3%; p < 0.05). Furthermore, the brain areas in the subcortical/cerebellum network and the frontoparietal network showed significant importance in both classifications. Together, our findings provide new insights to understand the neuropathology of SZ and further highlight the potential advantages of multimodal network properties for identifying SZ patients at different clinical stages.
Collapse
Affiliation(s)
- Jing Wang
- School of Biomedical Engineering, Guangzhou Xinhua University, Guangzhou, China
| | - Pengfei Ke
- School of Biomedical Sciences and Engineering, South China University of Technology, Guangzhou, China
- National Engineering Research Center for Tissue Restoration and Reconstruction, South China University of Technology, Guangzhou, China
| | - Jinyu Zang
- School of Biomedical Sciences and Engineering, South China University of Technology, Guangzhou, China
- National Engineering Research Center for Tissue Restoration and Reconstruction, South China University of Technology, Guangzhou, China
| | - Fengchun Wu
- The Affiliated Brain Hospital of Guangzhou Medical University, Guangzhou, China
- Guangdong Engineering Technology Research Center for Translational Medicine of Mental Disorders, Guangzhou, China
- *Correspondence: Fengchun Wu,
| | - Kai Wu
- School of Biomedical Sciences and Engineering, South China University of Technology, Guangzhou, China
- The Affiliated Brain Hospital of Guangzhou Medical University, Guangzhou, China
- Guangdong Engineering Technology Research Center for Translational Medicine of Mental Disorders, Guangzhou, China
- Guangdong Engineering Technology Research Center for Diagnosis and Rehabilitation of Dementia, Guangzhou, China
- National Engineering Research Center for Tissue Restoration and Reconstruction, South China University of Technology, Guangzhou, China
- Key Laboratory of Biomedical Engineering of Guangdong Province, South China University of Technology, Guangzhou, China
- Institute for Healthcare Artificial Intelligence Application, Guangdong Second Provincial General Hospital, Guangzhou, China
- Department of Nuclear Medicine and Radiology, Institute of Development, Aging and Cancer, Tohoku University, Sendai, Japan
- Kai Wu,
| |
Collapse
|
37
|
Wen Y, Zhou C, Chen L, Deng Y, Cleusix M, Jenni R, Conus P, Do KQ, Xin L. Bridging structural MRI with cognitive function for individual level classification of early psychosis via deep learning. Front Psychiatry 2022; 13:1075564. [PMID: 36704734 PMCID: PMC9871589 DOI: 10.3389/fpsyt.2022.1075564] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/20/2022] [Accepted: 12/21/2022] [Indexed: 01/12/2023] Open
Abstract
INTRODUCTION Recent efforts have been made to apply machine learning and deep learning approaches to the automated classification of schizophrenia using structural magnetic resonance imaging (sMRI) at the individual level. However, these approaches are less accurate on early psychosis (EP) since there are mild structural brain changes at early stage. As cognitive impairments is one main feature in psychosis, in this study we apply a multi-task deep learning framework using sMRI with inclusion of cognitive assessment to facilitate the classification of patients with EP from healthy individuals. METHOD Unlike previous studies, we used sMRI as the direct input to perform EP classifications and cognitive estimations. The proposed deep learning model does not require time-consuming volumetric or surface based analysis and can provide additionally cognition predictions. Experiments were conducted on an in-house data set with 77 subjects and a public ABCD HCP-EP data set with 164 subjects. RESULTS We achieved 74.9 ± 4.3% five-fold cross-validated accuracy and an area under the curve of 71.1 ± 4.1% on EP classification with the inclusion of cognitive estimations. DISCUSSION We reveal the feasibility of automated cognitive estimation using sMRI by deep learning models, and also demonstrate the implicit adoption of cognitive measures as additional information to facilitate EP classifications from healthy controls.
Collapse
Affiliation(s)
- Yang Wen
- Key Laboratory of Digital Media Technology of Sichuan Province, School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China.,Animal Imaging and Technology Core, Center for Biomedical Imaging, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland.,Laboratory for Functional and Metabolic Imaging, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - Chuan Zhou
- Key Laboratory of Digital Media Technology of Sichuan Province, School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China.,Institute of Electronic and Information Engineering of UESTC in Guangdong, Dongguan, Guangdong, China
| | - Leiting Chen
- Key Laboratory of Digital Media Technology of Sichuan Province, School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China.,Institute of Electronic and Information Engineering of UESTC in Guangdong, Dongguan, Guangdong, China
| | - Yu Deng
- Department of Biomedical Engineering, King's College London, London, United Kingdom
| | - Martine Cleusix
- Department of Psychiatry, Center for Psychiatric Neuroscience, Centre Hospitalier Universitaire Vaudois and University of Lausanne, Lausanne, Switzerland
| | - Raoul Jenni
- Department of Psychiatry, Center for Psychiatric Neuroscience, Centre Hospitalier Universitaire Vaudois and University of Lausanne, Lausanne, Switzerland
| | - Philippe Conus
- Service of General Psychiatry, Department of Psychiatry, Centre Hospitalier Universitaire Vaudois and University of Lausanne, Lausanne, Switzerland
| | - Kim Q Do
- Department of Psychiatry, Center for Psychiatric Neuroscience, Centre Hospitalier Universitaire Vaudois and University of Lausanne, Lausanne, Switzerland
| | - Lijing Xin
- Animal Imaging and Technology Core, Center for Biomedical Imaging, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| |
Collapse
|
38
|
|
39
|
Shoeibi A, Sadeghi D, Moridian P, Ghassemi N, Heras J, Alizadehsani R, Khadem A, Kong Y, Nahavandi S, Zhang YD, Gorriz JM. Automatic Diagnosis of Schizophrenia in EEG Signals Using CNN-LSTM Models. Front Neuroinform 2021; 15:777977. [PMID: 34899226 PMCID: PMC8657145 DOI: 10.3389/fninf.2021.777977] [Citation(s) in RCA: 35] [Impact Index Per Article: 11.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2021] [Accepted: 10/22/2021] [Indexed: 11/13/2022] Open
Abstract
Schizophrenia (SZ) is a mental disorder whereby due to the secretion of specific chemicals in the brain, the function of some brain regions is out of balance, leading to the lack of coordination between thoughts, actions, and emotions. This study provides various intelligent deep learning (DL)-based methods for automated SZ diagnosis via electroencephalography (EEG) signals. The obtained results are compared with those of conventional intelligent methods. To implement the proposed methods, the dataset of the Institute of Psychiatry and Neurology in Warsaw, Poland, has been used. First, EEG signals were divided into 25 s time frames and then were normalized by z-score or norm L2. In the classification step, two different approaches were considered for SZ diagnosis via EEG signals. In this step, the classification of EEG signals was first carried out by conventional machine learning methods, e.g., support vector machine, k-nearest neighbors, decision tree, naïve Bayes, random forest, extremely randomized trees, and bagging. Various proposed DL models, namely, long short-term memories (LSTMs), one-dimensional convolutional networks (1D-CNNs), and 1D-CNN-LSTMs, were used in the following. In this step, the DL models were implemented and compared with different activation functions. Among the proposed DL models, the CNN-LSTM architecture has had the best performance. In this architecture, the ReLU activation function with the z-score and L2-combined normalization was used. The proposed CNN-LSTM model has achieved an accuracy percentage of 99.25%, better than the results of most former studies in this field. It is worth mentioning that to perform all simulations, the k-fold cross-validation method with k = 5 has been used.
Collapse
Affiliation(s)
- Afshin Shoeibi
- Faculty of Electrical Engineering, K. N. Toosi University of Technology, Tehran, Iran
| | - Delaram Sadeghi
- Department of Medical Engineering, Islamic Azad University of Mashhad, Mashhad, Iran
| | - Parisa Moridian
- Faculty of Engineering, Islamic Azad University of Science and Research, Tehran, Iran
| | - Navid Ghassemi
- Faculty of Electrical Engineering, K. N. Toosi University of Technology, Tehran, Iran
| | - Jónathan Heras
- Department of Mathematics and Computer Science, University of La Rioja, Logroño, Spain
| | - Roohallah Alizadehsani
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Geelong, VIC, Australia
| | - Ali Khadem
- Faculty of Electrical Engineering, K. N. Toosi University of Technology, Tehran, Iran
| | - Yinan Kong
- School of Engineering, Macquarie University, Sydney, NSW, Australia
| | - Saeid Nahavandi
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Geelong, VIC, Australia
| | - Yu-Dong Zhang
- Department of Informatics, University of Leicester, Leicester, United Kingdom
| | - Juan Manuel Gorriz
- Department of Signal Theory, Telematics and Communications, ETS of Computer and Telecommunications Engineering, University of Granada, Granada, Spain
| |
Collapse
|
40
|
Rahaman MA, Chen J, Fu Z, Lewis N, Iraji A, Calhoun VD. Multi-modal deep learning of functional and structural neuroimaging and genomic data to predict mental illness. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:3267-3272. [PMID: 34891938 DOI: 10.1109/embc46164.2021.9630693] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Neuropsychiatric disorders such as schizophrenia are very heterogeneous in nature and typically diagnosed using self-reported symptoms. This makes it difficult to pose a confident prediction on the cases and does not provide insight into the underlying neural and biological mechanisms of these disorders. Combining neuroimaging and genomic data with a multi-modal 'predictome' paves the way for biologically informed markers and may improve prediction reliability. With that, we develop a multi-modal deep learning framework by fusing data from different modalities to capture the interaction between the latent features and evaluate their complementary information in characterizing schizophrenia. Our deep model uses structural MRI, functional MRI, and genome-wide polymorphism data to perform the classification task. It includes a multi-layer feed-forward network, an encoder, and a long short-term memory (LSTM) unit with attention to learn the latent features and adopt a joint training scheme capturing synergies between the modalities. The hybrid network also uses different regularizers for addressing the inherent overfitting and modality-specific bias in the multi-modal setup. Next, we run the network through a saliency model to analyze the learned features. Integrating modalities enhances the performance of the classifier, and our framework acquired 88% (P < 0.0001) accuracy on a dataset of 437 subjects. The trimodal accuracy is comparable to the state-of-the-art performance on a data collection of this size and outperforms the unimodal and bimodal baselines we compared. Model introspection was used to expose the salient neural features and genes/biological pathways associated with schizophrenia. To our best knowledge, this is the first approach that fuses genomic information with structural and functional MRI biomarkers for predicting schizophrenia. We believe this type of modality blending can better explain the disorder's dynamics by adding cross-modal prospects.Clinical Relevance- This study combinedly learns imaging and genomic features for the classification of schizophrenia. The data fusion scheme extracts modality interactions, and the saliency experiments report multiple functional and structural networks closely connected to the disorder.
Collapse
|
41
|
Multi-Class brain normality and abnormality diagnosis using modified Faster R-CNN. Int J Med Inform 2021; 155:104576. [PMID: 34555555 DOI: 10.1016/j.ijmedinf.2021.104576] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2021] [Revised: 09/10/2021] [Accepted: 09/13/2021] [Indexed: 11/24/2022]
Abstract
BACKGROUND AND OBJECTIVE The detection and analysis of brain disorders through medical imaging techniques are extremely important to get treatment on time and sustain a healthy lifestyle. Disorders cause permanent brain damage and alleviate the lifespan. Moreover, the classification of large volumes of medical image data manually by medicine experts is tiring, time-consuming, and prone to errors. This study aims to diagnose brain normality and abnormalities using a novel ResNet50 modified Faster Regions with Convolutional Neural Network(R-CNN) model. The classification task is performed into multiple classes which are hemorrhage, hydrocephalus, and normal. The proposed model both determines the borders of the normal/abnormal parts and classifies them with the highest accuracy. METHODS To provide a comprehensive performance analysis in the classification problem, Machine Learning(ML) and Deep Learning(DL) techniques were discussed. Artificial Neural Network(ANN), AdaBoost(AB), Decision Tree(DT), Logistic Regression(LR), Naive Bayes(NB), Random Forest(RF), and Support Vector Machine(SVM) were used as ML models. Besides, various Convolutional Neural Network(CNN) models and proposed ResNet50 modified Faster R-CNN model were used as DL models. Methods were validated using a novel brain dataset that contains both normal and abnormal images. RESULTS Based on results, LR obtained the highest result among ML methods and DenseNet201 obtained the highest results among CNN models with the accuracy of 84.80% and 85.68% for the classification task, respectively. Besides, the accuracy obtained by the proposed model is 99.75%. CONCLUSIONS Experimental results demonstrate that the proposed model has yielded better performance for detection and classification tasks. This artificial intelligence(AI) framework can be utilized as a computer-aided medical decision support system for medical experts.
Collapse
|
42
|
Cao M, Yang M, Qin C, Zhu X, Chen Y, Wang J, Liu T. Using DeepGCN to identify the autism spectrum disorder from multi-site resting-state data. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.103015] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
|
43
|
Fully-channel regional attention network for disease-location recognition with tongue images. Artif Intell Med 2021; 118:102110. [PMID: 34412836 DOI: 10.1016/j.artmed.2021.102110] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2020] [Revised: 04/06/2021] [Accepted: 05/11/2021] [Indexed: 12/28/2022]
Abstract
OBJECTIVE Using the deep learning model to realize tongue image-based disease location recognition and focus on solving two problems: 1. The ability of the general convolution network to model detailed regional tongue features is weak; 2. Ignoring the group relationship between convolution channels, which caused the high redundancy of the model. METHODS To enhance the convolutional neural networks. In this paper, a stochastic region pooling method is proposed to gain detailed regional features. Also, an inner-imaging channel relationship modeling method is proposed to model multi-region relations on all channels. Moreover, we combine it with the spatial attention mechanism. RESULTS The tongue image dataset with the clinical disease-location label is established. Abundant experiments are carried out on it. The experimental results show that the proposed method can effectively model the regional details of tongue image and improve the performance of disease location recognition. CONCLUSION In this paper, we construct the tongue image dataset with disease-location labels to mine the relationship between tongue images and disease locations. A novel fully-channel regional attention network is proposed to model the local detail tongue features and improve the modeling efficiency. SIGNIFICANCE The applications of deep learning in tongue image disease-location recognition and the proposed innovative models have guiding significance for other assistant diagnostic tasks. The proposed model provides an example of efficient modeling of detailed tongue features, which is of great guiding significance for other auxiliary diagnosis applications.
Collapse
|
44
|
Zhang Z, Li G, Xu Y, Tang X. Application of Artificial Intelligence in the MRI Classification Task of Human Brain Neurological and Psychiatric Diseases: A Scoping Review. Diagnostics (Basel) 2021; 11:1402. [PMID: 34441336 PMCID: PMC8392727 DOI: 10.3390/diagnostics11081402] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2021] [Revised: 07/21/2021] [Accepted: 07/21/2021] [Indexed: 12/12/2022] Open
Abstract
Artificial intelligence (AI) for medical imaging is a technology with great potential. An in-depth understanding of the principles and applications of magnetic resonance imaging (MRI), machine learning (ML), and deep learning (DL) is fundamental for developing AI-based algorithms that can meet the requirements of clinical diagnosis and have excellent quality and efficiency. Moreover, a more comprehensive understanding of applications and opportunities would help to implement AI-based methods in an ethical and sustainable manner. This review first summarizes recent research advances in ML and DL techniques for classifying human brain magnetic resonance images. Then, the application of ML and DL methods to six typical neurological and psychiatric diseases is summarized, including Alzheimer's disease (AD), Parkinson's disease (PD), major depressive disorder (MDD), schizophrenia (SCZ), attention-deficit/hyperactivity disorder (ADHD), and autism spectrum disorder (ASD). Finally, the limitations of the existing research are discussed, and possible future research directions are proposed.
Collapse
Affiliation(s)
- Zhao Zhang
- 715-3 Teaching Building No.5, Department of Biomedical Engineering, School of Life Sciences, Beijing Institute of Technology, 5 South Zhongguancun Road, Haidian District, Beijing 100081, China; (Z.Z.); (G.L.)
| | - Guangfei Li
- 715-3 Teaching Building No.5, Department of Biomedical Engineering, School of Life Sciences, Beijing Institute of Technology, 5 South Zhongguancun Road, Haidian District, Beijing 100081, China; (Z.Z.); (G.L.)
| | - Yong Xu
- Department of Cardiology, Chinese PLA General Hospital, Beijing 100853, China;
| | - Xiaoying Tang
- 715-3 Teaching Building No.5, Department of Biomedical Engineering, School of Life Sciences, Beijing Institute of Technology, 5 South Zhongguancun Road, Haidian District, Beijing 100081, China; (Z.Z.); (G.L.)
| |
Collapse
|
45
|
Mousavian M, Chen J, Traylor Z, Greening S. Depression detection from sMRI and rs-fMRI images using machine learning. J Intell Inf Syst 2021. [DOI: 10.1007/s10844-021-00653-w] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
46
|
Yamaguchi H, Hashimoto Y, Sugihara G, Miyata J, Murai T, Takahashi H, Honda M, Hishimoto A, Yamashita Y. Three-Dimensional Convolutional Autoencoder Extracts Features of Structural Brain Images With a "Diagnostic Label-Free" Approach: Application to Schizophrenia Datasets. Front Neurosci 2021; 15:652987. [PMID: 34305514 PMCID: PMC8294943 DOI: 10.3389/fnins.2021.652987] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2021] [Accepted: 06/07/2021] [Indexed: 01/17/2023] Open
Abstract
There has been increasing interest in performing psychiatric brain imaging studies using deep learning. However, most studies in this field disregard three-dimensional (3D) spatial information and targeted disease discrimination, without considering the genetic and clinical heterogeneity of psychiatric disorders. The purpose of this study was to investigate the efficacy of a 3D convolutional autoencoder (3D-CAE) for extracting features related to psychiatric disorders without diagnostic labels. The network was trained using a Kyoto University dataset including 82 patients with schizophrenia (SZ) and 90 healthy subjects (HS) and was evaluated using Center for Biomedical Research Excellence (COBRE) datasets, including 71 SZ patients and 71 HS. We created 16 3D-CAE models with different channels and convolutions to explore the effective range of hyperparameters for psychiatric brain imaging. The number of blocks containing two convolutional layers and one pooling layer was set, ranging from 1 block to 4 blocks. The number of channels in the extraction layer varied from 1, 4, 16, and 32 channels. The proposed 3D-CAEs were successfully reproduced into 3D structural magnetic resonance imaging (MRI) scans with sufficiently low errors. In addition, the features extracted using 3D-CAE retained the relation to clinical information. We explored the appropriate hyperparameter range of 3D-CAE, and it was suggested that a model with 3 blocks may be related to extracting features for predicting the dose of medication and symptom severity in schizophrenia.
Collapse
Affiliation(s)
- Hiroyuki Yamaguchi
- Department of Information Medicine, National Center of Neurology and Psychiatry, National Institute of Neuroscience, Tokyo, Japan.,Department of Psychiatry, School of Medicine, Yokohama City University, Yokohama, Japan
| | - Yuki Hashimoto
- Department of Information Medicine, National Center of Neurology and Psychiatry, National Institute of Neuroscience, Tokyo, Japan
| | - Genichi Sugihara
- Department of Psychiatry and Behavioral Sciences, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan
| | - Jun Miyata
- Department of Psychiatry, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Toshiya Murai
- Department of Psychiatry, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Hidehiko Takahashi
- Department of Psychiatry and Behavioral Sciences, Graduate School of Medical and Dental Sciences, Tokyo Medical and Dental University, Tokyo, Japan
| | - Manabu Honda
- Department of Information Medicine, National Center of Neurology and Psychiatry, National Institute of Neuroscience, Tokyo, Japan
| | - Akitoyo Hishimoto
- Department of Psychiatry, School of Medicine, Yokohama City University, Yokohama, Japan
| | - Yuichi Yamashita
- Department of Information Medicine, National Center of Neurology and Psychiatry, National Institute of Neuroscience, Tokyo, Japan
| |
Collapse
|
47
|
Yang M, Cao M, Chen Y, Chen Y, Fan G, Li C, Wang J, Liu T. Large-Scale Brain Functional Network Integration for Discrimination of Autism Using a 3-D Deep Learning Model. Front Hum Neurosci 2021; 15:687288. [PMID: 34149385 PMCID: PMC8206477 DOI: 10.3389/fnhum.2021.687288] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2021] [Accepted: 05/03/2021] [Indexed: 11/13/2022] Open
Abstract
GOAL Brain functional networks (BFNs) constructed using resting-state functional magnetic resonance imaging (fMRI) have proven to be an effective way to understand aberrant functional connectivity in autism spectrum disorder (ASD) patients. It is still challenging to utilize these features as potential biomarkers for discrimination of ASD. The purpose of this work is to classify ASD and normal controls (NCs) using BFNs derived from rs-fMRI. METHODS A deep learning framework was proposed that integrated convolutional neural network (CNN) and channel-wise attention mechanism to model both intra- and inter-BFN associations simultaneously for ASD diagnosis. We investigate the effects of each BFN on performance and performed inter-network connectivity analysis between each pair of BFNs. We compared the performance of our CNN model with some state-of-the-art algorithms using functional connectivity features. RESULTS We collected 79 ASD patients and 105 NCs from the ABIDE-I dataset. The mean accuracy of our classification algorithm was 77.74% for classification of ASD versus NCs. CONCLUSION The proposed model is able to integrate information from multiple BFNs to improve detection accuracy of ASD. SIGNIFICANCE These findings suggest that large-scale BFNs is promising to serve as reliable biomarkers for diagnosis of ASD.
Collapse
Affiliation(s)
- Ming Yang
- The Key Laboratory of Biomedical Information Engineering of Ministry of Education, Institute of Health and Rehabilitation Science, School of Life Sciences and Technology, Xi’an Jiaotong University, Xi’an, China
- National Engineering Research Center for Healthcare Devices, Guangzhou, China
- The Key Laboratory of Neuro-informatics and Rehabilitation Engineering of Ministry of Civil Affairs, Xi’an, China
| | - Menglin Cao
- The Key Laboratory of Biomedical Information Engineering of Ministry of Education, Institute of Health and Rehabilitation Science, School of Life Sciences and Technology, Xi’an Jiaotong University, Xi’an, China
- National Engineering Research Center for Healthcare Devices, Guangzhou, China
- The Key Laboratory of Neuro-informatics and Rehabilitation Engineering of Ministry of Civil Affairs, Xi’an, China
| | - Yuhao Chen
- The Key Laboratory of Biomedical Information Engineering of Ministry of Education, Institute of Health and Rehabilitation Science, School of Life Sciences and Technology, Xi’an Jiaotong University, Xi’an, China
- National Engineering Research Center for Healthcare Devices, Guangzhou, China
- The Key Laboratory of Neuro-informatics and Rehabilitation Engineering of Ministry of Civil Affairs, Xi’an, China
| | | | - Geng Fan
- The Key Laboratory of Biomedical Information Engineering of Ministry of Education, Institute of Health and Rehabilitation Science, School of Life Sciences and Technology, Xi’an Jiaotong University, Xi’an, China
- National Engineering Research Center for Healthcare Devices, Guangzhou, China
- The Key Laboratory of Neuro-informatics and Rehabilitation Engineering of Ministry of Civil Affairs, Xi’an, China
| | - Chenxi Li
- The Key Laboratory of Biomedical Information Engineering of Ministry of Education, Institute of Health and Rehabilitation Science, School of Life Sciences and Technology, Xi’an Jiaotong University, Xi’an, China
- National Engineering Research Center for Healthcare Devices, Guangzhou, China
- The Key Laboratory of Neuro-informatics and Rehabilitation Engineering of Ministry of Civil Affairs, Xi’an, China
| | - Jue Wang
- The Key Laboratory of Biomedical Information Engineering of Ministry of Education, Institute of Health and Rehabilitation Science, School of Life Sciences and Technology, Xi’an Jiaotong University, Xi’an, China
- National Engineering Research Center for Healthcare Devices, Guangzhou, China
- The Key Laboratory of Neuro-informatics and Rehabilitation Engineering of Ministry of Civil Affairs, Xi’an, China
| | - Tian Liu
- The Key Laboratory of Biomedical Information Engineering of Ministry of Education, Institute of Health and Rehabilitation Science, School of Life Sciences and Technology, Xi’an Jiaotong University, Xi’an, China
- National Engineering Research Center for Healthcare Devices, Guangzhou, China
- The Key Laboratory of Neuro-informatics and Rehabilitation Engineering of Ministry of Civil Affairs, Xi’an, China
| |
Collapse
|
48
|
Statistical and Machine Learning Link Selection Methods for Brain Functional Networks: Review and Comparison. Brain Sci 2021; 11:brainsci11060735. [PMID: 34073098 PMCID: PMC8227272 DOI: 10.3390/brainsci11060735] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2021] [Revised: 05/24/2021] [Accepted: 05/28/2021] [Indexed: 11/28/2022] Open
Abstract
Network-based representations have introduced a revolution in neuroscience, expanding the understanding of the brain from the activity of individual regions to the interactions between them. This augmented network view comes at the cost of high dimensionality, which hinders both our capacity of deciphering the main mechanisms behind pathologies, and the significance of any statistical and/or machine learning task used in processing this data. A link selection method, allowing to remove irrelevant connections in a given scenario, is an obvious solution that provides improved utilization of these network representations. In this contribution we review a large set of statistical and machine learning link selection methods and evaluate them on real brain functional networks. Results indicate that most methods perform in a qualitatively similar way, with NBS (Network Based Statistics) winning in terms of quantity of retained information, AnovaNet in terms of stability and ExT (Extra Trees) in terms of lower computational cost. While machine learning methods are conceptually more complex than statistical ones, they do not yield a clear advantage. At the same time, the high heterogeneity in the set of links retained by each method suggests that they are offering complementary views to the data. The implications of these results in neuroscience tasks are finally discussed.
Collapse
|
49
|
Masoudi B, Daneshvar S, Razavi SN. Multi-modal neuroimaging feature fusion via 3D Convolutional Neural Network architecture for schizophrenia diagnosis. INTELL DATA ANAL 2021. [DOI: 10.3233/ida-205113] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022]
Abstract
Early and precise diagnosis of schizophrenia disorder (SZ) has an essential role in the quality of a patient’s life and future treatments. Structural and functional neuroimaging provides robust biomarkers for understanding the anatomical and functional changes associated with SZ. Each of the neuroimaging techniques shows only a different perspective on the functional or structural of the brain, while multi-modal fusion can reveal latent connections in the brain. In this paper, we propose an approach for the fusion of structural and functional brain data with a deep learning-based model to take advantage of data fusion and increase the accuracy of schizophrenia disorder diagnosis. The proposed method consists of an architecture of 3D convolutional neural networks (CNNs) that applied to magnetic resonance imaging (MRI), functional magnetic resonance imaging (fMRI), and diffusion tensor imaging (DTI) extracted features. We use 3D MRI patches, fMRI spatial independent component analysis (ICA) map, and DTI fractional anisotropy (FA) as model inputs. Our method is validated on the COBRE dataset, and an average accuracy of 99.35% is obtained. The proposed method demonstrates promising classification performance and can be applied to real data.
Collapse
Affiliation(s)
- Babak Masoudi
- Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran
| | - Sabalan Daneshvar
- Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran
- Department of Electronic and Computer Engineering, College of Engineering, Design and Physical Sciences, Brunel University, London, UK
| | - Seyed Naser Razavi
- Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran
| |
Collapse
|
50
|
Quaak M, van de Mortel L, Thomas RM, van Wingen G. Deep learning applications for the classification of psychiatric disorders using neuroimaging data: Systematic review and meta-analysis. Neuroimage Clin 2021; 30:102584. [PMID: 33677240 PMCID: PMC8209481 DOI: 10.1016/j.nicl.2021.102584] [Citation(s) in RCA: 31] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2020] [Revised: 01/18/2021] [Accepted: 01/29/2021] [Indexed: 12/20/2022]
Abstract
Deep learning (DL) methods have been increasingly applied to neuroimaging data to identify patients with psychiatric and neurological disorders. This review provides an overview of the different DL applications within psychiatry and compares DL model accuracy to standard machine learning (SML). Fifty-three articles were included for qualitative analysis, primarily investigating autism spectrum disorder (ASD; n = 22), schizophrenia (SZ; n = 22) and attention-deficit/hyperactivity disorder (ADHD; n = 9). Thirty-two of the thirty-five studies that directly compared DL to SML reported a higher accuracy for DL. Only sixteen studies could be included in a meta-regression to quantitatively compare DL and SML performance. This showed a higher odds ratio for DL models, though the comparison attained significance only for ASD. Our results suggest that deep learning of neuroimaging data is a promising tool for the classification of individual psychiatric patients. However, it is not yet used to its full potential: most studies use pre-engineered features, whereas one of the main advantages of DL is its ability to learn representations of minimally processed data. Our current evaluation is limited by minimal reporting of performance measures to enable quantitative comparisons, and the restriction to ADHD, SZ and ASD as current research focusses on large publicly available datasets. To truly uncover the added value of DL, we need carefully designed comparisons of SML and DL models which are yet rarely performed.
Collapse
Affiliation(s)
- Mirjam Quaak
- Amsterdam UMC, University of Amsterdam, Department of Psychiatry, Meibergdreef 5, 1105 AZ Amsterdam, The Netherlands
| | - Laurens van de Mortel
- Amsterdam UMC, University of Amsterdam, Department of Psychiatry, Meibergdreef 5, 1105 AZ Amsterdam, The Netherlands
| | - Rajat Mani Thomas
- Amsterdam UMC, University of Amsterdam, Department of Psychiatry, Meibergdreef 5, 1105 AZ Amsterdam, The Netherlands
| | - Guido van Wingen
- Amsterdam UMC, University of Amsterdam, Department of Psychiatry, Meibergdreef 5, 1105 AZ Amsterdam, The Netherlands.
| |
Collapse
|