3D-CNN based discrimination of schizophrenia using resting-state fMRI

Investigating the discrimination ability of 3D convolutional neural networks applied to altered brain MRI parametric maps

Convolutional neural networks (CNNs) are gradually being recognized in the neuroimaging community as a powerful tool for image analysis. Despite their outstanding performances, some aspects of CNN functioning are still not fully understood by human operators. We postulated that the interpretability of CNNs applied to neuroimaging data could be improved by investigating their behavior when they are fed data with known characteristics. We analyzed the ability of 3D CNNs to discriminate between original and altered whole-brain parametric maps derived from diffusion-weighted magnetic resonance imaging. The alteration consisted in linearly changing the voxel intensity of either one (monoregion) or two (biregion) anatomical regions in each brain volume, but without mimicking any neuropathology. Performing ten-fold cross-validation and using a hold-out set for testing, we assessed the CNNs' discrimination ability according to the intensity of the altered regions, comparing the latter's size and relative position. Monoregion CNNs showed that the larger the modified region, the smaller the intensity increase needed to achieve good performances. Biregion CNNs systematically outperformed monoregion CNNs, but could only detect one of the two target regions when tested on the corresponding monoregion images. Exploiting prior information on training data allowed for a better understanding of CNN behavior, especially when altered regions were combined. This can inform about the complexity of CNN pattern retrieval and elucidate misclassified examples, particularly relevant for pathological data. The proposed analytical approach may serve to gain insights into CNN behavior and guide the design of enhanced detection systems exploiting our prior knowledge.

Identification of important gene signatures in schizophrenia through feature fusion and genetic algorithm

Schizophrenia is a debilitating psychiatric disorder that can significantly affect a patient's quality of life and lead to permanent brain damage. Although medical research has identified certain genetic risk factors, the specific pathogenesis of the disorder remains unclear. Despite the prevalence of research employing magnetic resonance imaging, few studies have focused on the gene level and gene expression profile involving a large number of screened genes. However, the high dimensionality of genetic data presents a great challenge to accurately modeling the data. To tackle the current challenges, this study presents a novel feature selection strategy that utilizes heuristic feature fusion and a multi-objective optimization genetic algorithm. The goal is to improve classification performance and identify the key gene subset for schizophrenia diagnostics. Traditional gene screening techniques are inadequate for accurately determining the precise number of key genes associated with schizophrenia. Our innovative approach integrates a filter-based feature selection method to reduce data dimensionality and a multi-objective optimization genetic algorithm for improved classification tasks. By combining the filtering and wrapper methods, our strategy leverages their respective strengths in a deliberate manner, leading to superior classification accuracy and a more efficient selection of relevant genes. This approach has demonstrated significant improvements in classification results across 11 out of 14 relevant datasets. The performance on the remaining three datasets is comparable to the existing methods. Furthermore, visual and enrichment analyses have confirmed the practicality of our proposed method as a promising tool for the early detection of schizophrenia.

Assistive tools for classifying neurological disorders using fMRI and deep learning: A guide and example

BACKGROUND

Deep-learning (DL) methods are rapidly changing the way researchers classify neurological disorders. For example, combining functional magnetic resonance imaging (fMRI) and DL has helped researchers identify functional biomarkers of neurological disorders (e.g., brain activation and connectivity) and pilot innovative diagnostic models. However, the knowledge required to perform DL analyses is often domain-specific and is not widely taught in the brain sciences (e.g., psychology, neuroscience, and cognitive science). Conversely, neurological diagnoses and neuroimaging training (e.g., fMRI) are largely restricted to the brain and medical sciences. In turn, these disciplinary knowledge barriers and distinct specializations can act as hurdles that prevent the combination of fMRI and DL pipelines. The complexity of fMRI and DL methods also hinders their clinical adoption and generalization to real-world diagnoses. For example, most current models are not designed for clinical settings or use by nonspecialized populations such as students, clinicians, and healthcare workers. Accordingly, there is a growing area of assistive tools (e.g., software and programming packages) that aim to streamline and increase the accessibility of fMRI and DL pipelines for the diagnoses of neurological disorders.

OBJECTIVES AND METHODS

In this study, we present an introductory guide to some popular DL and fMRI assistive tools. We also create an example autism spectrum disorder (ASD) classification model using assistive tools (e.g., Optuna, GIFT, and the ABIDE preprocessed repository), fMRI, and a convolutional neural network.

RESULTS

In turn, we provide researchers with a guide to assistive tools and give an example of a streamlined fMRI and DL pipeline.

CONCLUSIONS

We are confident that this study can help more researchers enter the field and create accessible fMRI and deep-learning diagnostic models for neurological disorders.

Explainability of three-dimensional convolutional neural networks for functional magnetic resonance imaging of Alzheimer's disease classification based on gradient-weighted class activation mapping

Currently, numerous studies focus on employing fMRI-based deep neural networks to diagnose neurological disorders such as Alzheimer's Disease (AD), yet only a handful have provided results regarding explainability. We address this gap by applying several prevalent explainability methods such as gradient-weighted class activation mapping (Grad-CAM) to an fMRI-based 3D-VGG16 network for AD diagnosis to improve the model's explainability. The aim is to explore the specific Region of Interest (ROI) of brain the model primarily focuses on when making predictions, as well as whether there are differences in these ROIs between AD and normal controls (NCs). First, we utilized multiple resting-state functional activity maps including ALFF, fALFF, ReHo, and VMHC to reduce the complexity of fMRI data, which differed from many studies that utilized raw fMRI data. Compared to methods utilizing raw fMRI data, this manual feature extraction approach may potentially alleviate the model's burden. Subsequently, 3D-VGG16 were employed for AD classification, where the final fully connected layers were replaced with a Global Average Pooling (GAP) layer, aimed at mitigating overfitting while preserving spatial information within the feature maps. The model achieved a maximum of 96.4% accuracy on the test set. Finally, several 3D CAM methods were employed to interpret the models. In the explainability results of the models with relatively high accuracy, the highlighted ROIs were primarily located in the precuneus and the hippocampus for AD subjects, while the models focused on the entire brain for NC. This supports current research on ROIs involved in AD. We believe that explaining deep learning models would not only provide support for existing research on brain disorders, but also offer important referential recommendations for the study of currently unknown etiologies.

Exploring Schizophrenia Classification Through Multimodal MRI and Deep Graph Neural Networks: Unveiling Brain Region-Specific Weight Discrepancies and Their Association With Cell-Type Specific Transcriptomic Features

BACKGROUND AND HYPOTHESIS

Schizophrenia (SZ) is a prevalent mental disorder that imposes significant health burdens. Diagnostic accuracy remains challenging due to clinical subjectivity. To address this issue, we explore magnetic resonance imaging (MRI) as a tool to enhance SZ diagnosis and provide objective references and biomarkers. Using deep learning with graph convolution, we represent MRI data as graphs, aligning with brain structure, and improving feature extraction, and classification. Integration of multiple modalities is expected to enhance classification.

STUDY DESIGN

Our study enrolled 683 SZ patients and 606 healthy controls from 7 hospitals, collecting structural MRI and functional MRI data. Both data types were represented as graphs, processed by 2 graph attention networks, and fused for classification. Grad-CAM with graph convolution ensured interpretability, and partial least squares analyzed gene expression in brain regions.

STUDY RESULTS

Our method excelled in the classification task, achieving 83.32% accuracy, 83.41% sensitivity, and 83.20% specificity in 10-fold cross-validation, surpassing traditional methods. And our multimodal approach outperformed unimodal methods. Grad-CAM identified potential brain biomarkers consistent with gene analysis and prior research.

CONCLUSIONS

Our study demonstrates the effectiveness of deep learning with graph attention networks, surpassing previous SZ diagnostic methods. Multimodal MRI's superiority over unimodal MRI confirms our initial hypothesis. Identifying potential brain biomarkers alongside gene biomarkers holds promise for advancing objective SZ diagnosis and research in SZ.

A review of the applications of generative adversarial networks to structural and functional MRI based diagnostic classification of brain disorders

Structural and functional MRI (magnetic resonance imaging) based diagnostic classification using machine learning has long held promise, but there are many roadblocks to achieving their potential. While traditional machine learning models suffered from their inability to capture the complex non-linear mapping, deep learning models tend to overfit the model. This is because there is data scarcity and imbalanced classes in neuroimaging; it is expensive to acquire data from human subjects and even more so in clinical populations. Due to their ability to augment data by learning underlying distributions, generative adversarial networks (GAN) provide a potential solution to this problem. Here, we provide a methodological primer on GANs and review the applications of GANs to classification of mental health disorders from neuroimaging data such as functional MRI and showcase the progress made thus far. We also highlight gaps in methodology as well as interpretability that are yet to be addressed. This provides directions about how the field can move forward. We suggest that since there are a range of methodological choices available to users, it is critical for users to interact with method developers so that the latter can tailor their development according to the users' needs. The field can be enriched by such synthesis between method developers and users in neuroimaging.

Determination of Alzheimer's disease based on morphology and atrophy using machine learning combined with automated segmentation

BACKGROUND

To evaluate the degree of cerebral atrophy for Alzheimer's disease (AD), voxel-based morphometry has been performed with magnetic resonance imaging. Detailed morphological changes in a specific tissue area having the most evidence of atrophy were not considered by the machine-learning technique.

PURPOSE

To develop a machine-learning system that can capture morphology features for determination of atrophy of brain tissue in early-stage AD and classification of healthy participants or patients.

MATERIAL AND METHODS

Three-dimensional T1-weighted (3D-T1W) data were obtained from AD Neuroimaging Initiative (200 healthy controls and 200 patients with early-stage AD). Automated segmentation of 3D-T1W data was performed. Deep learning (DL) and support vector machine (SVM) were trained using 66-segmented volume values as input and AD diagnosis as output. DL was performed using 66 volume values or gray matter (GM) and white matter (WM) volume values. SVM learning was performed using 66 volume values and six regions with high variable importance. 3D convolutional neural network (3D-CNN) was trained using the segmented images. Accuracy and area under curve (AUC) were obtained. Variable importance was evaluated from logistic regression analysis.

RESULTS

DL for GM and WM volume values, accuracy 0.6; SVM for all volume values, accuracy 0.82 and AUC 0.81; DL for all volume values, accuracy 0.82 and AUC 0.8; 3D-CNN using segmental images of the whole brain, accuracy 0.5 and AUC 0.51. SVM using volume values of six regions, accuracy 0.82; image-based 3D-CNN, highest accuracy 0.69.

CONCLUSION

Our results show that atrophic features are more considerable than morphological features in the early detection of AD.

Classification of Alzheimer's disease: application of a transfer learning deep Q-network method

Early diagnosis is crucial to slowing the progression of Alzheimer's disease (AD), so it is urgent to find an effective diagnostic method for AD. This study intended to investigate whether the transfer learning approach of deep Q-network (DQN) could effectively distinguish AD patients using local metrics of resting-state functional magnetic resonance imaging (rs-fMRI) as features. This study included 1310 subjects from the Consortium for Reliability and Reproducibility (CoRR) and 50 subjects from the Alzheimer's Disease Neuroimaging Initiative (ADNI) GO/2. The amplitude of low-frequency fluctuation (ALFF), fractional ALFF (fALFF) and percent amplitude of fluctuation (PerAF) were extracted as features using the Power 264 atlas. Based on gender bias in AD, we searched for transferable similar parts between the CoRR feature matrix and the ADNI feature matrix, resulting in the CoRR similar feature matrix served as the source domain and the ADNI similar feature matrix served as the target domain. A DQN classifier was pre-trained in the source domain and transferred to the target domain. Finally, the transferred DQN classifier was used to classify AD and healthy controls (HC). A permutation test was performed. The DQN transfer learning achieved a classification accuracy of 86.66% (p < 0.01), recall of 83.33% and precision of 83.33%. The findings suggested that the transfer learning approach using DQN could be an effective way to distinguish AD from HC. It also revealed the potential value of local brain activity in AD clinical diagnosis.

Preserving specificity in federated graph learning for fMRI-based neurological disorder identification

Resting-state functional magnetic resonance imaging (rs-fMRI) offers a non-invasive approach to examining abnormal brain connectivity associated with brain disorders. Graph neural network (GNN) gains popularity in fMRI representation learning and brain disorder analysis with powerful graph representation capabilities. Training a general GNN often necessitates a large-scale dataset from multiple imaging centers/sites, but centralizing multi-site data generally faces inherent challenges related to data privacy, security, and storage burden. Federated Learning (FL) enables collaborative model training without centralized multi-site fMRI data. Unfortunately, previous FL approaches for fMRI analysis often ignore site-specificity, including demographic factors such as age, gender, and education level. To this end, we propose a specificity-aware federated graph learning (SFGL) framework for rs-fMRI analysis and automated brain disorder identification, with a server and multiple clients/sites for federated model aggregation and prediction. At each client, our model consists of a shared and a personalized branch, where parameters of the shared branch are sent to the server while those of the personalized branch remain local. This can facilitate knowledge sharing among sites and also helps preserve site specificity. In the shared branch, we employ a spatio-temporal attention graph isomorphism network to learn dynamic fMRI representations. In the personalized branch, we integrate vectorized demographic information (i.e., age, gender, and education years) and functional connectivity networks to preserve site-specific characteristics. Representations generated by the two branches are then fused for classification. Experimental results on two fMRI datasets with a total of 1218 subjects suggest that SFGL outperforms several state-of-the-art approaches.

Discriminative analysis of schizophrenia patients using an integrated model combining 3D CNN with 2D CNN: A multimodal MR image and connectomics analysis

OBJECTIVE

Few studies have applied deep learning to the discriminative analysis of schizophrenia (SZ) patients using the fusional features of multimodal MRI data. Here, we proposed an integrated model combining a 3D convolutional neural network (CNN) with a 2D CNN to classify SZ patients.

METHOD

Structural MRI (sMRI) and resting-state functional MRI (rs-fMRI) data were acquired for 140 SZ patients and 205 normal controls. We computed structural connectivity (SC) from the sMRI data as well as functional connectivity (FC), amplitude of low-frequency fluctuation (ALFF), and regional homogeneity (ReHo) from the rs-fMRI data. The 3D images of T1, ReHo, and ALFF were used as the inputs for the 3D CNN model, while the SC and FC matrices were used as the inputs for the 2D CNN model. Moreover, we added squeeze and excitation blocks (SE-blocks) to each layer of the integrated model and used a support vector machine (SVM) to replace the softmax classifier.

RESULTS

The integrated model proposed in this study, using the fusional features of the T1 images, and the matrices of FC, showed the best performance. The use of the SE-blocks and SVM classifiers significantly improved the performance of the integrated model, in which the accuracy, sensitivity, specificity, area under the curve, and F1-score were 89.86%, 86.21%, 92.50%, 89.35%, and 87.72%, respectively.

CONCLUSIONS

Our findings indicated that an integrated model combining 3D CNN with 2D CNN is a promising method to improve the classification performance of SZ patients and has potential for the clinical diagnosis of psychiatric diseases.

Predicting suicidality in late-life depression by 3D convolutional neural network and cross-sample entropy analysis of resting-state fMRI

BACKGROUND

Predicting suicide is a pressing issue among older adults; however, predicting its risk is difficult. Capitalizing on the recent development of machine learning, considerable progress has been made in predicting complex behavior such as suicide. As depression remained the strongest risk for suicide, we aimed to apply deep learning algorithms to identify suicidality in a group with late-life depression (LLD).

METHODS

We enrolled 83 patients with LLD, 35 of which were non-suicidal and 48 were suicidal, including 26 with only suicidal ideation and 22 with past suicide attempts, for resting-state functional magnetic resonance imaging (MRI). Cross-sample entropy (CSE) analysis was conducted to examine the complexity of MRI signals among brain regions. Three-dimensional (3D) convolutional neural networks (CNNs) were used, and the classification accuracy in each brain region was averaged to predict suicidality after sixfold cross-validation.

RESULTS

We found brain regions with a mean accuracy above 75% to predict suicidality located mostly in default mode, fronto-parietal, and cingulo-opercular resting-state networks. The models with right amygdala and left caudate provided the most reliable accuracy in all cross-validation folds, indicating their neurobiological importance in late-life suicide.

CONCLUSION

Combining CSE analysis and the 3D CNN, several brain regions were found to be associated with suicidality.

Automatic recognition of schizophrenia from brain-network features using graph convolutional neural network

Schizophrenia is a severe mental illness that imposes considerable economic burden on families and society. However, its clinical diagnosis primarily relies on scales and doctors' clinical experience and lacks an objective and accurate diagnostic approach. In recent years, graph convolutional neural networks (GCN) have been used to assist in psychiatric diagnosis owing to their ability to learn spatial-association information. Therefore, this study proposes a schizophrenia automatic recognition model based on graph convolutional neural network. Herein, the resting-state electroencephalography (EEG) data of 103 first-episode schizophrenia patients and 92 normal controls (NCs) were obtained. The automatic recognition model was trained with a nodal feature matrix that comprised the time and frequency-domain features of the EEG signals and local features of the brain network. The most significant regions that contributed to the model classification were identified, and the correlation between the node topological features of each significant region and clinical evaluation metrics was explored. Experiments were conducted to evaluate the performance of the model using 10-fold cross-validation. The best performance in the theta frequency band with a 6 s epoch length and phase-locked value. The recognition accuracy was 90.01%. The most significant region for identifying with first-episode schizophrenia patients and NCs was located in the parietal lobe. The results of this study verify the applicability of the proposed novel method for the identification and diagnosis of schizophrenia.

Artificial intelligence in psychiatry research, diagnosis, and therapy

Psychiatric disorders are now responsible for the largest proportion of the global burden of disease, and even more challenges have been seen during the COVID-19 pandemic. Artificial intelligence (AI) is commonly used to facilitate the early detection of disease, understand disease progression, and discover new treatments in the fields of both physical and mental health. The present review provides a broad overview of AI methodology and its applications in data acquisition and processing, feature extraction and characterization, psychiatric disorder classification, potential biomarker detection, real-time monitoring, and interventions in psychiatric disorders. We also comprehensively summarize AI applications with regard to the early warning, diagnosis, prognosis, and treatment of specific psychiatric disorders, including depression, schizophrenia, autism spectrum disorder, attention-deficit/hyperactivity disorder, addiction, sleep disorders, and Alzheimer's disease. The advantages and disadvantages of AI in psychiatry are clarified. We foresee a new wave of research opportunities to facilitate and improve AI technology and its long-term implications in psychiatry during and after the COVID-19 era.

A state-of-the-art review of functional magnetic resonance imaging technique integrated with advanced statistical modeling and machine learning for primary headache diagnosis

Primary headache is a very common and burdensome functional headache worldwide, which can be classified as migraine, tension-type headache (TTH), trigeminal autonomic cephalalgia (TAC), and other primary headaches. Managing and treating these different categories require distinct approaches, and accurate diagnosis is crucial. Functional magnetic resonance imaging (fMRI) has become a research hotspot to explore primary headache. By examining the interrelationships between activated brain regions and improving temporal and spatial resolution, fMRI can distinguish between primary headaches and their subtypes. Currently the most commonly used is the cortical brain mapping technique, which is based on blood oxygen level-dependent functional magnetic resonance imaging (BOLD-fMRI). This review sheds light on the state-of-the-art advancements in data analysis based on fMRI technology for primary headaches along with their subtypes. It encompasses not only the conventional analysis methodologies employed to unravel pathophysiological mechanisms, but also deep-learning approaches that integrate these techniques with advanced statistical modeling and machine learning. The aim is to highlight cutting-edge fMRI technologies and provide new insights into the diagnosis of primary headaches.

A meta-analysis and systematic review of single vs. multimodal neuroimaging techniques in the classification of psychosis

BACKGROUND

Psychotic disorders are characterized by structural and functional abnormalities in brain networks. Neuroimaging techniques map and characterize such abnormalities using unique features (e.g., structural integrity, coactivation). However, it is unclear if a specific method, or a combination of modalities, is particularly effective in identifying differences in brain networks of someone with a psychotic disorder.

METHODS

A systematic meta-analysis evaluated machine learning classification of schizophrenia spectrum disorders in comparison to healthy control participants using various neuroimaging modalities (i.e., T1-weighted imaging (T1), diffusion tensor imaging (DTI), resting state functional connectivity (rs-FC), or some combination (multimodal)). Criteria for manuscript inclusion included whole-brain analyses and cross-validation to provide a complete picture regarding the predictive ability of large-scale brain systems in psychosis. For this meta-analysis, we searched Ovid MEDLINE, PubMed, PsychInfo, Google Scholar, and Web of Science published between inception and March 13th 2023. Prediction results were averaged for studies using the same dataset, but parallel analyses were run that included studies with pooled sample across many datasets. We assessed bias through funnel plot asymmetry. A bivariate regression model determined whether differences in imaging modality, demographics, and preprocessing methods moderated classification. Separate models were run for studies with internal prediction (via cross-validation) and external prediction.

RESULTS

93 studies were identified for quantitative review (30 T1, 9 DTI, 40 rs-FC, and 14 multimodal). As a whole, all modalities reliably differentiated those with schizophrenia spectrum disorders from controls (OR = 2.64 (95%CI = 2.33 to 2.95)). However, classification was relatively similar across modalities: no differences were seen across modalities in the classification of independent internal data, and a small advantage was seen for rs-FC studies relative to T1 studies in classification in external datasets. We found large amounts of heterogeneity across results resulting in significant signs of bias in funnel plots and Egger's tests. Results remained similar, however, when studies were restricted to those with less heterogeneity, with continued small advantages for rs-FC relative to structural measures. Notably, in all cases, no significant differences were seen between multimodal and unimodal approaches, with rs-FC and unimodal studies reporting largely overlapping classification performance. Differences in demographics and analysis or denoising were not associated with changes in classification scores.

CONCLUSIONS

The results of this study suggest that neuroimaging approaches have promise in the classification of psychosis. Interestingly, at present most modalities perform similarly in the classification of psychosis, with slight advantages for rs-FC relative to structural modalities in some specific cases. Notably, results differed substantially across studies, with suggestions of biased effect sizes, particularly highlighting the need for more studies using external prediction and large sample sizes. Adopting more rigorous and systematized standards will add significant value toward understanding and treating this critical population.

Discriminative analysis of schizophrenia patients using graph convolutional networks: A combined multimodal MRI and connectomics analysis

Introduction

Recent studies in human brain connectomics with multimodal magnetic resonance imaging (MRI) data have widely reported abnormalities in brain structure, function and connectivity associated with schizophrenia (SZ). However, most previous discriminative studies of SZ patients were based on MRI features of brain regions, ignoring the complex relationships within brain networks.

Methods

We applied a graph convolutional network (GCN) to discriminating SZ patients using the features of brain region and connectivity derived from a combined multimodal MRI and connectomics analysis. Structural magnetic resonance imaging (sMRI) and resting-state functional magnetic resonance imaging (rs-fMRI) data were acquired from 140 SZ patients and 205 normal controls. Eighteen types of brain graphs were constructed for each subject using 3 types of node features, 3 types of edge features, and 2 brain atlases. We investigated the performance of 18 brain graphs and used the TopK pooling layers to highlight salient brain regions (nodes in the graph).

Results

The GCN model, which used functional connectivity as edge features and multimodal features (sMRI + fMRI) of brain regions as node features, obtained the highest average accuracy of 95.8%, and outperformed other existing classification studies in SZ patients. In the explainability analysis, we reported that the top 10 salient brain regions, predominantly distributed in the prefrontal and occipital cortices, were mainly involved in the systems of emotion and visual processing.

Discussion

Our findings demonstrated that GCN with a combined multimodal MRI and connectomics analysis can effectively improve the classification of SZ at an individual level, indicating a promising direction for the diagnosis of SZ patients. The code is available at https://github.com/CXY-scut/GCN-SZ.git.

Automated detection of schizophrenia using deep learning: a review for the last decade

Schizophrenia (SZ) is a devastating mental disorder that disrupts higher brain functions like thought, perception, etc., with a profound impact on the individual's life. Deep learning (DL) can detect SZ automatically by learning signal data characteristics hierarchically without the need for feature engineering associated with traditional machine learning. We performed a systematic review of DL models for SZ detection. Various deep models like long short-term memory, convolution neural networks, AlexNet, etc., and composite methods have been published based on electroencephalographic signals, and structural and/or functional magnetic resonance imaging acquired from SZ patients and healthy patients control subjects in diverse public and private datasets. The studies, the study datasets, and model methodologies are reported in detail. In addition, the challenges of DL models for SZ diagnosis and future works are discussed.

A Class Activation Map-Based Interpretable Transfer Learning Model for Automated Detection of ADHD from fMRI Data

Automatic detection of Attention Deficit Hyperactivity Disorder (ADHD) based on the functional Magnetic Resonance Imaging (fMRI) through Deep Learning (DL) is becoming a quite useful methodology due to the curse of-dimensionality problem of the data is solved. Also, this method proposes an invasive and robust solution to the variances in data acquisition and class distribution imbalances. In this paper, a transfer learning approach, specifically ResNet-50 type pre-trained 2D-Convolutional Neural Network (CNN) was used to automatically classify ADHD and healthy children. The results demonstrated that ResNet-50 architecture with 10-k cross-validation (CV) achieves an overall classification accuracy of 93.45%. The interpretation of the results was done via the Class Activation Map (CAM) analysis which showed that children with ADHD differed from controls in a wide range of brain areas including frontal, parietal and temporal lobes.

Optimized adaptive neuro-fuzzy inference system based on hybrid grey wolf-bat algorithm for schizophrenia recognition from EEG signals

Schizophrenia is a chronic mental disorder that impairs a person's thinking capacity, feelings and emotions, behavioural traits, etc., Emotional distortions, delusions, hallucinations, and incoherent speech are all some of the symptoms of schizophrenia, and cause disruption of routine activities. Computer-assisted diagnosis of schizophrenia is significantly needed to give its patients a higher quality of life. Hence, an improved adaptive neuro-fuzzy inference system based on the Hybrid Grey Wolf-Bat Algorithm for accurate prediction of schizophrenia from multi-channel EEG signals is presented in this study. The EEG signals are pre-processed using a Butterworth band pass filter and wICA initially, from which statistical, time-domain, frequency-domain, and spectral features are extracted. Discriminating features are selected using the ReliefF algorithm and are then forwarded to ANFIS for classification into either schizophrenic or normal. ANFIS is optimized by the Hybrid Grey Wolf-Bat Algorithm (HWBO) for better efficiency. The method is experimented on two separate EEG datasets-1 and 2, demonstrating an accuracy of 99.54% and 99.35%, respectively, with appreciable F1-score and MCC. Further experiments reveal the efficiency of the Hybrid Wolf-Bat algorithm in optimizing the ANFIS parameters when compared with traditional ANFIS model and other proven algorithms like genetic algorithm-ANFIS, particle optimization-ANFIS, crow search optimization algorithm-ANFIS and ant colony optimization algorithm-ANFIS, showing high R2 value and low RSME value. To provide a bias free classification, tenfold cross validation is performed which produced an accuracy of 97.8% and 98.5% on the two datasets respectively. Experimental outcomes demonstrate the superiority of the Hybrid Grey Wolf-Bat Algorithm over the similar techniques in predicting schizophrenia.

Deep Learning Classification of Treatment Response in Diabetic Painful Neuropathy: A Combined Machine Learning and Magnetic Resonance Neuroimaging Methodological Study

Functional magnetic resonance imaging (fMRI) has been shown successfully to assess and stratify patients with painful diabetic peripheral neuropathy (pDPN). This supports the idea of using neuroimaging as a mechanism-based technique to individualise therapy for patients with painful DPN. The aim of this study was to use deep learning to predict treatment response in patients with pDPN using resting state functional imaging (rs-fMRI). We divided 43 painful pDPN patients into responders and non-responders to lidocaine treatment (responders n = 29 and non-responders n = 14). We used rs-fMRI to extract functional connectivity features, using group independent component analysis (gICA), and performed automated treatment response deep learning classification with three-dimensional convolutional neural networks (3D-CNN). Using gICA we achieved an area under the receiver operating characteristic curve (AUC) of 96.60% and F1-Score of 95% in a ten-fold cross validation (CV) experiment using our described 3D-CNN algorithm. To our knowledge, this is the first study utilising deep learning methods to classify treatment response in pDPN.

Automatic diagnosis of late-life depression by 3D convolutional neural networks and cross-sample Entropy analysis from resting-state fMRI

Resting-state fMRI has been widely used in investigating the pathophysiology of late-life depression (LLD). Unlike the conventional linear approach, cross-sample entropy (CSE) analysis shows the nonlinear property in fMRI signals between brain regions. Moreover, recent advances in deep learning, such as convolutional neural networks (CNNs), provide a timely application for understanding LLD. Accurate and prompt diagnosis is essential in LLD; hence, this study aimed to combine CNN and CSE analysis to discriminate LLD patients and non-depressed comparison older adults based on brain resting-state fMRI signals. Seventy-seven older adults, including 49 patients and 28 comparison older adults, were included for fMRI scans. Three-dimensional CSEs with volumes corresponding to 90 seed regions of interest of each participant were developed and fed into models for disease classification and depression severity prediction. We obtained a diagnostic accuracy > 85% in the superior frontal gyrus (left dorsolateral and right orbital parts), left insula, and right middle occipital gyrus. With a mean root-mean-square error (RMSE) of 2.41, three separate models were required to predict depressive symptoms in the severe, moderate, and mild depression groups. The CSE volumes in the left inferior parietal lobule, left parahippocampal gyrus, and left postcentral gyrus performed best in each respective model. Combined complexity analysis and deep learning algorithms can classify patients with LLD from comparison older adults and predict symptom severity based on fMRI data. Such application can be utilized in precision medicine for disease detection and symptom monitoring in LLD.

Deep learning in neuroimaging data analysis: Applications, challenges, and solutions

Methods for the analysis of neuroimaging data have advanced significantly since the beginning of neuroscience as a scientific discipline. Today, sophisticated statistical procedures allow us to examine complex multivariate patterns, however most of them are still constrained by assuming inherent linearity of neural processes. Here, we discuss a group of machine learning methods, called deep learning, which have drawn much attention in and outside the field of neuroscience in recent years and hold the potential to surpass the mentioned limitations. Firstly, we describe and explain the essential concepts in deep learning: the structure and the computational operations that allow deep models to learn. After that, we move to the most common applications of deep learning in neuroimaging data analysis: prediction of outcome, interpretation of internal representations, generation of synthetic data and segmentation. In the next section we present issues that deep learning poses, which concerns multidimensionality and multimodality of data, overfitting and computational cost, and propose possible solutions. Lastly, we discuss the current reach of DL usage in all the common applications in neuroimaging data analysis, where we consider the promise of multimodality, capability of processing raw data, and advanced visualization strategies. We identify research gaps, such as focusing on a limited number of criterion variables and the lack of a well-defined strategy for choosing architecture and hyperparameters. Furthermore, we talk about the possibility of conducting research with constructs that have been ignored so far or/and moving toward frameworks, such as RDoC, the potential of transfer learning and generation of synthetic data.

Deep learning for neural decoding in motor cortex

Objective. Neural decoding is an important tool in neural engineering and neural data analysis. Of various machine learning algorithms adopted for neural decoding, the recently introduced deep learning is promising to excel. Therefore, we sought to apply deep learning to decode movement trajectories from the activity of motor cortical neurons.Approach. In this paper, we assessed the performance of deep learning methods in three different decoding schemes, concurrent, time-delay, and spatiotemporal. In the concurrent decoding scheme where the input to the network is the neural activity coincidental to the movement, deep learning networks including artificial neural network (ANN) and long-short term memory (LSTM) were applied to decode movement and compared with traditional machine learning algorithms. Both ANN and LSTM were further evaluated in the time-delay decoding scheme in which temporal delays are allowed between neural signals and movements. Lastly, in the spatiotemporal decoding scheme, we trained convolutional neural network (CNN) to extract movement information from images representing the spatial arrangement of neurons, their activity, and connectomes (i.e. the relative strengths of connectivity between neurons) and combined CNN and ANN to develop a hybrid spatiotemporal network. To reveal the input features of the CNN in the hybrid network that deep learning discovered for movement decoding, we performed a sensitivity analysis and identified specific regions in the spatial domain.Main results. Deep learning networks (ANN and LSTM) outperformed traditional machine learning algorithms in the concurrent decoding scheme. The results of ANN and LSTM in the time-delay decoding scheme showed that including neural data from time points preceding movement enabled decoders to perform more robustly when the temporal relationship between the neural activity and movement dynamically changes over time. In the spatiotemporal decoding scheme, the hybrid spatiotemporal network containing the concurrent ANN decoder outperformed single-network concurrent decoders.Significance. Taken together, our study demonstrates that deep learning could become a robust and effective method for the neural decoding of behavior.

Deep multimodal predictome for studying mental disorders

Characterizing neuropsychiatric disorders is challenging due to heterogeneity in the population. We propose combining structural and functional neuroimaging and genomic data in a multimodal classification framework to leverage their complementary information. Our objectives are two-fold (i) to improve the classification of disorders and (ii) to introspect the concepts learned to explore underlying neural and biological mechanisms linked to mental disorders. Previous multimodal studies have focused on naïve neural networks, mostly perceptron, to learn modality-wise features and often assume equal contribution from each modality. Our focus is on the development of neural networks for feature learning and implementing an adaptive control unit for the fusion phase. Our mid fusion with attention model includes a multilayer feed-forward network, an autoencoder, a bi-directional long short-term memory unit with attention as the features extractor, and a linear attention module for controlling modality-specific influence. The proposed model acquired 92% (p < .0001) accuracy in schizophrenia prediction, outperforming several other state-of-the-art models applied to unimodal or multimodal data. Post hoc feature analyses uncovered critical neural features and genes/biological pathways associated with schizophrenia. The proposed model effectively combines multimodal neuroimaging and genomics data for predicting mental disorders. Interpreting salient features identified by the model may advance our understanding of their underlying etiological mechanisms.

Multimodal MRI-Based Whole-Brain Assessment in Patients In Anoxoischemic Coma by Using 3D Convolutional Neural Networks

Background

There is an unfulfilled need to find the best way to automatically capture, analyze, organize, and merge structural and functional brain magnetic resonance imaging (MRI) data to ultimately extract relevant signals that can assist the medical decision process at the bedside of patients in postanoxic coma. We aimed to develop and validate a deep learning model to leverage multimodal 3D MRI whole-brain times series for an early evaluation of brain damages related to anoxoischemic coma.

Methods

This proof-of-concept, prospective, cohort study was undertaken at the intensive care unit affiliated with the University Hospital (Toulouse, France), between March 2018 and May 2020. All patients were scanned in coma state at least 2 days (4 ± 2 days) after cardiac arrest. Over the same period, age-matched healthy volunteers were recruited and included. Brain MRI quantification encompassed both “functional data” from regions of interest (precuneus and posterior cingulate cortex) with whole-brain functional connectivity analysis and “structural data” (gray matter volume, T1-weighted, fractional anisotropy, and mean diffusivity). A specifically designed 3D convolutional neuronal network (CNN) was created to allow conscious state discrimination (coma vs. controls) by using raw MRI indices as the input. A voxel-wise visualization method based on the study of convolutional filters was applied to support CNN outcome. The Ethics Committee of the University Teaching Hospital of Toulouse, France (2018-A31) approved the study and informed consent was obtained from all participants.

Results

The final cohort consisted of 29 patients in postanoxic coma and 34 healthy volunteers. Coma patients were successfully discerned from controls by using 3D CNN in combination with different MR indices. The best accuracy was achieved by functional MRI data, in particular with resting-state functional MRI of the posterior cingulate cortex, with an accuracy of 0.96 (range 0.94–0.98) on the test set from 10-time repeated tenfold cross-validation. Even more satisfactory performances were achieved through the majority voting strategy, which was able to compensate for mistakes from single MR indices. Visualization maps allowed us to identify the most relevant regions for each MRI index, notably regions previously described as possibly being involved in consciousness emergence. Interestingly, a posteriori analysis of misclassified patients indicated that they may present some common functional MRI traits with controls, which suggests further favorable outcomes.

Conclusions

A fully automated identification of clinically relevant signals from complex multimodal neuroimaging data is a major research topic that may bring a radical paradigm shift in the neuroprognostication of patients with severe brain injury. We report for the first time a successful discrimination between patients in postanoxic coma patients from people serving as controls by using 3D CNN whole-brain structural and functional MRI data.

Clinical Trial Numberhttp://ClinicalTrials.gov (No. NCT03482115).

Supplementary Information

The online version contains supplementary material available at 10.1007/s12028-022-01525-z.

SSPNet: An interpretable 3D-CNN for classification of schizophrenia using phase maps of resting-state complex-valued fMRI data

Convolutional neural networks (CNNs) have shown promising results in classifying individuals with mental disorders such as schizophrenia using resting-state fMRI data. However, complex-valued fMRI data is rarely used since additional phase data introduces high-level noise though it is potentially useful information for the context of classification. As such, we propose to use spatial source phase (SSP) maps derived from complex-valued fMRI data as the CNN input. The SSP maps are not only less noisy, but also more sensitive to spatial activation changes caused by mental disorders than magnitude maps. We build a 3D-CNN framework with two convolutional layers (named SSPNet) to fully explore the 3D structure and voxel-level relationships from the SSP maps. Two interpretability modules, consisting of saliency map generation and gradient-weighted class activation mapping (Grad-CAM), are incorporated into the well-trained SSPNet to provide additional information helpful for understanding the output. Experimental results from classifying schizophrenia patients (SZs) and healthy controls (HCs) show that the proposed SSPNet significantly improved accuracy and AUC compared to CNN using magnitude maps extracted from either magnitude-only (by 23.4 and 23.6% for DMN) or complex-valued fMRI data (by 10.6 and 5.8% for DMN). SSPNet captured more prominent HC-SZ differences in saliency maps, and Grad-CAM localized all contributing brain regions with opposite strengths for HCs and SZs within SSP maps. These results indicate the potential of SSPNet as a sensitive tool that may be useful for the development of brain-based biomarkers of mental disorders.

Going deep into schizophrenia with artificial intelligence

Despite years of research, the mechanisms governing the onset, relapse, symptomatology, and treatment of schizophrenia (SZ) remain elusive. The lack of appropriate analytic tools to deal with the heterogeneity and complexity of SZ may be one of the reasons behind this situation. Deep learning, a subfield of artificial intelligence (AI) inspired by the nervous system, has recently provided an accessible way of modeling and analyzing complex, high-dimensional, nonlinear systems. The unprecedented accuracy of deep learning algorithms in classification and prediction tasks has revolutionized a wide range of scientific fields and is rapidly permeating SZ research. Deep learning has the potential of becoming a valuable aid for clinicians in the prediction, diagnosis, and treatment of SZ, especially in combination with principles from Bayesian statistics. Furthermore, deep learning could become a powerful tool for uncovering the mechanisms underlying SZ thanks to a growing number of techniques designed for improving model interpretability and causal reasoning. The purpose of this article is to introduce SZ researchers to the field of deep learning and review its latest applications in SZ research. In general, existing studies have yielded impressive results in classification and outcome prediction tasks. However, methodological concerns related to the assessment of model performance in several studies, the widespread use of small training datasets, and the little clinical value of some models suggest that some of these results should be taken with caution.

Alzheimer disease stages identification based on correlation transfer function system using resting-state functional magnetic resonance imaging

Alzheimer’s disease (AD) affects the quality of life as it causes; memory loss, difficulty in thinking, learning, and performing familiar tasks. Resting-state functional magnetic resonance imaging (rs-fMRI) has been widely used to investigate and analyze different brain regions for AD identification. This study investigates the effectiveness of using correlated transfer function (CorrTF) as a new biomarker to extract the essential features from rs-fMRI, along with support vector machine (SVM) ordered hierarchically, in order to distinguish between the different AD stages. Additionally, we explored the regions, showing significant changes based on the CorrTF extracted features’ strength among different AD stages. First, the process was initialized by applying the preprocessing on rs-fMRI data samples to reduce noise and retain the essential information. Then, the automated anatomical labeling (AAL) atlas was employed to divide the brain into 116 regions, where the intensity time series was calculated, and the CorrTF features were extracted for each region. The proposed framework employed the SVM classifier in two different methodologies, hierarchical and flat multi-classification schemes, to differentiate between the different AD stages for early detection purposes. The ADNI rs-fMRI dataset, employed in this study, consists of 167, 102, 129, and 114 normal, early, late mild cognitive impairment (MCI), and AD subjects, respectively. The proposed schemes achieved an average accuracy of 98.2% and 95.5% for hierarchical and flat multi-classification tasks, respectively, calculated using ten folds cross-validation. Therefore, CorrTF is considered a promising biomarker for AD early-stage identification. Moreover, the significant changes in the strengths of CorrTF connections among the different AD stages can help us identify and explore the affected brain regions and their latent associations during the progression of AD.

A Comprehensive Survey on the Detection, Classification, and Challenges of Neurological Disorders

Simple Summary

This study represents a resourceful review article that can deliver resources on neurological diseases and their implemented classification algorithms to reveal the future direction of researchers. Researchers interested in studying neurological diseases and previously implemented techniques in this field can follow this article. Various challenges occur in detecting different stages of the disorders. A limited amount of labeled and unlabeled datasets and other limitations is represented in this article to assist them in finding out the directions. The authors’ purpose for composing this article is to make a straightforward and concrete path for researchers to quickly find the way and the scope in this field for implementing future research on neurological disease detection.

Abstract

Neurological disorders (NDs) are becoming more common, posing a concern to pregnant women, parents, healthy infants, and children. Neurological disorders arise in a wide variety of forms, each with its own set of origins, complications, and results. In recent years, the intricacy of brain functionalities has received a better understanding due to neuroimaging modalities, such as magnetic resonance imaging (MRI), magnetoencephalography (MEG), and positron emission tomography (PET), etc. With high-performance computational tools and various machine learning (ML) and deep learning (DL) methods, these modalities have discovered exciting possibilities for identifying and diagnosing neurological disorders. This study follows a computer-aided diagnosis methodology, leading to an overview of pre-processing and feature extraction techniques. The performance of existing ML and DL approaches for detecting NDs is critically reviewed and compared in this article. A comprehensive portion of this study also shows various modalities and disease-specified datasets that detect and records images, signals, and speeches, etc. Limited related works are also summarized on NDs, as this domain has significantly fewer works focused on disease and detection criteria. Some of the standard evaluation metrics are also presented in this study for better result analysis and comparison. This research has also been outlined in a consistent workflow. At the conclusion, a mandatory discussion section has been included to elaborate on open research challenges and directions for future work in this emerging field.

ConCeptCNN: A novel multi-filter convolutional neural network for the prediction of neurodevelopmental disorders using brain connectome

BACKGROUND

Deep convolutional neural network (CNN) and its derivatives have recently shown great promise in the prediction of brain disorders using brain connectome data. Existing deep CNN methods using single global row and column convolutional filters have limited ability to extract discriminative information from brain connectome for prediction tasks.

PURPOSE

This paper presents a novel deep Connectome-Inception CNN (ConCeptCNN) model, which is developed based on multiple convolutional filters. The proposed model is used to extract topological features from brain connectome data for neurological disorders classification and analysis.

METHODS

The ConCeptCNN uses multiple vector-shaped filters extract topological information from the brain connectome at different levels for complementary feature embeddings of brain connectome. The proposed model is validated using two datasets: the Neuro Bureau ADHD-200 dataset and the Cincinnati Early Prediction Study (CINEPS) dataset.

RESULTS

In a cross-validation experiment, the ConCeptCNN achieved a prediction accuracy of 78.7% for the detection of ADHD in adolescents and an accuracy of 81.6% for the prediction of cognitive deficits at 2-year corrected age in very preterm infants. In addition to the classification tasks, the ConCeptCNN identified several brain regions that are discriminative to neurodevelopmental disorders.

CONCLUSIONS

We compared the ConCeptCNN with several peer CNN methods. The results demonstrated that proposed model improves overall classification performance of neurodevelopmental disorders prediction tasks. This article is protected by copyright. All rights reserved.

A computer-aid multi-task light-weight network for macroscopic feces diagnosis

The abnormal traits and colors of feces typically indicate that the patients are probably suffering from tumor or digestive-system diseases. Thus a fast, accurate and automatic health diagnosis system based on feces is urgently necessary for improving the examination speed and reducing the infection risk. The rarity of the pathological images would deteriorate the accuracy performance of the trained models. In order to alleviate this problem, we employ augmentation and over-sampling to expand the samples of the classes that have few samples in the training batch. In order to achieve an impressive recognition performance and leverage the latent correlation between the traits and colors of feces pathological samples, a multi-task network is developed to recognize colors and traits of the macroscopic feces images. The parameter number of a single multi-task network is generally much smaller than the total parameter number of multiple single-task networks, so the storage cost is reduced. The loss function of the multi-task network is the weighted sum of the losses of the two tasks. In this paper, the weights of the tasks are determined according to their difficulty levels that are measured by the fitted linear functions. The sufficient experiments confirm that the proposed method can yield higher accuracies, and the efficiency is also improved.

Discriminative Analysis of Schizophrenia Patients Using Topological Properties of Structural and Functional Brain Networks: A Multimodal Magnetic Resonance Imaging Study

Interest in the application of machine learning (ML) techniques to multimodal magnetic resonance imaging (MRI) data for the diagnosis of schizophrenia (SZ) at the individual level is growing. However, a few studies have applied the features of structural and functional brain networks derived from multimodal MRI data to the discriminative analysis of SZ patients at different clinical stages. In this study, 205 normal controls (NCs), 61 first-episode drug-naive SZ (FESZ) patients, and 79 chronic SZ (CSZ) patients were recruited. We acquired their structural MRI, diffusion tensor imaging, and resting-state functional MRI data and constructed brain networks for each participant, including the gray matter network (GMN), white matter network (WMN), and functional brain network (FBN). We then calculated 3 nodal properties for each brain network, including degree centrality, nodal efficiency, and betweenness centrality. Two classifications (SZ vs. NC and FESZ vs. CSZ) were performed using five ML algorithms. We found that the SVM classifier with the input features of the combination of nodal properties of both the GMN and FBN achieved the best performance to discriminate SZ patients from NCs [accuracy, 81.2%; area under the receiver operating characteristic curve (AUC), 85.2%; p &lt; 0.05]. Moreover, the SVM classifier with the input features of the combination of the nodal properties of both the GMN and WMN achieved the best performance to discriminate FESZ from CSZ patients (accuracy, 86.2%; AUC, 92.3%; p &lt; 0.05). Furthermore, the brain areas in the subcortical/cerebellum network and the frontoparietal network showed significant importance in both classifications. Together, our findings provide new insights to understand the neuropathology of SZ and further highlight the potential advantages of multimodal network properties for identifying SZ patients at different clinical stages.

Bridging structural MRI with cognitive function for individual level classification of early psychosis via deep learning

INTRODUCTION

Recent efforts have been made to apply machine learning and deep learning approaches to the automated classification of schizophrenia using structural magnetic resonance imaging (sMRI) at the individual level. However, these approaches are less accurate on early psychosis (EP) since there are mild structural brain changes at early stage. As cognitive impairments is one main feature in psychosis, in this study we apply a multi-task deep learning framework using sMRI with inclusion of cognitive assessment to facilitate the classification of patients with EP from healthy individuals.

METHOD

Unlike previous studies, we used sMRI as the direct input to perform EP classifications and cognitive estimations. The proposed deep learning model does not require time-consuming volumetric or surface based analysis and can provide additionally cognition predictions. Experiments were conducted on an in-house data set with 77 subjects and a public ABCD HCP-EP data set with 164 subjects.

RESULTS

We achieved 74.9 ± 4.3% five-fold cross-validated accuracy and an area under the curve of 71.1 ± 4.1% on EP classification with the inclusion of cognitive estimations.

DISCUSSION

We reveal the feasibility of automated cognitive estimation using sMRI by deep learning models, and also demonstrate the implicit adoption of cognitive measures as additional information to facilitate EP classifications from healthy controls.

Automatic Diagnosis of Schizophrenia in EEG Signals Using CNN-LSTM Models

Schizophrenia (SZ) is a mental disorder whereby due to the secretion of specific chemicals in the brain, the function of some brain regions is out of balance, leading to the lack of coordination between thoughts, actions, and emotions. This study provides various intelligent deep learning (DL)-based methods for automated SZ diagnosis via electroencephalography (EEG) signals. The obtained results are compared with those of conventional intelligent methods. To implement the proposed methods, the dataset of the Institute of Psychiatry and Neurology in Warsaw, Poland, has been used. First, EEG signals were divided into 25 s time frames and then were normalized by z-score or norm L2. In the classification step, two different approaches were considered for SZ diagnosis via EEG signals. In this step, the classification of EEG signals was first carried out by conventional machine learning methods, e.g., support vector machine, k-nearest neighbors, decision tree, naïve Bayes, random forest, extremely randomized trees, and bagging. Various proposed DL models, namely, long short-term memories (LSTMs), one-dimensional convolutional networks (1D-CNNs), and 1D-CNN-LSTMs, were used in the following. In this step, the DL models were implemented and compared with different activation functions. Among the proposed DL models, the CNN-LSTM architecture has had the best performance. In this architecture, the ReLU activation function with the z-score and L2-combined normalization was used. The proposed CNN-LSTM model has achieved an accuracy percentage of 99.25%, better than the results of most former studies in this field. It is worth mentioning that to perform all simulations, the k-fold cross-validation method with k = 5 has been used.

Multi-modal deep learning of functional and structural neuroimaging and genomic data to predict mental illness

Neuropsychiatric disorders such as schizophrenia are very heterogeneous in nature and typically diagnosed using self-reported symptoms. This makes it difficult to pose a confident prediction on the cases and does not provide insight into the underlying neural and biological mechanisms of these disorders. Combining neuroimaging and genomic data with a multi-modal 'predictome' paves the way for biologically informed markers and may improve prediction reliability. With that, we develop a multi-modal deep learning framework by fusing data from different modalities to capture the interaction between the latent features and evaluate their complementary information in characterizing schizophrenia. Our deep model uses structural MRI, functional MRI, and genome-wide polymorphism data to perform the classification task. It includes a multi-layer feed-forward network, an encoder, and a long short-term memory (LSTM) unit with attention to learn the latent features and adopt a joint training scheme capturing synergies between the modalities. The hybrid network also uses different regularizers for addressing the inherent overfitting and modality-specific bias in the multi-modal setup. Next, we run the network through a saliency model to analyze the learned features. Integrating modalities enhances the performance of the classifier, and our framework acquired 88% (P < 0.0001) accuracy on a dataset of 437 subjects. The trimodal accuracy is comparable to the state-of-the-art performance on a data collection of this size and outperforms the unimodal and bimodal baselines we compared. Model introspection was used to expose the salient neural features and genes/biological pathways associated with schizophrenia. To our best knowledge, this is the first approach that fuses genomic information with structural and functional MRI biomarkers for predicting schizophrenia. We believe this type of modality blending can better explain the disorder's dynamics by adding cross-modal prospects.Clinical Relevance- This study combinedly learns imaging and genomic features for the classification of schizophrenia. The data fusion scheme extracts modality interactions, and the saliency experiments report multiple functional and structural networks closely connected to the disorder.

Multi-Class brain normality and abnormality diagnosis using modified Faster R-CNN

BACKGROUND AND OBJECTIVE

The detection and analysis of brain disorders through medical imaging techniques are extremely important to get treatment on time and sustain a healthy lifestyle. Disorders cause permanent brain damage and alleviate the lifespan. Moreover, the classification of large volumes of medical image data manually by medicine experts is tiring, time-consuming, and prone to errors. This study aims to diagnose brain normality and abnormalities using a novel ResNet50 modified Faster Regions with Convolutional Neural Network(R-CNN) model. The classification task is performed into multiple classes which are hemorrhage, hydrocephalus, and normal. The proposed model both determines the borders of the normal/abnormal parts and classifies them with the highest accuracy.

METHODS

To provide a comprehensive performance analysis in the classification problem, Machine Learning(ML) and Deep Learning(DL) techniques were discussed. Artificial Neural Network(ANN), AdaBoost(AB), Decision Tree(DT), Logistic Regression(LR), Naive Bayes(NB), Random Forest(RF), and Support Vector Machine(SVM) were used as ML models. Besides, various Convolutional Neural Network(CNN) models and proposed ResNet50 modified Faster R-CNN model were used as DL models. Methods were validated using a novel brain dataset that contains both normal and abnormal images.

RESULTS

Based on results, LR obtained the highest result among ML methods and DenseNet201 obtained the highest results among CNN models with the accuracy of 84.80% and 85.68% for the classification task, respectively. Besides, the accuracy obtained by the proposed model is 99.75%.

CONCLUSIONS

Experimental results demonstrate that the proposed model has yielded better performance for detection and classification tasks. This artificial intelligence(AI) framework can be utilized as a computer-aided medical decision support system for medical experts.

Fully-channel regional attention network for disease-location recognition with tongue images

OBJECTIVE

Using the deep learning model to realize tongue image-based disease location recognition and focus on solving two problems: 1. The ability of the general convolution network to model detailed regional tongue features is weak; 2. Ignoring the group relationship between convolution channels, which caused the high redundancy of the model.

METHODS

To enhance the convolutional neural networks. In this paper, a stochastic region pooling method is proposed to gain detailed regional features. Also, an inner-imaging channel relationship modeling method is proposed to model multi-region relations on all channels. Moreover, we combine it with the spatial attention mechanism.

RESULTS

The tongue image dataset with the clinical disease-location label is established. Abundant experiments are carried out on it. The experimental results show that the proposed method can effectively model the regional details of tongue image and improve the performance of disease location recognition.

CONCLUSION

In this paper, we construct the tongue image dataset with disease-location labels to mine the relationship between tongue images and disease locations. A novel fully-channel regional attention network is proposed to model the local detail tongue features and improve the modeling efficiency.

SIGNIFICANCE

The applications of deep learning in tongue image disease-location recognition and the proposed innovative models have guiding significance for other assistant diagnostic tasks. The proposed model provides an example of efficient modeling of detailed tongue features, which is of great guiding significance for other auxiliary diagnosis applications.

Application of Artificial Intelligence in the MRI Classification Task of Human Brain Neurological and Psychiatric Diseases: A Scoping Review

Artificial intelligence (AI) for medical imaging is a technology with great potential. An in-depth understanding of the principles and applications of magnetic resonance imaging (MRI), machine learning (ML), and deep learning (DL) is fundamental for developing AI-based algorithms that can meet the requirements of clinical diagnosis and have excellent quality and efficiency. Moreover, a more comprehensive understanding of applications and opportunities would help to implement AI-based methods in an ethical and sustainable manner. This review first summarizes recent research advances in ML and DL techniques for classifying human brain magnetic resonance images. Then, the application of ML and DL methods to six typical neurological and psychiatric diseases is summarized, including Alzheimer's disease (AD), Parkinson's disease (PD), major depressive disorder (MDD), schizophrenia (SCZ), attention-deficit/hyperactivity disorder (ADHD), and autism spectrum disorder (ASD). Finally, the limitations of the existing research are discussed, and possible future research directions are proposed.

Three-Dimensional Convolutional Autoencoder Extracts Features of Structural Brain Images With a "Diagnostic Label-Free" Approach: Application to Schizophrenia Datasets

There has been increasing interest in performing psychiatric brain imaging studies using deep learning. However, most studies in this field disregard three-dimensional (3D) spatial information and targeted disease discrimination, without considering the genetic and clinical heterogeneity of psychiatric disorders. The purpose of this study was to investigate the efficacy of a 3D convolutional autoencoder (3D-CAE) for extracting features related to psychiatric disorders without diagnostic labels. The network was trained using a Kyoto University dataset including 82 patients with schizophrenia (SZ) and 90 healthy subjects (HS) and was evaluated using Center for Biomedical Research Excellence (COBRE) datasets, including 71 SZ patients and 71 HS. We created 16 3D-CAE models with different channels and convolutions to explore the effective range of hyperparameters for psychiatric brain imaging. The number of blocks containing two convolutional layers and one pooling layer was set, ranging from 1 block to 4 blocks. The number of channels in the extraction layer varied from 1, 4, 16, and 32 channels. The proposed 3D-CAEs were successfully reproduced into 3D structural magnetic resonance imaging (MRI) scans with sufficiently low errors. In addition, the features extracted using 3D-CAE retained the relation to clinical information. We explored the appropriate hyperparameter range of 3D-CAE, and it was suggested that a model with 3 blocks may be related to extracting features for predicting the dose of medication and symptom severity in schizophrenia.

Large-Scale Brain Functional Network Integration for Discrimination of Autism Using a 3-D Deep Learning Model

GOAL

Brain functional networks (BFNs) constructed using resting-state functional magnetic resonance imaging (fMRI) have proven to be an effective way to understand aberrant functional connectivity in autism spectrum disorder (ASD) patients. It is still challenging to utilize these features as potential biomarkers for discrimination of ASD. The purpose of this work is to classify ASD and normal controls (NCs) using BFNs derived from rs-fMRI.

METHODS

A deep learning framework was proposed that integrated convolutional neural network (CNN) and channel-wise attention mechanism to model both intra- and inter-BFN associations simultaneously for ASD diagnosis. We investigate the effects of each BFN on performance and performed inter-network connectivity analysis between each pair of BFNs. We compared the performance of our CNN model with some state-of-the-art algorithms using functional connectivity features.

RESULTS

We collected 79 ASD patients and 105 NCs from the ABIDE-I dataset. The mean accuracy of our classification algorithm was 77.74% for classification of ASD versus NCs.

CONCLUSION

The proposed model is able to integrate information from multiple BFNs to improve detection accuracy of ASD.

SIGNIFICANCE

These findings suggest that large-scale BFNs is promising to serve as reliable biomarkers for diagnosis of ASD.

Statistical and Machine Learning Link Selection Methods for Brain Functional Networks: Review and Comparison

Network-based representations have introduced a revolution in neuroscience, expanding the understanding of the brain from the activity of individual regions to the interactions between them. This augmented network view comes at the cost of high dimensionality, which hinders both our capacity of deciphering the main mechanisms behind pathologies, and the significance of any statistical and/or machine learning task used in processing this data. A link selection method, allowing to remove irrelevant connections in a given scenario, is an obvious solution that provides improved utilization of these network representations. In this contribution we review a large set of statistical and machine learning link selection methods and evaluate them on real brain functional networks. Results indicate that most methods perform in a qualitatively similar way, with NBS (Network Based Statistics) winning in terms of quantity of retained information, AnovaNet in terms of stability and ExT (Extra Trees) in terms of lower computational cost. While machine learning methods are conceptually more complex than statistical ones, they do not yield a clear advantage. At the same time, the high heterogeneity in the set of links retained by each method suggests that they are offering complementary views to the data. The implications of these results in neuroscience tasks are finally discussed.

Multi-modal neuroimaging feature fusion via 3D Convolutional Neural Network architecture for schizophrenia diagnosis

Early and precise diagnosis of schizophrenia disorder (SZ) has an essential role in the quality of a patient’s life and future treatments. Structural and functional neuroimaging provides robust biomarkers for understanding the anatomical and functional changes associated with SZ. Each of the neuroimaging techniques shows only a different perspective on the functional or structural of the brain, while multi-modal fusion can reveal latent connections in the brain. In this paper, we propose an approach for the fusion of structural and functional brain data with a deep learning-based model to take advantage of data fusion and increase the accuracy of schizophrenia disorder diagnosis. The proposed method consists of an architecture of 3D convolutional neural networks (CNNs) that applied to magnetic resonance imaging (MRI), functional magnetic resonance imaging (fMRI), and diffusion tensor imaging (DTI) extracted features. We use 3D MRI patches, fMRI spatial independent component analysis (ICA) map, and DTI fractional anisotropy (FA) as model inputs. Our method is validated on the COBRE dataset, and an average accuracy of 99.35% is obtained. The proposed method demonstrates promising classification performance and can be applied to real data.

Deep learning applications for the classification of psychiatric disorders using neuroimaging data: Systematic review and meta-analysis

Deep learning (DL) methods have been increasingly applied to neuroimaging data to identify patients with psychiatric and neurological disorders. This review provides an overview of the different DL applications within psychiatry and compares DL model accuracy to standard machine learning (SML). Fifty-three articles were included for qualitative analysis, primarily investigating autism spectrum disorder (ASD; n = 22), schizophrenia (SZ; n = 22) and attention-deficit/hyperactivity disorder (ADHD; n = 9). Thirty-two of the thirty-five studies that directly compared DL to SML reported a higher accuracy for DL. Only sixteen studies could be included in a meta-regression to quantitatively compare DL and SML performance. This showed a higher odds ratio for DL models, though the comparison attained significance only for ASD. Our results suggest that deep learning of neuroimaging data is a promising tool for the classification of individual psychiatric patients. However, it is not yet used to its full potential: most studies use pre-engineered features, whereas one of the main advantages of DL is its ability to learn representations of minimally processed data. Our current evaluation is limited by minimal reporting of performance measures to enable quantitative comparisons, and the restriction to ADHD, SZ and ASD as current research focusses on large publicly available datasets. To truly uncover the added value of DL, we need carefully designed comparisons of SML and DL models which are yet rarely performed.