A Mutual Multi-Scale Triplet Graph Convolutional Network for Classification of Brain Disorders Using Functional or Structural Connectivity

Using dynamic graph convolutional network to identify individuals with major depression disorder

Objective and quantitative neuroimaging biomarkers are crucial for early diagnosis of major depressive disorder (MDD). However, previous studies using machine learning (ML) to distinguish MDD have often used small sample sizes and overlooked MDD's neural connectome and mechanism. To address these gaps, we applied Dynamic Graph Convolutional Nets (DGCNs) to a large multi-site dataset of 2317 resting state functional MRI (RS-fMRI) scans from 1081 MDD patients and 1236 healthy controls from 16 Rest-meta-MDD consortium sites. Our DGCN model combined with the personal whole brain functional connectivity (FC) network achieved an accuracy of 82.5 % (95 % CI:81.6-83.4 %, AUC:0.869), outperforming other universal ML classifiers. The most prominent domains for classification were mainly in the default mode network, fronto-parietal and cingulo-opercular network. Our study supports the stability and efficacy of using DGCN to characterize MDD and demonstrates its potential in enhancing neurobiological comprehension of MDD by detecting clinically related disorders in FC network topologies.

Exploring Schizophrenia Classification Through Multimodal MRI and Deep Graph Neural Networks: Unveiling Brain Region-Specific Weight Discrepancies and Their Association With Cell-Type Specific Transcriptomic Features

BACKGROUND AND HYPOTHESIS

Schizophrenia (SZ) is a prevalent mental disorder that imposes significant health burdens. Diagnostic accuracy remains challenging due to clinical subjectivity. To address this issue, we explore magnetic resonance imaging (MRI) as a tool to enhance SZ diagnosis and provide objective references and biomarkers. Using deep learning with graph convolution, we represent MRI data as graphs, aligning with brain structure, and improving feature extraction, and classification. Integration of multiple modalities is expected to enhance classification.

STUDY DESIGN

Our study enrolled 683 SZ patients and 606 healthy controls from 7 hospitals, collecting structural MRI and functional MRI data. Both data types were represented as graphs, processed by 2 graph attention networks, and fused for classification. Grad-CAM with graph convolution ensured interpretability, and partial least squares analyzed gene expression in brain regions.

STUDY RESULTS

Our method excelled in the classification task, achieving 83.32% accuracy, 83.41% sensitivity, and 83.20% specificity in 10-fold cross-validation, surpassing traditional methods. And our multimodal approach outperformed unimodal methods. Grad-CAM identified potential brain biomarkers consistent with gene analysis and prior research.

CONCLUSIONS

Our study demonstrates the effectiveness of deep learning with graph attention networks, surpassing previous SZ diagnostic methods. Multimodal MRI's superiority over unimodal MRI confirms our initial hypothesis. Identifying potential brain biomarkers alongside gene biomarkers holds promise for advancing objective SZ diagnosis and research in SZ.

Jointly constrained group sparse connectivity representation improves early diagnosis of Alzheimer's disease on routinely acquired T1-weighted imaging-based brain network

Background

Radiomics-based morphological brain networks (radMBN) constructed from routinely acquired structural MRI (sMRI) data have gained attention in Alzheimer's disease (AD). However, the radMBN suffers from limited characterization of AD because sMRI only characterizes anatomical changes and is not a direct measure of neuronal pathology or brain activity.

Purpose

To establish a group sparse representation of the radMBN under a joint constraint of group-level white matter fiber connectivity and individual-level sMRI regional similarity (JCGS-radMBN).

Methods

Two publicly available datasets were adopted, including 120 subjects from ADNI with both T1-weighted image (T1WI) and diffusion MRI (dMRI) for JCGS-radMBN construction, 818 subjects from ADNI and 200 subjects solely with T1WI from AIBL for validation in early AD diagnosis. Specifically, the JCGS-radMBN was conducted by jointly estimating non-zero connections among subjects, with the regularization term constrained by group-level white matter fiber connectivity and individual-level sMRI regional similarity. Then, a triplet graph convolutional network was adopted for early AD diagnosis. The discriminative brain connections were identified using a two-sample t-test, and the neurobiological interpretation was validated by correlating the discriminative brain connections with cognitive scores.

Results

The JCGS-radMBN exhibited superior classification performance over five brain network construction methods. For the typical NC vs. AD classification, the JCGS-radMBN increased by 1-30% in accuracy over the alternatives on ADNI and AIBL. The discriminative brain connections exhibited a strong connectivity to hippocampus, parahippocampal gyrus, and basal ganglia, and had significant correlation with MMSE scores.

Conclusion

The proposed JCGS-radMBN facilitated the AD characterization of brain network established on routinely acquired imaging modality of sMRI.

Supplementary Information

The online version of this article (10.1007/s13755-023-00269-0) contains supplementary material, which is available to authorized users.

Riemannian manifold-based disentangled representation learning for multi-site functional connectivity analysis

Functional connectivity (FC), derived from resting-state functional magnetic resonance imaging (rs-fMRI), has been widely used to characterize brain abnormalities in disorders. FC is usually defined as a correlation matrix that is a symmetric positive definite (SPD) matrix lying on the Riemannian manifold. Recently, a number of learning-based methods have been proposed for FC analysis, while the geometric properties of Riemannian manifold have not yet been fully explored in previous studies. Also, most existing methods are designed to target one imaging site of fMRI data, which may result in limited training data for learning reliable and robust models. In this paper, we propose a novel Riemannian Manifold-based Disentangled Representation Learning (RM-DRL) framework which is capable of learning invariant representations from fMRI data across multiple sites for brain disorder diagnosis. In RM-DRL, we first employ an SPD-based encoder module to learn a latent unified representation of FC from different sites, which can preserve the Riemannian geometry of the SPD matrices. In latent space, a disentangled representation module is then designed to split the learned features into domain-specific and domain-invariant parts, respectively. Finally, a decoder module is introduced to ensure that sufficient information can be preserved during disentanglement learning. These designs allow us to introduce four types of training objectives to improve the disentanglement learning. Our RM-DRL method is evaluated on the public multi-site ABIDE dataset, showing superior performance compared with several state-of-the-art methods.

Combined graph convolutional networks with a multi-connection pattern to identify tremor-dominant Parkinson's disease and Essential tremor with resting tremor

Essential tremor with resting tremor (rET) and tremor-dominant Parkinson's disease (tPD) share many similar clinical symptoms, leading to frequent misdiagnoses. Functional connectivity (FC) matrix analysis derived from resting-state functional MRI (Rs-fMRI) offers a promising approach for early diagnosis and for exploring FC network pathogenesis in rET and tPD. However, methods relying solely on a single connection pattern may overlook the complementary roles of different connectivity patterns, resulting in reduced diagnostic differentiation. Therefore, we propose a multi-pattern connection Graph Convolutional Network (MCGCN) method to integrate information from various connection modes, distinguishing between rET and healthy controls (HC), tPD and HC, and rET and tPD. We constructed FC matrices using three different connectivity modes for each subject and used these as inputs to the MCGCN model for disease classification. The classification performance of the model was evaluated for each connectivity mode. Subsequently, gradient-weighted class activation mapping (Grad-CAM) was used to identify the most discriminative brain regions. The important brain regions identified were primarily distributed within cerebellar-motor and non-motor cortical networks. Compared with single-pattern GCN, our proposed MCGCN model demonstrated superior classification accuracy, underscoring the advantages of integrating multiple connectivity modes. Specifically, the model achieved an average accuracy of 88.0% for distinguishing rET from HC, 88.8% for rET from tPD, and 89.6% for tPD from HC. Our findings indicate that combining graph convolutional networks with multi-connection patterns can not only effectively discriminate between tPD, rET, and HC but also enhance our understanding of the functional network mechanisms underlying rET and tPD.

Hierarchical Graph Convolutional Network Built by Multiscale Atlases for Brain Disorder Diagnosis Using Functional Connectivity

Functional connectivity network (FCN) data from functional magnetic resonance imaging (fMRI) is increasingly used for the diagnosis of brain disorders. However, state-of-the-art studies used to build the FCN using a single brain parcellation atlas at a certain spatial scale, which largely neglected functional interactions across different spatial scales in hierarchical manners. In this study, we propose a novel framework to perform multiscale FCN analysis for brain disorder diagnosis. We first use a set of well-defined multiscale atlases to compute multiscale FCNs. Then, we utilize biologically meaningful brain hierarchical relationships among the regions in multiscale atlases to perform nodal pooling across multiple spatial scales, namely "Atlas-guided Pooling (AP)." Accordingly, we propose a multiscale-atlases-based hierarchical graph convolutional network (MAHGCN), built on the stacked layers of graph convolution and the AP, for a comprehensive extraction of diagnostic information from multiscale FCNs. Experiments on neuroimaging data from 1792 subjects demonstrate the effectiveness of our proposed method in the diagnoses of Alzheimer's disease (AD), the prodromal stage of AD [i.e., mild cognitive impairment (MCI)], as well as autism spectrum disorder (ASD), with the accuracy of 88.9%, 78.6%, and 72.7%, respectively. All results show significant advantages of our proposed method over other competing methods. This study not only demonstrates the feasibility of brain disorder diagnosis using resting-state fMRI empowered by deep learning but also highlights that the functional interactions in the multiscale brain hierarchy are worth being explored and integrated into deep learning network architectures for a better understanding of the neuropathology of brain disorders. The codes for MAHGCN are publicly available at "https://github.com/MianxinLiu/MAHGCN-code."

CE-GAN: Community Evolutionary Generative Adversarial Network for Alzheimer's Disease Risk Prediction

In the studies of neurodegenerative diseases such as Alzheimer's Disease (AD), researchers often focus on the associations among multi-omics pathogeny based on imaging genetics data. However, current studies overlook the communities in brain networks, leading to inaccurate models of disease development. This paper explores the developmental patterns of AD from the perspective of community evolution. We first establish a mathematical model to describe functional degeneration in the brain as the community evolution driven by entropy information propagation. Next, we propose an interpretable Community Evolutionary Generative Adversarial Network (CE-GAN) to predict disease risk. In the generator of CE-GAN, community evolutionary convolutions are designed to capture the evolutionary patterns of AD. The experiments are conducted using functional magnetic resonance imaging (fMRI) data and single nucleotide polymorphism (SNP) data. CE-GAN achieves 91.67% accuracy and 91.83% area under curve (AUC) in AD risk prediction tasks, surpassing advanced methods on the same dataset. In addition, we validated the effectiveness of CE-GAN for pathogeny extraction. The source code of this work is available at https://github.com/fmri123456/CE-GAN.

s2MRI-ADNet: an interpretable deep learning framework integrating Euclidean-graph representations of Alzheimer's disease solely from structural MRI

OBJECTIVE

To establish a multi-dimensional representation solely on structural MRI (sMRI) for early diagnosis of AD.

METHODS

A total of 3377 participants' sMRI from four independent databases were retrospectively identified to construct an interpretable deep learning model that integrated multi-dimensional representations of AD solely on sMRI (called s2MRI-ADNet) by a dual-channel learning strategy of gray matter volume (GMV) from Euclidean space and the regional radiomics similarity network (R2SN) from graph space. Specifically, the GMV feature map learning channel (called GMV-Channel) was to take into consideration spatial information of both long-range spatial relations and detailed localization information, while the node feature and connectivity strength learning channel (called NFCS-Channel) was to characterize the graph-structured R2SN network by a separable learning strategy.

RESULTS

The s2MRI-ADNet achieved a superior classification accuracy of 92.1% and 91.4% under intra-database and inter-database cross-validation. The GMV-Channel and NFCS-Channel captured complementary group-discriminative brain regions, revealing a complementary interpretation of the multi-dimensional representation of brain structure in Euclidean and graph spaces respectively. Besides, the generalizable and reproducible interpretation of the multi-dimensional representation in capturing complementary group-discriminative brain regions revealed a significant correlation between the four independent databases (p < 0.05). Significant associations (p < 0.05) between attention scores and brain abnormality, between classification scores and clinical measure of cognitive ability, CSF biomarker, metabolism, and genetic risk score also provided solid neurobiological interpretation.

CONCLUSION

The s2MRI-ADNet solely on sMRI could leverage the complementary multi-dimensional representations of AD in Euclidean and graph spaces, and achieved superior performance in the early diagnosis of AD, facilitating its potential in both clinical translation and popularization.

H-Net: Heterogeneous Neural Network for Multi-Classification of Neuropsychiatric Disorders

Clinical studies have proved that both structural magnetic resonance imaging (sMRI) and functional magnetic resonance imaging (fMRI) are implicitly associated with neuropsychiatric disorders (NDs), and integrating multi-modal to the binary classification of NDs has been thoroughly explored. However, accurately classifying multiple classes of NDs remains a challenge due to the complexity of disease subclass. In our study, we develop a heterogeneous neural network (H-Net) that integrates sMRI and fMRI modes for classifying multi-class NDs. To account for the differences between the two modes, H-Net adopts a heterogeneous neural network strategy to extract information from each mode. Specifically, H-Net includes an multi-layer perceptron based (MLP-based) encoder, a graph attention network based (GAT-based) encoder, and a cross-modality transformer block. The MLP-based and GAT-based encoders extract semantic features from sMRI and features from fMRI, respectively, while the cross-modality transformer block models the attention of two types of features. In H-Net, the proposed MLP-mixer block and cross-modality alignment are powerful tools for improving the multi-classification performance of NDs. H-Net is validate on the public dataset (CNP), where H-Net achieves 90% classification accuracy in diagnosing multi-class NDs. Furthermore, we demonstrate the complementarity of the two MRI modalities in improving the identification of multi-class NDs. Both visual and statistical analyses show the differences between ND subclasses.

MCAN: Multimodal Causal Adversarial Networks for Dynamic Effective Connectivity Learning From fMRI and EEG Data

Dynamic effective connectivity (DEC) is the accumulation of effective connectivity in the time dimension, which can describe the continuous neural activities in the brain. Recently, learning DEC from functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) data has attracted the attention of neuroinformatics researchers. However, the current methods fail to consider the gap between the fMRI and EEG modality, which can not precisely learn the DEC network from multimodal data. In this paper, we propose a multimodal causal adversarial network for DEC learning, named MCAN. The MCAN contains two modules: multimodal causal generator and multimodal causal discriminator. First, MCAN employs a multimodal causal generator with an attention-guided layer to produce a posterior signal and output a set of DEC networks. Then, the proposed method uses a multimodal causal discriminator to unsupervised calculate the joint gradient, which directs the update of the whole network. The experimental results on simulated data sets show that MCAN is superior to other state-of-the-art methods in learning the network structure of DEC and can effectively estimate the brain states. The experimental results on real data sets show that MCAN can better reveal abnormal patterns of brain activity and has good application potential in brain network analysis.

Enhancing Major Depressive Disorder Diagnosis With Dynamic-Static Fusion Graph Neural Networks

Major Depressive Disorder (MDD) is a debilitating, complex mental condition with unclear mechanisms hindering diagnostic progress. Research links MDD to abnormal brain connectivity using functional magnetic resonance imaging (fMRI). Yet, existing fMRI-based MDD models suffer from limitations, including neglecting dynamic network traits, lacking interpretability, and struggling with small datasets. We present DSFGNN, a novel graph neural network framework addressing these issues for improved MDD diagnosis. DSFGNN employs a graph isomorphism encoder to model static and dynamic brain networks, achieving effective fusion of temporal and spatial information through a spatiotemporal attention mechanism, thereby enhancing interpretability. Furthermore, we incorporate a causal disentangling module and orthogonal regularization module to augment the model's expressiveness. We evaluate DSFGNN on the Rest-meta-MDD dataset, yielding superior results compared to the best baseline. Besides, extensive ablation studies and interpretability analysis confirm DSFGNN's effectiveness and potential for biomarker discovery.

A novel interactive deep cascade spectral graph convolutional network with multi-relational graphs for disease prediction

Graph neural networks (GNNs) have recently grown in popularity for disease prediction. Existing GNN-based methods primarily build the graph topological structure around a single modality and combine it with other modalities to acquire feature representations of acquisitions. The complicated relationship in each modality, however, may not be well highlighted due to its specificity. Further, relatively shallow networks restrict adequate extraction of high-level features, affecting disease prediction performance. Accordingly, this paper develops a new interactive deep cascade spectral graph convolutional network with multi-relational graphs (IDCGN) for disease prediction tasks. Its crucial points lie in constructing multiple relational graphs and dual cascade spectral graph convolution branches with interaction (DCSGBI). Specifically, the former designs a pairwise imaging-based edge generator and a pairwise non-imaging-based edge generator from different modalities by devising two learnable networks, which adaptively capture graph structures and provide various views of the same acquisition to aid in disease diagnosis. Again, DCSGBI is established to enrich high-level semantic information and low-level details of disease data. It devises a cascade spectral graph convolution operator for each branch and incorporates the interaction strategy between different branches into the network, successfully forming a deep model and capturing complementary information from diverse branches. In this manner, more favorable and sufficient features are learned for a reliable diagnosis. Experiments on several disease datasets reveal that IDCGN exceeds state-of-the-art models and achieves promising results.

Randomizing Human Brain Function Representation for Brain Disease Diagnosis

Resting-state fMRI (rs-fMRI) is an effective tool for quantifying functional connectivity (FC), which plays a crucial role in exploring various brain diseases. Due to the high dimensionality of fMRI data, FC is typically computed based on the region of interest (ROI), whose parcellation relies on a pre-defined atlas. However, utilizing the brain atlas poses several challenges including 1) subjective selection bias in choosing from various brain atlases, 2) parcellation of each subject's brain with the same atlas yet disregarding individual specificity; 3) lack of interaction between brain region parcellation and downstream ROI-based FC analysis. To address these limitations, we propose a novel randomizing strategy for generating brain function representation to facilitate neural disease diagnosis. Specifically, we randomly sample brain patches, thus avoiding ROI parcellations of the brain atlas. Then, we introduce a new brain function representation framework for the sampled patches. Each patch has its function description by referring to anchor patches, as well as the position description. Furthermore, we design an adaptive-selection-assisted Transformer network to optimize and integrate the function representations of all sampled patches within each brain for neural disease diagnosis. To validate our framework, we conduct extensive evaluations on three datasets, and the experimental results establish the effectiveness and generality of our proposed method, offering a promising avenue for advancing neural disease diagnosis beyond the confines of traditional atlas-based methods. Our code is available at https://github.com/mjliu2020/RandomFR.

Gradient Matching Federated Domain Adaptation for Brain Image Classification

Federated learning has shown its unique advantages in many different tasks, including brain image analysis. It provides a new way to train deep learning models while protecting the privacy of medical image data from multiple sites. However, previous studies suggest that domain shift across different sites may influence the performance of federated models. As a solution, we propose a gradient matching federated domain adaptation (GM-FedDA) method for brain image classification, aiming to reduce domain discrepancy with the assistance of a public image dataset and train robust local federated models for target sites. It mainly includes two stages: 1) pretraining stage; we propose a one-common-source adversarial domain adaptation (OCS-ADA) strategy, i.e., adopting ADA with gradient matching loss to pretrain encoders for reducing domain shift at each target site (private data) with the assistance of a common source domain (public data) and 2) fine-tuning stage; we develop a gradient matching federated (GM-Fed) fine-tuning method for updating local federated models pretrained with the OCS-ADA strategy, i.e., pushing the optimization direction of a local federated model toward its specific local minimum by minimizing gradient matching loss between sites. Using fully connected networks as local models, we validate our method with the diagnostic classification tasks of schizophrenia and major depressive disorder based on multisite resting-state functional MRI (fMRI), respectively. Results show that the proposed GM-FedDA method outperforms other commonly used methods, suggesting the potential of our method in brain imaging analysis and other fields, which need to utilize multisite data while preserving data privacy.

Adversarial Learning Based Node-Edge Graph Attention Networks for Autism Spectrum Disorder Identification

Graph neural networks (GNNs) have received increasing interest in the medical imaging field given their powerful graph embedding ability to characterize the non-Euclidean structure of brain networks based on magnetic resonance imaging (MRI) data. However, previous studies are largely node-centralized and ignore edge features for graph classification tasks, resulting in moderate performance of graph classification accuracy. Moreover, the generalizability of GNN model is still far from satisfactory in brain disorder [e.g., autism spectrum disorder (ASD)] identification due to considerable individual differences in symptoms among patients as well as data heterogeneity among different sites. In order to address the above limitations, this study proposes a novel adversarial learning-based node-edge graph attention network (AL-NEGAT) for ASD identification based on multimodal MRI data. First, both node and edge features are modeled based on structural and functional MRI data to leverage complementary brain information and preserved in the constructed weighted adjacent matrix for individuals through the attention mechanism in the proposed NEGAT. Second, two AL methods are employed to improve the generalizability of NEGAT. Finally, a gradient-based saliency map strategy is utilized for model interpretation to identify important brain regions and connections contributing to the classification. Experimental results based on the public Autism Brain Imaging Data Exchange I (ABIDE I) data demonstrate that the proposed framework achieves a classification accuracy of 74.7% between ASD and typical developing (TD) groups based on 1007 subjects across 17 different sites and outperforms the state-of-the-art methods, indicating satisfying classification ability and generalizability of the proposed AL-NEGAT model. Our work provides a powerful tool for brain disorder identification.

Adaptive spatial-temporal neural network for ADHD identification using functional fMRI

Computer aided diagnosis methods play an important role in Attention Deficit Hyperactivity Disorder (ADHD) identification. Dynamic functional connectivity (dFC) analysis has been widely used for ADHD diagnosis based on resting-state functional magnetic resonance imaging (rs-fMRI), which can help capture abnormalities of brain activity. However, most existing dFC-based methods only focus on dependencies between two adjacent timestamps, ignoring global dynamic evolution patterns. Furthermore, the majority of these methods fail to adaptively learn dFCs. In this paper, we propose an adaptive spatial-temporal neural network (ASTNet) comprising three modules for ADHD identification based on rs-fMRI time series. Specifically, we first partition rs-fMRI time series into multiple segments using non-overlapping sliding windows. Then, adaptive functional connectivity generation (AFCG) is used to model spatial relationships among regions-of-interest (ROIs) with adaptive dFCs as input. Finally, we employ a temporal dependency mining (TDM) module which combines local and global branches to capture global temporal dependencies from the spatially-dependent pattern sequences. Experimental results on the ADHD-200 dataset demonstrate the superiority of the proposed ASTNet over competing approaches in automated ADHD classification.

Triplet-constrained deep hashing for chest X-ray image retrieval in COVID-19 assessment

Radiology images of the chest, such as computer tomography scans and X-rays, have been prominently used in computer-aided COVID-19 analysis. Learning-based radiology image retrieval has attracted increasing attention recently, which generally involves image feature extraction and finding matches in extensive image databases based on query images. Many deep hashing methods have been developed for chest radiology image search due to the high efficiency of retrieval using hash codes. However, they often overlook the complex triple associations between images; that is, images belonging to the same category tend to share similar characteristics and vice versa. To this end, we develop a triplet-constrained deep hashing (TCDH) framework for chest radiology image retrieval to facilitate automated analysis of COVID-19. The TCDH consists of two phases, including (a) feature extraction and (b) image retrieval. For feature extraction, we have introduced a triplet constraint and an image reconstruction task to enhance discriminative ability of learned features, and these features are then converted into binary hash codes to capture semantic information. Specifically, the triplet constraint is designed to pull closer samples within the same category and push apart samples from different categories. Additionally, an auxiliary image reconstruction task is employed during feature extraction to help effectively capture anatomical structures of images. For image retrieval, we utilize learned hash codes to conduct searches for medical images. Extensive experiments on 30,386 chest X-ray images demonstrate the superiority of the proposed method over several state-of-the-art approaches in automated image search. The code is now available online.

Constructing hierarchical attentive functional brain networks for early AD diagnosis

Analyzing functional brain networks (FBN) with deep learning has demonstrated great potential for brain disorder diagnosis. The conventional construction of FBN is typically conducted at a single scale with a predefined brain region atlas. However, numerous studies have identified that the structure and function of the brain are hierarchically organized in nature. This urges the need of representing FBN in a hierarchical manner for more effective analysis of the complementary diagnostic insights at different scales. To this end, this paper proposes to build hierarchical FBNs adaptively within the Transformer framework. Specifically, a sparse attention-based node-merging module is designed to work alongside the conventional network feature extraction modules in each layer. The proposed module generates coarser nodes for further FBN construction and analysis by combining fine-grained nodes. By stacking multiple such layers, a hierarchical representation of FBN can be adaptively learned in an end-to-end manner. The hierarchical structure can not only integrate the complementary information from multiscale FBN for joint analysis, but also reduce the model complexity due to decreasing node sizes. Moreover, this paper argues that the nodes defined by the existing atlases are not necessarily the optimal starting level to build FBN hierarchy and exploring finer nodes may further enrich the FBN representation. In this regard, each predefined node in an atlas is split into multiple sub-nodes, overcoming the scale limitation of the existing atlases. Extensive experiments conducted on various data sets consistently demonstrate the superior performance of the proposed method over the competing methods.

Spectral Graph Neural Network-Based Multi-Atlas Brain Network Fusion for Major Depressive Disorder Diagnosis

Major Depressive Disorder (MDD) imposes a substantial burden within the healthcare domain, impacting millions of individuals worldwide. Functional Magnetic Resonance Imaging (fMRI) has emerged as a promising tool for the objective diagnosis of MDD, enabling the investigation of functional connectivity patterns in the brain associated with this disorder. However, most existing methods focus on a single brain atlas, which limits their ability to capture the complex, multi-scale nature of functional brain networks. To address these limitations, we propose a novel multi-atlas fusion method that incorporates early and late fusion in a unified framework. Our method introduces the concept of the holistic Functional Connectivity Network (FCN), which captures both intra-atlas relationships within individual atlases and inter-regional relationships between atlases with different brain parcellation scales. This comprehensive representation enables the identification of potential disease-related patterns associated with MDD in the early stage of our framework. Moreover, by decoding the holistic FCN from various perspectives through multiple spectral Graph Convolutional Neural Networks and fusing their results with decision-level ensembles, we further improve the performance of MDD diagnosis. Our approach is easily implemented with minimal modifications to existing model structures and demonstrates a robust performance across different baseline models. Our method, evaluated on public resting-state fMRI datasets, surpasses the current multi-atlas fusion methods, enhancing the accuracy of MDD diagnosis. The proposed novel multi-atlas fusion framework provides a more reliable MDD diagnostic technique. Experimental results show our approach outperforms both single- and multi-atlas-based methods, demonstrating its effectiveness in advancing MDD diagnosis.

The Use of Generative Adversarial Network and Graph Convolution Network for Neuroimaging-Based Diagnostic Classification

Functional connectivity (FC) obtained from resting-state functional magnetic resonance imaging has been integrated with machine learning algorithms to deliver consistent and reliable brain disease classification outcomes. However, in classical learning procedures, custom-built specialized feature selection techniques are typically used to filter out uninformative features from FC patterns to generalize efficiently on the datasets. The ability of convolutional neural networks (CNN) and other deep learning models to extract informative features from data with grid structure (such as images) has led to the surge in popularity of these techniques. However, the designs of many existing CNN models still fail to exploit the relationships between entities of graph-structure data (such as networks). Therefore, graph convolution network (GCN) has been suggested as a means for uncovering the intricate structure of brain network data, which has the potential to substantially improve classification accuracy. Furthermore, overfitting in classifiers can be largely attributed to the limited number of available training samples. Recently, the generative adversarial network (GAN) has been widely used in the medical field for its generative aspect that can generate synthesis images to cope with the problems of data scarcity and patient privacy. In our previous work, GCN and GAN have been designed to investigate FC patterns to perform diagnosis tasks, and their effectiveness has been tested on the ABIDE-I dataset. In this paper, the models will be further applied to FC data derived from more public datasets (ADHD, ABIDE-II, and ADNI) and our in-house dataset (PTSD) to justify their generalization on all types of data. The results of a number of experiments show the powerful characteristic of GAN to mimic FC data to achieve high performance in disease prediction. When employing GAN for data augmentation, the diagnostic accuracy across ADHD-200, ABIDE-II, and ADNI datasets surpasses that of other machine learning models, including results achieved with BrainNetCNN. Specifically, in ADHD, the accuracy increased from 67.74% to 73.96% with GAN, in ABIDE-II from 70.36% to 77.40%, and in ADNI, reaching 52.84% and 88.56% for multiclass and binary classification, respectively. GCN also obtains decent results, with the best accuracy in ADHD datasets at 71.38% for multinomial and 75% for binary classification, respectively, and the second-best accuracy in the ABIDE-II dataset (72.28% and 75.16%, respectively). Both GAN and GCN achieved the highest accuracy for the PTSD dataset, reaching 97.76%. However, there are still some limitations that can be improved. Both methods have many opportunities for the prediction and diagnosis of diseases.

Ordinal Pattern Tree: A New Representation Method for Brain Network Analysis

Brain networks, describing the functional or structural interactions of brain with graph theory, have been widely used for brain imaging analysis. Currently, several network representation methods have been developed for describing and analyzing brain networks. However, most of these methods ignored the valuable weighted information of the edges in brain networks. In this paper, we propose a new representation method (i.e., ordinal pattern tree) for brain network analysis. Compared with the existing network representation methods, the proposed ordinal pattern tree (OPT) can not only leverage the weighted information of the edges but also express the hierarchical relationships of nodes in brain networks. On OPT, nodes are connected by ordinal edges which are constructed by using the ordinal pattern relationships of weighted edges. We represent brain networks as OPTs and further develop a new graph kernel called optimal transport (OT) based ordinal pattern tree (OT-OPT) kernel to measure the similarity between paired brain networks. In OT-OPT kernel, the OT distances are used to calculate the transport costs between the nodes on the OPTs. Based on these OT distances, we use exponential function to calculate OT-OPT kernel which is proved to be positive definite. To evaluate the effectiveness of the proposed method, we perform classification and regression experiments on ADHD-200, ABIDE and ADNI datasets. The experimental results demonstrate that our proposed method outperforms the state-of-the-art graph methods in the classification and regression tasks.

A novel spatiotemporal graph convolutional network framework for functional connectivity biomarkers identification of Alzheimer's disease

BACKGROUND

Functional connectivity (FC) biomarkers play a crucial role in the early diagnosis and mechanistic study of Alzheimer's disease (AD). However, the identification of effective FC biomarkers remains challenging. In this study, we introduce a novel approach, the spatiotemporal graph convolutional network (ST-GCN) combined with the gradient-based class activation mapping (Grad-CAM) model (STGC-GCAM), to effectively identify FC biomarkers for AD.

METHODS

This multi-center cross-racial retrospective study involved 2,272 participants, including 1,105 cognitively normal (CN) subjects, 790 mild cognitive impairment (MCI) individuals, and 377 AD patients. All participants underwent functional magnetic resonance imaging (fMRI) and T1-weighted MRI scans. In this study, firstly, we optimized the STGC-GCAM model to enhance classification accuracy. Secondly, we identified novel AD-associated biomarkers using the optimized model. Thirdly, we validated the imaging biomarkers using Kaplan-Meier analysis. Lastly, we performed correlation analysis and causal mediation analysis to confirm the physiological significance of the identified biomarkers.

RESULTS

The STGC-GCAM model demonstrated great classification performance (The average area under the curve (AUC) values for different categories were: CN vs MCI = 0.98, CN vs AD = 0.95, MCI vs AD = 0.96, stable MCI vs progressive MCI = 0.79). Notably, the model identified specific brain regions, including the sensorimotor network (SMN), visual network (VN), and default mode network (DMN), as key differentiators between patients and CN individuals. These brain regions exhibited significant associations with the severity of cognitive impairment (p < 0.05). Moreover, the topological features of important brain regions demonstrated excellent predictive capability for the conversion from MCI to AD (Hazard ratio = 3.885, p < 0.001). Additionally, our findings revealed that the topological features of these brain regions mediated the impact of amyloid beta (Aβ) deposition (bootstrapped average causal mediation effect: β = -0.01 [-0.025, 0.00], p < 0.001) and brain glucose metabolism (bootstrapped average causal mediation effect: β = -0.02 [-0.04, -0.001], p < 0.001) on cognitive status.

CONCLUSIONS

This study presents the STGC-GCAM framework, which identifies FC biomarkers using a large multi-site fMRI dataset.

Few-Shot Domain-Adaptive Anomaly Detection for Cross-Site Brain Images

Early screening is essential for effective intervention and treatment of individuals with mental disorders. Functional magnetic resonance imaging (fMRI) is a noninvasive tool for depicting neural activity and has demonstrated strong potential as a technique for identifying mental disorders. Due to the difficulty in data collection and diagnosis, imaging data from patients are rare at a single site, whereas abundant healthy control data are available from public datasets. However, joint use of these data from multiple sites for classification model training is hindered by cross-domain distribution discrepancy and diverse label spaces. Herein, we propose few-shot domain-adaptive anomaly detection (FAAD) to achieve cross-site anomaly detection of brain images based on only a few labeled samples. We introduce domain adaptation to mitigate cross-domain distribution discrepancy and jointly align the general and conditional feature distributions of imaging data across multiple sites. We utilize fMRI data of healthy subjects in the Human Connectome Project (HCP) as the source domain and fMRI images from six independent sites, including patients with mental disorders and demographically matched healthy controls, as target domains. Experiments showed the superiority of the proposed method compared with binary classification, traditional anomaly detection methods, and several recognized domain adaptation methods.

A Multi-Graph Cross-Attention-Based Region-Aware Feature Fusion Network Using Multi-Template for Brain Disorder Diagnosis

Functional connectivity (FC) networks based on resting-state functional magnetic imaging (rs-fMRI) are reliable and sensitive for brain disorder diagnosis. However, most existing methods are limited by using a single template, which may be insufficient to reveal complex brain connectivities. Furthermore, these methods usually neglect the complementary information between static and dynamic brain networks, and the functional divergence among different brain regions, leading to suboptimal diagnosis performance. To address these limitations, we propose a novel multi-graph cross-attention based region-aware feature fusion network (MGCA-RAFFNet) by using multi-template for brain disorder diagnosis. Specifically, we first employ multi-template to parcellate the brain space into different regions of interest (ROIs). Then, a multi-graph cross-attention network (MGCAN), including static and dynamic graph convolutions, is developed to explore the deep features contained in multi-template data, which can effectively analyze complex interaction patterns of brain networks for each template, and further adopt a dual-view cross-attention (DVCA) to acquire complementary information. Finally, to efficiently fuse multiple static-dynamic features, we design a region-aware feature fusion network (RAFFNet), which is beneficial to improve the feature discrimination by considering the underlying relations among static-dynamic features in different brain regions. Our proposed method is evaluated on both public ADNI-2 and ABIDE-I datasets for diagnosing mild cognitive impairment (MCI) and autism spectrum disorder (ASD). Extensive experiments demonstrate that the proposed method outperforms the state-of-the-art methods. Our source code is available at https://github.com/mylbuaa/MGCA-RAFFNet.

Deep Fusion of Multi-Template Using Spatio-Temporal Weighted Multi-Hypergraph Convolutional Networks for Brain Disease Analysis

Conventional functional connectivity network (FCN) based on resting-state fMRI (rs-fMRI) can only reflect the relationship between pairwise brain regions. Thus, the hyper-connectivity network (HCN) has been widely used to reveal high-order interactions among multiple brain regions. However, existing HCN models are essentially spatial HCN, which reflect the spatial relevance of multiple brain regions, but ignore the temporal correlation among multiple time points. Furthermore, the majority of HCN construction and learning frameworks are limited to using a single template, while the multi-template carries richer information. To address these issues, we first employ multiple templates to parcellate the rs-fMRI into different brain regions. Then, based on the multi-template data, we propose a spatio-temporal weighted HCN (STW-HCN) to capture more comprehensive high-order temporal and spatial properties of brain activity. Next, a novel deep fusion model of multi-template called spatio-temporal weighted multi-hypergraph convolutional network (STW-MHGCN) is proposed to fuse the STW-HCN of multiple templates, which extracts the deep interrelation information between different templates. Finally, we evaluate our method on the ADNI-2 and ABIDE-I datasets for mild cognitive impairment (MCI) and autism spectrum disorder (ASD) analysis. Experimental results demonstrate that the proposed method is superior to the state-of-the-art approaches in MCI and ASD classification, and the abnormal spatio-temporal hyper-edges discovered by our method have significant significance for the brain abnormalities analysis of MCI and ASD.

Adversarially Trained Persistent Homology Based Graph Convolutional Network for Disease Identification Using Brain Connectivity

Brain disease propagation is associated with characteristic alterations in the structural and functional connectivity networks of the brain. To identify disease-specific network representations, graph convolutional networks (GCNs) have been used because of their powerful graph embedding ability to characterize the non-Euclidean structure of brain networks. However, existing GCNs generally focus on learning the discriminative region of interest (ROI) features, often ignoring important topological information that enables the integration of connectome patterns of brain activity. In addition, most methods fail to consider the vulnerability of GCNs to perturbations in network properties of the brain, which considerably degrades the reliability of diagnosis results. In this study, we propose an adversarially trained persistent homology-based graph convolutional network (ATPGCN) to capture disease-specific brain connectome patterns and classify brain diseases. First, the brain functional/structural connectivity is constructed using different neuroimaging modalities. Then, we develop a novel strategy that concatenates the persistent homology features from a brain algebraic topology analysis with readout features of the global pooling layer of a GCN model to collaboratively learn the individual-level representation. Finally, we simulate the adversarial perturbations by targeting the risk ROIs from clinical prior, and incorporate them into a training loop to evaluate the robustness of the model. The experimental results on three independent datasets demonstrate that ATPGCN outperforms existing classification methods in disease identification and is robust to minor perturbations in network architecture. Our code is available at https://github.com/CYB08/ATPGCN.

A Causality-Aware Graph Convolutional Network Framework for Rigidity Assessment in Parkinsonians

Rigidity is one of the common motor disorders in Parkinson's disease (PD), which lead to life quality deterioration. The widely-used rating-scale-based approach for rigidity assessment still depends on the availability of experienced neurologists and is limited by rating subjectivity. Given the recent successful applications of quantitative susceptibility mapping (QSM) in auxiliary PD diagnosis, automated assessment of PD rigidity can be essentially achieved through QSM analysis. However, a major challenge is the performance instability due to the confounding factors (e.g., noise and distribution shift) which conceal the truly-causal features. Therefore, we propose a causality-aware graph convolutional network (GCN) framework, where causal feature selection is combined with causal invariance to ensure that causality-informed model decisions are reached. Firstly, a GCN model that integrates causal feature selection is systematically constructed at three graph levels: node, structure, and representation. In this model, a causal diagram is learned to extract a subgraph with truly-causal information. Secondly, a non-causal perturbation strategy is developed along with an invariance constraint to ensure the stability of the assessment results under different distributions, and thus avoid spurious correlations caused by distribution shifts. The superiority of the proposed method is shown by extensive experiments and the clinical value is revealed by the direct relevance of selected brain regions to rigidity in PD. Besides, its extensibility is verified on other two tasks: PD bradykinesia and mental state for Alzheimer's disease. Overall, we provide a clinically-potential tool for automated and stable assessment of PD rigidity. Our source code will be available at https://github.com/SJTUBME-QianLab/Causality-Aware-Rigidity.

Preserving specificity in federated graph learning for fMRI-based neurological disorder identification

Resting-state functional magnetic resonance imaging (rs-fMRI) offers a non-invasive approach to examining abnormal brain connectivity associated with brain disorders. Graph neural network (GNN) gains popularity in fMRI representation learning and brain disorder analysis with powerful graph representation capabilities. Training a general GNN often necessitates a large-scale dataset from multiple imaging centers/sites, but centralizing multi-site data generally faces inherent challenges related to data privacy, security, and storage burden. Federated Learning (FL) enables collaborative model training without centralized multi-site fMRI data. Unfortunately, previous FL approaches for fMRI analysis often ignore site-specificity, including demographic factors such as age, gender, and education level. To this end, we propose a specificity-aware federated graph learning (SFGL) framework for rs-fMRI analysis and automated brain disorder identification, with a server and multiple clients/sites for federated model aggregation and prediction. At each client, our model consists of a shared and a personalized branch, where parameters of the shared branch are sent to the server while those of the personalized branch remain local. This can facilitate knowledge sharing among sites and also helps preserve site specificity. In the shared branch, we employ a spatio-temporal attention graph isomorphism network to learn dynamic fMRI representations. In the personalized branch, we integrate vectorized demographic information (i.e., age, gender, and education years) and functional connectivity networks to preserve site-specific characteristics. Representations generated by the two branches are then fused for classification. Experimental results on two fMRI datasets with a total of 1218 subjects suggest that SFGL outperforms several state-of-the-art approaches.

A multi-view convolutional neural network method combining attention mechanism for diagnosing autism spectrum disorder

Autism Spectrum Disorder (ASD) is a neurodevelopmental condition whose current psychiatric diagnostic process is subjective and behavior-based. In contrast, functional magnetic resonance imaging (fMRI) can objectively measure brain activity and is useful for identifying brain disorders. However, the ASD diagnostic models employed to date have not reached satisfactory levels of accuracy. This study proposes the use of MAACNN, a method that utilizes multi-view convolutional neural networks (CNNs) in conjunction with attention mechanisms for identifying ASD in multi-scale fMRI. The proposed algorithm effectively combines unsupervised and supervised learning. In the initial stage, we employ stacked denoising autoencoders, an unsupervised learning method for feature extraction, which provides different nodes to adapt to multi-scale data. In the subsequent stage, we perform supervised learning by employing multi-view CNNs for classification and obtain the final results. Finally, multi-scale data fusion is achieved by using the attention fusion mechanism. The ABIDE dataset is used to evaluate the model we proposed., and the experimental results show that MAACNN achieves superior performance with 75.12% accuracy and 0.79 AUC on ABIDE-I, and 72.88% accuracy and 0.76 AUC on ABIDE-II. The proposed method significantly contributes to the clinical diagnosis of ASD.

Data-driven multimodal fusion: approaches and applications in psychiatric research

In the era of big data, where vast amounts of information are being generated and collected at an unprecedented rate, there is a pressing demand for innovative data-driven multi-modal fusion methods. These methods aim to integrate diverse neuroimaging perspectives to extract meaningful insights and attain a more comprehensive understanding of complex psychiatric disorders. However, analyzing each modality separately may only reveal partial insights or miss out on important correlations between different types of data. This is where data-driven multi-modal fusion techniques come into play. By combining information from multiple modalities in a synergistic manner, these methods enable us to uncover hidden patterns and relationships that would otherwise remain unnoticed. In this paper, we present an extensive overview of data-driven multimodal fusion approaches with or without prior information, with specific emphasis on canonical correlation analysis and independent component analysis. The applications of such fusion methods are wide-ranging and allow us to incorporate multiple factors such as genetics, environment, cognition, and treatment outcomes across various brain disorders. After summarizing the diverse neuropsychiatric magnetic resonance imaging fusion applications, we further discuss the emerging neuroimaging analyzing trends in big data, such as N-way multimodal fusion, deep learning approaches, and clinical translation. Overall, multimodal fusion emerges as an imperative approach providing valuable insights into the underlying neural basis of mental disorders, which can uncover subtle abnormalities or potential biomarkers that may benefit targeted treatments and personalized medical interventions.

The Combination of a Graph Neural Network Technique and Brain Imaging to Diagnose Neurological Disorders: A Review and Outlook

Neurological disorders (NDs), such as Alzheimer's disease, have been a threat to human health all over the world. It is of great importance to diagnose ND through combining artificial intelligence technology and brain imaging. A graph neural network (GNN) can model and analyze the brain, imaging from morphology, anatomical structure, function features, and other aspects, thus becoming one of the best deep learning models in the diagnosis of ND. Some researchers have investigated the application of GNN in the medical field, but the scope is broad, and its application to NDs is less frequent and not detailed enough. This review focuses on the research progress of GNNs in the diagnosis of ND. Firstly, we systematically investigated the GNN framework of ND, including graph construction, graph convolution, graph pooling, and graph prediction. Secondly, we investigated common NDs using the GNN diagnostic model in terms of data modality, number of subjects, and diagnostic accuracy. Thirdly, we discussed some research challenges and future research directions. The results of this review may be a valuable contribution to the ongoing intersection of artificial intelligence technology and brain imaging.

Dynamic weighted hypergraph convolutional network for brain functional connectome analysis

The hypergraph structure has been utilized to characterize the brain functional connectome (FC) by capturing the high order relationships among multiple brain regions of interest (ROIs) compared with a simple graph. Accordingly, hypergraph neural network (HGNN) models have emerged and provided efficient tools for hypergraph embedding learning. However, most existing HGNN models can only be applied to pre-constructed hypergraphs with a static structure during model training, which might not be a sufficient representation of the complex brain networks. In this study, we propose a dynamic weighted hypergraph convolutional network (dwHGCN) framework to consider a dynamic hypergraph with learnable hyperedge weights. Specifically, we generate hyperedges based on sparse representation and calculate the hyper similarity as node features. The hypergraph and node features are fed into a neural network model, where the hyperedge weights are updated adaptively during training. The dwHGCN facilitates the learning of brain FC features by assigning larger weights to hyperedges with higher discriminative power. The weighting strategy also improves the interpretability of the model by identifying the highly active interactions among ROIs shared by a common hyperedge. We validate the performance of the proposed model on two classification tasks with three paradigms functional magnetic resonance imaging (fMRI) data from Philadelphia Neurodevelopmental Cohort. Experimental results demonstrate the superiority of our proposed method over existing hypergraph neural networks. We believe our model can be applied to other applications in neuroimaging for its strength in representation learning and interpretation.

Graph-matching distance between individuals' functional connectomes varies with relatedness, age, and cognitive score

Functional connectomes (FCs), represented by networks or graphs that summarize coactivation patterns between pairs of brain regions, have been related at a population level to age, sex, cognitive/behavioral scores, life experience, genetics, and disease/disorders. However, quantifying FC differences between individuals also provides a rich source of information with which to map to differences in those individuals' biology, experience, genetics or behavior. In this study, graph matching is used to create a novel inter-individual FC metric, called swap distance, that quantifies the distance between pairs of individuals' partial FCs, with a smaller swap distance indicating the individuals have more similar FC. We apply graph matching to align FCs between individuals from the the Human Connectome ProjectN = 997 and find that swap distance (i) increases with increasing familial distance, (ii) increases with subjects' ages, (iii) is smaller for pairs of females compared to pairs of males, and (iv) is larger for females with lower cognitive scores compared to females with larger cognitive scores. Regions that contributed most to individuals' swap distances were in higher-order networks, that is, default-mode and fronto-parietal, that underlie executive function and memory. These higher-order networks' regions also had swap frequencies that varied monotonically with familial relatedness of the individuals in question. We posit that the proposed graph matching technique provides a novel way to study inter-subject differences in FC and enables quantification of how FC may vary with age, relatedness, sex, and behavior.

Discriminative analysis of schizophrenia patients using graph convolutional networks: A combined multimodal MRI and connectomics analysis

Introduction

Recent studies in human brain connectomics with multimodal magnetic resonance imaging (MRI) data have widely reported abnormalities in brain structure, function and connectivity associated with schizophrenia (SZ). However, most previous discriminative studies of SZ patients were based on MRI features of brain regions, ignoring the complex relationships within brain networks.

Methods

We applied a graph convolutional network (GCN) to discriminating SZ patients using the features of brain region and connectivity derived from a combined multimodal MRI and connectomics analysis. Structural magnetic resonance imaging (sMRI) and resting-state functional magnetic resonance imaging (rs-fMRI) data were acquired from 140 SZ patients and 205 normal controls. Eighteen types of brain graphs were constructed for each subject using 3 types of node features, 3 types of edge features, and 2 brain atlases. We investigated the performance of 18 brain graphs and used the TopK pooling layers to highlight salient brain regions (nodes in the graph).

Results

The GCN model, which used functional connectivity as edge features and multimodal features (sMRI + fMRI) of brain regions as node features, obtained the highest average accuracy of 95.8%, and outperformed other existing classification studies in SZ patients. In the explainability analysis, we reported that the top 10 salient brain regions, predominantly distributed in the prefrontal and occipital cortices, were mainly involved in the systems of emotion and visual processing.

Discussion

Our findings demonstrated that GCN with a combined multimodal MRI and connectomics analysis can effectively improve the classification of SZ at an individual level, indicating a promising direction for the diagnosis of SZ patients. The code is available at https://github.com/CXY-scut/GCN-SZ.git.

Multicenter and Multichannel Pooling GCN for Early AD Diagnosis Based on Dual-Modality Fused Brain Network

For significant memory concern (SMC) and mild cognitive impairment (MCI), their classification performance is limited by confounding features, diverse imaging protocols, and limited sample size. To address the above limitations, we introduce a dual-modality fused brain connectivity network combining resting-state functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI), and propose three mechanisms in the current graph convolutional network (GCN) to improve classifier performance. First, we introduce a DTI-strength penalty term for constructing functional connectivity networks. Stronger structural connectivity and bigger structural strength diversity between groups provide a higher opportunity for retaining connectivity information. Second, a multi-center attention graph with each node representing a subject is proposed to consider the influence of data source, gender, acquisition equipment, and disease status of those training samples in GCN. The attention mechanism captures their different impacts on edge weights. Third, we propose a multi-channel mechanism to improve filter performance, assigning different filters to features based on feature statistics. Applying those nodes with low-quality features to perform convolution would also deteriorate filter performance. Therefore, we further propose a pooling mechanism, which introduces the disease status information of those training samples to evaluate the quality of nodes. Finally, we obtain the final classification results by inputting the multi-center attention graph into the multi-channel pooling GCN. The proposed method is tested on three datasets (i.e., an ADNI 2 dataset, an ADNI 3 dataset, and an in-house dataset). Experimental results indicate that the proposed method is effective and superior to other related algorithms, with a mean classification accuracy of 93.05% in our binary classification tasks. Our code is available at: https://github.com/Xuegang-S.

Classification of Brain Disorders in rs-fMRI via Local-to-Global Graph Neural Networks

Recently, functional brain network has been used for the classification of brain disorders, such as Autism Spectrum Disorder (ASD) and Alzheimer's disease (AD). Existing methods either ignore the non-imaging information associated with the subjects and the relationship between the subjects, or cannot identify and analyze disease-related local brain regions and biomarkers, leading to inaccurate classification results. This paper proposes a local-to-global graph neural network (LG-GNN) to address this issue. A local ROI-GNN is designed to learn feature embeddings of local brain regions and identify biomarkers, and a global Subject-GNN is then established to learn the relationship between the subjects with the embeddings generated by the local ROI-GNN and the non-imaging information. The local ROI-GNN contains a self-attention based pooling module to preserve the embeddings most important for the classification. The global Subject-GNN contains an adaptive weight aggregation block to generate the multi-scale feature embedding corresponding to each subject. The proposed LG-GNN is thoroughly validated using two public datasets for ASD and AD classification. The experimental results demonstrated that it achieves the state-of-the-art performance in terms of various evaluation metrics.

GATE: Graph CCA for Temporal Self-Supervised Learning for Label-Efficient fMRI Analysis

In this work, we focus on the challenging task, neuro-disease classification, using functional magnetic resonance imaging (fMRI). In population graph-based disease analysis, graph convolutional neural networks (GCNs) have achieved remarkable success. However, these achievements are inseparable from abundant labeled data and sensitive to spurious signals. To improve fMRI representation learning and classification under a label-efficient setting, we propose a novel and theory-driven self-supervised learning (SSL) framework on GCNs, namely Graph CCA for Temporal sElf-supervised learning on fMRI analysis (GATE). Concretely, it is demanding to design a suitable and effective SSL strategy to extract formation and robust features for fMRI. To this end, we investigate several new graph augmentation strategies from fMRI dynamic functional connectives (FC) for SSL training. Further, we leverage canonical-correlation analysis (CCA) on different temporal embeddings and present the theoretical implications. Consequently, this yields a novel two-step GCN learning procedure comprised of (i) SSL on an unlabeled fMRI population graph and (ii) fine-tuning on a small labeled fMRI dataset for a classification task. Our method is tested on two independent fMRI datasets, demonstrating superior performance on autism and dementia diagnosis. Our code is available at https://github.com/LarryUESTC/GATE.

MNC-Net: Multi-task graph structure learning based on node clustering for early Parkinson's disease diagnosis

PURPOSE

The identification of early-stage Parkinson's disease (PD) is important for the effective management of patients, affecting their treatment and prognosis. Recently, structural brain networks (SBNs) have been used to diagnose PD. However, how to mine abnormal patterns from high-dimensional SBNs has been a challenge due to the complex topology of the brain. Meanwhile, the existing prediction mechanisms of deep learning models are often complicated, and it is difficult to extract effective interpretations. In addition, most works only focus on the classification of imaging and ignore clinical scores in practical applications, which limits the ability of the model. Inspired by the regional modularity of SBNs, we adopted graph learning from the perspective of node clustering to construct an interpretable framework for PD classification.

METHODS

In this study, a multi-task graph structure learning framework based on node clustering (MNC-Net) is proposed for the early diagnosis of PD. Specifically, we modeled complex SBNs into modular graphs that facilitated the representation learning of abnormal patterns. Traditional graph neural networks are optimized through graph structure learning based on node clustering, which identifies potentially abnormal brain regions and reduces the impact of irrelevant noise. Furthermore, we employed a regression task to link clinical scores to disease classification, and incorporated latent domain information into model training through multi-task learning.

RESULTS

We validated the proposed approach on the Parkinsons Progression Markers Initiative dataset. Experimental results showed that our MNC-Net effectively separated the early-stage PD from healthy controls(HC) with an accuracy of 95.5%. The t-SNE figures have showed that our graph structure learning method can capture more efficient and discriminatory features. Furthermore, node clustering parameters were used as important weights to extract salient task-related brain regions(ROIs). These ROIs are involved in the development of mood disorders, tremors, imbalances and other symptoms, highlighting the importance of memory, language and mild motor function in early PD. In addition, statistical results from clinical scores confirmed that our model could capture abnormal connectivity that was significantly different between PD and HC. These results are consistent with previous studies, demonstrating the interpretability of our methods.

Ensemble graph neural network model for classification of major depressive disorder using whole-brain functional connectivity

Major depressive disorder (MDD) is characterized by impairments in mood and cognitive functioning, and it is a prominent source of global disability and stress. A functional magnetic resonance imaging (fMRI) can aid clinicians in their assessments of individuals for the identification of MDD. Herein, we employ a deep learning approach to the issue of MDD classification. Resting-state fMRI data from 821 individuals with MDD and 765 healthy controls (HCs) is employed for investigation. An ensemble model based on graph neural network (GNN) has been created with the goal of identifying patients with MDD among HCs as well as differentiation between first-episode and recurrent MDDs. The graph convolutional network (GCN), graph attention network (GAT), and GraphSAGE models serve as a base models for the ensemble model that was developed with individual whole-brain functional networks. The ensemble's performance is evaluated using upsampling and downsampling, along with 10-fold cross-validation. The ensemble model achieved an upsampling accuracy of 71.18% and a downsampling accuracy of 70.24% for MDD and HC classification. While comparing first-episode patients with recurrent patients, the upsampling accuracy is 77.78% and the downsampling accuracy is 71.96%. According to the findings of this study, the proposed GNN-based ensemble model achieves a higher level of accuracy and suggests that our model produces can assist healthcare professionals in identifying MDD.

Exploring high-order correlations with deep-broad learning for autism spectrum disorder diagnosis

Recently, a lot of research has been conducted on diagnosing neurological disorders, such as autism spectrum disorder (ASD). Functional magnetic resonance imaging (fMRI) is the commonly used technique to assist in the diagnosis of ASD. In the past years, some conventional methods have been proposed to extract the low-order functional connectivity network features for ASD diagnosis, which ignore the complexity and global features of the brain network. Most deep learning-based methods generally have a large number of parameters that need to be adjusted during the learning process. To overcome the limitations mentioned above, we propose a novel deep-broad learning method for learning the higher-order brain functional connectivity network features to assist in ASD diagnosis. Specifically, we first construct the high-order functional connectivity network that describes global correlations of the brain regions based on hypergraph, and then we use the deep-broad learning method to extract the high-dimensional feature representations for brain networks sequentially. The evaluation of the proposed method is conducted on Autism Brain Imaging Data Exchange (ABIDE) dataset. The results show that our proposed method can achieve 71.8% accuracy on the multi-center dataset and 70.6% average accuracy on 17 single-center datasets, which are the best results compared with the state-of-the-art methods. Experimental results demonstrate that our method can describe the global features of the brain regions and get rich discriminative information for the classification task.

Multi-modal intermediate integrative methods in neuropsychiatric disorders: A review

The etiology of neuropsychiatric disorders involves complex biological processes at different omics layers, such as genomics, transcriptomics, epigenetics, proteomics, and metabolomics. The advent of high-throughput technology, as well as the availability of large open-source datasets, has ushered in a new era in system biology, necessitating the integration of various types of omics data. The complexity of biological mechanisms, the limitations of integrative strategies, and the heterogeneity of multi-omics data have all presented significant challenges to computational scientists. In comparison to early and late integration, intermediate integration may transform each data type into appropriate intermediate representations using various data transformation techniques, allowing it to capture more complementary information contained in each omics and highlight new interactions across omics layers. Here, we reviewed multi-modal intermediate integrative techniques based on component analysis, matrix factorization, similarity network, multiple kernel learning, Bayesian network, artificial neural networks, and graph transformation, as well as their applications in neuropsychiatric domains. We depicted advancements in these approaches and compared the strengths and weaknesses of each method examined. We believe that our findings will aid researchers in their understanding of the transformation and integration of multi-omics data in neuropsychiatric disorders.

Extracting default mode network based on graph neural network for resting state fMRI study

Functional magnetic resonance imaging (fMRI)-based study of functional connections in the brain has been highlighted by numerous human and animal studies recently, which have provided significant information to explain a wide range of pathological conditions and behavioral characteristics. In this paper, we propose the use of a graph neural network, a deep learning technique called graphSAGE, to investigate resting state fMRI (rs-fMRI) and extract the default mode network (DMN). Comparing typical methods such as seed-based correlation, independent component analysis, and dictionary learning, real data experiment results showed that the graphSAGE is more robust, reliable, and defines a clearer region of interests. In addition, graphSAGE requires fewer and more relaxed assumptions, and considers the single subject analysis and group subjects analysis simultaneously.

The alterations of brain functional connectivity networks in major depressive disorder detected by machine learning through multisite rs-fMRI data

BACKGROUND

The current diagnosis of major depressive disorder (MDD) is mainly based on the patient's self-report and clinical symptoms. Machine learning methods are used to identify MDD using resting-state functional magnetic resonance imaging (rs-fMRI) data. However, due to large site differences in multisite rs-fMRI data and the difficulty of sample collection, most of the current machine learning studies use small sample sizes of rs-fMRI datasets to detect the alterations of functional connectivity (FC) or network attribute (NA), which may affect the reliability of the experimental results.

METHODS

Multisite rs-fMRI data were used to increase the size of the sample, and then we extracted the functional connectivity (FC) and network attribute (NA) features from 1611 rs-fMRI data (832 patients with MDD (MDDs) and 779 healthy controls (HCs)). ComBat algorithm was used to harmonize the data variances caused by the multisite effect, and multivariate linear regression was used to remove age and sex covariates. Two-sample t-test and wrapper-based feature selection methods (support vector machine recursive feature elimination with cross-validation (SVM-RFECV) and LightGBM's "feature_importances_" function) were used to select important features. The Shapley additive explanations (SHAP) method was used to assign the contribution of features to the best classification effect model.

RESULTS

The best result was obtained from the LinearSVM model trained with the 136 important features selected by SVMRFE-CV. In the nested five-fold cross-validation (consisting of an outer and an inner loop of five-fold cross-validation) of 1611 data, the model achieved the accuracy, sensitivity, and specificity of 68.90 %, 71.75 %, and 65.84 %, respectively. The 136 important features were tested in a small dataset and obtained excellent classification results after balancing the ratio between patients with depression and HCs.

CONCLUSIONS

The combined use of FC and NA features is effective for classifying MDDs and HCs. The important FC and NA features extracted from the large sample dataset have some generalization performance and may be used as a reference for the altered brain functional connectivity networks in MDD.

An enhanced multi-modal brain graph network for classifying neuropsychiatric disorders

It has been proven that neuropsychiatric disorders (NDs) can be associated with both structures and functions of brain regions. Thus, data about structures and functions could be usefully combined in a comprehensive analysis. While brain structural MRI (sMRI) images contain anatomic and morphological information about NDs, functional MRI (fMRI) images carry complementary information. However, efficient extraction and fusion of sMRI and fMRI data remains challenging. In this study, we develop an enhanced multi-modal graph convolutional network (MME-GCN) in a binary classification between patients with NDs and healthy controls, based on the fusion of the structural and functional graphs of the brain region. First, based on the same brain atlas, we construct structural and functional graphs from sMRI and fMRI data, respectively. Second, we use machine learning to extract important features from the structural graph network. Third, we use these extracted features to adjust the corresponding edge weights in the functional graph network. Finally, we train a multi-layer GCN and use it in binary classification task. MME-GCN achieved 93.71% classification accuracy on the open data set provided by the Consortium for Neuropsychiatric Phenomics. In addition, we analyzed the important features selected from the structural graph and verified them in the functional graph. Using MME-GCN, we found several specific brain connections important to NDs.

Multi-View Feature Enhancement Based on Self-Attention Mechanism Graph Convolutional Network for Autism Spectrum Disorder Diagnosis

Functional connectivity (FC) network based on resting-state functional magnetic resonance imaging (rs-fMRI) has become an important tool to explore and understand the brain, which can provide objective basis for the diagnosis of neurodegenerative diseases, such as autism spectrum disorder (ASD). However, most functional connectivity (FC) networks only consider the unilateral features of nodes or edges, and the interaction between them is ignored. In fact, their integration can provide more comprehensive and crucial information in the diagnosis. To address this issue, a new multi-view brain network feature enhancement method based on self-attention mechanism graph convolutional network (SA-GCN) is proposed in this article, which can enhance node features through the connection relationship among different nodes, and then extract deep-seated and more discriminative features. Specifically, we first plug the pooling operation of self-attention mechanism into graph convolutional network (GCN), which can consider the node features and topology of graph network at the same time and then capture more discriminative features. In addition, the sample size is augmented by a "sliding window" strategy, which is beneficial to avoid overfitting and enhance the generalization ability. Furthermore, to fully explore the complex connection relationship among brain regions, we constructed the low-order functional graph network (Lo-FGN) and the high-order functional graph network (Ho-FGN) and enhance the features of the two functional graph networks (FGNs) based on SA-GCN. The experimental results on benchmark datasets show that: (1) SA-GCN can play a role in feature enhancement and can effectively extract more discriminative features, and (2) the integration of Lo-FGN and Ho-FGN can achieve the best ASD classification accuracy (79.9%), which reveals the information complementarity between them.

MAMF-GCN: Multi-scale adaptive multi-channel fusion deep graph convolutional network for predicting mental disorder

PURPOSE

Existing diagnoses of mental disorders rely on symptoms, patient descriptions, and scales, which are not objective enough. We attempt to explore an objective diagnostic method on fMRI data. Graph neural networks (GNN) have been paid more attention recently because of their advantages in processing unstructured relational data, especially for fMRI data. However, how to deeply embed and well-integrate with different modalities and scales on GNN is still a challenge. Instead of reaching a high degree of fusion, existing GCN methods simply combine image and non-image data. Most graph convolutional network (GCN) models use shallow structures, making it challenging to learn about potential information. Furthermore, current graph construction approaches usually use a single specific brain atlas, limiting the analysis and results.

METHOD

In this paper, a multi-scale adaptive multi-channel fusion deep graph convolutional network based on an attention mechanism (MAMF-GCN) is proposed to better integrate features of modalities and different atlas by exploiting multi-channel correlation. An encoder automatically combines one channel with non-imaging data to generate similarity weights between subjects using a similarity perception mechanism. Other channels generate multi-scale imaging features of fMRI data after processing in the different atlas. Multi-modal information is fused using an adaptive convolution module that applies a deep graph convolutional network (GCN) to extract information from richer hidden layers.

RESULTS

To demonstrate the effectiveness of our approach, we evaluate the performance of the proposed method on the Autism Brain Imaging Data Exchange (ABIDE) dataset and the Major Depressive Disorder (MDD) dataset. The experimental result shows that the proposed method outperforms many state-of-the-art methods in node classification performance. An extensive group of experiments on two disease prediction tasks demonstrates that the performance of the proposed MAMF-GCN on MDD/ABIDE dataset is improved by 3.37%-39.83% and 12.59%-32.92%, respectively. Moreover, our proposed method has also shown very effective performance in real-life clinical diagnosis. The comprehensive experiments demonstrate that our method is effective for node classification with brain disorders diagnosis.

CONCLUSION

The proposed MAMF-GCN method simultaneously extracts specific and common embeddings from the topology composed of multi-scale imaging features, phenotypic information, and their combinations, then learning adaptive embedding weights by attention mechanism, which can capture and fuse the multi-scale essential embeddings to improve the classification performance of brain disorder diagnosis.

SSPNet: An interpretable 3D-CNN for classification of schizophrenia using phase maps of resting-state complex-valued fMRI data

Convolutional neural networks (CNNs) have shown promising results in classifying individuals with mental disorders such as schizophrenia using resting-state fMRI data. However, complex-valued fMRI data is rarely used since additional phase data introduces high-level noise though it is potentially useful information for the context of classification. As such, we propose to use spatial source phase (SSP) maps derived from complex-valued fMRI data as the CNN input. The SSP maps are not only less noisy, but also more sensitive to spatial activation changes caused by mental disorders than magnitude maps. We build a 3D-CNN framework with two convolutional layers (named SSPNet) to fully explore the 3D structure and voxel-level relationships from the SSP maps. Two interpretability modules, consisting of saliency map generation and gradient-weighted class activation mapping (Grad-CAM), are incorporated into the well-trained SSPNet to provide additional information helpful for understanding the output. Experimental results from classifying schizophrenia patients (SZs) and healthy controls (HCs) show that the proposed SSPNet significantly improved accuracy and AUC compared to CNN using magnitude maps extracted from either magnitude-only (by 23.4 and 23.6% for DMN) or complex-valued fMRI data (by 10.6 and 5.8% for DMN). SSPNet captured more prominent HC-SZ differences in saliency maps, and Grad-CAM localized all contributing brain regions with opposite strengths for HCs and SZs within SSP maps. These results indicate the potential of SSPNet as a sensitive tool that may be useful for the development of brain-based biomarkers of mental disorders.