Understanding Graph Isomorphism Network for rs-fMRI Functional Connectivity Analysis

Siamese Graph Convolutional Network quantifies increasing structure-function discrepancy over the cognitive decline continuum

BACKGROUND AND OBJECTIVE

Alzheimer's disease dementia (ADD) is well known to induce alterations in both structural and functional brain connectivity. However, reported changes in connectivity are mostly limited to global/local network features, which have poor specificity for diagnostic purposes. Following recent advances in machine learning, deep neural networks, particularly Graph Neural Network (GNN) based approaches, have found applications in brain research as well. The majority of existing applications of GNNs employ a single network (uni-modal or structure/function unified), despite the widely accepted view that there is a nontrivial interdependence between the brain's structural connectivity and the neural activity patterns, which is hypothesized to be disrupted in ADD. This disruption is quantified as a discrepancy score by the proposed "structure-function discrepancy learning network" (sfDLN) and its distribution is studied over the spectrum of clinical cognitive decline. The measured discrepancy score is utilized as a diagnostic biomarker and is compared with state-of-the-art diagnostic classifiers.

METHODS

sfDLN is a GNN with a siamese architecture built on the hypothesis that the mismatch between structural and functional connectivity patterns increases over the cognitive decline spectrum, starting from subjective cognitive impairment (SCI), passing through a mid-stage mild cognitive impairment (MCI), and ending up with ADD. The structural brain connectome (sNET) built using diffusion MRI-based tractography and the novel, sparse (lean) functional brain connectome (ℓNET) built using fMRI are input to sfDLN. The siamese sfDLN is trained to extract connectome representations and a discrepancy (dissimilarity) score that complies with the proposed hypothesis and is blindly tested on an MCI group.

RESULTS

The sfDLN generated structure-function discrepancy scores show high disparity between ADD and SCI subjects. Leave-one-out experiments of SCI-ADD classification over a cohort of 42 subjects reach 88% accuracy, surpassing state-of-the-art GNN-based classifiers in the literature. Furthermore, a blind assessment over a cohort of 46 MCI subjects confirmed that it captures the intermediary character of the MCI group. GNNExplainer module employed to investigate the anatomical determinants of the observed discrepancy confirms that sfDLN attends to cortical regions neurologically relevant to ADD.

CONCLUSION

In support of our hypothesis, the harmony between the structural and functional organization of the brain degrades with increasing cognitive decline. This discrepancy, shown to be rooted in brain regions neurologically relevant to ADD, can be quantified by sfDLN and outperforms state-of-the-art GNN-based ADD classification methods when used as a biomarker.

PyMulSim: a method for computing node similarities between multilayer networks via graph isomorphism networks

BACKGROUND

In bioinformatics, interactions are modelled as networks, based on graph models. Generally, these support a single-layer structure which incorporates a specific entity (i.e., node) and only one type of link (i.e., edge). However, real-world biological systems consisting of biological objects belonging to heterogeneous entities, and these operate and influence each other in multiple contexts, simultaneously. Usually, node similarities are investigated to assess the relatedness between biological objects in a network of interest, and node embeddings are widely used for studying novel interaction from a topological point of view. About that, the state-of-the-art presents several methods for evaluating the node similarity inside a given network, but methodologies able to evaluate similarities between pairs of nodes belonging to different networks are missing. The latter are crucial for studies that relate different biological networks, e.g., for Network Alignment or to evaluate the possible evolution of the interactions of a little-known network on the basis of a well-known one. Existing methods are ineffective in evaluating nodes outside their structure, even more so in the context of multilayer networks, in which the topic still exploits approaches adapted from static networks. In this paper, we presented pyMulSim, a novel method for computing the pairwise similarities between nodes belonging to different multilayer networks. It uses a Graph Isomorphism Network (GIN) for the representative learning of node features, that uses for processing the embeddings and computing the similarities between the pairs of nodes of different multilayer networks.

RESULTS

Our experimentation investigated the performance of our method. Results show that our method effectively evaluates the similarities between the biological objects of a source multilayer network to a target one, based on the analysis of the node embeddings. Results have been also assessed for different noise levels, also through statistical significance analyses properly performed for this purpose.

CONCLUSIONS

PyMulSim is a novel method for computing the pairwise similarities between nodes belonging to different multilayer networks, by using a GIN for learning node embeddings. It has been evaluated both in terms of performance and validity, reporting a high degree of reliability.

Adversarial Learning Based Node-Edge Graph Attention Networks for Autism Spectrum Disorder Identification

Graph neural networks (GNNs) have received increasing interest in the medical imaging field given their powerful graph embedding ability to characterize the non-Euclidean structure of brain networks based on magnetic resonance imaging (MRI) data. However, previous studies are largely node-centralized and ignore edge features for graph classification tasks, resulting in moderate performance of graph classification accuracy. Moreover, the generalizability of GNN model is still far from satisfactory in brain disorder [e.g., autism spectrum disorder (ASD)] identification due to considerable individual differences in symptoms among patients as well as data heterogeneity among different sites. In order to address the above limitations, this study proposes a novel adversarial learning-based node-edge graph attention network (AL-NEGAT) for ASD identification based on multimodal MRI data. First, both node and edge features are modeled based on structural and functional MRI data to leverage complementary brain information and preserved in the constructed weighted adjacent matrix for individuals through the attention mechanism in the proposed NEGAT. Second, two AL methods are employed to improve the generalizability of NEGAT. Finally, a gradient-based saliency map strategy is utilized for model interpretation to identify important brain regions and connections contributing to the classification. Experimental results based on the public Autism Brain Imaging Data Exchange I (ABIDE I) data demonstrate that the proposed framework achieves a classification accuracy of 74.7% between ASD and typical developing (TD) groups based on 1007 subjects across 17 different sites and outperforms the state-of-the-art methods, indicating satisfying classification ability and generalizability of the proposed AL-NEGAT model. Our work provides a powerful tool for brain disorder identification.

Cross-Domain Identification of Multisite Major Depressive Disorder Using End-to-End Brain Dynamic Attention Network

Establishing objective and quantitative imaging markers at individual level can assist in accurate diagnosis of Major Depressive Disorder (MDD). However, the clinical heterogeneity of MDD and the shift to multisite data decreased identification accuracy. To address these issues, the Brain Dynamic Attention Network (BDANet) is innovatively proposed, and analyzed bimodal scans from 2055 participants of the Rest-meta-MDD consortium. The end-to-end BDANet contains two crucial components. The Dynamic BrainGraph Generator dynamically focuses and represents topological relationships between Regions of Interest, overcoming limitations of static methods. The Ensemble Classifier is constructed to obfuscate domain sources to achieve inter-domain alignment. Finally, BDANet dynamically generates sample-specific brain graphs by downstream recognition tasks. The proposed BDANet achieved an accuracy of 81.6%. The regions with high attribution for classification were mainly located in the insula, cingulate cortex and auditory cortex. The level of brain connectivity in p24 region was negatively correlated ( [Formula: see text]) with the severity of MDD. Additionally, sex differences in connectivity strength were observed in specific brain regions and functional subnetworks ( [Formula: see text] or [Formula: see text]). These findings based on a large multisite dataset support the conclusion that BDANet can better solve the problem of the clinical heterogeneity of MDD and the shift of multisite data. It also illustrates the potential utility of BDANet for personalized accurate identification, treatment and intervention of MDD.

Preserving specificity in federated graph learning for fMRI-based neurological disorder identification

Resting-state functional magnetic resonance imaging (rs-fMRI) offers a non-invasive approach to examining abnormal brain connectivity associated with brain disorders. Graph neural network (GNN) gains popularity in fMRI representation learning and brain disorder analysis with powerful graph representation capabilities. Training a general GNN often necessitates a large-scale dataset from multiple imaging centers/sites, but centralizing multi-site data generally faces inherent challenges related to data privacy, security, and storage burden. Federated Learning (FL) enables collaborative model training without centralized multi-site fMRI data. Unfortunately, previous FL approaches for fMRI analysis often ignore site-specificity, including demographic factors such as age, gender, and education level. To this end, we propose a specificity-aware federated graph learning (SFGL) framework for rs-fMRI analysis and automated brain disorder identification, with a server and multiple clients/sites for federated model aggregation and prediction. At each client, our model consists of a shared and a personalized branch, where parameters of the shared branch are sent to the server while those of the personalized branch remain local. This can facilitate knowledge sharing among sites and also helps preserve site specificity. In the shared branch, we employ a spatio-temporal attention graph isomorphism network to learn dynamic fMRI representations. In the personalized branch, we integrate vectorized demographic information (i.e., age, gender, and education years) and functional connectivity networks to preserve site-specific characteristics. Representations generated by the two branches are then fused for classification. Experimental results on two fMRI datasets with a total of 1218 subjects suggest that SFGL outperforms several state-of-the-art approaches.

Predicting treatment response in adolescents and young adults with major depressive episodes from fMRI using graph isomorphism network

BACKGROUND

Major depressive episode (MDE) is the main clinical feature of mood disorders (major depressive disorder and bipolar disorder) in adolescents and young adults and accounts for most of the disease course. However, 30%-40% of MDE patients not responding to clinical first-line interventions. It is crucial to predict treatment response in the early stages and identify biomarkers associated with treatment response. Graph Isomorphism Network (GIN), a deep learning method, is promising for predicting treatment response for individual MDE patients with more powerful representation ability to capture the features of brain functional connectivity.

METHODS

In this study, GIN was used to predict individual treatment response in 198 adolescents and young adults with MDE. The most discriminating regions were also identified for the treatment response prediction.

RESULTS

Using GIN approach, the baseline functional connectivity could predict 79.8% responders and 67.4% non-responders to treatment (accuracy 74.24%). Furthermore, the most discriminating brain regions were mainly involved in paralimbic and subcortical areas.

CONCLUSIONS

GIN has shown potential in predicting treatment response for individual patients, which may enable personalized treatment decisions. Furthermore, targeted interventions focused on modulating the activity and connectivity within paralimbic and subcortical regions could potentially improve treatment outcomes and enable personalized interventions for adolescents and young adults with MDE.

Modularity-Constrained Dynamic Representation Learning for Interpretable Brain Disorder Analysis with Functional MRI

Resting-state functional MRI (rs-fMRI) is increasingly used to detect altered functional connectivity patterns caused by brain disorders, thereby facilitating objective quantification of brain pathology. Existing studies typically extract fMRI features using various machine/deep learning methods, but the generated imaging biomarkers are often challenging to interpret. Besides, the brain operates as a modular system with many cognitive/topological modules, where each module contains subsets of densely inter-connected regions-of-interest (ROIs) that are sparsely connected to ROIs in other modules. However, current methods cannot effectively characterize brain modularity. This paper proposes a modularity-constrained dynamic representation learning (MDRL) framework for interpretable brain disorder analysis with rs-fMRI. The MDRL consists of 3 parts: (1) dynamic graph construction, (2) modularity-constrained spatiotemporal graph neural network (MSGNN) for dynamic feature learning, and (3) prediction and biomarker detection. In particular, the MSGNN is designed to learn spatiotemporal dynamic representations of fMRI, constrained by 3 functional modules (i.e., central executive network, salience network, and default mode network). To enhance discriminative ability of learned features, we encourage the MSGNN to reconstruct network topology of input graphs. Experimental results on two public and one private datasets with a total of 1,155 subjects validate that our MDRL outperforms several state-of-the-art methods in fMRI-based brain disorder analysis. The detected fMRI biomarkers have good explainability and can be potentially used to improve clinical diagnosis.

Multi-Scale Dynamic Graph Learning for Brain Disorder Detection With Functional MRI

Resting-state functional magnetic resonance imaging (rs-fMRI) has been widely used in the detection of brain disorders such as autism spectrum disorder based on various machine/deep learning techniques. Learning-based methods typically rely on functional connectivity networks (FCNs) derived from blood-oxygen-level-dependent time series of rs-fMRI data to capture interactions between brain regions-of-interest (ROIs). Graph neural networks have been recently used to extract fMRI features from graph-structured FCNs, but cannot effectively characterize spatiotemporal dynamics of FCNs, e.g., the functional connectivity of brain ROIs is dynamically changing in a short period of time. Also, many studies usually focus on single-scale topology of FCN, thereby ignoring the potential complementary topological information of FCN at different spatial resolutions. To this end, in this paper, we propose a multi-scale dynamic graph learning (MDGL) framework to capture multi-scale spatiotemporal dynamic representations of rs-fMRI data for automated brain disorder diagnosis. The MDGL framework consists of three major components: 1) multi-scale dynamic FCN construction using multiple brain atlases to model multi-scale topological information, 2) multi-scale dynamic graph representation learning to capture spatiotemporal information conveyed in fMRI data, and 3) multi-scale feature fusion and classification. Experimental results on two datasets show that MDGL outperforms several state-of-the-art methods.

FP-MAP: an extensive library of fingerprint-based molecular activity prediction tools

Discovering new drugs for disease treatment is challenging, requiring a multidisciplinary effort as well as time, and resources. With a view to improving hit discovery and lead compound identification, machine learning (ML) approaches are being increasingly used in the decision-making process. Although a number of ML-based studies have been published, most studies only report fragments of the wider range of bioactivities wherein each model typically focuses on a particular disease. This study introduces FP-MAP, an extensive atlas of fingerprint-based prediction models that covers a diverse range of activities including neglected tropical diseases (caused by viral, bacterial and parasitic pathogens) as well as other targets implicated in diseases such as Alzheimer's. To arrive at the best predictive models, performance of ≈4,000 classification/regression models were evaluated on different bioactivity data sets using 12 different molecular fingerprints. The best performing models that achieved test set AUC values of 0.62-0.99 have been integrated into an easy-to-use graphical user interface that can be downloaded from https://gitlab.com/vishsoft/fpmap.

GLACIER: GLASS-BOX TRANSFORMER FOR INTERPRETABLE DYNAMIC NEUROIMAGING

Deep learning models can perform as well or better than humans in many tasks, especially vision related. Almost exclusively, these models are used to perform classification or prediction. However, deep learning models are usually of black-box nature, and it is often difficult to interpret the model or the features. The lack of interpretability causes a restrain from applying deep learning to fields such as neuroimaging, where the results must be transparent, and interpretable. Therefore, we present a 'glass-box' deep learning model and apply it to the field of neuroimaging. Our model mixes spatial and temporal dimensions in succession to estimate dynamic connectivity between the brain's intrinsic networks. The interpretable connectivity matrices produced by our model result in beating state-of-the-art models on many tasks using multiple functional MRI datasets. More importantly, our model estimates task-based flexible connectivity matrices, unlike static methods such as Pearson's correlation coefficients.

Graph-in-Graph (GiG): Learning interpretable latent graphs in non-Euclidean domain for biological and healthcare applications

Graphs are a powerful tool for representing and analyzing unstructured, non-Euclidean data ubiquitous in the healthcare domain. Two prominent examples are molecule property prediction and brain connectome analysis. Importantly, recent works have shown that considering relationships between input data samples has a positive regularizing effect on the downstream task in healthcare applications. These relationships are naturally modeled by a (possibly unknown) graph structure between input samples. In this work, we propose Graph-in-Graph (GiG), a neural network architecture for protein classification and brain imaging applications that exploits the graph representation of the input data samples and their latent relation. We assume an initially unknown latent-graph structure between graph-valued input data and propose to learn a parametric model for message passing within and across input graph samples, end-to-end along with the latent structure connecting the input graphs. Further, we introduce a Node Degree Distribution Loss (NDDL) that regularizes the predicted latent relationships structure. This regularization can significantly improve the downstream task. Moreover, the obtained latent graph can represent patient population models or networks of molecule clusters, providing a level of interpretability and knowledge discovery in the input domain, which is of particular value in healthcare.

Recent Advances in Explainable Artificial Intelligence for Magnetic Resonance Imaging

Advances in artificial intelligence (AI), especially deep learning (DL), have facilitated magnetic resonance imaging (MRI) data analysis, enabling AI-assisted medical image diagnoses and prognoses. However, most of the DL models are considered as "black boxes". There is an unmet need to demystify DL models so domain experts can trust these high-performance DL models. This has resulted in a sub-domain of AI research called explainable artificial intelligence (XAI). In the last decade, many experts have dedicated their efforts to developing novel XAI methods that are competent at visualizing and explaining the logic behind data-driven DL models. However, XAI techniques are still in their infancy for medical MRI image analysis. This study aims to outline the XAI applications that are able to interpret DL models for MRI data analysis. We first introduce several common MRI data modalities. Then, a brief history of DL models is discussed. Next, we highlight XAI frameworks and elaborate on the principles of multiple popular XAI methods. Moreover, studies on XAI applications in MRI image analysis are reviewed across the tissues/organs of the human body. A quantitative analysis is conducted to reveal the insights of MRI researchers on these XAI techniques. Finally, evaluations of XAI methods are discussed. This survey presents recent advances in the XAI domain for explaining the DL models that have been utilized in MRI applications.

Graph propagation network captures individual specificity of the relationship between functional and structural connectivity

Functional connectivity (FC) network characterizes the functional interactions between brain regions and is considered to root in the underlying structural connectivity (SC) network. If this is the case, individual variations in SC should cause corresponding individual variations in FC. However, divergences exist in the correspondence between direct SC and FC and researchers still cannot capture individual differences in FC via direct SC. As brain regions may interact through multi-hop indirect SC pathways, we conceived that one can capture the individual specific SC-FC relationship via incorporating indirect SC pathways appropriately. In this study, we designed graph propagation network (GPN) that models the information propagation between brain regions based on the SC network. Effects of interactions through multi-hop SC pathways naturally emerge from the multilayer information propagation in GPN. We predicted the individual differences in FC network based on SC network via multilayer GPN and results indicate that multilayer GPN incorporating effects of multi-hop indirect SCs greatly enhances the ability to predict individual FC network. Furthermore, the SC-FC relationship evaluated via the prediction accuracy is negatively correlated with the functional gradient, suggesting that the SC-FC relationship gradually uncouples along the functional hierarchy spanning from unimodal to transmodal cortex. We also revealed important intermediate brain regions along multi-hop SC pathways involving in the individual SC-FC relationship. These results suggest that multilayer GPN can serve as a method to establish individual SC-FC relationship at the macroneuroimaging level.

Few-shot learning via graph embeddings with convolutional networks for low-data molecular property prediction

Graph neural networks and convolutional architectures have proven to be pivotal in improving the prediction of molecular properties in drug discovery. However, this is fundamentally a low data problem that is incompatible with regular deep learning approaches. Contemporary deep networks require large amounts of training data, which severely limits the prediction of new molecular entities from limited available data. In this paper, we address the challenge of low data in molecular property prediction by: (1) defining a set of deep learning architectures that accept compound chemical structures in the form of molecular graphs, (2) creating a few-shot learning strategy across graph neural networks and convolutional neural networks to leverage the rich information of graph embeddings, and (3) proposing a two-module meta-learning framework to learn from task-transferable knowledge and predict molecular properties on few-shot data. Furthermore, we conduct multiple experiments on two benchmark multiproperty datasets to demonstrate a superior performance over conventional graph-based baselines. ROC-AUC results for 10-shot experiments show an average improvement of $$+11.37\%$$ + 11.37 % on Tox21 and $$+0.53\%$$ + 0.53 % on SIDER, which are representative small-sized biological datasets for molecular property prediction.

Ensemble graph neural network model for classification of major depressive disorder using whole-brain functional connectivity

Major depressive disorder (MDD) is characterized by impairments in mood and cognitive functioning, and it is a prominent source of global disability and stress. A functional magnetic resonance imaging (fMRI) can aid clinicians in their assessments of individuals for the identification of MDD. Herein, we employ a deep learning approach to the issue of MDD classification. Resting-state fMRI data from 821 individuals with MDD and 765 healthy controls (HCs) is employed for investigation. An ensemble model based on graph neural network (GNN) has been created with the goal of identifying patients with MDD among HCs as well as differentiation between first-episode and recurrent MDDs. The graph convolutional network (GCN), graph attention network (GAT), and GraphSAGE models serve as a base models for the ensemble model that was developed with individual whole-brain functional networks. The ensemble's performance is evaluated using upsampling and downsampling, along with 10-fold cross-validation. The ensemble model achieved an upsampling accuracy of 71.18% and a downsampling accuracy of 70.24% for MDD and HC classification. While comparing first-episode patients with recurrent patients, the upsampling accuracy is 77.78% and the downsampling accuracy is 71.96%. According to the findings of this study, the proposed GNN-based ensemble model achieves a higher level of accuracy and suggests that our model produces can assist healthcare professionals in identifying MDD.

Through the looking glass: Deep interpretable dynamic directed connectivity in resting fMRI

Brain network interactions are commonly assessed via functional (network) connectivity, captured as an undirected matrix of Pearson correlation coefficients. Functional connectivity can represent static and dynamic relations, but often these are modeled using a fixed choice for the data window Alternatively, deep learning models may flexibly learn various representations from the same data based on the model architecture and the training task. However, the representations produced by deep learning models are often difficult to interpret and require additional posthoc methods, e.g., saliency maps. In this work, we integrate the strengths of deep learning and functional connectivity methods while also mitigating their weaknesses. With interpretability in mind, we present a deep learning architecture that exposes a directed graph layer that represents what the model has learned about relevant brain connectivity. A surprising benefit of this architectural interpretability is significantly improved accuracy in discriminating controls and patients with schizophrenia, autism, and dementia, as well as age and gender prediction from functional MRI data. We also resolve the window size selection problem for dynamic directed connectivity estimation as we estimate windowing functions from the data, capturing what is needed to estimate the graph at each time-point. We demonstrate efficacy of our method in comparison with multiple existing models that focus on classification accuracy, unlike our interpretability-focused architecture. Using the same data but training different models on their own discriminative tasks we are able to estimate task-specific directed connectivity matrices for each subject. Results show that the proposed approach is also more robust to confounding factors compared to standard dynamic functional connectivity models. The dynamic patterns captured by our model are naturally interpretable since they highlight the intervals in the signal that are most important for the prediction. The proposed approach reveals that differences in connectivity among sensorimotor networks relative to default-mode networks are an important indicator of dementia and gender. Dysconnectivity between networks, specially sensorimotor and visual, is linked with schizophrenic patients, however schizophrenic patients show increased intra-network default-mode connectivity compared to healthy controls. Sensorimotor connectivity was important for both dementia and schizophrenia prediction, but schizophrenia is more related to dysconnectivity between networks whereas, dementia bio-markers were mostly intra-network connectivity.

Sparse Hierarchical Representation Learning on Functional Brain Networks for Prediction of Autism Severity Levels

Machine learning algorithms have been widely applied in diagnostic tools for autism spectrum disorder (ASD), revealing an altered brain connectivity. However, little is known about whether an magnetic resonance imaging (MRI)-based brain network is related to the severity of ASD symptoms in a large-scale cohort. We propose a graph convolution neural network-based framework that can generate sparse hierarchical graph representations for functional brain connectivity. Instead of assigning initial features for each node, we utilized a feature extractor to derive node features and the extracted representations can be fed to a hierarchical graph self-attention framework to effectively represent the entire graph. By incorporating connectivity embeddings in the feature extractor, we propose adjacency embedding networks to characterize the heterogeneous representations of the brain connectivity. Our proposed model variants outperform the benchmarking model with different configurations of adjacency embedding networks and types of functional connectivity matrices. Using this approach with the best configuration (SHEN atlas for node definition, Tikhonov correlation for connectivity estimation, and identity-adjacency embedding), we were able to predict individual ASD severity levels with a meaningful accuracy: the mean absolute error (MAE) and correlation between predicted and observed ASD severity scores resulted in 0.96, and r = 0.61 (P < 0.0001), respectively. To obtain a better understanding on how to generate better representations, we investigate the relationships between the extracted feature embeddings and the graph theory-based nodal measurements using canonical correlation analysis. Finally, we visualized the model to identify the most contributive functional connections for predicting ASD severity scores.

Identification of Young High-Functioning Autism Individuals Based on Functional Connectome Using Graph Isomorphism Network: A Pilot Study

Accumulated studies have determined the changes in functional connectivity in autism spectrum disorder (ASD) and spurred the application of machine learning for classifying ASD. Graph Neural Network provides a new method for network analysis in brain disorders to identify the underlying network features associated with functional deficits. Here, we proposed an improved model of Graph Isomorphism Network (GIN) that implements the Weisfeiler-Lehman (WL) graph isomorphism test to learn the graph features while taking into account the importance of each node in the classification to improve the interpretability of the algorithm. We applied the proposed method on multisite datasets of resting-state functional connectome from Autism Brain Imaging Data Exchange (ABIDE) after stringent quality control. The proposed method outperformed other commonly used classification methods on five different evaluation metrics. We also identified salient ROIs in visual and frontoparietal control networks, which could provide potential neuroimaging biomarkers for ASD identification.

EEG-GAT: Graph Attention Networks for Classification of Electroencephalogram (EEG) Signals

Graph neural networks (GNN) are an emerging framework in the deep learning community. In most GNN applications, the graph topology of data samples is provided in the dataset. Specifically, the graph shift operator (GSO), which could be adjacency, graph Laplacian, or their normalizations, is known a priori. However we often have no knowledge of the grand-truth graph topology underlying real-world datasets. One example of this is to extract subject-invariant features from physiological electroencephalogram (EEG) to predict a cognitive task. Previous methods use electrode sites to represent a node in the graph and connect them in various ways to hand-engineer a GSO e.g., i) each pair of electrode sites is connected to form a complete graph, ii) a specific number of electrode sites are connected to form a k-nearest neighbor graph, iii) each pair of electrode site is connected only if the Euclidean distance is within a heuristic threshold. In this paper, we overcome this limitation by parameterizing the GSO using a multi-head attention mechanism to explore the functional neural connectivity subject to a cognitive task between different electrode sites, and simultaneously learn the unsupervised graph topology in conjunction with the parameters of graph convolutional kernels.

Seizure detection by brain-connectivity analysis using dynamic graph isomorphism network

Epilepsy is a neurological disease caused by ab-normal neural electrical discharges. Electroencephalography (EEG) is a powerful tool to measure the brain electrical activity and has been widely used for seizure detection. Manual EEG analysis is labor-intensive and time-consuming. Automatic seizure detection is urgently demanded for long-time seizure monitoring. Many methods have been proposed for automatic seizure detection based on EEG signals. However, most of the existing methods are patient-specific with limited generaliz-ability. Few studies investigate inter-patient seizure detection, which remains challenging. The aim of the present study is therefore to develop advanced algorithms for efficient inter-patient seizure detection using EEG. To this end, dynamic brain network is employed to capture the spatiotemporal dynamics of the connectivity among brain regions. A novel graph neural network referred to as graph isomorphic network is proposed for effective local-global spatiotemporal feature extraction and seizure classification. The proposed method is evaluated with the CHB-MIT open dataset with a ten-fold cross-validation. The results reveal excellent performance for the proposed method, with accuracy, sensitivity, and specificity of 96.2%, 95.4%, and 97.0% respectively, significantly higher than the results reported in the literature. Our results provide useful information for inter-patient seizure detection, particularly for long-time ambulatory seizure monitoring.

Forecasting brain activity based on models of spatiotemporal brain dynamics: A comparison of graph neural network architectures

Comprehending the interplay between spatial and temporal characteristics of neural dynamics can contribute to our understanding of information processing in the human brain. Graph neural networks (GNNs) provide a new possibility to interpret graph-structured signals like those observed in complex brain networks. In our study we compare different spatiotemporal GNN architectures and study their ability to model neural activity distributions obtained in functional MRI (fMRI) studies. We evaluate the performance of the GNN models on a variety of scenarios in MRI studies and also compare it to a VAR model, which is currently often used for directed functional connectivity analysis. We show that by learning localized functional interactions on the anatomical substrate, GNN-based approaches are able to robustly scale to large network studies, even when available data are scarce. By including anatomical connectivity as the physical substrate for information propagation, such GNNs also provide a multimodal perspective on directed connectivity analysis, offering a novel possibility to investigate the spatiotemporal dynamics in brain networks.

A deep graph neural network architecture for modelling spatio-temporal dynamics in resting-state functional MRI data

Resting-state functional magnetic resonance imaging (rs-fMRI) has been successfully employed to understand the organisation of the human brain. Typically, the brain is parcellated into regions of interest (ROIs) and modelled as a graph where each ROI represents a node and association measures between ROI-specific blood-oxygen-level-dependent (BOLD) time series are edges. Recently, graph neural networks (GNNs) have seen a surge in popularity due to their success in modelling unstructured relational data. The latest developments with GNNs, however, have not yet been fully exploited for the analysis of rs-fMRI data, particularly with regards to its spatio-temporal dynamics. In this paper, we present a novel deep neural network architecture which combines both GNNs and temporal convolutional networks (TCNs) in order to learn from both the spatial and temporal components of rs-fMRI data in an end-to-end fashion. In particular, this corresponds to intra-feature learning (i.e., learning temporal dynamics with TCNs) as well as inter-feature learning (i.e., leveraging interactions between ROI-wise dynamics with GNNs). We evaluate our model with an ablation study using 35,159 samples from the UK Biobank rs-fMRI database, as well as in the smaller Human Connectome Project (HCP) dataset, both in a unimodal and in a multimodal fashion. We also demonstrate that out architecture contains explainability-related features which easily map to realistic neurobiological insights. We suggest that this model could lay the groundwork for future deep learning architectures focused on leveraging the inherently and inextricably spatio-temporal nature of rs-fMRI data.

Sex Differences of Cerebellum and Cerebrum: Evidence from Graph Convolutional Network

This work aims to exploit a novel graph neural network to predict the sex of the brain topological network, and to find the sex differences in the cerebrum and cerebellum. A two-branch multi-scale graph convolutional network (TMGCN) is designed to analyze the sex differences of the brain. Two complementary templates are used to construct cerebrum and cerebellum networks, respectively, followed by a two-branch sub-network with multi-scale filters and a trainable weighted fusion strategy for the final prediction. Finally, a trainable graph topk-pooling layer is utilized in our model to visualize key brain regions relevant to the prediction. The proposed TMGCN achieves a prediction accuracy of 84.48%. In the cerebellum, the bilateral Crus I-II, lobule VI and VIIb, and the posterior vermis (VI-X) are discriminative for this task. As for the cerebrum, the discriminative brain regions consist of the bilateral inferior temporal gyrus, the bilateral fusiform gyrus, the bilateral parahippocampal gyrus, the bilateral cingulate gyrus, the bilateral medial ventral occipital cortex, the bilateral lateral occipital cortex, the bilateral amygdala, and the bilateral hippocampus. This study tackles the sex prediction problem from a more comprehensive view, and may provide the resting-state fMRI evidence for further study of sex differences in the cerebellum and cerebrum.

EEG-GNN: Graph Neural Networks for Classification of Electroencephalogram (EEG) Signals

Convolutional neural networks (CNN) have been frequently used to extract subject-invariant features from electroencephalogram (EEG) for classification tasks. This approach holds the underlying assumption that electrodes are equidistant analogous to pixels of an image and hence fails to explore/exploit the complex functional neural connectivity between different electrode sites. We overcome this limitation by tailoring the concepts of convolution and pooling applied to 2D grid-like inputs for the functional network of electrode sites. Furthermore, we develop various graph neural network (GNN) models that project electrodes onto the nodes of a graph, where the node features are represented as EEG channel samples collected over a trial, and nodes can be connected by weighted/unweighted edges according to a flexible policy formulated by a neuroscientist. The empirical evaluations show that our proposed GNN-based framework outperforms standard CNN classifiers across ErrP, and RSVP datasets, as well as allowing neuroscientific interpretability and explainability to deep learning methods tailored to EEG related classification problems. Another practical advantage of our GNN-based framework is that it can be used in EEG channel selection, which is critical for reducing computational cost, and designing portable EEG headsets.

Deep graph neural network-based prediction of acute suicidal ideation in young adults

Precise remote evaluation of both suicide risk and psychiatric disorders is critical for suicide prevention as well as for psychiatric well-being. Using questionnaires is an alternative to labor-intensive diagnostic interviews in a large general population, but previous models for predicting suicide attempts suffered from low sensitivity. We developed and validated a deep graph neural network model that increased the prediction sensitivity of suicide risk in young adults (n = 17,482 for training; n = 14,238 for testing) using multi-dimensional questionnaires and suicidal ideation within 2 weeks as the prediction target. The best model achieved a sensitivity of 76.3%, specificity of 83.4%, and an area under curve of 0.878 (95% confidence interval, 0.855-0.899). We demonstrated that multi-dimensional deep features covering depression, anxiety, resilience, self-esteem, and clinico-demographic information contribute to the prediction of suicidal ideation. Our model might be useful for the remote evaluation of suicide risk in the general population of young adults for specific situations such as the COVID-19 pandemic.

Connectome-based individual prediction of cognitive behaviors via graph propagation network reveals directed brain network topology

Objective. Brain connectivity network supports the information flow underlying human cognitions and should reflect the individual variability in human cognitive behaviors. Various studies have utilized brain connectivity to predict individual differences in human behaviors. However, traditional studies viewed brain connectivity network as a one-dimensional vector, a method which neglects topological properties of brain connectivity network.Approach. To utilize these topological properties, we proposed that graph neural network (GNN) which combines graph theory and neural network can be adopted. Different from previous node-driven GNNs that parameterize on the node feature transformation, we designed an edge-driven GNN named graph propagation network (GPN) that parameterizes on the information propagation within brain connectivity network.Main results.Edge-driven GPN outperforms various baseline models such as node-driven GNN and traditional partial least square regression in predicting the individual total cognition based on the resting-state functional connectome. GPN also reveals a directed network topology encoding the information flow, indicating that higher-order association cortices such as dorsolateral prefrontal, inferior frontal and inferior parietal cortices are responsible for the information integration underlying total cognition.Significance. These results suggest that edge-driven GPN can better explore topological structures of brain connectivity network and can serve as a new method to associate brain connectome and human behaviors.

A graph neural network framework for causal inference in brain networks

A central question in neuroscience is how self-organizing dynamic interactions in the brain emerge on their relatively static structural backbone. Due to the complexity of spatial and temporal dependencies between different brain areas, fully comprehending the interplay between structure and function is still challenging and an area of intense research. In this paper we present a graph neural network (GNN) framework, to describe functional interactions based on the structural anatomical layout. A GNN allows us to process graph-structured spatio-temporal signals, providing a possibility to combine structural information derived from diffusion tensor imaging (DTI) with temporal neural activity profiles, like that observed in functional magnetic resonance imaging (fMRI). Moreover, dynamic interactions between different brain regions discovered by this data-driven approach can provide a multi-modal measure of causal connectivity strength. We assess the proposed model's accuracy by evaluating its capabilities to replicate empirically observed neural activation profiles, and compare the performance to those of a vector auto regression (VAR), like that typically used in Granger causality. We show that GNNs are able to capture long-term dependencies in data and also computationally scale up to the analysis of large-scale networks. Finally we confirm that features learned by a GNN can generalize across MRI scanner types and acquisition protocols, by demonstrating that the performance on small datasets can be improved by pre-training the GNN on data from an earlier study. We conclude that the proposed multi-modal GNN framework can provide a novel perspective on the structure-function relationship in the brain. Accordingly this approach appears to be promising for the characterization of the information flow in brain networks.

Generalizable Machine Learning in Neuroscience Using Graph Neural Networks

Although a number of studies have explored deep learning in neuroscience, the application of these algorithms to neural systems on a microscopic scale, i.e. parameters relevant to lower scales of organization, remains relatively novel. Motivated by advances in whole-brain imaging, we examined the performance of deep learning models on microscopic neural dynamics and resulting emergent behaviors using calcium imaging data from the nematode C. elegans. As one of the only species for which neuron-level dynamics can be recorded, C. elegans serves as the ideal organism for designing and testing models bridging recent advances in deep learning and established concepts in neuroscience. We show that neural networks perform remarkably well on both neuron-level dynamics prediction and behavioral state classification. In addition, we compared the performance of structure agnostic neural networks and graph neural networks to investigate if graph structure can be exploited as a favourable inductive bias. To perform this experiment, we designed a graph neural network which explicitly infers relations between neurons from neural activity and leverages the inferred graph structure during computations. In our experiments, we found that graph neural networks generally outperformed structure agnostic models and excel in generalization on unseen organisms, implying a potential path to generalizable machine learning in neuroscience.

Multi-Hops Functional Connectivity Improves Individual Prediction of Fusiform Face Activation via a Graph Neural Network

Brain connectivity plays an important role in determining the brain region’s function. Previous researchers proposed that the brain region’s function is characterized by that region’s input and output connectivity profiles. Following this proposal, numerous studies have investigated the relationship between connectivity and function. However, this proposal only utilizes direct connectivity profiles and thus is deficient in explaining individual differences in the brain region’s function. To overcome this problem, we proposed that a brain region’s function is characterized by that region’s multi-hops connectivity profile. To test this proposal, we used multi-hops functional connectivity to predict the individual face activation of the right fusiform face area (rFFA) via a multi-layer graph neural network and showed that the prediction performance is essentially improved. Results also indicated that the two-layer graph neural network is the best in characterizing rFFA’s face activation and revealed a hierarchical network for the face processing of rFFA.