Deep learning models of cognitive processes constrained by human brain connectomes

Adversarially Trained Persistent Homology Based Graph Convolutional Network for Disease Identification Using Brain Connectivity

Brain disease propagation is associated with characteristic alterations in the structural and functional connectivity networks of the brain. To identify disease-specific network representations, graph convolutional networks (GCNs) have been used because of their powerful graph embedding ability to characterize the non-Euclidean structure of brain networks. However, existing GCNs generally focus on learning the discriminative region of interest (ROI) features, often ignoring important topological information that enables the integration of connectome patterns of brain activity. In addition, most methods fail to consider the vulnerability of GCNs to perturbations in network properties of the brain, which considerably degrades the reliability of diagnosis results. In this study, we propose an adversarially trained persistent homology-based graph convolutional network (ATPGCN) to capture disease-specific brain connectome patterns and classify brain diseases. First, the brain functional/structural connectivity is constructed using different neuroimaging modalities. Then, we develop a novel strategy that concatenates the persistent homology features from a brain algebraic topology analysis with readout features of the global pooling layer of a GCN model to collaboratively learn the individual-level representation. Finally, we simulate the adversarial perturbations by targeting the risk ROIs from clinical prior, and incorporate them into a training loop to evaluate the robustness of the model. The experimental results on three independent datasets demonstrate that ATPGCN outperforms existing classification methods in disease identification and is robust to minor perturbations in network architecture. Our code is available at https://github.com/CYB08/ATPGCN.

TSP-GNN: a novel neuropsychiatric disorder classification framework based on task-specific prior knowledge and graph neural network

Neuropsychiatric disorder (ND) is often accompanied by abnormal functional connectivity (FC) patterns in specific task contexts. The distinctive task-specific FC patterns can provide valuable features for ND classification models using deep learning. However, most previous studies rely solely on the whole-brain FC matrix without considering the prior knowledge of task-specific FC patterns. Insight by the decoding studies on brain-behavior relationship, we develop TSP-GNN, which extracts task-specific prior (TSP) connectome patterns and employs graph neural network (GNN) for disease classification. TSP-GNN was validated using publicly available datasets. Our results demonstrate that different ND types show distinct task-specific connectivity patterns. Compared with the whole-brain node characteristics, utilizing task-specific nodes enhances the accuracy of ND classification. TSP-GNN comprises the first attempt to incorporate prior task-specific connectome patterns and the power of deep learning. This study elucidates the association between brain dysfunction and specific cognitive processes, offering valuable insights into the cognitive mechanism of neuropsychiatric disease.

Brain decoding of the Human Connectome Project tasks in a dense individual fMRI dataset

Brain decoding aims to infer cognitive states from patterns of brain activity. Substantial inter-individual variations in functional brain organization challenge accurate decoding performed at the group level. In this paper, we tested whether accurate brain decoding models can be trained entirely at the individual level. We trained several classifiers on a dense individual functional magnetic resonance imaging (fMRI) dataset for which six participants completed the entire Human Connectome Project (HCP) task battery >13 times over ten separate fMRI sessions. We evaluated nine decoding methods, from Support Vector Machines (SVM) and Multi-Layer Perceptron (MLP) to Graph Convolutional Neural Networks (GCN). All decoders were trained to classify single fMRI volumes into 21 experimental conditions simultaneously, using ∼7 h of fMRI data per participant. The best prediction accuracies were achieved with GCN and MLP models, whose performance (57-67 % accuracy) approached state-of-the-art accuracy (76 %) with models trained at the group level on >1 K hours of data from the original HCP sample. Our SVM model also performed very well (54-62 % accuracy). Feature importance maps derived from MLP -our best-performing model- revealed informative features in regions relevant to particular cognitive domains, notably in the motor cortex. We also observed that inter-subject classification achieved substantially lower accuracy than subject-specific models, indicating that our decoders learned individual-specific features. This work demonstrates that densely-sampled neuroimaging datasets can be used to train accurate brain decoding models at the individual level. We expect this work to become a useful benchmark for techniques that improve model generalization across multiple subjects and acquisition conditions.

Benchmarking explanation methods for mental state decoding with deep learning models

Deep learning (DL) models find increasing application in mental state decoding, where researchers seek to understand the mapping between mental states (e.g., experiencing anger or joy) and brain activity by identifying those spatial and temporal features of brain activity that allow to accurately identify (i.e., decode) these states. Once a DL model has been trained to accurately decode a set of mental states, neuroimaging researchers often make use of methods from explainable artificial intelligence research to understand the model's learned mappings between mental states and brain activity. Here, we benchmark prominent explanation methods in a mental state decoding analysis of multiple functional Magnetic Resonance Imaging (fMRI) datasets. Our findings demonstrate a gradient between two key characteristics of an explanation in mental state decoding, namely, its faithfulness and its alignment with other empirical evidence on the mapping between brain activity and decoded mental state: explanation methods with high explanation faithfulness, which capture the model's decision process well, generally provide explanations that align less well with other empirical evidence than the explanations of methods with less faithfulness. Based on our findings, we provide guidance for neuroimaging researchers on how to choose an explanation method to gain insight into the mental state decoding decisions of DL models.

Interpreting mental state decoding with deep learning models

In mental state decoding, researchers aim to identify the set of mental states (e.g., experiencing happiness or fear) that can be reliably identified from the activity patterns of a brain region (or network). Deep learning (DL) models are highly promising for mental state decoding because of their unmatched ability to learn versatile representations of complex data. However, their widespread application in mental state decoding is hindered by their lack of interpretability, difficulties in applying them to small datasets, and in ensuring their reproducibility and robustness. We recommend approaching these challenges by leveraging recent advances in explainable artificial intelligence (XAI) and transfer learning, and also provide recommendations on how to improve the reproducibility and robustness of DL models in mental state decoding.