A Deep Network Model on Dynamic Functional Connectivity With Applications to Gender Classification and Intelligence Prediction

Spatio-temporal learning and explaining for dynamic functional connectivity analysis: Application to depression

BACKGROUND

Functional connectivity has been shown to fluctuate over time. The present study aimed to identifying major depressive disorders (MDD) with dynamic functional connectivity (dFC) from resting-state fMRI data, which would be helpful to produce tools of early depression diagnosis and enhance our understanding of depressive etiology.

METHODS

The resting-state fMRI data of 178 subjects were collected, including 89 MDD and 89 healthy controls. We propose a spatio-temporal learning and explaining framework for dFC analysis. A yet effective spatio-temporal model is developed to classifying MDD from healthy controls with dFCs. The model is a stacking neural network model, which learns network structure information by a multi-layer perceptron based spatial encoder, and learns time-varying patterns by a Transformer based temporal encoder. We propose to explain the spatio-temporal model with a two-stage explanation method of importance feature extracting and disorder-relevant pattern exploring. The layer-wise relevance propagation (LRP) method is introduced to extract the most relevant input features in the model, and the attention mechanism with LRP is applied to extract the important time steps of dFCs. The disorder-relevant functional connections, brain regions, and brain states in the model are further explored and identified.

RESULTS

We achieved the best classification performance in identifying MDD from healthy controls with dFC data. The top important functional connectivity, brain regions, and dynamic states closely related to MDD have been identified.

LIMITATIONS

The data preprocessing may affect the classification performance of the model, and this study needs further validation in a larger patient population.

CONCLUSIONS

The experimental results demonstrate that the proposed spatio-temporal model could effectively classify MDD, and uncover structural and temporal patterns of dFCs in depression.

Measuring functional connectivity in frequency-domain helps to better characterize brain function

Resting-state functional connectivity (FC) is widely used in multivariate pattern analysis of functional magnetic resonance imaging (fMRI), including identifying the locations of putative brain functional borders, predicting individual phenotypes, and diagnosing clinical mental diseases. However, limited attention has been paid to the analysis of functional interactions from a frequency perspective. In this study, by contrasting coherence-based and correlation-based FC with two machine learning tasks, we observed that measuring FC in the frequency domain helped to identify finer functional subregions and achieve better pattern discrimination capability relative to the temporal correlation. This study has proven the feasibility of coherence in the analysis of fMRI, and the results indicate that modeling functional interactions in the frequency domain may provide richer information than that in the time domain, which may provide a new perspective on the analysis of functional neuroimaging.

Rich-club organization of whole-brain spatio-temporal multilayer functional connectivity networks

Objective

In this work, we propose a novel method for constructing whole-brain spatio-temporal multilayer functional connectivity networks (FCNs) and four innovative rich-club metrics.

Methods

Spatio-temporal multilayer FCNs achieve a high-order representation of the spatio-temporal dynamic characteristics of brain networks by combining the sliding time window method with graph theory and hypergraph theory. The four proposed rich-club scales are based on the dynamic changes in rich-club node identity, providing a parameterized description of the topological dynamic characteristics of brain networks from both temporal and spatial perspectives. The proposed method was validated in three independent differential analysis experiments: male-female gender difference analysis, analysis of abnormality in patients with autism spectrum disorders (ASD), and individual difference analysis.

Results

The proposed method yielded results consistent with previous relevant studies and revealed some innovative findings. For instance, the dynamic topological characteristics of specific white matter regions effectively reflected individual differences. The increased abnormality in internal functional connectivity within the basal ganglia may be a contributing factor to the occurrence of repetitive or restrictive behaviors in ASD patients.

Conclusion

The proposed methodology provides an efficacious approach for constructing whole-brain spatio-temporal multilayer FCNs and conducting analysis of their dynamic topological structures. The dynamic topological characteristics of spatio-temporal multilayer FCNs may offer new insights into physiological variations and pathological abnormalities in neuroscience.

Predictability of intelligence and age from structural connectomes

In this study, structural images of 1048 healthy subjects from the Human Connectome Project Young Adult study and 94 from ADNI-3 study were processed by an in-house tractography pipeline and analyzed together with pre-processed data of the same subjects from braingraph.org. Whole brain structural connectome features were used to build a simple correlation-based regression machine learning model to predict intelligence and age of healthy subjects. Our results showed that different forms of intelligence as well as age are predictable to a certain degree from diffusion tensor imaging detecting anatomical fiber tracts in the living human brain. Though we did not identify significant differences in the prediction capability for the investigated features depending on the imaging feature extraction method, we did find that crystallized intelligence was consistently better predictable than fluid intelligence from structural connectivity data through all datasets. Our findings suggest a practical and scalable processing and analysis framework to explore broader research topics employing brain MR imaging.

Multi-level and joint attention networks on brain functional connectivity for cross-cognitive prediction

Deep learning on resting-state functional MRI (rs-fMRI) has shown great success in predicting a single cognition or mental disease. Nevertheless, cognitive functions or mental diseases may share neural mechanisms that can benefit their prediction/classification. We propose a multi-level and joint attention (ML-Joint-Att) network to learn high-order representations of brain functional connectivities that are specific and shared across multiple tasks. We design the ML-Joint-Att network with edge and node convolutional operators, an adaptive inception module, and three attention modules, including network-wise, region-wise, and region-wise joint attention modules. The adaptive inception learns brain functional connectivity at multiple spatial scales. The network-wise and region-wise attention modules take the multi-scale functional connectivities as input and learn features at the network and regional levels for individual tasks. Moreover, the joint attention module is designed as region-wise joint attention to learn shared brain features that contribute to and compensate for the prediction of multiple tasks. We employed the Adolescent Brain Cognitive Development (ABCD) dataset (n =9092) to evaluate the ML-Joint-Att network for the prediction of cognitive flexibility and inhibition. Our experiments demonstrated the usefulness of the three attention modules and identified brain functional connectivities and regions specific and common between cognitive flexibility and inhibition. In particular, the joint attention module can significantly improve the prediction of both cognitive functions. Moreover, leave-one-site cross-validation showed that the ML-Joint-Att network is robust to independent samples obtained from different sites of the ABCD study. Our network outperformed existing machine learning techniques, including Brain Bias Set (BBS), spatio-temporal graph convolution network (ST-GCN), and BrainNetCNN. We demonstrated the generalization of our method to other applications, such as the prediction of fluid intelligence and crystallized intelligence, which also outperformed the ST-GCN and BrainNetCNN.

Accounting for temporal variability in functional magnetic resonance imaging improves prediction of intelligence

Neuroimaging-based prediction methods for intelligence have seen a rapid development. Among different neuroimaging modalities, prediction using functional connectivity (FC) has shown great promise. Most literature has focused on prediction using static FC, with limited investigations on the merits of such analysis compared to prediction using dynamic FC or region-level functional magnetic resonance imaging (fMRI) times series that encode temporal variability. To account for the temporal dynamics in fMRI, we propose a bi-directional long short-term memory (bi-LSTM) approach that incorporates feature selection mechanism. The proposed pipeline is implemented via an efficient algorithm and applied for predicting intelligence using region-level time series and dynamic FC. We compare the prediction performance using different fMRI features acquired from the Adolescent Brain Cognitive Development (ABCD) study involving nearly 7000 individuals. Our detailed analysis illustrates the consistently inferior performance of static FC compared to region-level time series or dynamic FC for single and combined rest and task fMRI experiments. The joint analysis of task and rest fMRI leads to improved intelligence prediction under all models compared to using fMRI from only one experiment. In addition, the proposed bi-LSTM pipeline based on region-level time series identifies several shared and differential important brain regions across fMRI experiments that drive intelligence prediction. A test-retest analysis of the selected regions shows strong reliability across cross-validation folds. Given the large sample size of ABCD study, our results provide strong evidence that superior prediction of intelligence can be achieved by accounting for temporal variations in fMRI.

BolT: Fused window transformers for fMRI time series analysis

Deep-learning models have enabled performance leaps in analysis of high-dimensional functional MRI (fMRI) data. Yet, many previous methods are suboptimally sensitive for contextual representations across diverse time scales. Here, we present BolT, a blood-oxygen-level-dependent transformer model, for analyzing multi-variate fMRI time series. BolT leverages a cascade of transformer encoders equipped with a novel fused window attention mechanism. Encoding is performed on temporally-overlapped windows within the time series to capture local representations. To integrate information temporally, cross-window attention is computed between base tokens in each window and fringe tokens from neighboring windows. To gradually transition from local to global representations, the extent of window overlap and thereby number of fringe tokens are progressively increased across the cascade. Finally, a novel cross-window regularization is employed to align high-level classification features across the time series. Comprehensive experiments on large-scale public datasets demonstrate the superior performance of BolT against state-of-the-art methods. Furthermore, explanatory analyses to identify landmark time points and regions that contribute most significantly to model decisions corroborate prominent neuroscientific findings in the literature.

Static and Dynamic Functional Connectivity Alterations in Alzheimer's Disease and Neuropsychiatric Diseases

To date, numerous studies have documented various alterations in resting brain activity in Alzheimer's disease (AD) and other neuropsychiatric diseases. In particular, disease-related alterations of functional connectivity (FC) in the resting state networks (RSN) have been documented. Altered FC in RSN is useful not only for interpreting the phenotype of diseases but also for diagnosing the diseases. More recently, several studies proposed the dynamics of resting-brain activity as a useful marker for detecting altered RSNs related to AD and other diseases. In contrast to previous studies, which focused on FC calculated using an entire fMRI scan (static FC), these newer studies focused the on temporal dynamics of FC within the scan (dynamic FC) to provide more sensitive measures to characterize RSNs. However, despite the increasing popularity of dFC, several studies cautioned that the results obtained in commonly used analyses for dFC require careful interpretation. In this mini-review, we review recent studies exploring alterations of static and dynamic functional connectivity in AD and other neuropsychiatric diseases. We then discuss how to utilize and interpret dFC for studying resting brain activity in diseases.

A deep graph neural network architecture for modelling spatio-temporal dynamics in resting-state functional MRI data

Resting-state functional magnetic resonance imaging (rs-fMRI) has been successfully employed to understand the organisation of the human brain. Typically, the brain is parcellated into regions of interest (ROIs) and modelled as a graph where each ROI represents a node and association measures between ROI-specific blood-oxygen-level-dependent (BOLD) time series are edges. Recently, graph neural networks (GNNs) have seen a surge in popularity due to their success in modelling unstructured relational data. The latest developments with GNNs, however, have not yet been fully exploited for the analysis of rs-fMRI data, particularly with regards to its spatio-temporal dynamics. In this paper, we present a novel deep neural network architecture which combines both GNNs and temporal convolutional networks (TCNs) in order to learn from both the spatial and temporal components of rs-fMRI data in an end-to-end fashion. In particular, this corresponds to intra-feature learning (i.e., learning temporal dynamics with TCNs) as well as inter-feature learning (i.e., leveraging interactions between ROI-wise dynamics with GNNs). We evaluate our model with an ablation study using 35,159 samples from the UK Biobank rs-fMRI database, as well as in the smaller Human Connectome Project (HCP) dataset, both in a unimodal and in a multimodal fashion. We also demonstrate that out architecture contains explainability-related features which easily map to realistic neurobiological insights. We suggest that this model could lay the groundwork for future deep learning architectures focused on leveraging the inherently and inextricably spatio-temporal nature of rs-fMRI data.

DC-tCNN: A Deep Model for EEG-based Detection of Dim Targets

OBJECTIVE

Dim target detection in remote sensing images is a significant and challenging problem. In this work, we seek to explore event-related brain responses of dim target detection tasks and extend the brain-computer interface (BCI) systems to this task for efficiency enhancement.

METHODS

We develop a BCI paradigm named Asynchronous Visual Evoked Paradigm (AVEP), in which subjects are required to search the dim targets within satellite images when their scalp electroencephalography (EEG) signals are simultaneously recorded. In the paradigm, stimulus onset time and target onset time are asynchronous because subjects need enough time to confirm whether there are targets of interest in the presented serial images. We further propose a Domain adaptive and Channel-wise attention-based Time-domain Convolutional Neural Network (DC-tCNN) to solve the single-trial EEG classification problem for the AVEP task. In this model, we design a multi-scale CNN module combined with a channel-wise attention module to effectively extract event-related brain responses underlying EEG signals. Meanwhile, domain adaptation is proposed to mitigate cross-subject distribution discrepancy.

RESULTS

The results demonstrate the superior performance and better generalizability of this model in classifying the single-trial EEG data of AVEP task in contrast to typical EEG deep learning networks. Visualization analyses of spatiotemporal features also illustrate the effectiveness and interpretability of our proposed paradigm and learning model.

CONCLUSION

The proposed paradigm and model can effectively explore ambiguous event-related brain responses on EEG-based dim target detection tasks.

SIGNIFICANCE

Our work can provide a valuable reference for BCI-based image detection of dim targets.

Multi-Source Domain Adaptation Techniques for Mitigating Batch Effects: A Comparative Study

The past decade has seen an increasing number of applications of deep learning (DL) techniques to biomedical fields, especially in neuroimaging-based analysis. Such DL-based methods are generally data-intensive and require a large number of training instances, which might be infeasible to acquire from a single acquisition site, especially for data, such as fMRI scans, due to the time and costs that they demand. We can attempt to address this issue by combining fMRI data from various sites, thereby creating a bigger heterogeneous dataset. Unfortunately, the inherent differences in the combined data, known as batch effects, often hamper learning a model. To mitigate this issue, techniques such as multi-source domain adaptation [Multi-source Domain Adversarial Networks (MSDA)] aim at learning an effective classification function that uses (learned) domain-invariant latent features. This article analyzes and compares the performance of various popular MSDA methods [MDAN, Domain AggRegation Networks (DARN), Multi-Domain Matching Networks (MDMN), and Moment Matching for MSDA (M3SDA)] at predicting different labels (illness, age, and sex) of images from two public rs-fMRI datasets: ABIDE 1and ADHD-200. It also evaluates the impact of various conditions such as class imbalance, the number of sites along with a comparison of the degree of adaptation of each of the methods, thereby presenting the effectiveness of MSDA models in neuroimaging-based applications.

ConCeptCNN: A novel multi-filter convolutional neural network for the prediction of neurodevelopmental disorders using brain connectome

BACKGROUND

Deep convolutional neural network (CNN) and its derivatives have recently shown great promise in the prediction of brain disorders using brain connectome data. Existing deep CNN methods using single global row and column convolutional filters have limited ability to extract discriminative information from brain connectome for prediction tasks.

PURPOSE

This paper presents a novel deep Connectome-Inception CNN (ConCeptCNN) model, which is developed based on multiple convolutional filters. The proposed model is used to extract topological features from brain connectome data for neurological disorders classification and analysis.

METHODS

The ConCeptCNN uses multiple vector-shaped filters extract topological information from the brain connectome at different levels for complementary feature embeddings of brain connectome. The proposed model is validated using two datasets: the Neuro Bureau ADHD-200 dataset and the Cincinnati Early Prediction Study (CINEPS) dataset.

RESULTS

In a cross-validation experiment, the ConCeptCNN achieved a prediction accuracy of 78.7% for the detection of ADHD in adolescents and an accuracy of 81.6% for the prediction of cognitive deficits at 2-year corrected age in very preterm infants. In addition to the classification tasks, the ConCeptCNN identified several brain regions that are discriminative to neurodevelopmental disorders.

CONCLUSIONS

We compared the ConCeptCNN with several peer CNN methods. The results demonstrated that proposed model improves overall classification performance of neurodevelopmental disorders prediction tasks. This article is protected by copyright. All rights reserved.

Feeding the machine: Challenges to reproducible predictive modeling in resting-state connectomics

In this critical review, we examine the application of predictive models, for example, classifiers, trained using machine learning (ML) to assist in interpretation of functional neuroimaging data. Our primary goal is to summarize how ML is being applied and critically assess common practices. Our review covers 250 studies published using ML and resting-state functional MRI (fMRI) to infer various dimensions of the human functional connectome. Results for holdout ("lockbox") performance was, on average, ∼13% less accurate than performance measured through cross-validation alone, highlighting the importance of lockbox data, which was included in only 16% of the studies. There was also a concerning lack of transparency across the key steps in training and evaluating predictive models. The summary of this literature underscores the importance of the use of a lockbox and highlights several methodological pitfalls that can be addressed by the imaging community. We argue that, ideally, studies are motivated both by the reproducibility and generalizability of findings as well as the potential clinical significance of the insights. We offer recommendations for principled integration of machine learning into the clinical neurosciences with the goal of advancing imaging biomarkers of brain disorders, understanding causative determinants for health risks, and parsing heterogeneous patient outcomes.

Representation learning of resting state fMRI with variational autoencoder

Resting state functional magnetic resonance imaging (rsfMRI) data exhibits complex but structured patterns. However, the underlying origins are unclear and entangled in rsfMRI data. Here we establish a variational auto-encoder, as a generative model trainable with unsupervised learning, to disentangle the unknown sources of rsfMRI activity. After being trained with large data from the Human Connectome Project, the model has learned to represent and generate patterns of cortical activity and connectivity using latent variables. The latent representation and its trajectory represent the spatiotemporal characteristics of rsfMRI activity. The latent variables reflect the principal gradients of the latent trajectory and drive activity changes in cortical networks. Representational geometry captured as covariance or correlation between latent variables, rather than cortical connectivity, can be used as a more reliable feature to accurately identify subjects from a large group, even if only a short period of data is available in each subject. Our results demonstrate that VAE is a valuable addition to existing tools, particularly suited for unsupervised representation learning of resting state fMRI activity.

Wiring of higher-order cortical areas: Spatiotemporal development of cortical hierarchy

A hierarchical development of cortical areas was suggested over a century ago, but the diversity and complexity of cortical hierarchy properties have so far prevented a formal demonstration. The aim of this review is to clarify the similarities and differences in the developmental processes underlying cortical development of primary and higher-order areas. We start by recapitulating the historical and recent advances underlying the biological principle of cortical hierarchy in adults. We then revisit the arguments for a hierarchical maturation of cortical areas, and further integrate the principles of cortical areas specification during embryonic and postnatal development. We highlight how the dramatic expansion in cortical size might have contributed to the increased number of association areas sustaining cognitive complexification in evolution. Finally, we summarize the recent observations of an alteration of cortical hierarchy in neuropsychiatric disorders and discuss their potential developmental origins.

Tracking whole-brain connectivity dynamics in the resting-state fMRI with post-facial paralysis synkinesis

BACKGROUND

Resting-state functional magnetic resonance imaging (rs-fMRI) is widely applied to explore abnormal functional connectivity (FC) in patients with post-facial paralysis synkinesis (PFPS). However, most studies considered steady spatial-temporal signal interactions between distinct brain regions during the period of scanning.

OBJECTIVE

In this study, we aim to investigate abnormal dynamic functional connectivity (dFC) in PFPS patients.

METHODS

We enrolled 31 PFPS patients and 19 healthy controls. All participants underwent rs-fMRI. Sliding windows approach was applied to construct dFC matrices. Next, these matrices were clustered into distinct states using the k-means clustering algorithm.

RESULTS

We found that it was not the dFC patterns, but rather the temporal properties including the mean dwell time (MDT) and occurrence frequencies, that showed a significant difference between PFPS patients and healthy controls. Two randomly clustered dFC states were recognized for both groups. Among them, State 1 showed significantly lower connectivity compared to State 2 in patients group. Compared to healthy controls, the duration spent by the PFPS patients in the state with lower connectivity significantly increased and is positively correlated with the better facial function.

CONCLUSIONS

In conclusion, aberrant dFC is a fundamental feature of brain dysfunction in PFPS patients, which is associated with the facial nerve function. These findings may contribute to a better understanding of the abnormal brain reorganization mechanisms in PFPS patients.