Assessing effects of prenatal alcohol exposure using group-wise sparse representation of fMRI data

An explainable deep learning framework for characterizing and interpreting human brain states

Deep learning approaches have been widely adopted in the medical image analysis field. However, a most of existing deep learning approaches focus on achieving promising performances such as classification, detection, and segmentation, and much less effort is devoted to the explanation of the designed models. Similarly, in the brain imaging field, many deep learning approaches have been designed and applied to characterize and predict human brain states. However, these models lack interpretation. In response, we propose a novel domain knowledge informed self-attention graph pooling-based (SAGPool) graph convolutional neural network to study human brain states. Specifically, the dense individualized and common connectivity-based cortical landmarks system (DICCCOL, structural brain connectivity profiles) and holistic atlases of functional networks and interactions system (HAFNI, functional brain connectivity profiles) are integrated with the SAGPool model to better characterize and interpret the brain states. Extensive experiments are designed and carried out on the large-scale human connectome project (HCP) Q1 and S1200 dataset. Promising brain state classification performances are observed (e.g., an average of 93.7% for seven-task classification and 100% for binary classification). In addition, the importance of the brain regions, which contributes most to the accurate classification, is successfully quantified and visualized. A thorough neuroscientific interpretation suggests that these extracted brain regions and their importance calculated from self-attention graph pooling layer offer substantial explainability.

Modeling spatio-temporal patterns of holistic functional brain networks via multi-head guided attention graph neural networks (Multi-Head GAGNNs)

Mounting evidence has demonstrated that complex brain function processes are realized by the interaction of holistic functional brain networks which are spatially distributed across specific brain regions in a temporally dynamic fashion. Therefore, modeling spatio-temporal patterns of holistic functional brain networks plays an important role in understanding brain function. Compared to traditional modeling methods such as principal component analysis, independent component analysis, and sparse coding, superior performance has been achieved by recent deep learning methodologies. However, there are still two limitations of existing deep learning approaches for functional brain network modeling. They either (1) merely modeled a single targeted network and ignored holistic ones at one time, or (2) underutilized both spatial and temporal features of fMRI during network modeling, and the spatial/temporal accuracy was thus not warranted. To address these limitations, we proposed a novel Multi-Head Guided Attention Graph Neural Network (Multi-Head GAGNN) to simultaneously model both spatial and temporal patterns of holistic functional brain networks. Specifically, a spatial Multi-Head Attention Graph U-Net was first adopted to model the spatial patterns of multiple brain networks, and a temporal Multi-Head Guided Attention Network was then introduced to model the corresponding temporal patterns under the guidance of modeled spatial patterns. Based on seven task fMRI datasets from the public Human Connectome Project and resting state fMRI datasets from the public Autism Brain Imaging Data Exchange I of 1448 subjects, the proposed Multi-Head GAGNN showed superior ability and generalizability in modeling both spatial and temporal patterns of holistic functional brain networks in individual brains compared to other state-of-the-art (SOTA) models. Furthermore, the modeled spatio-temporal patterns of functional brain networks via the proposed Multi-Head GAGNN can better predict the individual cognitive behavioral measures compared to the other SOTA models. This study provided a novel and powerful tool for brain function modeling as well as for understanding the brain-cognitive behavior associations.

The effects of lutein and zeaxanthin on resting state functional connectivity in older Caucasian adults: a randomized controlled trial

The carotenoids lutein (L) and zeaxanthin (Z) accumulate in retinal regions of the eye and have long been shown to benefit visual health. A growing literature suggests cognitive benefits as well, particularly in older adults. The present randomized controlled trial sought to investigate the effects of L and Z on brain function using resting state functional magnetic resonance imaging (fMRI). It was hypothesized that L and Z supplementation would (1) improve intra-network integrity of default mode network (DMN) and (2) reduce inter-network connectivity between DMN and other resting state networks. 48 community-dwelling older adults (mean age = 72 years) were randomly assigned to receive a daily L (10 mg) and Z (2 mg) supplement or a placebo for 1 year. Resting state fMRI data were acquired at baseline and post-intervention. A dictionary learning and sparse coding computational framework, based on machine learning principles, was used to investigate intervention-related changes in functional connectivity. DMN integrity was evaluated by calculating spatial overlap rate with a well-established DMN template provided in the neuroscience literature. Inter-network connectivity was evaluated via time series correlations between DMN and nine other resting state networks. Contrary to expectation, results indicated that L and Z significantly increased rather than decreased inter-network connectivity (Cohen's d = 0.89). A significant intra-network effect on DMN integrity was not observed. Rather than restoring what has been described in the available literature as a "youth-like" pattern of intrinsic brain activity, L and Z may facilitate the aging brain's capacity for compensation by enhancing integration between networks that tend to be functionally segregated earlier in the lifespan.

Differentiable neural architecture search for optimal spatial/temporal brain function network decomposition

It has been a key topic to decompose the brain's spatial/temporal function networks from 4D functional magnetic resonance imaging (fMRI) data. With the advantages of robust and meaningful brain pattern extraction, deep neural networks have been shown to be more powerful and flexible in fMRI data modeling than other traditional methods. However, the challenge of designing neural network architecture for high-dimensional and complex fMRI data has also been realized recently. In this paper, we propose a new spatial/temporal differentiable neural architecture search algorithm (ST-DARTS) for optimal brain network decomposition. The core idea of ST-DARTS is to optimize the inner cell structure of the vanilla recurrent neural network (RNN) in order to effectively decompose spatial/temporal brain function networks from fMRI data. Based on the evaluations on all seven fMRI tasks in human connectome project (HCP) dataset, the ST-DARTS model is shown to perform promisingly, both spatially (i.e., it can recognize the most stimuli-correlated spatial brain network activation that is very similar to the benchmark) and temporally (i.e., its temporal activity is highly positively correlated with the task-design). To further improve the efficiency of ST-DARTS model, we introduce a flexible early-stopping mechanism, named as ST-DARTS+, which further improves experimental results significantly. To our best knowledge, the proposed ST-DARTS and ST-DARTS+ models are among the early efforts in optimally decomposing spatial/temporal brain function networks from fMRI data with neural architecture search strategy and they demonstrate great promise.

Modeling task-based fMRI data via deep belief network with neural architecture search

It has been shown that deep neural networks are powerful and flexible models that can be applied on fMRI data with superb representation ability over traditional methods. However, a challenge of neural network architecture design has also attracted attention: due to the high dimension of fMRI volume images, the manual process of network model design is very time-consuming and not optimal. To tackle this problem, we proposed an unsupervised neural architecture search (NAS) framework on a deep belief network (DBN) that models volumetric fMRI data, named NAS-DBN. The NAS-DBN framework is based on Particle Swarm Optimization (PSO) where the swarms of neural architectures can evolve and converge to a feasible optimal solution. The experiments showed that the proposed NAS-DBN framework can quickly find a robust architecture of DBN, yielding a hierarchy organization of functional brain networks (FBNs) and temporal responses. Compared with 3 manually designed DBNs, the proposed NAS-DBN has the lowest testing loss of 0.0197, suggesting an overall performance improvement of up to 47.9 %. For each task, the NAS-DBN identified 260 FBNs, including task-specific FBNs and resting state networks (RSN), which have high overlap rates to general linear model (GLM) derived templates and independent component analysis (ICA) derived RSN templates. The average overlap rate of NAS-DBN to GLM on 20 task-specific FBNs is as high as 0.536, indicating a performance improvement of up to 63.9 % in respect of network modeling. Besides, we showed that the NAS-DBN can simultaneously generate temporal responses that resemble the task designs very well, and it was observed that widespread overlaps between FBNs from different layers of NAS-DBN model form a hierarchical organization of FBNs. Our NAS-DBN framework contributes an effective, unsupervised NAS method for modeling volumetric task fMRI data.

Automatic recognition of holistic functional brain networks using iteratively optimized convolutional neural networks (IO-CNN) with weak label initialization

fMRI data decomposition techniques have advanced significantly from shallow models such as Independent Component Analysis (ICA) and Sparse Coding and Dictionary Learning (SCDL) to deep learning models such Deep Belief Networks (DBN) and Convolutional Autoencoder (DCAE). However, interpretations of those decomposed networks are still open questions due to the lack of functional brain atlases, no correspondence across decomposed or reconstructed networks across different subjects, and significant individual variabilities. Recent studies showed that deep learning, especially deep convolutional neural networks (CNN), has extraordinary ability of accommodating spatial object patterns, e.g., our recent works using 3D CNN for fMRI-derived network classifications achieved high accuracy with a remarkable tolerance for mistakenly labelled training brain networks. However, the training data preparation is one of the biggest obstacles in these supervised deep learning models for functional brain network map recognitions, since manual labelling requires tedious and time-consuming labours which will sometimes even introduce label mistakes. Especially for mapping functional networks in large scale datasets such as hundreds of thousands of brain networks used in this paper, the manual labelling method will become almost infeasible. In response, in this work, we tackled both the network recognition and training data labelling tasks by proposing a new iteratively optimized deep learning CNN (IO-CNN) framework with an automatic weak label initialization, which enables the functional brain networks recognition task to a fully automatic large-scale classification procedure. Our extensive experiments based on ABIDE-II 1099 brains' fMRI data showed the great promise of our IO-CNN framework.

Joint representation of connectome-scale structural and functional profiles for identification of consistent cortical landmarks in macaque brain

Discovery and representation of common structural and functional cortical architectures has been a significant yet challenging problem for years. Due to the remarkable variability of structural and functional cortical architectures in human brain, it is challenging to jointly represent a common cortical architecture which can comprehensively encode both structure and function characteristics. In order to better understand this challenge and considering that macaque monkey brain has much less variability in structure and function compared with human brain, in this paper, we propose a novel computational framework to apply our DICCCOL (Dense Individualized and Common Connectivity-based Cortical Landmarks) and HAFNI (Holistic Atlases of Functional Networks and Interactions) frameworks on macaque brains, in order to jointly represent structural and functional connectome-scale profiles for identification of a set of consistent and common cortical landmarks across different macaque brains based on multimodal DTI and resting state fMRI (rsfMRI) data. Experimental results demonstrate that 100 consistent and common cortical landmarks are successfully identified via the proposed framework, each of which has reasonably accurate anatomical, structural fiber connection pattern, and functional correspondences across different macaque brains. This set of 100 landmarks offer novel insights into the structural and functional cortical architectures in macaque brains.

Joint representation of consistent structural and functional profiles for identification of common cortical landmarks

In the brain mapping field, there have been significant interests in representation of structural/functional profiles to establish structural/functional landmark correspondences across individuals and populations. For example, from the structural perspective, our previous studies have identified hundreds of consistent DICCCOL (dense individualized and common connectivity-based cortical landmarks) landmarks across individuals and populations, each of which possess consistent DTI-derived fiber connection patterns. From the functional perspective, a large collection of well-characterized HAFNI (holistic atlases of functional networks and interactions) networks based on sparse representation of whole-brain fMRI signals have been identified in our prior studies. However, due to the remarkable variability of structural and functional architectures in the human brain, it is challenging for earlier studies to jointly represent the connectome-scale structural and functional profiles for establishing a common cortical architecture which can comprehensively encode both structural and functional characteristics across individuals. To address this challenge, we propose an effective computational framework to jointly represent the structural and functional profiles for identification of consistent and common cortical landmarks with both structural and functional correspondences across different brains based on DTI and fMRI data. Experimental results demonstrate that 55 structurally and functionally common cortical landmarks can be successfully identified.

A Dictionary Learning Approach for Signal Sampling in Task-Based fMRI for Reduction of Big Data

The exponential growth of fMRI big data offers researchers an unprecedented opportunity to explore functional brain networks. However, this opportunity has not been fully explored yet due to the lack of effective and efficient tools for handling such fMRI big data. One major challenge is that computing capabilities still lag behind the growth of large-scale fMRI databases, e.g., it takes many days to perform dictionary learning and sparse coding of whole-brain fMRI data for an fMRI database of average size. Therefore, how to reduce the data size but without losing important information becomes a more and more pressing issue. To address this problem, we propose a signal sampling approach for significant fMRI data reduction before performing structurally-guided dictionary learning and sparse coding of whole brain's fMRI data. We compared the proposed structurally guided sampling method with no sampling, random sampling and uniform sampling schemes, and experiments on the Human Connectome Project (HCP) task fMRI data demonstrated that the proposed method can achieve more than 15 times speed-up without sacrificing the accuracy in identifying task-evoked functional brain networks.

Sparse coding reveals greater functional connectivity in female brains during naturalistic emotional experience

Functional neuroimaging is widely used to examine changes in brain function associated with age, gender or neuropsychiatric conditions. FMRI (functional magnetic resonance imaging) studies employ either laboratory-designed tasks that engage the brain with abstracted and repeated stimuli, or resting state paradigms with little behavioral constraint. Recently, novel neuroimaging paradigms using naturalistic stimuli are gaining increasing attraction, as they offer an ecologically-valid condition to approximate brain function in real life. Wider application of naturalistic paradigms in exploring individual differences in brain function, however, awaits further advances in statistical methods for modeling dynamic and complex dataset. Here, we developed a novel data-driven strategy that employs group sparse representation to assess gender differences in brain responses during naturalistic emotional experience. Comparing to independent component analysis (ICA), sparse coding algorithm considers the intrinsic sparsity of neural coding and thus could be more suitable in modeling dynamic whole-brain fMRI signals. An online dictionary learning and sparse coding algorithm was applied to the aggregated fMRI signals from both groups, which was subsequently factorized into a common time series signal dictionary matrix and the associated weight coefficient matrix. Our results demonstrate that group sparse representation can effectively identify gender differences in functional brain network during natural viewing, with improved sensitivity and reliability over ICA-based method. Group sparse representation hence offers a superior data-driven strategy for examining brain function during naturalistic conditions, with great potential for clinical application in neuropsychiatric disorders.

Constructing fine-granularity functional brain network atlases via deep convolutional autoencoder

State-of-the-art functional brain network reconstruction methods such as independent component analysis (ICA) or sparse coding of whole-brain fMRI data can effectively infer many thousands of volumetric brain network maps from a large number of human brains. However, due to the variability of individual brain networks and the large scale of such networks needed for statistically meaningful group-level analysis, it is still a challenging and open problem to derive group-wise common networks as network atlases. Inspired by the superior spatial pattern description ability of the deep convolutional neural networks (CNNs), a novel deep 3D convolutional autoencoder (CAE) network is designed here to extract spatial brain network features effectively, based on which an Apache Spark enabled computational framework is developed for fast clustering of larger number of network maps into fine-granularity atlases. To evaluate this framework, 10 resting state networks (RSNs) were manually labeled from the sparsely decomposed networks of Human Connectome Project (HCP) fMRI data and 5275 network training samples were obtained, in total. Then the deep CAE models are trained by these functional networks' spatial maps, and the learned features are used to refine the original 10 RSNs into 17 network atlases that possess fine-granularity functional network patterns. Interestingly, it turned out that some manually mislabeled outliers in training networks can be corrected by the deep CAE derived features. More importantly, fine granularities of networks can be identified and they reveal unique network patterns specific to different brain task states. By further applying this method to a dataset of mild traumatic brain injury study, it shows that the technique can effectively identify abnormal small networks in brain injury patients in comparison with controls. In general, our work presents a promising deep learning and big data analysis solution for modeling functional connectomes, with fine granularities, based on fMRI data.

An Optimized Method for Bayesian Connectivity Change Point Model

The brain undergoes functional dynamic changes at all times. Investigating functional dynamics has been recently verified to be helpful for detecting psychological conditions and powerful for analyzing disease-related abnormalities of the brain. This article aims to detect functional dynamics. Specifically, we focus on how to effectively distinguish corresponding functional connectivity and change points from functional magnetic resonance imaging (fMRI) data. By combining Bayesian connectivity change point model (BCCPM), a modified genetic algorithm (GA) is presented to optimize the evolutionary procedure toward the most probable distributions of real change points in fMRI. We randomly initialize different binary indicator vectors to represent different distributions of change points. Each indicator vector represents an individual in GA, and together they form an initial population. Then we calculate Bayesian posterior probability and use it as the fitness of each individual. Finally, we evolve individuals of current generation toward the next higher fitness generation by a series of modified genetic operators. After several evolutionary procedures, individuals in the final generation may have outstanding fitness and the one with highest fitness can represent the most likely change point distribution in the corresponding fMRI data. Furthermore, the most probable change point distribution could be resolved. We test the optimized method for BCCPM on several synthesized data sets, and the experimental results verify that the proposed model produces higher accuracy results with lower time consumption. Also, we apply the new model to real block-designed task-based fMRI data set and excellent results are obtained.

Radiological studies of fetal alcohol spectrum disorders in humans and animal models: An updated comprehensive review

Fetal Alcohol Spectrum Disorders encompass a wide range of birth defects in children born to mothers who consumed alcohol during pregnancy. Typical mental impairments in FASD include difficulties in life adaptation and learning and memory, deficits in attention, visuospatial skills, language and speech disabilities, mood disorders and motor disabilities. Multimodal imaging methods have enabled in vivo studies of the teratogenic effects of alcohol on the central nervous system, giving more insight into the FASD phenotype. This paper offers an up-to-date comprehensive review of radiological findings in the central nervous system in studies of prenatal alcohol exposure in both humans and translational animal models, including Magnetic Resonance Imaging, Computed Tomography, Positron Emission Tomography, Single Photon Emission Tomography and Ultrasonography.

Automatic Recognition of fMRI-Derived Functional Networks Using 3-D Convolutional Neural Networks

Current functional magnetic resonance imaging (fMRI) data modeling techniques, such as independent component analysis and sparse coding methods, can effectively reconstruct dozens or hundreds of concurrent interacting functional brain networks simultaneously from the whole brain fMRI signals. However, such reconstructed networks have no correspondences across different subjects. Thus, automatic, effective, and accurate classification and recognition of these large numbers of fMRI-derived functional brain networks are very important for subsequent steps of functional brain analysis in cognitive and clinical neuroscience applications. However, this task is still a challenging and open problem due to the tremendous variability of various types of functional brain networks and the presence of various sources of noises. In recognition of the fact that convolutional neural networks (CNN) has superior capability of representing spatial patterns with huge variability and dealing with large noises, in this paper, we design, apply, and evaluate a deep 3-D CNN framework for automatic, effective, and accurate classification and recognition of large number of functional brain networks reconstructed by sparse representation of whole-brain fMRI signals. Our extensive experimental results based on the Human Connectome Project fMRI data showed that the proposed deep 3-D CNN can effectively and robustly perform functional networks classification and recognition tasks, while maintaining a high tolerance for mistakenly labeled training instances. This study provides a new deep learning approach for modeling functional connectomes based on fMRI data.

Task fMRI data analysis based on supervised stochastic coordinate coding

Task functional magnetic resonance imaging (fMRI) has been widely employed for brain activation detection and brain network analysis. Modeling rich information from spatially-organized collection of fMRI time series is challenging because of the intrinsic complexity. Hypothesis-driven methods, such as the general linear model (GLM), which regress exterior stimulus from voxel-wise functional brain activity, are limited due to overlooking the complexity of brain activities and the diversity of concurrent brain networks. Recently, sparse representation and dictionary learning methods have attracted increasing interests in task fMRI data analysis. The major advantage of this methodology is its promise in reconstructing concurrent brain networks systematically. However, this data-driven strategy is, to some extent, arbitrary and does not sufficiently utilize the prior information of task design and neuroscience knowledge. To bridge this gap, we here propose a novel supervised sparse representation and dictionary learning framework based on stochastic coordinate coding (SCC) algorithm for task fMRI data analysis, in which certain brain networks are learned with known information such as pre-defined temporal patterns and spatial network patterns, and at the same time other networks are learned automatically from data. Our proposed method has been applied to two independent task fMRI datasets, and qualitative and quantitative evaluations have shown that our method provides a new and effective framework for task fMRI data analysis.

Assessing the effects of cocaine dependence and pathological gambling using group-wise sparse representation of natural stimulus FMRI data

Assessing functional brain activation patterns in neuropsychiatric disorders such as cocaine dependence (CD) or pathological gambling (PG) under naturalistic stimuli has received rising interest in recent years. In this paper, we propose and apply a novel group-wise sparse representation framework to assess differences in neural responses to naturalistic stimuli across multiple groups of participants (healthy control, cocaine dependence, pathological gambling). Specifically, natural stimulus fMRI (N-fMRI) signals from all three groups of subjects are aggregated into a big data matrix, which is then decomposed into a common signal basis dictionary and associated weight coefficient matrices via an effective online dictionary learning and sparse coding method. The coefficient matrices associated with each common dictionary atom are statistically assessed for each group separately. With the inter-group comparisons based on the group-wise correspondence established by the common dictionary, our experimental results demonstrated that the group-wise sparse coding and representation strategy can effectively and specifically detect brain networks/regions affected by different pathological conditions of the brain under naturalistic stimuli.

Temporal Dynamics Assessment of Spatial Overlap Pattern of Functional Brain Networks Reveals Novel Functional Architecture of Cerebral Cortex

OBJECTIVE

Various studies in the brain mapping field have demonstrated that there exist multiple concurrent functional networks that are spatially overlapped and interacting with each other during specific task performance to jointly realize the total brain function. Assessing such spatial overlap patterns of functional networks (SOPFNs) based on functional magnetic resonance imaging (fMRI) has thus received increasing interest for brain function studies. However, there are still two crucial issues to be addressed. First, the SOPFNs are assessed over the entire fMRI scan assuming the temporal stationarity, while possibly time-dependent dynamics of the SOPFNs is not sufficiently explored. Second, the SOPFNs are assessed within individual subjects, while group-wise consistency of the SOPFNs is largely unknown.

METHODS

To address the two issues, we propose a novel computational framework of group-wise sparse representation of whole-brain fMRI temporal segments to assess the temporal dynamic spatial patterns of SOPFNs that are consistent across different subjects.

RESULTS

Experimental results based on the recently publicly released Human Connectome Project grayordinate task fMRI data demonstrate that meaningful SOPFNs exhibiting dynamic spatial patterns across different time periods are effectively and robustly identified based on the reconstructed concurrent functional networks via the proposed framework. Specifically, those SOPFNs locate significantly more on gyral regions than on sulcal regions across different time periods.

CONCLUSION

These results reveal novel functional architecture of cortical gyri and sulci.

SIGNIFICANCE

Moreover, these results help better understand functional dynamics mechanisms of cerebral cortex in the future.