Hierarchical Spatio-Temporal Modeling of Naturalistic Functional Magnetic Resonance Imaging Signals via Two-Stage Deep Belief Network With Neural Architecture Search

A survey of brain functional network extraction methods using fMRI data

Functional network (FN) analyses play a pivotal role in uncovering insights into brain function and understanding the pathophysiology of various brain disorders. This paper focuses on classical and advanced methods for deriving brain FNs from functional magnetic resonance imaging (fMRI) data. We systematically review their foundational principles, advantages, shortcomings, and interrelations, encompassing both static and dynamic FN extraction approaches. In the context of static FN extraction, we present hypothesis-driven methods such as region of interest (ROI)-based approaches as well as data-driven methods including matrix decomposition, clustering, and deep learning. For dynamic FN extraction, both window-based and windowless methods are surveyed with respect to the estimation of time-varying FN and the subsequent computation of FN states. We also discuss the scope of application of the various methods and avenues for future improvements.

A hybrid spatiotemporal deep belief network and sparse representation-based framework reveals multilevel core functional components in decoding multitask fMRI signals

Decoding human brain activity on various task-based functional brain imaging data is of great significance for uncovering the functioning mechanism of the human mind. Currently, most feature extraction model-based methods for brain state decoding are shallow machine learning models, which may struggle to capture complex and precise spatiotemporal patterns of brain activity from the highly noisy fMRI raw data. Moreover, although decoding models based on deep learning methods benefit from their multilayer structure that could extract spatiotemporal features at multiscale, the relatively large populations of fMRI datasets are indispensable, and the explainability of their results is elusive. To address the above problems, we proposed a computational framework based on hybrid spatiotemporal deep belief network and sparse representations to differentiate multitask fMRI (tfMRI) signals. Using a relatively small cohort of tfMRI data as a test bed, our framework can achieve an average classification accuracy of 97.86% and define the multilevel temporal and spatial patterns of multiple cognitive tasks. Intriguingly, our model can characterize the key components for differentiating the multitask fMRI signals. Overall, the proposed framework can identify the interpretable and discriminative fMRI composition patterns at multiple scales, offering an effective methodology for basic neuroscience and clinical research with relatively small cohorts.

Hierarchical Individual Naturalistic Functional Brain Networks with Group Consistency uncovered by a Two-Stage NAS-Volumetric Sparse DBN Framework

The functional magnetic resonance imaging under naturalistic paradigm (NfMRI) showed great advantages in identifying complex and interactive functional brain networks due to its dynamics and multimodal information. In recent years, various deep learning models, such as deep convolutional autoencoder (DCAE), deep belief network (DBN) and volumetric sparse deep belief network (vsDBN), can obtain hierarchical functional brain networks (FBN) and temporal features from fMRI data. Among them, the vsDBN model revealed a good capability in identifying hierarchical FBNs by modelling fMRI volume images. However, due to the high dimensionality of fMRI volumes and the diverse training parameters of deep learning methods, especially the network architecture that is the most critical parameter for uncovering the hierarchical organization of human brain function, researchers still face challenges in designing an appropriate deep learning framework with automatic network architecture optimization to model volumetric NfMRI. In addition, most of the existing deep learning models ignore the group-wise consistency and inter-subject variation properties embedded in NfMRI volumes. To solve these problems, we proposed a two-stage neural architecture search and vs DBN model (two-stage NAS-vsDBN model) to identify the hierarchical human brain spatio-temporal features possessing both group-consistency and individual-uniqueness under naturalistic condition. Moreover, our model defined reliable network structure for modelling volumetric NfMRI data via NAS framework, and the group-level and individual-level FBNs and associated temporal features exhibited great consistency. In general, our method well identified the hierarchical temporal and spatial features of the brain function and revealed the crucial properties of neural processes under natural viewing condition.Significance StatementIn this paper, we proposed and applied a novel analytical strategy - a two-stage NAS-vsDBN model to identify both group-level and individual-level spatio-temporal features at multi-scales from volumetric NfMRI data. The proposed PSO-based NAS framework can find optimal neural structure for both group-wise and individual-level vs-DBN models. Furthermore, with well-established correspondence between two stages of vsDBN models, our model can effectively detect group-level FBNs that reveal the consistency in neural processes across subjects and individual-level FBNs that maintain the subject specific variability, verifying the inherent property of brain function under naturalistic condition.