1
|
Liu Y, Ge E, He M, Liu Z, Zhao S, Hu X, Qiang N, Zhu D, Liu T, Ge B. Mapping dynamic spatial patterns of brain function with spatial-wise attention. J Neural Eng 2024; 21:026005. [PMID: 38407988 DOI: 10.1088/1741-2552/ad2cea] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2023] [Accepted: 02/02/2024] [Indexed: 02/28/2024]
Abstract
Objective: Using functional magnetic resonance imaging (fMRI) and deep learning to discover the spatial pattern of brain function, or functional brain networks (FBNs) has been attracted many reseachers. Most existing works focus on static FBNs or dynamic functional connectivity among fixed spatial network nodes, but ignore the potential dynamic/time-varying characteristics of the spatial networks themselves. And most of works based on the assumption of linearity and independence, that oversimplify the relationship between blood-oxygen level dependence signal changes and the heterogeneity of neuronal activity within voxels.Approach: To overcome these problems, we proposed a novel spatial-wise attention (SA) based method called Spatial and Channel-wise Attention Autoencoder (SCAAE) to discover the dynamic FBNs without the assumptions of linearity or independence. The core idea of SCAAE is to apply the SA to generate FBNs directly, relying solely on the spatial information present in fMRI volumes. Specifically, we trained the SCAAE in a self-supervised manner, using the autoencoder to guide the SA to focus on the activation regions. Experimental results show that the SA can generate multiple meaningful FBNs at each fMRI time point, which spatial similarity are close to the FBNs derived by known classical methods, such as independent component analysis.Main results: To validate the generalization of the method, we evaluate the approach on HCP-rest, HCP-task and ADHD-200 dataset. The results demonstrate that SA mechanism can be used to discover time-varying FBNs, and the identified dynamic FBNs over time clearly show the process of time-varying spatial patterns fading in and out.Significance: Thus we provide a novel method to understand human brain better. Code is available athttps://github.com/WhatAboutMyStar/SCAAE.
Collapse
Affiliation(s)
- Yiheng Liu
- School of Physics & Information Technology, Shaanxi Normal University, Xi'an, People's Republic of China
- Key Laboratory of Modern Teaching Technology, Ministry of Education, Shaanxi Normal University, Xi'an, People's Republic of China
| | - Enjie Ge
- School of Physics & Information Technology, Shaanxi Normal University, Xi'an, People's Republic of China
| | - Mengshen He
- School of Physics & Information Technology, Shaanxi Normal University, Xi'an, People's Republic of China
| | - Zhengliang Liu
- School of Computing, University of Georgia, Athens, GA, United States of America
| | - Shijie Zhao
- Shenzhen Research Institute of Northwestern Polytechnical University, Shenzhen, People's Republic of China
| | - Xintao Hu
- School of Automation, Northwestern Polytechnical University, Xi'an, People's Republic of China
| | - Ning Qiang
- School of Physics & Information Technology, Shaanxi Normal University, Xi'an, People's Republic of China
| | - Dajiang Zhu
- Department of Computer Science, University of Texas at Arlington, Arlington, TX, United States of America
| | - Tianming Liu
- School of Computing, University of Georgia, Athens, GA, United States of America
| | - Bao Ge
- School of Physics & Information Technology, Shaanxi Normal University, Xi'an, People's Republic of China
- Key Laboratory of Modern Teaching Technology, Ministry of Education, Shaanxi Normal University, Xi'an, People's Republic of China
| |
Collapse
|
2
|
Pan H, Mao Y, Liu P, Li Y, Wei G, Qiao X, Ren Y, Zhao F. Extracting transition features among brain states based on coarse-grained similarity measurement for autism spectrum disorder analysis. Med Phys 2023; 50:6269-6282. [PMID: 36995984 DOI: 10.1002/mp.16406] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Revised: 03/13/2023] [Accepted: 03/23/2023] [Indexed: 03/31/2023] Open
Abstract
BACKGROUND The abnormal brain functional connectivity (FC) of patients with mental diseases is closely linked to the transition features among brain states. However, the current research on state transition will produce certain division deviations in the measurement method of state division, and also ignore the transition features among multiple states that contain more abundant information for analyzing brain diseases. PURPOSE To investigate the potential of the proposed method based on coarse-grained similarity measurement to solve the problem of state division, and consider the transition features among multiple states to analyze the FC abnormalities of autism spectrum disorder (ASD) patients. METHODS We used resting-state functional magnetic resonance imaging to examine 45 ASD and 47 healthy controls (HC). The FC between brain regions was calculated by the sliding window and correlation algorithm, and a novel coarse-grained similarity measure method was used to cluster the FC networks into five states, and then extract the features both of the state itself and the transition features among multiple states for analysis and diagnosis. RESULTS (1) The state as divided by the coarse-grained measurement method improves the diagnostic performance of individuals with ASD compared with previous methods. (2) The transition features among multiple states can provide complementary information to the features of the state itself in the ASD analysis and diagnosis. (3) ASD individuals have different brain state transitions than HC. Specifically, the abnormalities in intra- and inter-network connectivity of ASD patients mainly occur in the default mode network, the visual network, and the cerebellum. CONCLUSIONS Such results demonstrate that our approach with new measurements and new features is effective and promising in brain state analysis and ASD diagnosis.
Collapse
Affiliation(s)
- Hongxin Pan
- School of Computer Science and Technology, Shandong Technology and Business University, Yantai, China
| | - Yanyan Mao
- School of Computer Science and Technology, Shandong Technology and Business University, Yantai, China
| | - Peiqiang Liu
- School of Computer Science and Technology, Shandong Technology and Business University, Yantai, China
| | - Yuan Li
- School of Management Science and Engineering, Shandong Technology and Business University, Yantai, China
| | - Guanglan Wei
- Information Network Center, Shandong Second Provincial General Hospital, Jinan, China
| | - Xiaoyan Qiao
- School of Mathematics and Information Science, Shandong Technology and Business University, Yantai, China
| | - Yande Ren
- Department of Radiology, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China
| | - Feng Zhao
- School of Computer Science and Technology, Shandong Technology and Business University, Yantai, China
| |
Collapse
|
3
|
Zhao L, Wu Z, Dai H, Liu Z, Hu X, Zhang T, Zhu D, Liu T. A generic framework for embedding human brain function with temporally correlated autoencoder. Med Image Anal 2023; 89:102892. [PMID: 37482031 DOI: 10.1016/j.media.2023.102892] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Revised: 03/19/2023] [Accepted: 07/06/2023] [Indexed: 07/25/2023]
Abstract
Learning an effective and compact representation of human brain function from high-dimensional fMRI data is crucial for studying the brain's functional organization. Traditional representation methods such as independent component analysis (ICA) and sparse dictionary learning (SDL) mainly rely on matrix decomposition which represents the brain function as spatial brain networks and the corresponding temporal patterns. The correspondence of those brain networks across individuals are built by viewing them as one-hot vectors and then performing the matching. However, those one-hot vectors do not encode the regularity and/or variability of different brains very well, and thus are limited in effectively representing the functional brain activities across individuals and among different time points. To address this problem, in this paper, we formulate the human brain functional representation as an embedding problem, and propose a novel embedding framework based on the Transformer model to encode the brain function in a compact, stereotyped and comparable latent space where the brain activities are represented as dense embedding vectors. We evaluate the proposed embedding framework on the publicly available Human Connectome Project (HCP) task fMRI dataset. The experiments on brain state prediction task indicate the effectiveness and generalizability of the learned embedding. We also explore the interpretability of the learned embedding from both spatial and temporal perspective. In general, our approach provides novel insights on representing the regularity and variability of human brain function in a general, comparable, and stereotyped latent space.
Collapse
Affiliation(s)
- Lin Zhao
- School of Computing, The University of Georgia, Athens 30602, USA
| | - Zihao Wu
- School of Computing, The University of Georgia, Athens 30602, USA
| | - Haixing Dai
- School of Computing, The University of Georgia, Athens 30602, USA
| | - Zhengliang Liu
- School of Computing, The University of Georgia, Athens 30602, USA
| | - Xintao Hu
- School of Automation, Northwestern Polytechnical University, Xi'an 710072, China
| | - Tuo Zhang
- School of Automation, Northwestern Polytechnical University, Xi'an 710072, China
| | - Dajiang Zhu
- Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington 76019, USA.
| | - Tianming Liu
- School of Computing, The University of Georgia, Athens 30602, USA.
| |
Collapse
|
4
|
Gaga R. Editorial for "Evaluation of Spatial Attentive Deep Learning for Automatic Placental Segmentation on Longitudinal MRI". J Magn Reson Imaging 2022; 57:1541-1542. [PMID: 35979891 DOI: 10.1002/jmri.28401] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Accepted: 07/08/2022] [Indexed: 11/11/2022] Open
Affiliation(s)
- Remus Gaga
- 2nd Pediatric Clinic, Clinical Emergency Hospital for Children, Cluj-Napoca, Cluj, Romania
| |
Collapse
|
5
|
Decoding Task-Based fMRI Data with Graph Neural Networks, Considering Individual Differences. Brain Sci 2022; 12:brainsci12081094. [PMID: 36009157 PMCID: PMC9405908 DOI: 10.3390/brainsci12081094] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2022] [Revised: 08/01/2022] [Accepted: 08/06/2022] [Indexed: 12/05/2022] Open
Abstract
Task fMRI provides an opportunity to analyze the working mechanisms of the human brain during specific experimental paradigms. Deep learning models have increasingly been applied for decoding and encoding purposes study to representations in task fMRI data. More recently, graph neural networks, or neural networks models designed to leverage the properties of graph representations, have recently shown promise in task fMRI decoding studies. Here, we propose an end-to-end graph convolutional network (GCN) framework with three convolutional layers to classify task fMRI data from the Human Connectome Project dataset. We compared the predictive performance of our GCN model across four of the most widely used node embedding algorithms—NetMF, RandNE, Node2Vec, and Walklets—to automatically extract the structural properties of the nodes in the functional graph. The empirical results indicated that our GCN framework accurately predicted individual differences (0.978 and 0.976) with the NetMF and RandNE embedding methods, respectively. Furthermore, to assess the effects of individual differences, we tested the classification performance of the model on sub-datasets divided according to gender and fluid intelligence. Experimental results indicated significant differences in the classification predictions of gender, but not high/low fluid intelligence fMRI data. Our experiments yielded promising results and demonstrated the superior ability of our GCN in modeling task fMRI data.
Collapse
|
6
|
Karakasis PA, Liavas AP, Sidiropoulos ND, Simos PG, Papadaki E. Multisubject Task-Related fMRI Data Processing via a Two-Stage Generalized Canonical Correlation Analysis. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2022; 31:4011-4022. [PMID: 35588408 DOI: 10.1109/tip.2022.3159125] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Functional magnetic resonance imaging (fMRI) is one of the most popular methods for studying the human brain. Task-related fMRI data processing aims to determine which brain areas are activated when a specific task is performed and is usually based on the Blood Oxygen Level Dependent (BOLD) signal. The background BOLD signal also reflects systematic fluctuations in regional brain activity which are attributed to the existence of resting-state brain networks. We propose a new fMRI data generating model which takes into consideration the existence of common task-related and resting-state components. We first estimate the common task-related temporal component, via two successive stages of generalized canonical correlation analysis and, then, we estimate the common task-related spatial component, leading to a task-related activation map. The experimental tests of our method with synthetic data reveal that we are able to obtain very accurate temporal and spatial estimates even at very low Signal to Noise Ratio (SNR), which is usually the case in fMRI data processing. The tests with real-world fMRI data show significant advantages over standard procedures based on General Linear Models (GLMs).
Collapse
|
7
|
Ren Y, Xu S, Tao Z, Song L, He X. Hierarchical Spatio-Temporal Modeling of Naturalistic Functional Magnetic Resonance Imaging Signals via Two-Stage Deep Belief Network With Neural Architecture Search. Front Neurosci 2021; 15:794955. [PMID: 34955738 PMCID: PMC8692564 DOI: 10.3389/fnins.2021.794955] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2021] [Accepted: 11/19/2021] [Indexed: 11/28/2022] Open
Abstract
Naturalistic functional magnetic resonance imaging (NfMRI) has become an effective tool to study brain functional activities in real-life context, which reduces the anxiety or boredom due to difficult or repetitive tasks and avoids the problem of unreliable collection of brain activity caused by the subjects’ microsleeps during resting state. Recent studies have made efforts on characterizing the brain’s hierarchical organizations from fMRI data by various deep learning models. However, most of those models have ignored the properties of group-wise consistency and inter-subject difference in brain function under naturalistic paradigm. Another critical issue is how to determine the optimal neural architecture of deep learning models, as manual design of neural architecture is time-consuming and less reliable. To tackle these problems, we proposed a two-stage deep belief network (DBN) with neural architecture search (NAS) combined framework (two-stage NAS-DBN) to model both the group-consistent and individual-specific naturalistic functional brain networks (FBNs), which reflected the hierarchical organization of brain function and the nature of brain functional activities under naturalistic paradigm. Moreover, the test-retest reliability and spatial overlap rate of the FBNs identified by our model reveal better performance than that of widely used traditional methods. In general, our model provides a promising method for characterizing hierarchical spatiotemporal features under the natural paradigm.
Collapse
Affiliation(s)
- Yudan Ren
- School of Information Science and Technology, Northwest University, Xi'an, China
| | - Shuhan Xu
- School of Information Science and Technology, Northwest University, Xi'an, China
| | - Zeyang Tao
- School of Information Science and Technology, Northwest University, Xi'an, China
| | - Limei Song
- School of Information Science and Technology, Northwest University, Xi'an, China
| | - Xiaowei He
- School of Information Science and Technology, Northwest University, Xi'an, China
| |
Collapse
|