Characterizing functional brain networks via Spatio-Temporal Attention 4D Convolutional Neural Networks (STA-4DCNNs)

A survey of brain functional network extraction methods using fMRI data

Functional network (FN) analyses play a pivotal role in uncovering insights into brain function and understanding the pathophysiology of various brain disorders. This paper focuses on classical and advanced methods for deriving brain FNs from functional magnetic resonance imaging (fMRI) data. We systematically review their foundational principles, advantages, shortcomings, and interrelations, encompassing both static and dynamic FN extraction approaches. In the context of static FN extraction, we present hypothesis-driven methods such as region of interest (ROI)-based approaches as well as data-driven methods including matrix decomposition, clustering, and deep learning. For dynamic FN extraction, both window-based and windowless methods are surveyed with respect to the estimation of time-varying FN and the subsequent computation of FN states. We also discuss the scope of application of the various methods and avenues for future improvements.

fMRI-based spatio-temporal parcellations of the human brain

PURPOSE OF REVIEW

Human brain parcellation based on functional magnetic resonance imaging (fMRI) plays an essential role in neuroscience research. By segmenting vast and intricate fMRI data into functionally similar units, researchers can better decipher the brain's structure in both healthy and diseased states. This article reviews current methodologies and ideas in this field, while also outlining the obstacles and directions for future research.

RECENT FINDINGS

Traditional brain parcellation techniques, which often rely on cytoarchitectonic criteria, overlook the functional and temporal information accessible through fMRI. The adoption of machine learning techniques, notably deep learning, offers the potential to harness both spatial and temporal information for more nuanced brain segmentation. However, the search for a one-size-fits-all solution to brain segmentation is impractical, with the choice between group-level or individual-level models and the intended downstream analysis influencing the optimal parcellation strategy. Additionally, evaluating these models is complicated by our incomplete understanding of brain function and the absence of a definitive "ground truth".

SUMMARY

While recent methodological advancements have significantly enhanced our grasp of the brain's spatial and temporal dynamics, challenges persist in advancing fMRI-based spatio-temporal representations. Future efforts will likely focus on refining model evaluation and selection as well as developing methods that offer clear interpretability for clinical usage, thereby facilitating further breakthroughs in our comprehension of the brain.

ADHD diagnosis guided by functional brain networks combined with domain knowledge

Utilizing functional magnetic resonance imaging (fMRI) to model functional brain networks (FBNs) is increasingly prominent in attention-deficit/hyperactivity disorder (ADHD) research, revealing neural impact and mechanisms through the exploration of activated brain regions. However, current FBNs-based methods face two major challenges. The primary challenge stems from the limitations of existing modeling methods in accurately capturing both regional correlations and long-distance dependencies (LDDs) within the dynamic brain, thereby affecting the diagnostic accuracy of FBNs as biomarkers. Additionally, limited sample size and class imbalance also pose a challenge to the learning performance of the model. To address the issues, we propose an automated diagnostic framework, which integrates modeling, multimodal fusion, and classification into a unified process. It aims to extract representative FBNs and efficiently incorporate domain knowledge to guide ADHD classification. Our work mainly includes three-fold: 1) A multi-head attention-based region-enhancement module (MAREM) is designed to simultaneously capture regional correlations and LDDs across the entire sequence of brain activity, which facilitates the construction of representative FBNs. 2) The multimodal supplementary learning module (MSLM) is proposed to integrate domain knowledge from phenotype data with FBNs from neuroimaging data, achieving information complementarity and alleviating the problems of insufficient medical data and unbalanced sample categories. 3) An ADHD automatic diagnosis framework guided by FBNs and domain knowledge (ADF-FAD) is proposed to help doctors make more accurate decisions, which is applied to the ADHD-200 dataset to confirm its effectiveness. The results indicate that the FBNs extracted by MAREM perform well in modeling and classification. After with MSLM, the model achieves accuracy of 92.4%, 74.4%, and 80% at NYU, PU, and KKI, respectively, demonstrating its ability to effectively capture crucial information related to ADHD diagnosis. Codes are available at https://github.com/zhuimengxuebao/ADF-FAD.

A hybrid CNN-SVM model for enhanced autism diagnosis

Autism is a representative disorder of pervasive developmental disorder. It exerts influence upon an individual's behavior and performance, potentially co-occurring with other mental illnesses. Consequently, an effective diagnostic approach proves to be invaluable in both therapeutic interventions and the timely provision of medical support. Currently, most scholars' research primarily relies on neuroimaging techniques for auxiliary diagnosis and does not take into account the distinctive features of autism's social impediments. In order to address this deficiency, this paper introduces a novel convolutional neural network-support vector machine model that integrates resting state functional magnetic resonance imaging data with the social responsiveness scale metrics for the diagnostic assessment of autism. We selected 821 subjects containing the social responsiveness scale measure from the publicly available Autism Brain Imaging Data Exchange dataset, including 379 subjects with autism spectrum disorder and 442 typical controls. After preprocessing of fMRI data, we compute the static and dynamic functional connectivity for each subject. Subsequently, convolutional neural networks and attention mechanisms are utilized to extracts their respective features. The extracted features, combined with the social responsiveness scale features, are then employed as novel inputs for the support vector machine to categorize autistic patients and typical controls. The proposed model identifies salient features within the static and dynamic functional connectivity, offering a possible biological foundation for clinical diagnosis. By incorporating the behavioral assessments, the model achieves a remarkable classification accuracy of 94.30%, providing a more reliable support for auxiliary diagnosis.

Brain Structural Connectivity Guided Vision Transformers for Identification of Functional Connectivity Characteristics in Preterm Neonates

Preterm birth is the leading cause of death in children under five years old, and is associated with a wide sequence of complications in both short and long term. In view of rapid neurodevelopment during the neonatal period, preterm neonates may exhibit considerable functional alterations compared to term ones. However, the identified functional alterations in previous studies merely achieve moderate classification performance, while more accurate functional characteristics with satisfying discrimination ability for better diagnosis and therapeutic treatment is underexplored. To address this problem, we propose a novel brain structural connectivity (SC) guided Vision Transformer (SCG-ViT) to identify functional connectivity (FC) differences among three neonatal groups: preterm, preterm with early postnatal experience, and term. Particularly, inspired by the neuroscience-derived information, a novel patch token of SC/FC matrix is defined, and the SC matrix is then adopted as an effective mask into the ViT model to screen out input FC patch embeddings with weaker SC, and to focus on stronger ones for better classification and identification of FC differences among the three groups. The experimental results on multi-modal MRI data of 437 neonatal brains from publicly released Developing Human Connectome Project (dHCP) demonstrate that SCG-ViT achieves superior classification ability compared to baseline models, and successfully identifies holistically different FC patterns among the three groups. Moreover, these different FCs are significantly correlated with the differential gene expressions of the three groups. In summary, SCG-ViT provides a powerfully brain-guided pipeline of adopting large-scale and data-intensive deep learning models for medical imaging-based diagnosis.

Modeling default mode network patterns via a universal spatio-temporal brain attention skip network

Designing a comprehensive four-dimensional resting-state functional magnetic resonance imaging (4D Rs-fMRI) based default mode network (DMN) modeling methodology to reveal the spatio-temporal patterns of individual DMN, is crucial for understanding the cognitive mechanisms of the brain and the pathogenesis of psychiatric disorders. However, there are still two limitations of existing approaches for DMN modeling. The approaches either (1) simply split the spatio-temporal components and ignore the overall character of the spatio-temporal patterns or (2) are biased in the process of feature extraction for DMN modeling, and their spatio-temporal accuracy is thus not warranted. To this end, we propose a novel Spatio-Temporal Brain Attention Skip Network (STBAS-Net) to model the personalized spatio-temporal patterns of the DMN. STBAS-Net consists of spatial and temporal components, where the multi-head attention skip connection block in the spatial component achieves detailed feature extraction and enhancement in the shallow stage. Under the guidance of spatial information, we technically fuse multiple spatio-temporal information in the temporal component, which dexterously exploits the overall spatio-temporal features and achieves mutual constraints of spatio-temporal patterns to characterize the spatio-temporal patterns of the DMN. We verify the proposed STBAS-Net on a publicly released 4D Rs-fMRI dataset and an EMCI dataset. The experimental results show that compared with existing advanced methods, the proposed network can more accurately model the personalized spatio-temporal patterns of the human brain DMN and successfully identify abnormal spatio-temporal patterns in EMCI patients. This study provides a potential tool for revealing the spatio-temporal patterns of the human brain DMN and is expected to provide an effective methodological framework for future exploration of abnormal brain spatio-temporal patterns and modeling of other functional brain networks.

Towards an accurate autism spectrum disorder diagnosis: multiple connectome views from fMRI data

Functional connectome has revealed remarkable potential in the diagnosis of neurological disorders, e.g. autism spectrum disorder. However, existing studies have primarily focused on a single connectivity pattern, such as full correlation, partial correlation, or causality. Such an approach fails in discovering the potential complementary topology information of FCNs at different connection patterns, resulting in lower diagnostic performance. Consequently, toward an accurate autism spectrum disorder diagnosis, a straightforward ambition is to combine the multiple connectivity patterns for the diagnosis of neurological disorders. To this end, we conduct functional magnetic resonance imaging data to construct multiple brain networks with different connectivity patterns and employ kernel combination techniques to fuse information from different brain connectivity patterns for autism diagnosis. To verify the effectiveness of our approach, we assess the performance of the proposed method on the Autism Brain Imaging Data Exchange dataset for diagnosing autism spectrum disorder. The experimental findings demonstrate that our method achieves precise autism spectrum disorder diagnosis with exceptional accuracy (91.30%), sensitivity (91.48%), and specificity (91.11%).

Preserving specificity in federated graph learning for fMRI-based neurological disorder identification

Resting-state functional magnetic resonance imaging (rs-fMRI) offers a non-invasive approach to examining abnormal brain connectivity associated with brain disorders. Graph neural network (GNN) gains popularity in fMRI representation learning and brain disorder analysis with powerful graph representation capabilities. Training a general GNN often necessitates a large-scale dataset from multiple imaging centers/sites, but centralizing multi-site data generally faces inherent challenges related to data privacy, security, and storage burden. Federated Learning (FL) enables collaborative model training without centralized multi-site fMRI data. Unfortunately, previous FL approaches for fMRI analysis often ignore site-specificity, including demographic factors such as age, gender, and education level. To this end, we propose a specificity-aware federated graph learning (SFGL) framework for rs-fMRI analysis and automated brain disorder identification, with a server and multiple clients/sites for federated model aggregation and prediction. At each client, our model consists of a shared and a personalized branch, where parameters of the shared branch are sent to the server while those of the personalized branch remain local. This can facilitate knowledge sharing among sites and also helps preserve site specificity. In the shared branch, we employ a spatio-temporal attention graph isomorphism network to learn dynamic fMRI representations. In the personalized branch, we integrate vectorized demographic information (i.e., age, gender, and education years) and functional connectivity networks to preserve site-specific characteristics. Representations generated by the two branches are then fused for classification. Experimental results on two fMRI datasets with a total of 1218 subjects suggest that SFGL outperforms several state-of-the-art approaches.

An Umbrella Review of the Fusion of fMRI and AI in Autism

The role of functional magnetic resonance imaging (fMRI) is assuming an increasingly central role in autism diagnosis. The integration of Artificial Intelligence (AI) into the realm of applications further contributes to its development. This study's objective is to analyze emerging themes in this domain through an umbrella review, encompassing systematic reviews. The research methodology was based on a structured process for conducting a literature narrative review, using an umbrella review in PubMed and Scopus. Rigorous criteria, a standard checklist, and a qualification process were meticulously applied. The findings include 20 systematic reviews that underscore key themes in autism research, particularly emphasizing the significance of technological integration, including the pivotal roles of fMRI and AI. This study also highlights the enigmatic role of oxytocin. While acknowledging the immense potential in this field, the outcome does not evade acknowledging the significant challenges and limitations. Intriguingly, there is a growing emphasis on research and innovation in AI, whereas aspects related to the integration of healthcare processes, such as regulation, acceptance, informed consent, and data security, receive comparatively less attention. Additionally, the integration of these findings into Personalized Medicine (PM) represents a promising yet relatively unexplored area within autism research. This study concludes by encouraging scholars to focus on the critical themes of health domain integration, vital for the routine implementation of these applications.

Application of Deep Learning for Prediction of Alzheimer's Disease in PET/MR Imaging

Alzheimer's disease (AD) is a progressive neurodegenerative disorder that affects millions of people worldwide. Positron emission tomography/magnetic resonance (PET/MR) imaging is a promising technique that combines the advantages of PET and MR to provide both functional and structural information of the brain. Deep learning (DL) is a subfield of machine learning (ML) and artificial intelligence (AI) that focuses on developing algorithms and models inspired by the structure and function of the human brain's neural networks. DL has been applied to various aspects of PET/MR imaging in AD, such as image segmentation, image reconstruction, diagnosis and prediction, and visualization of pathological features. In this review, we introduce the basic concepts and types of DL algorithms, such as feed forward neural networks, convolutional neural networks, recurrent neural networks, and autoencoders. We then summarize the current applications and challenges of DL in PET/MR imaging in AD, and discuss the future directions and opportunities for automated diagnosis, predictions of models, and personalized medicine. We conclude that DL has great potential to improve the quality and efficiency of PET/MR imaging in AD, and to provide new insights into the pathophysiology and treatment of this devastating disease.

Deep Learning Aided Neuroimaging and Brain Regulation

Currently, deep learning aided medical imaging is becoming the hot spot of AI frontier application and the future development trend of precision neuroscience. This review aimed to render comprehensive and informative insights into the recent progress of deep learning and its applications in medical imaging for brain monitoring and regulation. The article starts by providing an overview of the current methods for brain imaging, highlighting their limitations and introducing the potential benefits of using deep learning techniques to overcome these limitations. Then, we further delve into the details of deep learning, explaining the basic concepts and providing examples of how it can be used in medical imaging. One of the key strengths is its thorough discussion of the different types of deep learning models that can be used in medical imaging including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and generative adversarial network (GAN) assisted magnetic resonance imaging (MRI), positron emission tomography (PET)/computed tomography (CT), electroencephalography (EEG)/magnetoencephalography (MEG), optical imaging, and other imaging modalities. Overall, our review on deep learning aided medical imaging for brain monitoring and regulation provides a referrable glance for the intersection of deep learning aided neuroimaging and brain regulation.

Multi-head attention-based masked sequence model for mapping functional brain networks

The investigation of functional brain networks (FBNs) using task-based functional magnetic resonance imaging (tfMRI) has gained significant attention in the field of neuroimaging. Despite the availability of several methods for constructing FBNs, including traditional methods like GLM and deep learning methods such as spatiotemporal self-attention mechanism (STAAE), these methods have design and training limitations. Specifically, they do not consider the intrinsic characteristics of fMRI data, such as the possibility that the same signal value at different time points could represent different brain states and meanings. Furthermore, they overlook prior knowledge, such as task designs, during training. This study aims to overcome these limitations and develop a more efficient model by drawing inspiration from techniques in the field of natural language processing (NLP). The proposed model, called the Multi-head Attention-based Masked Sequence Model (MAMSM), uses a multi-headed attention mechanism and mask training approach to learn different states corresponding to the same voxel values. Additionally, it combines cosine similarity and task design curves to construct a novel loss function. The MAMSM was applied to seven task state datasets from the Human Connectome Project (HCP) tfMRI dataset. Experimental results showed that the features acquired by the MAMSM model exhibit a Pearson correlation coefficient with the task design curves above 0.95 on average. Moreover, the model can extract more meaningful networks beyond the known task-related brain networks. The experimental results demonstrated that MAMSM has great potential in advancing the understanding of functional brain networks.