fMRI volume classification using a 3D convolutional neural network robust to shifted and scaled neuronal activations

Automating the Human Connectome Project's Temporal ICA Pipeline

Functional magnetic resonance imaging (fMRI) data are dominated by noise and artifacts, with only a small fraction of the variance relating to neural activity. Temporal independent component analysis (tICA) is a recently developed method that enables selective denoising of fMRI artifacts related to physiology such as respiration. However, an automated and easy to use pipeline for tICA has not previously been available; instead, two manual steps have been necessary: 1) setting the group spatial ICA dimensionality after MELODIC's Incremental Group-PCA (MIGP) and 2) labeling tICA components as artifacts versus signals. Moreover, guidance has been lacking as to how many subjects and timepoints are needed to adequately re-estimate the temporal ICA decomposition and what alternatives are available for smaller groups or even individual subjects. Here, we introduce a nine-step fully automated tICA pipeline which removes global artifacts from fMRI dense timeseries after sICA+FIX cleaning and MSMAll alignment driven by functionally relevant areal features. Additionally, we have developed an automated "reclean" Pipeline for improved spatial ICA (sICA) artifact removal. Two major automated components of the pipeline are 1) an automatic group spatial ICA (sICA) dimensionality selection for MIGP data enabled by fitting multiple Wishart distributions; 2) a hierarchical classifier to distinguish group tICA signal components from artifactual components, equipped with a combination of handcrafted features from domain expert knowledge and latent features obtained via self-supervised learning on spatial maps. We demonstrate that the dimensionality estimated for the MIGP data from HCP Young Adult 3T and 7T datasets is comparable to previous manual tICA estimates, and that the group sICA decomposition is highly reproducible. We also show that the tICA classifier achieved over 0.98 Precision-Recall Area Under Curve (PR-AUC) and that the correctly classified components account for over 95% of the tICA-represented variance on multiple held-out evaluation datasets including the HCP-Young Adult, HCP-Aging and HCP-Development datasets under various settings. Our automated tICA pipeline is now available as part of the HCP pipelines, providing a powerful and user-friendly tool for the neuroimaging community.

Multimodal deep neural decoding reveals highly resolved spatiotemporal profile of visual object representation in humans

Perception and categorization of objects in a visual scene are essential to grasp the surrounding situation. Recently, neural decoding schemes, such as machine learning in functional magnetic resonance imaging (fMRI), has been employed to elucidate the underlying neural mechanisms. However, it remains unclear as to how spatially distributed brain regions temporally represent visual object categories and sub-categories. One promising strategy to address this issue is neural decoding with concurrently obtained neural response data of high spatial and temporal resolution. In this study, we explored the spatial and temporal organization of visual object representations using concurrent fMRI and electroencephalography (EEG), combined with neural decoding using deep neural networks (DNNs). We hypothesized that neural decoding by multimodal neural data with DNN would show high classification performance in visual object categorization (faces or non-face objects) and sub-categorization within faces and objects. Visualization of the fMRI DNN was more sensitive than that in the univariate approach and revealed that visual categorization occurred in brain-wide regions. Interestingly, the EEG DNN valued the earlier phase of neural responses for categorization and the later phase of neural responses for sub-categorization. Combination of the two DNNs improved the classification performance for both categorization and sub-categorization compared with fMRI DNN or EEG DNN alone. These deep learning-based results demonstrate a categorization principle in which visual objects are represented in a spatially organized and coarse-to-fine manner, and provide strong evidence of the ability of multimodal deep learning to uncover spatiotemporal neural machinery in sensory processing.

A Class Activation Map-Based Interpretable Transfer Learning Model for Automated Detection of ADHD from fMRI Data

Automatic detection of Attention Deficit Hyperactivity Disorder (ADHD) based on the functional Magnetic Resonance Imaging (fMRI) through Deep Learning (DL) is becoming a quite useful methodology due to the curse of-dimensionality problem of the data is solved. Also, this method proposes an invasive and robust solution to the variances in data acquisition and class distribution imbalances. In this paper, a transfer learning approach, specifically ResNet-50 type pre-trained 2D-Convolutional Neural Network (CNN) was used to automatically classify ADHD and healthy children. The results demonstrated that ResNet-50 architecture with 10-k cross-validation (CV) achieves an overall classification accuracy of 93.45%. The interpretation of the results was done via the Class Activation Map (CAM) analysis which showed that children with ADHD differed from controls in a wide range of brain areas including frontal, parietal and temporal lobes.

General psychopathology factor (p-factor) prediction using resting-state functional connectivity and a scanner-generalization neural network

The general psychopathology factor (p-factor) represents shared variance across mental disorders based on psychopathologic symptoms. The Adolescent Brain Cognitive Development (ABCD) Study offers an unprecedented opportunity to investigate functional networks (FNs) from functional magnetic resonance imaging (fMRI) associated with the psychopathology of an adolescent cohort (n > 10,000). However, the heterogeneities associated with the use of multiple sites and multiple scanners in the ABCD Study need to be overcome to improve the prediction of the p-factor using fMRI. We proposed a scanner-generalization neural network (SGNN) to predict the individual p-factor by systematically reducing the scanner effect for resting-state functional connectivity (RSFC). We included 6905 adolescents from 18 sites whose fMRI data were collected using either Siemens or GE scanners. The p-factor was estimated based on the Child Behavior Checklist (CBCL) scores available in the ABCD study using exploratory factor analysis. We evaluated the Pearson's correlation coefficients (CCs) for p-factor prediction via leave-one/two-site-out cross-validation (LOSOCV/LTSOCV) and identified important FNs from the weight features (WFs) of the SGNN. The CCs were higher for the SGNN than for alternative models when using both LOSOCV (0.1631 ± 0.0673 for the SGNN vs. 0.1497 ± 0.0710 for kernel ridge regression [KRR]; p < 0.05 from a two-tailed paired t-test) and LTSOCV (0.1469 ± 0.0381 for the SGNN vs. 0.1394 ± 0.0359 for KRR; p = 0.01). It was found that (a) the default-mode and dorsal attention FNs were important for p-factor prediction, and (b) the intra-visual FN was important for scanner generalization. We demonstrated the efficacy of our novel SGNN model for p-factor prediction while simultaneously eliminating scanner-related confounding effects for RSFC.

On the benefits of self-taught learning for brain decoding

CONTEXT

We study the benefits of using a large public neuroimaging database composed of functional magnetic resonance imaging (fMRI) statistic maps, in a self-taught learning framework, for improving brain decoding on new tasks. First, we leverage the NeuroVault database to train, on a selection of relevant statistic maps, a convolutional autoencoder to reconstruct these maps. Then, we use this trained encoder to initialize a supervised convolutional neural network to classify tasks or cognitive processes of unseen statistic maps from large collections of the NeuroVault database.

RESULTS

We show that such a self-taught learning process always improves the performance of the classifiers, but the magnitude of the benefits strongly depends on the number of samples available both for pretraining and fine-tuning the models and on the complexity of the targeted downstream task.

CONCLUSION

The pretrained model improves the classification performance and displays more generalizable features, less sensitive to individual differences.

Deep learning in neuroimaging data analysis: Applications, challenges, and solutions

Methods for the analysis of neuroimaging data have advanced significantly since the beginning of neuroscience as a scientific discipline. Today, sophisticated statistical procedures allow us to examine complex multivariate patterns, however most of them are still constrained by assuming inherent linearity of neural processes. Here, we discuss a group of machine learning methods, called deep learning, which have drawn much attention in and outside the field of neuroscience in recent years and hold the potential to surpass the mentioned limitations. Firstly, we describe and explain the essential concepts in deep learning: the structure and the computational operations that allow deep models to learn. After that, we move to the most common applications of deep learning in neuroimaging data analysis: prediction of outcome, interpretation of internal representations, generation of synthetic data and segmentation. In the next section we present issues that deep learning poses, which concerns multidimensionality and multimodality of data, overfitting and computational cost, and propose possible solutions. Lastly, we discuss the current reach of DL usage in all the common applications in neuroimaging data analysis, where we consider the promise of multimodality, capability of processing raw data, and advanced visualization strategies. We identify research gaps, such as focusing on a limited number of criterion variables and the lack of a well-defined strategy for choosing architecture and hyperparameters. Furthermore, we talk about the possibility of conducting research with constructs that have been ignored so far or/and moving toward frameworks, such as RDoC, the potential of transfer learning and generation of synthetic data.

Deep Metric Representation Learning for Clinical Resting State fMRI

With growing size of resting state fMRI datasets and advances in deep learning methods, there are ever increasing opportunities to leverage progress in deep learning to solve challenging tasks in neuroimaging. In this work, we build upon recent advances in deep metric learning, to learn embeddings of rs-fMRI data, which can then be potentially used for several downstream tasks. We propose an efficient training method for our model and compare our method with other widely used models. Our experimental results indicate that deep metric learning can be used as an additional refinement step to learn representations of fMRI data, that significantly improves performance on downstream modeling tasks.

Attention module improves both performance and interpretability of four-dimensional functional magnetic resonance imaging decoding neural network

Decoding brain cognitive states from neuroimaging signals is an important topic in neuroscience. In recent years, deep neural networks (DNNs) have been recruited for multiple brain state decoding and achieved good performance. However, the open question of how to interpret the DNN black box remains unanswered. Capitalizing on advances in machine learning, we integrated attention modules into brain decoders to facilitate an in‐depth interpretation of DNN channels. A four‐dimensional (4D) convolution operation was also included to extract temporo‐spatial interaction within the fMRI signal. The experiments showed that the proposed model obtains a very high accuracy (97.4%) and outperforms previous researches on the seven different task benchmarks from the Human Connectome Project (HCP) dataset. The visualization analysis further illustrated the hierarchical emergence of task‐specific masks with depth. Finally, the model was retrained to regress individual traits within the HCP and to classify viewing images from the BOLD5000 dataset, respectively. Transfer learning also achieves good performance. Further visualization analysis shows that, after transfer learning, low‐level attention masks remained similar to the source domain, whereas high‐level attention masks changed adaptively. In conclusion, the proposed 4D model with attention module performed well and facilitated interpretation of DNNs, which is helpful for subsequent research.

Mixed-effects multilevel analysis followed by canonical correlation analysis is an effective fMRI tool for the investigation of idiosyncrasies

We report that regions-of-interest (ROIs) associated with idiosyncratic individual behavior can be identified from functional magnetic resonance imaging (fMRI) data using statistical approaches that explicitly model individual variability in neuronal activations, such as mixed-effects multilevel analysis (MEMA). We also show that the relationship between neuronal activation in fMRI and behavioral data can be modeled using canonical correlation analysis (CCA). A real-world dataset for the neuronal response to nicotine use was acquired using a custom-made MRI-compatible apparatus for the smoking of electronic cigarettes (e-cigarettes). Nineteen participants smoked e-cigarettes in an MRI scanner using the apparatus with two experimental conditions: e-cigarettes with nicotine (ECIG) and sham e-cigarettes without nicotine (SCIG) and subjective ratings were collected. The right insula was identified in the ECIG condition from the χ² -test of the MEMA but not from the t-test, and the corresponding activations were significantly associated with the similarity scores (r = -.52, p = .041, confidence interval [CI] = [-0.78, -0.17]) and the urge-to-smoke scores (r = .73, p <.001, CI = [0.52, 0.88]). From the contrast between the two conditions (i.e., ECIG > SCIG), the right orbitofrontal cortex was identified from the χ² -tests, and the corresponding neuronal activations showed a statistically meaningful association with similarity (r = -.58, p = .01, CI = [-0.84, -0.17]) and the urge to smoke (r = .34, p = .15, CI = [0.09, 0.56]). The validity of our analysis pipeline (i.e., MEMA followed by CCA) was further evaluated using the fMRI and behavioral data acquired from the working memory and gambling tasks available from the Human Connectome Project.

Cigarette craving modulation is more feasible than resistance modulation for heavy cigarette smokers: empirical evidence from functional MRI data

BACKGROUND

Modulation of cigarette craving and neuronal activations from nicotine-dependent cigarette smokers using real-time functional MRI (rtfMRI)-based neurofeedback (rtfMRI-NF) has been previously reported.

OBJECTIVES

The aim of this study was to evaluate the efficacy of rtfMRI-NF training in reducing cigarette cravings using fMRI data acquired before and after training.

METHODS

Treatment-seeking male heavy cigarette smokers (N = 14) were enrolled and randomly assigned to two conditions related to rtfMRI-NF training aiming at resisting the urge to smoke. In one condition, subjects underwent conventional rtfMRI-NF training using neuronal activity as the neurofeedback signal (activity-based) within regions-of-interest (ROIs) implicated in cigarette craving. In another condition, subjects underwent rtfMRI-NF training with additional functional connectivity information included in the neurofeedback signal (functional connectivity-added). Before and after rtfMRI-NF training at each of two visits, participants underwent two fMRI runs with cigarette smoking stimuli and were asked to crave or resist the urge to smoke without neurofeedback. Cigarette craving-related or resistance-related regions were identified using a general linear model followed by paired t-tests and were evaluated using regression analysis on the basis of neuronal activation and subjective craving scores (CRSs).

RESULTS

Visual areas were mainly implicated in craving, whereas the superior frontal areas were associated with resistance. The degree of (a) CRS reduction and (b) the correlation between neuronal activation and CRSs were statistically significant (P < 0.05) in the functional connectivity-added neurofeedback group for craving-related ROIs.

CONCLUSION

Our study demonstrated the feasibility of altering cigarette craving in craving-related ROIs but not in resistance-related ROIs via rtfMRI-NF training.

PSSPNN: PatchShuffle Stochastic Pooling Neural Network for an Explainable Diagnosis of COVID-19 with Multiple-Way Data Augmentation

AIM

COVID-19 has caused large death tolls all over the world. Accurate diagnosis is of significant importance for early treatment.

METHODS

In this study, we proposed a novel PSSPNN model for classification between COVID-19, secondary pulmonary tuberculosis, community-captured pneumonia, and healthy subjects. PSSPNN entails five improvements: we first proposed the n-conv stochastic pooling module. Second, a novel stochastic pooling neural network was proposed. Third, PatchShuffle was introduced as a regularization term. Fourth, an improved multiple-way data augmentation was used. Fifth, Grad-CAM was utilized to interpret our AI model.

RESULTS

The 10 runs with random seed on the test set showed our algorithm achieved a microaveraged F1 score of 95.79%. Moreover, our method is better than nine state-of-the-art approaches.

CONCLUSION

This proposed PSSPNN will help assist radiologists to make diagnosis more quickly and accurately on COVID-19 cases.