Linking brain-wide multivoxel activation patterns to behaviour: Examples from language and math

Perspective-taking is associated with increased discriminability of affective states in the ventromedial prefrontal cortex

Recent work using multivariate-pattern analysis (MVPA) on functional magnetic resonance imaging (fMRI) data has found that distinct affective states produce correspondingly distinct patterns of neural activity in the cerebral cortex. However, it is unclear whether individual differences in the distinctiveness of neural patterns evoked by affective stimuli underlie empathic abilities such as perspective-taking (PT). Accordingly, we examined whether we could predict PT tendency from the classification of blood-oxygen-level-dependent (BOLD) fMRI activation patterns while participants (n = 57) imagined themselves in affectively charged scenarios. We used an MVPA searchlight analysis to map where in the brain activity patterns permitted the classification of four affective states: happiness, sadness, fear and disgust. Classification accuracy was significantly above chance levels in most of the prefrontal cortex and in the posterior medial cortices. Furthermore, participants' self-reported PT was positively associated with classification accuracy in the ventromedial prefrontal cortex and insula. This finding has implications for understanding affective processing in the prefrontal cortex and for interpreting the cognitive significance of classifiable affective brain states. Our multivariate approach suggests that PT ability may rely on the grain of internally simulated affective representations rather than simply the global strength.

Brain activity patterns of phonemic representations are atypical in beginning readers with family risk for dyslexia

There is an ongoing debate whether phonological deficits in dyslexics should be attributed to (a) less specified representations of speech sounds, like suggested by studies in young children with a familial risk for dyslexia, or (b) to an impaired access to these phonemic representations, as suggested by studies in adults with dyslexia. These conflicting findings are rooted in between study differences in sample characteristics and/or testing techniques. The current study uses the same multivariate functional MRI (fMRI) approach as previously used in adults with dyslexia to investigate phonemic representations in 30 beginning readers with a familial risk and 24 beginning readers without a familial risk of dyslexia, of whom 20 were later retrospectively classified as dyslexic. Based on fMRI response patterns evoked by listening to different utterances of /bA/ and /dA/ sounds, multivoxel analyses indicate that the underlying activation patterns of the two phonemes were distinct in children with a low family risk but not in children with high family risk. However, no group differences were observed between children that were later classified as typical versus dyslexic readers, regardless of their family risk status, indicating that poor phonemic representations constitute a risk for dyslexia but are not sufficient to result in reading problems. We hypothesize that poor phonemic representations are trait (family risk) and not state (dyslexia) dependent, and that representational deficits only lead to reading difficulties when they are present in conjunction with other neuroanatomical or-functional deficits.

The effect of machine learning regression algorithms and sample size on individualized behavioral prediction with functional connectivity features

Individualized behavioral/cognitive prediction using machine learning (ML) regression approaches is becoming increasingly applied. The specific ML regression algorithm and sample size are two key factors that non-trivially influence prediction accuracies. However, the effects of the ML regression algorithm and sample size on individualized behavioral/cognitive prediction performance have not been comprehensively assessed. To address this issue, the present study included six commonly used ML regression algorithms: ordinary least squares (OLS) regression, least absolute shrinkage and selection operator (LASSO) regression, ridge regression, elastic-net regression, linear support vector regression (LSVR), and relevance vector regression (RVR), to perform specific behavioral/cognitive predictions based on different sample sizes. Specifically, the publicly available resting-state functional MRI (rs-fMRI) dataset from the Human Connectome Project (HCP) was used, and whole-brain resting-state functional connectivity (rsFC) or rsFC strength (rsFCS) were extracted as prediction features. Twenty-five sample sizes (ranged from 20 to 700) were studied by sub-sampling from the entire HCP cohort. The analyses showed that rsFC-based LASSO regression performed remarkably worse than the other algorithms, and rsFCS-based OLS regression performed markedly worse than the other algorithms. Regardless of the algorithm and feature type, both the prediction accuracy and its stability exponentially increased with increasing sample size. The specific patterns of the observed algorithm and sample size effects were well replicated in the prediction using re-testing fMRI data, data processed by different imaging preprocessing schemes, and different behavioral/cognitive scores, thus indicating excellent robustness/generalization of the effects. The current findings provide critical insight into how the selected ML regression algorithm and sample size influence individualized predictions of behavior/cognition and offer important guidance for choosing the ML regression algorithm or sample size in relevant investigations.

Arithmetic in the developing brain: A review of brain imaging studies

Brain imaging studies on academic achievement offer an exciting window on experience-dependent cortical plasticity, as they allow us to understand how developing brains change when children acquire culturally transmitted skills. This contribution focuses on the learning of arithmetic, which is quintessential to mathematical development. The nascent body of brain imaging studies reveals that arithmetic recruits a large set of interconnected areas, including prefrontal, posterior parietal, occipito-temporal and hippocampal areas. This network undergoes developmental changes in its function, connectivity and structure, which are not yet fully understood. This network only partially overlaps with what has been found in adults, and clear differences are observed in the recruitment of the hippocampus, which are related to the development of arithmetic fact retrieval. Despite these emerging trends, the literature remains scattered, particularly in the context of atypical development. Acknowledging the distributed nature of the arithmetic network, future studies should focus on connectivity and analytic approaches that investigate patterns of brain activity, coupled with a careful design of the arithmetic tasks and assessments of arithmetic strategies. Such studies will produce a more comprehensive understanding of how the arithmetical brain unfolds, how it changes over time, and how it is impaired in atypical development.

When minds matter for moral judgment: intent information is neurally encoded for harmful but not impure acts

Recent behavioral evidence indicates a key role for intent in moral judgments of harmful acts (e.g. assault) but not impure acts (e.g. incest). We tested whether the neural responses in regions for mental state reasoning, including the right temporoparietal junction (RTPJ), are greater when people evaluate harmful vs impure violations. In addition, using multivoxel pattern analysis, we investigated whether the voxel-wise pattern in these regions distinguishes intentional from accidental actions, for either kind of violation. The RTPJ was preferentially recruited in response to harmful vs impure acts. Moreover, although its response was equally high for intentional and accidental acts, the voxel-wise pattern in the RTPJ distinguished intentional from accidental acts in the harm domain but not the purity domain. Finally, we found that the degree to which the RTPJ discriminated between intentional and accidental acts predicted the impact of intent information on moral judgments but again only in the harm domain. These findings reveal intent to be a uniquely critical factor for moral evaluations of harmful vs impure acts and shed light on the neural computations for mental state reasoning.

A Window into the Brain: Advances in Psychiatric fMRI

Functional magnetic resonance imaging (fMRI) plays a key role in modern psychiatric research. It provides a means to assay differences in brain systems that underlie psychiatric illness, treatment response, and properties of brain structure and function that convey risk factor for mental diseases. Here we review recent advances in fMRI methods in general use and progress made in understanding the neural basis of mental illness. Drawing on concepts and findings from psychiatric fMRI, we propose that mental illness may not be associated with abnormalities in specific local regions but rather corresponds to variation in the overall organization of functional communication throughout the brain network. Future research may need to integrate neuroimaging information drawn from different analysis methods and delineate spatial and temporal patterns of brain responses that are specific to certain types of psychiatric disorders.

Comparing within-subject classification and regularization methods in fMRI for large and small sample sizes

In recent years, a variety of multivariate classifier models have been applied to fMRI, with different modeling assumptions. When classifying high-dimensional fMRI data, we must also regularize to improve model stability, and the interactions between classifier and regularization techniques are still being investigated. Classifiers are usually compared on large, multisubject fMRI datasets. However, it is unclear how classifier/regularizer models perform for within-subject analyses, as a function of signal strength and sample size. We compare four standard classifiers: Linear and Quadratic Discriminants, Logistic Regression and Support Vector Machines. Classification was performed on data in the linear kernel (covariance) feature space, and classifiers are tuned with four commonly-used regularizers: Principal Component and Independent Component Analysis, and penalization of kernel features using L₁ and L₂ norms. We evaluated prediction accuracy (P) and spatial reproducibility (R) of all classifier/regularizer combinations on single-subject analyses, over a range of three different block task contrasts and sample sizes for a BOLD fMRI experiment. We show that the classifier model has a small impact on signal detection, compared to the choice of regularizer. PCA maximizes reproducibility and global SNR, whereas Lp -norms tend to maximize prediction. ICA produces low reproducibility, and prediction accuracy is classifier-dependent. However, trade-offs in (P,R) depend partly on the optimization criterion, and PCA-based models are able to explore the widest range of (P,R) values. These trends are consistent across task contrasts and data sizes (training samples range from 6 to 96 scans). In addition, the trends in classifier performance are consistent for ROI-based classifier analyses.

Neural Correlates of Subliminal Language Processing

Language is a high-level cognitive function, so exploring the neural correlates of unconscious language processing is essential for understanding the limits of unconscious processing in general. The results of several functional magnetic resonance imaging studies have suggested that unconscious lexical and semantic processing is confined to the posterior temporal lobe, without involvement of the frontal lobe—the regions that are indispensable for conscious language processing. However, previous studies employed a similarly designed masked priming paradigm with briefly presented single and contextually unrelated words. It is thus possible, that the stimulation level was insufficiently strong to be detected in the high-level frontal regions. Here, in a high-resolution fMRI and multivariate pattern analysis study we explored the neural correlates of subliminal language processing using a novel paradigm, where written meaningful sentences were suppressed from awareness for extended duration using continuous flash suppression. We found that subjectively and objectively invisible meaningful sentences and unpronounceable nonwords could be discriminated not only in the left posterior superior temporal sulcus (STS), but critically, also in the left middle frontal gyrus. We conclude that frontal lobes play a role in unconscious language processing and that activation of the frontal lobes per se might not be sufficient for achieving conscious awareness.

Decoding moral judgments from neural representations of intentions

Intentional harms are typically judged to be morally worse than accidental harms. Distinguishing between intentional harms and accidents depends on the capacity for mental state reasoning (i.e., reasoning about beliefs and intentions), which is supported by a group of brain regions including the right temporo-parietal junction (RTPJ). Prior research has found that interfering with activity in RTPJ can impair mental state reasoning for moral judgment and that high-functioning individuals with autism spectrum disorders make moral judgments based less on intent information than neurotypical participants. Three experiments, using multivoxel pattern analysis, find that (i) in neurotypical adults, the RTPJ shows reliable and distinct spatial patterns of responses across voxels for intentional vs. accidental harms, and (ii) individual differences in this neural pattern predict differences in participants' moral judgments. These effects are specific to RTPJ. By contrast, (iii) this distinction was absent in adults with autism spectrum disorders. We conclude that multivoxel pattern analysis can detect features of mental state representations (e.g., intent), and that the corresponding neural patterns are behaviorally and clinically relevant.

Paradigm free mapping with sparse regression automatically detects single-trial functional magnetic resonance imaging blood oxygenation level dependent responses

The ability to detect single trial responses in functional magnetic resonance imaging (fMRI) studies is essential, particularly if investigating learning or adaptation processes or unpredictable events. We recently introduced paradigm free mapping (PFM), an analysis method that detects single trial blood oxygenation level dependent (BOLD) responses without specifying prior information on the timing of the events. PFM is based on the deconvolution of the fMRI signal using a linear hemodynamic convolution model. Our previous PFM method (Caballero-Gaudes et al., 2011: Hum Brain Mapp) used the ridge regression estimator for signal deconvolution and required a baseline signal period for statistical inference. In this work, we investigate the application of sparse regression techniques in PFM. In particular, a novel PFM approach is developed using the Dantzig selector estimator, solved via an efficient homotopy procedure, along with statistical model selection criteria. Simulation results demonstrated that, using the Bayesian information criterion to select the regularization parameter, this method obtains high detection rates of the BOLD responses, comparable with a model-based analysis, but requiring no information on the timing of the events and being robust against hemodynamic response function variability. The practical operation of this sparse PFM method was assessed with single-trial fMRI data acquired at 7T, where it automatically detected all task-related events, and was an improvement on our previous PFM method, as it does not require the definition of a baseline state and amplitude thresholding and does not compromise on specificity and sensitivity.

A sparse representation-based algorithm for pattern localization in brain imaging data analysis

Considering the two-class classification problem in brain imaging data analysis, we propose a sparse representation-based multi-variate pattern analysis (MVPA) algorithm to localize brain activation patterns corresponding to different stimulus classes/brain states respectively. Feature selection can be modeled as a sparse representation (or sparse regression) problem. Such technique has been successfully applied to voxel selection in fMRI data analysis. However, single selection based on sparse representation or other methods is prone to obtain a subset of the most informative features rather than all. Herein, our proposed algorithm recursively eliminates informative features selected by a sparse regression method until the decoding accuracy based on the remaining features drops to a threshold close to chance level. In this way, the resultant feature set including all the identified features is expected to involve all the informative features for discrimination. According to the signs of the sparse regression weights, these selected features are separated into two sets corresponding to two stimulus classes/brain states. Next, in order to remove irrelevant/noisy features in the two selected feature sets, we perform a nonparametric permutation test at the individual subject level or the group level. In data analysis, we verified our algorithm with a toy data set and an intrinsic signal optical imaging data set. The results show that our algorithm has accurately localized two class-related patterns. As an application example, we used our algorithm on a functional magnetic resonance imaging (fMRI) data set. Two sets of informative voxels, corresponding to two semantic categories (i.e., “old people” and “young people”), respectively, are obtained in the human brain.

Classification of structural MRI images in Alzheimer's disease from the perspective of ill-posed problems

BACKGROUND

Machine learning neuroimaging researchers have often relied on regularization techniques when classifying MRI images. Although these were originally introduced to deal with "ill-posed" problems it is rare to find studies that evaluate the ill-posedness of MRI image classification problems. In addition, to avoid the effects of the "curse of dimensionality" very often dimension reduction is applied to the data.

METHODOLOGY

Baseline structural MRI data from cognitively normal and Alzheimer's disease (AD) patients from the AD Neuroimaging Initiative database were used in this study. We evaluated here the ill-posedness of this classification problem across different dimensions and sample sizes and its relationship to the performance of regularized logistic regression (RLR), linear support vector machine (SVM) and linear regression classifier (LRC). In addition, these methods were compared with their principal components space counterparts.

PRINCIPAL FINDINGS

In voxel space the prediction performance of all methods increased as sample sizes increased. They were not only relatively robust to the increase of dimension, but they often showed improvements in accuracy. We linked this behavior to improvements in conditioning of the linear kernels matrices. In general the RLR and SVM performed similarly. Surprisingly, the LRC was often very competitive when the linear kernel matrices were best conditioned. Finally, when comparing these methods in voxel and principal component spaces, we did not find large differences in prediction performance.

CONCLUSIONS AND SIGNIFICANCE

We analyzed the problem of classifying AD MRI images from the perspective of linear ill-posed problems. We demonstrate empirically the impact of the linear kernel matrix conditioning on different classifiers' performance. This dependence is characterized across sample sizes and dimensions. In this context we also show that increased dimensionality does not necessarily degrade performance of machine learning methods. In general, this depends on the nature of the problem and the type of machine learning method.

Combining graph and machine learning methods to analyze differences in functional connectivity across sex

In this work we combine machine learning methods and graph theoretical analysis to investigate gender associated differences in resting state brain network connectivity. The set of all correlations computed from the fMRI resting state data is used as input features for classification. Two ensemble learning methods are used to perform the detection of the set of discriminative edges between groups (males vs. females) of brain networks: 1) Random Forest and 2) an ensemble method based on least angle shrinkage and selection operator (lasso) regressors. Permutation testing is used not only to assess significance of classification accuracy but also to evaluate significance of feature selection. Finally, these methods are applied to data downloaded from the Connectome Project website. Our results suggest that gender differences in brain function may be related to sexually dimorphic regional connectivity between specific critical nodes via gender-discriminative edges.

Total variation regularization for fMRI-based prediction of behavior

While medical imaging typically provides massive amounts of data, the extraction of relevant information for predictive diagnosis remains a difficult challenge. Functional magnetic resonance imaging (fMRI) data, that provide an indirect measure of task-related or spontaneous neuronal activity, are classically analyzed in a mass-univariate procedure yielding statistical parametric maps. This analysis framework disregards some important principles of brain organization: population coding, distributed and overlapping representations. Multivariate pattern analysis, i.e., the prediction of behavioral variables from brain activation patterns better captures this structure. To cope with the high dimensionality of the data, the learning method has to be regularized. However, the spatial structure of the image is not taken into account in standard regularization methods, so that the extracted features are often hard to interpret. More informative and interpretable results can be obtained with the l(1) norm of the image gradient, also known as its total variation (TV), as regularization. We apply for the first time this method to fMRI data, and show that TV regularization is well suited to the purpose of brain mapping while being a powerful tool for brain decoding. Moreover, this article presents the first use of TV regularization for classification.

How does a child solve 7 + 8? Decoding brain activity patterns associated with counting and retrieval strategies

Cognitive development and learning are characterized by diminished reliance on effortful procedures and increased use of memory-based problem solving. Here we identify the neural correlates of this strategy shift in 7-9-year-old children at an important developmental period for arithmetic skill acquisition. Univariate and multivariate approaches were used to contrast brain responses between two groups of children who relied primarily on either retrieval or procedural counting strategies. Children who used retrieval strategies showed greater responses in the left ventrolateral prefrontal cortex; notably, this was the only brain region which showed univariate differences in signal intensity between the two groups. In contrast, multivariate analysis revealed distinct multivoxel activity patterns in bilateral hippocampus, posterior parietal cortex and left ventrolateral prefrontal cortex regions between the two groups. These results demonstrate that retrieval and counting strategies during early learning are characterized by distinct patterns of activity in a distributed network of brain regions involved in arithmetic problem solving and controlled retrieval of arithmetic facts. Our findings suggest that the reorganization and refinement of neural activity patterns in multiple brain regions plays a dominant role in the transition to memory-based arithmetic problem solving. Our findings further demonstrate how multivariate approaches can provide novel insights into fine-scale developmental changes in the brain. More generally, our study illustrates how brain imaging and developmental research can be integrated to investigate fundamental aspects of neurocognitive development.

The dynamics of ontogeny: a neuroconstructivist perspective on genes, brains, cognition and behavior

For years, the view that the human cognitive system is as a Swiss army knife with innately specified functional modules that come online one by one or can be impaired independently of other modules, has dominated cognitive science. In this chapter, we start out by questioning this view and argue it needs to be replaced by a dynamic neuroconstructivist approach in which genes, brain, behavior, and environment interact multidirectionally throughout development. Using examples from the recent literature, we then highlight how a static modular view of the brain remains deeply ingrained in (1) behavioral, (2) neuroimaging, and (3) genetics research on typical and atypical cognitive development. Finally, we discuss future contributions of the neuroconstructivist approach to developmental research in particular, and cognitive neuroscience in general.

Neuroimaging of the developing brain: taking "developing" seriously

With a few notable exceptions, many studies, be they behavioral, neuroimaging, or genetic, are snapshots that compare one child group to one adult group, which capture only two points in time and tell the scientist nothing about the mechanisms underlying neural trajectories over developmental time. Thus, a distinction needs to be drawn between child neuroimaging and developmental neuroimaging, the latter approach being relevant not just to children, but to adults and the ageing brain.