Task modulation of spatiotemporal dynamics in semantic brain networks: An EEG/MEG study

Decoding Semantics from Dynamic Brain Activation Patterns: From Trials to Task in EEG/MEG Source Space

The temporal dynamics within the semantic brain network and its dependence on stimulus and task parameters are still not well understood. Here, we addressed this by decoding task as well as stimulus information from source-estimated EEG/MEG human data. We presented the same visual word stimuli in a lexical decision (LD) and three semantic decision (SD) tasks. The meanings of the presented words varied across five semantic categories. Source space decoding was applied over time in five ROIs in the left hemisphere (anterior and posterior temporal lobe, inferior frontal gyrus, primary visual areas, and angular gyrus) and one in the right hemisphere (anterior temporal lobe). Task decoding produced sustained significant effects in all ROIs from 50 to 100 ms, both when categorizing tasks with different semantic demands (LD-SD) as well as for similar semantic tasks (SD-SD). In contrast, a semantic word category could only be decoded in lATL, rATL, PTC, and IFG, between 250 and 500 ms. Furthermore, we compared two approaches to source space decoding: conventional ROI-by-ROI decoding and combined-ROI decoding with back-projected activation patterns. The former produced more reliable results for word category decoding while the latter was more informative for task decoding. This indicates that task effects are distributed across the whole semantic network while stimulus effects are more focal. Our results demonstrate that the semantic network is widely distributed but that bilateral anterior temporal lobes together with control regions are particularly relevant for the processing of semantic information.

Neural manifolds carry reactivation of phonetic representations during semantic processing

Traditional models of speech perception posit that neural activity encodes speech through a hierarchy of cognitive processes, from low-level representations of acoustic and phonetic features to high-level semantic encoding. Yet it remains unknown how neural representations are transformed across levels of the speech hierarchy. Here, we analyzed unique microelectrode array recordings of neuronal spiking activity from the human left anterior superior temporal gyrus, a brain region at the interface between phonetic and semantic speech processing, during a semantic categorization task and natural speech perception. We identified distinct neural manifolds for semantic and phonetic features, with a functional separation of the corresponding low-dimensional trajectories. Moreover, phonetic and semantic representations were encoded concurrently and reflected in power increases in the beta and low-gamma local field potentials, suggesting top-down predictive and bottom-up cumulative processes. Our results are the first to demonstrate mechanisms for hierarchical speech transformations that are specific to neuronal population dynamics.

Recurrent connectivity supports higher-level visual and semantic object representations in the brain

Visual object recognition has been traditionally conceptualised as a predominantly feedforward process through the ventral visual pathway. While feedforward artificial neural networks (ANNs) can achieve human-level classification on some image-labelling tasks, it's unclear whether computational models of vision alone can accurately capture the evolving spatiotemporal neural dynamics. Here, we probe these dynamics using a combination of representational similarity and connectivity analyses of fMRI and MEG data recorded during the recognition of familiar, unambiguous objects. Modelling the visual and semantic properties of our stimuli using an artificial neural network as well as a semantic feature model, we find that unique aspects of the neural architecture and connectivity dynamics relate to visual and semantic object properties. Critically, we show that recurrent processing between the anterior and posterior ventral temporal cortex relates to higher-level visual properties prior to semantic object properties, in addition to semantic-related feedback from the frontal lobe to the ventral temporal lobe between 250 and 500 ms after stimulus onset. These results demonstrate the distinct contributions made by semantic object properties in explaining neural activity and connectivity, highlighting it as a core part of object recognition not fully accounted for by current biologically inspired neural networks.

An eye on semantics: a study on the influence of concreteness and predictability on early fixation durations

We used eye-tracking during natural reading to study how semantic control and representation mechanisms interact for the successful comprehension of sentences, by manipulating sentence context and single-word meaning. Specifically, we examined whether a word's semantic characteristic (concreteness) affects first fixation and gaze durations (FFDs and GDs) and whether it interacts with the predictability of a word. We used a linear mixed effects model including several possible psycholinguistic covariates. We found a small but reliable main effect of concreteness and replicated a predictability effect on FFDs, but we found no interaction between the two. The results parallel previous findings of additive effects of predictability (context) and frequency (lexical level) in fixation times. Our findings suggest that the semantics of a word and the context created by the preceding words additively influence early stages of word processing in natural sentence reading.

Time-Lagged Multidimensional Pattern Connectivity (TL-MDPC): An EEG/MEG pattern transformation based functional connectivity metric

Functional and effective connectivity methods are essential to study the complex information flow in brain networks underlying human cognition. Only recently have connectivity methods begun to emerge that make use of the full multidimensional information contained in patterns of brain activation, rather than unidimensional summary measures of these patterns. To date, these methods have mostly been applied to fMRI data, and no method allows vertex-to-vertex transformations with the temporal specificity of EEG/MEG data. Here, we introduce time-lagged multidimensional pattern connectivity (TL-MDPC) as a novel bivariate functional connectivity metric for EEG/MEG research. TL-MDPC estimates the vertex-to-vertex transformations among multiple brain regions and across different latency ranges. It determines how well patterns in ROI X at time point tx can linearly predict patterns of ROI Y at time point ty. In the present study, we use simulations to demonstrate TL-MDPC's increased sensitivity to multidimensional effects compared to a unidimensional approach across realistic choices of number of trials and signal-to-noise ratios. We applied TL-MDPC, as well as its unidimensional counterpart, to an existing dataset varying the depth of semantic processing of visually presented words by contrasting a semantic decision and a lexical decision task. TL-MDPC detected significant effects beginning very early on, and showed stronger task modulations than the unidimensional approach, suggesting that it is capable of capturing more information. With TL-MDPC only, we observed rich connectivity between core semantic representation (left and right anterior temporal lobes) and semantic control (inferior frontal gyrus and posterior temporal cortex) areas with greater semantic demands. TL-MDPC is a promising approach to identify multidimensional connectivity patterns, typically missed by unidimensional approaches.

Multiple functions of the angular gyrus at high temporal resolution

Here, the functions of the angular gyrus (AG) are evaluated in the light of current evidence from transcranial magnetic/electric stimulation (TMS/TES) and EEG/MEG studies. 65 TMS/TES and 52 EEG/MEG studies were examined in this review. TMS/TES literature points to a causal role in semantic processing, word and number processing, attention and visual search, self-guided movement, memory, and self-processing. EEG/MEG studies reported AG effects at latencies varying between 32 and 800 ms in a wide range of domains, with a high probability to detect an effect at 300-350 ms post-stimulus onset. A three-phase unifying model revolving around the process of sensemaking is then suggested: (1) early AG involvement in defining the current context, within the first 200 ms, with a bias toward the right hemisphere; (2) attention re-orientation and retrieval of relevant information within 200-500 ms; and (3) cross-modal integration at late latencies with a bias toward the left hemisphere. This sensemaking process can favour accuracy (e.g. for word and number processing) or plausibility (e.g. for comprehension and social cognition). Such functions of the AG depend on the status of other connected regions. The much-debated semantic role is also discussed as follows: (1) there is a strong TMS/TES evidence for a causal semantic role, (2) current EEG/MEG evidence is however weak, but (3) the existing arguments against a semantic role for the AG are not strong. Some outstanding questions for future research are proposed. This review recognizes that cracking the role(s) of the AG in cognition is possible only when its exact contributions within the default mode network are teased apart.

The role of the angular gyrus in semantic cognition: a synthesis of five functional neuroimaging studies

Semantic knowledge is central to human cognition. The angular gyrus (AG) is widely considered a key brain region for semantic cognition. However, the role of the AG in semantic processing is controversial. Key controversies concern response polarity (activation vs. deactivation) and its relation to task difficulty, lateralization (left vs. right AG), and functional-anatomical subdivision (PGa vs. PGp subregions). Here, we combined the fMRI data of five studies on semantic processing (n = 172) and analyzed the response profiles from the same anatomical regions-of-interest for left and right PGa and PGp. We found that the AG was consistently deactivated during non-semantic conditions, whereas response polarity during semantic conditions was inconsistent. However, the AG consistently showed relative response differences between semantic and non-semantic conditions, and between different semantic conditions. A combined analysis across all studies revealed that AG responses could be best explained by separable effects of task difficulty and semantic processing demand. Task difficulty effects were stronger in PGa than PGp, regardless of hemisphere. Semantic effects were stronger in left than right AG, regardless of subregion. These results suggest that the AG is engaged in both domain-general task-difficulty-related processes and domain-specific semantic processes. In semantic processing, we propose that left AG acts as a "multimodal convergence zone" that binds different semantic features associated with the same concept, enabling efficient access to task-relevant features.

Towards an objective evaluation of EEG/MEG source estimation methods - The linear approach

The spatial resolution of EEG/MEG source estimates, often described in terms of source leakage in the context of the inverse problem, poses constraints on the inferences that can be drawn from EEG/MEG source estimation results. Software packages for EEG/MEG data analysis offer a large choice of source estimation methods but few tools to experimental researchers for methods evaluation and comparison. Here, we describe a framework and tools for objective and intuitive resolution analysis of EEG/MEG source estimation based on linear systems analysis, and apply those to the most widely used distributed source estimation methods such as L2-minimum-norm estimation (L2-MNE) and linearly constrained minimum variance (LCMV) beamformers. Within this framework it is possible to define resolution metrics that define meaningful aspects of source estimation results (such as localization accuracy in terms of peak localization error, PLE, and spatial extent in terms of spatial deviation, SD) that are relevant to the task at hand and can easily be visualized. At the core of this framework is the resolution matrix, which describes the potential leakage from and into point sources (point-spread and cross-talk functions, or PSFs and CTFs, respectively). Importantly, for linear methods these functions allow generalizations to multiple sources or complex source distributions. This paper provides a tutorial-style introduction into linear EEG/MEG source estimation and resolution analysis aimed at experimental (rather than methods-oriented) researchers. We used this framework to demonstrate how L2-MNE-type as well as LCMV beamforming methods can be evaluated in practice using software tools that have only recently become available for routine use. Our novel methods comparison includes PLE and SD for a larger number of methods than in similar previous studies, such as unweighted, depth-weighted and normalized L2-MNE methods (including dSPM, sLORETA, eLORETA) and two LCMV beamformers. The results demonstrate that some methods can achieve low and even zero PLE for PSFs. However, their SD as well as both PLE and SD for CTFs are far less optimal for all methods, in particular for deep cortical areas. We hope that our paper will encourage EEG/MEG researchers to apply this approach to their own tasks at hand.