In defense of abstract conceptual representations

Decoding Semantics from Dynamic Brain Activation Patterns: From Trials to Task in EEG/MEG Source Space

The temporal dynamics within the semantic brain network and its dependence on stimulus and task parameters are still not well understood. Here, we addressed this by decoding task as well as stimulus information from source-estimated EEG/MEG human data. We presented the same visual word stimuli in a lexical decision (LD) and three semantic decision (SD) tasks. The meanings of the presented words varied across five semantic categories. Source space decoding was applied over time in five ROIs in the left hemisphere (anterior and posterior temporal lobe, inferior frontal gyrus, primary visual areas, and angular gyrus) and one in the right hemisphere (anterior temporal lobe). Task decoding produced sustained significant effects in all ROIs from 50 to 100 ms, both when categorizing tasks with different semantic demands (LD-SD) as well as for similar semantic tasks (SD-SD). In contrast, a semantic word category could only be decoded in lATL, rATL, PTC, and IFG, between 250 and 500 ms. Furthermore, we compared two approaches to source space decoding: conventional ROI-by-ROI decoding and combined-ROI decoding with back-projected activation patterns. The former produced more reliable results for word category decoding while the latter was more informative for task decoding. This indicates that task effects are distributed across the whole semantic network while stimulus effects are more focal. Our results demonstrate that the semantic network is widely distributed but that bilateral anterior temporal lobes together with control regions are particularly relevant for the processing of semantic information.

Experiential grounding of abstract concepts: Processing of abstract mental state concepts engages brain regions involved in mentalizing, automatic speech, and lip movements

concepts like mental state concepts lack a physical referent, which can be directly perceived. Classical theories therefore claim that abstract concepts require amodal representations detached from experiential brain systems. However, grounded cognition approaches suggest an involvement of modal experiential brain regions in the processing of abstract concepts. In the present functional magnetic resonance imaging study, we investigated the relation of the processing of abstract mental state concepts to modal experiential brain systems in a fine-grained fashion. Participants performed lexical decisions on abstract mental state as well as on verbal association concepts as control category. Experiential brain systems related to the processing of mental states, generating verbal associations, automatic speech as well as hand and lip movements were determined by corresponding localizer tasks. Processing of abstract mental state concepts neuroanatomically overlapped with activity patterns associated with processing of mental states, generating verbal associations, automatic speech and lip movements. Hence, mental state concepts activate the mentalizing brain network, complemented by perceptual-motor brain regions involved in simulation of visual or action features associated with social interactions, linguistic brain regions as well as face-motor brain regions recruited for articulation. The present results provide compelling evidence for the rich grounding of abstract mental state concepts in experiential brain systems related to mentalizing, verbal communication and mouth action.

How the intrinsic functional connectivity patterns of the semantic network support semantic processing

Semantic processing, a core of language comprehension, involves the activation of brain regions dispersed extensively across the frontal, temporal, and parietal cortices that compose the semantic network. To comprehend the functional structure of this semantic network and how it prepares for semantic processing, we investigated its intrinsic functional connectivity (FC) and the relation between this pattern and semantic processing ability in a large sample from the Human Connectome Project (HCP) dataset. We first defined a well-studied brain network for semantic processing, and then we characterized the within-network connectivity (WNC) and the between-network connectivity (BNC) within this network using a voxel-based global brain connectivity (GBC) method based on resting-state functional magnetic resonance imaging (fMRI). The results showed that 97.73% of the voxels in the semantic network displayed considerably greater WNC than BNC, demonstrating that the semantic network is a fairly encapsulated network. Moreover, multiple connector hubs in the semantic network were identified after applying the criterion of WNC > 1 SD above the mean WNC of the semantic network. More importantly, three of these connector hubs (i.e., the left anterior temporal lobe, angular gyrus, and orbital part of the inferior frontal gyrus) were reliably associated with semantic processing ability. Our findings suggest that the three identified regions use WNC as the central mechanism for supporting semantic processing and that task-independent spontaneous connectivity in the semantic network is essential for semantic processing.

The shared and unique neural correlates of personal semantic, general semantic, and episodic memory

One of the most common distinctions in long-term memory is that between semantic (i.e., general world knowledge) and episodic (i.e., recollection of contextually specific events from one's past). However, emerging cognitive neuroscience data suggest a surprisingly large overlap between the neural correlates of semantic and episodic memory. Moreover, personal semantic memories (i.e., knowledge about the self and one's life) have been studied little and do not easily fit into the standard semantic-episodic dichotomy. Here, we used fMRI to record brain activity while 48 participants verified statements concerning general facts, autobiographical facts, repeated events, and unique events. In multivariate analysis, all four types of memory involved activity within a common network bilaterally (e.g., frontal pole, paracingulate gyrus, medial frontal cortex, middle/superior temporal gyrus, precuneus, posterior cingulate, angular gyrus) and some areas of the medial temporal lobe. Yet the four memory types differentially engaged this network, increasing in activity from general to autobiographical facts, from autobiographical facts to repeated events, and from repeated to unique events. Our data are compatible with a component process model, in which declarative memory types rely on different weightings of the same elementary processes, such as perceptual imagery, spatial features, and self-reflection.

Concepts require flexible grounding

Research on semantic memory has a problem. On the one hand, a robust body of evidence implicates sensorimotor regions in conceptual processing. On the other hand, a different body of evidence implicates a modality independent semantic system. The standard solution to this tension is to posit a hub-and-spoke system with modality independent hubs and modality specific spokes. In this paper, I argue in support of an alternative view of grounding which remains committed to neural reenactment but emphasizes the multimodal and multilevel nature of the semantic system. This view is built upon the recognition that abstraction is a design feature of concepts. Semantic memory employs hierarchically structured representations to capture different degrees of abstraction. Grounding does not work the way that many embodied approaches have assumed.

Dissociable contributions of the medial parietal cortex to recognition memory

Human neuroimaging studies of episodic memory retrieval routinely observe the engagement of specific cortical regions beyond the medial temporal lobe. Of these, medial parietal cortex (MPC) is of particular interest given its ubiquitous, and yet distinct, functional characteristics during different types of retrieval tasks. Specifically, while recognition memory and autobiographical recall tasks are both used to probe episodic retrieval, these paradigms consistently drive distinct patterns of response within MPC. This dissociation adds to growing evidence suggesting a common principle of functional organization across memory related brain structures, specifically regarding the control or content demands of memory-based decisions. To carefully examine this putative organization, we used a high-resolution fMRI dataset collected at ultra-high field (7T) while subjects performed thousands of recognition-memory trials to identify MPC regions responsive to recognition-decisions or semantic content of stimuli within and across individuals. We observed interleaving, though distinct, functional subregions of MPC where responses were sensitive to either recognition decisions or the semantic representation of stimuli, but rarely both. In addition, this functional dissociation within MPC was further accentuated by distinct profiles of connectivity bias with the hippocampus during task and rest. Finally, we show that recent observations of person and place selectivity within MPC reflect category specific responses from within identified semantic regions that are sensitive to mnemonic demands. Together, these data better account for how distinct patterns of MPC responses can occur as a result of task demands during episodic retrieval and may reflect a common principle of organization throughout hippocampal-neocortical memory systems.

Conceptual representations in the default, control and attention networks are task-dependent and cross-modal

Conceptual knowledge is central to human cognition. Neuroimaging studies suggest that conceptual processing involves modality-specific and multimodal brain regions in a task-dependent fashion. However, it remains unclear (1) to what extent conceptual feature representations are also modulated by the task, (2) whether conceptual representations in multimodal regions are indeed cross-modal, and (3) how the conceptual system relates to the large-scale functional brain networks. To address these issues, we conducted multivariate pattern analyses on fMRI data. 40 participants performed three tasks-lexical decision, sound judgment, and action judgment-on written words. We found that (1) conceptual feature representations are strongly modulated by the task, (2) conceptual representations in several multimodal regions are cross-modal, and (3) conceptual feature retrieval involves the default, frontoparietal control, and dorsal attention networks. Conceptual representations in these large-scale networks are task-dependent and cross-modal. Our findings support theories that assume conceptual processing to rely on a flexible, multi-level architecture.

Modelling perception as a hierarchical competition differentiates imagined, veridical, and hallucinated percepts

Mental imagery is a process by which thoughts become experienced with sensory characteristics. Yet, it is not clear why mental images appear diminished compared to veridical images, nor how mental images are phenomenologically distinct from hallucinations, another type of non-veridical sensory experience. Current evidence suggests that imagination and veridical perception share neural resources. If so, we argue that considering how neural representations of externally generated stimuli (i.e. sensory input) and internally generated stimuli (i.e. thoughts) might interfere with one another can sufficiently differentiate between veridical, imaginary, and hallucinatory perception. We here use a simple computational model of a serially connected, hierarchical network with bidirectional information flow to emulate the primate visual system. We show that modelling even first approximations of neural competition can more coherently explain imagery phenomenology than non-competitive models. Our simulations predict that, without competing sensory input, imagined stimuli should ubiquitously dominate hierarchical representations. However, with competition, imagination should dominate high-level representations but largely fail to outcompete sensory inputs at lower processing levels. To interpret our findings, we assume that low-level stimulus information (e.g. in early visual cortices) contributes most to the sensory aspects of perceptual experience, while high-level stimulus information (e.g. towards temporal regions) contributes most to its abstract aspects. Our findings therefore suggest that ongoing bottom-up inputs during waking life may prevent imagination from overriding veridical sensory experience. In contrast, internally generated stimuli may be hallucinated when sensory input is dampened or eradicated. Our approach can explain individual differences in imagery, along with aspects of daydreaming, hallucinations, and non-visual mental imagery.

The influence of conceptual concreteness on the reading acquisition and integration of novel words into semantic memory via thematic relations

Plenty of studies have been conducted to reveal neurocognitive underpinnings of conceptual representation. Compared with that of concrete concepts, the neurocognitive correlates of abstract concepts remain elusive. The current study aimed to investigate the influence of conceptual concreteness on the reading acquisition and integration of novel words into semantic memory. We constructed two-sentence contexts in which two-character pseudowords were embedded as novel words. Participants read the contexts to infer the meaning of novel words which were either concrete or abstract, and then performed a lexical decision task and a cued-recall memory task. In lexical decision task, primed by the learned novel words, their corresponding concepts, thematically related or unrelated words as well as unlearned pseudowords were judged whether they were words or not. In memory task, participants were presented with the novel words and asked to write down their meaning. The contextual reading and memory test can demonstrate the modulation of conceptual concreteness on novel word learning and the lexical decision task can reveal whether concrete and abstract novel words are integrated into semantic memory similarly or not. During contextual reading, abstract novel words presented for the first time elicited a larger N400 than concrete ones. In memory task, the meaning of concrete novel words was recollected better than abstract novel words. These results indicate that abstract novel words are more difficult to acquire during contextual reading, and to retain afterwards. For lexical decision task behavioral and ERPs were graded, with the longest reaction time, the lowest accuracy and the largest N400s for the unrelated words, then the thematically related words and finally the corresponding concepts of the novel words, regardless of conceptual concreteness. The results suggest that both concrete and abstract novel words can be integrated into semantic memory via thematic relations. These findings are discussed in terms of differential representational framework which posits that concrete words connect with each other via semantic similarities, and abstract ones via thematic relations.

Different computational relations in language are captured by distinct brain systems

A critical way for humans to acquire information is through language, yet whether and how language experience drives specific neural semantic representations is still poorly understood. We considered statistical properties captured by 3 different computational principles of language (simple co-occurrence, network-(graph)-topological relations, and neural-network-vector-embedding relations) and tested the extent to which they can explain the neural patterns of semantic representations, measured by 2 functional magnetic resonance imaging experiments that shared common semantic processes. Distinct graph-topological word relations, and not simple co-occurrence or neural-network-vector-embedding relations, had unique explanatory power for the neural patterns in the anterior temporal lobe (capturing graph-common-neighbors), inferior frontal gyrus, and posterior middle/inferior temporal gyrus (capturing graph-shortest-path). These results were relatively specific to language: they were not explained by sensory-motor similarities and the same computational relations of visual objects (based on visual image database) showed effects in the visual cortex in the picture naming experiment. That is, different topological properties within language and the same topological computations (common-neighbors) for language and visual inputs are captured by different brain regions. These findings reveal the specific neural semantic representations along graph-topological properties of language, highlighting the information type-specific and statistical property-specific manner of semantic representations in the human brain.

The role of the angular gyrus in semantic cognition: a synthesis of five functional neuroimaging studies

Semantic knowledge is central to human cognition. The angular gyrus (AG) is widely considered a key brain region for semantic cognition. However, the role of the AG in semantic processing is controversial. Key controversies concern response polarity (activation vs. deactivation) and its relation to task difficulty, lateralization (left vs. right AG), and functional-anatomical subdivision (PGa vs. PGp subregions). Here, we combined the fMRI data of five studies on semantic processing (n = 172) and analyzed the response profiles from the same anatomical regions-of-interest for left and right PGa and PGp. We found that the AG was consistently deactivated during non-semantic conditions, whereas response polarity during semantic conditions was inconsistent. However, the AG consistently showed relative response differences between semantic and non-semantic conditions, and between different semantic conditions. A combined analysis across all studies revealed that AG responses could be best explained by separable effects of task difficulty and semantic processing demand. Task difficulty effects were stronger in PGa than PGp, regardless of hemisphere. Semantic effects were stronger in left than right AG, regardless of subregion. These results suggest that the AG is engaged in both domain-general task-difficulty-related processes and domain-specific semantic processes. In semantic processing, we propose that left AG acts as a "multimodal convergence zone" that binds different semantic features associated with the same concept, enabling efficient access to task-relevant features.

Academic training increases grounding of scientific concepts in experiential brain systems

Scientific concepts typically transcendent our sensory experiences. Traditional approaches to science education therefore assume a shift towards amodal or verbal knowledge representations during academic training. Grounded cognition approaches, in contrast, predict a maintenance of grounding of the concepts in experiential brain networks or even an increase. To test these competing approaches, the present study investigated the semantic content of scientific psychological concepts and identified the corresponding neural circuits using functional magnetic resonance imaging (fMRI) in undergraduate psychology students (beginners) and in graduated psychologists (advanced learners). During fMRI scanning, participants were presented with words denoting scientific psychological concepts within a lexical decision task (e.g. "conditioning", "habituation"). The individual semantic property content of each concept was related to brain activity during abstract concept processing. In both beginners and advanced learners, visual and motor properties activated brain regions also involved in perception and action, while mental state properties increased activity in brain regions also recruited by emotional-social scene observation. Only in advanced learners, social constellation properties elicited brain activity overlapping with emotional-social scene observation. In line with grounded cognition approaches, the present results highlight the importance of experiential information for constituting the meaning of abstract scientific concepts during the course of academic training.

Distinct roles for the anterior temporal lobe and angular gyrus in the spatiotemporal cortical semantic network

Semantic knowledge is supported by numerous brain regions, but the spatiotemporal configuration of the network that links these areas remains an open question. The hub-and-spokes model posits that a central semantic hub coordinates this network. In this study, we explored distinct aspects that define a semantic hub, as reflected in the spatiotemporal modulation of neural activity and connectivity by semantic variables, from the earliest stages of semantic processing. We used source-reconstructed electro/magnetoencephalography, and investigated the concreteness contrast across three tasks. In a whole-cortex analysis, the left anterior temporal lobe (ATL) was the only area that showed modulation of evoked brain activity from 100 ms post-stimulus. Furthermore, using Dynamic Causal Modeling of the evoked responses, we investigated effective connectivity amongst the candidate semantic hub regions, that is, left ATL, supramarginal/angular gyrus (SMG/AG), middle temporal gyrus, and inferior frontal gyrus. We found that models with a single semantic hub showed the highest Bayesian evidence, and the hub region was found to change from ATL (within 250 ms) to SMG/AG (within 450 ms) over time. Our results support a single semantic hub view, with ATL showing sustained modulation of neural activity by semantics, and both ATL and AG underlying connectivity depending on the stage of semantic processing.

Dissociation between function and manipulation in semantic representations of motor impaired subjects: A new test

A fundamental problem in semantic cognition is the representation of human concepts in the brain. Much of the knowledge acquired in the last decades comes from the study of dissociations found in patients with acquired difficulties in language, perception, and action. In particular, some deficits involve loss of knowledge about tools. The dissociation between two relevant aspects of tools, function and manipulation, has been the focus of several studies. In this paper, a new test designed to study the dissociation between function and manipulation is proposed and normative values for a control population are provided. This novel test was additionally administered to and evaluated in a group of Parkinson's disease patients. The Graded-Controlled Hub-and-Spoke model of Lambon Ralph, Jefferies, Patterson and Rogers was used as a theoretical guide to interpret the results.

How the Brain Dynamically Constructs Sentence-Level Meanings From Word-Level Features

How are words connected to the thoughts they help to express? Recent brain imaging studies suggest that word representations are embodied in different neural systems through which the words are experienced. Building on this idea, embodied approaches such as the Concept Attribute Representations (CAR) theory represents concepts as a set of semantic features (attributes) mapped to different brain systems. An intriguing challenge to this theory is that people weigh concept attributes differently based on context, i.e., they construct meaning dynamically according to the combination of concepts that occur in the sentence. This research addresses this challenge through the Context-dEpendent meaning REpresentations in the BRAin (CEREBRA) neural network model. Based on changes in the brain images, CEREBRA quantifies the effect of sentence context on word meanings. Computational experiments demonstrated that words in different contexts have different representations, the changes observed in the concept attributes reveal unique conceptual combinations, and that the new representations are more similar to the other words in the sentence than to the original representations. Behavioral analysis further confirmed that the changes produced by CEREBRA are actionable knowledge that can be used to predict human responses. These experiments constitute a comprehensive evaluation of CEREBRA's context-based representations, showing that CARs can be dynamic and change based on context. Thus, CEREBRA is a useful tool for understanding how word meanings are represented in the brain, providing a framework for future interdisciplinary research on the mental lexicon.

Differential temporo-spatial pattern of electrical brain activity during the processing of abstract concepts related to mental states and verbal associations

Refined grounded cognition accounts propose that abstract concepts might be grounded in brain circuits involved in mentalizing. In the present event-related potential (ERP) study, we compared the time course of neural processing in response to semantically predefined abstract mental states and verbal association concepts during a lexical decision task. In addition to scalp ERPs, source estimates of underlying volume brain activity were determined to reveal spatio-temporal clusters of greater electrical brain activity to abstract mental state vs. verbal association concepts, and vice versa. Source estimates suggested early (onset 194 ms), but short-lived enhanced activity (offset 210 ms) to verbal association concepts in left occipital regions. Increased occipital activity might reflect retrieval of visual word form or access to visual conceptual features of associated words. Increased estimated source activity to mental state concepts was obtained in visuo-motor (superior parietal, pre- and postcentral areas) and mentalizing networks (lateral and medial prefrontal areas, insula, precuneus, temporo-parietal junction) with an onset of 212 ms, which extended to later time windows. The time course data indicated two processing phases: An initial conceptual access phase, in which linguistic and modal brain circuits rapidly process features depending on their relevance, and a later conceptual elaboration phase, in which elaborative processing within feature-specific networks further refines the concept. This study confirms the proposal that abstract concepts are based on representations in distinct neural circuits depending on their semantic feature content. The present research also highlights the importance of investigating sets of abstract concepts with a defined semantic content.

The temporal lobes and memory

Since the first description of the case of H.M. in the mid-1950s, the debate over the contribution of the mesial temporal lobe (MTL) to human memory functioning has not ceased to stimulate new experimental work and the development of new theoretical models. The early demonstration that despite their devastating memory loss patients with hippocampal damage are still able to learn a number of visuo-motor and visuo-perceptual skills at a normal rate and to be normally primed by verbal and visual material suggested that the term "memory" is actually an umbrella concept that includes very different brain plasticity phenomena and that MTL damage actually impairs only one of these. Subsequent research, which capitalized on a detailed anatomical description of MTL structures and on the close analysis of memory-related phenomena, tried to define the unique role of the MTL structures in brain plasticity and in the government of human behavior. A first hypothesis identified this role in the conscious forms of memory as opposed to implicit ones. In the last two decades, the emphasis has moved to the relational role of the hippocampus in binding together different pieces of unimodal information to provide unitary, multimodal representations of personal experiences.

Grounding (fairly) complex numerical knowledge: an educational example

In this article, we contextualize and discuss an on-line contribution to this special issue in which a video-recorded lecture demonstrates the teaching of an abstract mathematical concept, namely regression to the mean. We first motivate the pertinence of this example from the perspective of embodied cognition. Then, we identify mechanisms of teaching that reflect embodied cognitive practices, such as the concreteness fading approach. Rather than a comprehensive review of multiple extensive literatures, this article provides the interested reader with several sources or entries into those literatures.

Does Embodiment of Verbs Influence Predicate Metaphor Processing in a Second Language? Evidence From Picture Priming

Distinct from nominal metaphors, predicate metaphors entail metaphorical abstraction from concrete verbs, which generally involve more action and stronger motor simulation than nouns. It remains unclear whether and how the concrete, embodied aspects of verbs are connected with abstract, disembodied thinking in the brains of L2 learners. Since English predicate metaphors are unfamiliar to Chinese L2 learners, the study of embodiment effect on English predicate metaphor processing may provide new evidence for embodied cognition and categorization models that remain controversial, and offer practical insights into L2 metaphor processing and pedagogy. Hence, we aim to investigate whether the embodiment of verbs, via the activation of sensorimotor information, influences two groups of L2 learners during their comprehension of conventional and novel predicate metaphors. The results show a significant effect of embodiment: a stronger facilitation for novel predicate metaphors in both higher-level and lower-level groups, and a weaker facilitation for conventional predicate metaphors in the lower-level group. The findings demonstrate preliminary evidence for a graded effect of embodiment on predicate metaphors processing, modulated by L2 proficiency and metaphor novelty. The study supports a hybrid view of embodied cognition and reveals that sensorimotor aspects of verbs may be the intermediate entity involved in the indirect categorization.

Is rushing always faster than strolling? A reaction time study on the processing of sentences containing manner of motion verbs

In the context of the embodied cognition debate, an effect of motion verb associated speed information has previously been detected using eye-tracking, functional magnetic resonance imaging (fMRI), and reaction times (RT). The latter, for instance, was implemented by Wender and Weber (1982), who observed that participants were faster in detecting motion in sentences associated with fast motion compared to sentences associated with slow motion after having formed mental images of the sentences' content. It remains open whether the reported effects of speed are associated with automatic lexical-semantic retrieval processes or whether they reflect higher top-down cognitive processes. To answer this question, the paradigm by Wender and Weber (1982) was adopted and further elaborated in the present study. In Experiment 1 visualization instructions were eliminated. Additionally, the stimulus material was manipulated in regards to the agent of the described movement (human vs. object motion) in order to determine the representation's modality (visual vs. motoric). In Experiment 2, the task to detect motion was replaced by the task to judge sensicality. The results suggest that the prompt to perform mental imagery is not a precondition for the engagement of modal representations in this speed of motion paradigm and that the involved representations' modality is visual rather than motoric. However, the modal representations' involvement is dependent on the task. They thus do not seem to be part of the invariant semantic representation of manner of motion verbs.

Dual coding of knowledge in the human brain

How does the human brain code knowledge about the world? While disciplines such as artificial intelligence represent world knowledge based on human language, neurocognitive models of knowledge have been dominated by sensory embodiment, in which knowledge is derived from sensory/motor experience and supported by high-level sensory/motor and association cortices. The neural correlates of an alternative disembodied symbolic system had previously been difficult to establish. A recent line of studies exploring knowledge about visual properties, such as color, in visually deprived individuals converge to provide positive, compelling evidence for non-sensory, language-derived, knowledge representation in dorsal anterior temporal lobe and extended language network, in addition to the sensory-derived representations, leading to a sketch of a dual-coding knowledge neural framework.

An ALE meta-analytical review of the neural correlates of abstract and concrete words

Several clinical studies have reported a double dissociation between abstract and concrete concepts, suggesting that they are processed by at least partly different networks in the brain. However, neuroimaging data seem not in line with neuropsychological reports. Using the ALE method, we run a meta-analysis on 32 brain-activation imaging studies that considered only nouns and verbs. Five clusters were associated with concrete words, four clusters with abstract words. When only nouns were selected three left activation clusters were found to be associated with concrete stimuli and only one with abstract nouns (left IFG). These results confirm that concrete and abstract words processing involves at least partially segregated brain areas, the IFG being relevant for abstract nouns and verbs while more posterior temporoparietal-occipital regions seem to be crucial for processing concrete words, in contrast with the neuropsychological literature that suggests a temporal anterior involvement for concrete words. We investigated the possible reasons that produce different outcomes in neuroimaging and clinical studies.

Emotional Valence Precedes Semantic Maturation of Words: A Longitudinal Computational Study of Early Verbal Emotional Anchoring

We present a longitudinal computational study on the connection between emotional and amodal word representations from a developmental perspective. In this study, children's and adult word representations were generated using the latent semantic analysis (LSA) vector space model and Word Maturity methodology. Some children's word representations were used to set a mapping function between amodal and emotional word representations with a neural network model using ratings from 9-year-old children. The neural network was trained and validated in the child semantic space. Then, the resulting neural network was tested with adult word representations using ratings from an adult data set. Samples of 1210 and 5315 words were used in the child and the adult semantic spaces, respectively. Results suggested that the emotional valence of words can be predicted from amodal vector representations even at the child stage, and accurate emotional propagation was found in the adult word vector representations. In this way, different propagative processes were observed in the adult semantic space. These findings highlight a potential mechanism for early verbal emotional anchoring. Moreover, different multiple linear regression and mixed-effect models revealed moderation effects for the performance of the longitudinal computational model. First, words with early maturation and subsequent semantic definition promoted emotional propagation. Second, an interaction effect between age of acquisition and abstractness was found to explain model performance. The theoretical and methodological implications are discussed.

The Effects of Handwriting Experience on Literacy Learning

Previous research indicates that writing practice may be more beneficial than nonmotor practice for letter learning. Here, we report a training study comparing typing, visual, and writing learning conditions in adults (N = 42). We investigated the behavioral consequences of learning modality on literacy learning and evaluated the nature of the learned letter representations. Specifically, the study addressed three questions. First, are the benefits of handwriting practice due to motor learning per se or to other incidental factors? Second, do the benefits generalize to untrained tasks? And third, does handwriting practice lead to learning and strengthening only of motor representations or of other types of representations as well? Our results clearly show that handwriting compared with nonmotor practice produces faster learning and greater generalization to untrained tasks than previously reported. Furthermore, only handwriting practice leads to learning of both motor and amodal symbolic letter representations.

Rapid microstructural plasticity in the cortical semantic network following a short language learning session

Despite the clear importance of language in our life, our vital ability to quickly and effectively learn new words and meanings is neurobiologically poorly understood. Conventional knowledge maintains that language learning—especially in adulthood—is slow and laborious. Furthermore, its structural basis remains unclear. Even though behavioural manifestations of learning are evident near instantly, previous neuroimaging work across a range of semantic categories has largely studied neural changes associated with months or years of practice. Here, we address rapid neuroanatomical plasticity accompanying new lexicon acquisition, specifically focussing on the learning of action-related language, which has been linked to the brain’s motor systems. Our results show that it is possible to measure and to externally modulate (using transcranial magnetic stimulation (TMS) of motor cortex) cortical microanatomic reorganisation after mere minutes of new word learning. Learning-induced microstructural changes, as measured by diffusion kurtosis imaging (DKI) and machine learning-based analysis, were evident in prefrontal, temporal, and parietal neocortical sites, likely reflecting integrative lexico-semantic processing and formation of new memory circuits immediately during the learning tasks. These results suggest a structural basis for the rapid neocortical word encoding mechanism and reveal the causally interactive relationship of modal and associative brain regions in supporting learning and word acquisition.

This combined neuroimaging and brain stimulation study reveals rapid and distributed microstructural plasticity after a single immersive language learning session, demonstrating the causal relevance of the motor cortex in encoding the meaning of novel action words.

Perceptual and Interoceptive Strength Norms for 270 French Words

Perceptual experience through the five modalities (i.e., vision, hearing, touch, taste, and smell) has demonstrated its key role in semantics. Researchers also highlighted the role of interoceptive information in the grounded representation of concepts. However, to this day, there is no available data across these modalities in the French language. Therefore, the aim of this study was to circumvent this caveat. Participants aged between 18 and 50 completed an online survey in which we recorded scores of perceptual strength (PS), interoceptive information, imageability, concreteness, conceptual familiarity, and age of acquisition of 270 words of the French language. We also analysed the relationships between perceptual modalities and psycholinguistic variables. Results showed that vast majority of concepts were visually-dominant. Correlation analyses revealed that the five PS variables were strongly correlated with imageability, concreteness, and conceptual familiarity and highlight that PS variables index one aspect of the semantic representations of a word. On the other hand, high interoceptive scores were highlighted only for the less imageable and less concrete words, emphasizing their importance for the grounding of abstract concepts. Future research could use these norms in the investigation of the role of perceptual experience in the representation of concepts and their impact on word processing.

Task-Dependent Functional and Effective Connectivity during Conceptual Processing

Conceptual knowledge is central to cognition. Previous neuroimaging research indicates that conceptual processing involves both modality-specific perceptual-motor areas and multimodal convergence zones. For example, our previous functional magnetic resonance imaging (fMRI) study revealed that both modality-specific and multimodal regions respond to sound and action features of concepts in a task-dependent fashion (Kuhnke P, Kiefer M, Hartwigsen G. 2020b. Task-dependent recruitment of modality-specific and multimodal regions during conceptual processing. Cereb Cortex. 30:3938–3959.). However, it remains unknown whether and how modality-specific and multimodal areas interact during conceptual tasks. Here, we asked 1) whether multimodal and modality-specific areas are functionally coupled during conceptual processing, 2) whether their coupling depends on the task, 3) whether information flows top-down, bottom-up or both, and 4) whether their coupling is behaviorally relevant. We combined psychophysiological interaction analyses with dynamic causal modeling on the fMRI data of our previous study. We found that functional coupling between multimodal and modality-specific areas strongly depended on the task, involved both top-down and bottom-up information flow, and predicted conceptually guided behavior. Notably, we also found coupling between different modality-specific areas and between different multimodal areas. These results suggest that functional coupling in the conceptual system is extensive, reciprocal, task-dependent, and behaviorally relevant. We propose a new model of the conceptual system that incorporates task-dependent functional interactions between modality-specific and multimodal areas.

At the Neural Intersection Between Language and Emotion

What role does language play in emotion? Behavioral research shows that emotion words such as "anger" and "fear" alter emotion experience, but questions still remain about mechanism. Here, we review the neuroscience literature to examine whether neural processes associated with semantics are also involved in emotion. Our review suggests that brain regions involved in the semantic processing of words: (i) are engaged during experiences of emotion, (ii) coordinate with brain regions involved in affect to create emotions, (iii) hold representational content for emotion, and (iv) may be necessary for constructing emotional experience. We relate these findings with respect to four theoretical relationships between language and emotion, which we refer to as "non-interactive," "interactive," "constitutive," and "deterministic." We conclude that findings are most consistent with the interactive and constitutive views with initial evidence suggestive of a constitutive view, in particular. We close with several future directions that may help test hypotheses of the constitutive view.

Heterogeneity in abstract verbs: An ERP study

It has been well documented that different types of nouns and action verbs are associated with behavioral and neural differences. In contrast, abstract verbs (e.g., think, dissolve) are often treated as a homogeneous category. We compared event-related potentials recorded during a syntactic classification task of four verb types; 1) abstract mental, 2) abstract emotional, 3) abstract nonbodily, and 4) concrete. Abstract nonbodily state verbs showed a sustained negativity at frontocentral electrodes and sustained positivity at parietal and occipital electrodes beginning 400 ms post-stimulus onset relative to abstract mental state and concrete verbs. Discrete source localization revealed a right inferior parietal source for all verbs and a distributed source estimation localized sources that distinguished between abstract mental state and abstract nonbodily state verbs to bilateral parietal cortex, left temporal cortex and right ventromedial prefrontal cortex. These findings suggest that different types of abstract verbs are associated with representational differences.

Left posterior inferior parietal cortex causally supports the retrieval of action knowledge

Conceptual knowledge is central to human cognition. The left posterior inferior parietal lobe (pIPL) is implicated by neuroimaging studies as a multimodal hub representing conceptual knowledge related to various perceptual-motor modalities. However, the causal role of left pIPL in conceptual processing remains unclear. Here, we transiently disrupted left pIPL function with transcranial magnetic stimulation (TMS) to probe its causal relevance for the retrieval of action and sound knowledge. We compared effective TMS over left pIPL with sham TMS, while healthy participants performed three different tasks-lexical decision, action judgment, and sound judgment-on words with a high or low association to actions and sounds. We found that pIPL-TMS selectively impaired action judgments on low sound-low action words. For the first time, we directly related computational simulations of the TMS-induced electrical field to behavioral performance, which revealed that stronger stimulation of left pIPL is associated with worse performance for action but not sound judgments. These results indicate that left pIPL causally supports conceptual processing when action knowledge is task-relevant and cannot be compensated by sound knowledge. Our findings suggest that left pIPL is specialized for the retrieval of action knowledge, challenging the view of left pIPL as a multimodal conceptual hub.

Challenges and Opportunities for Grounding Cognition

According to the grounded perspective, cognition emerges from the interaction of classic cognitive processes with the modalities, the body, and the environment. Rather than being an autonomous impenetrable module, cognition incorporates these other domains intrinsically into its operation. The Situated Action Cycle offers one way of understanding how the modalities, the body, and the environment become integrated to ground cognition. Seven challenges and opportunities are raised for this perspective: (1) How does cognition emerge from the Situated Action Cycle and in turn support it? (2) How can we move beyond simply equating embodiment with action, additionally establishing how embodiment arises in the autonomic, neuroendocrine, immune, cardiovascular, respiratory, digestive, and integumentary systems? (3) How can we better understand the mechanisms underlying multimodal simulation, its functions across the Situated Action Cycle, and its integration with other representational systems? (4) How can we develop and assess theoretical accounts of symbolic processing from the grounded perspective (perhaps using the construct of simulators)? (5) How can we move beyond the simplistic distinction between concrete and abstract concepts, instead addressing how concepts about the external and internal worlds pattern to support the Situated Action Cycle? (6) How do individual differences emerge from different populations of situational memories as the Situated Action Cycle manifests itself differently across individuals? (7) How can constructs from grounded cognition provide insight into the replication and generalization crises, perhaps from a quantum perspective on mechanisms (as exemplified by simulators).

Transient Neural Activation of Abstract Relations on an Incidental Analogy Task

Although a large proportion of the lexicon consists of abstract concepts, little is known about how they are represented by the brain. Here, we investigated how the mind represents relations shared between sets of mental representations that are superficially unrelated, such as car-engine and dog-tongue, but that nonetheless share a more general, abstract relation, such as whole-part. Participants saw a pair of words on each trial and were asked to indicate whether they could think of a relation between them. Importantly, they were not explicitly asked whether different word pairs shared the same relation, as in analogical reasoning tasks. We observed representational similarity for abstract relations in regions in the "conceptual hub" network, even when controlling for semantic relatedness between word pairs. By contrast, we did not observe representational similarity in regions previously implicated in explicit analogical reasoning. A given relation was sometimes repeated across sequential word pairs, allowing us to test for behavioral and neural priming of abstract relations. Indeed, we observed faster RTs and greater representational similarity for primed than unprimed trials, suggesting that mental representations of abstract relations are transiently activated on this incidental analogy task. Finally, we found a significant correlation between behavioral and neural priming across participants. To our knowledge, this is the first study to investigate relational priming using functional neuroimaging and to show that neural representations are strengthened by relational priming. This research shows how abstract concepts can be brought to mind momentarily, even when not required for task performance.

Relational Integration in the Human Brain: A Review and Synthesis

Relational integration is required when multiple explicit representations of relations between entities must be jointly considered to make inferences. We provide an overview of the neural substrate of relational integration in humans and the processes that support it, focusing on work on analogical and deductive reasoning. In addition to neural evidence, we consider behavioral and computational work that has informed neural investigations of the representations of individual relations and of relational integration. In very general terms, evidence from neuroimaging, neuropsychological, and neuromodulatory studies points to a small set of regions (generally left lateralized) that appear to constitute key substrates for component processes of relational integration. These include posterior parietal cortex, implicated in the representation of first-order relations (e.g., A:B); rostrolateral pFC, apparently central in integrating first-order relations so as to generate and/or evaluate higher-order relations (e.g., A:B::C:D); dorsolateral pFC, involved in maintaining relations in working memory; and ventrolateral pFC, implicated in interference control (e.g., inhibiting salient information that competes with relevant relations). Recent work has begun to link computational models of relational representation and reasoning with patterns of neural activity within these brain areas.

How Abstract (Non-embodied) Linguistic Representations Augment Cognitive Control

Recent scholarship emphasizes the scaffolding role of language for cognition. Language, it is claimed, is a cognition-enhancing niche (Clark, 2006), a programming tool for cognition (Lupyan and Bergen, 2016), even neuroenhancement (Dove, 2019) and augments cognitive functions such as memory, categorization, cognitive control, and meta-cognitive abilities ("thinking about thinking"). Yet, the notion that language enhances or augments cognition, and in particular, cognitive control does not easily fit in with embodied approaches to language processing, or so we will argue. Accounts aiming to explain how language enhances various cognitive functions often employ a notion of abstract representation. Yet, embodied approaches to language processing have it that language processing crucially, according to some accounts even exclusively, involves embodied, modality-specific, i.e., non-abstract representations. In coming to understand a particular phrase or sentence, a prior experience has to be simulated or reenacted. The representation thus activated is embodied (modality-specific) as sensorimotor regions of the brain are thereby recruited. In this paper, we will first discuss the notion of representation, clarify what it takes for a representation to be embodied or abstract, and distinguish between conceptual and (other) linguistic representations. We will then put forward a characterization of cognitive control and examine its representational infrastructure. The remainder of the paper will be devoted to arguing that language augments cognitive control. To that end, we will draw on two lines of research, which investigate how language augments cognitive control: (i) research on the availability of linguistic labels and (ii) research on the active usage of a linguistic code, specifically, in inner speech. Eventually, we will argue that the cognition-enhancing capacity of language can be explained once we assume that it provides us with (a) abstract, non-embodied representations and with (b) abstract, sparse linguistic representations that may serve as easy-to-manipulate placeholders for fully embodied or otherwise more detailed representations.

Interactions between language, thought, and perception: Cognitive and neural perspectives

ABSTRACT The role that language plays in shaping non-linguistic cognitive and perceptual systems has been the subject of much theoretical and experimental attention over the past half-century. Understanding how language interacts with non-linguistic systems can provide insight into broader constraints on cognitive and brain organization. The papers that form this volume investigate various ways in which linguistic structure can interact with and influence how speakers think about and perceive the world, and the related issue of the constraints that in turn shape linguistic representations. These theoretical and empirical contributions support deeper understanding of the interactions between language, thought, and perception, and motivate new approaches for developing directional predictions at both the neural and cognitive levels.

Morality in advertising: An fMRI study on persuasion in communication

Advertising slogans serve the function of persuasive communication by presenting catchy phrases. To decide whether a slogan is convincing or not, cognitive reasoning is assumed to be complemented by a more implicit and intuitive route of information processing, presumably similar to evaluating normative judgments in moral statements. We employed functional magnetic resonance imaging (fMRI) while Western male subjects judged advertising slogans and moral statements as another decision task with subjective nature. Compared to a neutral control condition that targeted declarative memory and to an aesthetic-related condition, the evaluation processes in both domains engaged the anterior medial prefrontal cortex (mPFC), which is associated with decision-making incorporating personal value. Conjoint activations were also observed in the left temporoparietal junction (TPJ) when compared to the aesthetics condition. Results are discussed with reference to domain-independence, a suspected difference to aesthetic-like appreciations, and functional organization in the mPFC and the TPJ.

Brain Activation During Conceptual Processing of Action and Sound Verbs

Grounded cognition approaches to conceptual representations postulate a close link between conceptual knowledge and the sensorimotor brain systems. The present fMRI study tested, whether a feature-specific representation of concepts, as previously demonstrated for nouns, can also be found for action- and sound-related verbs. Participants were presented with action- and soundrelated verbs along with pseudoverbs while performing a lexical decision task. Sound-related verbs activated auditory areas in the temporal cortex, whereas action-related verbs activated brain regions in the superior frontal gyrus and the cerebellum, albeit only at a more liberal threshold. This differential brain activation during conceptual verb processing partially overlapped with or was adjacent to brain regions activated during the functional localizers probing sound perception or action execution. Activity in brain areas involved in the processing of action information was parametrically modulated by ratings of action relevance. Comparisons of action- and sound-related verbs with pseudoverbs revealed activation for both verb categories in auditory and motor areas. In contrast to proposals of strong grounded cognition approaches, our study did not demonstrate a considerable overlap of activations for action- and sound-related verbs and for the corresponding functional localizer tasks. However, in line with weaker variants of grounded cognition theories, the differential activation pattern for action- and sound-related verbs was near corresponding sensorimotor brain regions depending on conceptual feature relevance. Possibly, action-sound coupling resulted in a mutual activation of the motor and the auditory system for both action- and sound-related verbs, thereby reducing the effect sizes for the differential contrasts.

No title, no theme: The joined neural space between speakers and listeners during production and comprehension of multi-sentence discourse

Speakers and listeners usually interact in larger discourses than single words or even single sentences. The goal of the present study was to identify the neural bases reflecting how the mental representation of the situation denoted in a multi-sentence discourse (situation model) is constructed and shared between speakers and listeners. An fMRI study using a variant of the ambiguous text paradigm was designed. Speakers (n = 15) produced ambiguous texts in the scanner and listeners (n = 27) subsequently listened to these texts in different states of ambiguity: preceded by a highly informative, intermediately informative or no title at all. Conventional BOLD activation analyses in listeners, as well as inter-subject correlation analyses between the speakers' and the listeners' hemodynamic time courses were performed. Critically, only the processing of disambiguated, coherent discourse with an intelligible situation model representation involved (shared) activation in bilateral lateral parietal and medial prefrontal regions. This shared spatiotemporal pattern of brain activation between the speaker and the listener suggests that the process of memory retrieval in medial prefrontal regions and the binding of retrieved information in the lateral parietal cortex constitutes a core mechanism underlying the communication of complex conceptual representations.

Task-Dependent Recruitment of Modality-Specific and Multimodal Regions during Conceptual Processing

Conceptual knowledge is central to cognitive abilities such as word comprehension. Previous neuroimaging evidence indicates that concepts are at least partly composed of perceptual and motor features that are represented in the same modality-specific brain regions involved in actual perception and action. However, it is unclear to what extent the retrieval of perceptual-motor features and the resulting engagement of modality-specific regions depend on the concurrent task. To address this issue, we measured brain activity in 40 young and healthy participants using functional magnetic resonance imaging, while they performed three different tasks-lexical decision, sound judgment, and action judgment-on words that independently varied in their association with sounds and actions. We found neural activation for sound and action features of concepts selectively when they were task-relevant in brain regions also activated during auditory and motor tasks, respectively, as well as in higher-level, multimodal regions which were recruited during both sound and action feature retrieval. For the first time, we show that not only modality-specific perceptual-motor areas but also multimodal regions are engaged in conceptual processing in a flexible, task-dependent fashion, responding selectively to task-relevant conceptual features.

Speaking in the Brain: The Interaction between Words and Syntax in Sentence Production

This neuroimaging study investigated the neural infrastructure of sentence-level language production. We compared brain activation patterns, as measured with BOLD-fMRI, during production of sentences that differed in verb argument structures (intransitives, transitives, ditransitives) and the lexical status of the verb (known verbs or pseudoverbs). The experiment consisted of 30 mini-blocks of six sentences each. Each mini-block started with an example for the type of sentence to be produced in that block. On each trial in the mini-blocks, participants were first given the (pseudo-)verb followed by three geometric shapes to serve as verb arguments in the sentences. Production of sentences with known verbs yielded greater activation compared to sentences with pseudoverbs in the core language network of the left inferior frontal gyrus, the left posterior middle temporal gyrus, and a more posterior middle temporal region extending into the angular gyrus, analogous to effects observed in language comprehension. Increasing the number of verb arguments led to greater activation in an overlapping left posterior middle temporal gyrus/angular gyrus area, particularly for known verbs, as well as in the bilateral precuneus. Thus, producing sentences with more complex structures using existing verbs leads to increased activation in the language network, suggesting some reliance on memory retrieval of stored lexical-syntactic information during sentence production. This study thus provides evidence from sentence-level language production in line with functional models of the language network that have so far been mainly based on single-word production, comprehension, and language processing in aphasia.

Connecting concepts in the brain by mapping cortical representations of semantic relations

In the brain, the semantic system is thought to store concepts. However, little is known about how it connects different concepts and infers semantic relations. To address this question, we collected hours of functional magnetic resonance imaging data from human subjects listening to natural stories. We developed a predictive model of the voxel-wise response and further applied it to thousands of new words. Our results suggest that both semantic categories and relations are represented by spatially overlapping cortical patterns, instead of anatomically segregated regions. Semantic relations that reflect conceptual progression from concreteness to abstractness are represented by cortical patterns of activation in the default mode network and deactivation in the frontoparietal attention network. We conclude that the human brain uses distributed networks to encode not only concepts but also relationships between concepts. In particular, the default mode network plays a central role in semantic processing for abstraction of concepts.

Overlapping connectivity patterns during semantic processing of abstract and concrete words revealed with multivariate Granger Causality analysis

. Abstract, unlike concrete, nouns refer to notions beyond our perception. Even though there is no consensus among linguists as to what exactly constitutes a concrete or abstract word, neuroscientists found clear evidence of a "concreteness" effect. This can, for instance, be seen in patients with language impairments due to brain injury or developmental disorder who are capable of perceiving one category better than another. Even though the results are inconclusive, neuroimaging studies on healthy subjects also provide a spatial and temporal account of differences in the processing of abstract versus concrete words. A description of the neural pathways during abstract word reading, the manner in which the connectivity patterns develop over the different stages of lexical and semantic processing compared to that of concrete word processing are still debated. We conducted a high-density EEG study on 24 healthy young volunteers using an implicit categorization task. From this, we obtained high spatio-temporal resolution data and, by means of source reconstruction, reduced the effect of signal mixing observed on scalp level. A multivariate, time-varying and directional method of analyzing connectivity based on the concept of Granger Causality (Partial Directed Coherence) revealed a dynamic network that transfers information from the right superior occipital lobe along the ventral and dorsal streams towards the anterior temporal and orbitofrontal lobes of both hemispheres. Some regions along these pathways appear to be primarily involved in either receiving or sending information. A clear difference in information transfer of abstract and concrete words was observed during the time window of semantic processing, specifically for information transferred towards the left anterior temporal lobe. Further exploratory analysis confirmed a generally stronger connectivity pattern for processing concrete words. We believe our study could guide future research towards a more refined theory of abstract word processing in the brain.

Brain Regions Involved in Conceptual Retrieval in Sighted and Blind People

If conceptual retrieval is partially based on the simulation of sensorimotor experience, people with a different sensorimotor experience, such as congenitally blind people, should retrieve concepts in a different way. However, studies investigating the neural basis of several conceptual domains (e.g., actions, objects, places) have shown a very limited impact of early visual deprivation. We approached this problem by investigating brain regions that encode the perceptual similarity of action and color concepts evoked by spoken words in sighted and congenitally blind people. At first, and in line with previous findings, a contrast between action and color concepts (independently of their perceptual similarity) revealed similar activations in sighted and blind people for action concepts and partially different activations for color concepts, but outside visual areas. On the other hand, adaptation analyses based on subjective ratings of perceptual similarity showed compelling differences across groups. Perceptually similar colors and actions induced adaptation in the posterior occipital cortex of sighted people only, overlapping with regions known to represent low-level visual features of those perceptual domains. Early-blind people instead showed a stronger adaptation for perceptually similar concepts in temporal regions, arguably indexing higher reliance on a lexical-semantic code to represent perceptual knowledge. Overall, our results show that visual deprivation does changes the neural bases of conceptual retrieval, but mostly at specific levels of representation supporting perceptual similarity discrimination, reconciling apparently contrasting findings in the field.