Neurobiological Mechanisms for Semantic Feature Extraction and Conceptual Flexibility

When abstract becomes concrete, naturalistic encoding of concepts in the brain

Language is acquired and processed in complex and dynamic naturalistic contexts, involving the simultaneous processing of connected speech, faces, bodies, objects, etc. How words and their associated concepts are encoded in the brain during real-world processing is still unknown. Here, the representational structure of concrete and abstract concepts was investigated during movie watching to address the extent to which brain responses dynamically change depending on visual context. First, across contexts, concrete and abstract concepts are shown to encode different experience-based information in separable sets of brain regions. However, these differences are reduced when multimodal context is considered. Specifically, the response profile of abstract words becomes more concrete-like when these are processed in visual scenes highly related to their meaning. Conversely, when the visual context is unrelated to a given concrete word, the activation pattern resembles more that of abstract conceptual processing. These results suggest that while concepts generally encode habitual experiences, the underlying neurobiological organisation is not fixed but depends dynamically on available contextual information.

Experiential grounding of abstract concepts: Processing of abstract mental state concepts engages brain regions involved in mentalizing, automatic speech, and lip movements

concepts like mental state concepts lack a physical referent, which can be directly perceived. Classical theories therefore claim that abstract concepts require amodal representations detached from experiential brain systems. However, grounded cognition approaches suggest an involvement of modal experiential brain regions in the processing of abstract concepts. In the present functional magnetic resonance imaging study, we investigated the relation of the processing of abstract mental state concepts to modal experiential brain systems in a fine-grained fashion. Participants performed lexical decisions on abstract mental state as well as on verbal association concepts as control category. Experiential brain systems related to the processing of mental states, generating verbal associations, automatic speech as well as hand and lip movements were determined by corresponding localizer tasks. Processing of abstract mental state concepts neuroanatomically overlapped with activity patterns associated with processing of mental states, generating verbal associations, automatic speech and lip movements. Hence, mental state concepts activate the mentalizing brain network, complemented by perceptual-motor brain regions involved in simulation of visual or action features associated with social interactions, linguistic brain regions as well as face-motor brain regions recruited for articulation. The present results provide compelling evidence for the rich grounding of abstract mental state concepts in experiential brain systems related to mentalizing, verbal communication and mouth action.

Breakdown of category-specific word representations in a brain-constrained neurocomputational model of semantic dementia

The neurobiological nature of semantic knowledge, i.e., the encoding and storage of conceptual information in the human brain, remains a poorly understood and hotly debated subject. Clinical data on semantic deficits and neuroimaging evidence from healthy individuals have suggested multiple cortical regions to be involved in the processing of meaning. These include semantic hubs (most notably, anterior temporal lobe, ATL) that take part in semantic processing in general as well as sensorimotor areas that process specific aspects/categories according to their modality. Biologically inspired neurocomputational models can help elucidate the exact roles of these regions in the functioning of the semantic system and, importantly, in its breakdown in neurological deficits. We used a neuroanatomically constrained computational model of frontotemporal cortices implicated in word acquisition and processing, and adapted it to simulate and explain the effects of semantic dementia (SD) on word processing abilities. SD is a devastating, yet insufficiently understood progressive neurodegenerative disease, characterised by semantic knowledge deterioration that is hypothesised to be specifically related to neural damage in the ATL. The behaviour of our brain-based model is in full accordance with clinical data-namely, word comprehension performance decreases as SD lesions in ATL progress, whereas word repetition abilities remain less affected. Furthermore, our model makes predictions about lesion- and category-specific effects of SD: our simulation results indicate that word processing should be more impaired for object- than for action-related words, and that degradation of white matter should produce more severe consequences than the same proportion of grey matter decay. In sum, the present results provide a neuromechanistic explanatory account of cortical-level language impairments observed during the onset and progress of semantic dementia.

Neurobiological mechanisms for language, symbols and concepts: Clues from brain-constrained deep neural networks

Neural networks are successfully used to imitate and model cognitive processes. However, to provide clues about the neurobiological mechanisms enabling human cognition, these models need to mimic the structure and function of real brains. Brain-constrained networks differ from classic neural networks by implementing brain similarities at different scales, ranging from the micro- and mesoscopic levels of neuronal function, local neuronal links and circuit interaction to large-scale anatomical structure and between-area connectivity. This review shows how brain-constrained neural networks can be applied to study in silico the formation of mechanisms for symbol and concept processing and to work towards neurobiological explanations of specifically human cognitive abilities. These include verbal working memory and learning of large vocabularies of symbols, semantic binding carried by specific areas of cortex, attention focusing and modulation driven by symbol type, and the acquisition of concrete and abstract concepts partly influenced by symbols. Neuronal assembly activity in the networks is analyzed to deliver putative mechanistic correlates of higher cognitive processes and to develop candidate explanations founded in established neurobiological principles.

Ecological Meanings: A Consensus Paper on Individual Differences and Contextual Influences in Embodied Language

Embodied theories of cognition consider many aspects of language and other cognitive domains as the result of sensory and motor processes. In this view, the appraisal and the use of concepts are based on mechanisms of simulation grounded on prior sensorimotor experiences. Even though these theories continue receiving attention and support, increasing evidence indicates the need to consider the flexible nature of the simulation process, and to accordingly refine embodied accounts. In this consensus paper, we discuss two potential sources of variability in experimental studies on embodiment of language: individual differences and context. Specifically, we show how factors contributing to individual differences may explain inconsistent findings in embodied language phenomena. These factors include sensorimotor or cultural experiences, imagery, context-related factors, and cognitive strategies. We also analyze the different contextual modulations, from single words to sentences and narratives, as well as the top-down and bottom-up influences. Similarly, we review recent efforts to include cultural and language diversity, aging, neurodegenerative diseases, and brain disorders, as well as bilingual evidence into the embodiment framework. We address the importance of considering individual differences and context in clinical studies to drive translational research more efficiently, and we indicate recommendations on how to correctly address these issues in future research. Systematically investigating individual differences and context may contribute to understanding the dynamic nature of simulation in language processes, refining embodied theories of cognition, and ultimately filling the gap between cognition in artificial experimental settings and cognition in the wild (i.e., in everyday life).

Grounded Cognition Entails Linguistic Relativity: A Neglected Implication of a Major Semantic Theory

According to the popular Grounded Cognition Model (GCM), the sensory and motor features of concepts, including word meanings, are stored directly within neural systems for perception and action. More precisely, the core claim is that these concrete conceptual features reuse some of the same modality-specific representations that serve to categorize experiences involving the relevant kinds of objects and events. Research in semantic typology, however, has shown that word meanings vary significantly across the roughly 6500 languages in the world. I argue that this crosslinguistic semantic diversity has significant yet previously unrecognized theoretical consequences for the GCM. In particular, to accommodate the typological data, the GCM must assume that the concrete features of word meanings are not merely stored within sensory/motor brain systems, but are represented there in ways that are, to a nontrivial degree, language-specific. Moreover, it must assume that these conceptual representations are also activated during the nonlinguistic processing of the relevant kinds of objects and events (e.g., during visual perception and action planning); otherwise, they would not really be grounded, which is to say, embedded inside sensory/motor systems. Crucially, however, such activations would constitute what is traditionally called linguistic relativity-that is, the influence of language-specific semantic structures on other forms of cognition. The overarching aim of this paper is to elaborate this argument more fully and explore its repercussions. To that end, I discuss in greater detail the key aspects of the GCM, the evidence for crosslinguistic semantic diversity, pertinent work on linguistic relativity, the central claim that the GCM entails linguistic relativity, some initial supporting results, and some important limitations and future directions.

Concepts require flexible grounding

Research on semantic memory has a problem. On the one hand, a robust body of evidence implicates sensorimotor regions in conceptual processing. On the other hand, a different body of evidence implicates a modality independent semantic system. The standard solution to this tension is to posit a hub-and-spoke system with modality independent hubs and modality specific spokes. In this paper, I argue in support of an alternative view of grounding which remains committed to neural reenactment but emphasizes the multimodal and multilevel nature of the semantic system. This view is built upon the recognition that abstraction is a design feature of concepts. Semantic memory employs hierarchically structured representations to capture different degrees of abstraction. Grounding does not work the way that many embodied approaches have assumed.

Predictive language comprehension in Parkinson's disease

Verb and action knowledge deficits are reported in persons with Parkinson's disease (PD), even in the absence of dementia or mild cognitive impairment. However, the impact of these deficits on combinatorial semantic processing is less well understood. Following on previous verb and action knowledge findings, we tested the hypothesis that PD impairs the ability to integrate event-based thematic fit information during online sentence processing. Specifically, we anticipated persons with PD with age-typical cognitive abilities would perform more poorly than healthy controls during a visual world paradigm task requiring participants to predict a target object constrained by the thematic fit of the agent-verb combination. Twenty-four PD and 24 healthy age-matched participants completed comprehensive neuropsychological assessments. We recorded participants' eye movements as they heard predictive sentences (The fisherman rocks the boat) alongside target, agent-related, verb-related, and unrelated images. We tested effects of group (PD/control) on gaze using growth curve models. There were no significant differences between PD and control participants, suggesting that PD participants successfully and rapidly use combinatory thematic fit information to predict upcoming language. Baseline sentences with no predictive information (e.g., Look at the drum) confirmed that groups showed equivalent sentence processing and eye movement patterns. Additionally, we conducted an exploratory analysis contrasting PD and controls' performance on low-motion-content versus high-motion-content verbs. This analysis revealed fewer predictive fixations in high-motion sentences only for healthy older adults. PD participants may adapt to their disease by relying on spared, non-action-simulation-based language processing mechanisms, although this conclusion is speculative, as the analyses of high- vs. low-motion items was highly limited by the study design. These findings provide novel evidence that individuals with PD match healthy adults in their ability to use verb meaning to predict upcoming nouns despite previous findings of verb semantic impairment in PD across a variety of tasks.

Motor imagery training to improve language processing: What are the arguments?

Studies showed that motor expertise was found to induce improvement in language processing. Grounded and situated approaches attributed this effect to an underlying automatic simulation of the motor experience elicited by action words, similar to motor imagery (MI), and suggest shared representations of action conceptualization. Interestingly, recent results also suggest that the mental simulation of action by MI training induces motor-system modifications and improves motor performance. Consequently, we hypothesize that, since MI training can induce motor-system modifications, it could be used to reinforce the functional connections between motor and language system, and could thus lead to improved language performance. Here, we explore these potential interactions by reviewing recent fundamental and clinical literature in the action-language and MI domains. We suggested that exploiting the link between action language and MI could open new avenues for complementary language improvement programs. We summarize the current literature to evaluate the rationale behind this novel training and to explore the mechanisms underlying MI and its impact on language performance.

Neural correlates of embodied action language processing: a systematic review and meta-analytic study

The neural correlates of action language processing are still debated within embodied cognition research and little is known about the flexible involvement of modality-specific pre-motor system and multimodal high-level temporo-parietal regions as a function of explicit and implicit tasks. A systematic review and the Activation likelihood estimation (ALE) meta-analyses on functional neuroimaging studies were performed to identify neural correlates of action language processing activated during explicit and implicit tasks. The contrast ALE meta-analysis revealed activation of modality-specific premotor area and inferior frontal areas during explicit action language tasks while a greater activation of posterior temporo-occipital areas emerged for implicit tasks. The conjunction analysis revealed overlap in the temporo-parietal multimodal high-level regions for both types of tasks. Functional specialization of the middle temporal gyrus was found where the more posterior-occipital part resulted activated during implicit action language tasks whereas the antero-lateral part was involved in explicit tasks. Our findings were discussed within a conceptual flexibility perspective about the involvement of both the modality-specific and multimodal brain system during action language processing depending on different types of tasks.

Kinesthetic motor-imagery training improves performance on lexical-semantic access

The objective of this study was to evaluate the effect of Motor Imagery (MI) training on language comprehension. In line with literature suggesting an intimate relationship between the language and the motor system, we proposed that a MI-training could improve language comprehension by facilitating lexico-semantic access. In two experiments, participants were assigned to a kinesthetic motor-imagery training (KMI) group, in which they had to imagine making upper-limb movements, or to a static visual imagery training (SVI) group, in which they had to mentally visualize pictures of landscapes. Differential impacts of both training protocols on two different language comprehension tasks (i.e., semantic categorization and sentence-picture matching task) were investigated. Experiment 1 showed that KMI training can induce better performance (shorter reaction times) than SVI training for the two language comprehension tasks, thus suggesting that a KMI-based motor activation can facilitate lexico-semantic access after only one training session. Experiment 2 aimed at replicating these results using a pre/post-training language assessment and a longer training period (four training sessions spread over four days). Although the improvement magnitude between pre- and post-training sessions was greater in the KMI group than in the SVI one on the semantic categorization task, the sentence-picture matching task tended to provide an opposite pattern of results. Overall, this series of experiments highlights for the first time that motor imagery can contribute to the improvement of lexical-semantic processing and could open new avenues on rehabilitation methods for language deficits.

Research on Feature Extraction and Chinese Translation Method of Internet-of-Things English Terminology

Feature extraction and Chinese translation of Internet-of-Things English terms are the basis of many natural language processing. Its main purpose is to extract rich semantic information from unstructured texts to allow computers to further calculate and process them to meet different types of NLP-based tasks. However, most of the current methods use simple neural network models to count the word frequency or probability of words in the text, and it is difficult to accurately understand and translate IoT English terms. In response to this problem, this study proposes a neural network for feature extraction and Chinese translation of IoT English terms based on LSTM, which can not only correctly extract and translate IoT English vocabulary but also realize the feature correspondence between English and Chinese. The neural network proposed in this study has been tested and trained on multiple datasets, and it basically fulfills the requirements of feature translation and Chinese translation of Internet-of-Things terms in English and has great potential in the follow-up work.

Differential temporo-spatial pattern of electrical brain activity during the processing of abstract concepts related to mental states and verbal associations

Refined grounded cognition accounts propose that abstract concepts might be grounded in brain circuits involved in mentalizing. In the present event-related potential (ERP) study, we compared the time course of neural processing in response to semantically predefined abstract mental states and verbal association concepts during a lexical decision task. In addition to scalp ERPs, source estimates of underlying volume brain activity were determined to reveal spatio-temporal clusters of greater electrical brain activity to abstract mental state vs. verbal association concepts, and vice versa. Source estimates suggested early (onset 194 ms), but short-lived enhanced activity (offset 210 ms) to verbal association concepts in left occipital regions. Increased occipital activity might reflect retrieval of visual word form or access to visual conceptual features of associated words. Increased estimated source activity to mental state concepts was obtained in visuo-motor (superior parietal, pre- and postcentral areas) and mentalizing networks (lateral and medial prefrontal areas, insula, precuneus, temporo-parietal junction) with an onset of 212 ms, which extended to later time windows. The time course data indicated two processing phases: An initial conceptual access phase, in which linguistic and modal brain circuits rapidly process features depending on their relevance, and a later conceptual elaboration phase, in which elaborative processing within feature-specific networks further refines the concept. This study confirms the proposal that abstract concepts are based on representations in distinct neural circuits depending on their semantic feature content. The present research also highlights the importance of investigating sets of abstract concepts with a defined semantic content.

Learning Low-Dimensional Semantics for Music and Language via Multi-Subject fMRI

Embodied Cognition (EC) states that semantics is encoded in the brain as firing patterns of neural circuits, which are learned according to the statistical structure of human multimodal experience. However, each human brain is idiosyncratically biased, according to its subjective experience, making this biological semantic machinery noisy with respect to semantics inherent to media, such as music and language. We propose to represent media semantics using low-dimensional vector embeddings by jointly modeling the functional Magnetic Resonance Imaging (fMRI) activity of several brains via Generalized Canonical Correlation Analysis (GCCA). We evaluate the semantic richness of the resulting latent space in appropriate semantic classification tasks: music genres and language topics. We show that the resulting unsupervised representations outperform the original high-dimensional fMRI voxel spaces in these downstream tasks while being more computationally efficient. Furthermore, we show that joint modeling of several subjects increases the semantic richness of the learned latent vector spaces as the number of subjects increases. Quantitative results and corresponding statistical significance testing demonstrate the instantiation of music and language semantics in the brain, thereby providing further evidence for multimodal embodied cognition as well as a method for extraction of media semantics from multi-subject brain dynamics.

Modelling concrete and abstract concepts using brain-constrained deep neural networks

A neurobiologically constrained deep neural network mimicking cortical areas relevant for sensorimotor, linguistic and conceptual processing was used to investigate the putative biological mechanisms underlying conceptual category formation and semantic feature extraction. Networks were trained to learn neural patterns representing specific objects and actions relevant to semantically 'ground' concrete and abstract concepts. Grounding sets consisted of three grounding patterns with neurons representing specific perceptual or action-related features; neurons were either unique to one pattern or shared between patterns of the same set. Concrete categories were modelled as pattern triplets overlapping in their 'shared neurons', thus implementing semantic feature sharing of all instances of a category. In contrast, abstract concepts had partially shared feature neurons common to only pairs of category instances, thus, exhibiting family resemblance, but lacking full feature overlap. Stimulation with concrete and abstract conceptual patterns and biologically realistic unsupervised learning caused formation of strongly connected cell assemblies (CAs) specific to individual grounding patterns, whose neurons were spread out across all areas of the deep network. After learning, the shared neurons of the instances of concrete concepts were more prominent in central areas when compared with peripheral sensorimotor ones, whereas for abstract concepts the converse pattern of results was observed, with central areas exhibiting relatively fewer neurons shared between pairs of category members. We interpret these results in light of the current knowledge about the relative difficulty children show when learning abstract words. Implications for future neurocomputational modelling experiments as well as neurobiological theories of semantic representation are discussed.

Rekindling Action Language: A Neuromodulatory Study on Parkinson's Disease Patients

Impairments of action semantics (a cognitive domain that critically engages motor brain networks) are pervasive in early Parkinson's disease (PD). However, no study has examined whether action semantic skills in persons with this disease can be influenced by non-invasive neuromodulation. Here, we recruited 22 PD patients and performed a five-day randomized, blinded, sham-controlled study to assess whether anodal transcranial direct current stimulation (atDCS) over the primary motor cortex, combined with cognitive training, can boost action-concept processing. On day 1, participants completed a picture-word association (PWA) task involving action-verb and object-noun conditions. They were then randomly assigned to either an atDCS (n = 11, 2 mA for 20 m) or a sham tDCS (n = 11, 2 mA for 30 s) group and performed an online PWA practice over three days. On day 5, they repeated the initial protocol. Relative to sham tDCS, the atDCS group exhibited faster reaction times for action (as opposed to object) concepts in the post-stimulation test. This result was exclusive to the atDCS group and held irrespective of the subjects' cognitive, executive, and motor skills, further attesting to its specificity. Our findings suggest that action-concept deficits in PD are distinctively grounded in motor networks and might be countered by direct neuromodulation of such circuits. Moreover, they provide new evidence for neurosemantic models and inform a thriving agenda in the embodied cognition framework.

No fruits without color: Cross-modal priming and EEG reveal different roles for different features across semantic categories

Category-specific impairments witnessed in patients with semantic deficits have broadly dissociated into natural and artificial kinds. However, how the category of food (more specifically, fruits and vegetables) fits into this distinction has been difficult to interpret, given a pattern of deficit that has inconsistently mapped onto either kind, despite its intuitive membership to the natural domain. The present study explores the effects of a manipulation of a visual sensory (i.e., color) or functional (i.e., orientation) feature on the consequential semantic processing of fruits and vegetables (and tools, by comparison), first at the behavioral and then at the neural level. The categorization of natural (i.e., fruits/vegetables) and artificial (i.e., utensils) entities was investigated via cross-modal priming. Reaction time analysis indicated a reduction in priming for color-modified natural entities and orientation-modified artificial entities. Standard event-related potentials (ERP) analysis was performed, in addition to linear classification. For natural entities, a N400 effect at central channel sites was observed for the color-modified condition compared relative to normal and orientation conditions, with this difference confirmed by classification analysis. Conversely, there was no significant difference between conditions for the artificial category in either analysis. These findings provide strong evidence that color is an integral property to the categorization of fruits/vegetables, thus substantiating the claim that feature-based processing guides as a function of semantic category.

Chinese-English bilinguals show linguistic-perceptual links in the brain associating short spoken phrases with corresponding real-world natural action sounds by semantic category

Higher cognitive functions such as linguistic comprehension must ultimately relate to perceptual systems in the brain, though how and why this forms remains unclear. Different brain networks that mediate perception when hearing real-world natural sounds has recently been proposed to respect a taxonomic model of acoustic-semantic categories. Using functional magnetic resonance imaging (fMRI) with Chinese/English bilingual listeners, the present study explored whether reception of short spoken phrases, in both Chinese (Mandarin) and English, describing corresponding sound-producing events would engage overlapping brain regions at a semantic category level. The results revealed a double-dissociation of cortical regions that were preferential for representing knowledge of human versus environmental action events, whether conveyed through natural sounds or the corresponding spoken phrases depicted by either language. These findings of cortical hubs exhibiting linguistic-perceptual knowledge links at a semantic category level should help to advance neurocomputational models of the neurodevelopment of language systems.

Primary Progressive Aphasia: Toward a Pathophysiological Synthesis

PURPOSE OF REVIEW

The term primary progressive aphasia (PPA) refers to a diverse group of dementias that present with prominent and early problems with speech and language. They present considerable challenges to clinicians and researchers.

RECENT FINDINGS

Here, we review critical issues around diagnosis of the three major PPA variants (semantic variant PPA, nonfluent/agrammatic variant PPA, logopenic variant PPA), as well as considering 'fragmentary' syndromes. We next consider issues around assessing disease stage, before discussing physiological phenotyping of proteinopathies across the PPA spectrum. We also review evidence for core central auditory impairments in PPA, outline critical challenges associated with treatment, discuss pathophysiological features of each major PPA variant, and conclude with thoughts on key challenges that remain to be addressed. New findings elucidating the pathophysiology of PPA represent a major step forward in our understanding of these diseases, with implications for diagnosis, care, management, and therapies.

Support vector machine based aphasia classification of transcranial magnetic stimulation language mapping in brain tumor patients

Repetitive TMS (rTMS) allows for non-invasive and transient disruption of local neuronal functioning. We used machine learning approaches to assess whether brain tumor patients can be accurately classified into aphasic and non-aphasic groups using their rTMS language mapping results as input features. Given that each tumor affects the subject-specific language networks differently, resulting in heterogenous rTMS functional mappings, we propose the use of machine learning strategies to classify potential patterns of rTMS language mapping results. We retrospectively included 90 patients with left perisylvian world health organization (WHO) grade II-IV gliomas that underwent presurgical navigated rTMS language mapping. Within our cohort, 29 of 90 (32.2%) patients suffered from at least mild aphasia as shown in the Aachen Aphasia Test based Berlin Aphasia Score (BAS). After spatial normalization to MNI 152 of all rTMS spots, we calculated the error rate (ER) in each stimulated cortical area (28 regions of interest, ROI) by automated anatomical labeling parcellation (AAL3) and IIT. We used a support vector machine (SVM) to classify significant areas in relation to aphasia. After feeding the ROIs into the SVM model, it revealed that in addition to age (w = 2.98), the ERs of the left supramarginal gyrus (w = 3.64), left inferior parietal gyrus (w = 2.28) and right pars triangularis (w = 1.34) contributed more than other features to the model. The model's sensitivity was 86.2%, the specificity was 82.0%, the overall accuracy was 85.5% and the AUC was 89.3%. Our results demonstrate an increased vulnerability of right inferior pars triangularis to rTMS in aphasic patients due to left perisylvian gliomas. This finding points towards a functional relevant involvement of the right pars triangularis in response to aphasia. The tumor location feature, specified by calculating overlaps with white and grey matter atlases, did not affect the SVM model. The left supramarginal gyrus as a feature improved our SVM model the most. Additionally, our results could point towards a decreasing potential for neuroplasticity with age.

Why Language Processing Recruits Modality Specific Brain Regions: It Is Not About Understanding Words, but About Modelling Situations

Whether language comprehension requires the participation of brain structures that evolved for perception and action has been a subject of intense debate. While brain-imaging evidence for the involvement of such modality-specific regions has grown, the fact that lesions to these structures do not necessarily erase word knowledge has invited the conclusion that language-induced activity in these structures might not be essential for word recognition. Why language processing recruits these structures remains unanswered, however. Here, we examine the original findings from a slightly different perspective. We first consider the 'original' function of structures in modality-specific brain regions that are recruited by language activity. We propose that these structures help elaborate 'internal forward models' in motor control (c.f. emulators). Emulators are brain systems that capture the relationship between an action and its sensory consequences. During language processing emulators could thus allow accessing associative memories. We further postulate the existence of a linguistic system that exploits, in a rule-based manner, emulators and other nonlinguistic brain systems, to gain complementary (and redundant) information during language processing. Emulators are therefore just one of several sources of information. We emphasize that whether a given word-form triggers activity in modality-specific brain regions depends on the linguistic context and not on the word-form as such. The role of modality-specific systems in language processing is thus not to help understanding words but to model the verbally depicted situation by supplying memorized context information. We present a model derived from these assumptions and provide predictions and perspectives for future research.

Time course of brain activity during the processing of motor- and vision-related abstract concepts: flexibility and task dependency

Grounded cognition theories assume that conceptual processing depends on modality-specific brain systems in a context-dependent fashion. Although the relation of abstract concepts to modality-specific systems is less obvious than for concrete concepts, recent behavioral and neuroimaging studies indicated a foundation of abstract concepts in vision and action. However, due to their poor temporal resolution, neuroimaging studies cannot determine whether sensorimotor activity reflects rapid access to conceptual information or later conceptual processes. The present study therefore assessed the time course of abstract concept processing using event-related potentials (ERPs) and compared ERP responses to abstract concepts with a strong relation to vision or action. We tested whether possible ERP effects to abstract word categories would emerge in early or in later time windows and whether these effects would depend on the depth of the conceptual task. In Experiment 1, a shallow lexical decision task, early feature-specific effects starting at 178 ms were revealed, but later effects beyond 300 ms were also observed. In Experiment 2, a deep conceptual decision task, feature-specific effects with an onset of 22 ms were obtained, but effects again extended beyond 300 ms. In congruency with earlier neuroimaging work, the present feature-specific ERP effects suggest a grounding of abstract concepts in modal brain systems. The presence of early and late feature-specific effects indicates that sensorimotor activity observed in neuroimaging experiments may reflect both rapid conceptual and later post-conceptual processing. Results furthermore suggest that a deep conceptual task accelerates access to conceptual sensorimotor features, thereby demonstrating conceptual flexibility.

Positive effects of grasping virtual objects on memory for novel words in a second language

Theories of embodied cognition describe language processing and representation as inherently connected to the sensorimotor experiences collected during acquisition. While children grasp their world, collect bodily experiences and name them, in second language (L2), students learn bilingual word lists. Experimental evidence shows that embodiment by mean of gestures enhances memory for words in L2. However, no study has been conducted on the effects of grasping in L2. In a virtual scenario, we trained 46 participants on 18 two- and three-syllabic words of Vimmi, an artificial corpus created for experimental purposes. The words were assigned concrete meanings of graspable objects. Six words were learned audio-visually, by reading the words projected on the wall and by hearing them. Another 6 words were trained by observation of virtual objects. Another 6 words were learned by observation and additional grasping the virtual objects. Thereafter participants were subministered free, cued recall, and reaction time tests in order to assess the word retention and the word recognition. After 30 days, the recall tests were repeated remotely to assess the memory in the long term. The results show that grasping of virtual objects can lead to superior memory performance and to lower reaction times during recognition.

Brain Activation During Conceptual Processing of Action and Sound Verbs

Grounded cognition approaches to conceptual representations postulate a close link between conceptual knowledge and the sensorimotor brain systems. The present fMRI study tested, whether a feature-specific representation of concepts, as previously demonstrated for nouns, can also be found for action- and sound-related verbs. Participants were presented with action- and soundrelated verbs along with pseudoverbs while performing a lexical decision task. Sound-related verbs activated auditory areas in the temporal cortex, whereas action-related verbs activated brain regions in the superior frontal gyrus and the cerebellum, albeit only at a more liberal threshold. This differential brain activation during conceptual verb processing partially overlapped with or was adjacent to brain regions activated during the functional localizers probing sound perception or action execution. Activity in brain areas involved in the processing of action information was parametrically modulated by ratings of action relevance. Comparisons of action- and sound-related verbs with pseudoverbs revealed activation for both verb categories in auditory and motor areas. In contrast to proposals of strong grounded cognition approaches, our study did not demonstrate a considerable overlap of activations for action- and sound-related verbs and for the corresponding functional localizer tasks. However, in line with weaker variants of grounded cognition theories, the differential activation pattern for action- and sound-related verbs was near corresponding sensorimotor brain regions depending on conceptual feature relevance. Possibly, action-sound coupling resulted in a mutual activation of the motor and the auditory system for both action- and sound-related verbs, thereby reducing the effect sizes for the differential contrasts.

No title, no theme: The joined neural space between speakers and listeners during production and comprehension of multi-sentence discourse

Speakers and listeners usually interact in larger discourses than single words or even single sentences. The goal of the present study was to identify the neural bases reflecting how the mental representation of the situation denoted in a multi-sentence discourse (situation model) is constructed and shared between speakers and listeners. An fMRI study using a variant of the ambiguous text paradigm was designed. Speakers (n = 15) produced ambiguous texts in the scanner and listeners (n = 27) subsequently listened to these texts in different states of ambiguity: preceded by a highly informative, intermediately informative or no title at all. Conventional BOLD activation analyses in listeners, as well as inter-subject correlation analyses between the speakers' and the listeners' hemodynamic time courses were performed. Critically, only the processing of disambiguated, coherent discourse with an intelligible situation model representation involved (shared) activation in bilateral lateral parietal and medial prefrontal regions. This shared spatiotemporal pattern of brain activation between the speaker and the listener suggests that the process of memory retrieval in medial prefrontal regions and the binding of retrieved information in the lateral parietal cortex constitutes a core mechanism underlying the communication of complex conceptual representations.

Task-Dependent Recruitment of Modality-Specific and Multimodal Regions during Conceptual Processing

Conceptual knowledge is central to cognitive abilities such as word comprehension. Previous neuroimaging evidence indicates that concepts are at least partly composed of perceptual and motor features that are represented in the same modality-specific brain regions involved in actual perception and action. However, it is unclear to what extent the retrieval of perceptual-motor features and the resulting engagement of modality-specific regions depend on the concurrent task. To address this issue, we measured brain activity in 40 young and healthy participants using functional magnetic resonance imaging, while they performed three different tasks-lexical decision, sound judgment, and action judgment-on words that independently varied in their association with sounds and actions. We found neural activation for sound and action features of concepts selectively when they were task-relevant in brain regions also activated during auditory and motor tasks, respectively, as well as in higher-level, multimodal regions which were recruited during both sound and action feature retrieval. For the first time, we show that not only modality-specific perceptual-motor areas but also multimodal regions are engaged in conceptual processing in a flexible, task-dependent fashion, responding selectively to task-relevant conceptual features.

Processing of Action and Sound Verbs in Context: An FMRI Study

Recent theories propose a flexible recruitment of sensory and motor brain regions during conceptual processing depending on context and task. The present functional magnetic resonance imaging study investigated the influence of context and task on conceptual processing of action and sound verbs. Participants first performed an explicit semantic context decision task, in which action and sound verbs were presented together with a context noun. The same verbs were repeatedly presented in a subsequent implicit lexical decision task together with new action and sound verbs. Thereafter, motor and acoustic localizer tasks were administered to identify brain regions involved in perception and action. During the explicit task, we found differential activations to action and sound verbs near corresponding sensorimotor brain regions. During the implicit lexical decision task, differences between action and sound verbs were absent. However, feature-specific repetition effects were observed near corresponding sensorimotor brain regions. The present results suggest flexible conceptual representations depending on context and task. Feature-specific effects were observed only near, but not within corresponding sensorimotor brain regions, as defined by the localizer tasks. Our results therefore only provide limited evidence in favor of grounded cognition theories assuming a close link between the conceptual and the sensorimotor systems.

The case of CAUSE: neurobiological mechanisms for grounding an abstract concept

How can we understand causal relationships and how can we understand words such as 'cause'? Some theorists assume that the underlying abstract concept is given to us, and that perceptual correlation provides the relevant hints towards inferring causation from perceived real-life events. A different approach emphasizes the role of actions and their typical consequences for the emergence of the concept of causation and the application of the related term. A model of causation is proposed that highlights the family resemblance between causal actions and postulates that symbols are necessary for binding together the different partially shared semantic features of subsets of causal actions and their goals. Linguistic symbols are proposed to play a key role in binding the different subsets of semantic features of the abstract concept. The model is spelt out at the neuromechanistic level of distributed cortical circuits and the cognitive functions they carry. The model is discussed in light of behavioural and neuroscience evidence, and questions for future research are highlighted. In sum, taking causation as a concrete example, I argue that abstract concepts and words can be learnt and grounded in real-life interaction, and that the neurobiological mechanisms realizing such abstract semantic grounding are within our grasp.This article is part of the theme issue 'Varieties of abstract concepts: development, use and representation in the brain'.