Neural reuse of action perception circuits for language, concepts and communication

Action verb processing specifically modulates motor behaviour and sensorimotor neuronal oscillations

Understanding action-related language recruits the brain’s motor system and can interact with motor behaviour. The current study shows MEG oscillatory patterns during verb-motor priming. Hand and foot verbs were followed by hand or foot responses, with faster reaction times for congruent conditions. In ROIs placed in the hand/arm and foot/leg portions of the sensorimotor cortex, this behavioural priming effect was accompanied by modulations in MEG oscillatory patterns preceding the responses. Power suppression in the alpha/beta frequency bands was reduced in congruent conditions in the body-part-specific ROIs. These results imply that the verb-motor priming effect may be a direct consequence of motor cortex contributions to action word processing.

The topographical organization of motor processing: An ALE meta-analysis on six action domains and the relevance of Broca's region

Action is a cover term used to refer to a large set of motor processes differing in domain specificities (e.g. execution or observation). Here we review neuroimaging evidence on action processing (N = 416; Subjects = 5912) using quantitative Activation Likelihood Estimation (ALE) and Meta-Analytic Connectivity Modeling (MACM) approaches to delineate the functional specificities of six domains: (1) Action Execution, (2) Action Imitation, (3) Motor Imagery, (4) Action Observation, (5) Motor Learning, (6) Motor Preparation. Our results show distinct functional patterns for the different domains with convergence in posterior BA44 (pBA44) for execution, imitation and imagery processing. The functional connectivity network seeding in the motor-based localized cluster of pBA44 differs from the connectivity network seeding in the (language-related) anterior BA44. The two networks implement distinct cognitive functions. We propose that the motor-related network encompassing pBA44 is recruited when processing movements requiring a mental representation of the action itself.

Is there a prediction network? Meta-analytic evidence for a cortical-subcortical network likely subserving prediction

Predictive coding is an increasingly influential and ambitious concept in neuroscience viewing the brain as a 'hypothesis testing machine' that constantly strives to minimize prediction error, the gap between its predictions and the actual sensory input. Despite the invaluable contribution of this framework to the formulation of brain function, its neuroanatomical foundations have not been fully defined. To address this gap, we conducted activation likelihood estimation (ALE) meta-analysis of 39 neuroimaging studies of three functional domains (action perception, language and music) inherently involving prediction. The ALE analysis revealed a widely distributed brain network encompassing regions within the inferior and middle frontal gyri, anterior insula, premotor cortex, pre-supplementary motor area, temporoparietal junction, striatum, thalamus/subthalamus and the cerebellum. This network is proposed to subserve domain-general prediction and its relevance to motor control, attention, implicit learning and social cognition is discussed in light of the predictive coding scheme. Better understanding of the presented network may help advance treatments of neuropsychiatric conditions related to aberrant prediction processing and promote cognitive enhancement in healthy individuals.

More than a scaffold: Language is a neuroenhancement

What role does language play in our thoughts? A longstanding proposal that has gained traction among supporters of embodied or grounded cognition suggests that it serves as a cognitive scaffold. This idea turns on the fact that language-with its ability to capture statistical regularities, leverage culturally acquired information, and engage grounded metaphors-is an effective and readily available support for our thinking. In this essay, I argue that language should be viewed as more than this; it should be viewed as a neuroenhancement. The neurologically realized language system is an important subcomponent of a flexible, multimodal, and multilevel conceptual system. It is not merely a source for information about the world but also a computational add-on that extends our conceptual reach. This approach provides a compelling explanation of the course of development, our facility with abstract concepts, and even the scope of language-specific influences on cognition.

When words are upside down: Language-space associations in children and adults

The nonlinguistic sensorimotor experiences and contexts that accompany language learning are often assumed to play an integral role in meaning representation. However, despite embodied models of language comprehension being well established in the literature, evidence is mainly derived from adult studies. For example, it has been shown that for adult comprehenders there is a close link between language and space, resulting in automatic reactivation of a referent's spatial location during word processing. In the current study, we investigated whether this link can also be found in young children (4;8-7;5 years;months). In a Stroop-like paradigm, our participants responded to a colored circle on the screen with an upward or downward arm movement after auditorily perceiving a task-irrelevant noun. We chose noun's with referents that are typically located either in upper or lower space (e.g., "sun" vs. "shoe"). Although it was not necessary to process the meaning of the nouns in order to fulfill the task, we found a spatial compatibility effect that was similar to language-space associations previously reported for adult participants. We conclude that sensorimotor experiences in the spatial domain seem to play a role during meaning processing from early childhood onward, a finding crucial for embodied models of language comprehension.

A replication attempt of hemispheric differences in semantic-relatedness judgments (Zwaan & Yaxley, 2003)

In a study by Zwaan and Yaxley (2003, Cognition, 87, B79-B86), participants judged the semantic relatedness of word pairs presented one above the other either in the left or right visual field with all related pairs requiring right-handed responses. If the vertical orientation of the word pairs matched their referents' typical vertical orientation ("roof" above "basement") a match effect was observed, but only when the word pair was presented in the left visual field. We replicated this study with response side as an additional factor and found a main effect of match, as well as a Simon effect with faster responses when the required response matched the visual field in which the word pair was presented. We did not, however, observe an interaction between the match effect and the visual field. This challenges the assumption that coarse semantic representations, including spatial properties of objects, are mainly processed in the right hemisphere.

Auditory-Motor Matching in Vocal Recognition and Imitative Learning

Songbirds possess mirror neurons (MNs) activating during the perception and execution of specific features of songs. These neurons are located in high vocal center (HVC), a premotor nucleus implicated in song perception, production and learning, making worth to inquire their properties and functions in vocal recognition and imitative learning. By integrating a body of brain and behavioral data, we discuss neurophysiology, anatomical, computational properties and possible functions of songbird MNs. We state that the neurophysiological properties of songbird MNs depends on sensorimotor regions that are outside the auditory neural system. Interestingly, songbirds MNs can be the result of the specific type of song representation possessed by some songbird species. At the functional level, we discuss whether songbird MNs are involved in others' song recognition, by dissecting the function of recognition in various different but possible overlapping processes: action-oriented perception, discriminative-oriented perception and identification of the signaler. We conclude that songbird MNs may be involved in recognizing other singer's vocalizations, while their role in imitative learning still require to solve how auditory feedback are used to correct own vocal performance to match the tutor song. Finally, we compare songbird and human mirror responses, hypothesizing a case of convergent evolution, and proposing new experimental directions.

How meaning unfolds in neural time: Embodied reactivations can precede multimodal semantic effects during language processing

Research on how the brain construes meaning during language use has prompted two conflicting accounts. According to the 'grounded view', word understanding involves quick reactivations of sensorimotor (embodied) experiences evoked by the stimuli, with simultaneous or later engagement of multimodal (conceptual) systems integrating information from various sensory streams. Contrariwise, for the 'symbolic view', this capacity depends crucially on multimodal operations, with embodied systems playing epiphenomenal roles after comprehension. To test these contradictory hypotheses, the present magnetoencephalography study assessed implicit semantic access to grammatically constrained action and non-action verbs (n = 100 per category) while measuring spatiotemporally precise signals from the primary motor cortex (M1, a core region subserving bodily movements) and the anterior temporal lobe (ATL, a putative multimodal semantic hub). Convergent evidence from sensor- and source-level analyses revealed that increased modulations for action verbs occurred earlier in M1 (∼130-190 ms) than in specific ATL hubs (∼250-410 ms). Moreover, machine-learning decoding showed that trial-by-trial classification peaks emerged faster in M1 (∼100-175 ms) than in the ATL (∼345-500 ms), with over 71% accuracy in both cases. Considering their latencies, these results challenge the 'symbolic view' and its implication that sensorimotor mechanisms play only secondary roles in semantic processing. Instead, our findings support the 'grounded view', showing that early semantic effects are critically driven by embodied reactivations and that these cannot be reduced to post-comprehension epiphenomena, even when words are individually classified. Briefly, our study offers non-trivial insights to constrain fine-grained models of language and understand how meaning unfolds in neural time.

Language as a disruptive technology: abstract concepts, embodiment and the flexible mind

A growing body of evidence suggests that cognition is embodied and grounded. Abstract concepts, though, remain a significant theoretical challenge. A number of researchers have proposed that language makes an important contribution to our capacity to acquire and employ concepts, particularly abstract ones. In this essay, I critically examine this suggestion and ultimately defend a version of it. I argue that a successful account of how language augments cognition should emphasize its symbolic properties and incorporate a view of embodiment that recognizes the flexible, multimodal and task-related nature of action, emotion and perception systems. On this view, language is an ontogenetically disruptive cognitive technology that expands our conceptual reach.This article is part of the theme issue 'Varieties of abstract concepts: development, use and representation in the brain'.

Error-based learning and lexical competition in word production: Evidence from multilingual naming

We tested whether learning associated to lexical selection is error-based, and whether lexical selection is competitive by assessing the after-effects of producing words on subsequent production of semantic competitors differing in degree of error (translation equivalents). Speakers named pictures or words in one language (part A), and then named the same set of pictures (old set) and a new set in another language (part B). RTs for the old set (i.e., translation equivalents) were larger than for the new set (i.e., items which not have been named previously in another language). Supporting that learning is error-based, this cost was mostly larger after naming in a language with a higher degree of error (L2 vs. L1). Supporting that lexical selection is competitive, after naming in a language with a high degree of error (L3), the cost was larger for naming in another language with a high degree of error (L2 vs. L1).

Language as a Predictor of Motor Recovery: The Case for a More Global Approach to Stroke Rehabilitation

Stroke is the third leading cause of death in the developed world and the primary cause of adult disability. The most common site of stroke is the middle cerebral artery (MCA), an artery that supplies a range of areas involved in both language and motor function. As a consequence, many stroke patients experience a combination of language and motor deficits. Indeed, those suffering from Broca's aphasia have an 80% chance of also suffering hemiplegia. Despite the prevalence of multifaceted disability in patients, the current trend in both clinical trials and clinical practice is toward compartmentalization of dysfunction. In this article, we review evidence that aphasia and hemiplegia do not just coexist, but that they interact. We review a number of clinical reports describing how therapies for one type of deficit can improve recovery in the other and vice versa. We go on to describe how language deficits should be seen as a warning to clinicians that the patient is likely to experience motor impairment and slower motor recovery, aiding clinicians to optimize their choice of therapy. We explore these findings and offer a tentative link between language and arm function through their shared need for sequential action, which we term fluency. We propose that area BA44 (part of Broca's area) acts as a hub for fluency in both movement and language, both in terms of production and comprehension.

Neuronal correlates of label facilitated tactile perception

It is a long-standing question in neurolinguistics, to what extent language can have a causal effect on perception. A recent behavioural study reported that participants improved their discrimination ability of Braille-like tactile stimuli after one week of implicit association training with language stimuli being co-presented redundantly with the tactile stimuli. In that experiment subjects were exposed twice a day for 1 h to the joint presentation of tactile stimuli presented to the fingertip and auditorily presented pseudowords. Their discrimination ability improved only for those tactile stimuli that were consistently paired with pseudowords, but not for those that were discordantly paired with different pseudowords. Thereby, a causal effect of verbal labels on tactile perception has been demonstrated under controlled laboratory conditions. This raises the question as to what the neuronal mechanisms underlying this implicit learning effect are. Here, we present fMRI data collected before and after the aforementioned behavioral learning to test for changes in brain connectivity as the underlying mechanism of the observed behavioral effects. The comparison of pre- and post-training revealed a language-driven increase in connectivity strength between auditory and secondary somatosensory cortex and the hippocampus as an association-learning related region.

Conceptual and Perceptual Dimensions of Word Meaning Are Recovered Rapidly and in Parallel during Reading

A single word (the noun “ elephant”) encapsulates a complex multidimensional meaning, including both perceptual (“ big”, “ gray”, “ trumpeting”) and conceptual (“ mammal”, “ can be found in India”) features. Opposing theories make different predictions as to whether different features (also conceivable as dimensions of the semantic space) are stored in similar neural regions and recovered with similar temporal dynamics during word reading. In this magnetoencephalography study, we tracked the brain activity of healthy human participants while reading single words varying orthogonally across three semantic dimensions: two perceptual ones (i.e., the average implied real-world size and the average strength of association with a prototypical sound) and a conceptual one (i.e., the semantic category). The results indicate that perceptual and conceptual representations are supported by partially segregated neural networks: Whereas visual and auditory dimensions are encoded in the phase coherence of low-frequency oscillations of occipital and superior temporal regions, respectively, semantic features are encoded in the power of low-frequency oscillations of anterior temporal and inferior parietal areas. However, despite the differences, these representations appear to emerge at the same latency: around 200 msec after stimulus onset. Taken together, these findings suggest that perceptual and conceptual dimensions of the semantic space are recovered automatically, rapidly, and in parallel during word reading.

A Neurobiologically Constrained Cortex Model of Semantic Grounding With Spiking Neurons and Brain-Like Connectivity

One of the most controversial debates in cognitive neuroscience concerns the cortical locus of semantic knowledge and processing in the human brain. Experimental data revealed the existence of various cortical regions relevant for meaning processing, ranging from semantic hubs generally involved in semantic processing to modality-preferential sensorimotor areas involved in the processing of specific conceptual categories. Why and how the brain uses such complex organization for conceptualization can be investigated using biologically constrained neurocomputational models. Here, we improve pre-existing neurocomputational models of semantics by incorporating spiking neurons and a rich connectivity structure between the model ‘areas’ to mimic important features of the underlying neural substrate. Semantic learning and symbol grounding in action and perception were simulated by associative learning between co-activated neuron populations in frontal, temporal and occipital areas. As a result of Hebbian learning of the correlation structure of symbol, perception and action information, distributed cell assembly circuits emerged across various cortices of the network. These semantic circuits showed category-specific topographical distributions, reaching into motor and visual areas for action- and visually-related words, respectively. All types of semantic circuits included large numbers of neurons in multimodal connector hub areas, which is explained by cortical connectivity structure and the resultant convergence of phonological and semantic information on these zones. Importantly, these semantic hub areas exhibited some category-specificity, which was less pronounced than that observed in primary and secondary modality-preferential cortices. The present neurocomputational model integrates seemingly divergent experimental results about conceptualization and explains both semantic hubs and category-specific areas as an emergent process causally determined by two major factors: neuroanatomical connectivity structure and correlated neuronal activation during language learning.

The neural oscillatory markers of phonetic convergence during verbal interaction

During a conversation, the neural processes supporting speech production and perception overlap in time and, based on context, expectations and the dynamics of interaction, they are also continuously modulated in real time. Recently, the growing interest in the neural dynamics underlying interactive tasks, in particular in the language domain, has mainly tackled the temporal aspects of turn-taking in dialogs. Besides temporal coordination, an under-investigated phenomenon is the implicit convergence of the speakers toward a shared phonetic space. Here, we used dual electroencephalography (dual-EEG) to record brain signals from subjects involved in a relatively constrained interactive task where they were asked to take turns in chaining words according to a phonetic rhyming rule. We quantified participants' initial phonetic fingerprints and tracked their phonetic convergence during the interaction via a robust and automatic speaker verification technique. Results show that phonetic convergence is associated to left frontal alpha/low-beta desynchronization during speech preparation and by high-beta suppression before and during listening to speech in right centro-parietal and left frontal sectors, respectively. By this work, we provide evidence that mutual adaptation of speech phonetic targets, correlates with specific alpha and beta oscillatory dynamics. Alpha and beta oscillatory dynamics may index the coordination of the "when" as well as the "how" speech interaction takes place, reinforcing the suggestion that perception and production processes are highly interdependent and co-constructed during a conversation.

The case of CAUSE: neurobiological mechanisms for grounding an abstract concept

How can we understand causal relationships and how can we understand words such as 'cause'? Some theorists assume that the underlying abstract concept is given to us, and that perceptual correlation provides the relevant hints towards inferring causation from perceived real-life events. A different approach emphasizes the role of actions and their typical consequences for the emergence of the concept of causation and the application of the related term. A model of causation is proposed that highlights the family resemblance between causal actions and postulates that symbols are necessary for binding together the different partially shared semantic features of subsets of causal actions and their goals. Linguistic symbols are proposed to play a key role in binding the different subsets of semantic features of the abstract concept. The model is spelt out at the neuromechanistic level of distributed cortical circuits and the cognitive functions they carry. The model is discussed in light of behavioural and neuroscience evidence, and questions for future research are highlighted. In sum, taking causation as a concrete example, I argue that abstract concepts and words can be learnt and grounded in real-life interaction, and that the neurobiological mechanisms realizing such abstract semantic grounding are within our grasp.This article is part of the theme issue 'Varieties of abstract concepts: development, use and representation in the brain'.

Progressive Compromise of Nouns and Action Verbs in Posterior Cortical Atrophy

Processing of nouns and action verbs can be differentially compromised following lesions to posterior and anterior/motor brain regions, respectively. However, little is known about how these deficits progress in the course of neurodegeneration. To address this issue, we assessed productive lexical skills in a patient with posterior cortical atrophy (PCA) at two different stages of his pathology. On both occasions, he underwent a structural brain imaging protocol and completed semantic fluency tasks requiring retrieval of animals (nouns) and actions (verbs). Imaging results were compared with those of controls via voxel-based morphometry (VBM), whereas fluency performance was compared to age-matched norms through Crawford's t-tests. In the first assessment, the patient exhibited atrophy of more posterior regions supporting multimodal semantics (medial temporal and lingual gyri), together with a selective deficit in noun fluency. Then, by the second assessment, the patient's atrophy had progressed mainly toward fronto-motor regions (rolandic operculum, inferior and superior frontal gyri) and subcortical motor hubs (cerebellum, thalamus), and his fluency impairments had extended to action verbs. These results offer unprecedented evidence of the specificity of the pathways related to noun and action-verb impairments in the course of neurodegeneration, highlighting the latter's critical dependence on damage to regions supporting motor functions, as opposed to multimodal semantic processes.

Action Semantics at the Bottom of the Brain: Insights From Dysplastic Cerebellar Gangliocytoma

Recent embodied cognition research shows that access to action verbs in shallow-processing tasks becomes selectively compromised upon atrophy of the cerebellum, a critical motor region. Here we assessed whether cerebellar damage also disturbs explicit semantic processing of action pictures and its integration with ongoing motor responses. We evaluated a cognitively preserved 33-year-old man with severe dysplastic cerebellar gangliocytoma (Lhermitte-Duclos disease), encompassing most of the right cerebellum and the posterior part of the left cerebellum. The patient and eight healthy controls completed two semantic association tasks (involving pictures of objects and actions, respectively) that required motor responses. Accuracy results via Crawford’s modified t-tests revealed that the patient was selectively impaired in action association. Moreover, reaction-time analysis through Crawford’s Revised Standardized Difference Test showed that, while processing of action concepts involved slower manual responses in controls, no such effect was observed in the patient, suggesting that motor-semantic integration dynamics may be compromised following cerebellar damage. Notably, a Bayesian Test for a Deficit allowing for Covariates revealed that these patterns remained after covarying for executive performance, indicating that they were not secondary to extra-linguistic impairments. Taken together, our results extend incipient findings on the embodied functions of the cerebellum, offering unprecedented evidence of its crucial role in processing non-verbal action meanings and integrating them with concomitant movements. These findings illuminate the relatively unexplored semantic functions of this region while calling for extensions of motor cognition models.

Flexibility in Language Action Interaction: The Influence of Movement Type

Recent neuropsychological studies in neurological patients and healthy subjects suggest a close functional relationship between the brain systems for language and action. Facilitation and inhibition effects of motor system activity on language processing have been demonstrated as well as causal effects in the reverse direction, from language processes on motor excitability or performance. However, as the documented effects between motor and language systems were sometimes facilitatory and sometimes inhibitory, the “sign” of these effects still remains to be explained. In a previous study, we reported a word-category-specific differential impairment of verbal working memory for concordant arm- and leg-related action words brought about by complex sequential movements of the hands and feet. In this article, we seek to determine whether the sign of the functional interaction between language and action systems of the human brain can be changed in a predictable manner by changing movement type. We here report that the sign of the effect of motor movement on action word memory can be reversed from interference to facilitation if, instead of complex movement sequences, simple repetitive movements are performed. Specifically, when engaged in finger tapping, subjects were able to remember relatively more arm-related action words (as compared to control conditions), thus documenting an enhancement of working memory brought about by simple hand movements. In contrast, when performing complex sequences of finger movements, an effector-specific degradation of action word memory was found. By manipulating the sign of the effect in accord with theory-driven predictions, these findings provide support for shared neural bases for motor movement and verbal working memory for action-related words and strengthen the argument that motor systems play a causal and functionally relevant role in language processing semantically related to action.

Agnosia for bird calls

The cognitive organisation of nonverbal auditory knowledge remains poorly defined. Deficits of environmental sound as well as word and visual object knowledge are well-recognised in semantic dementia. However, it is unclear how auditory cognition breaks down in this disorder and how this relates to deficits in other knowledge modalities. We had the opportunity to study a patient with a typical syndrome of semantic dementia who had extensive premorbid knowledge of birds, allowing us to assess the impact of the disease on the processing of auditory in relation to visual and verbal attributes of this specific knowledge category. We designed a novel neuropsychological test to probe knowledge of particular avian characteristics (size, behaviour [migratory or nonmigratory], habitat [whether or not primarily water-dwelling]) in the nonverbal auditory, visual and verbal modalities, based on a uniform two-alternative-forced-choice procedure. The patient's performance was compared to healthy older individuals of similar birding experience. We further compared his performance on this test of bird knowledge with his knowledge of familiar human voices and faces. Relative to healthy birder controls, the patient showed marked deficits of bird call and bird name knowledge but relatively preserved knowledge of avian visual attributes and retained knowledge of human voices and faces. In both the auditory and visual modalities, his knowledge of the avian characteristics of size and behaviour was intact whereas his knowledge of the associated characteristic of habitat was deficient. This case provides further evidence that nonverbal auditory knowledge has a fractionated organisation that can be differentially targeted in semantic dementia.

•

The cognitive organisation of auditory semantics is poorly understood.

•

We assessed multimodal avian knowledge in a birder with semantic dementia.

•

The patient had auditory (but not visual) agnosia for birds versus healthy birders.

•

Auditory knowledge of avian attributes and human voices were differentially affected.

•

This case illuminates the fractionated organisation of nonverbal auditory knowledge.