Active perception: sensorimotor circuits as a cortical basis for language

A neuropsychological perspective on the link between language and praxis in modern humans

Hypotheses about the emergence of human cognitive abilities postulate strong evolutionary links between language and praxis, including the possibility that language was originally gestural. The present review considers functional and neuroanatomical links between language and praxis in brain-damaged patients with aphasia and/or apraxia. The neural systems supporting these functions are predominantly located in the left hemisphere. There are many parallels between action and language for recognition, imitation and gestural communication suggesting that they rely partially on large, common networks, differentially recruited depending on the nature of the task. However, this relationship is not unequivocal and the production and understanding of gestural communication are dependent on the context in apraxic patients and remains to be clarified in aphasic patients. The phonological, semantic and syntactic levels of language seem to share some common cognitive resources with the praxic system. In conclusion, neuropsychological observations do not allow support or rejection of the hypothesis that gestural communication may have constituted an evolutionary link between tool use and language. Rather they suggest that the complexity of human behaviour is based on large interconnected networks and on the evolution of specific properties within strategic areas of the left cerebral hemisphere.

From action to language: comparative perspectives on primate tool use, gesture and the evolution of human language

The papers in this Special Issue examine tool use and manual gestures in primates as a window on the evolution of the human capacity for language. Neurophysiological research has supported the hypothesis of a close association between some aspects of human action organization and of language representation, in both phonology and semantics. Tool use provides an excellent experimental context to investigate analogies between action organization and linguistic syntax. Contributors report and contextualize experimental evidence from monkeys, great apes, humans and fossil hominins, and consider the nature and the extent of overlaps between the neural representations of tool use, manual gestures and linguistic processes.

Individual and social learning processes involved in the acquisition and generalization of tool use in macaques

Macaques can efficiently use several tools, but their capacity to discriminate the relevant physical features of a tool and the social factors contributing to their acquisition are still poorly explored. In a series of studies, we investigated macaques' ability to generalize the use of a stick as a tool to new objects having different physical features (study 1), or to new contexts, requiring them to adapt the previously learned motor strategy (study 2). We then assessed whether the observation of a skilled model might facilitate tool-use learning by naive observer monkeys (study 3). Results of study 1 and study 2 showed that monkeys trained to use a tool generalize this ability to tools of different shape and length, and learn to adapt their motor strategy to a new task. Study 3 demonstrated that observing a skilled model increases the observers' manipulations of a stick, thus facilitating the individual discovery of the relevant properties of this object as a tool. These findings support the view that in macaques, the motor system can be modified through tool use and that it has a limited capacity to adjust the learnt motor skills to a new context. Social factors, although important to facilitate the interaction with tools, are not crucial for tool-use learning.

Motor-language coupling: direct evidence from early Parkinson's disease and intracranial cortical recordings

Language and action systems are functionally coupled in the brain as demonstrated by converging evidence using Functional magnetic resonance imaging (fMRI), electroencephalography (EEG), transcranial magnetic stimulation (TMS), and lesion studies. In particular, this coupling has been demonstrated using the action-sentence compatibility effect (ACE) in which motor activity and language interact. The ACE task requires participants to listen to sentences that described actions typically performed with an open hand (e.g., clapping), a closed hand (e.g., hammering), or without any hand action (neutral); and to press a large button with either an open hand position or closed hand position immediately upon comprehending each sentence. The ACE is defined as a longer reaction time (RT) in the action-sentence incompatible conditions than in the compatible conditions. Here we investigated direct motor-language coupling in two novel and uniquely informative ways. First, we measured the behavioural ACE in patients with motor impairment (early Parkinson's disease - EPD), and second, in epileptic patients with direct electrocorticography (ECoG) recordings. In experiment 1, EPD participants with preserved general cognitive repertoire, showed a much diminished ACE relative to non-EPD volunteers. Moreover, a correlation between ACE performance and action-verb processing (kissing and dancing test - KDT) was observed. Direct cortical recordings (ECoG) in motor and language areas (experiment 2) demonstrated simultaneous bidirectional effects: motor preparation affected language processing (N400 at left inferior frontal gyrus and middle/superior temporal gyrus), and language processing affected activity in movement-related areas (motor potential at premotor and M1). Our findings show that the ACE paradigm requires ongoing integration of preserved motor and language coupling (abolished in EPD) and engages motor-temporal cortices in a bidirectional way. In addition, both experiments suggest the presence of a motor-language network which is not restricted to somatotopically defined brain areas. These results open new pathways in the fields of motor diseases, theoretical approaches to language understanding, and models of action-perception coupling.

Spectrotemporal processing drives fast access to memory traces for spoken words

The Mismatch Negativity (MMN) component of the event-related potentials is generated when a detectable spectrotemporal feature of the incoming sound does not match the sensory model set up by preceding repeated stimuli. MMN is enhanced at frontocentral scalp sites for deviant words when compared to acoustically similar deviant pseudowords, suggesting that automatic access to long-term memory traces for spoken words contributes to MMN generation. Does spectrotemporal feature matching also drive automatic lexical access? To test this, we recorded human auditory event-related potentials (ERPs) to disyllabic spoken words and pseudowords within a passive oddball paradigm. We first aimed at replicating the word-related MMN enhancement effect for Spanish, thereby adding to the available cross-linguistic evidence (e.g., Finnish, English). We then probed its resilience to spectrotemporal perturbation by inserting short (20 ms) and long (120 ms) silent gaps between first and second syllables of deviant and standard stimuli. A significantly enhanced, frontocentrally distributed MMN to deviant words was found for stimuli with no gap. The long gap yielded no deviant word MMN, showing that prior expectations of word form limits in a given language influence deviance detection processes. Crucially, the insertion of a short gap suppressed deviant word MMN enhancement at frontocentral sites. We propose that spectrotemporal point-wise matching constitutes a core mechanism for fast serial computations in audition and language, bridging sensory and long-term memory systems.

First decade of research on constrained-induced treatment approaches for aphasia rehabilitation

Approaches for treating poststroke language impairments (aphasia) based on constraint-induced (CI) principles were first introduced in 2001. CI principles as previously applied to upper extremity and locomotor retraining in stroke survivors were derived from basic neuroscience. They comprise forced-use of the affected modality, a gradual rebuilding of targeted functions using a highly intensive treatment protocol, administered in a behaviorally relevant context. CI-based approaches have stimulated considerable neurorehabilitation research interest in the past decade. The original CI aphasia treatment protocol was tailored to improve functional communication in chronic aphasia (ie, 6-12mo after stroke) and more recently, it has been adapted to treat language impairments in acute stroke survivors as well. Moreover, CI therapy applied to aphasia has been used as a model to assess language network plasticity in response to treatment using functional imaging techniques. In the following article, we review the first 10 years of behavioral and functional brain imaging research on CI-based approaches for aphasia rehabilitation.

Losing the sound of concepts: damage to auditory association cortex impairs the processing of sound-related concepts

Conceptual knowledge is classically supposed to be abstract and represented in an amodal unitary system, distinct from the sensory and motor brain systems. A more recent embodiment view of conceptual knowledge, however, proposes that concepts are grounded in distributed modality-specific brain areas which typically process sensory or action-related object information. Recent neuroimaging evidence suggested the significance of left auditory association cortex encompassing posterior superior and middle temporal gyrus in coding conceptual sound features of everyday objects. However, a causal role of this region in processing conceptual sound information has yet to be established. Here we had the unique chance to investigate a patient, JR, with a focal lesion in left posterior superior and middle temporal gyrus. To test the necessity of this region in conceptual and perceptual processing of sound information we administered four different experimental tasks to JR: Visual word recognition, category fluency, sound recognition and voice classification. Compared with a matched control group, patient JR was consistently impaired in conceptual processing of sound-related everyday objects (e.g., "bell"), while performance for non-sound-related everyday objects (e.g., "armchair"), animals, whether they typically produce sounds (e.g., "frog") or not (e.g., "tortoise"), and musical instruments (e.g., "guitar") was intact. An analogous deficit pattern in JR was also obtained for perceptual recognition of the corresponding sounds. Hence, damage to left auditory association cortex specifically impairs perceptual and conceptual processing of sounds from everyday objects. In support of modality-specific theories, these findings strongly evidence the necessity of auditory association cortex in coding sound-related conceptual information.

Grasp it loudly! Supporting actions with semantically congruent spoken action words

Evidence for cross-talk between motor and language brain structures has accumulated over the past several years. However, while a significant amount of research has focused on the interaction between language perception and action, little attention has been paid to the potential impact of language production on overt motor behaviour.

The aim of the present study was to test whether verbalizing during a grasp-to-displace action would affect motor behaviour and, if so, whether this effect would depend on the semantic content of the pronounced word (Experiment I). Furthermore, we sought to test the stability of such effects in a different group of participants and investigate at which stage of the motor act language intervenes (Experiment II). For this, participants were asked to reach, grasp and displace an object while overtly pronouncing verbal descriptions of the action (“grasp” and “put down”) or unrelated words (e.g. “butterfly” and “pigeon”).

Fine-grained analyses of several kinematic parameters such as velocity peaks revealed that when participants produced action-related words their movements became faster compared to conditions in which they did not verbalize or in which they produced words that were not related to the action. These effects likely result from the functional interaction between semantic retrieval of the words and the planning and programming of the action.

Therefore, links between (action) language and motor structures are significant to the point that language can refine overt motor behaviour.

Passive motion paradigm: an alternative to optimal control

IN THE LAST YEARS, OPTIMAL CONTROL THEORY (OCT) HAS EMERGED AS THE LEADING APPROACH FOR INVESTIGATING NEURAL CONTROL OF MOVEMENT AND MOTOR COGNITION FOR TWO COMPLEMENTARY RESEARCH LINES: behavioral neuroscience and humanoid robotics. In both cases, there are general problems that need to be addressed, such as the "degrees of freedom (DoFs) problem," the common core of production, observation, reasoning, and learning of "actions." OCT, directly derived from engineering design techniques of control systems quantifies task goals as "cost functions" and uses the sophisticated formal tools of optimal control to obtain desired behavior (and predictions). We propose an alternative "softer" approach passive motion paradigm (PMP) that we believe is closer to the biomechanics and cybernetics of action. The basic idea is that actions (overt as well as covert) are the consequences of an internal simulation process that "animates" the body schema with the attractor dynamics of force fields induced by the goal and task-specific constraints. This internal simulation offers the brain a way to dynamically link motor redundancy with task-oriented constraints "at runtime," hence solving the "DoFs problem" without explicit kinematic inversion and cost function computation. We argue that the function of such computational machinery is not only restricted to shaping motor output during action execution but also to provide the self with information on the feasibility, consequence, understanding and meaning of "potential actions." In this sense, taking into account recent developments in neuroscience (motor imagery, simulation theory of covert actions, mirror neuron system) and in embodied robotics, PMP offers a novel framework for understanding motor cognition that goes beyond the engineering control paradigm provided by OCT. Therefore, the paper is at the same time a review of the PMP rationale, as a computational theory, and a perspective presentation of how to develop it for designing better cognitive architectures.

The evolution of syntax: an exaptationist perspective

The evolution of language required elaboration of a number of independent mechanisms in the hominin lineage, including systems involved in signaling, semantics, and syntax. Two perspectives on the evolution of syntax can be contrasted. The “continuist” perspective seeks the evolutionary roots of complex human syntax in simpler combinatory systems used in animal communication systems, such as iteration and sequencing. The “exaptationist” perspective posits evolutionary change of function, so that systems today used for linguistic communication might previously have served quite different functions in earlier hominids. I argue that abundant biological evidence supports an exaptationist perspective, in general, and that it must be taken seriously when considering language evolution. When applied to syntax, this suggests that core computational components used today in language could have originally served non-linguistic functions such as motor control, non-verbal thought, or spatial reasoning. I outline three specific exaptationist hypotheses for spoken language. These three hypotheses each posit a change of functionality in a precursor circuit, and its transformation into a neural circuit or region specifically involved in language today. Hypothesis 1 suggests that the precursor mechanism for intentional vocal control, specifically direct cortical control over the larynx, was manual motor control subserved by the cortico-spinal tract. The second is that the arcuate fasciculus, which today connects syntactic and lexical regions, had its origin in intracortical connections subserving vocal imitation. The third is that the specialized components of Broca’s area, specifically BA 45, had their origins in non-linguistic motor control, and specifically hierarchical planning of action. I conclude by illustrating the importance of both homology (studied via primates) and convergence (typically analyzed in birds) for testing such evolutionary hypotheses.

Inflection in action: Semantic motor system activation to noun- and verb-containing phrases is modulated by the presence of overt grammatical markers

A recent breakthrough in understanding brain-language mechanisms is the discovery of local motor cortex activations that index specific meaning features of words, phrases and sentences. The words "talk" and "walk" activate different parts of the motor cortex, reflecting the body part relationship of actions the linguistic items are typically used to speak about. It has been suggested that such semantic motor mapping can be explained by behaviorist theories, based on conditioning mechanisms also effective in Pavlov's dog when it salivates to bell sounds. In contrast, a neurobiological approach to language predicts modulation of semantic activation by grammatical, including inflectional-morphological, information. Here, we test these competing predictions by putting action words into different phrasal contexts invoking morphosyntactic and morphophonological processes and demonstrate that semantic motor mappings are modulated by grammatical sentence properties, especially the presence of overtly realized inflectional affixes on nouns or verbs embedded in grammatical phrases. Mechanistic neuroscience theories taking into account both meaning and grammar, including morphology and syntax, are required to explain these observations. A direct comparison between phrases containing nouns and verbs revealed a tendency towards greater activation to noun phrases in left-inferior premotor cortex and posterior Broca's region (BA 44), thus questioning previous suggestions that left inferiorfrontal areas might be dedicated to verb processing per se.

The brain basis of language processing: from structure to function

Language processing is a trait of human species. The knowledge about its neurobiological basis has been increased considerably over the past decades. Different brain regions in the left and right hemisphere have been identified to support particular language functions. Networks involving the temporal cortex and the inferior frontal cortex with a clear left lateralization were shown to support syntactic processes, whereas less lateralized temporo-frontal networks subserve semantic processes. These networks have been substantiated both by functional as well as by structural connectivity data. Electrophysiological measures indicate that within these networks syntactic processes of local structure building precede the assignment of grammatical and semantic relations in a sentence. Suprasegmental prosodic information overtly available in the acoustic language input is processed predominantly in a temporo-frontal network in the right hemisphere associated with a clear electrophysiological marker. Studies with patients suffering from lesions in the corpus callosum reveal that the posterior portion of this structure plays a crucial role in the interaction of syntactic and prosodic information during language processing.

Acting in perspective: the role of body and language as social tools

We investigated how the reach-to-grasp movement is influenced by the presence of another person (friend or non-friend), who was either invisible (behind) or located in different positions with respect to an object and to the agent, and by the perspective conveyed by linguistic pronouns ("I", "You"). The interaction between social relationship and relative position influenced the latency of both maximal fingers aperture and velocity peak, showing shorter latencies in the presence of a non-friend than in the presence of a friend. However, whereas the relative position of a non-friend did not affect the kinematics of the movement, the position of a friend mattered: latencies were significantly shorter with friends only in positions allowing them to easily reach for the object. Finally, the investigation of the overall reaching movement time showed an interaction between the speaker and the pronoun: participants reached the object more quickly when the other spoke, particularly if she used the "I" pronoun. This suggests that speaking, and particularly using the "I" pronoun, evokes a potential action. Implications of the results for embodied cognition are discussed.

Literal, fictive and metaphorical motion sentences preserve the motion component of the verb: a TMS study

We used Transcranial Magnetic Stimulation (TMS) to assess whether reading literal, non-literal (i.e., metaphorical, idiomatic) and fictive motion sentences modulates the activity of the motor system. Sentences were divided into three segments visually presented one at a time: the noun phrase, the verb and the final part of the sentence. Single pulse-TMS was delivered at the end of the sentence over the leg motor area in the left hemisphere and motor evoked potentials (MEPs) were recorded from the right gastrocnemius and tibialis anterior muscles. MEPs were larger when participants were presented with literal, fictive and metaphorical motion sentences than with idiomatic motion or mental sentences. These results suggest that the excitability of the motor system is modulated by the motor component of the verb, which is preserved in fictive and metaphorical motion sentences.

The lateralization of motor cortex activation to action-words

What determines the laterality of activation in motor cortex for words whose meaning is related to bodily actions? It has been suggested that the neuronal representation of the meaning of action-words is shaped by individual experience. However, core language functions are left-lateralized in the majority of both right- and left-handers. It is still an open question to what degree connections between left-hemispheric core language areas and right-hemispheric motor areas can play a role in semantics. We investigated laterality of brain activation using fMRI in right- and left-handed participants in response to visually presented hand-related action-words, namely uni- and bi-manual actions (such as "throw" and "clap"). These stimulus groups were matched with respect to general (hand-) action-relatedness, but differed with respect to whether they are usually performed with the dominant hand or both hands. We may expect generally more left-hemispheric motor cortex activation for hand-related words in both handedness groups, with possibly more bilateral activation for left-handers compared to right-handers. In our study, both participant groups activated motor cortex bilaterally for bi-manual words. Interestingly, both groups also showed a left-lateralized activation pattern to uni-manual words. We argue that this reflects the effect of left-hemispheric language dominance on the formation of semantic brain circuits on the basis of Hebbian correlation learning.

You can count on the motor cortex: finger counting habits modulate motor cortex activation evoked by numbers

The embodied cognition framework suggests that neural systems for perception and action are engaged during higher cognitive processes. In an event-related fMRI study, we tested this claim for the abstract domain of numerical symbol processing: is the human cortical motor system part of the representation of numbers, and is organization of numerical knowledge influenced by individual finger counting habits? Developmental studies suggest a link between numerals and finger counting habits due to the acquisition of numerical skills through finger counting in childhood. In the present study, digits 1 to 9 and the corresponding number words were presented visually to adults with different finger counting habits, i.e. left- and right-starters who reported that they usually start counting small numbers with their left and right hand, respectively. Despite the absence of overt hand movements, the hemisphere contralateral to the hand used for counting small numbers was activated when small numbers were presented. The correspondence between finger counting habits and hemispheric motor activation is consistent with an intrinsic functional link between finger counting and number processing.

Bidirectional semantic interference between action and speech

Research on embodied cognition assumes that language processing involves modal simulations that recruit the same neural systems that are usually used for action execution. If this is true, one should find evidence for bidirectional crosstalk between action and language. Using a direct matching paradigm, this study tested if action-languages interactions are bidirectional (Experiments 1 and 2), and whether the effect of crosstalk between action perception and language production is due to facilitation or interference (Experiment 3). Replicating previous findings, we found evidence for crosstalk when manual actions had to be performed simultaneously to action-word perception (Experiment 1) and also when language had to be produced during simultaneous perception of hand actions (Experiment 2). These findings suggest a clear bidirectional relationship between action and language. The latter crosstalk effect was due to interference between action and language (Experiment 3). By extending previous research of embodied cognition, the present findings provide novel evidence suggesting that bidirectional functional relations between action and language are based on similar conceptual-semantic representations.

When do you grasp the idea? MEG evidence for instantaneous idiom understanding

We investigated the time-course of cortical activation during comprehension of literal and idiomatic sentences using MEG and anatomically guided distributed source analysis. Previous fMRI work had shown that the comprehension of sentences including action-related words elicits somatotopic semantic activation along the motor strip, reflecting meaning aspects of constituent words. Furthermore, idioms more strongly activated temporal pole and prefrontal cortex than literal sentences. Here we show that, compared to literal sentences, processing of idioms in a silent reading task modulates anterior fronto-temporal activity very early-on, already 150-250 ms after the sentences' critical disambiguating words ("kick the habit"). In parallel, the meaning of action words embedded in sentences is reflected by somatotopic activation of precentral motor systems. As neural reflections of constituent parts of idiomatic sentences are manifest at the same early latencies as brain indexes of idiomatic vs. literal meaning processing, we suggest that within ¼ of a second, compositional and abstract context-driven semantic processes in parallel contribute to the understanding of idiom meaning.

A cognitive architecture for the coordination of utterances

Dialog partners coordinate with each other to reach a common goal. The analogy with other joint activities has sparked interesting observations (e.g., about the norms governing turn-taking) and has informed studies of linguistic alignment in dialog. However, the parallels between language and action have not been fully explored, especially with regard to the mechanisms that support moment-by-moment coordination during language use in conversation. We review the literature on joint actions to show (i) what sorts of mechanisms allow coordination and (ii) which types of experimental paradigms can be informative of the nature of such mechanisms. Regarding (i), there is converging evidence that the actions of others can be represented in the same format as one’s own actions. Furthermore, the predicted actions of others are taken into account in the planning of one’s own actions. Similarly, we propose that interlocutors are able to coordinate their acts of production because they can represent their partner’s utterances. They can then use these representations to build predictions, which they take into account when planning self-generated utterances. Regarding (ii), we propose a new methodology to study interactive language. Psycholinguistic tasks that have traditionally been used to study individual language production are distributed across two participants, who either produce two utterances simultaneously or complete each other’s utterances.

The relation between body semantics and spatial body representations

The present study addressed the relation between body semantics (i.e. semantic knowledge about the human body) and spatial body representations, by presenting participants with word pairs, one below the other, referring to body parts. The spatial position of the word pairs could be congruent (e.g. EYE / MOUTH) or incongruent (MOUTH / EYE) with respect to the spatial position of the words' referents. In addition, the spatial distance between the words' referents was varied, resulting in word pairs referring to body parts that are close (e.g. EYE / MOUTH) or far in space (e.g. EYE / FOOT). A spatial congruency effect was observed when subjects made an iconicity judgment (Experiments 2 and 3) but not when making a semantic relatedness judgment (Experiment 1). In addition, when making a semantic relatedness judgment (Experiment 1) reaction times increased with increased distance between the body parts but when making an iconicity judgment (Experiments 2 and 3) reaction times decreased with increased distance. These findings suggest that the processing of body-semantics results in the activation of a detailed visuo-spatial body representation that is modulated by the specific task requirements. We discuss these new data with respect to theories of embodied cognition and body semantics.

What is so special about embodied simulation?

Simulation theories of social cognition abound in the literature, but it is often unclear what simulation means and how it works. The discovery of mirror neurons, responding both to action execution and observation, suggested an embodied approach to mental simulation. Over the past few years this approach has been hotly debated and alternative accounts have been proposed. We discuss these accounts and argue that they fail to capture the uniqueness of embodied simulation (ES). ES theory provides a unitary account of basic social cognition, demonstrating that people reuse their own mental states or processes represented with a bodily format in functionally attributing them to others.

Flexibility in embodied lexical-semantic representations

According to an embodied view of language comprehension, language concepts are grounded in our perceptual systems. Evidence for the idea that concepts are grounded in areas involved in action and perception comes from both behavioral and neuroimaging studies (Glenberg [1997]: Behav Brain Sci 20:1-55; Barsalou [1999]: Behav Brain Sci 22:577-660; Pulvermueller [1999]: Behav Brain Sci 22:253-336; Barsalou et al. [2003]: Trends Cogn Sci 7:84-91). However, the results from several studies indicate that the activation of information in perception and action areas is not a purely automatic process (Raposo et al. [2009]: Neuropsychologia 47:388-396; Rueschemeyer et al. [2007]: J Cogn Neurosci 19:855-865). These findings suggest that embodied representations are flexible. In these studies, flexibility is characterized by the relative presence or absence of activation in our perceptual systems. However, even if the context in which a word is presented does not undermine a motor interpretation, it is possible that the degree to which a modality-specific region contributes to a representation depends on the context in which conceptual features are retrieved. In the present study, we investigated this issue by presenting word stimuli for which both motor and visual properties (e.g., Tennis ball, Boxing glove) were important in constituting the concept. Conform with the idea that language representations are flexible and context dependent, we demonstrate that the degree to which a modality-specific region contributes to a representation considerably changes as a function of context.

Songs to Syntax

Language comprises a central component of a complex that is sometimes called “the human capacity.” This complex seems to have crystallized fairly recently among a small group in East Africa of whom people are all descendants. Common descent has been important in the evolution of the brain, such that avian and mammalian brains may be largely homologous, particularly in the case of brain regions involved in auditory perception, vocalization and auditory memory. There has been convergent evolution of the capacity for auditory-vocal learning, and possibly for structuring of external vocalizations, such that apes lack the abilities that are shared between songbirds and humans. Language’s recent evolutionary origin suggests that the computational machinery underlying syntax arose via the introduction of a single, simple, combinatorial operation. Further, the relation of a simple combinatorial syntax to the sensory-motor and thought systems reveals language to be asymmetric in design: while it precisely matches the representations required for inner mental thought, acting as the “glue” that binds together other internal cognitive and sensory modalities, at the same time it poses computational difficulties for externalization, that is, parsing and speech or signed production. Despite this mismatch, language syntax leads directly to the rich cognitive array that marks us as a symbolic species.

A role for the motor system in binding abstract emotional meaning

Sensorimotor areas activate to action- and object-related words, but their role in abstract meaning processing is still debated. Abstract emotion words denoting body internal states are a critical test case because they lack referential links to objects. If actions expressing emotion are crucial for learning correspondences between word forms and emotions, emotion word–evoked activity should emerge in motor brain systems controlling the face and arms, which typically express emotions. To test this hypothesis, we recruited 18 native speakers and used event-related functional magnetic resonance imaging to compare brain activation evoked by abstract emotion words to that by face- and arm-related action words. In addition to limbic regions, emotion words indeed sparked precentral cortex, including body-part–specific areas activated somatotopically by face words or arm words. Control items, including hash mark strings and animal words, failed to activate precentral areas. We conclude that, similar to their role in action word processing, activation of frontocentral motor systems in the dorsal stream reflects the semantic binding of sign and meaning of abstract words denoting emotions and possibly other body internal states.

Evidence of sound symbolism in simple vocalizations

The question of the arbitrariness of language is among the oldest in cognitive sciences, and it relates to the nature of the associations between vocal sounds and their meaning. Growing evidence seems to support sound symbolism, claiming for a naturally constrained mapping of meaning into sounds. Most of such evidence, however, comes from studies based on the interpretation of pseudowords, and to date, there is little empirical evidence that sound symbolism can affect phonatory behavior. In the present study, we asked participants to utter the letter /a/ in response to visual stimuli varying in shape, luminance, and size, and we observed consistent sound symbolic effects on vocalizations. Utterances' loudness was modulated by stimulus shape and luminance. Moreover, stimulus shape consistently modulated the frequency of the third formant (F3). This finding reveals an automatic mapping of specific visual attributes into phonological features of vocalizations. Furthermore, it suggests that sound-meaning associations are reciprocal, affecting active (production) as well as passive (comprehension) linguistic behavior.

Resting-state functional connectivity indexes reading competence in children and adults

Task-based neuroimaging studies face the challenge of developing tasks capable of equivalently probing reading networks across different age groups. Resting-state fMRI, which requires no specific task, circumvents these difficulties. Here, in 25 children (8-14 years) and 25 adults (21-46 years), we examined the extent to which individual differences in reading competence can be related to resting-state functional connectivity (RSFC) of regions implicated in reading. In both age groups, reading standard scores correlated positively with RSFC between the left precentral gyrus and other motor regions, and between Broca's and Wernicke's areas. This suggests that, regardless of age group, stronger coupling among motor regions, as well as between language/speech regions, subserves better reading, presumably reflecting automatized articulation. We also observed divergent RSFC-behavior relationships in children and adults, particularly those anchored in the left fusiform gyrus (FFG) (the visual word form area). In adults, but not children, better reading performance was associated with stronger positive correlations between FFG and phonology-related regions (Broca's area and the left inferior parietal lobule), and with stronger negative relationships between FFG and regions of the "task-negative" default network. These results suggest that both positive RSFC (functional coupling) between reading regions and negative RSFC (functional segregation) between a reading region and default network regions are important for automatized reading, characteristic of adult readers. Together, our task-independent RSFC findings highlight the importance of appreciating developmental changes in the neural correlates of reading competence, and suggest that RSFC may serve to facilitate the identification of reading disorders in different age groups.

How the motor system handles nouns: a behavioral study

It is an open question whether the motor system is involved during understanding of concrete nouns, as it is for concrete verbs. To clarify this issue, we carried out a behavioral experiment using a go-no go paradigm with an early and delayed go-signal delivery. Italian nouns referring to concrete objects (hand-related or foot-related) and abstract entities served as stimuli. Right-handed participants read the stimuli and responded when the presented word was concrete using the left or right hand. At the early go-signal, slower right-hand responses were found for hand-related nouns compared to foot-related nouns. The opposite pattern was found for the left hand. These findings demonstrate an early lateralized modulation of the motor system during noun processing, most likely crucial for noun comprehension.

Shared language: overlap and segregation of the neuronal infrastructure for speaking and listening revealed by functional MRI

Whether the brain's speech-production system is also involved in speech comprehension is a topic of much debate. Research has focused on whether motor areas are involved in listening, but overlap between speaking and listening might occur not only at primary sensory and motor levels, but also at linguistic levels (where semantic, lexical, and syntactic processes occur). Using functional MRI adaptation during speech comprehension and production, we found that the brain areas involved in semantic, lexical, and syntactic processing are mostly the same for speaking and for listening. Effects of primary processing load (indicative of sensory and motor processes) overlapped in auditory cortex and left inferior frontal cortex, but not in motor cortex, where processing load affected activity only in speaking. These results indicate that the linguistic parts of the language system are used for both speaking and listening, but that the motor system does not seem to provide a crucial contribution to listening.

Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans

Several stories in the popular media have speculated that it may be possible to infer from the brain which word a person is speaking or even thinking. While recent studies have demonstrated that brain signals can give detailed information about actual and imagined actions, such as different types of limb movements or spoken words, concrete experimental evidence for the possibility to 'read the mind', i.e. to interpret internally-generated speech, has been scarce. In this study, we found that it is possible to use signals recorded from the surface of the brain (electrocorticography) to discriminate the vowels and consonants embedded in spoken and in imagined words, and we defined the cortical areas that held the most information about discrimination of vowels and consonants. The results shed light on the distinct mechanisms associated with production of vowels and consonants, and could provide the basis for brain-based communication using imagined speech.

Vocal pitch discrimination in the motor system

Speech production can be broadly separated into two distinct components: Phonation and Articulation. These two aspects require the efficient control of several phono-articulatory effectors. Speech is indeed generated by the vibration of the vocal-folds in the larynx (F0) followed by ''filtering" by articulators, to select certain resonant frequencies out of that wave (F1, F2, F3, etc.). Recently it has been demonstrated that the motor representation of articulators (lips and tongue) participates in the discrimination of articulatory sounds (lips- and tongue-related speech sounds). Here we investigate whether the results obtained on articulatory sounds discrimination could be extended to phonation by applying a dual-pulse TMS protocol while subjects had to discriminate F0-shifted vocal utterances [a]. Stimulation over the larynx motor representation, compared to the control site (tongue/lips motor cortex), induced a reduction in RT for stimuli including a subtle pitch shift. We demonstrate that vocal pitch discrimination, in analogy with the articulatory component, requires the contribution of the motor system and that this effect is somatotopically organized.

Different distal-proximal movement balances in right- and left-hand writing may hint at differential premotor cortex involvement

Right-handed people generally write with their right hand. Language expressed in script is thus performed with the hand also preferred for skilled motor tasks. This may suggest an efficient functional interaction between the language area of Broca and the adjacent ventral premotor cortex (PMv) in the left (dominant) hemisphere. Pilot observations suggested that distal movements are particularly implicated in cursive writing with the right hand and proximal movements in left-hand writing, which generated ideas concerning hemisphere-specific roles of PMv and dorsal premotor cortex (PMd). Now we examined upper-limb movements in 30 right-handed participants during right- and left-hand writing, respectively. Quantitative description of distal and proximal movements demonstrated a significant difference between movements in right- and left-hand writing (p<.001, Wilcoxon signed-rank test). A Distal Movement Excess (DME) characterized writing with the right hand, while proximal and distal movements similarly contributed to left-hand writing. Although differences between non-language drawings were not tested, we propose that the DME in right-hand writing may reflect functional dominance of PMv in the left hemisphere. More proximal movements in left-hand writing might be related to PMd dominance in right-hemisphere motor control, logically implicated in spatial visuomotor transformations as seen in reaching.

Modulation of the motor system during visual and auditory language processing

Studies of embodied cognition have demonstrated the engagement of the motor system when people process action-related words and concepts. However, research using transcranial magnetic stimulation (TMS) to examine linguistic modulation in primary motor cortex has produced inconsistent results. Some studies report that action words produce an increase in corticospinal excitability; others, a decrease. Given the differences in methodology and modality, we re-examined this issue, comparing conditions in which participants either read or listened to the same set of action words. In separate blocks of trials, participants were presented with lists of words in the visual and auditory modality, and a TMS pulse was applied over left motor cortex, either 150 or 300 ms after the word onset. Motor evoked potentials (MEPs) elicited were larger following the presentation of action words compared with control words. However, this effect was only observed when the words were presented visually; no changes in MEPs were found when the words were presented auditorily. A review of the TMS literature on action word processing reveals a similar modality effect on corticospinal excitability. We discuss different hypotheses that might account for this differential modulation of action semantics by vision and audition.

Gesturing Meaning: Non-action Words Activate the Motor System

Across cultures, speakers produce iconic gestures, which add – through the movement of the speakers’ hands – a pictorial dimension to the speakers’ message. These gestures capture not only the motor content but also the visuospatial content of the message. Here, we provide first evidence for a direct link between the representation of perceptual information and the motor system that can account for these observations. Across four experiments, participants’ hand movements captured both shapes that were directly perceived, and shapes that were only implicitly activated by unrelated semantic judgments of object words. These results were obtained even though the objects were not associated with any motor behaviors that would match the gestures the participants had to produce. Moreover, implied shape affected not only gesture selection processes but also their actual execution – as measured by the shape of hand motion through space – revealing intimate links between implied shape representation and motor output. The results are discussed in terms of ideomotor theories of action and perception, and provide one avenue for explaining the ubiquitous phenomenon of iconic gestures.

From sounds to words: a neurocomputational model of adaptation, inhibition and memory processes in auditory change detection

Most animals detect sudden changes in trains of repeated stimuli but only some can learn a wide range of sensory patterns and recognise them later, a skill crucial for the evolutionary success of higher mammals. Here we use a neural model mimicking the cortical anatomy of sensory and motor areas and their connections to explain brain activity indexing auditory change and memory access. Our simulations indicate that while neuronal adaptation and local inhibition of cortical activity can explain aspects of change detection as observed when a repeated unfamiliar sound changes in frequency, the brain dynamics elicited by auditory stimulation with well-known patterns (such as meaningful words) cannot be accounted for on the basis of adaptation and inhibition alone. Specifically, we show that the stronger brain responses observed to familiar stimuli in passive oddball tasks are best explained in terms of activation of memory circuits that emerged in the cortex during the learning of these stimuli. Such memory circuits, and the activation enhancement they entail, are absent for unfamiliar stimuli. The model illustrates how basic neurobiological mechanisms, including neuronal adaptation, lateral inhibition, and Hebbian learning, underlie neuronal assembly formation and dynamics, and differentially contribute to the brain's major change detection response, the mismatch negativity.

Applauding with closed hands: neural signature of action-sentence compatibility effects

BACKGROUND

Behavioral studies have provided evidence for an action-sentence compatibility effect (ACE) that suggests a coupling of motor mechanisms and action-sentence comprehension. When both processes are concurrent, the action sentence primes the actual movement, and simultaneously, the action affects comprehension. The aim of the present study was to investigate brain markers of bidirectional impact of language comprehension and motor processes.

METHODOLOGY/PRINCIPAL FINDINGS

Participants listened to sentences describing an action that involved an open hand, a closed hand, or no manual action. Each participant was asked to press a button to indicate his/her understanding of the sentence. Each participant was assigned a hand-shape, either closed or open, which had to be used to activate the button. There were two groups (depending on the assigned hand-shape) and three categories (compatible, incompatible and neutral) defined according to the compatibility between the response and the sentence. ACEs were found in both groups. Brain markers of semantic processing exhibited an N400-like component around the Cz electrode position. This component distinguishes between compatible and incompatible, with a greater negative deflection for incompatible. Motor response elicited a motor potential (MP) and a re-afferent potential (RAP), which are both enhanced in the compatible condition.

CONCLUSIONS/SIGNIFICANCE

The present findings provide the first ACE cortical measurements of semantic processing and the motor response. N400-like effects suggest that incompatibility with motor processes interferes in sentence comprehension in a semantic fashion. Modulation of motor potentials (MP and RAP) revealed a multimodal semantic facilitation of the motor response. Both results provide neural evidence of an action-sentence bidirectional relationship. Our results suggest that ACE is not an epiphenomenal post-sentence comprehension process. In contrast, motor-language integration occurring during the verb onset supports a genuine and ongoing brain motor-language interaction.