Temporal dynamics of the action-sentence compatibility effect

Embodiment and gestural realization of ergative verbs

In this study, we examined the gestural embodiment of active, passive, and active-form/passive-sense voices of ergative verbs in English. We analyzed gestures produced by presenters talking about a variety of subjects in a set of videos. We used several Chi-square tests to find out what type of gesture (representational, beat, and pointing gestures) co-occurred more frequently with active, passive, and active-form/passive-sense voices of ergative verbs. The results showed that representational gestures occurred more frequently with active than passive and active-form/passive-sense voices of ergative verbs. Furthermore, representational gestures occurred more frequently with active voices of ergative verbs having human subjects than non-human subjects. This was also the case with active-form/passive-sense sentences. Based on these results, it is suggested that form of a sentence is an influential factor in the process of embodying the situation that is described by that sentence. Active voice of an English ergative verb is more likely to be accompanied by representational gestures and is embodied more strongly than passive and active-form/passive-sense voices of that verb.

Embodied Processing at Six Linguistic Granularity Levels: A Consensus Paper

Language processing is influenced by sensorimotor experiences. Here, we review behavioral evidence for embodied and grounded influences in language processing across six linguistic levels of granularity. We examine (a) sub-word features, discussing grounded influences on iconicity (systematic associations between word form and meaning); (b) words, discussing boundary conditions and generalizations for the simulation of color, sensory modality, and spatial position; (c) sentences, discussing boundary conditions and applications of action direction simulation; (d) texts, discussing how the teaching of simulation can improve comprehension in beginning readers; (e) conversations, discussing how multi-modal cues improve turn taking and alignment; and (f) text corpora, discussing how distributional semantic models can reveal how grounded and embodied knowledge is encoded in texts. These approaches are converging on a convincing account of the psychology of language, but at the same time, there are important criticisms of the embodied approach and of specific experimental paradigms. The surest way forward requires the adoption of a wide array of scientific methods. By providing complimentary evidence, a combination of multiple methods on various levels of granularity can help us gain a more complete understanding of the role of embodiment and grounding in language processing.

A Review of Studies Supporting Metaphorical Embodiment

This paper presents a review of studies that have provided evidence supporting metaphorical embodiment. These studies are divided into three categories of behavioral, neuroimaging, and corpus studies. After summing up the findings of these studies, it is concluded that metaphorical embodiment is supported by these three lines of research. This is followed by a review of a number of studies that have measured sensorimotor and action effector strengths of various concepts. Then, the idea of sensorimotor and action effector strength of concepts is linked to metaphorical embodiment to present the main idea of the paper. Based on the findings of studies that have measured sensorimotor and action effector strengths of concepts, it is suggested that the degree of involvement of sensorimotor systems in mental simulation of metaphoric actions may not be at the same level in all metaphors. It depends on the sensorimotor strength of the base of the metaphor in various modalities. If the base of a metaphor has a high degree of perceptual strength in a certain modality, that modality plays the most important role in the processing of that metaphor, while other modalities take less important roles. In other words, depending on the sensorimotor strengths of the base of a metaphor in various modalities, those modalities have various levels of importance in the processing of that metaphor. If the base of the metaphor is weak in all modalities, modal resources can come into play to process that metaphor.

Spatiotemporal Dynamics of Activation in Motor and Language Areas Suggest a Compensatory Role of the Motor Cortex in Second Language Processing

The involvement of the motor cortex in language understanding has been intensively discussed in the framework of embodied cognition. Although some studies have provided evidence for the involvement of the motor cortex in different receptive language tasks, the role that it plays in language perception and understanding is still unclear. In the present study, we explored the degree of involvement of language and motor areas in a visually presented sentence comprehension task, modulated by language proficiency (L1: native language, L2: second language) and linguistic abstractness (literal, metaphorical, and abstract). Magnetoencephalography data were recorded from 26 late Chinese learners of English. A cluster-based permutation F test was performed on the amplitude of the source waveform for each motor and language region of interest (ROI). Results showed a significant effect of language proficiency in both language and motor ROIs, manifested as overall greater involvement of language ROIs (short insular gyri and planum polare of the superior temporal gyrus) in the L1 than the L2 during 300-500 ms, and overall greater involvement of motor ROI (central sulcus) in the L2 than the L1 during 600-800 ms. We interpreted the over-recruitment of the motor area in the L2 as a higher demand for cognitive resources to compensate for the inadequate engagement of the language network. In general, our results indicate a compensatory role of the motor cortex in L2 understanding.

The experiential basis of compatibility effects in reading-by-rotating paradigms

The current study originates from inconsistent findings within the framework of embodied language processing, specifically in the reading-by-rotating literature: whereas some studies report a match advantage (e.g., Zwaan and Taylor (J Exp Psychol 135:1, 2006)), i.e., shorter reading times when the direction of a linguistically conveyed manual rotation matched rather than mismatched the direction of an actually to be performed manual rotation Claus (Acta Psychol 156:104-113, 2015) found a mismatch advantage. The current study addresses two explanations that were previously discussed as potentially responsible for this inconsistency: on the one hand, differences in the knob devices employed; on the other hand, differences in the perspectives adopted by the readers depending on the number of characters involved in the narratives. Concurrently, the study exploits individual differences in motoric experience to explore the experiential basis of action-sentence compatibility effects. The results are inconclusive with respect to the two explanations. However, in their overall picture, they contribute suggestive considerations for the ongoing debate on action-simulation effects by pointing to the potential role of interindividual variation in motoric experience.

Up right, not right up: Primacy of verticality in both language and movement

When describing motion along both the horizontal and vertical axes, languages from different families express the elements encoding verticality before those coding for horizontality (e.g., going up right instead of right up). In light of the motor grounding of language, the present study investigated whether the prevalence of verticality in Path expression also governs the trajectory of arm biological movements. Using a 3D virtual-reality setting, we tracked the kinematics of hand pointing movements in five spatial directions, two of which implied the vertical and horizontal vectors equally (i.e., up right +45° and bottom right −45°). Movement onset could be prompted by visual or auditory verbal cues, the latter being canonical in French (“en haut à droite”/up right) or not (“à droite en haut”/right up). In two experiments, analyses of the index finger kinematics revealed a significant effect of gravity, with earlier acceleration, velocity, and deceleration peaks for upward (+45°) than downward (−45°) movements, irrespective of the instructions. Remarkably, confirming the linguistic observations, we found that vertical kinematic parameters occurred earlier than horizontal ones for upward movements, both for visual and congruent verbal cues. Non-canonical verbal instructions significantly affected this temporal dynamic: for upward movements, the horizontal and vertical components temporally aligned, while they reversed for downward movements where the kinematics of the vertical axis was delayed with respect to that of the horizontal one. This temporal dynamic is so deeply anchored that non-canonical verbal instructions allowed for horizontality to precede verticality only for movements that do not fight against gravity. Altogether, our findings provide new insights into the embodiment of language by revealing that linguistic path may reflect the organization of biological movements, giving priority to the vertical axis.

The action-sentence compatibility effect (ACE): Meta-analysis of a benchmark finding for embodiment

The embodied account of language comprehension has been one of the most influential theoretical developments in the recent decades addressing the question how humans comprehend and represent language. To examine its assumptions, many studies have made use of behavioral paradigms involving basic compatibility effects. The action-sentence compatibility effect (ACE) is one of the most influential of these compatibility effects and is the most widely cited evidence for the embodied account of language comprehension. However, recently there have been difficulties in extending or even in reliably replicating the ACE. The conflicting findings concerning the ACE and its extensions lead to the discussion of whether the ACE is indeed a reliable effect. In a first step we conducted a meta-analysis using a random-effects model. This analysis revealed a small but significant effect size of the ACE. Furthermore, the task-parameter Delay occurred as a factor of interest in whether the ACE appears with positive or negative effect direction. A second meta-analytic approach (Fisher's method) supports these findings. Additionally, an analysis of publication bias suggests that there is bias in the ACE literature. In post-hoc analyses of the recent multi-lab investigation of the ACE (Morey et al., 2021), evidence for individual differences in the ACE was found. However, further analyses indicate that these differences are likely due to item-specific variability and the specific way in which items were assigned to conditions in the counterbalancing lists.

What matters is the underlying experience: Similar motor responses during processing observed hand actions and hand-related verbs

It is well-accepted that processing observed actions involves at some extent the same neural mechanisms responsible for action execution. More recently, it has been forwarded that also the processing of verbs expressing a specific motor content is subserved by the neural mechanisms allowing individuals to perform the content expressed by that linguistic material. This view is also known as embodiment and contrasts with a more classical approach to language processing that considers it as amodal. In the present study, we used a go/no-go paradigm, in which participants were requested to respond to real words and pictures and refrain from responding when presented stimuli were pseudowords and scrambled images. Real stimuli included pictures depicting hand- and foot-related actions and verbs expressing hand- and foot-related actions. We, therefore, directly compared the modulation of hand motor responses during the observation of actions and the presentation of verbs, expressing actions in the same category. The results have shown that participants gave slower hand motor responses during the observation of hand actions and the processing of hand-related verbs as than observed foot actions and related verbs. These findings support embodiment showing that whatever the modality of presentation (observed action or verb), the modulation of hand motor responses was similar, thus suggesting that processing seen actions and related verbs shares common mechanisms most likely involving the motor system and the underlying motor experience.

The neural inhibition network is causally involved in the disembodiment effect of linguistic negation

Negation applied to action contexts reduces the activation of the motor system. According to the Reusing Inhibition for Negation (RIN) hypothesis, such "disembodiment" effect occurs because understanding negations engages the reuse of inhibitory control mechanisms. Here, we investigated whether the right inferior frontal gyrus (rIFG) - a key area of the inhibitory control system - contributes to primary motor cortex (M1) processing of negated action-sentences. Using a perturb-and-measure paradigm, we applied off-line low-frequency repetitive TMS (rTMS) over the rIFG, before performing a reading task involving action and attentional sentences presented in both affirmative or negative form. During the reading task, motor excitability was assessed by recording motor-evoked potentials (MEPs) induced by single-pulse TMS (spTMS) over the left M1, at two loci in the sentence: the verb or the object. Results show that after sham stimulation (baseline), motor excitability measured on the verb, was reduced for negative, compared to affirmative action sentences. Crucially, neuromodulation of rIFG suppressed this inhibitory effect of negation, since motor excitability was equaled for negative and affirmative action sentences. As expected, no effect of negation was observed for attentional sentences or when the pulse was delivered over the object. Our study confirms that understanding negative action sentences inhibits M1. This effect took place at an early stage of semantic processing (i.e., while processing the verb in our task), and faded at a later time-point. Critically, by highlighting a causal role of rIFG in this motor inhibition, we provide direct neurophysiological support to the RIN hypothesis.

A pre-registered, multi-lab non-replication of the action-sentence compatibility effect (ACE)

The Action-sentence Compatibility Effect (ACE) is a well-known demonstration of the role of motor activity in the comprehension of language. Participants are asked to make sensibility judgments on sentences by producing movements toward the body or away from the body. The ACE is the finding that movements are faster when the direction of the movement (e.g., toward) matches the direction of the action in the to-be-judged sentence (e.g., Art gave you the pen describes action toward you). We report on a pre-registered, multi-lab replication of one version of the ACE. The results show that none of the 18 labs involved in the study observed a reliable ACE, and that the meta-analytic estimate of the size of the ACE was essentially zero.

Adjectives Modulate Sensorimotor Activation Driven by Nouns

We performed three experiments to investigate whether adjectives can modulate the sensorimotor activation elicited by nouns. In Experiment 1, nouns of graspable objects were used as stimuli. Participants had to decide if each noun referred to a natural or artifact, by performing either a precision or a power reach-to-grasp movement. Response grasp could be compatible or incompatible with the grasp typically used to manipulate the objects to which the nouns referred. The results revealed faster reaction times (RTs) in compatible than in incompatible trials. In Experiment 2, the nouns were combined with adjectives expressing either disadvantageous information about object graspability (e.g., sharp) or information about object color (e.g., reddish). No difference in RTs between compatible and incompatible conditions was found when disadvantageous adjectives were used. Conversely, a compatibility effect occurred when color adjectives were combined with nouns referring to natural objects. Finally, in Experiment 3 the nouns were combined with adjectives expressing tactile or shape proprieties of the objects (e.g., long or smooth). Results revealed faster RTs in compatible than in incompatible condition for both noun categories. Taken together, our findings suggest that adjectives can shape the sensorimotor activation elicited by nouns of graspable objects, highlighting that language simulation goes beyond the single-word level.

The Embodied Penman: Effector-Specific Motor-Language Integration During Handwriting

Several studies have illuminated how processing manual action verbs (MaVs) affects the programming or execution of concurrent hand movements. Here, to circumvent key confounds in extant designs, we conducted the first assessment of motor-language integration during handwriting-a task in which linguistic and motoric processes are co-substantiated. Participants copied MaVs, non-manual action verbs, and non-action verbs as we collected measures of motor programming and motor execution. Programming latencies were similar across conditions, but execution was faster for MaVs than for the other categories, regardless of whether word meanings were accessed implicitly or explicitly. In line with the Hand-Action-Network Dynamic Language Embodiment (HANDLE) model, such findings suggest that effector-congruent verbs can prime manual movements even during highly automatized tasks in which motoric and verbal processes are naturally intertwined. Our paradigm opens new avenues for fine-grained explorations of embodied language processes.

Embodiment Effect on the Comprehension of Mandarin Manual Action Language: An ERP Study

Embodiment theories argue that language comprehension involves activating specific sensory-motor systems in the brain. Previous research performed in English and other Indo-European languages suggests that, when compared to compatible sentences referring to the same actions performed sequentially (e.g. 'After cleaning the wound he unrolled the bandage'), incompatible sentences describing two manual actions performed simultaneously by an agent (e.g. 'While cleaning the wound he unrolled the bandage') were worse understood and increased the event-related potentials (ERPs) component N400. The present ERP research aims to further investigate brain response to motor compatibility in native Mandarin speakers. The Chinese experimental sentences described two manual actions either in incongruent conditions marked by the Chinese parallel structure … … ('yibian…yibian…'/'while… while…') or congruent conditions marked by the sequential structure … … ('xian…ranhou…'/'firstly… then…'). The last action clause elicited larger fronto-central N400 in the incongruent condition, which reveals that there are semantic embodiment effects on the comprehension of Mandarin manual action language.

Motor resonance during linguistic processing as shown by EEG in a naturalistic VR environment

Embodied cognition studies have shown motor resonance during action language processing, indicating that linguistic representations are at least partially multimodal. However, constraints of this activation linked to linguistic and extra-linguistic context, function and timing have not yet been fully explored. Importantly, embodied cognition binds social and physical contexts to cognition, suggesting that more ecologically valid contexts will yield more valid measures of cognitive processing. Herein, we measured cortical motor activation during language processing in a fully immersive Cave automatic virtual environment (CAVE). EEG was recorded while participants engaged in a Go/No-Go task. They heard action verbs and, for Go trials, performed a corresponding action on a virtual object. ERSP (event-related spectral perturbation) was calculated during verb processing, corresponding to the pattern of power suppression (event-related desynchronization - ERD) and enhancement (event-related synchronization - ERS) relative to the reference interval. Significant ERD emerged during verb processing in both the µ (8-13 Hz) and beta band (20-30 Hz) for both Go and No-Go trials. µ ERD emerged in the 400-500 msec time window, associated with lexical-semantic processing. Greater µ ERD emerged for Go compared to No-Go trials. The present results provide compelling evidence in a naturalistic setting of how motor and linguistic processes interact.

East is not right: Spatial compatibility differs between egocentric and cardinal retrieval

Four experiments examined perceptuo-motor associations involved in spatial knowledge encoding and retrieval. Participants learned spatial information by studying a map or by navigating through a real environment and then verified spatial descriptions based on either egocentric or cardinal directional terms. Participants moved the computer mouse to a YES or NO button to verify each statement. We tracked mouse cursor trajectories to examine perceptuo-motor associations in spatial knowledge. An encoding hypothesis predicts that perceptuo-motor associations depend on the involvement of perceptions and actions during encoding, regardless of how spatial knowledge would be used. The retrieval hypothesis predicts that perceptuo-motor associations change as a function of retrieval demands, regardless of how they are learned. The results supported the retrieval hypothesis. Participants showed action compatibility effects with egocentric retrieval, regardless of how spatial information was learned. With well-developed spatial knowledge, a reliable compatibility effect emerged during egocentric retrieval, but no or limited compatibility effects emerged with cardinal retrieval. With less-developed knowledge, the compatibility effects evident during cardinal retrieval suggest a process of egocentric recoding. Other factors of environment learning, such as location proximity and orientation changes, also impacted the compatibility effect, as revealed in the temporal dynamics of mouse movements. Taken together, the results demonstrate that retrieval demands differentially rely upon perceptuo-motor associations in long-term spatial knowledge. This effect is also modulated by environment experience, proximity of learned locations, and experienced orientations.

A moving story: Whole-body motor training selectively improves the appraisal of action meanings in naturalistic narratives

This study examined whether systematic whole-body stimulation and increased attention to visuospatial motion patterns can enhance the appraisal of action meanings evoked by naturalistic texts. Participants listened to action and neutral (non-action) narratives before and after videogame-based bodily training, and responded to questions on information realized by verbs (denoting abstract and action processes) and circumstances (conveying locative or temporal details, for example). Strategically, we worked with dyslexic children, whose potential comprehension deficits could give room to post-training improvements. Results showed a selective boost in understanding of action information, even when controlling for baseline performance. Also, this effect proved uninfluenced by short-term memory skills, and it was absent when training relied on non-action videogames requiring minimal bodily engagement. Of note, the movements described in the texts did not match those performed by participants, suggesting that well-established effector- and direction-specific language embodiment effects may be accompanied by more coarse-grained sensorimotor resonance, driven by activation of motor and visuospatial sensory systems. In sum, the stimulation of movement-related mechanisms seems to selectively boost the appraisal of actions evoked by naturalistic texts. By demonstrating such links between two real-life activities, our study offers an empirical tie between embodied and situated accounts of cognition.

The poverty of embodied cognition

In recent years, there has been rapidly growing interest in embodied cognition, a multifaceted theoretical proposition that (1) cognitive processes are influenced by the body, (2) cognition exists in the service of action, (3) cognition is situated in the environment, and (4) cognition may occur without internal representations. Many proponents view embodied cognition as the next great paradigm shift for cognitive science. In this article, we critically examine the core ideas from embodied cognition, taking a "thought exercise" approach. We first note that the basic principles from embodiment theory are either unacceptably vague (e.g., the premise that perception is influenced by the body) or they offer nothing new (e.g., cognition evolved to optimize survival, emotions affect cognition, perception-action couplings are important). We next suggest that, for the vast majority of classic findings in cognitive science, embodied cognition offers no scientifically valuable insight. In most cases, the theory has no logical connections to the phenomena, other than some trivially true ideas. Beyond classic laboratory findings, embodiment theory is also unable to adequately address the basic experiences of cognitive life.

Just out of reach: On the reliability of the action-sentence compatibility effect

The action-sentence compatibility effect (ACE; Glenberg & Kaschak, 2002), a hallmark finding in Embodied Cognition, implicates the motor system in language comprehension. In the ACE, people process sentences implying movement toward or away from themselves, responding with actions toward or away from their bodies. These processes interact, implying a linkage between linguistic and motor systems. From a theoretical perspective, the ACE has been extremely influential, being widely cited evidence in favor of embodied cognition. The present study began as an attempt to extend the ACE in a new direction, but eventually became a series of attempts to simply replicate the effect. Across 8 experiments, I tested whether the ACE extends to a novel mouse-tracking method and/or is susceptible to higher-order cognitive influences. In 3 experiments, attempts were made to "disembody" the ACE by presenting participants' names on the computer screen (as in Markman & Brendl, 2005). In each experiment, the ACE could not be disembodied, because the ACE did not occur. In further experiments, the ACE was not observed in reading times, regardless of response mode (mouse movements vs. button-presses) or stimuli, including those from the original research. Similarly, no ACE was observed in physical movement times. Bayes Factor analyses of the current experiments, and the previous ACE literature, suggest that the evidence for the ACE is generally weak: Many studies considered as positive evidence actually support the null hypothesis, and very few published results offer strong evidence for the ACE. Implications for the embodiment hypothesis are discussed.

A touch with words: Dynamic synergies between manual actions and language

Manual actions are a hallmark of humanness. Their underlying neural circuitry gives rise to species-specific skills and interacts with language processes. In particular, multiple studies show that hand-related expressions - verbal units evoking manual activity - variously affect concurrent manual actions, yielding apparently controversial results (interference, facilitation, or null effects) in varied time windows. Through a systematic review of 108 experiments, we show that such effects are driven by several factors, such as the level of verbal processing, action complexity, and the time-lag between linguistic and motor processes. We reconcile key empirical patterns by introducing the Hand-Action-Network Dynamic Language Embodiment (HANDLE) model, an integrative framework based on neural coupling dynamics and predictive-coding principles. To conclude, we assess HANDLE against the backdrop of other action-cognition theories, illustrate its potential applications to understand high-level deficits in motor disorders, and discuss key challenges for further development. In sum, our work aligns with the 'pragmatic turn', moving away from passive and static representationalist perspectives to a more dynamic, enactive, and embodied conceptualization of cognitive processes.

The protagonist's first perspective influences the encoding of spatial information in narratives

Three experiments examined the first-perspective alignment effect that is observed when retrieving spatial information from memory about described environments. Participants read narratives that described the viewpoint of a protagonist in fictitious environments and then pointed to the memorized locations of described objects from imagined perspectives. Results from Experiments 1 and 2 showed that performance was best when participants responded from the protagonist's first perspective even though object locations were described from a different perspective. In Experiment 3, in which participants were physically oriented with the perspective used to describe object locations, performance from that description perspective was better than that from the protagonist's first perspective, which was, in turn, better than performance from other perspectives. These findings suggest that when reading narratives, people default to using a reference frame that is aligned with their own facing direction, although physical movement may facilitate retrieval from other perspectives.

The Action-Sentence Compatibility Effect in ASL: the role of semantics vs. perception

Embodied theories of cognition propose that humans use sensorimotor systems in processing language. The Action-Sentence Compatibility Effect (ACE) refers to the finding that motor responses are facilitated after comprehending sentences that imply movement in the same direction. In sign languages there is a potential conflict between sensorimotor systems and linguistic semantics: movement away from the signer is perceived as motion toward the comprehender. We examined whether perceptual processing of sign movement or verb semantics modulate the ACE. Deaf ASL signers performed a semantic judgment task while viewing signed sentences expressing toward or away motion. We found a significant congruency effect relative to the verb's semantics rather than to the perceived motion. This result indicates that (a) the motor system is involved in the comprehension of a visual-manual language, and (b) motor simulations for sign language are modulated by verb semantics rather than by the perceived visual motion of the hands.

Time course of action representations evoked during sentence comprehension

The nature of hand-action representations evoked during language comprehension was investigated using a variant of the visual-world paradigm in which eye fixations were monitored while subjects viewed a screen displaying four hand postures and listened to sentences describing an actor using or lifting a manipulable object. Displayed postures were related to either a functional (using) or volumetric (lifting) interaction with an object that matched or did not match the object mentioned in the sentence. Subjects were instructed to select the hand posture that matched the action described in the sentence. Even before the manipulable object was mentioned in the sentence, some sentence contexts allowed subjects to infer the object's identity and the type of action performed with it and eye fixations immediately favored the corresponding hand posture. This effect was assumed to be the result of ongoing motor or perceptual imagery in which the action described in the sentence was mentally simulated. In addition, the hand posture related to the manipulable object mentioned in a sentence, but not related to the described action (e.g., a writing posture in the context of a sentence that describes lifting, but not using, a pencil), was favored over other hand postures not related to the object. This effect was attributed to motor resonance arising from conceptual processing of the manipulable object, without regard to the remainder of the sentence context.

Viewing photos and reading nouns of natural graspable objects similarly modulate motor responses

It is well known that the observation of graspable objects recruits the same motor representations involved in their actual manipulation. Recent evidence suggests that the presentation of nouns referring to graspable objects may exert similar effects. So far, however, it is not clear to what extent the modulation of the motor system during object observation overlaps with that related to noun processing. To address this issue, 2 behavioral experiments were carried out using a go-no go paradigm. Healthy participants were presented with photos and nouns of graspable and non-graspable natural objects. Also scrambled images and pseudowords obtained from the original stimuli were used. At a go-signal onset (150 ms after stimulus presentation) participants had to press a key when the stimulus referred to a real object, using their right (Experiment 1) or left (Experiment 2) hand, and refrain from responding when a scrambled image or a pseudoword was presented. Slower responses were found for both photos and nouns of graspable objects as compared to non-graspable objects, independent of the responding hand. These findings suggest that processing seen graspable objects and written nouns referring to graspable objects similarly modulates the motor system.

A test of motor skill-specific action embodiment in ice-hockey players

To further our understanding of the role of the motor system in comprehending action-related sentences, we compared action experts (athletes) to visual experts (fans) and novices when responding with an action-specific effector (either hand or foot). These conditions allowed inferences about the degree and specificity of embodiment in language comprehension. Ice hockey players, fans and novices made speeded judgments regarding the congruence between an auditorily presented sentence and a subsequently presented picture. Picture stimuli consisted of either hockey or everyday items. Half of these pictures 'matched' the action implied in the preceding sentence. Further, the action in these images involved either primarily the hand or the foot. For everyday items, action-matched items were responded to faster than action-mismatched items. However, only the players and fans showed the action-match effect for hockey items. There were no consistent effector-stimuli compatibility effects, nor skill-based interactions with compatibility, suggesting that the action-match effect was not based on motor ability per se, but rather a construction of the action based on knowledge or visual experience with the hockey related sentences.

Motor activation in literal and non-literal sentences: does time matter?

Despite the impressive amount of evidence showing involvement of the sensorimotor systems in language processing, important questions remain unsolved among which the relationship between non-literal uses of language and sensorimotor activation. The literature did not yet provide a univocal answer on whether the comprehension of non-literal, abstract motion sentences engages the same neural networks recruited for literal sentences. A previous TMS study using the same experimental materials of the present study showed activation for literal, fictive and metaphoric motion sentences but not for idiomatic ones. To evaluate whether this may depend on insufficient time for elaborating the idiomatic meaning, we conducted a behavioral experiment that used a sensibility judgment task performed by pressing a button either with a hand finger or with a foot. Motor activation is known to be sensitive to the action-congruency of the effector used for responding. Therefore, all other things being equal, significant differences between response emitted with an action-congruent or incongruent effector (foot vs. hand) may be attributed to motor activation. Foot-related action verbs were embedded in sentences conveying literal motion, fictive motion, metaphoric motion or idiomatic motion. Mental sentences were employed as a control condition. foot responses were significantly faster than finger responses but only in literal motion sentences. We hypothesize that motor activation may arise in early phases of comprehension processes (i.e., upon reading the verb) for then decaying as a function of the strength of the semantic motion component of the verb.

Encoding and updating spatial information presented in narratives

Four experiments investigated whether directional spatial relations encoded by reading narratives are updated following described protagonist rotations. Participants memorized locations of objects described in short stories that placed them, as the protagonist, in remote settings. After reading a description that the protagonist rotated to the left or the right of the initial orientation, participants made judgements about object relations in the described environment (Experiment 1). Before making these judgements, participants were instructed to physically rotate to match (Experiment 2) or mismatch (Experiment 4) the protagonist's described rotation and in Experiments 3 and 4 to also visualize the changed relations following rotation. Participants' performance suggested that they relied on the initial representation they constructed during encoding rather than on the updated protagonist-to-object relations. Participants' physical movement to match the described rotation and additional visualization instructions did not facilitate updating through a sensorimotor process. In these respects, updating spatial relations in situation models constructed from narratives differs from updating in perceptually experienced environments.

Processing of hand-related verbs specifically affects the planning and execution of arm reaching movements

Even though a growing body of research has shown that the processing of action language affects the planning and execution of motor acts, several aspects of this interaction are still hotly debated. The directionality (i.e. does understanding action-related language induce a facilitation or an interference with the corresponding action?), the time course, and the nature of the interaction (i.e. under what conditions does the phenomenon occur?) are largely unclear. To further explore this topic we exploited a go/no-go paradigm in which healthy participants were required to perform arm reaching movements toward a target when verbs expressing either hand or foot actions were shown, and to refrain from moving when abstract verbs were presented. We found that reaction times (RT) and percentages of errors increased when the verb involved the same effector used to give the response. This interference occurred very early, when the interval between verb presentation and the delivery of the go signal was 50 ms, and could be elicited until this delay was about 600 ms. In addition, RTs were faster when subjects used the right arm than when they used the left arm, suggesting that action–verb understanding is left-lateralized. Furthermore, when the color of the printed verb and not its meaning was the cue for movement execution the differences between RTs and error percentages between verb categories disappeared, unequivocally indicating that the phenomenon occurs only when the semantic content of a verb has to be retrieved. These results are compatible with the theory of embodied language, which hypothesizes that comprehending verbal descriptions of actions relies on an internal simulation of the sensory–motor experience of the action, and provide a new and detailed view of the interplay between action language and motor acts.

Motor-language coupling: direct evidence from early Parkinson's disease and intracranial cortical recordings

Language and action systems are functionally coupled in the brain as demonstrated by converging evidence using Functional magnetic resonance imaging (fMRI), electroencephalography (EEG), transcranial magnetic stimulation (TMS), and lesion studies. In particular, this coupling has been demonstrated using the action-sentence compatibility effect (ACE) in which motor activity and language interact. The ACE task requires participants to listen to sentences that described actions typically performed with an open hand (e.g., clapping), a closed hand (e.g., hammering), or without any hand action (neutral); and to press a large button with either an open hand position or closed hand position immediately upon comprehending each sentence. The ACE is defined as a longer reaction time (RT) in the action-sentence incompatible conditions than in the compatible conditions. Here we investigated direct motor-language coupling in two novel and uniquely informative ways. First, we measured the behavioural ACE in patients with motor impairment (early Parkinson's disease - EPD), and second, in epileptic patients with direct electrocorticography (ECoG) recordings. In experiment 1, EPD participants with preserved general cognitive repertoire, showed a much diminished ACE relative to non-EPD volunteers. Moreover, a correlation between ACE performance and action-verb processing (kissing and dancing test - KDT) was observed. Direct cortical recordings (ECoG) in motor and language areas (experiment 2) demonstrated simultaneous bidirectional effects: motor preparation affected language processing (N400 at left inferior frontal gyrus and middle/superior temporal gyrus), and language processing affected activity in movement-related areas (motor potential at premotor and M1). Our findings show that the ACE paradigm requires ongoing integration of preserved motor and language coupling (abolished in EPD) and engages motor-temporal cortices in a bidirectional way. In addition, both experiments suggest the presence of a motor-language network which is not restricted to somatotopically defined brain areas. These results open new pathways in the fields of motor diseases, theoretical approaches to language understanding, and models of action-perception coupling.

The action-sentence compatibility effect in Japanese sentences

The action-sentence compatibility effect (ACE) is a phenomenon in which a reader's response to a sentence is made faster when there is congruity between the action described in the sentence and the action that makes up the response. Previous studies showed the ACE occurs in action-related sentences in several languages. However, all these were SVO (verb-object) languages, in which verbs are placed before object nouns; this order is reversed in SOV languages. Moreover, those studies investigated hand responses. This study assessed the existence of the ACE in Japanese, an SOV language, and in foot responses. 24 female participants judged the sensibility of Japanese sentences that described actions and responded with either their foot or hand as an effector. Reaction times were significantly faster when there was congruity between the effector described in the sentences and the effector actually used for the response. However, sentence dependency was also found in the foot responses.

The influence of motor simulations on language comprehension

This study investigates the influence of activating specific motor codes on the comprehension of passages that describe the use of an object requiring similar motor manipulations. In three experiments, participants either imagined or pantomimed performing an action involving a common object. Participants then held the action in memory while reading a brief story, which described another object that required similar or different motor behaviors. Reading times were collected on the complementary actions. Finally, participants acted out the original action. In Exp. 1 and 2, reading slowed to the verbs. Exp. 2 revealed the slowing to be true interference, which disappeared in Exp. 3 when the action did not need to be recalled. The results suggest that readers activate motor codes when reading story actions, which supports an embodied view. The results also indicate that activated codes bound to an action will, at least briefly, impair reading about a complementary action requiring the same codes, consistent with Hommel's (2009) theory of event coding.

How the motor system handles nouns: a behavioral study

It is an open question whether the motor system is involved during understanding of concrete nouns, as it is for concrete verbs. To clarify this issue, we carried out a behavioral experiment using a go-no go paradigm with an early and delayed go-signal delivery. Italian nouns referring to concrete objects (hand-related or foot-related) and abstract entities served as stimuli. Right-handed participants read the stimuli and responded when the presented word was concrete using the left or right hand. At the early go-signal, slower right-hand responses were found for hand-related nouns compared to foot-related nouns. The opposite pattern was found for the left hand. These findings demonstrate an early lateralized modulation of the motor system during noun processing, most likely crucial for noun comprehension.

Symbol interdependency in symbolic and embodied cognition

Whether computational algorithms such as latent semantic analysis (LSA) can both extract meaning from language and advance theories of human cognition has become a topic of debate in cognitive science, whereby accounts of symbolic cognition and embodied cognition are often contrasted. Albeit for different reasons, in both accounts the importance of statistical regularities in linguistic surface structure tends to be underestimated. The current article gives an overview of the symbolic and embodied cognition accounts and shows how meaning induction attributed to a specific statistical process or to activation of embodied representations should be attributed to language itself. Specifically, the performance of LSA can be attributed to the linguistic surface structure, more than special characteristics of the algorithm, and embodiment findings attributed to perceptual simulations can be explained by distributional linguistic information.

The Two-Level Theory of verb meaning: An approach to integrating the semantics of action with the mirror neuron system

Verbs have two separate levels of meaning. One level reflects the uniqueness of every verb and is called the "root". The other level consists of a more austere representation that is shared by all the verbs in a given class and is called the "event structure template". We explore the following hypotheses about how, with specific reference to the motor features of action verbs, these two distinct levels of semantic representation might correspond to two distinct levels of the mirror neuron system. Hypothesis 1: Root-level motor features of verb meaning are partially subserved by somatotopically mapped mirror neurons in the left primary motor and/or premotor cortices. Hypothesis 2: Template-level motor features of verb meaning are partially subserved by representationally more schematic mirror neurons in Brodmann area 44 of the left inferior frontal gyrus. Evidence has been accumulating in support of the general neuroanatomical claims made by these two hypotheses-namely, that each level of verb meaning is associated with the designated cortical areas. However, as yet no studies have satisfied all the criteria necessary to support the more specific neurobiological claims made by the two hypotheses-namely, that each level of verb meaning is associated with mirror neurons in the pertinent brain regions. This would require demonstrating that within those regions the same neuronal populations are engaged during (a) the linguistic processing of particular motor features of verb meaning, (b) the execution of actions with the corresponding motor features, and (c) the observation of actions with the corresponding motor features.

Embodied language: a review of the role of the motor system in language comprehension

A growing body of research suggests that comprehending verbal descriptions of actions relies on an internal simulation of the described action. To assess this motor resonance account of language comprehension, we first review recent developments in the literature on perception and action, with a view towards language processing. We then examine studies of language processing from an action simulation perspective. We conclude by discussing several criteria that might be helpful with regard to assessing the role of motor resonance during language comprehension.

Grasp Cueing shows Obligatory Attention to Action Goals

To understand the grounding of cognitive mechanisms in perception and action, we used a simple detection task to determine how long it takes to predict an action goal from the perception of grasp postures and whether this prediction is under strategic control. Healthy observers detected visual probes over small or large objects after seeing either a precision grip or a power grip posture. Although the posture was uninformative it induced attention shifts to the grasp-congruent object within 350 ms. When the posture predicted target appearance over the grasp-incongruent object, observers’ initial strategic allocation of attention was overruled by the congruency between grasp and object. These results might help to characterize the human mirror neuron system and reveal how joint attention tunes early perceptual processes toward action prediction.