Do I need to have my hands free to understand hand-related language? Investigating the functional relevance of experiential simulations

No support for a causal role of primary motor cortex in construing meaning from language: An rTMS study

Embodied cognition theories predict a functional involvement of sensorimotor processes in language understanding. In a preregistered experiment, we tested this idea by investigating whether interfering with primary motor cortex (M1) activation can change how people construe meaning from action language. Participants were presented with sentences describing actions (e.g., "turning off the light") and asked to choose between two interpretations of their meaning, one more concrete (e.g., "flipping a switch") and another more abstract (e.g., "going to sleep"). Prior to this task, participants' M1 was disrupted using repetitive transcranial magnetic stimulation (rTMS). The results yielded strong evidence against the idea that M1-rTMS affects meaning construction (BF01 > 30). Additional analyses and control experiments suggest that the absence of effect cannot be accounted for by failure to inhibit M1, lack of construct validity of the task, or lack of power to detect a small effect. In sum, these results do not support a causal role for primary motor cortex in building meaning from action language.

Modal and amodal cognition: an overarching principle in various domains of psychology

Accounting for how the human mind represents the internal and external world is a crucial feature of many theories of human cognition. Central to this question is the distinction between modal as opposed to amodal representational formats. It has often been assumed that one but not both of these two types of representations underlie processing in specific domains of cognition (e.g., perception, mental imagery, and language). However, in this paper, we suggest that both formats play a major role in most cognitive domains. We believe that a comprehensive theory of cognition requires a solid understanding of these representational formats and their functional roles within and across different domains of cognition, the developmental trajectory of these representational formats, and their role in dysfunctional behavior. Here we sketch such an overarching perspective that brings together research from diverse subdisciplines of psychology on modal and amodal representational formats so as to unravel their functional principles and their interactions.

Sentence-based mental simulations: Evidence from behavioral experiments using garden-path sentences

Language comprehenders activate mental representations of sensorimotor experiences related to the content of utterances they process. However, it is still unclear whether these sensorimotor simulations are driven by associations with words or by a more complex process of meaning composition into larger linguistic expressions, such as sentences. In two experiments, we investigated whether comprehenders indeed create sentence-based simulations. Materials were constructed such that simulation effects could only emerge from sentence meaning and not from word-based associations alone. We additionally asked when during sentence processing these simulations are constructed, using a garden-path paradigm. Participants read either a garden-path sentence (e.g., "As Mary ate the egg was in the fridge") or a corresponding unambiguous control with the same meaning and words (e.g., "The egg was in the fridge as Mary ate"). Participants then judged whether a depicted entity was mentioned in the sentence or not. In both experiments, picture response times were faster when the picture was compatible (vs. incompatible) with the sentence-based interpretation of the target entity (e.g., both for garden-path and control sentence: an unpeeled egg), suggesting that participants created simulations based on the sentence content and only operating over the sentence as a whole.

The action-sentence compatibility effect (ACE): Meta-analysis of a benchmark finding for embodiment

The embodied account of language comprehension has been one of the most influential theoretical developments in the recent decades addressing the question how humans comprehend and represent language. To examine its assumptions, many studies have made use of behavioral paradigms involving basic compatibility effects. The action-sentence compatibility effect (ACE) is one of the most influential of these compatibility effects and is the most widely cited evidence for the embodied account of language comprehension. However, recently there have been difficulties in extending or even in reliably replicating the ACE. The conflicting findings concerning the ACE and its extensions lead to the discussion of whether the ACE is indeed a reliable effect. In a first step we conducted a meta-analysis using a random-effects model. This analysis revealed a small but significant effect size of the ACE. Furthermore, the task-parameter Delay occurred as a factor of interest in whether the ACE appears with positive or negative effect direction. A second meta-analytic approach (Fisher's method) supports these findings. Additionally, an analysis of publication bias suggests that there is bias in the ACE literature. In post-hoc analyses of the recent multi-lab investigation of the ACE (Morey et al., 2021), evidence for individual differences in the ACE was found. However, further analyses indicate that these differences are likely due to item-specific variability and the specific way in which items were assigned to conditions in the counterbalancing lists.

Towards Strong Inference in Research on Embodiment - Possibilities and Limitations of Causal Paradigms

A central question in the cognitive sciences is which role embodiment plays for high-level cognitive functions, such as conceptual processing. Here, we propose that one reason why progress regarding this question has been slow is a lacking focus on what Platt (1964) called “strong inference”. Strong inference is possible when results from an experimental paradigm are not merely consistent with a hypothesis, but they provide decisive evidence for one particular hypothesis compared to competing hypotheses. We discuss how causal paradigms, which test the functional relevance of sensory-motor processes for high-level cognitive functions, can move the field forward. In particular, we explore how congenital sensory-motor disorders, acquired sensory-motor deficits, and interference paradigms with healthy participants can be utilized as an opportunity to better understand the role of sensory experience in conceptual processing. Whereas all three approaches can bring about valuable insights, we highlight that the study of congenitally and acquired sensorimotor disorders is particularly effective in the case of conceptual domains with strong unimodal basis (e.g., colors), whereas interference paradigms with healthy participants have a broader application, avoid many of the practical and interpretational limitations of patient studies, and allow a systematic and step-wise progressive inference approach to causal mechanisms.

When words are upside down: Language-space associations in children and adults

The nonlinguistic sensorimotor experiences and contexts that accompany language learning are often assumed to play an integral role in meaning representation. However, despite embodied models of language comprehension being well established in the literature, evidence is mainly derived from adult studies. For example, it has been shown that for adult comprehenders there is a close link between language and space, resulting in automatic reactivation of a referent's spatial location during word processing. In the current study, we investigated whether this link can also be found in young children (4;8-7;5 years;months). In a Stroop-like paradigm, our participants responded to a colored circle on the screen with an upward or downward arm movement after auditorily perceiving a task-irrelevant noun. We chose noun's with referents that are typically located either in upper or lower space (e.g., "sun" vs. "shoe"). Although it was not necessary to process the meaning of the nouns in order to fulfill the task, we found a spatial compatibility effect that was similar to language-space associations previously reported for adult participants. We conclude that sensorimotor experiences in the spatial domain seem to play a role during meaning processing from early childhood onward, a finding crucial for embodied models of language comprehension.

How do German bilingual schoolchildren process German prepositions? - A study on language-motor interactions

While much support is found for embodied language processing in a first language (L1), evidence for embodiment in second language (L2) processing is rather sparse. In a recent study, we found support for L2 embodiment, but also an influence of L1 on L2 processing in adult learners. In the present study, we compared bilingual schoolchildren who speak German as one of their languages with monolingual German schoolchildren. We presented the German prepositions auf (on), über (above), and unter (under) in a Stroop-like task. Upward or downward responses were made depending on the font colour, resulting in compatible and incompatible trials. We found compatibility effects for all children, but in contrast to the adult sample, there were no processing differences between the children depending on the nature of their other language, suggesting that the processing of German prepositions of bilingual children is embodied in a similar way as in monolingual German children.

Response Hand Differentially Affects Action Word Processing

Recent approaches in the tradition of theories of semantic and conceptual “grounding” emphasize the role of perceptual and motor knowledge in language as well as action understanding. However, the role of the two cerebral hemispheres in integrating action-motor and language processes is not clear yet. The present study looked at the influence of a simultaneous motor tapping task on word processing. In a lexical decision task, uni-manual and bi-manual hand-related, and foot-related action verbs were presented in the left and right visual half-field. A group of healthy participants performed tapping with the left hand and lexical decisions with their right hand. In a second group of participants, the reversed hand response pattern was applied. The results showed that response hand had an influence on functional lateralization of word processing when responses were executed with the non-dominant hand. Projecting words to the ipsilateral hemisphere relative to the hemisphere performing lexical decisions led to significantly decreased performance. The results showed that left hand responses led to an increased accuracy for hand-related in contrast to foot-related action verbs. The findings suggest an influence of response hand on action word processing.

Symbol Grounding Without Direct Experience: Do Words Inherit Sensorimotor Activation From Purely Linguistic Context?

Theories of embodied cognition assume that concepts are grounded in non-linguistic, sensorimotor experience. In support of this assumption, previous studies have shown that upwards response movements are faster than downwards movements after participants have been presented with words whose referents are typically located in the upper vertical space (and vice versa for downwards responses). This is taken as evidence that processing these words reactivates sensorimotor experiential traces. This congruency effect was also found for novel words, after participants learned these words as labels for novel objects that they encountered either in their upper or lower visual field. While this indicates that direct experience with a word's referent is sufficient to evoke said congruency effects, the present study investigates whether this direct experience is also a necessary condition. To this end, we conducted five experiments in which participants learned novel words from purely linguistic input: Novel words were presented in pairs with real up- or down-words (Experiment 1); they were presented in natural sentences where they replaced these real words (Experiment 2); they were presented as new labels for these real words (Experiment 3); and they were presented as labels for novel combined concepts based on these real words (Experiment 4 and 5). In all five experiments, we did not find any congruency effects elicited by the novel words; however, participants were always able to make correct explicit judgements about the vertical dimension associated to the novel words. These results suggest that direct experience is necessary for reactivating experiential traces, but this reactivation is not a necessary condition for understanding (in the sense of storing and accessing) the corresponding aspects of word meaning.