Linguistic representations of motion do not depend on the visual motion system

Mental simulation of colour properties during language comprehension: influence of context and comprehension stages

Many studies have shown that mental simulation may occur during language comprehension. Supporting evidence is derived from the matching effects in the sentence-picture verification (SPV) task often used to assess mental simulations of object properties, such as size, orientation, and shape. However, mixed results have been obtained regarding object colour, with researchers reporting matching or mismatching effects. This study investigated the impact of colour information clarity within sentences on the process of mental simulation during language comprehension. Employing the SPV task and using novel objects, we examined whether there is a mental simulation of colour after excluding typical/atypical colour bias and how varying levels of colour information clarity in sentences influence the emergence of matching effects at different stages of comprehension. To address these issues, we conducted two experiments. In Experiment 1, the participants read normal sentences and subsequently engaged in picture verification with a novel object after a 500 ms delay. In Experiment 2, the participants encountered sentences containing both clear and unclear colour information and, after either a 0 ms or 1500 ms interval, completed picture verification tasks with a novel object. Null effects were found in the 500 ms condition for normal sentences and the 0 ms condition for unclear colour information sentences. A mismatching effect appeared in the 0 ms condition after clear colour information sentences, and a matching effect appeared in the 1500 ms condition for all sentences. The results indicated that after excluding colour bias, the participants still formed mental simulations of colour during language comprehension. Our results also indicated that ongoing colour simulation with time pressure impacted the participant responses. The participants ignored unclear colour information under time pressure, but without time pressure, they constructed simulations that were as detailed as possible, regardless of whether the implicit colour information in the sentence was clear.

Meaning in hand: Investigating shared mechanisms of motor imagery and sensorimotor simulation in language processing

There is substantial evidence to support grounded theories of semantic representation, however the mechanisms of simulation in most theories are underspecified. In the present study, we used an individual differences approach to test whether motor imagery may share some mechanisms with sensorimotor simulations engaged during semantic processing. We quantified individual differences in motor imagery ability via implicit imagery tasks and explicit imagery questionnaires and tested their relationship to sensorimotor effects in syntactic classification tasks. In Experiment 1 (N = 185) we tested relationships between motor imagery and semantic processing of body-object interaction meaning (BOI; the degree to which you can interact with a word's referent) and foot/leg action meaning. We observed two interactions between imagery ability measured on the Florida Praxis Imagery Questionnaire (FPIQ) and BOI effects in semantic processing (response time and accuracy). In both interactions poorer imagery ability was associated with null BOI effects, whereas better imagery was associated with BOI effects. We also observed faster and more accurate responses to verbs associated with more foot/leg action meaning than verbs with less foot/leg action meaning, but this foot/leg action effect did not significantly interact with individual differences in motor imagery. In Experiment 2 (N = 195) we tested whether the interactions observed in Experiment 1 were dependent on the object-directed nature of the actions, or whether similar effects would be observed for hand actions not associated with objects. We also expanded our investigation beyond hand and foot imagery to consider whole body imagery. We observed an interaction between performance on a hand laterality judgement task (HLJT; assessing hand motor imagery) and sensorimotor effects in semantic processing of verbs associated with hand/arm action meaning. Participants with the fastest responses on the most difficult trials of the HLJT showed no significant difference in their response times to words with high and low hand/arm action meaning. We also observed faster and more accurate responses to high relative to low embodiment verbs, but this sensorimotor effect did not interact with individual differences in motor imagery. The results suggest specific (and not general) associations, in that some, but not all forms of hand and object-directed motor imagery are related to sensorimotor effects in language processing of hand/arm action verbs and nouns describing objects that are easy to interact with. As such, hand and object-directed motor imagery may share mechanisms with sensorimotor simulation during semantic processing.

Ecological Meanings: A Consensus Paper on Individual Differences and Contextual Influences in Embodied Language

Embodied theories of cognition consider many aspects of language and other cognitive domains as the result of sensory and motor processes. In this view, the appraisal and the use of concepts are based on mechanisms of simulation grounded on prior sensorimotor experiences. Even though these theories continue receiving attention and support, increasing evidence indicates the need to consider the flexible nature of the simulation process, and to accordingly refine embodied accounts. In this consensus paper, we discuss two potential sources of variability in experimental studies on embodiment of language: individual differences and context. Specifically, we show how factors contributing to individual differences may explain inconsistent findings in embodied language phenomena. These factors include sensorimotor or cultural experiences, imagery, context-related factors, and cognitive strategies. We also analyze the different contextual modulations, from single words to sentences and narratives, as well as the top-down and bottom-up influences. Similarly, we review recent efforts to include cultural and language diversity, aging, neurodegenerative diseases, and brain disorders, as well as bilingual evidence into the embodiment framework. We address the importance of considering individual differences and context in clinical studies to drive translational research more efficiently, and we indicate recommendations on how to correctly address these issues in future research. Systematically investigating individual differences and context may contribute to understanding the dynamic nature of simulation in language processes, refining embodied theories of cognition, and ultimately filling the gap between cognition in artificial experimental settings and cognition in the wild (i.e., in everyday life).

The Time-Course of Sentence Meaning Composition. N400 Effects of the Interaction between Context-Induced and Lexically Stored Affordances

Contemporary semantic theories can be classified along two dimensions: (i) the way and time-course in which contextual factors influence sentence truth-conditions; and (ii) whether and to what extent comprehension involves sensory, motor and emotional processes. In order to explore this theoretical space, our ERP study investigates the time-course of the interaction between the lexically specified telic component of a noun (the function of the object to which the noun refers to, e.g., a funnel is generally used to pour liquids into containers) and an ad-hoc affordance contextually induced by the situation described in the discourse. We found that, if preceded by a neutral discourse context, a verb incongruent with the noun's telic component as in “She uses the funnel to hang her coat” elicited an enhanced N400 compared to a congruent verb as in “She uses the funnel to pour water into a container.” However, if the situation introduced in the preceding discourse induced a new function for the object as an ad-hoc affordance (e.g., the funnel is glued to the wall and the agent wants to hang the coat), we observed a crossing-over regarding the direction of the N400 effect: comparing the ad-hoc affordance-inducing context with the neutral context, the N400 for the incongruent verb was significantly reduced, whereas the N400 for the congruent verb was significantly enhanced. We explain these results as a consequence of the incorporation of the contextually triggered ad-hoc affordance into the meaning of the noun. Combining these results with an analysis of semantic similarity values between test sentences and contexts, we argue that one possibility is that the incorporation of an ad-hoc affordance may be explained on the basis of the mental simulation of concurrent motor information.

Color adaptation induced from linguistic description of color

Recent theories propose that language comprehension can influence perception at the low level of perceptual system. Here, we used an adaptation paradigm to test whether processing language caused color adaptation in the visual system. After prolonged exposure to a color linguistic context, which depicted red, green, or non-specific color scenes, participants immediately performed a color detection task, indicating whether they saw a green color square in the middle of a white screen or not. We found that participants were more likely to perceive the green color square after listening to discourses denoting red compared to discourses denoting green or conveying non-specific color information, revealing that language comprehension caused an adaptation aftereffect at the perceptual level. Therefore, semantic representation of color may have a common neural substrate with color perception. These results are in line with the simulation view of embodied language comprehension theory, which predicts that processing language reactivates the sensorimotor systems that are engaged during real experience.

Formal Semantics in the Neurology Clinic: Atypical Understanding of Aspectual Coercion in ALS Patients

Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disease of the motor system with subtle adverse effects on cognition. It is still unclear whether ALS also affects language and semantics, and if so, what aspects and processes exactly. We investigated how ALS patients understand verb phrases modified by temporal preposition phrases, e.g., "To watch TV for half an hour." Interpretation here requires operations such as aspectual coercion that add or delete elements from event structures, depending on temporal modifiers, and constraints on coercion, which make combinations with certain modifiers not viable. Using a theoretically-motivated experimental design, we observed that acceptance rates for aspectual coercion were abnormally high in ALS patients. The effect was largest for the more complex cases of coercion: not those that involve enrichment of event structures ("To switch on the TV in half an hour," where a number of failed attempts must be included in the interpretation) but those that, if applied, would result in deletion of event structure elements ("To repair the TV for half an hour"). Our experimental results are consistent with a deficit of constraints on coercion, and not with impaired semantic processes or representations, in line with recent studies suggesting that verb semantics is largely spared in ALS.

Arguments about the nature of concepts: Symbols, embodiment, and beyond

How are the meanings of words, events, and objects represented and organized in the brain? This question, perhaps more than any other in the field, probes some of the deepest and most foundational puzzles regarding the structure of the mind and brain. Accordingly, it has spawned a field of inquiry that is diverse and multidisciplinary, has led to the discovery of numerous empirical phenomena, and has spurred the development of a wide range of theoretical positions. This special issue brings together the most recent theoretical developments from the leaders in the field, representing a range of viewpoints on issues of fundamental significance to a theory of meaning representation. Here we introduce the special issue by way of pulling out some key themes that cut across the contributions that form this issue and situating those themes in the broader literature. The core issues around which research on conceptual representation can be organized are representational format, representational content, the organization of concepts in the brain, and the processing dynamics that govern interactions between the conceptual system and sensorimotor representations. We highlight areas in which consensus has formed; for those areas in which opinion is divided, we seek to clarify the relation of theory and evidence and to set in relief the bridging assumptions that undergird current discussions.

Interactions between motion and form processing in the human visual system

The predominant view of motion and form processing in the human visual system assumes that these two attributes are handled by separate and independent modules. Motion processing involves filtering by direction-selective sensors, followed by integration to solve the aperture problem. Form processing involves filtering by orientation-selective and size-selective receptive fields, followed by integration to encode object shape. It has long been known that motion signals can influence form processing in the well-known Gestalt principle of common fate; texture elements which share a common motion property are grouped into a single contour or texture region. However, recent research in psychophysics and neuroscience indicates that the influence of form signals on motion processing is more extensive than previously thought. First, the salience and apparent direction of moving lines depends on how the local orientation and direction of motion combine to match the receptive field properties of motion-selective neurons. Second, orientation signals generated by "motion-streaks" influence motion processing; motion sensitivity, apparent direction and adaptation are affected by simultaneously present orientation signals. Third, form signals generated by human body shape influence biological motion processing, as revealed by studies using point-light motion stimuli. Thus, form-motion integration seems to occur at several different levels of cortical processing, from V1 to STS.