A review of literature on the link between action observation and action language: advancing a shared semantic theory

The value of action observation in speech and language rehabilitation: A systematic review and meta-analysis

In this systematic review and meta-analysis, our aim was to identify and quantify evidence of action observation therapy (AOT) efficacy in managing language deficits in patients with aphasia. This study conducts two quantitative investigations: firstly, comparing the effects of AOT and conventional control therapy in different groups, and secondly, analyzing within AOT group to explore potential moderators of AOT effectiveness. Four databases were searched up until August 2023 to find studies utilizing AOT for aphasia management. Seven eligible studies were included. The main analyses revealed moderate evidence of improvement in naming tasks, with a large effect size (Hedge's g = 1.27, 95 %CI [0.44; 2.09], p = 0.003, I2 < 25) following AOT compared to control interventions. Furthermore, to be efficient, AOT should focus on human actions (e.g., running, jumping) rather than non-human actions (e.g., meowing or barking). These findings indicate that AOT is a promising alternative complementary approach for patients with aphasia. Future research should confirm the potential benefits of AOT with more randomized controlled studies and aim to clarify the minimal dose necessary and the possibility of transfer to various language tasks.

The cerebellum and the Mirror Neuron System: A matter of inhibition? From neurophysiological evidence to neuromodulatory implications. A narrative review

Mirror neurons show activity during both the execution (AE) and observation of actions (AO). The Mirror Neuron System (MNS) could be involved during motor imagery (MI) as well. Extensive research suggests that the cerebellum is interconnected with the MNS and may be critically involved in its activities. We gathered evidence on the cerebellum's role in MNS functions, both theoretically and experimentally. Evidence shows that the cerebellum plays a major role during AO and MI and that its lesions impair MNS functions likely because, by modulating the activity of cortical inhibitory interneurons with mirror properties, the cerebellum may contribute to visuomotor matching, which is fundamental for shaping mirror properties. Indeed, the cerebellum may strengthen sensory-motor patterns that minimise the discrepancy between predicted and actual outcome, both during AE and AO. Furthermore, through its connections with the hippocampus, the cerebellum might be involved in internal simulations of motor programs during MI. Finally, as cerebellar neuromodulation might improve its impact on MNS activity, we explored its potential neurophysiological and neurorehabilitation implications.

Shared representations of human actions across vision and language

Humans can recognize and communicate about many actions performed by others. How are actions organized in the mind, and is this organization shared across vision and language? We collected similarity judgments of human actions depicted through naturalistic videos and sentences, and tested four models of action categorization, defining actions at different levels of abstraction ranging from specific (action verb) to broad (action target: whether an action is directed towards an object, another person, or the self). The similarity judgments reflected a shared organization of action representations across videos and sentences, determined mainly by the target of actions, even after accounting for other semantic features. Furthermore, language model embeddings predicted the behavioral similarity of action videos and sentences, and captured information about the target of actions alongside unique semantic information. Together, our results show that action concepts are similarly organized in the mind across vision and language, and that this organization reflects socially relevant goals.

Kinematic observation reduces effect of aging on episodic memory performance

This study investigated the influence of kinematics observation (i.e., observing action from only the motion of the main joints of an actor) on episodic memory performance differences between young and older adults. To this end, 42 young (20-35 years) and 45 older (60-75 years) participants performed a free recall task in two different conditions: either after an encoding phase consisting of the visual presentation and reading of action verbs or after an encoding phase consisting of the visual presentation and naming of point-light displays of humans performing the same actions. Results showed a beneficial effect of point-light display encoding for both young and older participants but with a more pronounced benefit for the older participants compared to young adults. These findings are discussed in relation to the embodied view of memory which considers that memory is directly linked to the sensorimotor experiences and the environmental support hypothesis which postulates that elaborate processing can improve memory performance. In conclusion, kinematic observation could constitute an interesting potential intervention for supporting memory in older adults.

Total sleep deprivation reduces the table tennis anticipation performance of young men: A functional magnetic resonance imaging study

This study explored whether and how sleep deprivation (SD) affects sport-related anticipation. Twenty table tennis players and 28 non-athletes completed a table tennis anticipation task before and after 36 h SD. Functional magnetic resonance imaging (fMRI) data were acquired simultaneously. The results showed that, compared with the non-athletes, table tennis players had higher neural efficiency, manifested by their higher anticipation accuracy and lower frontal lobe activation. SD impaired anticipation performance, accompanied by decreased activation of the occipital and temporal lobes. Compensatory activation occurred in the left hippocampus and orbital part of the right inferior frontal gyrus (IFG) after SD in the table tennis player group, but not in the non-athlete group. The decreased accuracy of non-athletes was positively correlated with decreased activation of orbital part of the right IFG. This study's findings improve the understanding of the cognitive neuroscience mechanisms by which SD affects sport-related anticipation.

Brain Signatures of Embodied Semantics and Language: A Consensus Paper

According to embodied theories (including embodied, embedded, extended, enacted, situated, and grounded approaches to cognition), language representation is intrinsically linked to our interactions with the world around us, which is reflected in specific brain signatures during language processing and learning. Moving on from the original rivalry of embodied vs. amodal theories, this consensus paper addresses a series of carefully selected questions that aim at determining when and how rather than whether motor and perceptual processes are involved in language processes. We cover a wide range of research areas, from the neurophysiological signatures of embodied semantics, e.g., event-related potentials and fields as well as neural oscillations, to semantic processing and semantic priming effects on concrete and abstract words, to first and second language learning and, finally, the use of virtual reality for examining embodied semantics. Our common aim is to better understand the role of motor and perceptual processes in language representation as indexed by language comprehension and learning. We come to the consensus that, based on seminal research conducted in the field, future directions now call for enhancing the external validity of findings by acknowledging the multimodality, multidimensionality, flexibility and idiosyncrasy of embodied and situated language and semantic processes.

Embodiment of action-related language in the native and a late foreign language - An fMRI-study

Theories of embodied cognition postulate that language processing activates similar sensory-motor structures as when interacting with the environment. Only little is known about the neural substrate of embodiment in a foreign language (L2) as compared to the mother tongue (L1). In this fMRI study, we investigated embodiment of motor and non-motor action verbs in L1 and L2 including 31 late bilinguals. Half had German as L1 and French as L2, and the other half vice-versa. We collapsed across languages to avoid the confound between language and order of language acquisition. Region of interest analyses showed stronger activation in motor regions during L2 than during L1 processing, independently of the motor-relatedness of the verbs. Moreover, a stronger involvement of motor regions for motor-related as compared to non-motor-related verbs, similarly for L1 and L2, was found. Overall, the similarity between L1 and L2 embodiment seems to depend on individual and contextual factors.

Sentence-based mental simulations: Evidence from behavioral experiments using garden-path sentences

Language comprehenders activate mental representations of sensorimotor experiences related to the content of utterances they process. However, it is still unclear whether these sensorimotor simulations are driven by associations with words or by a more complex process of meaning composition into larger linguistic expressions, such as sentences. In two experiments, we investigated whether comprehenders indeed create sentence-based simulations. Materials were constructed such that simulation effects could only emerge from sentence meaning and not from word-based associations alone. We additionally asked when during sentence processing these simulations are constructed, using a garden-path paradigm. Participants read either a garden-path sentence (e.g., "As Mary ate the egg was in the fridge") or a corresponding unambiguous control with the same meaning and words (e.g., "The egg was in the fridge as Mary ate"). Participants then judged whether a depicted entity was mentioned in the sentence or not. In both experiments, picture response times were faster when the picture was compatible (vs. incompatible) with the sentence-based interpretation of the target entity (e.g., both for garden-path and control sentence: an unpeeled egg), suggesting that participants created simulations based on the sentence content and only operating over the sentence as a whole.

Point-light display: a new tool to improve verb recovery in patients with aphasia? A pilot study

Some studies have demonstrated that Action Observation (AO) could help patients with aphasia to recover use of verbs. However, the role of kinematics in this effect has remained unknown. The main aim was to assess the effectiveness of a complementary intervention based on the observation of action kinematics in patients with aphasia. Seven aphasic patients (3 males, 4 females) aged between 55 and 88 years participated in the studies. All patients received a classical intervention and an additional, specific intervention based on action observation. This consisted in visualizing a static image or a point-light sequence representing a human action and in trying to name the verb representing the action. In each session, 57 actions were visualized: 19 represented by a static drawing, 19 by a non-focalized point-light sequence, i.e., a point-light display with all dots in white, and 19 by a focalized point-light sequence, i.e., a point-light display (PLD) with the dots corresponding to the main limbs in yellow. Before (pre-test) and after (post-test) the intervention, each patient performed the same denomination task, in which all actions were presented in photographs. The results showed a significant improvement in performance between pre and post-test, but only when the actions were presented in focalized and non-focalized point-light sequences during the intervention. The presentation of action kinematics seems crucial in the recovery of verbs in patients with aphasia. This should be considered by speech therapists in their interventions.

The role of implicit motor simulation on action verb memory

Observation is known to improve memory for action. Previous findings linked such an effect with an easier relation processing of action components following observation compared to mere sentence reading. However, action observation also elicits implicit motor simulation, that is a processing of one's movement through the observer own motor system. We aimed to assess whether encoding of implicit motor simulation can also explain why observation is better than reading for action memory. To prevent influence of item relation processing, two studies about isolated action verbs learning were designed. In Experiment 1, action verbs were encoded with short videos of point-light human movements or with written definitions. Subsequent free recall indicated better memory for the verbs within the video clip condition. Experiment 2 compared two encoding conditions based on point-light human movement videos. Half of the verbs were learned with their normal corresponding movement (biological kinematic). For the other half of the verbs, the velocity of point-light movements was modified to create abnormal nonbiological kinematic actions. We observed better free recall for the verbs learned with biological kinematics. Taken together, those results suggest that action observation is beneficial because it allows the encoding of motor-related information (implicit motor simulation). Semantic resonance between linguistic and motor representations of action could also contribute to memory improvement. Contrary to previous studies, our results cannot be explained by an improvement of items relation processing. However, it suggests that the basic level of action verb memory is sensorimotor perception, such as implicit motor simulation.

PLAViMoP database: A new continuously assessed and collaborative 3D point-light display dataset

It was more than 45 years ago that Gunnar Johansson invented the point-light display technique. This showed for the first time that kinematics is crucial for action recognition, and that humans are very sensitive to their conspecifics' movements. As a result, many of today's researchers use point-light displays to better understand the mechanisms behind this recognition ability. In this paper, we propose PLAViMoP, a new database of 3D point-light displays representing everyday human actions (global and fine-motor control movements), sports movements, facial expressions, interactions, and robotic movements. Access to the database is free, at https://plavimop.prd.fr/en/motions . Moreover, it incorporates a search engine to facilitate action retrieval. In this paper, we describe the construction, functioning, and assessment of the PLAViMoP database. Each sequence was analyzed according to four parameters: type of movement, movement label, sex of the actor, and age of the actor. We provide both the mean scores for each assessment of each point-light display, and the comparisons between the different categories of sequences. Our results are discussed in the light of the literature and the suitability of our stimuli for research and applications.

Mapping relational links between motor imagery, action observation, action-related language, and action execution

Actions can be physically executed, observed, imagined, or simply thought about. Unifying mental processes, such as simulation, emulation, or predictive processing, are thought to underlie different action types, whether they are mental states, as in the case of motor imagery and action observation, or involve physical execution. While overlapping brain activity is typically observed across different actions which indicates commonalities, research interest is also concerned with investigating the distinct functional components of these action types. Unfortunately, untangling subtleties associated with the neurocognitive bases of different action types is a complex endeavour due to the high dimensional nature of their neural substrate (e.g., any action process is likely to activate multiple brain regions thereby having multiple dimensions to consider when comparing across them). This has impeded progress in action-related theorising and application. The present study addresses this challenge by using the novel approach of multidimensional modeling to reduce the high-dimensional neural substrate of four action-related behaviours (motor imagery, action observation, action-related language, and action execution), find the least number of dimensions that distinguish or relate these action types, and characterise their neurocognitive relational links. Data for the model comprised brain activations for action types from whole-brain analyses reported in 53 published articles. Eighty-two dimensions (i.e., 82 brain regions) for the action types were reduced to a three-dimensional model, that mapped action types in ordination space where the greater the distance between the action types, the more dissimilar they are. A series of one-way ANOVAs and post-hoc comparisons performed on the mean coordinates for each action type in the model showed that across all action types, action execution and concurrent action observation (AO)-motor imagery (MI) were most neurocognitively similar, while action execution and AO were most dissimilar. Most action types were similar on at least one neurocognitive dimension, the exception to this being action-related language. The import of the findings are discussed in terms of future research and implications for application.

The Link Between Action Verb Processing and Action Observation: A Developmental Study

In this work we aimed to assess the typical link in human development between action observation and language. For this, we studied, in 68 children aged 5-11 years of age, how action verbs can prime action representation. While children 7-8 years of age benefited from a congruent action verb prime when they had to judge an image representing an action, this effect was not present in 5-6-year-olds. Thus, the link between language and action observation changes during development at about age 7. We discussed these findings in consideration of current theories proposed to account for the action-language link.

Neural correlates of embodied action language processing: a systematic review and meta-analytic study

The neural correlates of action language processing are still debated within embodied cognition research and little is known about the flexible involvement of modality-specific pre-motor system and multimodal high-level temporo-parietal regions as a function of explicit and implicit tasks. A systematic review and the Activation likelihood estimation (ALE) meta-analyses on functional neuroimaging studies were performed to identify neural correlates of action language processing activated during explicit and implicit tasks. The contrast ALE meta-analysis revealed activation of modality-specific premotor area and inferior frontal areas during explicit action language tasks while a greater activation of posterior temporo-occipital areas emerged for implicit tasks. The conjunction analysis revealed overlap in the temporo-parietal multimodal high-level regions for both types of tasks. Functional specialization of the middle temporal gyrus was found where the more posterior-occipital part resulted activated during implicit action language tasks whereas the antero-lateral part was involved in explicit tasks. Our findings were discussed within a conceptual flexibility perspective about the involvement of both the modality-specific and multimodal brain system during action language processing depending on different types of tasks.

The link between language and action in aging

OBJECTIVE

Many studies have demonstrated the existence of a link between action verb processing and action. However, little is known about the changes in this relationship with aging.

METHOD

To assess this point, we compare the performances of younger and older people during a priming task consisting of judging whether an image contains a human after listening to an action verb.

RESULTS

In accordance with previous literature, the results showed that younger people were faster to detect the presence of a human in the image in congruent conditions, namely, when the action verb and the image refer to the same action. However, this effect was not present in older adults' participants.

CONCLUSION

These findings suggest that the link between action and language decreases with age. We discuss these findings in the context of the embodied view of cognition.