Contextual modulation of motor resonance during the observation of everyday actions

Contextual expectations shape the motor coding of movement kinematics during the prediction of observed actions: A TMS study

Contextual information may shape motor resonance and support intention understanding during observation of incomplete, ambiguous actions. It is unclear, however, whether this effect is contingent upon kinematics ambiguity or contextual information is continuously integrated with kinematics to predict the overarching action intention. Moreover, a differentiation between the motor mapping of the intention suggested by context or kinematics has not been clearly demonstrated. In a first action execution phase, 29 participants were asked to perform reaching-to-grasp movements towards big or small food objects with the intention to eat or to move; electromyography from the First Dorsal Interosseous (FDI) and Abductor Digiti Minimi (ADM) was recorded. Depending on object size, the intentions to eat or to move were differently implemented by a whole-hand or a precision grip kinematics, thus qualifying an action-muscle dissociation. Then, in a following action prediction task, the same participants were asked to observe an actor performing the same actions and to predict his/her intention while motor resonance was assessed for the same muscles. Of note, videos were interrupted at early or late action phases, and actions were embedded in contexts pointing toward an eating or a moving intention, congruently or incongruently with kinematics. We found greater involvement of the FDI or ADM in the execution of precision or whole-hand grips, respectively. Crucially, this pattern of activation was mirrored during observation of the same actions in congruent contexts, but it was cancelled out or reversed in the incongruent ones, either when videos were interrupted at either early or long phases of action deployment. Our results extend previous evidence by showing that contextual information shapes motor resonance not only under conditions of perceptual uncertainty but also when more informative kinematics is available.

Visual background information modulates motor contagions in humans

Motor contagions refer to implicit effects induced by the observation of actions made by others on one's own actions. A plethora of studies conducted over the last two decades have demonstrated that both observed and predicted actions can induce various kinds of motor contagions in a human observer. However, motor contagions have always been investigated with regard to different features of an observed action, and it remains unclear whether the background environment in which an observed action takes place modulates motor contagions as well. Here, we investigated participant movements in an empirical hand steering task during which the participants were required to move a cursor through a visual channel after being presented with videos of an actor performing the same task. We manipulated the congruency between the actions shown in the video and the background channels and examined whether and how they affected the participants' own movements. We observed a clear interaction between the observed action and its background. The movement time of the participants' actions tended to increase or decrease depending on whether they observed a faster or slower movement, respectively, and these changes were amplified if the background was not congruent with the action contained within it. These results suggest that background information can modulate motor contagions in humans.

Excitatory/inhibitory motor balance reflects individual differences during joint action coordination

Joint action (JA) is a continuous process of motor co-regulation based on the integration of contextual (top-down) and kinematic (bottom-up) cues from partners. The fine equilibrium between excitation and inhibition in sensorimotor circuits is, thus, central to such a dynamic process of action selection and execution. In a bimanual task adapted to become a unimanual JA task, the participant held a bottle (JA), while a confederate had to reach and unscrew either that bottle or another stabilized by a mechanical clamp (No_JA). Prior knowledge was manipulated in each trial such that the participant knew (K) or not (No_K) the target bottle in advance. Online transcranial magnetic stimulation (TMS) was administered at action-relevant landmarks to explore corticospinal excitability (CSE) and inhibition (cortical silent period [cSP]). CSE was modulated early on before the action started if prior information was available. In contrast, cSP modulation emerged later during the reaching action, regardless of prior information. These two indexes could thus reflect the concurrent elaboration of contextual priors (top-down) and the online sampling of partner's kinematic cues (bottom-up). Furthermore, participants selected either one of two possible behavioural strategies, preferring early or late force exertion on the bottle. One translates into a reduced risk of motor coordination failure and the other into reduced metabolic expenditure. Each strategy was characterised by a specific excitatory/inhibitory profile. In conclusion, the study of excitatory/inhibitory balance paves the way for the neurophysiological determination of individual differences in the combination of top-down and bottom-up processing during JA coordination.

Effects of Action Observation Plus Motor Imagery Administered by Immersive Virtual Reality on Hand Dexterity in Healthy Subjects

Action observation and motor imagery (AOMI) are commonly delivered through a laptop screen. Immersive virtual reality (VR) may enhance the observer's embodiment, a factor that may boost AOMI effects. The study aimed to investigate the effects on manual dexterity of AOMI delivered through immersive VR compared to AOMI administered through a laptop. To evaluate whether VR can enhance the effects of AOMI, forty-five young volunteers were enrolled and randomly assigned to the VR-AOMI group, who underwent AOMI through immersive VR, the AOMI group, who underwent AOMI through a laptop screen, or the control group, who observed landscape video clips. All participants underwent a 5-day treatment, consisting of 12 min per day. We investigated between and within-group differences after treatments relative to functional manual dexterity tasks using the Purdue Pegboard Test (PPT). This test included right hand (R), left hand (L), both hands (B), R + L + B, and assembly tasks. Additionally, we analyzed kinematics parameters including total and sub-phase duration, peak and mean velocity, and normalized jerk, during the Nine-Hole Peg Test to examine whether changes in functional scores may also occur through specific kinematic patterns. Participants were assessed at baseline (T0), after the first training session (T1), and at the end of training (T2). A significant time by group interaction and time effects were found for PPT, where both VR-AOMI and AOMI groups improved at the end of training. Larger PPT-L task improvements were found in the VR-AOMI group (d: 0.84, CI95: 0.09-1.58) compared to the AOMI group from T0 to T1. Immersive VR used for the delivery of AOMI speeded up hand dexterity improvements.

Contextual Priors Shape Action Understanding before and beyond the Unfolding of Movement Kinematics

Previous studies have shown that contextual information may aid in guessing the intention underlying others' actions in conditions of perceptual ambiguity. Here, we aimed to evaluate the temporal deployment of contextual influence on action prediction with increasing availability of kinematic information during the observation of ongoing actions. We used action videos depicting an actor grasping an object placed on a container to perform individual or interpersonal actions featuring different kinematic profiles. Crucially, the container could be of different colors. First, in a familiarization phase, the probability of co-occurrence between each action kinematics and color cues was implicitly manipulated to 80% and 20%, thus generating contextual priors. Then, in a testing phase, participants were asked to predict action outcome when the same action videos were occluded at five different timeframes of the entire movement, ranging from when the actor was still to when the grasp of the object was fully accomplished. In this phase, all possible action-contextual cues' associations were equally presented. The results showed that for all occlusion intervals, action prediction was more facilitated when action kinematics deployed in high- than low-probability contextual scenarios. Importantly, contextual priors shaped action prediction even in the latest occlusion intervals, where the kinematic cues clearly unveiled an action outcome that was previously associated with low-probability scenarios. These residual contextual effects were stronger in individuals with higher subclinical autistic traits. Our findings highlight the relative contribution of kinematic and contextual information to action understanding and provide evidence in favor of their continuous integration during action observation.

Motor cognition in plants: from thought to real experiments

Motor cognition involves the process of planning and executing goal-directed movements and recognizing, anticipating, and interpreting others' actions. Motor cognitive functions are generally associated with the presence of a brain and are ascribed only to humans and other animal species. A growing body of evidence suggests that aneural organisms, like climbing plants, exhibit behaviors driven by the intention to achieve goals, challenging our understanding of cognition. Here, we propose an inclusive perspective under motor cognition to explain climbing plants' behavior. We will first review our empirical research based on kinematical analysis to understand movement in pea plants. Then, we situate this empirical research within the current theoretical debate aimed at extending the principles of cognition to aneural organisms. A novel comparative perspective that considers the perception-action cycle, involving transforming perceived environmental elements into intended movement patterns, is provided.

The social relevance and the temporal constraints of motor resonance in humans

In humans, motor resonance effects can be tracked by measuring the enhancement of corticospinal excitability by action observation. Uncovering factors driving motor resonance is crucial for optimizing action observation paradigms in experimental and clinical settings. In the present study, we deepen motor resonance properties for grasping movements. Thirty-five healthy subjects underwent an action observation task presenting right-hand grasping movements differing from their action goal. Single-pulse transcranial magnetic stimulation was applied over the left primary motor cortex at 100, 200, or 300 ms from the onset of the visual stimulus depicting the action. Motor-evoked potentials were recorded from four muscles of the right hand and forearm. Results show a muscle-specific motor resonance effect at 200 ms after movement but selectively for observing a socially relevant grasp towards another human being. This effect correlates with observers' emotional empathy scores, and it was followed by inhibition of motor resonance at 300 ms post-stimulus onset. No motor resonance facilitation emerged while observing intransitive hand movement or object grasping. This evidence highlights the social side of motor resonance and its dependency on temporal factors.

Cognitive predictors of Social processing in congenital atypical development

According to current accounts of social cognition, the emergence of verbal and non-verbal components of social perception might rely on the acquisition of different cognitive abilities. These components might be differently sensitive to the pattern of neuropsychological impairments in congenital neurodevelopmental disorders. Here, we explored the association between social and non-social cognitive domains by administering subtests of the NEPSY-II battery to 92 patients with Intellectual and Developmental Disability (IDD). Regardless the level of intellectual functioning and presence of congenital brain malformations, results revealed that visuospatial skills predicted emotion recognition and verbal component of Theory of Mind, whereas imitation predicted the non-verbal one. Future interventions might focus on spatial and sensorimotor abilities to boost the development of social cognition in IDD.

A large-scale fMRI dataset for human action recognition

Human action recognition is a critical capability for our survival, allowing us to interact easily with the environment and others in everyday life. Although the neural basis of action recognition has been widely studied using a few action categories from simple contexts as stimuli, how the human brain recognizes diverse human actions in real-world environments still needs to be explored. Here, we present the Human Action Dataset (HAD), a large-scale functional magnetic resonance imaging (fMRI) dataset for human action recognition. HAD contains fMRI responses to 21,600 video clips from 30 participants. The video clips encompass 180 human action categories and offer a comprehensive coverage of complex activities in daily life. We demonstrate that the data are reliable within and across participants and, notably, capture rich representation information of the observed human actions. This extensive dataset, with its vast number of action categories and exemplars, has the potential to deepen our understanding of human action recognition in natural environments.

Error observation as a window on performance monitoring in social contexts? A systematic review

Living in a social world requires social monitoring, i.e., the ability to keep track of others' actions and mistakes. Here, we demonstrate the good reliability of the behavioral and neurophysiological indexes ascribed to social monitoring. We also show that no consensus exists on the cognitive bases of this phenomenology and discuss three alternative hypotheses: (i) the direct matching hypothesis, postulating that observed errors are processed through automatic simulation; (ii) the attentional hypothesis, considering errors as unexpected events that take resources away from task processing; and (iii) the goal representation hypothesis, which weighs social error monitoring depending on how relevant the other's task is to the observer's goals. To date, evidence on the role played by factors that could help to disentangle these hypotheses (e.g., the human vs. non-human nature of the actor, the error rate, and the reward context) is insufficient, although the goal representation hypothesis seems to receive more support. Theory-driven experimental designs are needed to enlighten this debate and clarify the role of error monitoring during interactive exchanges.

Effects of the social context on the neurophysiological correlates of observed error monitoring

Monitoring the motor performance of others, including the correctness of their actions, is crucial for the human behavior. However, while performance (and error) monitoring of the own actions has been studied extensively at the neurophysiological level, the corresponding studies on monitoring of others' errors are scarce, especially for ecological actions. Moreover, the role of the context of the observed action has not been sufficiently explored. To fill this gap, the present study investigated electroencephalographic (EEG) indices of error monitoring during observation of images of interrupted reach-to-grasp actions in social (an object held in another person's hand) and non-social (an object placed on a table) contexts. Analysis in time- and time-frequency domain showed that, at the level of conscious error awareness, there were no effects of the social context (observed error positivity was present for erroneous actions in both contexts). However, the effects of the context were present at the level of hand image processing: observing erroneous actions in the non-social context was related to larger occipito-temporal N1 and theta activity, while in the social context this pattern was reversed, i.e., larger N1 and theta activity were present for the correct actions. These results suggest that, in case of easily predictable ecological actions, action correctness is processed as early as at the level of hand image perception, since the hand posture conveys information about the action (e.g., motor intention). The social context of actions might make the correct actions more salient, possibly through the saliency of the correctly achieved common goal.

“Left and right prefrontal routes to action comprehension”

Successful action comprehension requires the integration of motor information and semantic cues about objects in context. Previous evidence suggests that while motor features are dorsally encoded in the fronto-parietal action observation network (AON); semantic features are ventrally processed in temporal structures. Importantly, these dorsal and ventral routes seem to be preferentially tuned to low (LSF) and high (HSF) spatial frequencies, respectively. Recently, we proposed a model of action comprehension where we hypothesized an additional route to action understanding whereby coarse LSF information about objects in context is projected to the dorsal AON via the prefrontal cortex (PFC), providing a prediction signal of the most likely intention afforded by them. Yet, this model awaits for experimental testing. To this end, we used a perturb-and-measure continuous theta burst stimulation (cTBS) approach, selectively disrupting neural activity in the left and right PFC and then evaluating the participant's ability to recognize filtered action stimuli containing only HSF or LSF. We find that stimulation over PFC triggered different spatial-frequency modulations depending on lateralization: left-cTBS and right-cTBS led to poorer performance on HSF and LSF action stimuli, respectively. Our findings suggest that left and right PFC exploit distinct spatial frequencies to support action comprehension, providing evidence for multiple routes to social perception in humans.

Motor invariants in action execution and perception

The nervous system is sensitive to statistical regularities of the external world and forms internal models of these regularities to predict environmental dynamics. Given the inherently social nature of human behavior, being capable of building reliable predictive models of others' actions may be essential for successful interaction. While social prediction might seem to be a daunting task, the study of human motor control has accumulated ample evidence that our movements follow a series of kinematic invariants, which can be used by observers to reduce their uncertainty during social exchanges. Here, we provide an overview of the most salient regularities that shape biological motion, examine the role of these invariants in recognizing others' actions, and speculate that anchoring socially-relevant perceptual decisions to such kinematic invariants provides a key computational advantage for inferring conspecifics' goals and intentions.

Decoding motor expertise from fine-tuned oscillatory network organization

Can motor expertise be robustly predicted by the organization of frequency-specific oscillatory brain networks? To answer this question, we recorded high-density electroencephalography (EEG) in expert Tango dancers and naïves while viewing and judging the correctness of Tango-specific movements and during resting. We calculated task-related and resting-state connectivity at different frequency-bands capturing task performance (delta [δ], 1.5-4 Hz), error monitoring (theta [θ], 4-8 Hz), and sensorimotor experience (mu [μ], 8-13 Hz), and derived topographical features using graph analysis. These features, together with canonical expertise measures (i.e., performance in action discrimination, time spent dancing Tango), were fed into a data-driven computational learning analysis to test whether behavioral and brain signatures robustly classified individuals depending on their expertise level. Unsurprisingly, behavioral measures showed optimal classification (100%) between dancers and naïves. When considering brain models, the task-based classification performed well (~73%), with maximal discrimination afforded by theta-band connectivity, a hallmark signature of error processing. Interestingly, mu connectivity during rest outperformed (100%) the task-based approach, matching the optimal classification of behavioral measures and thus emerging as a potential trait-like marker of sensorimotor network tuning by intense training. Overall, our findings underscore the power of fine-tuned oscillatory network signatures for capturing expertise-related differences and their potential value in the neuroprognosis of learning outcomes.

Kinematic-Based Classification of Social Gestures and Grasping by Humans and Machine Learning Techniques

The affective motion of humans conveys messages that other humans perceive and understand without conventional linguistic processing. This ability to classify human movement into meaningful gestures or segments plays also a critical role in creating social interaction between humans and robots. In the research presented here, grasping and social gesture recognition by humans and four machine learning techniques (k-Nearest Neighbor, Locality-Sensitive Hashing Forest, Random Forest and Support Vector Machine) is assessed by using human classification data as a reference for evaluating the classification performance of machine learning techniques for thirty hand/arm gestures. The gestures are rated according to the extent of grasping motion on one task and the extent to which the same gestures are perceived as social according to another task. The results indicate that humans clearly rate differently according to the two different tasks. The machine learning techniques provide a similar classification of the actions according to grasping kinematics and social quality. Furthermore, there is a strong association between gesture kinematics and judgments of grasping and the social quality of the hand/arm gestures. Our results support previous research on intention-from-movement understanding that demonstrates the reliance on kinematic information for perceiving the social aspects and intentions in different grasping actions as well as communicative point-light actions.

From Affordances to Abstract Words: The Flexibility of Sensorimotor Grounding

The sensorimotor system plays a critical role in several cognitive processes. Here, we review recent studies documenting this interplay at different levels. First, we concentrate on studies that have shown how the sensorimotor system is flexibly involved in interactions with objects. We report evidence demonstrating how social context and situations influence affordance activation, and then focus on tactile and kinesthetic components in body-object interactions. Then, we turn to word use, and review studies that have shown that not only concrete words, but also abstract words are grounded in the sensorimotor system. We report evidence that abstract concepts activate the mouth effector more than concrete concepts, and discuss this effect in light of studies on adults, children, and infants. Finally, we pinpoint possible sensorimotor mechanisms at play in the acquisition and use of abstract concepts. Overall, we show that the involvement of the sensorimotor system is flexibly modulated by context, and that its role can be integrated and flanked by that of other systems such as the linguistic system. We suggest that to unravel the role of the sensorimotor system in cognition, future research should fully explore the complexity of this intricate, and sometimes slippery, relation.

Contextual Priors Guide Perception and Motor Responses to Observed Actions

In everyday-life scenarios, prior expectations provided by the context in which actions are embedded support action prediction. However, it is still unclear how newly learned action-context associations can drive our perception and motor responses. To fill this gap, we measured behavioral (Experiment 1) and motor responses (Experiment 2) during two tasks requiring the prediction of occluded actions or geometrical shapes. Each task consisted of an implicit probabilistic learning and a test phase. During learning, we exposed participants to videos showing specific associations between a contextual cue and a particular action or shape. During the test phase, videos were earlier occluded to reduce the amount of sensorial information and induce participants to use the implicitly learned action/shape-context associations for disambiguation. Results showed that reliable contextual cues made participants more accurate in identifying the unfolding action or shape. Importantly, motor responses were modulated by contextual probability during action, but not shape prediction. Particularly, in conditions of perceptual uncertainty the motor system coded for the most probable action based on contextual informativeness, regardless of action kinematics. These findings suggest that contextual priors can shape motor responses to action observation beyond mere kinematics mapping.

Adults Do Not Distinguish Action Intentions Based on Movement Kinematics Presented in Naturalistic Settings

Predicting others’ actions is an essential part of acting in the social world. Action kinematics have been proposed to be a cue about others’ intentions. It is still an open question as to whether adults can use kinematic information in naturalistic settings when presented as a part of a richer visual scene than previously examined. We investigated adults’ intention perceptions from kinematics using naturalistic stimuli in two experiments. In experiment 1, thirty participants watched grasp-to-drink and grasp-to-place movements and identified the movement intention (to drink or to place), whilst their mouth-opening muscle activity was measured with electromyography (EMG) to examine participants’ motor simulation of the observed actions. We found anecdotal evidence that participants could correctly identify the intentions from the action kinematics, although we found no evidence for increased activation of their mylohyoid muscle during the observation of grasp-to-drink compared to grasp-to-place actions. In pre-registered experiment 2, fifty participants completed the same task online. With the increased statistical power, we found strong evidence that participants were not able to discriminate intentions based on movement kinematics. Together, our findings suggest that the role of action kinematics in intention perception is more complex than previously assumed. Although previous research indicates that under certain circumstances observers can perceive and act upon intention-specific kinematic information, perceptual differences in everyday scenes or the observers’ ability to use kinematic information in more naturalistic scenes seems limited.

Functional Correlates of Action Observation of Gait in Patients with Parkinson's Disease

Background

Action observation (AO) relies on the mirror neuron system (MNS) and has been proposed as a rehabilitation tool in Parkinson's disease (PD), in particular for gait disorder such as freezing of gait (FOG). In this study, we aimed to explore the brain functional correlates of the observation of human gait in PD patients with (FOG+) and without (FOG-) FOG and to investigate a possible relationship between AO-induced brain activation and gait performance.

Methods

Fifty-four participants were enrolled in the study (15 PD FOG+; 18 PD FOG-; 21 healthy subjects (HS)) which consisted of two tasks in two separate days: (i) gait assessment and (ii) task-fMRI during AO of gait. Differences between patients with PD (FOG+ and FOG-) and HS were assessed at the level of behavioral and functional analysis.

Results

Gait parameters, including gait velocity, stride length, and their coefficients of variability (CV), were different in PD patients compared to HS, whereas gait performance was similar between FOG+ and FOG-. The PD group, compared to HS, presented reduced functional activation in the frontal, cingulum, and parietooccipital regions. Reduced activity was more pronounced in the FOG+ group, compared to both HS and FOG- groups. Gait variability positively correlated with precuneus neural activity in the FOG+ group. Discussion. Patients with PD present a reduced functional activity during AO of gait, especially if FOG+. A baseline knowledge of the neural correlates of AO of gait in the clinical routine “on” status would help for the design of future AO rehabilitative interventions.

Proactive gaze is present during biological and non-biological motion observation

Others' action observation activates in the observer a coordinated hand-eye motor program, covert for the hand (i.e. motor resonance), and overt for the eye (i.e. proactive gaze), similar to that of the observed agent. The biological motion hypothesis of action anticipation claims that proactive gaze occurs only in the presence of biological motion, and that kinematic information is sufficient to determine the anticipation process. The results of the present study did not support the biological motion hypothesis of action anticipation. Specifically, proactive gaze was present during observation of both a biological accelerated-decelerated motion and a non-biological constant velocity motion (Experiment 1), in the presence of a barrier able to restrict differences between the two kinematics to the motion profile of individual markers prior to contact (Experiment 2), but only if an object was present at the end point of the movement trajectory (Experiment 3). Furthermore, proactive gaze was found independently of the presence of end effects temporally congruent with the instant in which the movement stopped (Experiments 4, and 5). We propose that the involvement of the observer's motor system is not restricted to when the agent moves with natural kinematics, and it is mandatory whenever the presence of an agent or a goal is evident, regardless of physical appearance, natural kinematics, and the possibility to identify the action behind the stimulus.

Autistic Traits Differently Account for Context-Based Predictions of Physical and Social Events

Autism is associated with difficulties in making predictions based on contextual cues. Here, we investigated whether the distribution of autistic traits in the general population, as measured through the Autistic Quotient (AQ), is associated with alterations of context-based predictions of social and non-social stimuli. Seventy-eight healthy participants performed a social task, requiring the prediction of the unfolding of an action as interpersonal (e.g., to give) or individual (e.g., to eat), and a non-social task, requiring the prediction of the appearance of a moving shape as a short (e.g., square) or a long (e.g., rectangle) figure. Both tasks consisted of (i) a familiarization phase, in which the association between each stimulus type and a contextual cue was manipulated with different probabilities of co-occurrence, and (ii) a testing phase, in which visual information was impoverished by early occlusion of video display, thus forcing participants to rely on previously learned context-based associations. Findings showed that the prediction of both social and non-social stimuli was facilitated when embedded in high-probability contexts. However, only the contextual modulation of non-social predictions was reduced in individuals with lower 'Attention switching' abilities. The results provide evidence for an association between weaker context-based expectations of non-social events and higher autistic traits.

Spatial frequency tuning of motor responses reveals differential contribution of dorsal and ventral systems to action comprehension

Understanding object-directed actions performed by others is central to everyday life. This ability is thought to rely on the interaction between the dorsal action observation network (AON) and a ventral object recognition pathway. On this view, the AON would encode action kinematics, and the ventral pathway, the most likely intention afforded by the objects. However, experimental evidence supporting this model is still scarce. Here, we aimed to disentangle the contribution of dorsal vs. ventral pathways to action comprehension by exploiting their differential tuning to low-spatial frequencies (LSFs) and high-spatial frequencies (HSFs). We filtered naturalistic action images to contain only LSF or HSF and measured behavioral performance and corticospinal excitability (CSE) using transcranial magnetic stimulation (TMS). Actions were embedded in congruent or incongruent scenarios as defined by the compatibility between grips and intentions afforded by the contextual objects. Behaviorally, participants were better at discriminating congruent actions in intact than LSF images. This effect was reversed for incongruent actions, with better performance for LSF than intact and HSF. These modulations were mirrored at the neurophysiological level, with greater CSE facilitation for congruent than incongruent actions for HSF and the opposite pattern for LSF images. Finally, only for LSF did we observe CSE modulations according to grip kinematics. While results point to differential dorsal (LSF) and ventral (HSF) contributions to action comprehension for grip and context encoding, respectively, the negative congruency effect for LSF images suggests that object processing may influence action perception not only through ventral-to-dorsal connections, but also through a dorsal-to-dorsal route involved in predictive processing.

The last chance to pass the ball: investigating the role of temporal expectation and motor resonance in processing temporal errors in motor actions

Humans can acquire information on others' motor outputs (action prediction) and intentions (action understanding) according to their individual motor repertoire and to the detected gesture's features (e.g. temporal patterns). We aimed at dissociating between action prediction and action understanding abilities in soccer players and novices observing soccer action videos including correct timing pass (CTP) or delayed pass (DP). First, we used an occluding paradigm to evaluate participants' ability to predict the correct time to pass the ball. Although soccer players showed reduced reaction times, all subjects showed a similar pattern of performance: during DP observation, responses appeared delayed with respect to the other conditions but anticipated with respect to the observed DP. In a separate experiment, we investigated the ability to recognize CTP vs DP and the modulation of primary motor cortex (M1) excitability associated to video observation. Only soccer players showed selective modulation of M1 according to the plausibility of the observed action, with increased excitability during the observation of the CTP and in a phase preceding the DP. In conclusion, action prediction ability seems to be independent from the individual motor repertoire. By contrast, only subjects with previously acquired sensorimotor skills are able to infer the observed action's long-term intention.

Action processing in the motor system: Transcranial Magnetic Stimulation (TMS) evidence of shared mechanisms in the visual and linguistic modalities

In two experiments, we compared the dynamics of corticospinal excitability when processing visually or linguistically presented tool-oriented hand actions in native speakers and sequential bilinguals. In a third experiment we used the same procedure to test non-motor, low-level stimuli, i.e. scrambled images and pseudo-words. Stimuli were presented in sequence: pictures (tool + tool-oriented hand action or their scrambled counterpart) and words (tool noun + tool-action verb or pseudo-words). Experiment 1 presented German linguistic stimuli to native speakers, while Experiment 2 presented English stimuli to non-natives. Experiment 3 tested Italian native speakers. Single-pulse trascranial magnetic stimulation (spTMS) was applied to the left motor cortex at five different timings: baseline, 200 ms after tool/noun onset, 150, 350 and 500 ms after hand/verb onset with motor-evoked potentials (MEPs) recorded from the first dorsal interosseous (FDI) and abductor digiti minimi (ADM) muscles. We report strong similarities in the dynamics of corticospinal excitability across the visual and linguistic modalities. MEPs' suppression started as early as 150 ms and lasted for the duration of stimulus presentation (500 ms). Moreover, we show that this modulation is absent for stimuli with no motor content. Overall, our study supports the notion of a core, overarching system of action semantics shared by different modalities.

Physical and observational practices of unusual actions prime action verb processing

Numerous studies have highlighted a strong relationship between language and sensorimotor processes, showing, for example, that perceiving an action influences subsequent language processing. Moreover, previous studies have demonstrated that the context in which actions are perceived is crucial to enable this action-language relationship. In particular, action verb processing is facilitated when an action is perceived in its usual context (e.g., someone watering a plant) but not in an unusual context (e.g., someone watering a computer). This difference could be explained in terms of experience; because people always practice actions in accordance with the context, they have no (visual or motor) experience related to the unusual context. The aim of the present study was to test this assumption by assessing and comparing the effect of physical practice and observational learning on the action-language relationship. The results of two experiments showed a facilitation effect of both training methods. Whereas usual actions systematically prime action verb processing, the link between action and language appears for unusual actions only after training by practicing (experiment 1, physical practice) or observing (experiment 2, observational learning). Overall, these findings support the role of experience in the activation of sensorimotor representations during action verb processing.

Low or High-Level Motor Coding? The Role of Stimulus Complexity

Transcranial magnetic stimulation (TMS) studies have shown that observing an action induces activity in the onlooker's motor system. In light of the muscle specificity and time-locked mirroring nature of the effect, this motor resonance has been traditionally viewed as an inner automatic replica of the observed movement. Notably, studies highlighting this aspect have classically considered movement in isolation (i.e., using non-realistic stimuli such as snapshots of hands detached from background). However, a few recent studies accounting for the role of contextual cues, motivational states, and social factors, have challenged this view by showing that motor resonance is not completely impervious to top-down modulations. A debate is still present. We reasoned that motor resonance reflects the inner replica of the observed movement only when its modulation is assessed during the observation of movements in isolation. Conversely, the presence of top-down modulations of motor resonance emerges when other high-level factors (i.e., contextual cues, past experience, social, and motivational states) are taken into account. Here, we attempt to lay out current TMS studies assessing this issue and discuss the results in terms of their potential to favor the inner replica or the top-down modulation hypothesis. In doing so, we seek to shed light on this actual debate and suggest specific avenues for future research, highlighting the need for a more ecological approach when studying motor resonance phenomenon.

Uncertainty in contextual and kinematic cues jointly modulates motor resonance in primary motor cortex

Contextual information accompanying others' actions modulates "motor resonance", i.e., neural activity within motor areas that is elicited by movement observation. One possibility is that we weigh and combine such information in a Bayesian manner according to their relative uncertainty. Therefore, contextual information becomes particularly useful when others' actions are difficult to discriminate. It is unclear, however, whether this uncertainty modulates the neural activity in primary motor cortex (M1) during movement observation. Here, we applied single-pulse transcranial magnetic stimulation (TMS) while subjects watched different grasping actions. We operationalized motor resonance as grip-specific modulation of corticomotor excitability measured in the index (FDI) versus the little finger abductor (ADM). We experimentally modulated either the availability of kinematic information ( experiment 1) or the reliability of contextual cues ( experiment 2). Our results indicate that even in the absence of movement kinematics, reliable contextual information is enough to trigger significant muscle-specific corticomotor excitability changes in M1, which are strongest when both kinematics and contextual information are available. These findings suggest that bottom-up mechanisms that activate motor representations as a function of the observed kinematics and top-down mechanisms that activate motor representations associated with arbitrary cues converge in M1. NEW & NOTEWORTHY Our study reveals new neurophysiological insights in support of the Bayesian account of action observation by showing that "motor resonance", i.e., neural activity evoked by observing others' actions, incorporates the uncertainty related to both contextual (prior beliefs) and kinematic (sensory evidence) cues. Notably, we show that muscle-specific modulation of M1 is strongest when context and movement kinematics are available, and it can be elicited even in the absence of movement kinematics.

Autistic traits predict poor integration between top-down contextual expectations and movement kinematics during action observation

Autism is associated with difficulties in predicting and understanding other people’s actions. There is evidence that autistic traits are distributed across a spectrum and that subclinical forms of autistic impairments can also be measured in the typical population. To investigate the association between autistic traits and motor responses to others’ actions, we quantified these traits and measured cortico-spinal excitability modulations in M1 during the observation of actions embedded in congruent, incongruent and ambiguous contexts. In keeping with previous studies, we found that actions observed in congruent contexts elicited an early facilitation of M1 responses, and actions observed in incongruent contexts, resulted in a later inhibition. Correlational analysis revealed no association between autistic traits and the facilitation for congruent contexts. However, we found a significant correlation between motor inhibition and autistic traits, specifically related to social skills and attention to details. Importantly, the influence of these factors was independent from each other, and from the observer’s gender. Thus, results suggest that individuals with higher social deficits and greater detail-processing style are more impaired in suppressing action simulation in M1 when a mismatch between kinematics and context occurs. This points to difficult integration between kinematics and contextual representations in the autistic-like brain.

When context modulates the influence of action observation on language processing

Numerous studies in the field of embodied cognition have shown a crosstalk between language and sensorimotor processes. In particular, it has been demonstrated that perceiving an action influences subsequent language processing. However, when studying the effect of action observation on language processing it has not been considered whether the context of action presentation could modulate this influence. To test this assumption, the participants in our study observed a prime, specifically a cartoon picture of a person performing an action in either a usual or an unusual context, and then had to perform a semantic decision task involving action verbs that could be congruent or incongruent with the action in the prime. Data analyses showed a significant difference on response times for congruent action verbs compared with incongruent action verbs in the usual context, whereas no difference was observed in the unusual context. This finding indicates that the influence of action observation on language appears only with usual actions, suggesting that the context of action presentation is crucial to enable the influence of action observation on action verbs processing.

Observation of an action with a congruent contextual background facilitates corticospinal excitability: A combined TMS and eye-tracking experiment

Action observation produces activity in similar regions of the brain to those involved in action execution and can offer an effective intervention for motor (re)learning, although optimal viewing conditions for such interventions remain to be established. In this experiment, single-pulse transcranial magnetic stimulation (TMS) and eye-tracking were used simultaneously to investigate the effect of manipulating background context on both corticospinal excitability and visual attention during action observation. Twenty-four participants observed four different videos: (i) a static hand holding a sponge (control condition); and an index finger-thumb pinch of a sponge against (ii) a plain black background; (iii) a background containing objects that were incongruent with the observed action; and (iv) a background containing objects that were congruent with the observed action. TMS was delivered to the hand representation of the left primary motor cortex, and motor evoked potentials were recorded from the first dorsal interosseous and abductor digiti minimi muscles of the right hand. Eye movements were recorded throughout the experiment. Results indicated that corticospinal excitability was facilitated during the congruent context condition compared to both the static hand and plain black background conditions. In addition, the number of fixations and percentage of time participants spent fixating on the background scene were significantly greater during the incongruent and congruent conditions compared to the static hand and plain black background conditions. These results indicate that the provision of additional visual information that is congruent with the observed movement contributes to a facilitation of corticospinal excitability by providing the observer with information regarding the goal and intention of the observed action. Providing congruent contextual information may enhance the efficacy of action observation interventions for motor (re)learning.

An Emotion-Enriched Context Influences the Effect of Action Observation on Cortical Excitability

Observing other people in action activates the “mirror neuron system” that serves for action comprehension and prediction. Recent evidence suggests that this function requires a high level codification triggered not only by components of motor behavior, but also by the environment where the action is embedded. An overlooked component of action perceiving is the one related to the emotional information provided by the context where the observed action takes place. Indeed, whether valence and arousal associated to an emotion might exert an influence on motor system activation during action observation has not been assessed so far. Here, cortico-spinal excitability of the left motor cortex was recorded in three groups of subjects. In the first condition, motor-evoked potential (MEPs) were recorded from a muscle involved in the grasping movement (i.e., abductor pollicis brevis, APB) while participants were watching the same reach-to-grasp movement embedded in contexts with negative emotional valence, but different levels of arousal: sadness (low arousal), and disgust (high arousal) (“Context plus Movement-APB” condition). In the second condition, MEPs were recorded from APB muscle while participants were observing static images representing the contexts in which the movement observed by participants in “Context plus Movement-APB” condition took place (“Context Only-APB” condition). Finally, in the third condition, MEPS were recorded from a muscle not involved in the grasping action, i.e., abductor digiti minimi, ADM, while participants were watching the same videos shown during the “Context plus Movement-APB” condition (“Context plus Movement-ADM” condition). Results showed a greater increase of cortical excitability only during the observation of the hand moving in the context eliciting disgust, and these changes were specific for the muscle involved in the observed action. Our findings show that the emotional context in which a movement occurs modulates motor resonance and that the combination of negative valence/high arousal drives the greater response in the observer’s mirror neuron system in a strictly muscle specific fashion.

Tracking the Time Course of Top-Down Contextual Effects on Motor Responses during Action Comprehension

Context plays a key role in coding high-level components of others' behavior, including the goal and the intention of an observed action. However, little is known about its possible role in shaping lower levels of action processing, such as simulating action kinematics and muscular activity. Furthermore, there is no evidence regarding the time course and the neural mechanisms subserving this modulation. To address these issues, we combined single-pulse transcranial magnetic stimulation and motor-evoked potentials while healthy humans watched videos of everyday actions embedded in congruent, incongruent, or ambiguous contexts. Video endings were occluded from view and participants had to predict action unfolding. Transcranial magnetic stimulation was delivered at 80, 240, and 400 ms after action onset. An earlier selective facilitation of motor resonance occurring at 240 ms was observed for actions embedded in congruent contexts, compared with those occurring in incongruent and ambiguous ones. Later on, at 400 ms, a selective inhibition of motor resonance was found for actions embedded in incongruent contexts, compared with those taking place in congruent and ambiguous ones. No modulations were observed at 80 ms. Together, these findings indicate that motor resonance can be modulated by contextual information with different timings, depending on the (in)congruency between the different levels of action representation. Furthermore, the different time course of these effects suggests that they stem from partially independent mechanisms, with the early facilitation directly involving M1, and the later inhibition recruiting high-level structures outside the motor system.

SIGNIFICANCE STATEMENT

Previous studies indicate that, when we observe other people's actions, the context in which actions take place influences intention understanding. However, little is known about the precise mechanisms involved in the contextual modulation of action representation (i.e., inhibition vs facilitation) and how they unfold in time. The present study sheds light on these aspects. Specifically, we show an early top-down facilitation (at ∼240 ms) and a later inhibition (at ∼400 ms) of motor resonance in response to actions observed in congruent and incongruent contexts, respectively.

Overt orienting of spatial attention and corticospinal excitability during action observation are unrelated

Observing moving body parts can automatically activate topographically corresponding motor representations in the primary motor cortex (M1), the so-called direct matching. Novel neurophysiological findings from social contexts are nonetheless proving that this process is not automatic as previously thought. The motor system can flexibly shift from imitative to incongruent motor preparation, when requested by a social gesture. In the present study we aim to bring an increase in the literature by assessing whether and how diverting overt spatial attention might affect motor preparation in contexts requiring interactive responses from the onlooker. Experiment 1 shows that overt attention—although anchored to an observed biological movement—can be captured by a target object as soon as a social request for it becomes evident. Experiment 2 reveals that the appearance of a short-lasting red dot in the contralateral space can divert attention from the target, but not from the biological movement. Nevertheless, transcranial magnetic stimulation (TMS) over M1 combined with electromyography (EMG) recordings (Experiment 3) indicates that attentional interference reduces corticospinal excitability related to the observed movement, but not motor preparation for a complementary action on the target. This work provides evidence that social motor preparation is impermeable to attentional interference and that a double dissociation is present between overt orienting of spatial attention and neurophysiological markers of action observation.