Responses to irrational actions in action observation and mentalising networks of the human brain

Three-stage Dynamic Brain-cognitive Model of Understanding Action Intention Displayed by Human Body Movements

The ability to comprehend the intention conveyed through human body movements is crucial for effective interpersonal interactions. If people can't understand the intention behind other individuals' isolated or interactive actions, their actions will become meaningless. Psychologists have investigated the cognitive processes and neural representations involved in understanding action intention, yet a cohesive theoretical explanation remains elusive. Hence, we mainly review existing literature related to neural correlates of action intention, and primarily propose a putative Three-stage Dynamic Brain-cognitive Model of understanding action intention, which involves body perception, action identification and intention understanding. Specifically, at the first stage, body parts/shapes are processed by those brain regions such as extrastriate and fusiform body areas; During the second stage, differentiating observed actions relies on configuring relationships between body parts, facilitated by the activation of the Mirror Neuron System; The last stage involves identifying various intention categories, utilizing the Mentalizing System for recruitment, and different activation patterns concerning the nature of the intentions participants dealing with. Finally, we delves into the clinical practice, like intervention training based on a theoretical model for individuals with autism spectrum disorders who encounter difficulties in interpersonal communication.

Brain mechanisms of social signalling in live social interactions with autistic and neurotypical adults

The simple act of watching another person can change a person's behaviour in subtle but important ways; the individual being watched is now capable of signalling to the watcher, and may use this opportunity to communicate to the watcher. Recent data shows that people will spontaneously imitate more when being watched. Here, we examine the neural and cognitive mechanisms of being watched during spontaneous social imitation in autistic and neurotypical adults using fNIRS brain imaging. Participants (n = 44) took part in a block-moving task where they were instructed only to copy the block sequence which people normally do using a straight low action trajectory. Here, the demonstrator sometimes used an atypical 'high' action trajectory, giving participants the opportunity to spontaneously copy the high trajectory even if this slowed their performance. The confederate who demonstrated each block sequence could watch the participant's actions or close her eyes, giving a factorial design with factors of trajectory (high/low) and watched (watched/unwatched). Throughout the task, brain signals were captured from bilateral temporal/parietal/occipital cortex using fNIRS. We found that all participants performed higher actions when being watched by the confederate than when not being watched, with no differences between autistic and neurotypical participants. The unwatched conditions were associated with higher activity of the right inferior parietal lobule in all participants and also engagement of left STS only in autistic participants. These findings are consistent with the claim that people engage different neural mechanisms when watched and unwatched and that participants with autism may engage additional brain mechanisms to match neurotypical behaviour and compensate for social difficulties. However, further studies will be needed to replicate these results in a larger sample of participants.

Functional role of the theory of mind network in integrating mentalistic prior information with action kinematics during action observation

Inferring intentions from verbal and nonverbal human behaviour is critical for everyday social life. Here, we combined Transcranial Magnetic Stimulation (TMS) with a behavioural priming paradigm to test whether key nodes of the Theory of Mind network (ToMn) contribute to understanding others' intentions by integrating prior knowledge about an agent with the observed action kinematics. We used a modified version of the Faked-Action Discrimination Task (FAD), a forced-choice paradigm in which participants watch videos of actors lifting a cube and judge whether the actors are trying to deceive them concerning the weight of the cube. Videos could be preceded or not by verbal description (prior) about the agent's truthful or deceitful intent. We applied single pulse TMS over three key nodes of the ToMn, namely dorsomedial prefrontal cortex (dmPFC), right posterior superior temporal sulcus (pSTS) and right temporo-parietal junction (rTPJ). Sham-TMS served as a control (baseline) condition. Following sham or rTPJ stimulation, we observed no consistent influence of priors on FAD performance. In contrast, following dmPFC stimulation, and to a lesser extent pSTS stimulation, truthful and deceitful actions were perceived as more deceptive only when the prior suggested a dishonest intention. These findings highlight a functional role of dmPFC and pSTS in coupling prior knowledge about deceptive intents with observed action kinematics in order to judge faked actions. Our study provides causal evidence that fronto-temporal nodes of the ToMn are functionally relevant to mental state inference during action observation.

Top-Down Attention Guidance Shapes Action Encoding in the pSTS

The posterior superior temporal sulcus (pSTS) is a brain region characterized by perceptual representations of human body actions that promote the understanding of observed behavior. Increasingly, action observation is recognized as being strongly shaped by the expectations of the observer (Kilner 2011; Koster-Hale and Saxe 2013; Patel et al. 2019). Therefore, to characterize top-down influences on action observation, we evaluated the statistical structure of multivariate activation patterns from the action observation network (AON) while observers attended to the different dimensions of action vignettes (the action kinematics, goal, or identity of avatars jumping or crouching). Decoding accuracy varied as a function of attention instruction in the right pSTS and left inferior frontal cortex (IFC), with the right pSTS classifying actions most accurately when observers attended to the action kinematics and the left IFC classifying most accurately when observed attended to the actor's goal. Functional connectivity also increased between the right pSTS and right IFC when observers attended to the actions portrayed in the vignettes. Our findings are evidence that the attentive state of the viewer modulates sensory representations in the pSTS, consistent with proposals that the pSTS occupies an interstitial zone mediating top-down context and bottom-up perceptual cues during action observation.

Weighted Brain Network Metrics for Decoding Action Intention Understanding Based on EEG

Background: Understanding the action intentions of others is important for social and human-robot interactions. Recently, many state-of-the-art approaches have been proposed for decoding action intention understanding. Although these methods have some advantages, it is still necessary to design other tools that can more efficiently classify the action intention understanding signals. New Method: Based on EEG, we first applied phase lag index (PLI) and weighted phase lag index (WPLI) to construct functional connectivity matrices in five frequency bands and 63 micro-time windows, then calculated nine graph metrics from these matrices and subsequently used the network metrics as features to classify different brain signals related to action intention understanding. Results: Compared with the single methods (PLI or WPLI), the combination method (PLI+WPLI) demonstrates some overwhelming victories. Most of the average classification accuracies exceed 70%, and some of them approach 80%. In statistical tests of brain network, many significantly different edges appear in the frontal, occipital, parietal, and temporal regions. Conclusions: Weighted brain networks can effectively retain data information. The integrated method proposed in this study is extremely effective for investigating action intention understanding. Both the mirror neuron and mentalizing systems participate as collaborators in the process of action intention understanding.

Seeing the Unexpected: How Brains Read Communicative Intent through Kinematics

Social interaction requires us to recognize subtle cues in behavior, such as kinematic differences in actions and gestures produced with different social intentions. Neuroscientific studies indicate that the putative mirror neuron system (pMNS) in the premotor cortex and mentalizing system (MS) in the medial prefrontal cortex support inferences about contextually unusual actions. However, little is known regarding the brain dynamics of these systems when viewing communicatively exaggerated kinematics. In an event-related functional magnetic resonance imaging experiment, 28 participants viewed stick-light videos of pantomime gestures, recorded in a previous study, which contained varying degrees of communicative exaggeration. Participants made either social or nonsocial classifications of the videos. Using participant responses and pantomime kinematics, we modeled the probability of each video being classified as communicative. Interregion connectivity and activity were modulated by kinematic exaggeration, depending on the task. In the Social Task, communicativeness of the gesture increased activation of several pMNS and MS regions and modulated top-down coupling from the MS to the pMNS, but engagement of the pMNS and MS was not found in the nonsocial task. Our results suggest that expectation violations can be a key cue for inferring communicative intention, extending previous findings from wholly unexpected actions to more subtle social signaling.

Neural processing of social interaction: Coordinate-based meta-analytic evidence from human neuroimaging studies

While the action observation and mentalizing networks are considered to play complementary roles in understanding others' goals and intentions, they might be concurrently engaged when processing social interactions. We assessed this hypothesis via three activation-likelihood-estimation meta-analyses of neuroimaging studies on the neural processing of: (a) social interactions, (b) individual actions by the action observation network, and (c) mental states by the mentalizing network. Conjunction analyses and direct comparisons unveiled overlapping and specific regions among the resulting maps. We report quantitative meta-analytic evidence for a "social interaction network" including key nodes of the action observation and mentalizing networks. An action-social interaction-mentalizing gradient of activity along the posterior temporal cortex highlighted a hierarchical processing of interactions, from visuomotor analyses decoding individual and shared intentions to in-depth inferences on actors' intentional states. The medial prefrontal cortex, possibly in conjunction with the amygdala, might provide additional information concerning the affective valence of the interaction. This evidence suggests that the functional architecture underlying the neural processing of interactions involves the joint involvement of the action observation and mentalizing networks. These data might inform the design of rehabilitative treatments for social cognition disorders in pathological conditions, and the assessment of their outcome in randomized controlled trials.

The way others move can influence what we choose

Whether pointing at a menu item or rifling through a clothes rack, when we choose we often move. We investigated whether people's tendency to copy the movements of others could influence their choices. Participants saw pairs of pictures in private and indicated which one they preferred. They then entered a virtual art gallery and saw the same picture pairs in the presence of a virtual character. Having observed the virtual character point to indicate her preference with either a high or low movement trajectory, participants indicated their preference. There was either an anatomical (same movement, same choice) or spatial correspondence (same movement, different choice) between the participant's pictures and those of the virtual character. We found that participants copied the movement made by the virtual character rather than her action goal (i.e., her choice of picture). This resulted in a shift towards the virtual character's preferences in the anatomical condition but away from her preferences in the spatial condition. This effect was driven by the observation of the virtual character's high pointing movements. In a further experiment, we did not find any significant differences in imitation behaviour in autism, although autistic participants were less consistent in their choices. Our findings demonstrate that we are not only influenced by other's choices but also the types of movements others make to indicate those choices.

Perceptual teleology: expectations of action efficiency bias social perception

Primates interpret conspecific behaviour as goal-directed and expect others to achieve goals by the most efficient means possible. While this teleological stance is prominent in evolutionary and developmental theories of social cognition, little is known about the underlying mechanisms. In predictive models of social cognition, a perceptual prediction of an ideal efficient trajectory would be generated from prior knowledge against which the observed action is evaluated, distorting the perception of unexpected inefficient actions. To test this, participants observed an actor reach for an object with a straight or arched trajectory on a touch screen. The actions were made efficient or inefficient by adding or removing an obstructing object. The action disappeared mid-trajectory and participants touched the last seen screen position of the hand. Judgements of inefficient actions were biased towards the efficient prediction (straight trajectories upward to avoid the obstruction, arched trajectories downward towards the target). These corrections increased when the obstruction's presence/absence was explicitly acknowledged, and when the efficient trajectory was explicitly predicted. Additional supplementary experiments demonstrated that these biases occur during ongoing visual perception and/or immediately after motion offset. The teleological stance is at least partly perceptual, providing an ideal reference trajectory against which actual behaviour is evaluated.

What Are You Doing With That Object? Comparing the Neural Responses of Action Understanding in Adolescents With and Without Autism

Understanding another's actions, including what they are doing and why they are doing it, can be difficult for individuals with autism spectrum disorder (ASD). This understanding is supported by the action observation (AON) and mentalizing (MZN) networks, as well as the superior temporal sulcus. We examined these areas in children with ASD and typically developing controls by having participants view eating and placing actions performed in conventional and unconventional ways while functional magnetic resonance images were collected. We found an effect of action-type, but not conventionality, in both groups, and a between groups difference only when viewing conventional eating actions. Findings suggest there are not global AON/MZN deficits in ASD, and observing unconventional actions may not spontaneously activate the MZN.

Spatiotemporal Phase Synchronization in Adaptive Reconfiguration from Action Observation Network to Mentalizing Network for Understanding Other's Action Intention

In action intention understanding, the mirror system is involved in perception-action matching process and the mentalizing system underlies higher-level intention inference. By analyzing the dynamic functional connectivity in α (8-12 Hz) and β (12-30 Hz) frequency bands over a "hand-cup interaction" observation task, this study investigates the topological transition from the action observation network (AON) to the mentalizing network (MZN), and estimates their functional relevance for intention identification from other's different action kinematics. Sequential brain microstates were extracted based on event-related potentials (ERPs), in which significantly differing neuronal responses were found in N170-P200 related to perceptually matching kinematic profiles and P400-700 involved in goal inference. Inter-electrode weighted phase lag index analysis on the ERP microstates revealed a shift of hub centrality salient in α frequency band, from the AON dominated by left-lateral frontal-premotor-temporal and temporal-parietooccipital synchronizations to the MZN consisting of more bilateral frontal-parietal and temporal-parietal synchronizations. As compared with usual actions, intention identification of unintelligible actions induces weaker synchronizations in the AON but dramatically increased connectivity in right frontal-temporal-parietal regions of the MZN, indicating a spatiotemporally complementary effect between the functional network configurations involved in mirror and mentalizing processes. Perceptual processing in observing usual/unintelligible actions decreases/increases requirements for intention inference, which would induce less/greater functional network reorganization on the way to mentalization. From the comparison, our study suggests that the adaptive topological changes from the AON to the MZN indicate implicit causal association between the mirror and mentalizing systems for decoding others' intentionality.

Brain regions involved in observing and trying to interpret dog behaviour

Humans and dogs have interacted for millennia. As a result, humans (and especially dog owners) sometimes try to interpret dog behaviour. While there is extensive research on the brain regions that are involved in mentalizing about other peoples’ behaviour, surprisingly little is known of whether we use these same brain regions to mentalize about animal behaviour. In this fMRI study we investigate whether brain regions involved in mentalizing about human behaviour are also engaged when observing dog behaviour. Here we show that these brain regions are more engaged when observing dog behaviour that is difficult to interpret compared to dog behaviour that is easy to interpret. Interestingly, these results were not only obtained when participants were instructed to infer reasons for the behaviour but also when they passively viewed the behaviour, indicating that these brain regions are activated by spontaneous mentalizing processes.

Moving higher and higher: imitators' movements are sensitive to observed trajectories regardless of action rationality

Humans sometimes perform actions which, at least superficially, appear suboptimal to the goal they are trying to achieve. Despite being able to identify these irrational actions from an early age, humans display a curious tendency to copy them. The current study recorded participants’ movements during an established imitation task and manipulated the rationality of the observed action in two ways. Participants observed videos of a model point to a series of targets with either a low, high or ‘superhigh’ trajectory either in the presence or absence of obstacles between her targets. The participants’ task was to watch which targets the model pointed to and then point to the same targets on the table in front of them. There were no obstacles between the participants’ targets. Firstly, we found that the peak height of participants’ movements between their targets was sensitive to the height of the model’s movements, even in the ‘superhigh’ condition where the model’s action was rated as irrational. Secondly, participants showed obstacle priming—the peak height of participants’ movements was higher after having observed the model move over obstacles to reach her targets, compared to when there were no obstacles between her targets. This suggests that participants code the environment of co-actors into their own motor programs, even when this compromises the efficiency of their own movements. We discuss the implications of our findings in terms of theories of imitation and obstacle priming.

Neural Activity while Imitating Emotional Faces is Related to Both Lower and Higher-Level Social Cognitive Performance

Imitation and observation of actions and facial emotional expressions activates the human fronto-parietal mirror network. There is skepticism regarding the role of this low-level network in more complex high-level social behaviour. We sought to test whether neural activation during an observation/imitation task was related to both lower and higher level social cognition. We employed an established observe/imitate task of emotional faces during functional MRI in 28 healthy adults, with final analyses based on 20 individuals following extensive quality control. Partial least squares (PLS) identified patterns of relationships between spatial activation and a battery of objective out-of-scanner assessments that index lower and higher-level social cognitive performance, including the Penn emotion recognition task, reading the mind in the eyes, the awareness of social inference test (TASIT) parts 1, 2, and 3, and the relationships across domains (RAD) test. Strikingly, activity in limbic, right inferior frontal, and inferior parietal areas during imitation of emotional faces correlated with performance on emotion evaluation (TASIT1), social inference - minimal (TASIT2), social inference - enriched (TASIT3), and the RAD tests. These results show a role for this network in both lower-level and higher-level social cognitive processes which are collectively critical for social functioning in everyday life.

The role of parietal cortex in overimitation: a study with fNIRS

Previous studies have shown right parietal activation in response to observing irrational actions. Behavioral studies show that people sometimes imitate irrational actions, a phenomenon called overimitation. However, limitations on movement in functional magnetic resonance imaging (fMRI) mean that the neural basis of overimitation has not been studied. To address this, our study employed a less restrictive neuroimaging technique, functional near-infrared spectroscopy (fNIRS). Measurements were taken while participants observed either rational or irrational movements before performing movements on a computerized puzzle task. Observing irrational actions produced greater activation in right anterior inferior parietal lobule (aIPL), replicating results from the fMRI literature. This is a proof of principle that fNIRS can be used as an alternative to fMRI in social cognition experiments, and that parietal cortex has a core role in responding to irrational actions.