Effects of agency on movement interference during observation of a moving dot stimulus

Guidelines for reporting action simulation studies (GRASS): Proposals to improve reporting of research in motor imagery and action observation

Researchers from multiple disciplines have studied the simulation of actions through motor imagery, action observation, or their combination. Procedures used in these studies vary considerably between research groups, and no standardized approach to reporting experimental protocols has been proposed. This has led to under-reporting of critical details, impairing the assessment, replication, synthesis, and potential clinical translation of effects. We provide an overview of issues related to the reporting of information in action simulation studies, and discuss the benefits of standardized reporting. We propose a series of checklists that identify key details of research protocols to include when reporting action simulation studies. Each checklist comprises A) essential methodological details, B) essential details that are relevant to a specific mode of action simulation, and C) further points that may be useful on a case-by-case basis. We anticipate that the use of these guidelines will improve the understanding, reproduction, and synthesis of studies using action simulation, and enhance the translation of research using motor imagery and action observation to applied and clinical settings.

From neural noise to co-adaptability: Rethinking the multifaceted architecture of motor variability

In the last decade, the source and the functional meaning of motor variability have attracted considerable attention in behavioral and brain sciences. This construct classically combined different levels of description, variable internal robustness or coherence, and multifaceted operational meanings. We provide here a comprehensive review of the literature with the primary aim of building a precise lexicon that goes beyond the generic and monolithic use of motor variability. In the pars destruens of the work, we model three domains of motor variability related to peculiar computational elements that influence fluctuations in motor outputs. Each domain is in turn characterized by multiple sub-domains. We begin with the domains of noise and differentiation. However, the main contribution of our model concerns the domain of adaptability, which refers to variation within the same exact motor representation. In particular, we use the terms learning and (social)fitting to specify the portions of motor variability that depend on our propensity to learn and on our largely constitutive propensity to be influenced by external factors. A particular focus is on motor variability in the context of the sub-domain named co-adaptability. Further groundbreaking challenges arise in the modeling of motor variability. Therefore, in a separate pars construens, we attempt to characterize these challenges, addressing both theoretical and experimental aspects as well as potential clinical implications for neurorehabilitation. All in all, our work suggests that motor variability is neither simply detrimental nor beneficial, and that studying its fluctuations can provide meaningful insights for future research.

Top-Down Modulation of Motor Priming by Belief About Animacy

Research has shown that people automatically imitate others and that this tendency is stronger when the other person is a human compared with a nonhuman agent. However, a controversial question is whether automatic imitation is also modulated by whether people believe the other person is a human. Although early research supported this hypothesis, not all studies reached the same conclusion and a recent meta-analysis found that there is currently neither evidence in favor nor against an influence of animacy beliefs on automatic imitation. One of the most prominent studies supporting such an influence is the study by Liepelt and Brass (2010), who found that automatic imitation was stronger when participants believed an ambiguous, gloved hand to be human, as opposed to wooden. In this registered report, we provide a high-powered replication of this study (N = 199). In contrast to Liepelt and Brass (2010), we did not find an effect of animacy beliefs on automatic imitation. However, we did find a correlation between automatic imitation and perceived self-other similarity. Together, these results suggest that the gloved hand procedure does not reliably influence automatic imitation, but interindividual differences in perceived similarity do.

Body Form Modulates the Prediction of Human and Artificial Behaviour from Gaze Observation

The future of human–robot collaboration relies on people’s ability to understand and predict robots' actions. The machine-like appearance of robots, as well as contextual information, may influence people’s ability to anticipate the behaviour of robots. We conducted six separate experiments to investigate how spatial cues and task instructions modulate people’s ability to understand what a robot is doing. Participants observed goal-directed and non-goal directed gaze shifts made by human and robot agents, as well as directional cues displayed by a triangle. We report that biasing an observer's attention, by showing just one object an agent can interact with, can improve people’s ability to understand what humanoid robots will do. Crucially, this cue had no impact on people’s ability to predict the upcoming behaviour of the triangle. Moreover, task instructions that focus on the visual and motor consequences of the observed gaze were found to influence mentalising abilities. We suggest that the human-like shape of an agent and its physical capabilities facilitate the prediction of an upcoming action. The reported findings expand current models of gaze perception and may have important implications for human–human and human–robot collaboration.

We like it ‘cause you take it: vicarious effects of approach/avoidance behaviours on observers

We present five studies investigating the effects of approach and avoidance behaviours when individuals do not enact them but, instead, learn that others have performed them. In Experiment 1, when participants read that a fictitious character (model) had approached a previously unknown product, they ascribed to this model a liking for the object. In contrast, they ascribed to the model a disliking for the avoided product. In Experiment 2, this result emerged, with a smaller effect size, even when it was clear that the behaviours followed specific instructions from a third party. The model had been a mere executor instead of behaving autonomously. Finally, in Experiments 3, 4, and 5, we showed, with direct and indirect measures of attitudes, that reading that the model had approached vs avoided products was sufficient to create preferences in the participant for the approached one, regardless of whether it was explained that the model was a mere executor. This research highlights the largely unexplored effects of vicarious approach/avoidance behaviours. Theoretical and practical implications and possible developments of this line of research are discussed.

Visual tracking at 4 months in preterm infants predicts 6.5-year cognition and attention

BACKGROUND

Visual tracking of moving objects requires sustained attention and prediction of the object's trajectory. We tested the hypothesis that measures of eye-head tracking of moving objects are associated to long-term neurodevelopment in very preterm infants.

METHODS

Visual tracking performance was assessed at 4 month's corrected age in 57 infants with gestational age <32 weeks. An object moved in front of the infant with sinusoidal or triangular (i.e. abrupt) turns of the direction. Gaze gain, smooth pursuit gain, and timing of gaze to object motion were analyzed. At 6.5 years the Wechsler Intelligence Scale for Children (WISC-IV), the Brown Attention Deficit Disorder (Brown ADD), and visual examination were performed.

RESULTS

Gaze gain and smooth pursuit gain at 4 months were strongly related to all WISC-IV parameters at 6.5 years. Gaze gain for the triangular and sinusoidal motion patterns related similarly to the cognitive scores. For the sinusoidal motion pattern, timing related to most Brown ADD parameters. There were no statistically significant differences in associations dependent on motion pattern. Visual function did not influence the results.

CONCLUSION

The ability to attend to and smoothly track a moving object in infancy is an early marker of cognition and attention at 6.5 years.

IMPACT

Potential long-term implications of infant visual tracking of moving objects for school-age neurodevelopment has not been previously studied in very preterm infants. Early coordination of eye and head movements in gaze gain, smooth pursuit, and timing of gaze to object motion are closely associated with cognition and attention at 6.5 years. As related functions at 6.5 years include perceptual and verbal skills, working memory, processing speed and attention, predictive elements in gaze tracking of moving objects might be a suitable target for future intervention studies.

Motor cognition in schizophrenia: Control of automatic imitation and mapping of action context are reduced

The ability to imitate is considered impaired in schizophrenia patients. This assumption, however, is based on heterogeneous studies mostly targeting voluntary imitation, e.g., pantomime. Studies on automatic imitation, however, and on underlying mechanisms of top-down inhibition of automatic imitation and contextual modulation in schizophrenia are highly limited. We employed two sensorimotor paradigms to examine imitation-inhibition and action context mapping in 37 schizophrenia patients and 36 matched controls. In the first experiment, participants performed finger lifts while observing a hand executing compatible or incompatible finger lifts from the third-person perspective. The compatibility or incompatibility of these finger lifts affected participants' reaction times (RTs). The comparison of between-condition RT differences shows a larger movement compatibility effect in schizophrenia than in controls. The second experiment involved finger lifts while watching a still hand, from the first-person perspective, with constrained fingers that either corresponded or did not correspond to the participants' response fingers. Here, schizophrenia patients showed a diminished RT slowing in corresponding constraint trials. While the former results provide evidence for an impaired control of imitation in patients with schizophrenia, the latter results indicate a reduced encoding of action context. In conclusion, this study provides the first evidence for deficits of top-down control of imitation and motor context processing in the same sample of schizophrenia patients.

Reluctance against the machine: Retrieval of observational stimulus-response episodes in online settings emerges when interacting with a human, but not with a computer partner

Observing how another person responds to a stimulus creates stimulus–response (SR) episodes. These can be retrieved from memory on later occasions, which means that observed responses are utilized for regulating one’s own actions. Until now, evidence for storage and retrieval of observationally acquired SR episodes was limited to dyadic face-to-face interactions between two partners who respond in an alternating fashion. In two preregistered studies (total N = 252), we demonstrate for the first time that observational SR episodes can also be acquired in online interactions: Robust retrieval effects emerged when observers believe to be interacting with another person. In turn, retrieval effects were absent when observers believe to be interacting with a computer. Our findings show that feature-based binding and retrieval principles are pervasive and also apply to social interactions, even under purely virtual conditions. We discuss implications of our findings for different explanatory accounts of social modulations of automatic imitation.

Theoretical Perspective on an Ideomotor Brain-Computer Interface: Toward a Naturalistic and Non-invasive Brain-Computer Interface Paradigm Based on Action-Effect Representation

Recent years have been marked by the fulgurant expansion of non-invasive Brain-Computer Interface (BCI) devices and applications in various contexts (medical, industrial etc.). This technology allows agents "to directly act with thoughts," bypassing the peripheral motor system. Interestingly, it is worth noting that typical non-invasive BCI paradigms remain distant from neuroscientific models of human voluntary action. Notably, bidirectional links between action and perception are constantly ignored in BCI experiments. In the current perspective article, we proposed an innovative BCI paradigm that is directly inspired by the ideomotor principle, which postulates that voluntary actions are driven by the anticipated representation of forthcoming perceptual effects. We believe that (1) adapting BCI paradigms could allow simple action-effect bindings and consequently action-effect predictions and (2) using neural underpinnings of those action-effect predictions as features of interest in AI methods, could lead to more accurate and naturalistic BCI-mediated actions.

Does agency matter? Neural processing of robotic movements in 4- and 8-year olds

Despite the increase in interactions between children and robots, our understanding of children's neural processing of robotic movements is limited. The current study theorized that motor resonance hinges on the agency of an actor: its ability to perform actions volitionally. As one of the first studies with a cross-sectional sample of preschoolers and older children and with a specific focus on robotic action (rather than abstract non-human action), the current study investigated whether the perceived agency of a robot moderated children's motor resonance for robotic movements, and whether this changed with age. Motor resonance was measured using electroencephalography (EEG) by assessing mu power while 4 and 8-year-olds observed actions performed by agentic versus non-agentic robots and humans. Results show that older children resonated more strongly with non-agentic than agentic robotic or human movement, while no such differences were found for preschoolers. This outcome is discussed in terms of a predictive coding account of motor resonance. Importantly, these findings contribute to the existing set of studies on this topic by showing that, while keeping all kinematic information constant, there is a clear developmental difference in how children process robotic movement depending on the level of agency of a robot.

Is There a 'Social' Brain? Implementations and Algorithms

A fundamental question in psychology and neuroscience is the extent to which cognitive and neural processes are specialised for social behaviour, or are shared with other 'non-social' cognitive, perceptual, and motor faculties. Here we apply the influential framework of Marr (1982) across research in humans, monkeys, and rodents to propose that information processing can be understood as 'social' or 'non-social' at different levels. We argue that processes can be socially specialised at the implementational and/or the algorithmic level, and that changing the goal of social behaviour can also change social specificity. This framework could provide important new insights into the nature of social behaviour across species, facilitate greater integration, and inspire novel theoretical and empirical approaches.

Visual Feedback of Object Motion Direction Influences the Timing of Grip Force Modulation During Object Manipulation

During manipulation, object slipping is prevented by modulating the grip force (GF) in synchrony with motion-related inertial forces, i.e., load force (LF). However, due to conduction delays of the sensory system, GF must be modulated in advance based on predictions of LF changes. It has been proposed that such predictive force control relies on internal representations, i.e., internal models, of the relation between the dynamic of the environment and movement kinematics. Somatosensory and visual feedback plays a primary role in building these internal representations. For instance, it has been shown that manipulation-dependent somatosensory signals contribute to building internal representations of gravity in normal and altered gravitational contexts. Furthermore, delaying the timing of visual feedback of object displacement has been shown to affect GF. Here, we explored whether and the extent to which spatial features of visual feedback movement, such as motion direction, may contribute to GF control. If this were the case, a spatial mismatch between actual (somatosensory) and visual feedback of object motion would elicit changes in GF modulation. We tested this hypothesis by asking participants to generate vertical object movements while visual feedback of object position was congruent (0° rotation) or incongruent (180° or 90°) with the actual object displacement. The role of vision on GF control was quantified by the temporal shift of GF modulation as a function of visual feedback orientation and actual object motion direction. GF control was affected by visual feedback when this was incongruent in the vertical (180°), but not horizontal dimension. Importantly, 180° visual feedback rotation delayed and anticipated GF modulation during upward and downward actual movements, respectively. Our findings suggest that during manipulation, spatial features of visual feedback motion are used to predict upcoming LF changes. Furthermore, the present study provides evidence that an internal model of gravity contributes to GF control by influencing sensory reweighting processes during object manipulation.

Toward Scalable Measures of Quality of Interaction

Motor resonance, the activation of an observer’s motor control system by another actor’s movements, has been claimed to be an indicator for quality of interaction. Motor interference as one of the consequences of the presence of resonance can be detected by analyzing an actor’s spatial movements. It has therefore been used as an indicator for the presence of motor resonance. Unfortunately, the experimental paradigm in which motor interference has been shown to be detectable is ecologically implausible both in terms of the types of movements employed and the number of repetitions required. In the presented experiment, we tested whether some of these experimental constraints can be relaxed or modified toward a more naturalistic behavior without losing the ability to detect the interference effect. In the literature, spatial variance has been analytically quantified in many different ways. This study found these analytical variations to be nonequivalent by implementing them. Back-and-forth transitive movements were tested for motor interference; the effect was found to be more robust than with left-right movements, although the direction of interference was opposite to that reported in the literature. We conclude that motor interference, when measured by spatial variation, lacks promise for embedding in naturalistic interaction scenarios because the effect sizes were small.

Biological motion and animacy belief induce similar effects on involuntary shifts of attention

Biological motion is salient to the human visual and motor systems and may be intrinsic to the perception of animacy. Evidence for the salience of visual stimuli moving with trajectories consistent with biological motion comes from studies showing that such stimuli can trigger shifts of attention in the direction of that motion. The present study was conducted to determine whether or not top-down beliefs about animacy can modify the salience of a nonbiologically moving stimulus to the visuomotor system. A nonpredictive cuing task was used in which a white dot moved from a central location toward a left- or right-sided target placeholder. The target randomly appeared at either location 200, 600, or 1,300 ms after the motion onset. Five groups of participants experienced different stimulus conditions: (1) biological motion, (2) inverted biological motion, (3) nonbiological motion, (4) animacy belief (paired with nonbiological motion), and (5) computer-generated belief (paired with nonbiological motion). Analysis of response times revealed that the motion in the biological motion and animacy belief groups, but not in the inverted and nonbiological motion groups, affected processing of the target information. These findings indicate that biological motion is salient to the visual system and that top-down beliefs regarding the animacy of the stimulus can tune the visual and motor systems to increase the salience of nonbiological motion.

Reduced Motor Interference in Preschoolers with Autism Spectrum Disorder and Williams Syndrome

Motor interference occurs when action execution is hindered by the observation of an incongruent action. The present study used a novel eye-tracking paradigm to test the motor interference effect in 22 preschoolers with autism spectrum disorder (ASD), 14 preschoolers with Williams syndrome (WS), and 18 typically developing (TD) peers. In TD children, performance of a pre-determined action was slower after the observation of an incongruent motor action and faster following observation of a congruent motor action, indicating a motor interference effect. In both the ASD and WS groups, performance was unaffected by the congruent versus incongruent nature of the observed motor action.

Top-down attentional factors modulate action priming in reach-to-grasp action

Previous studies report that viewing exaggerated, high-lifting reaches (versus direct reaches) primes higher vertical deviation in wrist trajectory in the observer's subsequent reaches (trajectory priming), but it is unclear to what extent this effect depends upon task instructions relevant to top-down attention. In two experiments, participants were instructed to gaze at a dot presented on a large monitor for a colour-change go signal that cued them to execute a direct reach to a target. In the background, the monitor also displayed life-sized films of a human model. The films were of the model either remaining still or reaching to grasp a target with either a direct trajectory or an exaggerated, high-lifting trajectory. When the dot traced the human model's wrist throughout her movement, a robust trajectory priming effect emerged. When the dot remained stationary in a central location but the human model reached in the background, the human model's trajectory did not alter the participants' trajectories. Finally, when the dot traced exaggerated and direct trajectories and the human model remained stationary, the dot's movement produced an attenuated, non-significant trajectory priming effect. These findings show that top-down attentional factors modulate trajectory priming. In addition, a moving non-human stimulus does not produce the same degree of action priming when contextual factors make salient its independence of human agency and/or intention.

From automata to animate beings: the scope and limits of attributing socialness to artificial agents

Understanding the mechanisms and consequences of attributing socialness to artificial agents has important implications for how we can use technology to lead more productive and fulfilling lives. Here, we integrate recent findings on the factors that shape behavioral and brain mechanisms that support social interactions between humans and artificial agents. We review how visual features of an agent, as well as knowledge factors within the human observer, shape attributions across dimensions of socialness. We explore how anthropomorphism and dehumanization further influence how we perceive and interact with artificial agents. Based on these findings, we argue that the cognitive reconstruction within the human observer is likely to be far more crucial in shaping our interactions with artificial agents than previously thought, while the artificial agent's visual features are possibly of lesser importance. We combine these findings to provide an integrative theoretical account based on the "like me" hypothesis, and discuss the key role played by the Theory-of-Mind network, especially the temporal parietal junction, in the shift from mechanistic to social attributions. We conclude by highlighting outstanding questions on the impact of long-term interactions with artificial agents on the behavioral and brain mechanisms of attributing socialness to these agents.

Deficits in Visuo-Motor Temporal Integration Impacts Manual Dexterity in Probable Developmental Coordination Disorder

The neurological basis of developmental coordination disorder (DCD) is thought to be deficits in the internal model and mirror-neuron system (MNS) in the parietal lobe and cerebellum. However, it is not clear if the visuo-motor temporal integration in the internal model and automatic-imitation function in the MNS differs between children with DCD and those with typical development (TD). The current study aimed to investigate these differences. Using the manual dexterity test of the Movement Assessment Battery for Children (second edition), the participants were either assigned to the probable DCD (pDCD) group or TD group. The former was comprised of 29 children with clumsy manual dexterity, while the latter consisted of 42 children with normal manual dexterity. Visuo-motor temporal integration ability and automatic-imitation function were measured using the delayed visual feedback detection task and motor interference task, respectively. Further, the current study investigated whether autism-spectrum disorder (ASD) traits, attention-deficit hyperactivity disorder (ADHD) traits, and depressive symptoms differed among the two groups, since these symptoms are frequent comorbidities of DCD. In addition, correlation and multiple regression analyses were performed to extract factors affecting clumsy manual dexterity. In the results, the delay-detection threshold (DDT) and steepness of the delay-detection probability curve, which indicated visuo-motor temporal integration ability, were significantly prolonged and decreased, respectively, in children with pDCD. The interference effect, which indicated automatic-imitation function, was also significantly reduced in this group. These results highlighted that children with clumsy manual dexterity have deficits in visuo-motor temporal integration and automatic-imitation function. There was a significant correlation between manual dexterity, and measures of visuo-motor temporal integration, and ASD traits and ADHD traits and ASD. Multiple regression analysis revealed that the DDT, which indicated visuo-motor temporal integration, was the greatest predictor of poor manual dexterity. The current results supported and provided further evidence for the internal model deficit hypothesis. Further, they suggested a neurorehabilitation technique that improved visuo-motor temporal integration could be therapeutically effective for children with DCD.

Eye movements may cause motor contagion effects

When a person executes a movement, the movement is more errorful while observing another person's actions that are incongruent rather than congruent with the executed action. This effect is known as "motor contagion". Accounts of this effect are often grounded in simulation mechanisms: increased movement error emerges because the motor codes associated with observed actions compete with motor codes of the goal action. It is also possible, however, that the increased movement error is linked to eye movements that are executed simultaneously with the hand movement because oculomotor and manual-motor systems are highly interconnected. In the present study, participants performed a motor contagion task in which they executed horizontal arm movements while observing a model making either vertical (incongruent) or horizontal (congruent) movements under three conditions: no instruction, maintain central fixation, or track the model's hand with the eyes. A significant motor contagion-like effect was only found in the 'track' condition. Thus, 'motor contagion' in the present task may be an artifact of simultaneously executed incongruent eye movements. These data are discussed in the context of stimulation and associative learning theories, and raise eye movements as a critical methodological consideration for future work on motor contagion.

The shaping of social perception by stimulus and knowledge cues to human animacy

Although robots are becoming an ever-growing presence in society, we do not hold the same expectations for robots as we do for humans, nor do we treat them the same. As such, the ability to recognize cues to human animacy is fundamental for guiding social interactions. We review literature that demonstrates cortical networks associated with person perception, action observation and mentalizing are sensitive to human animacy information. In addition, we show that most prior research has explored stimulus properties of artificial agents (humanness of appearance or motion), with less investigation into knowledge cues (whether an agent is believed to have human or artificial origins). Therefore, currently little is known about the relationship between stimulus and knowledge cues to human animacy in terms of cognitive and brain mechanisms. Using fMRI, an elaborate belief manipulation, and human and robot avatars, we found that knowledge cues to human animacy modulate engagement of person perception and mentalizing networks, while stimulus cues to human animacy had less impact on social brain networks. These findings demonstrate that self–other similarities are not only grounded in physical features but are also shaped by prior knowledge. More broadly, as artificial agents fulfil increasingly social roles, a challenge for roboticists will be to manage the impact of pre-conceived beliefs while optimizing human-like design.

Pictures of you: Dot stimuli cause motor contagion in presence of a still human form

In this study, we investigate which visual cues induce participants to encode a non-human motion stimulus in their motor system. Participants performed reach-to-grasp actions to a target after observing a dot moving in a direct or higher-arcing path across a screen. Dot motion occurred in the presence of a meaningless (scrambled human model) stimulus, a still human model, or a human model performing a direct or exaggeratedly curved reach to a target. Our results show that observing the dot displacement causes motor contagion (changes in the height of the observer's hand trajectory) when a human form was visually present in the background (either moving or still). No contagion was evident, however, when this human context was absent (i.e., human image scrambled and not identifiable). This indicates that visual cues suggestive of human agency can determine whether or not moving stimuli are encoded in the motor system.

Violating instructed human agency: An fMRI study on ocular tracking of biological and nonbiological motion stimuli

Previous studies have shown that beliefs about the human origin of a stimulus are capable of modulating the coupling of perception and action. Such beliefs can be based on top-down recognition of the identity of an actor or bottom-up observation of the behavior of the stimulus. Instructed human agency has been shown to lead to superior tracking performance of a moving dot as compared to instructed computer agency, especially when the dot followed a biological velocity profile and thus matched the predicted movement, whereas a violation of instructed human agency by a nonbiological dot motion impaired oculomotor tracking (Zwickel et al., 2012). This suggests that the instructed agency biases the selection of predictive models on the movement trajectory of the dot motion. The aim of the present fMRI study was to examine the neural correlates of top-down and bottom-up modulations of perception-action couplings by manipulating the instructed agency (human action vs. computer-generated action) and the observable behavior of the stimulus (biological vs. nonbiological velocity profile). To this end, participants performed an oculomotor tracking task in an MRI environment. Oculomotor tracking activated areas of the eye movement network. A right-hemisphere occipito-temporal cluster comprising the motion-sensitive area V5 showed a preference for the biological as compared to the nonbiological velocity profile. Importantly, a mismatch between instructed human agency and a nonbiological velocity profile primarily activated medial-frontal areas comprising the frontal pole, the paracingulate gyrus, and the anterior cingulate gyrus, as well as the cerebellum and the supplementary eye field as part of the eye movement network. This mismatch effect was specific to the instructed human agency and did not occur in conditions with a mismatch between instructed computer agency and a biological velocity profile. Our results support the hypothesis that humans activate a specific predictive model for biological movements based on their own motor expertise. A violation of this predictive model causes costs as the movement needs to be corrected in accordance with incoming (nonbiological) sensory information.

Embodied artificial agents for understanding human social cognition

In this paper, we propose that experimental protocols involving artificial agents, in particular the embodied humanoid robots, provide insightful information regarding social cognitive mechanisms in the human brain. Using artificial agents allows for manipulation and control of various parameters of behaviour, appearance and expressiveness in one of the interaction partners (the artificial agent), and for examining effect of these parameters on the other interaction partner (the human). At the same time, using artificial agents means introducing the presence of artificial, yet human-like, systems into the human social sphere. This allows for testing in a controlled, but ecologically valid, manner human fundamental mechanisms of social cognition both at the behavioural and at the neural level. This paper will review existing literature that reports studies in which artificial embodied agents have been used to study social cognition and will address the question of whether various mechanisms of social cognition (ranging from lower- to higher-order cognitive processes) are evoked by artificial agents to the same extent as by natural agents, humans in particular. Increasing the understanding of how behavioural and neural mechanisms of social cognition respond to artificial anthropomorphic agents provides empirical answers to the conundrum 'What is a social agent?'

So close yet so far: Motor anomalies impacting on social functioning in autism spectrum disorder

Difficulties in the social domain and motor anomalies have been widely investigated in Autism Spectrum Disorder (ASD). However, they have been generally considered as independent, and therefore tackled separately. Recent advances in neuroscience have hypothesized that the cortical motor system can play a role not only as a controller of elementary physical features of movement, but also in a complex domain as social cognition. Here, going beyond previous studies on ASD that described difficulties in the motor and in the social domain separately, we focus on the impact of motor mechanisms anomalies on social functioning. We consider behavioral, electrophysiological and neuroimaging findings supporting the idea that motor cognition is a critical "intermediate phenotype" for ASD. Motor cognition anomalies in ASD affect the processes of extraction, codification and subsequent translation of "external" social information into the motor system. Intriguingly, this alternative "motor" approach to the social domain difficulties in ASD may be promising to bridge the gap between recent experimental findings and clinical practice, potentially leading to refined preventive approaches and successful treatments.

The influence of environmental context in interpersonal observation-execution

Cyclical upper-limb movements involuntarily deviate from a primary movement direction when the actor concurrently observes incongruent biological motion. We examined whether environmental context influences such motor interference during interpersonal observation-execution. Participants executed continuous horizontal arm movements while observing congruent horizontal or incongruent curvilinear biological movements with or without the presence of an object positioned as an obstacle or distractor. When participants were observing a curvilinear movement, an object located within the movement space became an obstacle, and, thus, the curvilinear trajectory was essential to reach into horizontal space. When acting as a distractor, or with no object, the curvilinear trajectory was no longer essential. For observing horizontal movements, objects were located at the same relative locations as in the curvilinear movement condition. We found greater involuntary movement deviation when observing curvilinear than horizontal movements. Also, there was an influence of context only when observing horizontal movements, with greater deviation exhibited in the presence of a large obstacle. These findings suggest that the influence of environmental context is underpinned by the (mis-)matching of observed and executed actions as incongruent biological motion is primarily coded via bottom-up sensorimotor processes, whilst the congruent condition incorporates surrounding environmental features to modulate the bottom-up sensorimotor processes.

Kick with the finger: symbolic actions shape motor cortex excitability

A large body of research indicates that observing actions made by others is associated with corresponding motor facilitation of the observer's corticospinal system. However, it is still controversial whether this matching mechanism strictly reflects the kinematics of the observed action or its meaning. To test this issue, motor evoked potentials induced by single-pulse transcranial magnetic stimulation were recorded from hand and leg muscles while participants observed a symbolic action carried out with the index finger, but classically performed with the leg (i.e., a soccer penalty kick). A control condition in which participants observed a similar (but not symbolic) hand movement was also included. Results showed that motor facilitation occurs both in the observer's hand (first dorsal interosseous) and leg (quadriceps femoris) muscles. The present study provides evidence that both the kinematics and the symbolic value of an observed action are able to modulate motor cortex excitability. The human motor system is thus not only involved in mirroring observed actions but is also finely tuned to their symbolic value.

Motion trajectory information and agency influence motor learning during observational practice

Fundamental to performing actions is the acquisition of motor behaviours. We examined if motor learning, through observational practice, occurs by viewing an agent displaying naturalistic or constant velocity, and whether motion trajectory, as opposed to end-state, information is required. We also investigated if observational practice is sensitive to belief regarding the origin of an agent. Participants had to learn a novel movement sequence timing task, which required upper-limb movements to a series of targets within a pre-specified absolute and relative time goal. Experiment 1 showed learning after viewing naturalistic and constant velocity, but not end-state information. For Experiment 2, in addition to learning the movement sequence, participants observed a series of movement stimuli that were either the trained or new sequences and asked to rate their confidence on whether the observed sequence was the same or different to observational practice. The results indicated that agency belief modulates how naturalistic and constant velocity is coded. This indicated that the processes associated with belief are part of an interpretative predictive coding system where the association between belief and observed motion is determined. When motion is constant velocity, or believed to be computer-generated, coding occurs through top-down processes. When motion is naturalistic velocity, and believed to be human-generated, it is most likely coded by gaining access to bottom-up sensorimotor processes in the action-observation network.

Motor interference in interactive contexts

Action observation and execution share overlapping neural substrates, so that simultaneous activation by observation and execution modulates motor performance. Previous literature on simple prehension tasks has revealed that motor influence can be two-sided: facilitation for observed and performed congruent actions and interference for incongruent actions. But little is known of the specific modulations of motor performance in complex forms of interaction. Is it possible that the very same observed movement can lead either to interference or facilitation effects on a temporally overlapping congruent executed action, depending on the context? To answer this question participants were asked to perform a reach-to-grasp movement adopting a precision grip (PG) while: (i) observing a fixation cross, (ii) observing an actor performing a PG with interactive purposes, (iii) observing an actor performing a PG without interactive purposes. In particular, in the interactive condition the actor was shown trying to pour some sugar on a large cup located out of her reach but close to the participant watching the video, thus eliciting in reaction a complementary whole-hand grasp. Notably, fine-grained kinematic analysis for this condition revealed a specific delay in the grasping and reaching components and an increased trajectory deviation despite the observed and executed movement’s congruency. Moreover, early peaks of trajectory deviation seem to indicate that socially relevant stimuli are acknowledged by the motor system very early. These data suggest that interactive contexts can determine a prompt modulation of stimulus–response compatibility effects.

Top-down social modulation of interpersonal observation-execution

Cyclical upper limb movement can involuntarily deviate from its primary movement axis when the performer concurrently observes incongruent biological motion (i.e. interpersonal observation-execution). The current study examined the social modulation of such involuntary motor interference using a protocol that reflected everyday social interactions encountered in a naturalistic social setting. Eighteen participants executed cyclical horizontal arm movements during the observation of horizontal (congruent) or curvilinear (incongruent) biological motion. Both prior to, and during the interpersonal observation-execution task, participants also received a series of social words designed to prime a pro-social or anti-social attitude. The results showed greater orthogonal movement deviation, and thus interference, for the curvilinear compared to horizontal stimuli. Importantly, and opposite to most of the previous findings from work on automatic imitation and mimicry, there was a greater interference effect for the anti-social compared to pro-social prime condition. These findings demonstrate the importance of interpreting the context of social primes, and strongly support predictions of a comparison between the prime construct and the self-concept/-schema and the top-down response modulation of social incentives.

The use of virtual characters to assess and train non-verbal communication in high-functioning autism

High-functioning autism (HFA) is a neurodevelopmental disorder, which is characterized by life-long socio-communicative impairments on the one hand and preserved verbal and general learning and memory abilities on the other. One of the areas where particular difficulties are observable is the understanding of non-verbal communication cues. Thus, investigating the underlying psychological processes and neural mechanisms of non-verbal communication in HFA allows a better understanding of this disorder, and potentially enables the development of more efficient forms of psychotherapy and trainings. However, the research on non-verbal information processing in HFA faces several methodological challenges. The use of virtual characters (VCs) helps to overcome such challenges by enabling an ecologically valid experience of social presence, and by providing an experimental platform that can be systematically and fully controlled. To make this field of research accessible to a broader audience, we elaborate in the first part of the review the validity of using VCs in non-verbal behavior research on HFA, and we review current relevant paradigms and findings from social-cognitive neuroscience. In the second part, we argue for the use of VCs as either agents or avatars in the context of "transformed social interactions." This allows for the implementation of real-time social interaction in virtual experimental settings, which represents a more sensitive measure of socio-communicative impairments in HFA. Finally, we argue that VCs and environments are a valuable assistive, educational and therapeutic tool for HFA.

Sensorimotor synchronization with audio-visual stimuli: limited multisensory integration

Understanding how we synchronize our actions with stimuli from different sensory modalities plays a central role in helping to establish how we interact with our multisensory environment. Recent research has shown better performance with multisensory over unisensory stimuli; however, the type of stimuli used has mainly been auditory and tactile. The aim of this article was to expand our understanding of sensorimotor synchronization with multisensory audio-visual stimuli and compare these findings to their individual unisensory counterparts. This research also aims to assess the role of spatio-temporal structure for each sensory modality. The visual and/or auditory stimuli had either temporal or spatio-temporal information available and were presented to the participants in unimodal and bimodal conditions. Globally, the performance was significantly better for the bimodal compared to the unimodal conditions; however, this benefit was limited to only one of the bimodal conditions. In terms of the unimodal conditions, the level of synchronization with visual stimuli was better than auditory, and while there was an observed benefit with the spatio-temporal compared to temporal visual stimulus, this was not replicated with the auditory stimulus.

Dyadic movement synchronization while performing incongruent trajectories requires mutual adaptation

Unintentional movement synchronization is often emerging between interacting humans. In the present study, we investigate the extent to which the incongruence of movement trajectories has an influence on unintentional dyadic movement synchronization. During a target-directed tapping task, a participant repetitively moved between two targets in front of another participant who performed the same task in parallel but independently. When the movement path of one participant was changed by placing an obstacle between the targets, the degree of their unintentional movement synchronization was measured. Movement synchronization was observed despite of their substantially different movement trajectories. A deeper investigation of the participant's unintentional behavior shows, that although the actor who cleared the obstacle puts unintentional effort in establishing synchrony by increasing movement velocity-the other actor also unintentionally adjusted his/her behavior by increasing dwell times. Results are discussed in the light of joint action, movement interference and obstacle avoidance behavior.

Atypical interference effect of action observation in autism spectrum conditions

BACKGROUND

Observing incongruent actions interferes with ongoing action execution. This 'interference effect' is larger for observed biological actions than for non-biological actions. The current study used virtual reality to investigate the biological specificity of interference effects of action observation in autism spectrum conditions (ASC).

METHOD

High-functioning adults with ASC and age- and IQ-matched healthy controls performed horizontal sinusoidal arm movements whilst observing arm movements conducted by a virtual reality agent with either human or robot form, which moved with either biological motion or at a constant velocity. In another condition, participants made the same arm movements while observing a real human. Observed arm movements were either congruent or incongruent with executed arm movements. An interference effect was calculated as the average variance in the incongruent action dimension during observation of incongruent compared with congruent movements.

RESULTS

Control participants exhibited an interference effect when observing real human and virtual human agent incongruent movements but not when observing virtual robot agent movements. Individuals with ASC differed from controls in that they showed no interference effects for real human, virtual human or virtual robot movements.

CONCLUSIONS

The current study demonstrates atypical interference effects in ASC.

Motor resonance facilitates movement execution: an ERP and kinematic study

Action observation, simulation and execution share neural mechanisms that allow for a common motor representation. It is known that when these overlapping mechanisms are simultaneously activated by action observation and execution, motor performance is influenced by observation and vice versa. To understand the neural dynamics underlying this influence and to measure how variations in brain activity impact the precise kinematics of motor behavior, we coupled kinematics and electrophysiological recordings of participants while they performed and observed congruent or non-congruent actions or during action execution alone. We found that movement velocities and the trajectory deviations of the executed actions increased during the observation of congruent actions compared to the observation of non-congruent actions or action execution alone. This facilitation was also discernible in the motor-related potentials of the participants; the motor-related potentials were transiently more negative in the congruent condition around the onset of the executed movement, which occurred 300 ms after the onset of the observed movement. This facilitation seemed to depend not only on spatial congruency but also on the optimal temporal relationship of the observation and execution events.

Perceiving nonverbal behavior: neural correlates of processing movement fluency and contingency in dyadic interactions

Despite the fact that nonverbal dyadic social interactions are abundant in the environment, the neural mechanisms underlying their processing are not yet fully understood. Research in the field of social neuroscience has suggested that two neural networks appear to be involved in social understanding: (1) the action observation network (AON) and (2) the social neural network (SNN). The aim of this study was to determine the differential contributions of the AON and the SNN to the processing of nonverbal behavior as observed in dyadic social interactions. To this end, we used short computer animation sequences displaying dyadic social interactions between two virtual characters and systematically manipulated two key features of movement activity, which are known to influence the perception of meaning in nonverbal stimuli: (1) movement fluency and (2) contingency of movement patterns. A group of 21 male participants rated the "naturalness" of the observed scenes on a four-point scale while undergoing fMRI. Behavioral results showed that both fluency and contingency significantly influenced the "naturalness" experience of the presented animations. Neurally, the AON was preferentially engaged when processing contingent movement patterns, but did not discriminate between different degrees of movement fluency. In contrast, regions of the SNN were engaged more strongly when observing dyads with disturbed movement fluency. In conclusion, while the AON is involved in the general processing of contingent social actions, irrespective of their kinematic properties, the SNN is preferentially recruited when atypical kinematic properties prompt inferences about the agents' intentions.

Simultaneous action execution and observation optimise grasping actions

Action observation and execution share overlapping neural resonating mechanisms. In the present study, we sought to examine the effect of the activation of this system during concurrent movement observation and execution in a prehension task, when no a priori information about the requirements of grasping action was available. Although it is known that simultaneous activation by observation and execution influences motor performance, the importance of the delays of these two events and the specific effect of movement observation itself (and not the prediction of the to-be-observed movement) on action performance are poorly known. Fine-grained kinematic analysis of both the transport and grasp components of the movement should provide knowledge about the influence of movement observation on the precision and the performance of the executed movement. The experiment involved two real participants who were asked to grasp a different side of a single object that was composed of a large and a small part. In the first experiment, we measured how the transport component and the grasp component were affected by movement observation. We tested whether this influence was greater if the observed movement occurred just before the onset of movement (200 ms) or well before the onset of movement (1 s). In a second experiment, to reproduce the previous experiment and to verify the specificity of the grasping movements, we also included a condition consisting of pointing towards the object. Both experiments showed two main results. A general facilitation of the transport component was found when observing a simultaneous action, independent of its congruency. Moreover, a specific facilitation of the grasp component was present during the observation of a congruent action when movement execution and observation were nearly synchronised. While the general facilitation may arise from a competition between the two participants as they reached for the object, the specific facilitation of the grasp component seems to be directly related to mirror neuron system activity induced by action observation itself. Moreover, the time course of the events appears to be an essential factor for this modulation, implying the transitory activation of the mirror neuron system.

What is social about social perception research?

A growing consensus in social cognitive neuroscience holds that large portions of the primate visual brain are dedicated to the processing of social information, i.e., to those aspects of stimuli that are usually encountered in social interactions such as others' facial expressions, actions, and symbols. Yet, studies of social perception have mostly employed simple pictorial representations of conspecifics. These stimuli are social only in the restricted sense that they physically resemble objects with which the observer would typically interact. In an equally important sense, however, these stimuli might be regarded as "non-social": the observer knows that they are viewing pictures and might therefore not attribute current mental states to the stimuli or might do so in a qualitatively different way than in a real social interaction. Recent studies have demonstrated the importance of such higher-order conceptualization of the stimulus for social perceptual processing. Here, we assess the similarity between the various types of stimuli used in the laboratory and object classes encountered in real social interactions. We distinguish two different levels at which experimental stimuli can match social stimuli as encountered in everyday social settings: (1) the extent to which a stimulus' physical properties resemble those typically encountered in social interactions and (2) the higher-level conceptualization of the stimulus as indicating another person's mental states. We illustrate the significance of this distinction for social perception research and report new empirical evidence further highlighting the importance of mental state attribution for perceptual processing. Finally, we discuss the potential of this approach to inform studies of clinical conditions such as autism.

Visuomotor resonance in autism spectrum disorders

When we observe the actions performed by others, our motor system “resonates” along with that of the observed agent. Is a similar visuomotor resonant response observed in autism spectrum disorders (ASD)? Studies investigating action observation in ASD have yielded inconsistent findings. In this perspective article we examine behavioral and neuroscientific evidence in favor of visuomotor resonance in ASD, and consider the possible role of action-perception coupling in social cognition. We distinguish between different aspects of visuomotor resonance and conclude that while some aspects may be preserved in ASD, abnormalities exist in the way individuals with ASD convert visual information from observed actions into a program for motor execution. Such abnormalities, we surmise, may contribute to but also depend on the difficulties that individuals with ASD encounter during social interaction.

Movement coordination or movement interference: visual tracking and spontaneous coordination modulate rhythmic movement interference

When an actor performs a rhythmic limb movement while observing a spatially incongruent movement he or she exhibits increased movement orthogonal to the instructed motion. Known as rhythmic movement interference, this phenomenon has been interpreted as a motor contagion effect, whereby observing the incongruent movement interferes with the intended movement and results in a motor production error. Here we test the hypothesis that rhythmic movement interference is an emergent property of rhythmic coordination. Participants performed rhythmic limb movements at a self-selected tempo while observing a computer stimulus moving in a congruent or incongruent manner. The degree to which participants visually tracked the stimulus was manipulated to influence whether participants became spontaneously entrained to the stimulus or not. Consistent with the rhythmic coordination hypothesis, participants only exhibited the rhythmic movement interference effect when they became spontaneously entrained to the incongruent stimulus.

Moving just like you: motor interference depends on similar motility of agent and observer

Recent findings in neuroscience suggest an overlap between brain regions involved in the execution of movement and perception of another’s movement. This so-called “action-perception coupling” is supposed to serve our ability to automatically infer the goals and intentions of others by internal simulation of their actions. A consequence of this coupling is motor interference (MI), the effect of movement observation on the trajectory of one’s own movement. Previous studies emphasized that various features of the observed agent determine the degree of MI, but could not clarify how human-like an agent has to be for its movements to elicit MI and, more importantly, what ‘human-like’ means in the context of MI. Thus, we investigated in several experiments how different aspects of appearance and motility of the observed agent influence motor interference (MI). Participants performed arm movements in horizontal and vertical directions while observing videos of a human, a humanoid robot, or an industrial robot arm with either artificial (industrial) or human-like joint configurations. Our results show that, given a human-like joint configuration, MI was elicited by observing arm movements of both humanoid and industrial robots. However, if the joint configuration of the robot did not resemble that of the human arm, MI could longer be demonstrated. Our findings present evidence for the importance of human-like joint configuration rather than other human-like features for perception-action coupling when observing inanimate agents.

Visual gravity influences arm movement planning

When submitted to a visuomotor rotation, subjects show rapid adaptation of visually guided arm reaching movements, indicated by a progressive reduction in reaching errors. In this study, we wanted to make a step forward by investigating to what extent this adaptation also implies changes into the motor plan. Up to now, classical visuomotor rotation paradigms have been performed on the horizontal plane, where the reaching motor plan in general requires the same kinematics (i.e., straight path and symmetric velocity profile). To overcome this limitation, we considered vertical and horizontal movement directions requiring specific velocity profiles. This way, a change in the motor plan due to the visuomotor conflict would be measurable in terms of a modification in the velocity profile of the reaching movement. Ten subjects performed horizontal and vertical reaching movements while observing a rotated visual feedback of their motion. We found that adaptation to a visuomotor rotation produces a significant change in the motor plan, i.e., changes to the symmetry of velocity profiles. This suggests that the central nervous system takes into account the visual information to plan a future motion, even if this causes the adoption of nonoptimal motor plans in terms of energy consumption. However, the influence of vision on arm movement planning is not fixed, but rather changes as a function of the visual orientation of the movement. Indeed, a clear influence on motion planning can be observed only when the movement is visually presented as oriented along the vertical direction. Thus vision contributes differently to the planning of arm pointing movements depending on motion orientation in space.

Coordination dynamics in a socially situated nervous system

Traditional theories of cognitive science have typically accounted for the organization of human behavior by detailing requisite computational/representational functions and identifying neurological mechanisms that might perform these functions. Put simply, such approaches hold that neural activity causes behavior. This same general framework has been extended to accounts of human social behavior via concepts such as “common-coding” and “co-representation” and much recent neurological research has been devoted to brain structures that might execute these social-cognitive functions. Although these neural processes are unquestionably involved in the organization and control of human social interactions, there is good reason to question whether they should be accorded explanatory primacy. Alternatively, we propose that a full appreciation of the role of neural processes in social interactions requires appropriately situating them in their context of embodied-embedded constraints. To this end, we introduce concepts from dynamical systems theory and review research demonstrating that the organization of human behavior, including social behavior, can be accounted for in terms of self-organizing processes and lawful dynamics of animal-environment systems. Ultimately, we hope that these alternative concepts can complement the recent advances in cognitive neuroscience and thereby provide opportunities to develop a complete and coherent account of human social interaction.

Social top-down response modulation (STORM): a model of the control of mimicry in social interaction

As a distinct feature of human social interactions, spontaneous mimicry has been widely investigated in the past decade. Research suggests that mimicry is a subtle and flexible social behavior which plays an important role for communication and affiliation. However, fundamental questions like why and how people mimic still remain unclear. In this paper, we evaluate past theories of why people mimic and the brain systems that implement mimicry in social psychology and cognitive neuroscience. By reviewing recent behavioral and neuroimaging studies on the control of mimicry by social signals, we conclude that the subtlety and sophistication of mimicry in social contexts reflect a social top-down response modulation (STORM) which increases one's social advantage and this mechanism is most likely implemented by medial prefrontal cortex (mPFC). We suggest that this STORM account of mimicry is important for our understanding of social behavior and social cognition, and provides implications for future research in autism.

Ocular tracking of biological and nonbiological motion: the effect of instructed agency

Recent findings suggest that visuomotor performance is modulated by people's beliefs about the agency (e.g., animate vs. inanimate) behind the events they perceive. This study investigated the effect of instructed agency on ocular tracking of point-light motions with biological and nonbiological velocity profiles. The motions followed either a relatively simple (ellipse) or a more complex (scribble) trajectory, and agency was manipulated by informing the participants that the motions they saw were either human or computer generated. In line with previous findings, tracking performance was better for biological than for nonbiological motions, and this effect was particularly pronounced for the simpler (elliptical) motions. The biological advantage was also larger for the human than for the computer instruction condition, but only for a measure that captured the predictive component of smooth pursuit. These results suggest that ocular tracking is influenced by the internal forward model people choose to adopt.