The visible face of intention: why kinematics matters

Contextual expectations shape the motor coding of movement kinematics during the prediction of observed actions: A TMS study

Contextual information may shape motor resonance and support intention understanding during observation of incomplete, ambiguous actions. It is unclear, however, whether this effect is contingent upon kinematics ambiguity or contextual information is continuously integrated with kinematics to predict the overarching action intention. Moreover, a differentiation between the motor mapping of the intention suggested by context or kinematics has not been clearly demonstrated. In a first action execution phase, 29 participants were asked to perform reaching-to-grasp movements towards big or small food objects with the intention to eat or to move; electromyography from the First Dorsal Interosseous (FDI) and Abductor Digiti Minimi (ADM) was recorded. Depending on object size, the intentions to eat or to move were differently implemented by a whole-hand or a precision grip kinematics, thus qualifying an action-muscle dissociation. Then, in a following action prediction task, the same participants were asked to observe an actor performing the same actions and to predict his/her intention while motor resonance was assessed for the same muscles. Of note, videos were interrupted at early or late action phases, and actions were embedded in contexts pointing toward an eating or a moving intention, congruently or incongruently with kinematics. We found greater involvement of the FDI or ADM in the execution of precision or whole-hand grips, respectively. Crucially, this pattern of activation was mirrored during observation of the same actions in congruent contexts, but it was cancelled out or reversed in the incongruent ones, either when videos were interrupted at either early or long phases of action deployment. Our results extend previous evidence by showing that contextual information shapes motor resonance not only under conditions of perceptual uncertainty but also when more informative kinematics is available.

Action representations in prevention behavior: Evidence from motor execution

Human actions sometimes aim at preventing an event from occurring. How these to-be-prevented events are represented, however, is poorly understood. Recent proposals in the literature point to a possible divide between effect-producing, operant actions, and effect-precluding, prevention actions, suggesting that the control of operant actions relies on codes of environment-related effects whereas prevention actions do not. Here we report two experiments on this issue, showing that spatial features (Experiment 1) as well as temporal features (Experiment 2) of to-be-prevented events influence actions in the same way as corresponding features of to-be-produced effects. This implies that selecting and executing prevention actions relies on anticipated environmental changes, comparable to operant actions.

Motor invariants in action execution and perception

The nervous system is sensitive to statistical regularities of the external world and forms internal models of these regularities to predict environmental dynamics. Given the inherently social nature of human behavior, being capable of building reliable predictive models of others' actions may be essential for successful interaction. While social prediction might seem to be a daunting task, the study of human motor control has accumulated ample evidence that our movements follow a series of kinematic invariants, which can be used by observers to reduce their uncertainty during social exchanges. Here, we provide an overview of the most salient regularities that shape biological motion, examine the role of these invariants in recognizing others' actions, and speculate that anchoring socially-relevant perceptual decisions to such kinematic invariants provides a key computational advantage for inferring conspecifics' goals and intentions.

The Components of Interpersonal Synchrony in the Typical Population and in Autism: A Conceptual Analysis

Interpersonal synchrony - the tendency for social partners to temporally co-ordinate their behaviour when interacting - is a ubiquitous feature of social interactions. Synchronous interactions play a key role in development, and promote social bonding and a range of pro-social behavioural outcomes across the lifespan. The process of achieving and maintaining interpersonal synchrony is highly complex, with inputs required from across perceptual, temporal, motor, and socio-cognitive domains. In this conceptual analysis, we synthesise evidence from across these domains to establish the key components underpinning successful non-verbal interpersonal synchrony, how such processes interact, and factors that may moderate their operation. We also consider emerging evidence that interpersonal synchrony is reduced in autistic populations. We use our account of the components contributing to interpersonal synchrony in the typical population to identify potential points of divergence in interpersonal synchrony in autism. The relationship between interpersonal synchrony and broader aspects of social communication in autism are also considered, together with implications for future research.

Differences in functional brain organization during gesture recognition between autistic and neurotypical individuals

Persons with and without autism process sensory information differently. Differences in sensory processing are directly relevant to social functioning and communicative abilities, which are known to be hampered in persons with autism. We collected functional magnetic resonance imaging data from 25 autistic individuals and 25 neurotypical individuals while they performed a silent gesture recognition task. We exploited brain network topology, a holistic quantification of how networks within the brain are organized to provide new insights into how visual communicative signals are processed in autistic and neurotypical individuals. Performing graph theoretical analysis, we calculated two network properties of the action observation network: 'local efficiency', as a measure of network segregation, and 'global efficiency', as a measure of network integration. We found that persons with autism and neurotypical persons differ in how the action observation network is organized. Persons with autism utilize a more clustered, local-processing-oriented network configuration (i.e. higher local efficiency) rather than the more integrative network organization seen in neurotypicals (i.e. higher global efficiency). These results shed new light on the complex interplay between social and sensory processing in autism.

Liking as a balance between synchronization, complexity and novelty

Synchronization has been identified as a key aspect in social bonding. While synchronization could be maximized by increasing the predictability of an interaction, such predictability is in tension with individuals' level of interest, which is tied to the interaction's complexity and novelty. In this study, we tested the interplay between synchronization and interest. We asked 104 female dyads to play the Mirror Game, in which they had to move their hands as coordinately as possible, and then report how much they liked each other. Utilizing information theory and video processing tools, we found that a combination of movement synchronization and complexity explained liking almost two times better than movement synchronization alone. Moreover, we found that people initiated novel and challenging interactions, even though they paid a price-being less synchronized. Examining the interactions' dynamics, we found that people who liked each other moved in a more synchronized, complex, and novel manner during most of the interaction. This suggests that in addition to synchronization, maintaining interest may be critical for positive social bonding. Thus, we propose a new framework in which balancing synchronization and interest, rather than merely maximizing synchronization, optimizes the interaction quality.

Kinematic-Based Classification of Social Gestures and Grasping by Humans and Machine Learning Techniques

The affective motion of humans conveys messages that other humans perceive and understand without conventional linguistic processing. This ability to classify human movement into meaningful gestures or segments plays also a critical role in creating social interaction between humans and robots. In the research presented here, grasping and social gesture recognition by humans and four machine learning techniques (k-Nearest Neighbor, Locality-Sensitive Hashing Forest, Random Forest and Support Vector Machine) is assessed by using human classification data as a reference for evaluating the classification performance of machine learning techniques for thirty hand/arm gestures. The gestures are rated according to the extent of grasping motion on one task and the extent to which the same gestures are perceived as social according to another task. The results indicate that humans clearly rate differently according to the two different tasks. The machine learning techniques provide a similar classification of the actions according to grasping kinematics and social quality. Furthermore, there is a strong association between gesture kinematics and judgments of grasping and the social quality of the hand/arm gestures. Our results support previous research on intention-from-movement understanding that demonstrates the reliance on kinematic information for perceiving the social aspects and intentions in different grasping actions as well as communicative point-light actions.

Sowing the seeds of intentionality: Motor intentions in plants

Motor intention/intentionality has been investigated from a wide variety of perspectives: some researchers have, for example, been focusing on the purely physical and mechanical aspects underlying the control of action, while others have been concentrating on subjective intentionality. Basically, all approaches ranging from the neuroscientific to phenomenological-inspired ones have been used to investigate motor intentions. The current study set out to examine motor intentions in connection to plant behavior utilizing the final goal of plant action as the definition of its motor intention. Taking a wide-angle approach, the first part of the review is dedicated to examining philosophical and psychological studies on motor intentions. Recent data demonstrating that plant behavior does indeed seem goal-directed will then be reviewed as we ponder the possibility of purposeful or intentional plant responses to stimuli and stress conditions in their environment. The article will draw to a close as we examine current theories attempting to explain plants' overt behavior and corresponding covert representations.

Bridging the gap between emotion and joint action

Our daily human life is filled with a myriad of joint action moments, be it children playing, adults working together (i.e., team sports), or strangers navigating through a crowd. Joint action brings individuals (and embodiment of their emotions) together, in space and in time. Yet little is known about how individual emotions propagate through embodied presence in a group, and how joint action changes individual emotion. In fact, the multi-agent component is largely missing from neuroscience-based approaches to emotion, and reversely joint action research has not found a way yet to include emotion as one of the key parameters to model socio-motor interaction. In this review, we first identify the gap and then stockpile evidence showing strong entanglement between emotion and acting together from various branches of sciences. We propose an integrative approach to bridge the gap, highlight five research avenues to do so in behavioral neuroscience and digital sciences, and address some of the key challenges in the area faced by modern societies.

Action Observation Responses Are Influenced by Movement Kinematics and Target Identity

In order to inform the debate whether cortical areas related to action observation provide a pragmatic or a semantic representation of goal-directed actions, we performed 2 functional magnetic resonance imaging (fMRI) experiments in humans. The first experiment, involving observation of aimless arm movements, resulted in activation of most of the components known to support action execution and action observation. Given the absence of a target/goal in this experiment and the activation of parieto-premotor cortical areas, which were associated in the past with direction, amplitude, and velocity of movement of biological effectors, our findings suggest that during action observation we could be monitoring movement kinematics. With the second, double dissociation fMRI experiment, we revealed the components of the observation-related cortical network affected by 1) actions that have the same target/goal but different reaching and grasping kinematics and 2) actions that have very similar kinematics but different targets/goals. We found that certain areas related to action observation, including the mirror neuron ones, are informed about movement kinematics and/or target identity, hence providing a pragmatic rather than a semantic representation of goal-directed actions. Overall, our findings support a process-driven simulation-like mechanism of action understanding, in agreement with the theory of motor cognition, and question motor theories of action concept processing.

The communicative advantage: how kinematic signaling supports semantic comprehension

Humans are unique in their ability to communicate information through representational gestures which visually simulate an action (eg. moving hands as if opening a jar). Previous research indicates that the intention to communicate modulates the kinematics (e.g., velocity, size) of such gestures. If and how this modulation influences addressees' comprehension of gestures have not been investigated. Here we ask whether communicative kinematic modulation enhances semantic comprehension (i.e., identification) of gestures. We additionally investigate whether any comprehension advantage is due to enhanced early identification or late identification. Participants (n = 20) watched videos of representational gestures produced in a more- (n = 60) or less-communicative (n = 60) context and performed a forced-choice recognition task. We tested the isolated role of kinematics by removing visibility of actor's faces in Experiment I, and by reducing the stimuli to stick-light figures in Experiment II. Three video lengths were used to disentangle early identification from late identification. Accuracy and response time quantified main effects. Kinematic modulation was tested for correlations with task performance. We found higher gesture identification performance in more- compared to less-communicative gestures. However, early identification was only enhanced within a full visual context, while late identification occurred even when viewing isolated kinematics. Additionally, temporally segmented acts with more post-stroke holds were associated with higher accuracy. Our results demonstrate that communicative signaling, interacting with other visual cues, generally supports gesture identification, while kinematic modulation specifically enhances late identification in the absence of other cues. Results provide insights into mutual understanding processes as well as creating artificial communicative agents.

Informing, Coordinating, and Performing: A Perspective on Functions of Sensorimotor Communication

Sensorimotor communication is a form of communication instantiated through body movements that are guided by both instrumental, goal-directed intentions and communicative, social intentions. Depending on the social interaction context, sensorimotor communication can serve different functions. This article aims to disentangle three of these functions: (a) an informing function of body movements, to highlight action intentions for an observer; (b) a coordinating function of body movements, to facilitate real-time action prediction in joint action; and (c) a performing function of body movements, to elicit emotional or aesthetic experiences in an audience. We provide examples of research addressing these different functions as well as some influencing factors, relating to individual differences, task characteristics, and situational demands. The article concludes by discussing the benefits of a closer dialog between separate lines of research on sensorimotor communication across different social contexts.

Neural dynamics of grip and goal integration during the processing of others' actions with objects: An ERP study

Recent behavioural evidence suggests that when processing others’ actions, motor acts and goal-related information both contribute to action recognition. Yet the neuronal mechanisms underlying the dynamic integration of the two action dimensions remain unclear. This study aims to elucidate the ERP components underlying the processing and integration of grip and goal-related information. The electrophysiological activity of 28 adults was recorded during the processing of object-directed action photographs (e.g., writing with pencil) containing either grip violations (e.g. upright pencil grasped with atypical-grip), goal violations (e.g., upside-down pencil grasped with typical-grip), both grip and goal violations (e.g., upside-down pencil grasped with atypical-grip), or no violations. Participants judged whether actions were overall typical or not according to object typical use. Brain activity was sensitive to the congruency between grip and goal information on the N400, reflecting the semantic integration between the two dimensions. On earlier components, brain activity was affected by grip and goal typicality independently. Critically, goal typicality but not grip typicality affected brain activity on the N300, supporting an earlier role of goal-related representations in action recognition. Findings provide new insights on the neural temporal dynamics of the integration of motor acts and goal-related information during the processing of others’ actions.

What first drives visual attention during the recognition of object-directed actions? The role of kinematics and goal information

The recognition of others' object-directed actions is known to involve the decoding of both the visual kinematics of the action and the action goal. Yet whether action recognition is first guided by the processing of visual kinematics or by a prediction about the goal of the actor remains debated. In order to provide experimental evidence to this issue, the present study aimed at investigating whether visual attention would be preferentially captured by visual kinematics or by action goal information when processing others' actions. In a visual search task, participants were asked to find correct actions (e.g., drinking from glass) among distractor actions. Distractors actions contained grip and/or goal violations and could therefore share the correct goal and/or the correct grip with the target. The time course of fixation proportion on each distractor action has been taken as an indicator of visual attention allocation. Results show that visual attention is first captured by the distractor action with similar goal. Then the withdrawal of visual attention from the action distractor with similar goal suggests a later attentional capture by the action distractor with similar grip. Overall, results are in line with predictive approaches of action understanding, which assume that observers first make a prediction about the actor's goal before verifying this prediction using the visual kinematics of the action.

Observation of Both Skilled and Erroneous Object Lifting Can Improve Predictive Force Scaling in the Observer

Recent studies have highlighted that the observation of hand-object interactions can influence perceptual weight judgments made by an observer. Moreover, observing salient motor errors during object lifting allows individuals to update their internal sensorimotor representation about object weight. Embodying observed visuomotor cues for the planning of a motor command further enables individuals to accurately scale their fingertip forces when subsequently lifting the same object. However, it is still unknown whether the observation of a skilled lift is equally able to mediate predictive motor control in the observer. Here, we tested this hypothesis by asking participants to grasp and lift a manipulandum after observing an actor's lift. The object weight changed unpredictably (light or heavy) every fourth to sixth trial performed by the actor. Participants were informed that they would always lift the same weight as the actor and that, based on the experimental condition, they would have to observe skilled or erroneously performed lifts. Our results revealed that the observation of both skilled and erroneously performed lifts allows participants to update their internal sensorimotor object representation, in turn enabling them to predict force scaling accurately. These findings suggest that the observation of salient motor errors, as well as subtle features of skilled motor performance, are embodied in the observer's motor repertoire and can drive changes in predictive motor control.

What Can You See? Identifying Cues on Internal States From the Movements of Natural Social Interactions

In recent years, the field of Human-Robot Interaction (HRI) has seen an increasing demand for technologies that can recognize and adapt to human behaviors and internal states (e.g., emotions and intentions). Psychological research suggests that human movements are important for inferring internal states. There is, however, a need to better understand what kind of information can be extracted from movement data, particularly in unconstrained, natural interactions. The present study examines which internal states and social constructs humans identify from movement in naturalistic social interactions. Participants either viewed clips of the full scene or processed versions of it displaying 2D positional data. Then, they were asked to fill out questionnaires assessing their social perception of the viewed material. We analyzed whether the full scene clips were more informative than the 2D positional data clips. First, we calculated the inter-rater agreement between participants in both conditions. Then, we employed machine learning classifiers to predict the internal states of the individuals in the videos based on the ratings obtained. Although we found a higher inter-rater agreement for full scenes compared to positional data, the level of agreement in the latter case was still above chance, thus demonstrating that the internal states and social constructs under study were identifiable in both conditions. A factor analysis run on participants' responses showed that participants identified the constructs interaction imbalance, interaction valence and engagement regardless of video condition. The machine learning classifiers achieved a similar performance in both conditions, again supporting the idea that movement alone carries relevant information. Overall, our results suggest it is reasonable to expect a machine learning algorithm, and consequently a robot, to successfully decode and classify a range of internal states and social constructs using low-dimensional data (such as the movements and poses of observed individuals) as input.

A review and consideration on the kinematics of reach-to-grasp movements in macaque monkeys

The bases for understanding the neuronal mechanisms that underlie the control of reach-to-grasp movements among nonhuman primates, particularly macaques, has been widely studied. However, only a few kinematic descriptions of their prehensile actions are available. A thorough understanding of macaques' prehensile movements is manifestly critical, in light of their role in biomedical research as valuable models for studying neuromotor disorders and brain mechanisms, as well as for developing brain-machine interfaces to facilitate arm control. This article aims to review the current state of knowledge on the kinematics of grasping movements that macaques perform in naturalistic, seminaturalistic, and laboratory settings, to answer the following questions: Are kinematic signatures affected by the context within which the movement is performed? In what ways are kinematics of humans' and macaques' prehensile actions similar/dissimilar? Our analysis reflects the challenges involved in making comparisons across settings and species due to the heterogeneous picture in terms of the number of subjects, stimuli, conditions, and hands used. The kinematics of free-ranging macaques are characterized by distinctive features that are exhibited neither by macaques in laboratory setting nor by human subjects. The temporal incidence of key kinematic landmarks diverges significantly between species, indicating disparities in the overall organization of movement. Given such complexities, we attempt a synthesis of the extant body of evidence, intending to generate some significant implications for directions that future research might take to recognize the remaining gaps and pursue the insights and resolutions to generate an interpretation of movement kinematics that accounts for all settings and subjects.

Social modulation of object-directed but not image-directed actions

There has recently been an increased research focus on the influence of social factors on human cognition, attention, and action. While this represents an important step towards an ecologically valid description of real-world behaviour, this work has primarily examined dyads interacting with virtual stimuli i.e. on-screen images of objects. Though differences between actions to images and real items are known, their relative sensitivity to social factors is largely unknown. We argue that because images and real items elicit different neural representations, patterns of attention, and hand actions, a direct comparison between the magnitude of social effects while interacting with images and real objects is demanded. We examined patterns of reaching as individuals performed a shape-matching game. Images and real objects were used as stimuli, and social context was manipulated via the proximity of an observer. We found that social context interacted with stimulus type to modulate behaviour. Specifically, there was a delay in reaching for distant objects when a participant was facing another individual but this social effect only occurred when the stimuli were real objects. Our data suggest that even when images and real objects are arranged to share the affordance of reachability, they differ in their sensitivity to social influences. Therefore, the measurement of social effects using on-screen stimuli may poorly predict the social effects of actions directed towards real objects. Accordingly, generalizations between these two domains should be treated with caution.

Movement kinematics drive chain selection toward intention detection

Estimation of intentions from the observation of other people’s actions has been proposed to rely on the same motor chain organization supporting the execution of intentional actions. However, the nature of the mechanism by which a specific neuronal chain is selected among possible alternatives during action observation remains obscure. Our study shows that in absence of discriminative contextual cues, subtle changes in the kinematics of the observed action inform mapping to the most probable chain. These results shed light on the importance of kinematics for the attribution of intentions to actions.

The ability to understand intentions based on another’s movements is crucial for human interaction. This ability has been ascribed to the so-called motor chaining mechanism: anytime a motor chain is activated (e.g., grasp-to-drink), the observer attributes to the agent the corresponding intention (i.e., to drink) from the first motor act (i.e., the grasp). However, the mechanisms by which a specific chain is selected in the observer remain poorly understood. In the current study, we investigate the possibility that in the absence of discriminative contextual cues, slight kinematic variations in the observed grasp inform mapping to the most probable chain. Chaining of motor acts predicts that, in a sequential grasping task (e.g., grasp-to-drink), electromyographic (EMG) components that are required for the final act [e.g., the mouth-opening mylohyoid (MH) muscle] show anticipatory activation. To test this prediction, we used MH EMG, transcranial magnetic stimulation (TMS; MH motor-evoked potentials), and predictive models of movement kinematics to measure the level and timing of MH activation during the execution (Experiment 1) and the observation (Experiment 2) of reach-to-grasp actions. We found that MH-related corticobulbar excitability during grasping observation varied as a function of the goal (to drink or to pour) and the kinematics of the observed grasp. These results show that subtle changes in movement kinematics drive the selection of the most probable motor chain, allowing the observer to link an observed act to the agent’s intention.

Visuo-motor interference with a virtual partner is equally present in cooperative and competitive interactions

Automatic imitation of observed actions is thought to be a powerful mechanism, one that may mediate the reward value of interpersonal interactions, but that could also generate visuo-motor interference when interactions involve complementary movements. Since interpersonal coordination seems to be crucial both when cooperating and competing with others, the questions arises as to whether imitation-and thus visuo-motor interference-occurs in both scenarios. To address this issue, we asked human participants to engage in high- or low-interactive (Interactive or Cued condition, respectively), cooperative or competitive, joint reach-to-grasps with a virtual partner. More specifically, interactions occurred in: (i) a Cued condition, where participants simply adapted their movement timing to synchronize with (during cooperation) or anticipate (during competition) the virtual partner's grasp; (ii) an Interactive condition requiring the same adaptation, as well as a real-time selection of their action according to the virtual character's movement. To simulate a realistic human-human interaction, the virtual character would change its movement speed in consecutive trials according to participants' behaviour. Results demonstrate that visuo-motor interference-as indexed by movement kinematics (higher maximum wrist height during complementary compared to imitative power grips)-emerge in both cooperative and competitive motor interactions only when predictions about the partner's movements are needed to perform one's own action (interactive condition). These results support the idea that simulative imitation is heavily present when individuals need to match their behaviours closely.

Communicative intent modulates production and comprehension of actions and gestures: A Kinect study

Actions may be used to directly act on the world around us, or as a means of communication. Effective communication requires the addressee to recognize the act as being communicative. Humans are sensitive to ostensive communicative cues, such as direct eye gaze (Csibra & Gergely, 2009). However, there may be additional cues present in the action or gesture itself. Here we investigate features that characterize the initiation of a communicative interaction in both production and comprehension. We asked 40 participants to perform 31 pairs of object-directed actions and representational gestures in more- or less- communicative contexts. Data were collected using motion capture technology for kinematics and video recording for eye-gaze. With these data, we focused on two issues. First, if and how actions and gestures are systematically modulated when performed in a communicative context. Second, if observers exploit such kinematic information to classify an act as communicative. Our study showed that during production the communicative context modulates space-time dimensions of kinematics and elicits an increase in addressee-directed eye-gaze. Naïve participants detected communicative intent in actions and gestures preferentially using eye-gaze information, only utilizing kinematic information when eye-gaze was unavailable. Our study highlights the general communicative modulation of action and gesture kinematics during production but also shows that addressees only exploit this modulation to recognize communicative intention in the absence of eye-gaze. We discuss these findings in terms of distinctive but potentially overlapping functions of addressee directed eye-gaze and kinematic modulations within the wider context of human communication and learning.

Evidence for the representation of movement kinematics in the discharge of F5 mirror neurons during the observation of transitive and intransitive actions

Mirror neurons (MirNs) are sensorimotor neurons that fire both when an animal performs a goal-directed action and when the same animal observes another agent performing the same or a similar transitive action. It has been claimed that the observation of intransitive actions does not activate MirNs in a monkey’s brain. Prompted by recent evidence indicating that the discharge of MirNs is modulated also by non-object-directed actions, we investigated thoroughly the efficacy of intransitive actions to trigger MirNs’ discharge. Using representational similarity analysis, we also studied whether the elements constituting the visual scene presented to the monkey during the observation of actions (both transitive and intransitive) are represented in the discharge of MirNs. For this purpose, the moving hand was modeled by its kinematics and the object by features of its geometry. We found that MirNs respond to the observation of both transitive and intransitive actions and that the discharge differences evoked by the observation of object- and non-object-directed actions are correlated more with the kinematic differences of these actions than with the differences of the objects’ features. These findings support the view that observed action kinematics contribute to action mirroring.

NEW & NOTEWORTHY Mirror neurons in the monkey brain are thought to respond exclusively to the observation of object-directed actions. Here, we show that mirror neurons also respond to the observation of intransitive actions and that the kinematics of the observed movements are represented in their discharge. This finding supports the view that mirror neurons provide also a kinematics-based representation of actions.

Where Are You Throwing the Ball? I Better Watch Your Body, Not Just Your Arm!

The ability to intercept or avoid a moving object, whether to catch a ball, snatch one's prey, or avoid the path of a predator, is a skill that has been acquired throughout evolution by many species in the animal kingdom. This requires processing early visual cues in order to program anticipatory motor responses tuned to the forthcoming event. Here, we explore the nature of the early kinematics cues that could inform an observer about the future direction of a ball projected with an unconstrained overarm throw. Our goal was to pinpoint the body segments that, throughout the temporal course of the throwing action, could provide key cues for accurately predicting the side of the outgoing ball. We recorded whole-body kinematics from twenty non-expert participants performing unconstrained overarm throws at four different targets placed on a vertical plane at 6 m distance. In order to characterize the spatiotemporal structure of the information embedded in the kinematics of the throwing action about the outgoing ball direction, we introduced a novel combination of dimensionality reduction and machine learning techniques. The recorded kinematics clearly shows that throwing styles differed considerably across individuals, with corresponding inter-individual differences in the spatio-temporal structure of the thrower predictability. We found that for most participants it is possible to predict the region where the ball hit the target plane, with an accuracy above 80%, as early as 400-500 ms before ball release. Interestingly, the body parts that provided the most informative cues about the action outcome varied with the throwing style and during the time course of the throwing action. Not surprisingly, at the very end of the action, the throwing arm is the most informative body segment. However, cues allowing for predictions to be made earlier than 200 ms before release are typically associated to other body parts, such as the lower limbs and the contralateral arm. These findings are discussed in the context of the sport-science literature on throwing and catching interactive tasks, as well as from the wider perspective of the role of sensorimotor coupling in interpersonal social interactions.

Decoding social intentions in human prehensile actions: Insights from a combined kinematics-fMRI study

Consistent evidence suggests that the way we reach and grasp an object is modulated not only by object properties (e.g., size, shape, texture, fragility and weight), but also by the types of intention driving the action, among which the intention to interact with another agent (i.e., social intention). Action observation studies ascribe the neural substrate of this 'intentional' component to the putative mirror neuron (pMNS) and the mentalizing (MS) systems. How social intentions are translated into executed actions, however, has yet to be addressed. We conducted a kinematic and a functional Magnetic Resonance Imaging (fMRI) study considering a reach-to-grasp movement performed towards the same object positioned at the same location but with different intentions: passing it to another person (social condition) or putting it on a concave base (individual condition). Kinematics showed that individual and social intentions are characterized by different profiles, with a slower movement at the level of both the reaching (i.e., arm movement) and the grasping (i.e., hand aperture) components. fMRI results showed that: (i) distinct voxel pattern activity for the social and the individual condition are present within the pMNS and the MS during action execution; (ii) decoding accuracies of regions belonging to the pMNS and the MS are correlated, suggesting that these two systems could interact for the generation of appropriate motor commands. Results are discussed in terms of motor simulation and inferential processes as part of a hierarchical generative model for action intention understanding and generation of appropriate motor commands.

Shaping of Reach-to-Grasp Kinematics by Intentions: A Meta-Analysis

When we reach to grasp something, we need to take into account both the properties of the object we are grasping and the intention we have in mind. Previous research has found these constraints to be visible in the reach-to-grasp kinematics, but there is no consensus on which kinematic parameters are the most sensitive. To examine this, a systematic literature search and meta-analyses were performed. The search identified studies assessing how changes in either an object property or a prior intention affect reach-to-grasp kinematics in healthy participants. Hereafter, meta-analyses were conducted using a restricted maximum likelihood random effect model. The meta-analyses showed that changes in both object properties and prior intentions affected reach-to-grasp kinematics. Based on these results, the authors argue for a tripartition of the reach-to-grasp movement in which the accelerating part of the reach is primarily associated with transporting the hand to the object (i.e., extrinsic object properties), the decelerating part of the reach is used as a preparation for object manipulation (i.e., prepare the grasp or the subsequent action), and the grasp is associated with manipulating the object's intrinsic properties, especially object size.

Anticipatory action planning in blind and sighted individuals

Several studies on visually guided reach-to-grasp movements have documented that how objects are grasped differs depending on the actions one intends to perform subsequently. However, no previous study has examined whether this differential grasping may also occur without visual input. In this study, we used motion capture technology to investigate the influence of visual feedback and prior visual experience on the modulation of kinematics by intention in sighted (in both full-vision and no-vision conditions), early-blind and late-blind participants. Results provide evidence of modulation of kinematics by intention to a similar degree under both full-vision and no-vision conditions. Moreover, they demonstrate that prior visual experience has little impact on the tailoring of grasping movements to intention. This suggests that sequential action planning does not depend on visual input, and may instead be ascribed to the function of multisensory-motor cortical network that operates and develops not only in light, but also in darkness.

A method for detecting characteristic patterns in social interactions with an application to handover interactions

Social interactions are a defining behavioural trait of social animals. Discovering characteristic patterns in the display of such behaviour is one of the fundamental endeavours in behavioural biology and psychology, as this promises to facilitate the general understanding, classification, prediction and even automation of social interactions. We present a novel approach to study characteristic patterns, including both sequential and synchronous actions in social interactions. The key concept in our analysis is to represent social interactions as sequences of behavioural states and to focus on changes in behavioural states shown by individuals rather than on the duration for which they are displayed. We extend techniques from data mining and bioinformatics to detect frequent patterns in these sequences and to assess how these patterns vary across individuals or changes in interaction tasks. To illustrate our approach and to demonstrate its potential, we apply it to novel data on a simple physical interaction, where one person hands a cup to another person. Our findings advance the understanding of handover interactions, a benchmark scenario for social interactions. More generally, we suggest that our approach permits a general perspective for studying social interactions.

A method for detecting characteristic patterns in social interactions with an application to handover interactions

Social interactions are a defining behavioural trait of social animals. Discovering characteristic patterns in the display of such behaviour is one of the fundamental endeavours in behavioural biology and psychology, as this promises to facilitate the general understanding, classification, prediction and even automation of social interactions. We present a novel approach to study characteristic patterns, including both sequential and synchronous actions in social interactions. The key concept in our analysis is to represent social interactions as sequences of behavioural states and to focus on changes in behavioural states shown by individuals rather than on the duration for which they are displayed. We extend techniques from data mining and bioinformatics to detect frequent patterns in these sequences and to assess how these patterns vary across individuals or changes in interaction tasks. To illustrate our approach and to demonstrate its potential, we apply it to novel data on a simple physical interaction, where one person hands a cup to another person. Our findings advance the understanding of handover interactions, a benchmark scenario for social interactions. More generally, we suggest that our approach permits a general perspective for studying social interactions.

A Causal Role for Primary Motor Cortex in Perception of Observed Actions

It has been proposed that motor system activity during action observation may be modulated by the kinematics of observed actions. One purpose of this activity during action observation may be to predict the visual consequence of another person's action based on their movement kinematics. Here, we tested the hypothesis that the primary motor cortex (M1) may have a causal role in inferring information that is present in the kinematics of observed actions. Healthy participants completed an action perception task before and after applying continuous theta burst stimulation (cTBS) over left M1. A neurophysiological marker was used to quantify the extent of M1 disruption following cTBS and stratify our sample a priori to provide an internal control. We found that a disruption to M1 caused a reduction in an individual's sensitivity to interpret the kinematics of observed actions; the magnitude of suppression of motor excitability predicted this change in sensitivity.

Doing It Your Way: How Individual Movement Styles Affect Action Prediction

Individuals show significant variations in performing a motor act. Previous studies in the action observation literature have largely ignored this ubiquitous, if often unwanted, characteristic of motor performance, assuming movement patterns to be highly similar across repetitions and individuals. In the present study, we examined the possibility that individual variations in motor style directly influence the ability to understand and predict others' actions. To this end, we first recorded grasping movements performed with different intents and used a two-step cluster analysis to identify quantitatively 'clusters' of movements performed with similar movement styles (Experiment 1). Next, using videos of the same movements, we proceeded to examine the influence of these styles on the ability to judge intention from action observation (Experiments 2 and 3). We found that motor styles directly influenced observers' ability to 'read' others' intention, with some styles always being less 'readable' than others. These results provide experimental support for the significance of motor variability for action prediction, suggesting that the ability to predict what another person is likely to do next directly depends on her individual movement style.

Motor planning of goal-directed action is tuned by the emotional valence of the stimulus: a kinematic study

The basic underpinnings of homeostatic behavior include interacting with positive items and avoiding negative ones. As the planning aspects of goal-directed actions can be inferred from their movement features, we investigated the kinematics of interacting with emotion-laden stimuli. Participants were instructed to grasp emotion-laden stimuli and bring them toward their bodies while the kinematics of their wrist movement was measured. The results showed that the time to peak velocity increased for bringing pleasant stimuli towards the body compared to unpleasant and neutral ones, suggesting higher easiness in undertaking the task with pleasant stimuli. Furthermore, bringing unpleasant stimuli towards the body increased movement time in comparison with both pleasant and neutral ones while the time to peak velocity for unpleasant stimuli was the same as for that of neutral stimuli. There was no change in the trajectory length among emotional categories. We conclude that during the “reach-to-grasp” and “bring-to-the-body” movements, the valence of the stimuli affects the temporal but not the spatial kinematic features of motion. To the best of our knowledge, we show for the first time that the kinematic features of a goal-directed action are tuned by the emotional valence of the stimuli.

Faster but Less Careful Prehension in Presence of High, Rather than Low, Social Status Attendees

Ample evidence attests that social intention, elicited through gestures explicitly signaling a request of communicative intention, affects the patterning of hand movement kinematics. The current study goes beyond the effect of social intention and addresses whether the same action of reaching to grasp an object for placing it in an end target position within or without a monitoring attendee's peripersonal space, can be moulded by pure social factors in general, and by social facilitation in particular. A motion tracking system (Optotrak Certus) was used to record motor acts. We carefully avoided the usage of communicative intention by keeping constant both the visual information and the positional uncertainty of the end target position, while we systematically varied the social status of the attendee (a high, or a low social status) in separated blocks. Only thirty acts performed in the presence of a different social status attendee, revealed a significant change of kinematic parameterization of hand movement, independently of the attendee's distance. The amplitude of peak velocity reached by the hand during the reach-to-grasp and the lift-to-place phase of the movement was larger in the high rather than in the low social status condition. By contrast, the deceleration time of the reach-to-grasp phase and the maximum grasp aperture was smaller in the high rather than in the low social status condition. These results indicated that the hand movement was faster but less carefully shaped in presence of a high, but not of a low social status attendee. This kinematic patterning suggests that being monitored by a high rather than a low social status attendee might lead participants to experience evaluation apprehension that informs the control of motor execution. Motor execution would rely more on feedforward motor control in the presence of a high social status human attendee, vs. feedback motor control, in the presence of a low social status attendee.

Interactional leader-follower sensorimotor communication strategies during repetitive joint actions

Non-verbal communication is the basis of animal interactions. In dyadic leader-follower interactions, leaders master the ability to carve their motor behaviour in order to 'signal' their future actions and internal plans while these signals influence the behaviour of follower partners, who automatically tend to imitate the leader even in complementary interactions. Despite their usefulness, signalling and imitation have a biomechanical cost, and it is unclear how this cost-benefits trade-off is managed during repetitive dyadic interactions that present learnable regularities. We studied signalling and imitation dynamics (indexed by movement kinematics) in pairs of leaders and followers during a repetitive, rule-based, joint action. Trial-by-trial Bayesian model comparison was used to evaluate the relation between signalling, imitation and pair performance. The different models incorporate different hypotheses concerning the factors (past interactions versus online movements) influencing the leader's signalling (or follower's imitation) kinematics. This approach showed that (i) leaders' signalling strategy improves future couple performance, (ii) leaders used the history of past interactions to shape their signalling, (iii) followers' imitative behaviour is more strongly affected by the online movement of the leader. This study elucidates the ways online sensorimotor communication help individuals align their task representations and ultimately improves joint action performance.

Individual differences in reading social intentions from motor deviants

As social animals, it is crucial to understand others’ intention. But is it possible to detect social intention in two actions that have the exact same motor goal? In the present study, we presented participants with video clips of an individual reaching for and grasping an object to either use it (personal trial) or to give his partner the opportunity to use it (social trial). In Experiment 1, the ability of naïve participants to classify correctly social trials through simple observation of short video clips was tested. In addition, detection levels were analyzed as a function of individual scores in psychological questionnaires of motor imagery, visual imagery, and social cognition. Results revealed that the between-participant heterogeneity in the ability to distinguish social from personal actions was predicted by the social skill abilities. A second experiment was then conducted to assess what predictive mechanism could contribute to the detection of social intention. Video clips were sliced and normalized to control for either the reaction times (RTs) or/and the movement times (MTs) of the grasping action. Tested in a second group of participants, results showed that the detection of social intention relies on the variation of both RT and MT that are implicitly perceived in the grasping action. The ability to use implicitly these motor deviants for action-outcome understanding would be the key to intuitive social interaction.

Perceiving what you intend to do from what you do: evidence for embodiment in social interactions

Although action and perception are central components of our interactions with the external world, the most recent experimental investigations also support their implications in the emotional, decision-making, and goal ascription processes in social context. In this article, we review the existing literature supporting this view and highlighting a link between reach-to-grasp motor actions and social communicative processes. First, we discuss the most recent experimental findings showing how the social context subtly influences the execution of object-oriented motor actions. Then, we show that the kinematic characteristics of object-oriented motor actions are modulated by the actor's social intention. Finally, we demonstrate that naïve observers can implicitly take advantage of these kinematic effects for their own motor productions. Considered together, these data are compatible with the embodied cognition framework stating that cognition, and in our case social cognition, is grounded in knowledge associated with past sensory and motor experiences.

Social cues to joint actions: the role of shared goals

In daily life, we do not just move independently from how others move. Rather, the way we move conveys information about our cognitive and affective attitudes toward our conspecifics. However, the implicit social substrate of our movements is not easy to capture and isolate given the complexity of human interactive behaviors. In this perspective article we discuss the crucial conditions for exploring the impact of "interpersonal" cognitive/emotional dimensions on the motor behavior of individuals interacting in realistic contexts. We argue that testing interactions requires one to build up naturalistic and yet controlled scenarios where participants reciprocally adapt their movements in order to achieve an overarching "shared goal." We suggest that a shared goal is what singles out real interactions from situations where two or more individuals contingently but independently act next to each other, and that "interpersonal" socio-emotional dimensions might fail to affect co-agents' behaviors if real interactions are not at place. We report the results of a novel joint-grasping task suitable for exploring how individual sub-goals (i.e., correctly grasping an object) relate to, and depend from, the representation of "shared goals."

The new hybrids: Continuing debates on social perception

I evaluate several attempts to integrate standard theories of social cognition, either theory theory or simulation theory, with aspects of interaction theory, and especially with the concept of direct social perception. I refer to these as new hybrid theories of social cognition. One of the new hybrids accomplishes the integration only by weakening the concept of mindreading or by understanding mindreading as targeting the shared situation rather than the other's mental states. Hybrids that attempt to accommodate the idea of direct perception of mental states grant a phenomenological directness only by maintaining tacit (theory-based) inferences on the subpersonal level. If such inferential processes are thought to be extra-perceptual, then perception is neither sufficient nor direct for an understanding of intentions and emotions. Moreover, insistence on top-down inferential processes trades off against the possibility of plasticity in the perceptual system itself. I suggest that a better model than a hybrid theory would be a pluralist one. A pluralist approach to social cognition would treat theoretical inference, simulation, direct perception, interactive skills, etc. as different strategies. The real challenge is to work out a pluralist account of subpersonal processes.

Social intentions in Parkinson's disease patients: A kinematic study

Dysfunction of the dopaminergic system leads to motor, cognitive and motivational symptoms in brain disorders such as Parkinson's disease (PD). Moreover, the dopaminergic system plays an important role in social interactions. The dopaminergic input to the basal ganglia (BG) thought to integrate social cues during the planning and execution of voluntary movements remains, however, largely unexplored. Since PD provides a model to assess this function in humans, our study aimed to investigate the effects of social intentions on actions in non-demented PDpatients receiving dopamine replacement therapy (Levodopa = l-Dopa) and in neurologically healthy control participants. Patients' ability to modulate motor patterning depending on the intention motivating the action to be performed was evaluated both in "on" (with l-Dopa) and "off" (without l-Dopa) states. Participants were instructed to reach for and to grasp an object; they were then told to hand it to another person (social condition) or to place it on a concave frame (individual condition). A 'passive-observer' condition, which was similar to the 'individual' condition except for the presence of an onlooker who simply observed the scene, was also assessed to exclude the possibility that differences might be due to the presence of another person. Movement kinematics were recorded using a three-dimensional motion analysis system. Study results demonstrated that the controls and the PD patients in an 'on' state adopted different kinematic patterning for the 'social' and the 'individual' conditions; the PD patients in the 'off' state, instead, were unable to kinematically differentiate between the two conditions. These results suggest that l-Dopa treatment has positive effects on translating social intentions into specific motor patterns in PD patients.