Seeing mental states: An experimental strategy for measuring the observability of other minds

Contextual expectations shape the motor coding of movement kinematics during the prediction of observed actions: A TMS study

Contextual information may shape motor resonance and support intention understanding during observation of incomplete, ambiguous actions. It is unclear, however, whether this effect is contingent upon kinematics ambiguity or contextual information is continuously integrated with kinematics to predict the overarching action intention. Moreover, a differentiation between the motor mapping of the intention suggested by context or kinematics has not been clearly demonstrated. In a first action execution phase, 29 participants were asked to perform reaching-to-grasp movements towards big or small food objects with the intention to eat or to move; electromyography from the First Dorsal Interosseous (FDI) and Abductor Digiti Minimi (ADM) was recorded. Depending on object size, the intentions to eat or to move were differently implemented by a whole-hand or a precision grip kinematics, thus qualifying an action-muscle dissociation. Then, in a following action prediction task, the same participants were asked to observe an actor performing the same actions and to predict his/her intention while motor resonance was assessed for the same muscles. Of note, videos were interrupted at early or late action phases, and actions were embedded in contexts pointing toward an eating or a moving intention, congruently or incongruently with kinematics. We found greater involvement of the FDI or ADM in the execution of precision or whole-hand grips, respectively. Crucially, this pattern of activation was mirrored during observation of the same actions in congruent contexts, but it was cancelled out or reversed in the incongruent ones, either when videos were interrupted at either early or long phases of action deployment. Our results extend previous evidence by showing that contextual information shapes motor resonance not only under conditions of perceptual uncertainty but also when more informative kinematics is available.

Action prediction in psychosis

Aberrant motor-sensory predictive functions have been linked to symptoms of psychosis, particularly reduced attenuation of self-generated sensations and misattribution of self-generated actions. Building on the parallels between prediction of self- and other-generated actions, this study aims to investigate whether individuals with psychosis also demonstrate abnormal perceptions and predictions of others' actions. Patients with psychosis and matched controls completed a two-alternative object size discrimination task. In each trial, they observed reaching actions towards a small and a large object, with varying levels of temporal occlusion ranging from 10% to 80% of movement duration. Their task was to predict the size of the object that would be grasped. We employed a novel analytic approach to examine how object size information was encoded and read out across progressive levels of occlusion with single-trial resolution. Patients with psychosis exhibited an overall pattern of reduced and discontinuous evidence integration relative to controls, characterized by a period of null integration up to 20% of movement duration, during which they did not read any size information. Surprisingly, this drop in accuracy in the initial integration period was not accompanied by a reduction in confidence. Difficulties in action prediction were correlated with the severity of negative symptoms and impaired functioning in social relationships.

From neural noise to co-adaptability: Rethinking the multifaceted architecture of motor variability

In the last decade, the source and the functional meaning of motor variability have attracted considerable attention in behavioral and brain sciences. This construct classically combined different levels of description, variable internal robustness or coherence, and multifaceted operational meanings. We provide here a comprehensive review of the literature with the primary aim of building a precise lexicon that goes beyond the generic and monolithic use of motor variability. In the pars destruens of the work, we model three domains of motor variability related to peculiar computational elements that influence fluctuations in motor outputs. Each domain is in turn characterized by multiple sub-domains. We begin with the domains of noise and differentiation. However, the main contribution of our model concerns the domain of adaptability, which refers to variation within the same exact motor representation. In particular, we use the terms learning and (social)fitting to specify the portions of motor variability that depend on our propensity to learn and on our largely constitutive propensity to be influenced by external factors. A particular focus is on motor variability in the context of the sub-domain named co-adaptability. Further groundbreaking challenges arise in the modeling of motor variability. Therefore, in a separate pars construens, we attempt to characterize these challenges, addressing both theoretical and experimental aspects as well as potential clinical implications for neurorehabilitation. All in all, our work suggests that motor variability is neither simply detrimental nor beneficial, and that studying its fluctuations can provide meaningful insights for future research.

Observational learning of atypical biological kinematics in autism

Observing and voluntarily imitating the biological kinematics displayed by a model underpins the acquisition of new motor skills via sensorimotor processes linking perception with action. Differences in voluntary imitation in autism could be related to sensorimotor processing activity during action-observation of biological motion, as well as how sensorimotor integration processing occurs across imitation attempts. Using an observational practice protocol, which minimized the active contribution of the peripheral sensorimotor system, we examined the contribution of sensorimotor processing during action-observation. The data showed that autistic participants imitated both the temporal duration and atypical kinematic profile of the observed movement with a similar level of accuracy as neurotypical participants. These findings suggest the lower-level perception-action processes responsible for encoding biological kinematics during the action-observation phase of imitation are operational in autism. As there was no task-specific engagement of the peripheral sensorimotor system during observational practice, imitation difficulties in autism are most likely underpinned by sensorimotor integration issues related to the processing of efferent and (re)afferent sensorimotor information during trial-to-trial motor execution.

Beyond simple laboratory studies: Developing sophisticated models to study rich behavior

Psychology and neuroscience are concerned with the study of behavior, of internal cognitive processes, and their neural foundations. However, most laboratory studies use constrained experimental settings that greatly limit the range of behaviors that can be expressed. While focusing on restricted settings ensures methodological control, it risks impoverishing the object of study: by restricting behavior, we might miss key aspects of cognitive and neural functions. In this article, we argue that psychology and neuroscience should increasingly adopt innovative experimental designs, measurement methods, analysis techniques and sophisticated computational models to probe rich, ecologically valid forms of behavior, including social behavior. We discuss the challenges of studying rich forms of behavior as well as the novel opportunities offered by state-of-the-art methodologies and new sensing technologies, and we highlight the importance of developing sophisticated formal models. We exemplify our arguments by reviewing some recent streams of research in psychology, neuroscience and other fields (e.g., sports analytics, ethology and robotics) that have addressed rich forms of behavior in a model-based manner. We hope that these "success cases" will encourage psychologists and neuroscientists to extend their toolbox of techniques with sophisticated behavioral models - and to use them to study rich forms of behavior as well as the cognitive and neural processes that they engage.

Neural network-based Bluetooth synchronization of multiple wearable devices

Bluetooth-enabled wearables can be linked to form synchronized networks to provide insightful and representative data that is exceptionally beneficial in healthcare applications. However, synchronization can be affected by inevitable variations in the component's performance from their ideal behavior. Here, we report an application-level solution that embeds a Neural network to analyze and overcome these variations. The neural network examines the timing at each wearable node, recognizes time shifts, and fine-tunes a virtual clock to make them operate in unison and thus achieve synchronization. We demonstrate the integration of multiple Kinematics Detectors to provide synchronized motion capture at a high frequency (200 Hz) that could be used for performing spatial and temporal interpolation in movement assessments. The technique presented in this work is general and independent from the physical layer used, and it can be potentially applied to any wireless communication protocol.

Kinematic priming of action predictions

The ability to anticipate what others will do next is crucial for navigating social, interactive environments. Here, we develop an experimental and analytical framework to measure the implicit readout of prospective intention information from movement kinematics. Using a primed action categorization task, we first demonstrate implicit access to intention information by establishing a novel form of priming, which we term kinematic priming: subtle differences in movement kinematics prime action prediction. Next, using data collected from the same participants in a forced-choice intention discrimination task 1 h later, we quantify single-trial intention readout-the amount of intention information read by individual perceivers in individual kinematic primes-and assess whether it can be used to predict the amount of kinematic priming. We demonstrate that the amount of kinematic priming, as indexed by both response times (RTs) and initial fixations to a given probe, is directly proportional to the amount of intention information read by the individual perceiver at the single-trial level. These results demonstrate that human perceivers have rapid, implicit access to intention information encoded in movement kinematics and highlight the potential of our approach to reveal the computations that permit the readout of this information with single-subject, single-trial resolution.

A glimpse into social perception in light of vitality forms

The American psychoanalyst and developmental psychologist Daniel Stern's idea of vitality forms might suggest a new solution to explain how other minds are intensely expressed in their actions. Vitality forms characterize the expressive style of actions. The effective perception of vitality forms allows people to recognize the affective states and intentions of others in their actions, and could even open the possibility of properties of objects that are indicated by the given actions. Currently, neurophysiological studies present that there might be a neural mirror mechanism in the dorso-central insula (DCI), middle cingulate cortex (MCC), and other related cerebral areas, which serve to preferably perceive and deliver vitality forms of actions. In this article, possible types of vitality forms related to other minds, which have been brought to particular attention in recent years, have been collected and discussed in the following four areas: (1) Vitality forms on understanding non-verbal intention, (2) on understanding verbal intention, (3) vitality forms as grounding social cognition, and (4) as grounding social emotion. These four areas, however, might refer to an entirety of a binary actor-observer communicative landscape. In this review, we try to simplify the analysis by relying on two fundamental dimensions of criteria: first, the idea of vitality forms is conceived as the most basic way of observing subsequent higher-order dimensions of action, that is, understanding intention in the style of action. Thus, in the first two subsections, the relationships between vitality forms and their roles in understanding non-verbal and verbal intention have been discussed. Second, vitality forms could also be conceived as background conditions of all the other mental categories, that is, vitality forms can ground cognition and emotion in a social context. In the second dimension, the existence of social cognition or emotion depends on the existence of the stylistic kinematics of action. A grounding relation is used to distinguish a ground, that is, vitality forms, and its grounded mental categories. As relating with the domain of social perception, in this review, it has been discussed vitality forms possibly could ground social cognition and social emotion, respectively.

Visual estimation of the force applied by another person

As observers, we believe that we can visually estimate the force that another person is applying to a material. However, it is unclear what kind of cues we use to do this. We focused on two types of visual change that occur when actors push an elastic material from above with their fingers: visual shaking and visual indentation depth. The first one relates to a finger/hand shaking, known as an "induced tremor", and the second one relates to material deformation due to the application of force. We found that human observers mainly used visual shaking to estimate the force being applied by another person in a video clip. Overall, the apparent applied force was perceived to be stronger when the level of visual shaking was greater. We also found that observers mainly used visual indentation depth and visual shaking to estimate the softness rating of materials. Overall, the apparent softness was perceived to be greater when the visual indentation depth was larger and the level of visual shaking was lower, which indicates that observers use visual shaking to estimate the force being applied, and that estimated force is then used for an estimation of softness.

Intersecting kinematic encoding and readout of intention in autism

Observers with autism spectrum disorders (ASDs) find it difficult to read intentions from movements. However, the computational bases of these difficulties are unknown. Do these difficulties reflect an intention readout deficit, or are they more likely rooted in kinematic (dis-)similarities between typical and ASD kinematics? We combined motion tracking, psychophysics, and computational analyses to uncover single-trial intention readout computations in typically developing (TD) children (n = 35) and children with ASD (n = 35) who observed actions performed by TD children and children with ASD. Average intention discrimination performance was above chance for TD observers but not for ASD observers. However, single-trial analysis showed that both TD and ASD observers read single-trial variations in movement kinematics. TD readers were better able to identify intention-informative kinematic features during observation of TD actions; conversely, ASD readers were better able to identify intention-informative features during observation of ASD actions. Crucially, while TD observers were generally able to extract the intention information encoded in movement kinematics, those with autism were unable to do so. These results extend existing conceptions of mind reading in ASD by suggesting that intention reading difficulties reflect both an interaction failure, rooted in kinematic dissimilarity between TD and ASD kinematics (at the level of feature identification), and an individual readout deficit (at the level of information extraction), accompanied by an overall reduced sensitivity of intention readout to single-trial variations in movement kinematics.

Kinematic-Based Classification of Social Gestures and Grasping by Humans and Machine Learning Techniques

The affective motion of humans conveys messages that other humans perceive and understand without conventional linguistic processing. This ability to classify human movement into meaningful gestures or segments plays also a critical role in creating social interaction between humans and robots. In the research presented here, grasping and social gesture recognition by humans and four machine learning techniques (k-Nearest Neighbor, Locality-Sensitive Hashing Forest, Random Forest and Support Vector Machine) is assessed by using human classification data as a reference for evaluating the classification performance of machine learning techniques for thirty hand/arm gestures. The gestures are rated according to the extent of grasping motion on one task and the extent to which the same gestures are perceived as social according to another task. The results indicate that humans clearly rate differently according to the two different tasks. The machine learning techniques provide a similar classification of the actions according to grasping kinematics and social quality. Furthermore, there is a strong association between gesture kinematics and judgments of grasping and the social quality of the hand/arm gestures. Our results support previous research on intention-from-movement understanding that demonstrates the reliance on kinematic information for perceiving the social aspects and intentions in different grasping actions as well as communicative point-light actions.

Costs and benefits of communicating vigor

Why do we run toward people we love, but only walk toward others? One reason is to let them know we love them. In this commentary, we elaborate on how subjective utility information encoded in vigor is read out by others. We consider the potential implications for understanding and modeling the link between movements and decisions in social environments.

Evaluating the relative contributions of copying and reconstruction processes in cultural transmission episodes

The ability to transmit information between individuals through social learning is a foundational component of cultural evolution. However, how this transmission occurs is still debated. On the one hand, the copying account draws parallels with biological mechanisms for genetic inheritance, arguing that learners copy what they observe and novel variations occur through random copying errors. On the other hand, the reconstruction account claims that, rather than directly copying behaviour, learners reconstruct the information that they believe to be most relevant on the basis of pragmatic inference, environmental and contextual cues. Distinguishing these two accounts empirically is difficult based on data from typical transmission chain studies because the predictions they generate frequently overlap. In this study we present a methodological approach that generates different predictions of these accounts by manipulating the task context between model and learner in a transmission episode. We then report an empirical proof-of-concept that applies this approach. The results show that, when a model introduces context-dependent embedded signals to their actions that are not intended to be transmitted, it is possible to empirically distinguish between competing predictions made by these two accounts. Our approach can therefore serve to understand the underlying cognitive mechanisms at play in cultural transmission and can make important contributions to the debate between preservative and reconstructive schools of thought.

Musical Interaction Reveals Music as Embodied Language

Life and social sciences often focus on the social nature of music (and language alike). In biology, for example, the three main evolutionary hypotheses about music (i.e., sexual selection, parent-infant bond, and group cohesion) stress its intrinsically social character (Honing et al., 2015). Neurobiology thereby has investigated the neuronal and hormonal underpinnings of musicality for more than two decades (Chanda and Levitin, 2013; Salimpoor et al., 2015; Mehr et al., 2019). In line with these approaches, the present paper aims to suggest that the proper way to capture the social interactive nature of music (and, before it, musicality), is to conceive of it as an embodied language, rooted in culturally adapted brain structures (Clarke et al., 2015; D'Ausilio et al., 2015). This proposal heeds Ian Cross' call for an investigation of music as an "interactive communicative process" rather than "a manifestation of patterns in sound" (Cross, 2014), with an emphasis on its embodied and predictive (coding) aspects (Clark, 2016; Leman, 2016; Koelsch et al., 2019). In the present paper our goal is: (i) to propose a framework of music as embodied language based on a review of the major concepts that define joint musical action, with a particular emphasis on embodied music cognition and predictive processing, along with some relevant neural underpinnings; (ii) to summarize three experiments conducted in our laboratories (and recently published), which provide evidence for, and can be interpreted according to, the new conceptual framework. In doing so, we draw on both cognitive musicology and neuroscience to outline a comprehensive framework of musical interaction, exploring several aspects of making music in dyads, from a very basic proto-musical action, like tapping, to more sophisticated contexts, like playing a jazz standard and singing a hocket melody. Our framework combines embodied and predictive features, revolving around the concept of joint agency (Pacherie, 2012; Keller et al., 2016; Bolt and Loehr, 2017). If social interaction is the "default mode" by which human brains communicate with their environment (Hari et al., 2015), music and musicality conceived of as an embodied language may arguably provide a route toward its navigation.

Adults Do Not Distinguish Action Intentions Based on Movement Kinematics Presented in Naturalistic Settings

Predicting others’ actions is an essential part of acting in the social world. Action kinematics have been proposed to be a cue about others’ intentions. It is still an open question as to whether adults can use kinematic information in naturalistic settings when presented as a part of a richer visual scene than previously examined. We investigated adults’ intention perceptions from kinematics using naturalistic stimuli in two experiments. In experiment 1, thirty participants watched grasp-to-drink and grasp-to-place movements and identified the movement intention (to drink or to place), whilst their mouth-opening muscle activity was measured with electromyography (EMG) to examine participants’ motor simulation of the observed actions. We found anecdotal evidence that participants could correctly identify the intentions from the action kinematics, although we found no evidence for increased activation of their mylohyoid muscle during the observation of grasp-to-drink compared to grasp-to-place actions. In pre-registered experiment 2, fifty participants completed the same task online. With the increased statistical power, we found strong evidence that participants were not able to discriminate intentions based on movement kinematics. Together, our findings suggest that the role of action kinematics in intention perception is more complex than previously assumed. Although previous research indicates that under certain circumstances observers can perceive and act upon intention-specific kinematic information, perceptual differences in everyday scenes or the observers’ ability to use kinematic information in more naturalistic scenes seems limited.

Understanding joint action: Current theoretical and empirical approaches

Joint actions are omnipresent, ranging from a handshake between two people to the coordination of groups of people playing in an orchestra. We are highly skilled at coordinating our actions with those of others to reach common goals and rely on this ability throughout our daily lives. What are the social, cognitive and neural processes underlying this ability? How do others around us influence our task representations? How does joint action influence interpersonal interactions? How do language and gesture support joint action? What differentiates joint action from individual action? This article forms an introductory editorial to the field of joint action. It accompanies contributions to the special issue entitled "Current Issues in Joint Action Research". The issue brings together conceptual and empirical approaches on different topics, ranging from lower-level issues such as the link between perception and joint action, to higher-level issues such as language as a form of joint action.

Proactive gaze is present during biological and non-biological motion observation

Others' action observation activates in the observer a coordinated hand-eye motor program, covert for the hand (i.e. motor resonance), and overt for the eye (i.e. proactive gaze), similar to that of the observed agent. The biological motion hypothesis of action anticipation claims that proactive gaze occurs only in the presence of biological motion, and that kinematic information is sufficient to determine the anticipation process. The results of the present study did not support the biological motion hypothesis of action anticipation. Specifically, proactive gaze was present during observation of both a biological accelerated-decelerated motion and a non-biological constant velocity motion (Experiment 1), in the presence of a barrier able to restrict differences between the two kinematics to the motion profile of individual markers prior to contact (Experiment 2), but only if an object was present at the end point of the movement trajectory (Experiment 3). Furthermore, proactive gaze was found independently of the presence of end effects temporally congruent with the instant in which the movement stopped (Experiments 4, and 5). We propose that the involvement of the observer's motor system is not restricted to when the agent moves with natural kinematics, and it is mandatory whenever the presence of an agent or a goal is evident, regardless of physical appearance, natural kinematics, and the possibility to identify the action behind the stimulus.

The communicative advantage: how kinematic signaling supports semantic comprehension

Humans are unique in their ability to communicate information through representational gestures which visually simulate an action (eg. moving hands as if opening a jar). Previous research indicates that the intention to communicate modulates the kinematics (e.g., velocity, size) of such gestures. If and how this modulation influences addressees' comprehension of gestures have not been investigated. Here we ask whether communicative kinematic modulation enhances semantic comprehension (i.e., identification) of gestures. We additionally investigate whether any comprehension advantage is due to enhanced early identification or late identification. Participants (n = 20) watched videos of representational gestures produced in a more- (n = 60) or less-communicative (n = 60) context and performed a forced-choice recognition task. We tested the isolated role of kinematics by removing visibility of actor's faces in Experiment I, and by reducing the stimuli to stick-light figures in Experiment II. Three video lengths were used to disentangle early identification from late identification. Accuracy and response time quantified main effects. Kinematic modulation was tested for correlations with task performance. We found higher gesture identification performance in more- compared to less-communicative gestures. However, early identification was only enhanced within a full visual context, while late identification occurred even when viewing isolated kinematics. Additionally, temporally segmented acts with more post-stroke holds were associated with higher accuracy. Our results demonstrate that communicative signaling, interacting with other visual cues, generally supports gesture identification, while kinematic modulation specifically enhances late identification in the absence of other cues. Results provide insights into mutual understanding processes as well as creating artificial communicative agents.

Transient Disruption of the Inferior Parietal Lobule Impairs the Ability to Attribute Intention to Action

Although it is well established that fronto-parietal regions are active during action observation, whether they play a causal role in the ability to infer others' intentions from visual kinematics remains undetermined. In the experiments reported here, we combined offline continuous theta burst stimulation (cTBS) with computational modeling to reveal and causally probe single-trial computations in the inferior parietal lobule (IPL) and inferior frontal gyrus (IFG). Participants received cTBS over the left anterior IPL and the left IFG pars orbitalis in separate sessions before completing an intention discrimination task (discriminate intention of observed reach-to-grasp acts) or a kinematic discrimination task unrelated to intention (discriminate peak wrist height of the same acts). We targeted intention-sensitive regions whose fMRI activity, recorded when observing the same reach-to-grasp acts, could accurately discriminate intention. We found that transient disruption of activity of the left IPL, but not the IFG, impaired the observer's ability to attribute intention to action. Kinematic discrimination unrelated to intention, in contrast, was largely unaffected. Computational analyses of how encoding (mapping of intention to movement kinematics) and readout (mapping of kinematics to intention choices) intersect at the single-trial level revealed that IPL cTBS did not diminish the overall sensitivity of intention readout to movement kinematics. Rather, it selectively misaligned intention readout with respect to encoding, deteriorating mapping from informative kinematic features to intention choices. These results provide causal evidence of how the left anterior IPL computes mapping from kinematics to intentions.

Neural dynamics of grip and goal integration during the processing of others' actions with objects: An ERP study

Recent behavioural evidence suggests that when processing others’ actions, motor acts and goal-related information both contribute to action recognition. Yet the neuronal mechanisms underlying the dynamic integration of the two action dimensions remain unclear. This study aims to elucidate the ERP components underlying the processing and integration of grip and goal-related information. The electrophysiological activity of 28 adults was recorded during the processing of object-directed action photographs (e.g., writing with pencil) containing either grip violations (e.g. upright pencil grasped with atypical-grip), goal violations (e.g., upside-down pencil grasped with typical-grip), both grip and goal violations (e.g., upside-down pencil grasped with atypical-grip), or no violations. Participants judged whether actions were overall typical or not according to object typical use. Brain activity was sensitive to the congruency between grip and goal information on the N400, reflecting the semantic integration between the two dimensions. On earlier components, brain activity was affected by grip and goal typicality independently. Critically, goal typicality but not grip typicality affected brain activity on the N300, supporting an earlier role of goal-related representations in action recognition. Findings provide new insights on the neural temporal dynamics of the integration of motor acts and goal-related information during the processing of others’ actions.

Seeing the Unexpected: How Brains Read Communicative Intent through Kinematics

Social interaction requires us to recognize subtle cues in behavior, such as kinematic differences in actions and gestures produced with different social intentions. Neuroscientific studies indicate that the putative mirror neuron system (pMNS) in the premotor cortex and mentalizing system (MS) in the medial prefrontal cortex support inferences about contextually unusual actions. However, little is known regarding the brain dynamics of these systems when viewing communicatively exaggerated kinematics. In an event-related functional magnetic resonance imaging experiment, 28 participants viewed stick-light videos of pantomime gestures, recorded in a previous study, which contained varying degrees of communicative exaggeration. Participants made either social or nonsocial classifications of the videos. Using participant responses and pantomime kinematics, we modeled the probability of each video being classified as communicative. Interregion connectivity and activity were modulated by kinematic exaggeration, depending on the task. In the Social Task, communicativeness of the gesture increased activation of several pMNS and MS regions and modulated top-down coupling from the MS to the pMNS, but engagement of the pMNS and MS was not found in the nonsocial task. Our results suggest that expectation violations can be a key cue for inferring communicative intention, extending previous findings from wholly unexpected actions to more subtle social signaling.

Instructions to attend to an observed action increase imitation in autistic adults

This study investigated whether reduced visual attention to an observed action might account for altered imitation in autistic adults. A total of 22 autistic and 22 non-autistic adults observed and then imitated videos of a hand producing sequences of movements that differed in vertical elevation while their hand and eye movements were recorded. Participants first performed a block of imitation trials with general instructions to imitate the action. They then performed a second block with explicit instructions to attend closely to the characteristics of the movement. Imitation was quantified according to how much participants modulated their movement between the different heights of the observed movements. In the general instruction condition, the autistic group modulated their movements significantly less compared to the non-autistic group. However, following instructions to attend to the movement, the autistic group showed equivalent imitation modulation to the non-autistic group. Eye movement recording showed that the autistic group spent significantly less time looking at the hand movement for both instruction conditions. These findings show that visual attention contributes to altered voluntary imitation in autistic individuals and have implications for therapies involving imitation as well as for autistic people's ability to understand the actions of others.

Implicit preference towards slim bodies and weight-stigma modulate the understanding of observed familiar actions

Mounting research evidence suggests that motor resonance (MR, i.e., the mapping of others' actions onto one's own motor repertoire) can be influenced by diverse factors related to individual differences. However, no evidence has been reported so far on the effects of physical appearance and negative attitudes toward obesity to the mechanism of MR. Thirty-six participants (18 normal-weight and 18 overweight) performed a weight discrimination task, in which they were observing amateur actors reaching and grasping a light or heavy cube with or without deception (true vs. fake actions). At the end of each video clip, participants were instructed to indicate the correct cube size (light or heavy). Importantly, body similarity between observers and actors was manipulated by presenting videos of normal-weight or overweight actors. Fat phobic attitudes and automatic preference for normal-weight than obese people were also examined. Signal detection analysis (d') on the acquired accuracy data has shown that both normal- and overweight participants were able to better discriminate truthful actions when performed by the normal-weight as compared to overweight actors. Furthermore, this finding was negatively correlated with increased scores of fat phobic attitudes in both groups. Hence, for the first time, we provide experimental evidence of action simulation being modulated by an implicit visual sensitivity towards slim bodies.

Gaze and body cues interplay during interactive requests

Although observing other's gaze and body movements provides a crucial source of information to successfully interact with other people, it remains unclear whether observers weigh differently these cues and whether the convergence of gaze and body's directions determines facilitation effects. Here we aim to shed more light on this issue by testing the reliance upon these cues from both a behavioral and a neurophysiological perspective in a social interactive context. In Experiment 1, we manipulated the convergence between the direction of an actor's upper limb movement and gaze direction while he attempts to socially interact with the participants observing the scene. We determined the direction of gaze as well as the duration of participants' ocular fixations during the observation of the scene. In Experiment 2, we measured and correlated the effect of the body/gaze manipulation on corticospinal excitability and on the readiness to interact-a disposition to engage in social situations. Eye-tracking data revealed that participants fixated chiefly the actor's head when his hand and gaze directions were divergent. Possibly a strategy to disambiguate the scene. Whereas participants mainly fixated the actor's hand when he performed an interactive request toward the participants. From a neurophysiological point of view, the more participants felt involved in the interaction, the lower was motor preparation in the muscle potentially needed to fulfill the actor's request. We contend that social contexts are more likely to elicit motor preparation compared to non-social ones, and that muscular inhibition is a necessary mechanism in order to prevent unwanted overt reactions during action observation tasks.

Predicting Intentions from Motion: The Subject-Adversarial Adaptation Approach

This paper aims at investigating the action prediction problem from a pure kinematic perspective. Specifically, we address the problem of recognizing future actions, indeed human intentions, underlying a same initial (and apparently unrelated) motor act. This study is inspired by neuroscientific findings asserting that motor acts at the very onset are embedding information about the intention with which are performed, even when different intentions originate from a same class of movements. To demonstrate this claim in computational and empirical terms, we designed an ad hoc experiment and built a new 3D and 2D dataset where, in both training and testing, we analyze a same class of grasping movements underlying different intentions. We investigate how much the intention discriminants generalize across subjects, discovering that each subject tends to affect the prediction by his/her own bias. Inspired by the domain adaptation problem, we propose to interpret each subject as a domain, leading to a novel subject adversarial paradigm. The proposed approach favorably copes with our new problem, boosting the considered baseline features encoding 2D and 3D information and which do not exploit the subject information.

What Can You See? Identifying Cues on Internal States From the Movements of Natural Social Interactions

In recent years, the field of Human-Robot Interaction (HRI) has seen an increasing demand for technologies that can recognize and adapt to human behaviors and internal states (e.g., emotions and intentions). Psychological research suggests that human movements are important for inferring internal states. There is, however, a need to better understand what kind of information can be extracted from movement data, particularly in unconstrained, natural interactions. The present study examines which internal states and social constructs humans identify from movement in naturalistic social interactions. Participants either viewed clips of the full scene or processed versions of it displaying 2D positional data. Then, they were asked to fill out questionnaires assessing their social perception of the viewed material. We analyzed whether the full scene clips were more informative than the 2D positional data clips. First, we calculated the inter-rater agreement between participants in both conditions. Then, we employed machine learning classifiers to predict the internal states of the individuals in the videos based on the ratings obtained. Although we found a higher inter-rater agreement for full scenes compared to positional data, the level of agreement in the latter case was still above chance, thus demonstrating that the internal states and social constructs under study were identifiable in both conditions. A factor analysis run on participants' responses showed that participants identified the constructs interaction imbalance, interaction valence and engagement regardless of video condition. The machine learning classifiers achieved a similar performance in both conditions, again supporting the idea that movement alone carries relevant information. Overall, our results suggest it is reasonable to expect a machine learning algorithm, and consequently a robot, to successfully decode and classify a range of internal states and social constructs using low-dimensional data (such as the movements and poses of observed individuals) as input.

Pupils say more than a thousand words: Pupil size reflects how observed actions are interpreted

Humans attend to others' facial expressions and body language to better understand their emotions and predict goals and intentions. The eyes and its pupils reveal important social information. Because pupil size is beyond voluntary control yet reflective of a range of cognitive and affective processes, pupils in principal have the potential to convey whether others' actions are interpreted correctly or not. Here, we measured pupil size while participants observed video-clips showing reach-to-grasp arm movements. Expressors in the video-clips were playing a board game and moved a dowel to a new position. Participants' task was to decide whether the dowel was repositioned with the intention to be followed up by another move of the same expressor (personal intention) or whether the arm movement carried the implicit message that expressor's turn was over (social intention). Replicating earlier findings, results showed that participants recognized expressors' intentions on the basis of their arm kinematics. Results further showed that participants' pupil size was larger when observing actions reflecting personal compared to social intentions. Most interestingly, before participants indicated how they interpreted the observed actions by choosing to press one of two keys (corresponding to the personal or social intention), their pupils within a split second, had already given away how they interpreted the expressor's movement. In sum, this study underscores the importance of nonverbal behavior in helping social messages get across quickly. Revealing how actions are interpreted, pupils may provide additional feedback for effective social interactions.

Re-enacting the Bodily Self on Stage: Embodied Cognition Meets Psychoanalysis

The embodied approach to cognition consists in a range of theoretical proposals sharing the idea that our concepts are constitutively shaped by the physical and social constraints of our body and environment. Still far from a mutually enriching interplay, in recent years embodied and psychoanalytic approaches are converging on similar constructs as the ones of intersubjectivity, bodily self, and affective quality of verbal communication. Some efforts to cope with the sentient subject were already present in classical cognitivism: having expunged desires and conflicts from the cognitive harmony, bodily emotions re-emerged but only as a noisy dynamic friction. In contrast, the new, neural, embodied cognitive science with its focus on bodily effects/affects has enabled a dialogue between neuro-cognitive perspectives and clinic-psychological ones, through shared conceptual frameworks. I will address crucial issues that should be faced on this reconciling path. With reference to two kinds of contemporary addictions - internet addiction disorder and eating disorders - I will introduce a possible therapeutic approach that is built upon the core role of the acting-sentient bodily self in a dynamic-social and affective environment. In Psychoanalytic Psychodrama, the spontaneous re-enactment of a past (socially and physically constrained) experience is actualized by means of the other, the Auxiliary Ego. This allows homeostatic and social-emotional affects, i.e., drives and instincts, to be re-experienced by the agent, the Protagonist, in a safe scenario. The director-psychoanalyst smoothly traces back this simulation to the motivated, and constrained, early proximal embodied interactions with significant others, and to the related instinctual conflicting aims. The psychoanalytic reframing of classical psychodrama does not merely exploit its original cathartic function, rather stands out for exploring the interpersonal constitution of the self, through an actual "re-somatization" of psychoanalytic therapy. Unspoken/unspeakable feelings pop up on stage: the strength of this treatment mainly rests on re-establishing the priority of the embodied Self over the narrative Self. By pointing out the possible conflicts between these two selves, this method can broaden the embodied cognition perspective. The psychodramatic approach will be briefly discussed in light of connectionist models, to finally address linguistic and methodological pivotal issues.

The Social Brain Automatically Predicts Others' Future Mental States

Social life requires people to predict the future: people must anticipate others' thoughts, feelings, and actions to interact with them successfully. The theory of predictive coding suggests that the social brain may meet this need by automatically predicting others' social futures. If so, when representing others' current mental state, the brain should already start representing their future states. To test this hypothesis, we used fMRI to measure female and male human participants' neural representations of mental states. Representational similarity analysis revealed that neural patterns associated with mental states currently under consideration resembled patterns of likely future states more so than patterns of unlikely future states. This effect manifested in activity across the social brain network and in medial prefrontal cortex in particular. Repetition suppression analysis also supported the social predictive coding hypothesis: considering mental states presented in predictable sequences reduced activity in the precuneus relative to unpredictable sequences. In addition to demonstrating that the brain makes automatic predictions of others' social futures, the results also demonstrate that the brain leverages a 3D representational space to make these predictions. Proximity between mental states on the psychological dimensions of rationality, social impact, and valence explained much of the association between state-specific neural pattern similarity and state transition likelihood. Together, these findings suggest that the way the brain represents the social present gives people an automatic glimpse of the social future.SIGNIFICANCE STATEMENT When you see a ball in flight, your brain calculates, not just its static visual features such as size and shape, but also predicts its future trajectory. Here, we investigated whether the same might hold true in the social world: when we see someone flying into a rage, does our brain automatically predict their social trajectory? In this study, we scanned participants' brain activity while they judged others' mental states. We found that neural activity associated with a given state resembled activity associated with likely future states. Additionally, unpredictable sequences of states evoked more brain activity than predictable sequences, consistent with monitoring for, and updating from, prediction errors. These results suggest that the social brain automatically predicts others' future mental states.

The way others move can influence what we choose

Whether pointing at a menu item or rifling through a clothes rack, when we choose we often move. We investigated whether people's tendency to copy the movements of others could influence their choices. Participants saw pairs of pictures in private and indicated which one they preferred. They then entered a virtual art gallery and saw the same picture pairs in the presence of a virtual character. Having observed the virtual character point to indicate her preference with either a high or low movement trajectory, participants indicated their preference. There was either an anatomical (same movement, same choice) or spatial correspondence (same movement, different choice) between the participant's pictures and those of the virtual character. We found that participants copied the movement made by the virtual character rather than her action goal (i.e., her choice of picture). This resulted in a shift towards the virtual character's preferences in the anatomical condition but away from her preferences in the spatial condition. This effect was driven by the observation of the virtual character's high pointing movements. In a further experiment, we did not find any significant differences in imitation behaviour in autism, although autistic participants were less consistent in their choices. Our findings demonstrate that we are not only influenced by other's choices but also the types of movements others make to indicate those choices.

Movement kinematics drive chain selection toward intention detection

Estimation of intentions from the observation of other people’s actions has been proposed to rely on the same motor chain organization supporting the execution of intentional actions. However, the nature of the mechanism by which a specific neuronal chain is selected among possible alternatives during action observation remains obscure. Our study shows that in absence of discriminative contextual cues, subtle changes in the kinematics of the observed action inform mapping to the most probable chain. These results shed light on the importance of kinematics for the attribution of intentions to actions.

The ability to understand intentions based on another’s movements is crucial for human interaction. This ability has been ascribed to the so-called motor chaining mechanism: anytime a motor chain is activated (e.g., grasp-to-drink), the observer attributes to the agent the corresponding intention (i.e., to drink) from the first motor act (i.e., the grasp). However, the mechanisms by which a specific chain is selected in the observer remain poorly understood. In the current study, we investigate the possibility that in the absence of discriminative contextual cues, slight kinematic variations in the observed grasp inform mapping to the most probable chain. Chaining of motor acts predicts that, in a sequential grasping task (e.g., grasp-to-drink), electromyographic (EMG) components that are required for the final act [e.g., the mouth-opening mylohyoid (MH) muscle] show anticipatory activation. To test this prediction, we used MH EMG, transcranial magnetic stimulation (TMS; MH motor-evoked potentials), and predictive models of movement kinematics to measure the level and timing of MH activation during the execution (Experiment 1) and the observation (Experiment 2) of reach-to-grasp actions. We found that MH-related corticobulbar excitability during grasping observation varied as a function of the goal (to drink or to pour) and the kinematics of the observed grasp. These results show that subtle changes in movement kinematics drive the selection of the most probable motor chain, allowing the observer to link an observed act to the agent’s intention.

Look at Me: Early Gaze Engagement Enhances Corticospinal Excitability During Action Observation

Direct gaze is a powerful social cue able to capture the onlooker’s attention. Beside gaze, head and limb movements as well can provide relevant sources of information for social interaction. This study investigated the joint role of direct gaze and hand gestures on onlookers corticospinal excitability (CE). In two experiments we manipulated the temporal and spatial aspects of observed gaze and hand behavior to assess their role in affecting motor preparation. To do this, transcranial magnetic stimulation (TMS) on the primary motor cortex (M1) coupled with electromyography (EMG) recording was used in two experiments. In the crucial manipulation, we showed to participants four video clips of an actor who initially displayed eye contact while starting a social request gesture, and then completed the action while directing his gaze toward a salient object for the interaction. This way, the observed gaze potentially expressed the intention to interact. Eye tracking data confirmed that gaze manipulation was effective in drawing observers’ attention to the actor’s hand gesture. In the attempt to reveal possible time-locked modulations, we tracked CE at the onset and offset of the request gesture. Neurophysiological results showed an early CE modulation when the actor was about to start the request gesture looking straight to the participants, compared to when his gaze was averted from the gesture. This effect was time-locked to the kinematics of the actor’s arm movement. Overall, data from the two experiments seem to indicate that the joint contribution of direct gaze and precocious kinematic information, gained while a request gesture is on the verge of beginning, increases the subjective experience of involvement and allows observers to prepare for an appropriate social interaction. On the contrary, the separation of gaze cues and body kinematics can have adverse effects on social motor preparation. CE is highly susceptible to biological cues, such as averted gaze, which is able to automatically capture and divert observer’s attention. This point to the existence of heuristics based on early action and gaze cues that would allow observers to interact appropriately.

How attention gates social interactions

Social interactions are at the core of social life. However, humans selectively choose their exchange partners and do not engage in all available opportunities for social encounters. In this review, we argue that attentional systems play an important role in guiding the selection of social interactions. Supported by both classic and emerging literature, we identify and characterize the three core processes-perception, interpretation, and evaluation-that interact with attentional systems to modulate selective responses to social environments. Perceptual processes facilitate attentional prioritization of social cues. Interpretative processes link attention with understanding of cues' social meanings and agents' mental states. Evaluative processes determine the perceived value of the source of social information. The interplay between attention and these three routes of processing places attention in a powerful role to manage the selection of the vast amount of social information that individuals encounter on a daily basis and, in turn, gate the selection of social interactions.