Neural correlates of observing joint actions with shared intentions

Movement synchrony among dance performers predicts brain synchrony among dance spectators

Performing dance is an intrinsically social art form where at least one person moves while another person watches. Dancing in groups promotes social bonding, but how does group dance affect the people watching? A group of dancers and dance novices watched a 30 min dance video individually in an fMRI scanner. In a follow-up behavioural study, the same people watched the video again and provided continuous enjoyment ratings. Firstly, we computed cross-recurrence of continuous enjoyment ratings and inter-subject correlations (ISCs) in fMRI separately for both groups, and with the choreographer of the dance work. At both behavioural and neural levels, dancers responded more similarly to each other than novices. ISCs among dancers extended beyond brain areas involved in audio-visual integration and sensory areas of human movement perception into motor areas, suggesting greater sensorimotor familiarity with the observed dance movements in the expert group. Secondly, we show that dancers' brain activations and continuous ratings are more similar to the choreographer's ratings in keeping with sharing an aesthetic and artistic perspective when viewing the dance. Thirdly, we show that movement synchrony among performers is the best predictor of brain synchrony among both expert and novice spectators. This is consistent with the idea that changes in emergent movement synchrony are a key aesthetic feature of performing dance. Finally, ISCs across perceptual and motor brain areas were primarily driven by movement acceleration and synchrony, whereas ISCs in orbital and pre-frontal brain areas were overall weaker and better explained by the continuous enjoyment ratings of each group. Our findings provide strong evidence that the aesthetic appreciation of dance involves a common experience between dance spectators and the choreographer. Moreover, the similarity of brain activations and of enjoyment increases with shared knowledge of - and practice in - the artform that is being experienced, in this case contemporary performing dance.

Evaluations of dyadic synchrony: observers' traits influence estimation and enjoyment of synchrony in mirror-game movements

While evidence abounds that motor synchrony is a powerful form of 'social glue' for those involved, we have yet to understand how observers perceive motor synchrony: can observers estimate the degree of synchrony accurately? Is synchrony aesthetically pleasing? In two preregistered experiments (n = 161 each), we assess how accurately observers can estimate the degree of synchrony in dyads playing the mirror game, and how much observers enjoy watching these movements. We further assess whether accuracy and enjoyment are influenced by individual differences in self-reported embodied expertise (ability to reproduce movements, body awareness, body competence), psychosocial resources (extraversion, self-esteem), or social competencies (empathy, autistic traits), while objectively controlling for the degree of measured synchrony and complexity. The data revealed that observers' estimated synchrony with poor accuracy, showing a tendency to underestimate the level of synchrony. Accuracy for low synchrony improved with increasing body competence, while accuracy for high synchrony improved with increasing autistic traits. Observers' enjoyment of dyadic movements correlated positively with the degree of measured synchrony, the predictability of the movements, and the observer's empathy. Furthermore, very low enjoyment was associated with increased body perception. Our findings indicate that accuracy in perceiving synchrony is closely linked to embodiment, while aesthetic evaluations of action hinge on individual differences.

Socially evaluative contexts facilitate mentalizing

Our ability to understand others' minds stands at the foundation of human learning, communication, cooperation, and social life more broadly. Although humans' ability to mentalize has been well-studied throughout the cognitive sciences, little attention has been paid to whether and how mentalizing differs across contexts. Classic developmental studies have examined mentalizing within minimally social contexts, in which a single agent seeks a neutral inanimate object. Such object-directed acts may be common, but they are typically consequential only to the object-seeking agent themselves. Here, we review a host of indirect evidence suggesting that contexts providing the opportunity to evaluate prospective social partners may facilitate mentalizing across development. Our article calls on cognitive scientists to study mentalizing in contexts where it counts.

EEG Frequency Tagging Reveals the Integration of Form and Motion Cues into the Perception of Group Movement

The human brain has dedicated mechanisms for processing other people’s movements. Previous research has revealed how these mechanisms contribute to perceiving the movements of individuals but has left open how we perceive groups of people moving together. Across three experiments, we test whether movement perception depends on the spatiotemporal relationships among the movements of multiple agents. In Experiment 1, we combine EEG frequency tagging with apparent human motion and show that posture and movement perception can be dissociated at harmonically related frequencies of stimulus presentation. We then show that movement but not posture processing is enhanced when observing multiple agents move in synchrony. Movement processing was strongest for fluently moving synchronous groups (Experiment 2) and was perturbed by inversion (Experiment 3). Our findings suggest that processing group movement relies on binding body postures into movements and individual movements into groups. Enhanced perceptual processing of movement synchrony may form the basis for higher order social phenomena such as group alignment and its social consequences.

Frontoparietal, Cerebellum Network Codes for Accurate Intention Prediction in Altered Perceptual Conditions

Integrating and predicting the intentions and actions of others are critical components of social interactions, but the behavioral and neural bases of such mechanisms under altered perceptual conditions are poorly understood. In the present study, we recruited expert violinists and age-matched controls with no musical training and asked them to evaluate simplified dynamic stimuli of violinists playing in a piano or forte communicative intent while undergoing functional magnetic resonance imaging. We show that expertise is needed to successfully understand and evaluate communicative intentions in spatially and temporally altered visual representations of musical performance. Frontoparietal regions-such as the dorsolateral prefrontal cortex and the inferior parietal lobule and sulcus-and various subregions of the cerebellum-such as cerebellar lobules I-IV, V, VI, VIIb, VIIIa, X-a re recruited in the process. Functional connectivity between these brain areas reveals widespread organization, particularly in the dorsolateral prefrontal cortex, inferior frontal gyrus, inferior parietal sulcus, and in the cerebellum. This network may be essential to successfully assess communicative intent in ambiguous or complex visual scenes.

From Gaze Perception to Social Cognition: The Shared-Attention System

When two people look at the same object in the environment and are aware of each other's attentional state, they find themselves in a shared-attention episode. This can occur through intentional or incidental signaling and, in either case, causes an exchange of information between the two parties about the environment and each other's mental states. In this article, we give an overview of what is known about the building blocks of shared attention (gaze perception and joint attention) and focus on bringing to bear new findings on the initiation of shared attention that complement knowledge about gaze following and incorporate new insights from research into the sense of agency. We also present a neurocognitive model, incorporating first-, second-, and third-order social cognitive processes (the shared-attention system, or SAS), building on previous models and approaches. The SAS model aims to encompass perceptual, cognitive, and affective processes that contribute to and follow on from the establishment of shared attention. These processes include fundamental components of social cognition such as reward, affective evaluation, agency, empathy, and theory of mind.

The neural and cognitive time course of intention reasoning: Electrophysiological evidence from ERPs

Intention is a typical mental state in the theory of mind. However, to date, there have been theoretical debates on the conceptual structure of intention. The neural and cognitive time course of intention reasoning remains unclear. The present event-related potential (ERP) study had two purposes: first, to investigate the neural correlates of intention reasoning based on a differentiated conceptual structure distinguishing desire and intention; second, to investigate the neural basis of intention reasoning for different agents. Thus, we compared the neural activity elicited by intention reasoning for self and for others when the intention matched or mismatched the desire of the agent. The results revealed that three ERP components distinguished among different types of intention reasoning. A negative-going ERP deflection with right frontal distribution between 400 and 500 ms might reflect the cognitive conflict involved in intention reasoning, a right frontal late positive component might be associated with the categorisation of agents, and a centro-parietal late slow wave might indicate the conceptual mental operations associated with decoupling mechanisms in intention processing. These findings implied the neural and cognitive time course of intention reasoning and provided neural evidence for the differentiated conception of intention.

Seeing synchrony: A replication of the effects of task-irrelevant social information on perceptions of interpersonal coordination

The display of synchronous behaviour can be both an engaging spectacle and a source of important social information. When understood as a dynamical system, interpersonal synchrony has specific kinematic qualities that have been shown to shape social perceptions. Little research, however, has examined the converse relationship - are perceptions of the kinematics of interpersonal synchrony influenced by socially relevant, but task-irrelevant, information? To provide further insight to this question we conducted a pre-registered replication of Lumsden, Miles, and Macrae (2012). Participants (n = 191) rated the level of coordination present in dyads made up of individuals with either similar or dissimilar skin tones. Faithful to the original study, the results indicated that perceivers were sensitive to differing levels of interpersonal coordination, and judged dissimilar dyads to be less coordinated than dyads with a similar skin tone despite actual coordination levels being objectively equivalent. Extending Lumsden et al., the results also revealed a negative relationship between subclinical variation in social anxiety and the degree of perceived coordination. This work is discussed with respect to the perceptual and social factors that underlie judgements of interpersonal coordination.

The effects of social deprivation on adolescent development and mental health

Adolescence (the stage between 10 and 24 years) is a period of life characterised by heightened sensitivity to social stimuli and the increased need for peer interaction. The physical distancing measures mandated globally to contain the spread of COVID-19 are radically reducing adolescents' opportunities to engage in face-to-face social contact outside their household. In this interdisciplinary Viewpoint, we describe literature from a variety of domains that highlight how social deprivation in adolescence might have far-reaching consequences. Human studies have shown the importance of peer acceptance and peer influence in adolescence. Animal research has shown that social deprivation and isolation have unique effects on brain and behaviour in adolescence compared with other stages of life. However, the decrease in adolescent face-to-face contact might be less detrimental due to widespread access to digital forms of social interaction through technologies such as social media. The findings reviewed highlight how physical distancing might have a disproportionate effect on an age group for whom peer interaction is a vital aspect of development.

Raising the bar: Can dual scanning improve our understanding of joint action?

Two-person neuroscience (2 ​PN) is a recently introduced conceptual and methodological framework used to investigate the neural basis of human social interaction from simultaneous neuroimaging of two or more subjects (hyperscanning). In this study, we adopted a 2 ​PN approach and a multiple-brain connectivity model to investigate the neural basis of a form of cooperation called joint action. We hypothesized different intra-brain and inter-brain connectivity patterns when comparing the interpersonal properties of joint action with non-interpersonal conditions, with a focus on co-representation, a core ability at the basis of cooperation. 32 subjects were enrolled in dual-EEG recordings during a computerized joint action task including three conditions: one in which the dyad jointly acted to pursue a common goal (joint), one in which each subject interacted with the PC (PC), and one in which each subject performed the task individually (Solo). A combination of multiple-brain connectivity estimation and specific indices derived from graph theory allowed to compare interpersonal with non-interpersonal conditions in four different frequency bands. Our results indicate that all the indices were modulated by the interaction, and returned a significantly stronger integration of multiple-subject networks in the joint vs. PC and Solo conditions. A subsequent classification analysis showed that features based on multiple-brain indices led to a better discrimination between social and non-social conditions with respect to single-subject indices. Taken together, our results suggest that multiple-brain connectivity can provide a deeper insight into the understanding of the neural basis of cooperation in humans.

The neuroscience of unmet social needs

John Cacioppo has compared loneliness to hunger or thirst in that it signals that one needs to act and repair what is lacking. This paper reviews Cacioppo's and others' contributions to our understanding of neural mechanisms underlying social motivation in humans and in other social species. We focus particularly on the dopaminergic reward system and try to integrate evidence from animal models and human research. In rodents, objective social isolation leads to increased social motivation, mediated by the brains' mesolimbic dopamine system. In humans, social rejection can lead to either increased or decreased social motivation, and is associated with activity in the insular cortex; while chronic loneliness is typically associated with decreased social motivation but has been associated with altered dopaminergic responses in the striatum. This mixed pattern of cross-species similarities and differences may arise from the substantially different methods used to study unmet social needs across species, and suggests the need for more direct and deliberate cross-species comparative research in this critically important domain.

How Task Interactivity Shapes Action Observation

Action observation triggers imitation, a powerful mechanism permitting interpersonal coordination. Coordination, however, also occurs when the partners’ actions are nonimitative and physically incongruent. One influential theory postulates that this is achieved via top-down modulation of imitation exerted by prefrontal regions. Here, we rather argue that coordination depends on sharing a goal with the interacting partner: this shapes action observation, overriding involuntary imitation, through the predictive activity of the left ventral premotor cortex (lvPMc). During functional magnetic resonance imaging (fMRI), participants played music in turn with a virtual partner in interactive and noninteractive conditions requiring 50% of imitative/nonimitative responses. In a full-factorial design, both perceptual features and low-level motor requirements were kept constant throughout the experiment. Behaviorally, the interactive context minimized visuomotor interference due to the involuntary imitation of physically incongruent movements. This was paralleled by modulation of neural activity in the lvPMc, which was specifically recruited during the interactive task independently of the imitative/nonimitative nature of the social exchange. This lvPMc activity reflected the predictive decoding of the partner’s actions, as revealed by multivariate pattern analysis. This demonstrates that, during interactions, we process our partners’ behavior to prospectively infer their contribution to the shared goal achievement, generating motor predictions for cooperation beyond low-level imitation.

Characterizing functional pathways of the human olfactory system

The central processing pathways of the human olfactory system are not fully understood. The olfactory bulb projects directly to a number of cortical brain structures, but the distinct networks formed by projections from each of these structures to the rest of the brain have not been well-defined. Here, we used functional magnetic resonance imaging and k-means clustering to parcellate human primary olfactory cortex into clusters based on whole-brain functional connectivity patterns. Resulting clusters accurately corresponded to anterior olfactory nucleus, olfactory tubercle, and frontal and temporal piriform cortices, suggesting dissociable whole-brain networks formed by the subregions of primary olfactory cortex. This result was replicated in an independent data set. We then characterized the unique functional connectivity profiles of each subregion, producing a map of the large-scale processing pathways of the human olfactory system. These results provide insight into the functional and anatomical organization of the human olfactory system.

Neural processing of social interaction: Coordinate-based meta-analytic evidence from human neuroimaging studies

While the action observation and mentalizing networks are considered to play complementary roles in understanding others' goals and intentions, they might be concurrently engaged when processing social interactions. We assessed this hypothesis via three activation-likelihood-estimation meta-analyses of neuroimaging studies on the neural processing of: (a) social interactions, (b) individual actions by the action observation network, and (c) mental states by the mentalizing network. Conjunction analyses and direct comparisons unveiled overlapping and specific regions among the resulting maps. We report quantitative meta-analytic evidence for a "social interaction network" including key nodes of the action observation and mentalizing networks. An action-social interaction-mentalizing gradient of activity along the posterior temporal cortex highlighted a hierarchical processing of interactions, from visuomotor analyses decoding individual and shared intentions to in-depth inferences on actors' intentional states. The medial prefrontal cortex, possibly in conjunction with the amygdala, might provide additional information concerning the affective valence of the interaction. This evidence suggests that the functional architecture underlying the neural processing of interactions involves the joint involvement of the action observation and mentalizing networks. These data might inform the design of rehabilitative treatments for social cognition disorders in pathological conditions, and the assessment of their outcome in randomized controlled trials.

Visuo-motor interference with a virtual partner is equally present in cooperative and competitive interactions

Automatic imitation of observed actions is thought to be a powerful mechanism, one that may mediate the reward value of interpersonal interactions, but that could also generate visuo-motor interference when interactions involve complementary movements. Since interpersonal coordination seems to be crucial both when cooperating and competing with others, the questions arises as to whether imitation-and thus visuo-motor interference-occurs in both scenarios. To address this issue, we asked human participants to engage in high- or low-interactive (Interactive or Cued condition, respectively), cooperative or competitive, joint reach-to-grasps with a virtual partner. More specifically, interactions occurred in: (i) a Cued condition, where participants simply adapted their movement timing to synchronize with (during cooperation) or anticipate (during competition) the virtual partner's grasp; (ii) an Interactive condition requiring the same adaptation, as well as a real-time selection of their action according to the virtual character's movement. To simulate a realistic human-human interaction, the virtual character would change its movement speed in consecutive trials according to participants' behaviour. Results demonstrate that visuo-motor interference-as indexed by movement kinematics (higher maximum wrist height during complementary compared to imitative power grips)-emerge in both cooperative and competitive motor interactions only when predictions about the partner's movements are needed to perform one's own action (interactive condition). These results support the idea that simulative imitation is heavily present when individuals need to match their behaviours closely.

Brain correlates of recognition of communicative interactions from biological motion in schizophrenia

BACKGROUND

Recognition of communicative interactions is a complex social cognitive ability which is associated with a specific neural activity in healthy individuals. However, neural correlates of communicative interaction processing from whole-body motion have not been known in patients with schizophrenia (SCZ). Therefore, the current study aims to examine the neural activity associated with recognition of communicative interactions in SCZ by using displays of the dyadic interactions downgraded to minimalistic point-light presentations.

METHODS

Twenty-six healthy controls (HC) and 25 SCZ were asked to judge whether two agents presented only by point-light displays were communicating or acting independently. Task-related activity and functional connectivity of brain structures were examined with General Linear Model and Generalized Psychophysiological Interaction approach, respectively.

RESULTS

HC were significantly more efficient in recognizing each type of action than SCZ. At the neural level, the activity of the right posterior superior temporal sulcus (pSTS) was observed to be higher in HC compared with SCZ for communicative v. individual action processing. Importantly, increased connectivity of the right pSTS with structures associated with mentalizing (left pSTS) and mirroring networks (left frontal areas) was observed in HC, but not in SCZ, during the presentation of social interactions.

CONCLUSION

Under-recruitment of the right pSTS, a structure known to have a pivotal role in social processing, may also be of importance for higher-order social cognitive deficits in SCZ. Furthermore, decreased task-related connectivity of the right pSTS may result in reduced use of additional sources of information (for instance motor resonance signals) during social cognitive processing in schizophrenia.

Taking two to tango: fMRI analysis of improvised joint action with physical contact

Many forms of joint action involve physical coupling between the participants, such as when moving a sofa together or dancing a tango. We report the results of a novel two-person functional MRI study in which trained couple dancers engaged in bimanual contact with an experimenter standing next to the bore of the magnet, and in which the two alternated between being the leader and the follower of joint improvised movements. Leading showed a general pattern of self-orientation, being associated with brain areas involved in motor planning, navigation, sequencing, action monitoring, and error correction. In contrast, following showed a far more sensory, externally-oriented pattern, revealing areas involved in somatosensation, proprioception, motion tracking, social cognition, and outcome monitoring. We also had participants perform a "mutual" condition in which the movement patterns were pre-learned and the roles were symmetric, thereby minimizing any tendency toward either leading or following. The mutual condition showed greater activity in brain areas involved in mentalizing and social reward than did leading or following. Finally, the analysis of improvisation revealed the dual importance of motor-planning and working-memory areas. We discuss these results in terms of theories of both joint action and improvisation.

Joint action aesthetics

Synchronized movement is a ubiquitous feature of dance and music performance. Much research into the evolutionary origins of these cultural practices has focused on why humans perform rather than watch or listen to dance and music. In this study, we show that movement synchrony among a group of performers predicts the aesthetic appreciation of live dance performances. We developed a choreography that continuously manipulated group synchronization using a defined movement vocabulary based on arm swinging, walking and running. The choreography was performed live to four audiences, as we continuously tracked the performers' movements, and the spectators' affective responses. We computed dynamic synchrony among performers using cross recurrence analysis of data from wrist accelerometers, and implicit measures of arousal from spectators' heart rates. Additionally, a subset of spectators provided continuous ratings of enjoyment and perceived synchrony using tablet computers. Granger causality analyses demonstrate predictive relationships between synchrony, enjoyment ratings and spectator arousal, if audiences form a collectively consistent positive or negative aesthetic evaluation. Controlling for the influence of overall movement acceleration and visual change, we show that dance communicates group coordination via coupled movement dynamics among a group of performers. Our findings are in line with an evolutionary function of dance-and perhaps all performing arts-in transmitting social signals between groups of people. Human movement is the common denominator of dance, music and theatre. Acknowledging the time-sensitive and immediate nature of the performer-spectator relationship, our study makes a significant step towards an aesthetics of joint actions in the performing arts.

Statistical learning in social action contexts

Sensitivity to the regularities and structure contained within sequential, goal-directed actions is an important building block for generating expectations about the actions we observe. Until now, research on statistical learning for actions has solely focused on individual action sequences, but many actions in daily life involve multiple actors in various interaction contexts. The current study is the first to investigate the role of statistical learning in tracking regularities between actions performed by different actors, and whether the social context characterizing their interaction influences learning. That is, are observers more likely to track regularities across actors if they are perceived as acting jointly as opposed to in parallel? We tested adults and toddlers to explore whether social context guides statistical learning and-if so-whether it does so from early in development. In a between-subjects eye-tracking experiment, participants were primed with a social context cue between two actors who either shared a goal of playing together ('Joint' condition) or stated the intention to act alone ('Parallel' condition). In subsequent videos, the actors performed sequential actions in which, for certain action pairs, the first actor's action reliably predicted the second actor's action. We analyzed predictive eye movements to upcoming actions as a measure of learning, and found that both adults and toddlers learned the statistical regularities across actors when their actions caused an effect. Further, adults with high statistical learning performance were sensitive to social context: those who observed actors with a shared goal were more likely to correctly predict upcoming actions. In contrast, there was no effect of social context in the toddler group, regardless of learning performance. These findings shed light on how adults and toddlers perceive statistical regularities across actors depending on the nature of the observed social situation and the resulting effects.

Social Transmission of Experience of Agency: An Experimental Study

The sense of controlling one’s own actions is fundamental to normal human mental function, and also underlies concepts of social responsibility for action. However, it remains unclear how the wider social context of human action influences sense of agency. Using a simple experimental design, we investigated, for the first time, how observing the action of another person or a robot could potentially influence one’s own sense of agency. We assessed how observing another’s action might change the perceived temporal relationship between one’s own voluntary actions and their outcomes, which has been proposed as an implicit measure of sense of agency. Working in pairs, participants chose between two action alternatives, one rewarded more frequently than the other, while watching a rotating clock hand. They judged, in separate blocks, either the time of their own action, or the time of a tone that followed the action. These were compared to baseline judgements of actions alone, or tones alone, to calculate the perceptual shift of action toward outcome and vice versa. Our design focused on how these two dependent variables, which jointly provide an implicit measure of sense of agency, might be influenced by observing another’s action. In the observational group, each participant could see the other’s actions. Multivariate analysis showed that the perceived time of action and tone shifted progressively toward the actual time of outcome with repeated experience of this social situation. No such progressive change occurred in other groups for whom a barrier hid participants’ actions from each other. However, a similar effect was observed in the group that viewed movements of a human-like robotic hand, rather than actions of another person. This finding suggests that observing the actions of others increases the salience of the external outcomes of action and this effect is not unique to observing human agents. Social contexts in which we see others controlling external events may play an important role in mentally representing the impact of our own actions on the external world.

Observing joint action: Coordination creates commitment

Previous research has shown that interpersonal coordination enhances pro-social attitudes and behavior. Here, we extend this research by investigating whether the degree of coordination observed in a joint action enhances the perception of individuals' commitment to the joint action. In four experiments, participants viewed videos of joint actions. In the low coordination condition, two agents made independent individual contributions to a joint action. In the high coordination condition, the individual contributions were tightly linked. Participants judged whether and for how long the observed agents would resist a tempting outside option and remain engaged in the joint action. The results showed that participants were more likely to expect agents to resist outside options when observing joint actions with a high degree of coordination. This indicates that observing interpersonal coordination is sufficient to enhance the perception of commitment to joint action.