Exploring the brain basis of joint action: co-ordination of actions, goals and intentions

Favoritism or bias? Cooperation and competition under different intergroup relationships: evidence from EEG hyperscanning

Cooperation and competition are the most common forms of social interaction in various social relationships. Intergroup relationships have been posited to influence individuals' interpersonal interactions significantly. Using electroencephalography hyperscanning, this study aimed to establish whether intergroup relationships influence interpersonal cooperation and competition and the underlying neural mechanisms. According to the results, the in-group Coop-index is better than the out-group, whereas the out-group Comp-index is stronger than the in-group. The in-group functional connectivity between the frontal-central region and the right temporoparietal junction in the β band was stronger in competition than cooperation. The out-group functional connectivity between the frontal-central region and the left temporoparietal junction in the α band was stronger in cooperation than competition. In both cooperation and competition, the in-group exhibited higher interbrain synchronization between the prefrontal cortex and parietal region in the θ band, as well as between the frontal-central region and frontal-central region in the α band, compared to the out-group. The intrabrain phase-locking value in both the α and β bands can effectively predict performance in competition tasks. Interbrain phase-locking value in both the α and θ bands can be effectively predicted in a performance cooperation task. This study offers neuroscientific evidence for in-group favoritism and out-group bias at an interpersonal level.

Influence of interpersonal distance on collaborative performance in the joint Simon task-An fNIRS-based hyperscanning study

Collaboration is a critical skill in everyday life. It has been suggested that collaborative performance may be influenced by social factors such as interpersonal distance, which is defined as the perceived psychological distance between individuals. Previous literature has reported that close interpersonal distance may promote the level of self-other integration between interacting members, and in turn, enhance collaborative performance. These studies mainly focused on interdependent collaboration, which requires high levels of shared representations and self-other integration. However, little is known about the effect of interpersonal distance on independent collaboration (e.g., the joint Simon task), in which individuals perform the task independently while the final outcome is determined by the parties. To address this issue, we simultaneously measured the frontal activations of ninety-four pairs of participants using a functional near-infrared spectroscopy (fNIRS)-based hyperscanning technique while they performed a joint Simon task. Behavioral results showed that the Joint Simon Effect (JSE), defined as the RT difference between incongruent and congruent conditions indicating the level of self-other integration between collaborators, was larger in the friend group than in the stranger group. Consistently, the inter-brain neural synchronization (INS) across the dorsolateral and medial parts of the prefrontal cortex was also stronger in the friend group. In addition, INS in the left dorsolateral prefrontal cortex negatively predicted JSE only in the friend group. These results suggest that close interpersonal distance may enhance the shared mental representation among collaborators, which in turn influences their collaborative performance.

An ecological study protocol for the multimodal investigation of the neurophysiological underpinnings of dyadic joint action

A novel multimodal experimental setup and dyadic study protocol were designed to investigate the neurophysiological underpinnings of joint action through the synchronous acquisition of EEG, ECG, EMG, respiration and kinematic data from two individuals engaged in ecologic and naturalistic cooperative and competitive joint actions involving face-to-face real-time and real-space coordinated full body movements. Such studies are still missing because of difficulties encountered in recording reliable neurophysiological signals during gross body movements, in synchronizing multiple devices, and in defining suitable study protocols. The multimodal experimental setup includes the synchronous recording of EEG, ECG, EMG, respiration and kinematic signals of both individuals via two EEG amplifiers and a motion capture system that are synchronized via a single-board microcomputer and custom Python scripts. EEG is recorded using new dry sports electrode caps. The novel study protocol is designed to best exploit the multimodal data acquisitions. Table tennis is the dyadic motor task: it allows naturalistic and face-to-face interpersonal interactions, free in-time and in-space full body movement coordination, cooperative and competitive joint actions, and two task difficulty levels to mimic changing external conditions. Recording conditions-including minimum table tennis rally duration, sampling rate of kinematic data, total duration of neurophysiological recordings-were defined according to the requirements of a multilevel analytical approach including a neural level (hyperbrain functional connectivity, Graph Theoretical measures and Microstate analysis), a cognitive-behavioral level (integrated analysis of neural and kinematic data), and a social level (extending Network Physiology to neurophysiological data recorded from two interacting individuals). Four practical tests for table tennis skills were defined to select the study population, permitting to skill-match the dyad members and to form two groups of higher and lower skilled dyads to explore the influence of skill level on joint action performance. Psychometric instruments are included to assess personality traits and support interpretation of results. Studying joint action with our proposed protocol can advance the understanding of the neurophysiological mechanisms sustaining daily life joint actions and could help defining systems to predict cooperative or competitive behaviors before being overtly expressed, particularly useful in real-life contexts where social behavior is a main feature.

Inter-group conflict affects inter-brain synchrony during synchronized movements

Interpersonal synchrony refers to alignment in time of interacting individuals. Recent neuroimaging findings indicate that the inferior frontal gyrus (IFG) - a core region of the observation-execution system - is not only activated during tasks that involve synchrony, but also coupled between interaction partners, suggesting a key role for the IFG in mediating interpersonal synchrony. In this study we investigated whether inter-brain synchrony (IBS) is modulated by inter-group relationships. We examined this question in the context of the Israeli-Palestinian conflict - one of the world's most prolonged and intractable conflicts. Using functional Near Infra-Red Spectroscopy (fNIRS) hyperscanning, we measured IBS among ingroup vs. inter-group dyads (same-nationality dyads and Jewish-Palestinian dyads, respectively) while they performed a task entailing 2D movement synchrony. The results point to an increase in behavioral synchrony and greater enjoyment in the ingroup dyads, compared to the inter-group dyads. Critically, IBS in the left IFG significantly increased throughout task and it was higher among ingroup compared to inter-group dyads. Our findings highlight the effect of group membership on IBS plasticity.

Multi-layer adaptation of group coordination in musical ensembles

Group coordination passes through an efficient integration of multimodal sources of information. This study examines complex non-verbal communication by recording movement kinematics from conductors and two sections of violinists of an orchestra adapting to a perturbation affecting their normal pattern of sensorimotor communication (rotation of half a turn of the first violinists’ section). We show that different coordination signals are channeled through ancillary (head kinematics) and instrumental movements (bow kinematics). Each one of them affect coordination either at the inter-group or intra-group levels, therefore tapping into different modes of cooperation: complementary versus imitative coordination. Our study suggests that the co-regulation of group behavior is based on the exchange of information across several layers, each one of them tuned to carry specific coordinative signals. Multi-layer sensorimotor communication may be the key musicians and, more generally humans, use to flexibly communicate between each other in interactive sensorimotor tasks.

Holistic cognitive and neural processes: a fNIRS-hyperscanning study on interpersonal sensorimotor synchronization

Interpersonal sensorimotor synchronization (interpersonal SMS) is the foundation of complex human social interaction. Previous studies primarily focused on the individual cognitive processes of interpersonal SMS. However, all individuals compose an entire interaction system with emerged holistic properties during interpersonal SMS. Therefore, we proposed the `holistic cognitive and neural processes' of interpersonal SMS and defined quantitative measurements that included Holistic Correction Gain (HCG), Holistic Timekeeper Variance (HTV) and Holistic Motor Variance (HMV) based on linear error correction model and inter-brain couplings obtained by hyperscanning technique. We performed a joint-tapping experiment including bidirectional and unidirectional conditions using functional near-infrared spectroscopy (fNIRS) hyperscanning to evaluate effects of these holistic processes on synchronization performance. We found that the dyads' performance highly correlated with the integrated effect of holistic cognitive processes in both conditions. Each holistic cognitive process played different roles in interpersonal SMS. HCG was critical to maintain synchronization. HTV related to mentalizing others' behavior. Holistic neural process, the inter-brain coupling of right prefrontal cortex (PFC), was significantly different between bidirectional and unidirectional conditions, which suggested the existence of neural markers at holistic level in interpersonal SMS.

Taking two to tango: fMRI analysis of improvised joint action with physical contact

Many forms of joint action involve physical coupling between the participants, such as when moving a sofa together or dancing a tango. We report the results of a novel two-person functional MRI study in which trained couple dancers engaged in bimanual contact with an experimenter standing next to the bore of the magnet, and in which the two alternated between being the leader and the follower of joint improvised movements. Leading showed a general pattern of self-orientation, being associated with brain areas involved in motor planning, navigation, sequencing, action monitoring, and error correction. In contrast, following showed a far more sensory, externally-oriented pattern, revealing areas involved in somatosensation, proprioception, motion tracking, social cognition, and outcome monitoring. We also had participants perform a "mutual" condition in which the movement patterns were pre-learned and the roles were symmetric, thereby minimizing any tendency toward either leading or following. The mutual condition showed greater activity in brain areas involved in mentalizing and social reward than did leading or following. Finally, the analysis of improvisation revealed the dual importance of motor-planning and working-memory areas. We discuss these results in terms of theories of both joint action and improvisation.

Following during physically-coupled joint action engages motion area MT+/V5

Interpersonal coordination during joint action depends on the perception of the partner's movements. In many such situations - for example, while moving furniture together or dancing a tango - there are kinesthetic interactions between the partners due to the forces shared between them that allow them to directly perceive one another's movements. Joint action of this type often involves a contrast between the roles of leader and follower, where the leader imparts forces onto the follower, and the follower has to be responsive to these force-cues during movement. We carried out a novel 2-person functional MRI study with trained couple dancers engaged in bimanual contact with an experimenter standing next to the bore of the magnet, where the two alternated between being the leader and follower of joint improvised movements, all with the eyes closed. One brain area that was unexpectedly more active during following than leading was the region of MT+/V5. While classically described as an area for processing visual motion, it has more recently been shown to be responsive to tactile motion as well. We suggest that MT+/V5 responds to motion based on force-cues during joint haptic interaction, most especially when a follower responds to force-cues coming from a leader's movements.

Decoding social intentions in human prehensile actions: Insights from a combined kinematics-fMRI study

Consistent evidence suggests that the way we reach and grasp an object is modulated not only by object properties (e.g., size, shape, texture, fragility and weight), but also by the types of intention driving the action, among which the intention to interact with another agent (i.e., social intention). Action observation studies ascribe the neural substrate of this 'intentional' component to the putative mirror neuron (pMNS) and the mentalizing (MS) systems. How social intentions are translated into executed actions, however, has yet to be addressed. We conducted a kinematic and a functional Magnetic Resonance Imaging (fMRI) study considering a reach-to-grasp movement performed towards the same object positioned at the same location but with different intentions: passing it to another person (social condition) or putting it on a concave base (individual condition). Kinematics showed that individual and social intentions are characterized by different profiles, with a slower movement at the level of both the reaching (i.e., arm movement) and the grasping (i.e., hand aperture) components. fMRI results showed that: (i) distinct voxel pattern activity for the social and the individual condition are present within the pMNS and the MS during action execution; (ii) decoding accuracies of regions belonging to the pMNS and the MS are correlated, suggesting that these two systems could interact for the generation of appropriate motor commands. Results are discussed in terms of motor simulation and inferential processes as part of a hierarchical generative model for action intention understanding and generation of appropriate motor commands.

To Pass or Not to Pass: Modeling the Movement and Affordance Dynamics of a Pick and Place Task

Humans commonly engage in tasks that require or are made more efficient by coordinating with other humans. In this paper we introduce a task dynamics approach for modeling multi-agent interaction and decision making in a pick and place task where an agent must move an object from one location to another and decide whether to act alone or with a partner. Our aims were to identify and model (1) the affordance related dynamics that define an actor's choice to move an object alone or to pass it to their co-actor and (2) the trajectory dynamics of an actor's hand movements when moving to grasp, relocate, or pass the object. Using a virtual reality pick and place task, we demonstrate that both the decision to pass or not pass an object and the movement trajectories of the participants can be characterized in terms of a behavioral dynamics model. Simulations suggest that the proposed behavioral dynamics model exhibits features observed in human participants including hysteresis in decision making, non-straight line trajectories, and non-constant velocity profiles. The proposed model highlights how the same low-dimensional behavioral dynamics can operate to constrain multiple (and often nested) levels of human activity and suggests that knowledge of what, when, where and how to move or act during pick and place behavior may be defined by these low dimensional task dynamics and, thus, can emerge spontaneously and in real-time with little a priori planning.

Help me if I can't: Social interaction effects in adult contextual word learning

A major challenge in second language acquisition is to build up new vocabulary. How is it possible to identify the meaning of a new word among several possible referents? Adult learners typically use contextual information, which reduces the number of possible referents a new word can have. Alternatively, a social partner may facilitate word learning by directing the learner's attention toward the correct new word meaning. While much is known about the role of this form of 'joint attention' in first language acquisition, little is known about its efficacy in second language acquisition. Consequently, we introduce and validate a novel visual word learning game to evaluate how joint attention affects the contextual learning of new words in a second language. Adult learners either acquired new words in a constant or variable sentence context by playing the game with a knowledgeable partner, or by playing the game alone on a computer. Results clearly show that participants who learned new words in social interaction (i) are faster in identifying a correct new word referent in variable sentence contexts, and (ii) temporally coordinate their behavior with a social partner. Testing the learned words in a post-learning recall or recognition task showed that participants, who learned interactively, better recognized words originally learned in a variable context. While this result may suggest that interactive learning facilitates the allocation of attention to a target referent, the differences in the performance during recognition and recall call for further studies investigating the effect of social interaction on learning performance. In summary, we provide first evidence on the role joint attention in second language learning. Furthermore, the new interactive learning game offers itself to further testing in complex neuroimaging research, where the lack of appropriate experimental set-ups has so far limited the investigation of the neural basis of adult word learning in social interaction.

Imitation Learning Errors Are Affected by Visual Cues in Both Performance and Observation Phases

Mechanisms of action imitation were examined. Previous studies have suggested that success or failure of imitation is determined at the point of observing an action. In other words, cognitive processing after observation is not related to the success of imitation; 20 university students participated in each of three experiments in which they observed a series of object manipulations consisting of four elements (hands, tools, object, and end points) and then imitated the manipulations. In Experiment 1, a specific intially observed element was color coded, and the specific manipulated object at the imitation stage was identically color coded; participants accurately imitated the color coded element. In Experiment 2, a specific element was color coded at the observation but not at the imitation stage, and there were no effects of color coding on imitation. In Experiment 3, participants were verbally instructed to attend to a specific element at the imitation stage, but the verbal instructions had no effect. Thus, the success of imitation may not be determined at the stage of observing an action and color coding can provide a clue for imitation at the imitation stage.

Dynamics of Social Interaction: Kinematic Analysis of a Joint Action

Non-verbal social interaction between humans requires accurate understanding of the others’ actions. The cognitivist approach suggests that successful interaction depends on the creation of a shared representation of the task, where the pairing of perceptive and motor systems of partners allows inclusion of the other’s goal into the overarching representation. Activity of the Mirror Neurons System (MNS) is thought to be a crucial mechanism linking two individuals during a joint action through action observation. The construction of a shared representation of an interaction (i.e., joint action) depends upon sensorimotor cognitive processes that modulate the ability to adapt in time and space. We attempted to detect individuals’ behavioral/kinematic change resulting in a global amelioration of performance for both subjects when a common representation of the action is built using a repetitive joint action. We asked pairs of subjects to carry out a simple task where one puts a base in the middle of a table and the other places a parallelepiped fitting into the base, the crucial manipulation being that participants switched roles during the experiment. We aimed to show that a full comprehension of a joint action is not an automatic process. We found that, before switching the interactional role, the participant initially placing the base orientated it in a way that led to an uncomfortable action for participants placing the parallelepiped. However, after switching roles, the action’s kinematics by the participant who places the base changed in order to facilitate the action of the other. More precisely, our data shows significant modulation of the base angle in order to ease the completion of the joint action, highlighting the fact that a shared knowledge of the complete action facilitates the generation of a common representation. This evidence suggests the ability to establish an efficient shared representation of a joint action benefits from physically taking our partner’s perspective because simply observing the actions of others may not be enough.

Effects of a Fragmented View of One's Partner on Interpersonal Coordination in Dance

In this study we investigated the effects of a mirror-mediated, partial view of one’s dance partner on interpersonal coordination in dance duets. Fourteen participant pairs (dyads) were asked to perform a reflectionally-symmetric eight-segment dance-relevant arm movement sequence in two visual conditions: with one dancer facing the mirror and providing a partial view on the dance partner, or both dancers facing back to back with, for both dancers, no view on one’s partner. During an eight-count beat-preparation phase, the task was paced via a metronome at three TEMPI; 1.6, 1.9, and 2.3 Hz, which was subsequently removed after which the movement sequence continued in silence. Interpersonal coordination was assessed using two tri-axial wireless accelerometers, one fixed to each dancer, that allowed the off-line kinematic analyses of dyad correlation, mean relative phase and mean standard deviation of relative phase of the up–down movements of (one of) the hands of the two dancers. In addition, two independent raters estimated the realized movement frequencies and percentage of the trial duration that the dancers moved in sync. Repeated measure ANOVAs revealed systematic effects of tempo on the performance measures, a positive effect of the use of the mirror on the coordination of the dancers’ movements but no facilitating effect of the mirror on the dancers’ synchronization. Overall, the results support the contention that when dancing to an internalized rhythmic beat the use of a mirror provides an ecological means to stabilize interpersonal coordination in dance duets without an effect on synchronization.

Factors affecting athletes' motor behavior after the observation of scenes of cooperation and competition in competitive sport: the effect of sport attitude

Aim: This study delineated how observing sports scenes of cooperation or competition modulated an action of interaction, in expert athletes, depending on their specific sport attitude.

Method: In a kinematic study, athletes were divided into two groups depending on their attitude toward teammates (cooperative or competitive). Participants observed sport scenes of cooperation and competition (basketball, soccer, water polo, volleyball, and rugby) and then they reached for, picked up, and placed an object on the hand of a conspecific (giving action). Mixed-design ANOVAs were carried out on the mean values of grasping-reaching parameters.

Results: Data showed that the type of scene observed as well as the athletes’ attitude affected reach-to-grasp actions to give. In particular, the cooperative athletes were speeded when they observed scenes of cooperation compared to when they observed scenes of competition.

Discussion: Participants were speeded when executing a giving action after observing actions of cooperation. This occurred only when they had a cooperative attitude. A match between attitude and intended action seems to be a necessary prerequisite for observing an effect of the observed type of scene on the performed action. It is possible that the observation of scenes of competition activated motor strategies which interfered with the strategies adopted by the cooperative participants to execute a cooperative (giving) sequence.

Facilitation effect of observed motor deviants in a cooperative motor task: Evidence for direct perception of social intention in action

Spatiotemporal parameters of voluntary motor action may help optimize human social interactions. Yet it is unknown whether individuals performing a cooperative task spontaneously perceive subtly informative social cues emerging through voluntary actions. In the present study, an auditory cue was provided through headphones to an actor and a partner who faced each other. Depending on the pitch of the auditory cue, either the actor or the partner were required to grasp and move a wooden dowel under time constraints from a central to a lateral position. Before this main action, the actor performed a preparatory action under no time constraint, consisting in placing the wooden dowel on the central location when receiving either a neutral ("prêt"-ready) or an informative auditory cue relative to who will be asked to perform the main action (the actor: "moi"-me, or the partner: "lui"-him). Although the task focused on the main action, analysis of motor performances revealed that actors performed the preparatory action with longer reaction times and higher trajectories when informed that the partner would be performing the main action. In this same condition, partners executed the main actions with shorter reaction times and lower velocities, despite having received no previous informative cues. These results demonstrate that the mere observation of socially driven motor actions spontaneously influences the low-level kinematics of voluntary motor actions performed by the observer during a cooperative motor task. These findings indicate that social intention can be anticipated from the mere observation of action patterns.

Kick with the finger: symbolic actions shape motor cortex excitability

A large body of research indicates that observing actions made by others is associated with corresponding motor facilitation of the observer's corticospinal system. However, it is still controversial whether this matching mechanism strictly reflects the kinematics of the observed action or its meaning. To test this issue, motor evoked potentials induced by single-pulse transcranial magnetic stimulation were recorded from hand and leg muscles while participants observed a symbolic action carried out with the index finger, but classically performed with the leg (i.e., a soccer penalty kick). A control condition in which participants observed a similar (but not symbolic) hand movement was also included. Results showed that motor facilitation occurs both in the observer's hand (first dorsal interosseous) and leg (quadriceps femoris) muscles. The present study provides evidence that both the kinematics and the symbolic value of an observed action are able to modulate motor cortex excitability. The human motor system is thus not only involved in mirroring observed actions but is also finely tuned to their symbolic value.

Congruent and Incongruent Corticospinal Activations at the Level of Multiple Effectors

Motor resonance is defined as the subliminal activation of the motor system while observing actions performed by others. However, resonating with another person's actions is not always an appropriate response: In real life, people do not just imitate but rather respond in a suitable fashion. A growing body of neurophysiologic studies has demonstrated that motor resonance can be overridden by complementary motor responses (such as preparing a precision grip on a small object when seeing an open hand in sign of request). In this study, we investigated the relationship between congruent and incongruent corticospinal activations at the level of multiple effectors. The modulation of motor evoked potentials evoked by single-pulse TMS over the motor cortex was assessed in upper and lower limb muscles of participants observing a soccer player performing a penalty kick straight in their direction. Study results revealed a double dissociation: Seeing the soccer player kicking the ball triggered a motor resonance in the observer's lower limb, whereas the upper limb response afforded by the object was overridden. On the other hand, seeing the ball approaching the observers elicited a complementary motor activation in upper limbs while motor resonance in lower limbs disappeared. Control conditions showing lateral kicks, mimicked kicks, and a ball in penalty area were also included to test the motor coding of object affordances. Results point to a modulation of motor responses in different limbs over the course of action and in function of their relevance in different contexts. We contend that ecologically valid paradigms are now needed to shed light on the motor system functioning in complex forms of interaction.

Motor interference in interactive contexts

Action observation and execution share overlapping neural substrates, so that simultaneous activation by observation and execution modulates motor performance. Previous literature on simple prehension tasks has revealed that motor influence can be two-sided: facilitation for observed and performed congruent actions and interference for incongruent actions. But little is known of the specific modulations of motor performance in complex forms of interaction. Is it possible that the very same observed movement can lead either to interference or facilitation effects on a temporally overlapping congruent executed action, depending on the context? To answer this question participants were asked to perform a reach-to-grasp movement adopting a precision grip (PG) while: (i) observing a fixation cross, (ii) observing an actor performing a PG with interactive purposes, (iii) observing an actor performing a PG without interactive purposes. In particular, in the interactive condition the actor was shown trying to pour some sugar on a large cup located out of her reach but close to the participant watching the video, thus eliciting in reaction a complementary whole-hand grasp. Notably, fine-grained kinematic analysis for this condition revealed a specific delay in the grasping and reaching components and an increased trajectory deviation despite the observed and executed movement’s congruency. Moreover, early peaks of trajectory deviation seem to indicate that socially relevant stimuli are acknowledged by the motor system very early. These data suggest that interactive contexts can determine a prompt modulation of stimulus–response compatibility effects.

The visual amplification of goal-oriented movements counteracts acquired non-use in hemiparetic stroke patients

BACKGROUND

Stroke-induced impairments result from both primary and secondary causes, i.e. damage to the brain and the acquired non-use of the impaired limbs. Indeed, stroke patients often under-utilize their paretic limb despite sufficient residual motor function. We hypothesize that acquired non-use can be overcome by reinforcement-based training strategies.

METHODS

Hemiparetic stroke patients (n = 20, 11 males, 9 right-sided hemiparesis) were asked to reach targets appearing in either the real world or in a virtual environment. Sessions were divided into 3 phases: baseline, intervention and washout. During the intervention the movement of the virtual representation of the patients' paretic limb was amplified towards the target.

RESULTS

We found that the probability of using the paretic limb during washout was significantly higher in comparison to baseline. Patients showed generalization of these results by displaying a more substantial workspace in real world task. These gains correlated with changes in effector selection patterns.

CONCLUSIONS

The amplification of the movement of the paretic limb in a virtual environment promotes the use of the paretic limb in stroke patients. Our findings indicate that reinforcement-based therapies may be an effective approach for counteracting learned non-use and may modulate motor performance in the real world.

Complementary actions

Complementary colors are color pairs which, when combined in the right proportions, produce white or black. Complementary actions refer here to forms of social interaction wherein individuals adapt their joint actions according to a common aim. Notably, complementary actions are incongruent actions. But being incongruent is not sufficient to be complementary (i.e., to complete the action of another person). Successful complementary interactions are founded on the abilities: (i) to simulate another person's movements, (ii) to predict another person's future action/s, (iii) to produce an appropriate incongruent response which differ, while interacting, with observed ones, and (iv) to complete the social interaction by integrating the predicted effects of one's own action with those of another person. This definition clearly alludes to the functional importance of complementary actions in the perception-action cycle and prompts us to scrutinize what is taking place behind the scenes. Preliminary data on this topic have been provided by recent cutting-edge studies utilizing different research methods. This mini-review aims to provide an up-to-date overview of the processes and the specific activations underlying complementary actions.

Self-organized complementary joint action: Behavioral dynamics of an interpersonal collision-avoidance task

Understanding stable patterns of interpersonal movement coordination is essential to understanding successful social interaction and activity (i.e., joint action). Previous research investigating such coordination has primarily focused on the synchronization of simple rhythmic movements (e.g., finger/forearm oscillations or pendulum swinging). Very few studies, however, have explored the stable patterns of coordination that emerge during task-directed complementary coordination tasks. Thus, the aim of the current study was to investigate and model the behavioral dynamics of a complementary collision-avoidance task. Participant pairs performed a repetitive targeting task in which they moved computer stimuli back and forth between sets of target locations without colliding into each other. The results revealed that pairs quickly converged onto a stable, asymmetric pattern of movement coordination that reflected differential control across participants, with 1 participant adopting a more straight-line movement trajectory between targets, and the other participant adopting a more elliptical trajectory between targets. This asymmetric movement pattern was also characterized by a phase lag between participants and was essential to task success. Coupling directionality analysis and dynamical modeling revealed that this dynamic regime was due to participant-specific differences in the coupling functions that defined the task-dynamics of participant pairs. Collectively, the current findings provide evidence that the dynamical coordination processes previously identified to underlie simple motor synchronization can also support more complex, goal-directed, joint action behavior, and can participate the spontaneous emergence of complementary joint action roles.

I give you a cup, I get a cup: a kinematic study on social intention

While affordances have been intensively studied, the mechanisms according to how their activation is modulated by context are poorly understood. We investigated how the Agent׳s reach-to-grasp movement towards a target-object (e.g. a can) is influenced by the other׳s interaction with a second object (manipulative/functional) and by his/her eye-gaze communication. To manipulate physical context we showed participants two objects that could be linked by a spatial relation (e.g. can-knife, typically found in the same context), or by different functional relations. The functional relations could imply an action to perform with another person (functional-cooperative: e.g. can-glass), or on our own (functional-individual: e.g. can-straw). When objects were not related (e.g. can-toothbrush) participants had to refrain from responding. In order to respond, in the giving condition participants had to move the target object towards the other person, in the getting condition towards their own body. When participants (Agents) performed a reach-to-grasp movement to give the target object, in presence of eye-gaze communication they reached the wrist׳s acceleration peak faster if the Other previously interacted with the second object in accordance with its conventional use. Consistently participants reached faster the MFA when the objects were related by a functional-individual than a functional-cooperative relation. The Agent׳s getting response strongly affected the grasping component of the movement: in case of eye-gaze sharing, MFA was greater when the other previously performed a manipulative than a functional grip. Results reveal that humans have developed a sophisticated capability in detecting information from hand posture and eye-gaze, which are informative as to the Agent׳s intention.

The eye contact effect in request and emblematic hand gestures

Request and emblematic gestures, despite being both communicative gestures, do differ in terms of social valence. Indeed, only the former are used to initiate/maintain/terminate an actual interaction. If such a difference is at stake, a relevant social cue, i.e. eye contact, should have different impacts on the neuronal underpinnings of the two types of gesture. We measured blood oxygen level-dependent signals, using functional magnetic resonance imaging, while participants watched videos of an actor, either blindfolded or not, performing emblems, request gestures, or meaningless control movements. A left-lateralized network was more activated by both types of communicative gestures than by meaningless movements, regardless of the accessibility of the actor's eyes. Strikingly, when eye contact was taken into account as a factor, a right-lateralized network was more strongly activated by emblematic gestures performed by the non-blindfolded actor than by those performed by the blindfolded actor. Such modulation possibly reflects the integration of information conveyed by the eyes with the representation of emblems. Conversely, a wider right-lateralized network was more strongly activated by request gestures performed by the blindfolded than by those performed by the non-blindfolded actor. This probably reflects the effect of the conflict between the observed action and its associated contextual information, in which relevant social cues are missing.

How relevant is social interaction in second language learning?

Verbal language is the most widespread mode of human communication, and an intrinsically social activity. This claim is strengthened by evidence emerging from different fields, which clearly indicates that social interaction influences human communication, and more specifically, language learning. Indeed, research conducted with infants and children shows that interaction with a caregiver is necessary to acquire language. Further evidence on the influence of sociality on language comes from social and linguistic pathologies, in which deficits in social and linguistic abilities are tightly intertwined, as is the case for Autism, for example. However, studies on adult second language (L2) learning have been mostly focused on individualistic approaches, partly because of methodological constraints, especially of imaging methods. The question as to whether social interaction should be considered as a critical factor impacting upon adult language learning still remains underspecified. Here, we review evidence in support of the view that sociality plays a significant role in communication and language learning, in an attempt to emphasize factors that could facilitate this process in adult language learning. We suggest that sociality should be considered as a potentially influential factor in adult language learning and that future studies in this domain should explicitly target this factor.

Coordination dynamics in a socially situated nervous system

Traditional theories of cognitive science have typically accounted for the organization of human behavior by detailing requisite computational/representational functions and identifying neurological mechanisms that might perform these functions. Put simply, such approaches hold that neural activity causes behavior. This same general framework has been extended to accounts of human social behavior via concepts such as “common-coding” and “co-representation” and much recent neurological research has been devoted to brain structures that might execute these social-cognitive functions. Although these neural processes are unquestionably involved in the organization and control of human social interactions, there is good reason to question whether they should be accorded explanatory primacy. Alternatively, we propose that a full appreciation of the role of neural processes in social interactions requires appropriately situating them in their context of embodied-embedded constraints. To this end, we introduce concepts from dynamical systems theory and review research demonstrating that the organization of human behavior, including social behavior, can be accounted for in terms of self-organizing processes and lawful dynamics of animal-environment systems. Ultimately, we hope that these alternative concepts can complement the recent advances in cognitive neuroscience and thereby provide opportunities to develop a complete and coherent account of human social interaction.

Reconceptualizing second-person interaction

Over the last couple of decades, most neuroscientific research on social cognition has been dominated by a third-person paradigm in which participating subjects are not actively engaging with other agents but merely observe them. Recently this paradigm has been challenged by researchers who promote a second-person approach to social cognition, and emphasize the importance of dynamic, real-time interactions with others. The present article's contribution to this debate is twofold. First, we critically analyze the second-person challenge to social neuroscience, and assess the various ways in which the distinction between second- versus third-person modes of social cognition has been articulated. Second, we put forward an alternative conceptualization of this distinction-one that gives pride of place to the notion of reciprocity. We discuss the implications of our proposal for neuroscientific studies on social cognition.

The end-state comfort effect facilitates joint action

Motor experts can accurately predict the future actions of others by observing their movements. This report describes three experiments that investigate such predictions in everyday object manipulations and test whether these predictions facilitate responses to the actions of others. Observing video excerpts showing an actor reaching for a vertically mounted dial, participants in Experiment 1 needed to predict how the actor would rotate it. Their predictions were specific to the direction and extent of the dial rotation and improved proportionate to the length of the video clip shown. Testing whether such predictions facilitate responses, in the subsequent experiments responders had to undo an actor's actions, back-rotating a dial (Exp 2) and a bar (Exp 3). The responders' actions were initiated faster when the actors' movements obeyed the so-called end-state comfort principle than when they did not. Our experiments show that humans exploit the end-state comfort effect to tweak their predictions of the future actions of others. The results moreover suggest that the precision of these predictions is mediated by perceptual learning rather than by motor simulation.

EXTENDING FEYNMAN'S FORMALISMS FOR MODELING HUMAN JOINT ACTION COORDINATION

The recently developed Life-Space-Foam approach to goal-directed human action deals with individual actor dynamics. This paper applies the model to characterize the dynamics of co-action by two or more actors. This dynamics is modelled by (i) a two-term joint action (including cognitive/motivatonal potential and kinetic energy), and (ii) its associated adaptive path integral, representing an infinite-dimensional neural network. Its feedback adaptation loop has been derived from Bernstein's concepts of sensory corrections loop in human motor control and Brooks' subsumption architectures in robotics. Potential applications of the proposed model in human-robot interaction research are discussed.

Imitation of hand and tool actions is effector-independent

Following the theoretical notion that tools often extend one’s body, in the present study, we investigated whether imitation of hand or tool actions is modulated by effector-specific information. Subjects performed grasping actions toward an object with either a handheld tool or their right hand. Actions were initiated in response to pictures representing a grip at an object that could be congruent or incongruent with the required action (grip-type congruency). Importantly, actions could be cued by means of a tool cue, a hand cue, and a symbolic cue (effector-type congruency). For both hand and tool actions, an action congruency effect was observed, reflected in faster reaction times if the observed grip type was congruent with the required movement. However, neither hand actions nor tool actions were differentially affected by the effector represented in the picture (i.e., when performing a tool action, the action congruency effect was similar for tool cues and hand cues). This finding suggests that imitation of hand and tool actions is effector-independent and thereby supports generalist rather than specialist theories of imitation.

Exploring the Neural Basis of Real-Life Joint Action: Measuring Brain Activation during Joint Table Setting with Functional Near-Infrared Spectroscopy

Many every-day life situations require two or more individuals to execute actions together. Assessing brain activation during naturalistic tasks to uncover relevant processes underlying such real-life joint action situations has remained a methodological challenge. In the present study, we introduce a novel joint action paradigm that enables the assessment of brain activation during real-life joint action tasks using functional near-infrared spectroscopy (fNIRS). We monitored brain activation of participants who coordinated complex actions with a partner sitting opposite them. Participants performed table setting tasks, either alone (solo action) or in cooperation with a partner (joint action), or they observed the partner performing the task (action observation). Comparing joint action and solo action revealed stronger activation (higher [oxy-Hb]-concentration) during joint action in a number of areas. Among these were areas in the inferior parietal lobule (IPL) that additionally showed an overlap of activation during action observation and solo action. Areas with such a close link between action observation and action execution have been associated with action simulation processes. The magnitude of activation in these IPL areas also varied according to joint action type and its respective demand on action simulation. The results validate fNIRS as an imaging technique for exploring the functional correlates of interindividual action coordination in real-life settings and suggest that coordinating actions in real-life situations requires simulating the actions of the partner.

Covert motor activity on NoGo trials in a task sharing paradigm: evidence from the lateralized readiness potential

Previous studies on task sharing propose that a representation of the co-actor's task share is generated when two actors share a common task. An important function of co-representation seems to lie in the anticipation of others' upcoming actions, which is essential for one's own action planning, as it enables the rapid selection of an appropriate response. We utilized measures of lateralized motor activation, the lateralized readiness potential (LRP), in a task sharing paradigm to address the questions (1) whether the generation of a co-representation involves motor activity in the non-acting person when it is other agent's turn to respond, and (2) whether co-representation of the other's task share is generated from one's own egocentric perspective or from the perspective of the actor (allocentric). Results showed that although it was the other agent's turn to respond, the motor system of the non-acting person was activated prior to the other's response. Furthermore, motor activity was based on egocentric spatial properties. The findings support the tight functional coupling between one's own actions and actions produced by others, suggesting that the involvement of the motor system is crucial for social interaction.

Recruitment of both the mirror and the mentalizing networks when observing social interactions depicted by point-lights: a neuroimaging study

BACKGROUND

Understanding social interactions requires the ability to accurately interpret conspecifics' actions, sometimes only on the basis of subtle body language analysis. Here we address an important issue that has not yet received much attention in social neuroscience, that of an interaction between two agents. We attempted to isolate brain responses to two individuals interacting compared to two individuals acting independently.

METHODOLOGY/PRINCIPAL FINDINGS

We used minimalistic point-light displays to depict the characters, as they provide the most straightforward way to isolate mechanisms used to extract information from motion per se without any interference with other visual information. Functional magnetic resonance imaging (fMRI) method was used to determine which brain regions were recruited during the observation of two interacting agents, mimicking everyday social scenes. While the mirror and mentalizing networks are rarely concurrently active, we found that both of them might be needed to catch the social intentions carried by whole-body motion.

CONCLUSIONS/SIGNIFICANCE

These findings shed light on how motor cognition contributes to social cognition when social information is embedded in whole-body motion only. Finally, the approach described here provides a valuable and original tool for investigating the brain networks responsible for social understanding, in particular in psychiatric disorders.

Neuro-cognitive mechanisms of decision making in joint action: a human-robot interaction study

In this paper we present a model for action preparation and decision making in cooperative tasks that is inspired by recent experimental findings about the neuro-cognitive mechanisms supporting joint action in humans. It implements the coordination of actions and goals among the partners as a dynamic process that integrates contextual cues, shared task knowledge and predicted outcome of others' motor behavior. The control architecture is formalized by a system of coupled dynamic neural fields representing a distributed network of local but connected neural populations. Different pools of neurons encode task-relevant information about action means, task goals and context in the form of self-sustained activation patterns. These patterns are triggered by input from connected populations and evolve continuously in time under the influence of recurrent interactions. The dynamic model of joint action is evaluated in a task in which a robot and a human jointly construct a toy object. We show that the highly context sensitive mapping from action observation onto appropriate complementary actions allows coping with dynamically changing joint action situations.

Follow you, Follow me: Continuous Mutual Prediction and Adaptation in Joint Tapping

To study the mechanisms of coordination that are fundamental to successful interactions we carried out a joint finger tapping experiment in which pairs of participants were asked to maintain a given beat while synchronizing to an auditory signal coming from the other person or the computer. When both were hearing each other, the pair became a coupled, mutually and continuously adaptive unit of two “hyper-followers”, with their intertap intervals (ITIs) oscillating in opposite directions on a tap-to-tap basis. There was thus no evidence for the emergence of a leader–follower strategy. We also found that dyads were equally good at synchronizing with the irregular, but responsive other as with the predictable, unresponsive computer. However, they performed worse when the “other” was both irregular and unresponsive. We thus propose that interpersonal coordination is facilitated by the mutual abilities to (a) predict the other's subsequent action and (b) adapt accordingly on a millisecond timescale.

Granger causality mapping during joint actions reveals evidence for forward models that could overcome sensory-motor delays

Studies investigating joint actions have suggested a central role for the putative mirror neuron system (pMNS) because of the close link between perception and action provided by these brain regions [1], [2], [3]. In contrast, our previous functional magnetic resonance imaging (fMRI) experiment demonstrated that the BOLD response of the pMNS does not suggest that it directly integrates observed and executed actions during joint actions [4]. To test whether the pMNS might contribute indirectly to the integration process by sending information to brain areas responsible for this integration (integration network), here we used Granger causality mapping (GCM) [5]. We explored the directional information flow between the anterior sites of the pMNS and previously identified integrative brain regions. We found that the left BA44 sent more information than it received to both the integration network (left thalamus, right middle occipital gyrus and cerebellum) and more posterior nodes of the pMNS (BA2). Thus, during joint actions, two anatomically separate networks therefore seem effectively connected and the information flow is predominantly from anterior to posterior areas of the brain. These findings suggest that the pMNS is involved indirectly in joint actions by transforming observed and executed actions into a common code and is part of a generative model that could predict the future somatosensory and visual consequences of observed and executed actions in order to overcome otherwise inevitable neural delays.

Acting together in and beyond the mirror neuron system

Moving a set dinner table often takes two people, and doing so without spilling the glasses requires the close coordination of the two agents' actions. It has been argued that the mirror neuron system may be the key neural locus of such coordination. Instead, here we show that such coordination recruits two separable sets of areas: one that could translate between motor and visual codes and one that could integrate these information to achieve common goals. The former includes regions of the putative mirror neuron system, the latter, regions of the prefrontal, posterior parietal and temporal lobe adjacent to the putative mirror neuron system. Both networks were more active while participants cooperated with a human agent, responding to their actions, compared to a computer that did not, evidencing their social dimension. This finding shows that although the putative mirror neuron system can play a critical role in joint actions by translating both agents' actions into a common code, the flexible remapping of our own actions with those of others required during joint actions seems to be performed outside of the putative mirror neuron system.