Strategic communication and behavioral coupling in asymmetric joint action

Communicative modulations of early action components support the prediction of distal goals

The successful unfolding of many social interactions relies on our capacity to predict other people's action goals, whether these are proximal (i.e., immediate) or distal (i.e., upcoming). The present set of studies asks whether observers can predict the distal goal of two-step action sequences when presented with communicative modulations of the first movement component of the sequence. We conducted three online experiments in which we presented participants with animations of a box moving to a first target location before moving onwards to a final, either near or far, target location. The second movement component and the target locations were occluded. After observing the first movement, participants were asked to select the most likely final target location, i.e., the distal goal of the sequence. Experiment 1 showed that participants relied on the velocity modulations of the first movement to infer the distal goal. The results of Experiment 2 indicated that such predictions of distal goals are possible even when the second movement in the sequence does not contain any velocity information, thus suggesting that the information present in the first movement plays the major role in the process of linking movements to their distal goals. However, Experiment 3 showed that under some circumstances the second movement can also contribute to how observers predict a distal goal. We discuss these results in terms of the underlying simulation processes that enable observers to predict a distal goal from the observation of proximal communicative modulations.

Space-time mapping relationships in sensorimotor communication during asymmetric joint action

Background

Sensorimotor communication is frequently observed in complex joint actions and social interactions. However, it remains challenging to explore the cognitive foundations behind sensorimotor communication.

Methods

The present study extends previous research by introducing a single-person baseline condition and formulates two distinct categories of asymmetric joint action tasks: distance tasks and orientation tasks. This research investigates the action performance of 65 participants under various experimental conditions utilizing a 2 (cooperative intention: Coop, No-coop) × 2 (task characteristic: distance, orientation) × 4 (target: T1, T2, T3, T4) repeated-measures experimental design to investigate the cognitive mechanisms underlying sensorimotor communication between individuals.

Results

The results showed that (1) target key dwell time, motion time, total motion time, and maximum motion height in the Coop condition are more than in the No-coop condition. (2) In the distance task without cooperative intention, the dwell time of T4 is smaller than T1, T2, T3, and its variability of T1, T2, T3, and T4 were no different. In the distance task with cooperative intention, the dwell time and its variability of T1, T2, T3, and T4 displayed an increasing trend. (3) In the orientation task without cooperative intention, the dwell time of T1 is smaller than T2, T3, T4, and variability of the target keys T1, T2, T3, and T4 had no difference. In the orientation task with cooperative intention, the dwell time and variability of the target keys T1, T2, T3, and T4 had increasing trends.

Conclusions

Those findings underscore the importance of cooperative intention for sensorimotor communication. In the distance task with cooperative intention, message senders establish a mapping relationship characterized by "near-small, far-large" between the task distance and the individual's action characteristics through sensorimotor experience. In the orientation task with cooperative intention, message senders combined sensorimotor experience and verbal metaphors to establish a mapping relationship between task orientation and action characteristics, following the sequence of "left-up, right-up, left-down, right-down" to transmit the message to others.

Flexible Cultural Learning Through Action Coordination

The cultural transmission of technical know-how has proven vital to the success of our species. The broad diversity of learning contexts and social configurations, as well as the various kinds of coordinated interactions they involve, speaks to our capacity to flexibly adapt to and succeed in transmitting vital knowledge in various learning contexts. Although often recognized by ethnographers, the flexibility of cultural learning has so far received little attention in terms of cognitive mechanisms. We argue that a key feature of the flexibility of cultural learning is that both the models and learners recruit cognitive mechanisms of action coordination to modulate their behavior contingently on the behavior of their partner, generating a process of mutual adaptation supporting the successful transmission of technical skills in diverse and fluctuating learning environments. We propose that the study of cultural learning would benefit from the experimental methods, results, and insights of joint-action research and, complementarily, that the field of joint-action research could expand its scope by integrating a learning and cultural dimension. Bringing these two fields of research together promises to enrich our understanding of cultural learning, its contextual flexibility, and joint action coordination.

What's your point? Insights from virtual reality on the relation between intention and action in the production of pointing gestures

Human communication involves the process of translating intentions into communicative actions. But how exactly do our intentions surface in the visible communicative behavior we display? Here we focus on pointing gestures, a fundamental building block of everyday communication, and investigate whether and how different types of underlying intent modulate the kinematics of the pointing hand and the brain activity preceding the gestural movement. In a dynamic virtual reality environment, participants pointed at a referent to either share attention with their addressee, inform their addressee, or get their addressee to perform an action. Behaviorally, it was observed that these different underlying intentions modulated how long participants kept their arm and finger still, both prior to starting the movement and when keeping their pointing hand in apex position. In early planning stages, a neurophysiological distinction was observed between a gesture that is used to share attitudes and knowledge with another person versus a gesture that mainly uses that person as a means to perform an action. Together, these findings suggest that our intentions influence our actions from the earliest neurophysiological planning stages to the kinematic endpoint of the movement itself.

Understanding others' distal goals from proximal communicative actions

Many social interactions require individuals to coordinate their actions and to inform each other about their goals. Often these goals concern an immediate (i.e., proximal) action, as when people give each other a brief handshake, but they sometimes also refer to a future (i.e. distal) action, as when football players perform a passing sequence. The present study investigates whether observers can derive information about such distal goals by relying on kinematic modulations of an actor's instrumental actions. In Experiment 1 participants were presented with animations of a box being moved at different velocities towards an apparent endpoint. The distal goal, however, was for the object to be moved past this endpoint, to one of two occluded target locations. Participants then selected the location which they considered the likely distal goal of the action. As predicted, participants were able to detect differences in movement velocity and, based on these differences, systematically mapped the movements to the two distal goal locations. Adding a distal goal led to more variation in the way participants mapped the observed movements onto different target locations. The results of Experiments 2 and 3 indicated that this cannot be explained by difficulties in perceptual discrimination. Rather, the increased variability likely reflects differences in interpreting the underlying connection between proximal communicative actions and distal goals. The present findings extend previous research on sensorimotor communication by demonstrating that communicative action modulations are not restricted to predicting proximal goals but can also be used to infer more distal goals.

The multimodal nature of communicative efficiency in social interaction

How does communicative efficiency shape language use? We approach this question by studying it at the level of the dyad, and in terms of multimodal utterances. We investigate whether and how people minimize their joint speech and gesture efforts in face-to-face interactions, using linguistic and kinematic analyses. We zoom in on other-initiated repair-a conversational microcosm where people coordinate their utterances to solve problems with perceiving or understanding. We find that efforts in the spoken and gestural modalities are wielded in parallel across repair turns of different types, and that people repair conversational problems in the most cost-efficient way possible, minimizing the joint multimodal effort for the dyad as a whole. These results are in line with the principle of least collaborative effort in speech and with the reduction of joint costs in non-linguistic joint actions. The results extend our understanding of those coefficiency principles by revealing that they pertain to multimodal utterance design.

What makes us act together? On the cognitive models supporting humans’ decisions for joint action

We face tasks every day that we can solve alone but decide to solve together with others. When do we choose to act together vs. alone? How long do we persist in working together when doing so is difficult? Do we prefer to act together when times are uncertain? An open question in joint action research is under what conditions humans prefer to act together or alone to achieve a certain goal, and whether their preference is based on a utility calculus that takes into account the costs and benefits associated with individual and joint action alternatives. Research on cooperation reveals that frequent engagement in joint activities provides high survival benefits, as it allows individuals to achieve goals together that are otherwise unavailable. Yet, survival advantage does not wholly explain the reasons for human cooperative behavior. In fact, humans are motivated to cooperate even when it is not necessary to achieve an outcome. Research in cognitive science suggests that navigating the potential costs of joint actions is a challenge for humans, and that joint actions might provide individuals with rewards that go beyond the achievement of instrumental goals. We here address the influence of key factors on the decision to engage in joint action, such as the coordination costs arising when acting together compared to alone and the social and instrumental rewards expected when acting together compared to alone. Addressing these questions will provide critical insight for the design of cognitive models of human decisions for cooperation.

Reaction time coupling in a joint stimulus-response task: A matter of functional actions or likable agents?

Shaping one owns actions by observing others’ actions is driven by the deep-rooted mechanism of perception-action coupling. It typically occurs automatically, expressed as for example the unintentional synchronization of reaction times in interactive games. Theories on perception-action coupling highlight its benefits such as the joint coordination of actions to cooperatively perform tasks properly, the learning of novel actions from others, and the bonding with likable others. However, such functional aspects and how they shape perception-action coupling have never been compared quantitatively. Here we tested a total of hundred-fifteen participants that played a stimulus-response task while, in parallel, they observed videos of agents that played the exact same task several milliseconds in advance. We compared to what degree the reaction times of actions of agents, who varied their behavior in terms of functionality and likability in preceding prisoner dilemma games and quizzes, shape the reaction times of human test participants. To manipulate functionality and likability, we varied the predictability of cooperative behavior and correctness of actions of agents, respectively, resulting in likable (cooperative), dislikable (uncooperative), functional (correct actions), and dysfunctional (incorrect actions) agents. The results of three experiments showed that the participants’ reaction times correlated most with the reaction times of agents that expressed functional behavior. However, the likability of agents had no effects on reaction time correlations. These findings suggest that, at least in the current computer task, participants are more likely to adopt the timing of actions from people that perform correct actions than from people that they like.

Reproducible Inter-Personal Brain Coupling Measurements in Hyperscanning Settings With functional Near Infra-Red Spectroscopy

Despite a huge advancement in neuroimaging techniques and growing importance of inter-personal brain research, few studies assess the most appropriate computational methods to measure brain-brain coupling. Here, we focus on the signal processing methods to detect brain-coupling in dyads. From a public dataset of functional Near Infra-Red Spectroscopy signals (N=24 dyads), we derived a synthetic control condition by randomization, we investigated the effectiveness of four most used signal similarity metrics: Cross Correlation, Mutual Information, Wavelet Coherence and Dynamic Time Warping. We also accounted for temporal variations between signals by allowing for misalignments up to a maximum lag. Starting from the observed effect sizes, computed in terms of Cohen's d, the power analysis indicated that a high sample size ([Formula: see text]) would be required to detect significant brain-coupling. We therefore discuss the need for specialized statistical approaches and propose bootstrap as an alternative method to avoid over-penalizing the results. In our settings, and based on bootstrap analyses, Cross Correlation and Dynamic Time Warping outperform Mutual Information and Wavelet Coherence for all considered maximum lags, with reproducible results. These results highlight the need to set specific guidelines as the high degree of customization of the signal processing procedures prevents the comparability between studies, their reproducibility and, ultimately, undermines the possibility of extracting new knowledge.

Human adults prefer to cooperate even when it is costly

Joint actions are cooperative activities where humans coordinate their actions to achieve individual and shared goals. While the motivation to engage in joint action is clear when a goal cannot be achieved by individuals alone, we asked whether humans are motivated to act together even when acting together is not necessary and implies incurring additional costs compared to individual goal achievement. Using a utility-based empirical approach, we investigated the extent of humans' preference for joint action over individual action, when the instrumental costs of performing joint actions outweigh the benefits. The results of five experiments showed that human adults have a stable preference for joint action, even if individual action is more effective to achieve a certain goal. We propose that such preferences can be understood as ascribing additional reward value to performing actions together.

Addressing joint action challenges in HRI: Insights from psychology and philosophy

The vast expansion of research in human-robot interactions (HRI) these last decades has been accompanied by the design of increasingly skilled robots for engaging in joint actions with humans. However, these advances have encountered significant challenges to ensure fluent interactions and sustain human motivation through the different steps of joint action. After exploring current literature on joint action in HRI, leading to a more precise definition of these challenges, the present article proposes some perspectives borrowed from psychology and philosophy showing the key role of communication in human interactions. From mutual recognition between individuals to the expression of commitment and social expectations, we argue that communicative cues can facilitate coordination, prediction, and motivation in the context of joint action. The description of several notions thus suggests that some communicative capacities can be implemented in the context of joint action for HRI, leading to an integrated perspective of robotic communication.

Evaluating the relative contributions of copying and reconstruction processes in cultural transmission episodes

The ability to transmit information between individuals through social learning is a foundational component of cultural evolution. However, how this transmission occurs is still debated. On the one hand, the copying account draws parallels with biological mechanisms for genetic inheritance, arguing that learners copy what they observe and novel variations occur through random copying errors. On the other hand, the reconstruction account claims that, rather than directly copying behaviour, learners reconstruct the information that they believe to be most relevant on the basis of pragmatic inference, environmental and contextual cues. Distinguishing these two accounts empirically is difficult based on data from typical transmission chain studies because the predictions they generate frequently overlap. In this study we present a methodological approach that generates different predictions of these accounts by manipulating the task context between model and learner in a transmission episode. We then report an empirical proof-of-concept that applies this approach. The results show that, when a model introduces context-dependent embedded signals to their actions that are not intended to be transmitted, it is possible to empirically distinguish between competing predictions made by these two accounts. Our approach can therefore serve to understand the underlying cognitive mechanisms at play in cultural transmission and can make important contributions to the debate between preservative and reconstructive schools of thought.

Simultaneous self-other integration and segregation support real-time interpersonal coordination in a musical joint action task

The ability to distinguish between an individual's own actions and those of another person is a requirement for successful joint action, particularly in domains such as group music making where precise interpersonal coordination ensures perceptual overlap in the effects of co-performers' actions. We tested the hypothesis that such coordination benefits from simultaneous integration and segregation of information about 'self' and 'other' in an experiment using a musical joint action paradigm. Sixteen pairs of individuals with little or no musical training performed a dyadic synchronization task on a pair of electronic music boxes. The relationship between pitches produced by paired participants (same vs. different) and the relationship between movement frequencies required to trigger synchronous tones (congruent vs. incongruent) were varied in a repeated measures design. The results indicate that interpersonal coordination was most accurate when sounds were different in pitch but movement frequency was congruent. Under other conditions, participants often drifted apart, resulting in poor coordination, especially with same sounds and incongruent movements across co-performers. These findings suggest that interpersonal coordination was facilitated when simultaneous self-other integration and segregation occurred across sensory modalities in an asymmetrical manner where pitch relations favoured segregation via auditory streaming while movement congruence favoured integration via visuo-motor coupling. Such self-other representational balance may enable co-performers to maintain autonomous control while attending, anticipating, and adapting to each other's timing when joint action requires precise temporal coordination.

Mechanisms for mutual support in motor interactions

What is the key to successful interaction? Is it sufficient to represent a common goal, or does the way our partner achieves that goal count as well? How do we react when our partner misbehaves? We used a turn-taking music-like task requiring participants to play sequences of notes together with a partner, and we investigated how people adapt to a partner's error that violates their expectations. Errors consisted of either playing a wrong note of a sequence that the agents were playing together (thus preventing the achievement of the joint goal) or playing the expected note with an unexpected action. In both cases, we found post-error slowing and inaccuracy suggesting the participants' implicit tendency to correct the partner's error and produce the action that the partner should have done. We argue that these "joint" monitoring processes depend on the motor predictions made within a (dyadic) motor plan and may represent a basic mechanism for mutual support in motor interactions.

Haptic communication optimises joint decisions and affords implicit confidence sharing

Group decisions can outperform the choices of the best individual group members. Previous research suggested that optimal group decisions require individuals to communicate explicitly (e.g., verbally) their confidence levels. Our study addresses the untested hypothesis that implicit communication using a sensorimotor channel—haptic coupling—may afford optimal group decisions, too. We report that haptically coupled dyads solve a perceptual discrimination task more accurately than their best individual members; and five times faster than dyads using explicit communication. Furthermore, our computational analyses indicate that the haptic channel affords implicit confidence sharing. We found that dyads take leadership over the choice and communicate their confidence in it by modulating both the timing and the force of their movements. Our findings may pave the way to negotiation technologies using fast sensorimotor communication to solve problems in groups.

Crossmodal correspondences as common ground for joint action

When performing joint actions, people rely on common ground - shared information that provides the required basis for mutual understanding. Common ground can be based on people's interaction history or on knowledge and expectations people share, e.g., because they belong to the same culture or social class. Here, we suggest that people rely on yet another form of common ground, one that originates in their similarities in multisensory processing. Specifically, we focus on 'crossmodal correspondences' - nonarbitrary associations that people make between stimulus features in different sensory modalities, e.g., between stimuli in the auditory and the visual modality such as high-pitched sounds and small objects. Going beyond previous research that focused on investigating crossmodal correspondences in individuals, we propose that people can use these correspondences for communicating and coordinating with others. Initial support for our proposal comes from a communication game played in a public space (an art gallery) by pairs of visitors. We observed that pairs created nonverbal communication systems by spontaneously relying on 'crossmodal common ground'. Based on these results, we conclude that crossmodal correspondences not only occur within individuals but that they can also be actively used in joint action to facilitate the coordination between individuals.

Infants understand collaboration: Neural evidence for 9-month-olds' attribution of shared goals to coordinated joint actions

Interpreting others' actions as goal-directed, even when the actions are unfamiliar, is indispensable for social learning, and can be particularly important for infants, whose own action repertoire is limited. Indeed, young infants have been shown to attribute goals to unfamiliar actions as early as 3 months of age, but this ability appears restricted to actions performed by individuals. In contrast, attributing shared goals to actions performed by multiple individuals seems to emerge only in the second year of life. Considering the restrictions that this would impose on infants' understanding and learning from interactions in their environment, we reexamine this ability by introducing 9-month-old infants to simple joint actions, in which two agents coordinate their actions toward the same goal. To establish whether infants formed an expectation about future actions of these agents, infants' cortical activity was measured using functional near-infrared spectroscopy (fNIRS). The hemodynamic response, recorded in (p)STS, indicated that infants attributed goals to simultaneous and coordinated joint actions of two individuals. Thus, even prior to actively engaging in collaborative activities themselves, infants can attribute shared goals to observed joint actions, enabling infants to learn from, and about, the complementary roles of social interactions, a central characteristic of human culture.

The communicative advantage: how kinematic signaling supports semantic comprehension

Humans are unique in their ability to communicate information through representational gestures which visually simulate an action (eg. moving hands as if opening a jar). Previous research indicates that the intention to communicate modulates the kinematics (e.g., velocity, size) of such gestures. If and how this modulation influences addressees' comprehension of gestures have not been investigated. Here we ask whether communicative kinematic modulation enhances semantic comprehension (i.e., identification) of gestures. We additionally investigate whether any comprehension advantage is due to enhanced early identification or late identification. Participants (n = 20) watched videos of representational gestures produced in a more- (n = 60) or less-communicative (n = 60) context and performed a forced-choice recognition task. We tested the isolated role of kinematics by removing visibility of actor's faces in Experiment I, and by reducing the stimuli to stick-light figures in Experiment II. Three video lengths were used to disentangle early identification from late identification. Accuracy and response time quantified main effects. Kinematic modulation was tested for correlations with task performance. We found higher gesture identification performance in more- compared to less-communicative gestures. However, early identification was only enhanced within a full visual context, while late identification occurred even when viewing isolated kinematics. Additionally, temporally segmented acts with more post-stroke holds were associated with higher accuracy. Our results demonstrate that communicative signaling, interacting with other visual cues, generally supports gesture identification, while kinematic modulation specifically enhances late identification in the absence of other cues. Results provide insights into mutual understanding processes as well as creating artificial communicative agents.

An approach to social flexibility: Congruency effects during spontaneous word-by-word interaction

Cognitive flexibility is the ability to switch between different concepts or to adapt goal-directed behavior in a changing environment. Although, cognitive research on this ability has long been focused on the individual mind, it is becoming increasingly clear that cognitive flexibility plays a central role in our social life. This is particularly evident in turn-taking in verbal conversation, where cognitive flexibility of the individual becomes part of social flexibility in the dyadic interaction. In this work, we introduce a model that reveals different parameters that explain how people flexibly handle unexpected events in verbal conversation. In order to study hypotheses derived from the model, we use a novel experimental approach in which thirty pairs of participants engaged in a word-by-word interaction by taking turns in generating sentences word by word. Similar to well established individual cognitive tasks, participants needed to adapt their behavior in order to respond to their co-actor’s last utterance. With our experimental approach we could manipulate the interaction between participants: Either both participants had to construct a sentence with a common target word (congruent condition) or with distinct target words (incongruent condition). We further studied the relation between the interactive Word-by-Word task measures and classical individual-centered, cognitive tasks, namely the Number-Letter task, the Stop-Signal task, and the GoNogo task. In the Word-by-Word task, we found that participants had faster response times in congruent compared to incongruent trials, which replicates the primary findings of standard cognitive tasks measuring cognitive flexibility. Further, we found a significant correlation between the performance in the Word-by-Word task and the Stop-Signal task indicating that participants with a high cognitive flexibility in the Word-by-Word task also showed high inhibition control.

Social epistemic actions

We consider the ways humans engage in social epistemic actions, to guide each other's attention, prediction, and learning processes towards salient information, at the timescale of online social interaction and joint action. This parallels the active guidance of other's attention, prediction, and learning processes at the longer timescale of niche construction and cultural practices, as discussed in the target article.

Informing, Coordinating, and Performing: A Perspective on Functions of Sensorimotor Communication

Sensorimotor communication is a form of communication instantiated through body movements that are guided by both instrumental, goal-directed intentions and communicative, social intentions. Depending on the social interaction context, sensorimotor communication can serve different functions. This article aims to disentangle three of these functions: (a) an informing function of body movements, to highlight action intentions for an observer; (b) a coordinating function of body movements, to facilitate real-time action prediction in joint action; and (c) a performing function of body movements, to elicit emotional or aesthetic experiences in an audience. We provide examples of research addressing these different functions as well as some influencing factors, relating to individual differences, task characteristics, and situational demands. The article concludes by discussing the benefits of a closer dialog between separate lines of research on sensorimotor communication across different social contexts.

Motion tracking of parents' infant- versus adult-directed actions reveals general and action-specific modulations

Parents tend to modulate their movements when demonstrating actions to their infants. Thus far, these modulations have primarily been quantified by human raters and for entire interactions, thereby possibly overlooking the intricacy of such demonstrations. Using optical motion tracking, the precise modulations of parents' infant-directed actions were quantified and compared to adult-directed actions and between action types. Parents demonstrated four novel objects to their 14-month-old infants and adult confederates. Each object required a specific action to produce a unique effect (e.g. rattling). Parents were asked to demonstrate an object at least once before passing it to their demonstration partner, and they were subsequently free to exchange the object as often as desired. Infants' success at producing the objects' action-effects was coded during the demonstration session and their memory of the action-effects was tested after a several-minute delay. Indicating general modulations across actions, parents repeated demonstrations more often, performed the actions in closer proximity and demonstrated action-effects for longer when interacting with their infant compared to the adults. Meanwhile, modulations of movement size and velocity were specific to certain action-effect pairs. Furthermore, a 'just right' modulation of proximity was detected, since infants' learning, memory, and parents' prior evaluations of their infants' motor abilities, were related to demonstrations that were performed neither too far from nor too close to the infants. Together, these findings indicate that infant-directed action modulations are not solely overall exaggerations but are dependent upon the characteristics of the to-be learned actions, their effects, and the infant learners.

This is for you: Social modulations of proximal vs. distal space in collaborative interaction

Human spatial representations are shaped by affordances for action offered by the environment. A prototypical example is the organization of space into peripersonal (within reach) and extrapersonal (outside reach) regions, mirrored by proximal (this/here) and distal (that/there) linguistic expressions. The peri-/extrapersonal distinction has been widely investigated in individual contexts, but little is known about how spatial representations are modulated by interaction with other people. Is near/far coding of space dynamically adapted to the position of a partner when space, objects, and action goals are shared? Over two preregistered experiments based on a novel interactive paradigm, we show that, in individual and social contexts involving no direct collaboration, linguistic coding of locations as proximal or distal depends on their distance from the speaker's hand. In contrast, in the context of collaborative interactions involving turn-taking and role reversal, proximal space is shifted towards the partner, and linguistic coding of near space ('this' / 'here') is remapped onto the partner's action space.

Making the Environment an Informative Place: A Conceptual Analysis of Epistemic Policies and Sensorimotor Coordination

How do living organisms decide and act with limited and uncertain information? Here, we discuss two computational approaches to solving these challenging problems: a "cognitive" and a "sensorimotor" enrichment of stimuli, respectively. In both approaches, the key notion is that agents can strategically modulate their behavior in informative ways, e.g., to disambiguate amongst alternative hypotheses or to favor the perception of stimuli providing the information necessary to later act appropriately. We discuss how, despite their differences, both approaches appeal to the notion that actions must obey both epistemic (i.e., information-gathering or uncertainty-reducing) and pragmatic (i.e., goal- or reward-maximizing) imperatives and balance them. Our computationally-guided analysis reveals that epistemic behavior is fundamental to understanding several facets of cognitive processing, including perception, decision making, and social interaction.

Role-Specific Brain Activations in Leaders and Followers During Joint Action

Much of social interaction in human life requires that individuals perform different roles during joint actions, the most basic distinction being that between a leader and a follower. A number of neuroimaging studies have examined the brain networks for leading and following, but none have examined what effect prior expertise at these roles has on brain activations during joint motor tasks. Couple dancers (e.g., dancers of Tango, Salsa, and swing) are an ideal population in which examine such effects, since leaders and followers of partnered dances have similar overall levels of motor expertise at dancing, but can differ strikingly in their role-specific skill sets. To explore role-specific expertise effects on brain activations for the first time, we recruited nine skilled leaders and nine skilled followers of couple dances for a functional magnetic resonance imaging study. We employed a two-person scanning arrangement that allowed a more naturalistic interaction between two individuals. The dancers interacted physically with an experimenter standing next to the bore of the magnet so as to permit bimanual partnered movements. Together, they alternated between leading and following the joint movements. The results demonstrated that the brain activations during the acts of leading and following were enhanced by prior expertise at being a leader or follower, and that activity in task-specific brain areas tended to be positively correlated with the level of expertise at the corresponding role. These findings provide preliminary evidence that training at one role of a joint motor task can selectively enhance role-related brain activations.

Peripheral Network Connectivity Analyses for the Real-Time Tracking of Coupled Bodies in Motion

Dyadic interactions are ubiquitous in our lives, yet they are highly challenging to study. Many subtle aspects of coupled bodily dynamics continuously unfolding during such exchanges have not been empirically parameterized. As such, we have no formal statistical methods to describe the spontaneously self-emerging coordinating synergies within each actor’s body and across the dyad. Such cohesive motion patterns self-emerge and dissolve largely beneath the awareness of the actors and the observers. Consequently, hand coding methods may miss latent aspects of the phenomena. The present paper addresses this gap and provides new methods to quantify the moment-by-moment evolution of self-emerging cohesiveness during highly complex ballet routines. We use weighted directed graphs to represent the dyads as dynamically coupled networks unfolding in real-time, with activities captured by a grid of wearable sensors distributed across the dancers’ bodies. We introduce new visualization tools, signal parameterizations, and a statistical platform that integrates connectivity metrics with stochastic analyses to automatically detect coordination patterns and self-emerging cohesive coupling as they unfold in real-time. Potential applications of these new techniques are discussed in the context of personalized medicine, basic research, and the performing arts.

Identifying others' informative intentions from movement kinematics

Previous research has demonstrated that people can reliably distinguish between actions with different instrumental intentions on the basis of the kinematic signatures of these actions (Cavallo, Koul, Ansuini, Capozzi, & Becchio, 2016). It has also been demonstrated that different informative intentions result in distinct action kinematics (McEllin, Knoblich, & Sebanz, 2017). However, it is unknown whether people can discriminate between instrumental actions and actions performed with an informative intention, and between actions performed with different informative intentions, on the basis of kinematic cues produced in these actions. We addressed these questions using a visual discrimination paradigm in which participants were presented with point light animations of an actor playing a virtual xylophone. We systematically manipulated and amplified kinematic parameters that have been shown to reflect different informative intentions. We found that participants reliably used both spatial and temporal cues in order to discriminate between instrumental actions and actions performed with an informative intention, and between actions performed with different informative intentions. Our findings indicate that the informative cues produced in joint action and teaching go beyond serving a general informative purpose and can be used to infer specific informative intentions.

Communicative intent modulates production and comprehension of actions and gestures: A Kinect study

Actions may be used to directly act on the world around us, or as a means of communication. Effective communication requires the addressee to recognize the act as being communicative. Humans are sensitive to ostensive communicative cues, such as direct eye gaze (Csibra & Gergely, 2009). However, there may be additional cues present in the action or gesture itself. Here we investigate features that characterize the initiation of a communicative interaction in both production and comprehension. We asked 40 participants to perform 31 pairs of object-directed actions and representational gestures in more- or less- communicative contexts. Data were collected using motion capture technology for kinematics and video recording for eye-gaze. With these data, we focused on two issues. First, if and how actions and gestures are systematically modulated when performed in a communicative context. Second, if observers exploit such kinematic information to classify an act as communicative. Our study showed that during production the communicative context modulates space-time dimensions of kinematics and elicits an increase in addressee-directed eye-gaze. Naïve participants detected communicative intent in actions and gestures preferentially using eye-gaze information, only utilizing kinematic information when eye-gaze was unavailable. Our study highlights the general communicative modulation of action and gesture kinematics during production but also shows that addressees only exploit this modulation to recognize communicative intention in the absence of eye-gaze. We discuss these findings in terms of distinctive but potentially overlapping functions of addressee directed eye-gaze and kinematic modulations within the wider context of human communication and learning.

When Height Carries Weight: Communicating Hidden Object Properties for Joint Action

In the absence of pre-established communicative conventions, people create novel communication systems to successfully coordinate their actions toward a joint goal. In this study, we address two types of such novel communication systems: sensorimotor communication, where the kinematics of instrumental actions are systematically modulated, versus symbolic communication. We ask which of the two systems co-actors preferentially create when aiming to communicate about hidden object properties such as weight. The results of three experiments consistently show that actors who knew the weight of an object transmitted this weight information to their uninformed co-actors by systematically modulating their instrumental actions, grasping objects of particular weights at particular heights. This preference for sensorimotor communication was reduced in a fourth experiment where co-actors could communicate with weight-related symbols. Our findings demonstrate that the use of sensorimotor communication extends beyond the communication of spatial locations to non-spatial, hidden object properties.

Taking two to tango: fMRI analysis of improvised joint action with physical contact

Many forms of joint action involve physical coupling between the participants, such as when moving a sofa together or dancing a tango. We report the results of a novel two-person functional MRI study in which trained couple dancers engaged in bimanual contact with an experimenter standing next to the bore of the magnet, and in which the two alternated between being the leader and the follower of joint improvised movements. Leading showed a general pattern of self-orientation, being associated with brain areas involved in motor planning, navigation, sequencing, action monitoring, and error correction. In contrast, following showed a far more sensory, externally-oriented pattern, revealing areas involved in somatosensation, proprioception, motion tracking, social cognition, and outcome monitoring. We also had participants perform a "mutual" condition in which the movement patterns were pre-learned and the roles were symmetric, thereby minimizing any tendency toward either leading or following. The mutual condition showed greater activity in brain areas involved in mentalizing and social reward than did leading or following. Finally, the analysis of improvisation revealed the dual importance of motor-planning and working-memory areas. We discuss these results in terms of theories of both joint action and improvisation.

Negotiating between individual and joint goals in ensemble musical performance

Successful joint action requires negotiation, especially in the event of goal incongruence. This article addresses goal incongruence in joint musical performance by manipulating the congruence of score instructions (congruent/incongruent) regarding tempo (speed) and dynamics (sound intensity) given to piano duos. The aim is to investigate how co-performers negotiate incongruent instructions for tempo and dynamics by balancing the prioritisation of individual goals versus the joint outcome and how this negotiation is modulated by musical expertise and personality (locus of control). In total, 14 pairs of pianists, who were not informed of the congruence manipulation, were placed back-to-back and were directed to achieve a successful performance over four repeated performances without verbal communication. Interpersonal coordination generally improved from the first to final performance in the congruent and incongruent conditions for both the tempo and dynamics tasks. Furthermore, in incongruent conditions, results suggest that performers prioritise the joint performance in the tempo task, but prioritise their own performance in the dynamics task. Although individual performance appears to be modulated by musical expertise, the balance of individual/joint performance across the duo is not influenced by musical part (melody vs accompaniment), expressive instruction, musical expertise or locus of control.

A Common Force-Sharing Pattern in Joint Action That Consists of Four People

The authors examined the force-sharing patterns in a joint action performed by a group of two, three, or four people compared with a solo action. In the joint actions, 28 participants produced periodic isometric forces such that the sum of forces they produced cycled between 5% and 10% maximum voluntary contraction with the right hand at 1 Hz. In both the three- and four-person tasks, the correlation between forces produced by two of the three or four participants was negative, and the remaining one or two participants produced intermediate forces. The errors of force and interval and force variabilities were smaller in four- and three-people groups than individuals. Four- and three-people groups thus performed better than individuals.

Hyper-Transcranial Alternating Current Stimulation: Experimental Manipulation of Inter-Brain Synchrony

We walk together, we watch together, we win together: Interpersonally coordinated actions are omnipresent in everyday life, yet the associated neural mechanisms are not well understood. Available evidence suggests that the synchronization of oscillatory activity across brains may provide a mechanism for the temporal alignment of actions between two or more individuals. In an attempt to provide a direct test of this hypothesis, we applied transcranial alternating current stimulation simultaneously to two individuals (hyper-tACS) who were asked to drum in synchrony at a set pace. Thirty-eight female-female dyads performed the dyadic drumming in the course of 3 weeks under three different hyper-tACS stimulation conditions: same-phase-same-frequency; different-phase-different-frequency; sham. Based on available evidence and theoretical considerations, stimulation was applied over right frontal and parietal sites in the theta frequency range. We predicted that same-phase-same-frequency stimulation would improve interpersonal action coordination, expressed as the degree of synchrony in dyadic drumming, relative to the other two conditions. Contrary to expectations, both the same-phase-same-frequency and the different-phase-different-frequency conditions were associated with greater dyadic drumming asynchrony relative to the sham condition. No influence of hyper-tACS on behavioral performance was seen when participants were asked to drum separately in synchrony to a metronome. Individual and dyad preferred drumming tempo was also unaffected by hyper-tACS. We discuss limitations of the present version of the hyper-tACS paradigm, and suggest avenues for future research.

Where Are You Throwing the Ball? I Better Watch Your Body, Not Just Your Arm!

The ability to intercept or avoid a moving object, whether to catch a ball, snatch one's prey, or avoid the path of a predator, is a skill that has been acquired throughout evolution by many species in the animal kingdom. This requires processing early visual cues in order to program anticipatory motor responses tuned to the forthcoming event. Here, we explore the nature of the early kinematics cues that could inform an observer about the future direction of a ball projected with an unconstrained overarm throw. Our goal was to pinpoint the body segments that, throughout the temporal course of the throwing action, could provide key cues for accurately predicting the side of the outgoing ball. We recorded whole-body kinematics from twenty non-expert participants performing unconstrained overarm throws at four different targets placed on a vertical plane at 6 m distance. In order to characterize the spatiotemporal structure of the information embedded in the kinematics of the throwing action about the outgoing ball direction, we introduced a novel combination of dimensionality reduction and machine learning techniques. The recorded kinematics clearly shows that throwing styles differed considerably across individuals, with corresponding inter-individual differences in the spatio-temporal structure of the thrower predictability. We found that for most participants it is possible to predict the region where the ball hit the target plane, with an accuracy above 80%, as early as 400-500 ms before ball release. Interestingly, the body parts that provided the most informative cues about the action outcome varied with the throwing style and during the time course of the throwing action. Not surprisingly, at the very end of the action, the throwing arm is the most informative body segment. However, cues allowing for predictions to be made earlier than 200 ms before release are typically associated to other body parts, such as the lower limbs and the contralateral arm. These findings are discussed in the context of the sport-science literature on throwing and catching interactive tasks, as well as from the wider perspective of the role of sensorimotor coupling in interpersonal social interactions.

Following during physically-coupled joint action engages motion area MT+/V5

Interpersonal coordination during joint action depends on the perception of the partner's movements. In many such situations - for example, while moving furniture together or dancing a tango - there are kinesthetic interactions between the partners due to the forces shared between them that allow them to directly perceive one another's movements. Joint action of this type often involves a contrast between the roles of leader and follower, where the leader imparts forces onto the follower, and the follower has to be responsive to these force-cues during movement. We carried out a novel 2-person functional MRI study with trained couple dancers engaged in bimanual contact with an experimenter standing next to the bore of the magnet, where the two alternated between being the leader and follower of joint improvised movements, all with the eyes closed. One brain area that was unexpectedly more active during following than leading was the region of MT+/V5. While classically described as an area for processing visual motion, it has more recently been shown to be responsive to tactile motion as well. We suggest that MT+/V5 responds to motion based on force-cues during joint haptic interaction, most especially when a follower responds to force-cues coming from a leader's movements.

Autistic traits affect interpersonal motor coordination by modulating strategic use of role-based behavior

Background

Despite the fact that deficits in social communication and interaction are at the core of Autism Spectrum Conditions (ASC), no study has yet tested individuals on a continuum from neurotypical development to autism in an on-line, cooperative, joint action task. In our study, we aimed to assess whether the degree of autistic traits affects participants' ability to modulate their motor behavior while interacting in a Joint Grasping task and according to their given role.

Methods

Sixteen pairs of adult participants played a cooperative social interactive game in which they had to synchronize their reach-to-grasp movements. Pairs were comprised of one ASC and one neurotypical with no cognitive disability. In alternate experimental blocks, one participant knew what action to perform (instructed role) while the other had to infer it from his/her partner’s action (adaptive role). When in the adaptive condition, participants were told to respond with an action that was either opposite or similar to their partner. Participants also played a non-social control game in which they had to synchronize with a non-biological stimulus.

Results

In the social interactive task, higher degree of autistic traits predicted less ability to modulate joint action according to one’s interactive role. In the non-social task, autistic traits did not predict differences in movement preparation and planning, thus ruling out the possibility that social interactive task results were due to basic motor or executive function difficulties. Furthermore, when participants played the non-social game, the higher their autistic traits, the more they were interfered by the non-biological stimulus.

Conclusions

Our study shows for the first time that high autistic traits predict a stereotypical interaction style when individuals are required to modulate their movements in order to coordinate with their partner according to their role in a joint action task. Specifically, the infrequent emergence of role-based motor behavior modulation during on-line motor cooperation in participants with high autistic traits sheds light on the numerous difficulties ASC have in nonverbal social interactions.

Electronic supplementary material

The online version of this article (doi:10.1186/s13229-017-0141-0) contains supplementary material, which is available to authorized users.

Let's decide together: Differences between individual and joint delay discounting

This study addressed the question whether or not social collaboration has an effect on delay discounting, the tendency to prefer sooner but smaller over later but larger delivered rewards. We applied a novel paradigm in which participants executed choices between two gains in an individual and in a dyadic decision-making condition. We observed how participants reached mutual consent via joystick movement coordination and found lower discounting and a higher decisions’ efficiency. In order to establish the underlying mechanism for dyadic variation, we further tested whether these differences emerge from social facilitation or inner group interchange.

Vision adds to haptics when dyads perform a whole-body joint balance task

When two or more people aim to produce joint action outcomes they need to coordinate their individual actions in space and time. Successful joint action performance has been reported to depend, among others, on visual and somatosensory information provided to the joint actors. This study investigated whether and how the systematic manipulation of visual information modulates real-time joint action when dyads performed a whole-body joint balance task. To this end, we introduced the Joint Action Board (JAB) where partners guided a ball through a maze towards a virtual hole by jointly shifting their weight on the board under three visual conditions: (1) the Follower had neither visual access to the Leader nor to the maze; (2) the Follower had no visual access to the maze but to the Leader; (3) the Follower had full visual access to both the Leader and to the maze. Joint action performance was measured as completion time of the maze task; interpersonal coordination was examined by means of kinematic analyses of both partners' motor behaviour. We predicted that systematically adding visual to the available haptic information would result in a significant increase in joint performance and that Leaders would change their coordination behavior depending on these conditions. Results showed that adding visual information to haptics led to an increase in joint action performance in a Leader-Follower relationship in a joint balance task. In addition, interpersonal coordination behavior (i.e. sway range of motion, time-lag between partner's bodies etc.) changed dependent on the provided visual information between partners in the jointly executed task.

You cannot speak and listen at the same time: a probabilistic model of turn-taking

Turn-taking is a preverbal skill whose mastering constitutes an important precondition for many social interactions and joint actions. However, the cognitive mechanisms supporting turn-taking abilities are still poorly understood. Here, we propose a computational analysis of turn-taking in terms of two general mechanisms supporting joint actions: action prediction (e.g., recognizing the interlocutor's message and predicting the end of turn) and signaling (e.g., modifying one's own speech to make it more predictable and discriminable). We test the hypothesis that in a simulated conversational scenario dyads using these two mechanisms can recognize the utterances of their co-actors faster, which in turn permits them to give and take turns more efficiently. Furthermore, we discuss how turn-taking dynamics depend on the fact that agents cannot simultaneously use their internal models for both action (or messages) prediction and production, as these have different requirements-or, in other words, they cannot speak and listen at the same time with the same level of accuracy. Our results provide a computational-level characterization of turn-taking in terms of cognitive mechanisms of action prediction and signaling that are shared across various interaction and joint action domains.

Effects of Agent-Environment Symmetry on the Coordination Dynamics of Triadic Jumping

We investigated whether the patterns of coordination that emerged during a three-participant (triadic) jumping task were defined by the symmetries of the (multi) agent-environment task space. Triads were instructed to jump around different geometrical arrangements of hoops. The symmetry of the hoop geometry was manipulated to create two symmetrical and two asymmetrical participant-hoop configurations. Video and motion tracking recordings were employed to determine the frequencies of coordination misses (collisions or failed jumps) and during 20 successful jump sequences, the jump direction chosen (clockwise vs. counterclockwise) and the patterning of between participant temporal movement lags within and across jump events. The results revealed that the (a)symmetry of the joint action workspace significantly influenced the (a)symmetry of the jump direction dynamics and, more importantly, the (a)symmetry of the between participant coordination lags. The symmetrical participant-hoop configurations resulted in smaller overall movement lags and a more spontaneous, interchangeable leader/follower relationship between participants, whereas the asymmetrical participant-hoop configurations resulted in slightly larger overall movements lags and a more explicit, persistent asymmetry in the leader/follower relationship of participants. The degree to which the patterns of behavioral coordination that emerged were consistent with the theory of symmetry groups and spontaneous and explicit symmetry-breaking are discussed.

Anisotropy and Antagonism in the Coupling of Two Oscillators: Concepts and Applications for Between-Person Coordination

Coupled oscillators provide a pertinent model approach to study between-person movement dynamics. While ample literature in this respect has considered the influence of external/environmental constraints and/or effects of a difference between the two agents' individual component dynamics (e.g., mismatch in natural frequency), recent studies also started to more directly consider the interaction per-se. The current perspective paper sets forth that while movement coordination dynamics has mainly been studied alongside a model in which the coupling is considered isotropic (i.e., symmetrical; both oscillators coupled to same degree) or strictly unidirectional (e.g., for moving to a given external rhythm), between-agent coupling involves a natural anisotropy: components influence each other bidirectionally to different degrees. Furthermore, recent research from different areas has considered so-called antagonistic or "competitive" coupling, which refers to the idea that one component is positively coupled to the other (attractive interaction), while the coupling in the other direction is negative (repulsive interaction). Although the latter would be rather tricky to address in within-person coordination, it does have strong applications and implications for between-person dynamics, for instance in the study of competitive interactions in sports situations (e.g., attacker-defender) and conflicting social (movement) interactions. The paper concludes by offering a conceptual framework and perspectives for future studies on the dynamic anisotropic nature of the interaction in between-person contexts.

Joint Action: Mental Representations, Shared Information and General Mechanisms for Coordinating with Others

In joint action, multiple people coordinate their actions to perform a task together. This often requires precise temporal and spatial coordination. How do co-actors achieve this? How do they coordinate their actions toward a shared task goal? Here, we provide an overview of the mental representations involved in joint action, discuss how co-actors share sensorimotor information and what general mechanisms support coordination with others. By deliberately extending the review to aspects such as the cultural context in which a joint action takes place, we pay tribute to the complex and variable nature of this social phenomenon.

Spontaneous adaptation explains why people act faster when being imitated

The human ability to perform joint actions is often attributed to high-level cognitive processes. For example, the finding that action leaders act faster when imitated by their partners has been interpreted as evidence for anticipation of the other's actions (Pfister, Dignath, Hommel, & Kunde, 2013). In two experiments, we showed that a low-level mechanism can account for this finding. Action leaders were faster when imitated than when counterimitated, but only if they could observe their partner's actions (Exp. 1). Crucially, when due to our manipulation the partner's imitative actions became slower than the counterimitative actions, leaders also became slower when they were imitated, and faster when counterimitated (Exp. 2). Our results suggest that spontaneous temporal adaptation is a key mechanism in joint action tasks. We argue for a reconsideration of other phenomena that have traditionally been attributed solely to high-level processes.