Prediction in joint action: what, when, and where

Are We in Time? How Predictive Coding and Dynamical Systems Explain Musical Synchrony

Humans tend to anticipate events when they synchronize their actions with sound (such as when they clap to music), which has puzzled scientists for decades. What accounts for this anticipation? We review two theoretical mechanisms for synchrony: predictive coding and dynamical systems. Both theories are grounded in neural activation patterns, but there are important distinctions. We contrast their assumptions, their computations, and their musical applications to anticipatory synchronization.

Effect of Social Context on Cognitive and Motor Behavior: A Systematic Review

Human cognitive and motor behavior is influenced by the social contexts. The aim of this systematic review is to investigate the impact of the social contexts on human behaviors. A systematic search of the literature was performed via Pub-Med/Medline, Web of sciences, Google scholar, Science direct, Springer-Link and EMBASE and 68 articles were selected. After applying all the inclusion and exclusion criteria, 16 articles were retained. The results show that the presence of other people and the social context influence motor behavior (i.e. movement duration, trajectory behavior, maximum speed) and cognitive behavior (reaction time). Studies have shown an improvement in performance in the presence of other people compared to the individual situation. However, other studies showed that the presence of other people led to deterioration in performance compared to the individual situation. The improvement of behavior is attributed to the social phenomenon of facilitation while the deterioration was explained by the conduct theory or the distraction conflict theory. These social phenomena of facilitation or inhibition could be related to the perception-action theory, which interferes with interaction with other. This, in turn, seems to be associated with neural circuits of mirror neurons and motor system.

Music in the brain

Music is ubiquitous across human cultures - as a source of affective and pleasurable experience, moving us both physically and emotionally - and learning to play music shapes both brain structure and brain function. Music processing in the brain - namely, the perception of melody, harmony and rhythm - has traditionally been studied as an auditory phenomenon using passive listening paradigms. However, when listening to music, we actively generate predictions about what is likely to happen next. This enactive aspect has led to a more comprehensive understanding of music processing involving brain structures implicated in action, emotion and learning. Here we review the cognitive neuroscience literature of music perception. We show that music perception, action, emotion and learning all rest on the human brain's fundamental capacity for prediction - as formulated by the predictive coding of music model. This Review elucidates how this formulation of music perception and expertise in individuals can be extended to account for the dynamics and underlying brain mechanisms of collective music making. This in turn has important implications for human creativity as evinced by music improvisation. These recent advances shed new light on what makes music meaningful from a neuroscientific perspective.

Isolating Action Prediction from Action Integration in the Perception of Social Interactions

Previous research suggests that predictive mechanisms are essential in perceiving social interactions. However, these studies did not isolate action prediction (a priori expectations about how partners in an interaction react to one another) from action integration (a posteriori processing of both partner's actions). This study investigated action prediction during social interactions while controlling for integration confounds. Twenty participants viewed 3D animations depicting an action-reaction interaction between two actors. At the start of each action-reaction interaction, one actor performs a social action. Immediately after, instead of presenting the other actor's reaction, a black screen covers the animation for a short time (occlusion duration) until a still frame depicting a precise moment of the reaction is shown (reaction frame). The moment shown in the reaction frame is either temporally aligned with the occlusion duration or deviates by 150 ms or 300 ms. Fifty percent of the action-reaction trials were semantically congruent, and the remaining were incongruent, e.g., one actor offers to shake hands, and the other reciprocally shakes their hand (congruent action-reaction) versus one actor offers to shake hands, and the other leans down (incongruent action-reaction). Participants made fast congruency judgments. We hypothesized that judging the congruency of action-reaction sequences is aided by temporal predictions. The findings supported this hypothesis; linear speed-accuracy scores showed that congruency judgments were facilitated by a temporally aligned occlusion duration, and reaction frames compared to 300 ms deviations, thus suggesting that observers internally simulate the temporal unfolding of an observed social interction. Furthermore, we explored the link between participants with higher autistic traits and their sensitivity to temporal deviations. Overall, the study offers new evidence of prediction mechanisms underpinning the perception of social interactions in isolation from action integration confounds.

Liking as a balance between synchronization, complexity and novelty

Synchronization has been identified as a key aspect in social bonding. While synchronization could be maximized by increasing the predictability of an interaction, such predictability is in tension with individuals’ level of interest, which is tied to the interaction’s complexity and novelty. In this study, we tested the interplay between synchronization and interest. We asked 104 female dyads to play the Mirror Game, in which they had to move their hands as coordinately as possible, and then report how much they liked each other. Utilizing information theory and video processing tools, we found that a combination of movement synchronization and complexity explained liking almost two times better than movement synchronization alone. Moreover, we found that people initiated novel and challenging interactions, even though they paid a price—being less synchronized. Examining the interactions’ dynamics, we found that people who liked each other moved in a more synchronized, complex, and novel manner during most of the interaction. This suggests that in addition to synchronization, maintaining interest may be critical for positive social bonding. Thus, we propose a new framework in which balancing synchronization and interest, rather than merely maximizing synchronization, optimizes the interaction quality.

Addressing joint action challenges in HRI: Insights from psychology and philosophy

The vast expansion of research in human-robot interactions (HRI) these last decades has been accompanied by the design of increasingly skilled robots for engaging in joint actions with humans. However, these advances have encountered significant challenges to ensure fluent interactions and sustain human motivation through the different steps of joint action. After exploring current literature on joint action in HRI, leading to a more precise definition of these challenges, the present article proposes some perspectives borrowed from psychology and philosophy showing the key role of communication in human interactions. From mutual recognition between individuals to the expression of commitment and social expectations, we argue that communicative cues can facilitate coordination, prediction, and motivation in the context of joint action. The description of several notions thus suggests that some communicative capacities can be implemented in the context of joint action for HRI, leading to an integrated perspective of robotic communication.

Joint rushing alters internal timekeeping in non-musicians and musicians

Recent studies have shown that people engaging in joint rhythmic activity unintentionally increase their tempo. The same tempo increase does not occur when the same rhythmic activity is performed alone. This phenomenon is known as joint rushing. In two experiments, we investigated whether joint rushing is caused by correction mechanisms that facilitate sensorimotor synchronization. Because such correction mechanisms require perceptual input, joint rushing should discontinue when auditory feedback in a joint rhythmic activity is interrupted. This prediction was clearly supported in two experiments, one with musicians and one with non-musicians. Surprisingly, there was no indication that the amount of joint rushing differed between musicians and non-musicians. Furthermore, neither musicians nor non-musicians were able to return to the initially instructed tempo after feedback had been interrupted. This result indicates that joint rushing has a lasting effect on an internal timekeeper. An important question for future research is whether joint rushing is only a dysfunctional side effect of the way sensorimotor synchronization works or whether it has a function in enabling precise temporal coordination between different individuals.

A Virtual Sandbox Approach to Studying the Effect of Augmented Communication on Human-Robot Collaboration

Human-Robot Collaboration (HRC) has the potential for a paradigm shift in industrial production by complementing the strengths of industrial robots with human staff. However, exploring these scenarios in physical experimental settings is costly and difficult, e.g., due to safety considerations. We present a virtual reality application that allows the exploration of HRC work arrangements with autonomous robots and their effect on human behavior. Prior experimental studies conducted using this application demonstrated the benefits of augmenting an autonomous robot arm with communication channels on subjective aspects such as perceived stress. Motivated by current safety regulations that hinder HRC to expand its full potential, we explored the effects of the augmented communication on objective measures (collision rate and produced goods) within a virtual sandbox application. Explored through a safe and replicable setup, the goal was to determine whether communication channels that provide guidance and explanation on the robot can help mitigate safety hazards without interfering with the production effectiveness of both parties. This is based on the theoretical foundation that communication channels enable the robot to explain its action, helps the human collaboration partner to comprehend the current state of the shared task better, and react accordingly. Focused on the optimization of production output, reduced collision rate, and increased perception of safety, a between-subjects experimental study with two conditions (augmented communication vs non-augmented) was conducted. The results revealed a statistically significant difference in terms of production quantity output and collisions with the robot, favoring the augmented conditions. Additional statistically significant differences regarding self-reported perceived safety were found. The results of this study provide an entry point for future research regarding the augmentation of industrial robots with communication channels for safety purposes.

Taking apart what brings us together: The role of action prediction, perspective-taking, and theory of mind in joint action

The ability to act together with others to achieve common goals is crucial in life, yet there is no full consensus on the underlying cognitive skills. While influential theoretical accounts suggest that interaction requires sophisticated insights into others' minds, alternative views propose that high-level social skills might not be necessary because interactions are grounded on sensorimotor predictive mechanisms. At present, empirical evidence is insufficient to decide between the two. This study addressed this issue and explored the association between performance at joint action tasks and cognitive abilities in three domains-action prediction, perspective-taking, and theory of mind-in healthy adults (N = 58). We found that, while perspective-taking played a role in reading the behaviour of others independently of the social context, action prediction abilities specifically influenced the agents' performance in an interactive task but not in a control (social but non-interactive) task. In our study, performance at a theory of mind test did not play any role, as confirmed by Bayesian analyses. The results suggest that, in adults, sensorimotor predictive mechanisms might play a significant and specific role in supporting interpersonal coordination during motor interactions. We discuss the implications of our findings for the contrasting theoretical views described earlier and propose a way they might be partly reconciled.

Robots facilitate human language production

Despite recent developments in integrating autonomous and human-like robots into many aspects of everyday life, social interactions with robots are still a challenge. Here, we focus on a central tool for social interaction: verbal communication. We assess the extent to which humans co-represent (simulate and predict) a robot's verbal actions. During a joint picture naming task, participants took turns in naming objects together with a social robot (Pepper, Softbank Robotics). Previous findings using this task with human partners revealed internal simulations on behalf of the partner down to the level of selecting words from the mental lexicon, reflected in partner-elicited inhibitory effects on subsequent naming. Here, with the robot, the partner-elicited inhibitory effects were not observed. Instead, naming was facilitated, as revealed by faster naming of word categories co-named with the robot. This facilitation suggests that robots, unlike humans, are not simulated down to the level of lexical selection. Instead, a robot's speaking appears to be simulated at the initial level of language production where the meaning of the verbal message is generated, resulting in facilitated language production due to conceptual priming. We conclude that robots facilitate core conceptualization processes when humans transform thoughts to language during speaking.

Frontoparietal, Cerebellum Network Codes for Accurate Intention Prediction in Altered Perceptual Conditions

Integrating and predicting the intentions and actions of others are critical components of social interactions, but the behavioral and neural bases of such mechanisms under altered perceptual conditions are poorly understood. In the present study, we recruited expert violinists and age-matched controls with no musical training and asked them to evaluate simplified dynamic stimuli of violinists playing in a piano or forte communicative intent while undergoing functional magnetic resonance imaging. We show that expertise is needed to successfully understand and evaluate communicative intentions in spatially and temporally altered visual representations of musical performance. Frontoparietal regions-such as the dorsolateral prefrontal cortex and the inferior parietal lobule and sulcus-and various subregions of the cerebellum-such as cerebellar lobules I-IV, V, VI, VIIb, VIIIa, X-a re recruited in the process. Functional connectivity between these brain areas reveals widespread organization, particularly in the dorsolateral prefrontal cortex, inferior frontal gyrus, inferior parietal sulcus, and in the cerebellum. This network may be essential to successfully assess communicative intent in ambiguous or complex visual scenes.

Sensory Attenuation of the Auditory P2 Differentiates Self- from Partner-Produced Sounds during Joint Action

Successful human interaction relies on people's ability to differentiate between the sensory consequences of their own and others' actions. Research in solo action contexts has identified sensory attenuation, that is, the selective perceptual or neural dampening of the sensory consequences of self-produced actions, as a potential marker of the distinction between self- and externally produced sensory consequences. However, very little research has examined whether sensory attenuation distinguishes self- from partner-produced sensory consequences in joint action contexts. The current study examined whether sensory attenuation of the auditory N1 or P2 ERPs distinguishes self- from partner-produced tones when pairs of people coordinate their actions to produce tone sequences that match a metronome pace. We did not find evidence of auditory N1 attenuation for either self- or partner-produced tones. Instead, the auditory P2 was attenuated for self-produced tones compared to partner-produced tones within the joint action. These findings indicate that self-specific attenuation of the auditory P2 differentiates the sensory consequences of one's own from others' actions during joint action. These findings also corroborate recent evidence that N1 attenuation may be driven by general rather than action-specific processes and support a recently proposed functional dissociation between auditory N1 and P2 attenuation.

Interactive brains, social minds: Neural and physiological mechanisms of interpersonal action coordination

It is now widely accepted that inter-brain synchronization is an important and inevitable mechanism of interpersonal action coordination and social interaction behavior. This review of the current literature focuses first on the forward model for interpersonal action coordination and functional system theory for biological systems, two broadly similar concepts for adaptive system behavior. Further, we review interacting-brain and/or hyper-brain dynamics studies, to show the interplay between intra- and inter-brain connectivity resulting in hyper-brain network structure and network topology dynamics, and consider the functioning of interacting brains as a superordinate system. The concept of a superordinate system, or superorganism, is then evaluated with respect to neuronal and physiological systems group dynamics, which show further accompanying mechanisms of interpersonal interaction. We note that fundamental problems need to be resolved to better understand the neural mechanisms of interpersonal action coordination.

Adaptive Prediction for Social Contexts: The Cerebellar Contribution to Typical and Atypical Social Behaviors

Social interactions involve processes ranging from face recognition to understanding others' intentions. To guide appropriate behavior in a given context, social interactions rely on accurately predicting the outcomes of one's actions and the thoughts of others. Because social interactions are inherently dynamic, these predictions must be continuously adapted. The neural correlates of social processing have largely focused on emotion, mentalizing, and reward networks, without integration of systems involved in prediction. The cerebellum forms predictive models to calibrate movements and adapt them to changing situations, and cerebellar predictive modeling is thought to extend to nonmotor behaviors. Primary cerebellar dysfunction can produce social deficits, and atypical cerebellar structure and function are reported in autism, which is characterized by social communication challenges and atypical predictive processing. We examine the evidence that cerebellar-mediated predictions and adaptation play important roles in social processes and argue that disruptions in these processes contribute to autism.

“I See What You Did There”

Unpredictability in robot behaviour can cause difficulties in interacting with robots. However, for social interactions with robots, a degree of unpredictability in robot behaviour may be desirable for facilitating engagement and increasing the attribution of mental states to the robot. To generate a better conceptual understanding of predictability, we looked at two facets of predictability, namely, the ability to predict robot actions and the association of predictability as an attribute of the robot. We carried out a video human-robot interaction study where we manipulated whether participants could either see the cause of a robot’s responsive action or could not see this, because there was no cause, or because we obstructed the visual cues. Our results indicate that when the cause of the robot’s responsive actions was not visible, participants rated the robot as more unpredictable and less competent, compared to when it was visible. The relationship between seeing the cause of the responsive actions and the attribution of competence was partially mediated by the attribution of unpredictability to the robot. We argue that the effects of unpredictability may be mitigated when the robot identifies when a person may not be aware of what the robot wants to respond to and uses additional actions to make its response predictable.

Simultaneous self-other integration and segregation support real-time interpersonal coordination in a musical joint action task

The ability to distinguish between an individual's own actions and those of another person is a requirement for successful joint action, particularly in domains such as group music making where precise interpersonal coordination ensures perceptual overlap in the effects of co-performers' actions. We tested the hypothesis that such coordination benefits from simultaneous integration and segregation of information about 'self' and 'other' in an experiment using a musical joint action paradigm. Sixteen pairs of individuals with little or no musical training performed a dyadic synchronization task on a pair of electronic music boxes. The relationship between pitches produced by paired participants (same vs. different) and the relationship between movement frequencies required to trigger synchronous tones (congruent vs. incongruent) were varied in a repeated measures design. The results indicate that interpersonal coordination was most accurate when sounds were different in pitch but movement frequency was congruent. Under other conditions, participants often drifted apart, resulting in poor coordination, especially with same sounds and incongruent movements across co-performers. These findings suggest that interpersonal coordination was facilitated when simultaneous self-other integration and segregation occurred across sensory modalities in an asymmetrical manner where pitch relations favoured segregation via auditory streaming while movement congruence favoured integration via visuo-motor coupling. Such self-other representational balance may enable co-performers to maintain autonomous control while attending, anticipating, and adapting to each other's timing when joint action requires precise temporal coordination.

Back-off

Standstill behavior by a robot is deemed to be ineffective and inefficient to convey a robot’s intention to yield priority to another party in spatial interaction. Instead, robots could convey their intention and thus their next action via motion. We developed a back-off (BO) movement to communicate the intention of yielding priority to pedestrians at bottlenecks. To evaluate human sensory perception and subjective legibility, the BO is compared to three other motion strategies in a video study with 167 interviewees at the university and public spaces, where it excels regarding legibility. Implemented in a real encounter, objective motion behavior of 78 participants as a reaction to a stop-and-wait strategy, and two versions of BO (short and long), shows an improvement of the pedestrians’ efficiency in the second encounter with the robot’s short BO version compared to the stop strategy. Eventually, in the third encounter with all motion strategies, interaction causes only a small time consumption still required by the cognitive process of perceiving an object in the visual field. Hence, the design of kinematic parameters, BO path and time, exhibits the potential to increase the fluency of an interaction with robots at bottlenecks.

Joint control of visually guided actions involves concordant increases in behavioural and neural coupling

It is often necessary for individuals to coordinate their actions with others. In the real world, joint actions rely on the direct observation of co-actors and rhythmic cues. But how are joint actions coordinated when such cues are unavailable? To address this question, we recorded brain activity while pairs of participants guided a cursor to a target either individually (solo control) or together with a partner (joint control) from whom they were physically and visibly separated. Behavioural patterns revealed that joint action involved real-time coordination between co-actors and improved accuracy for the lower performing co-actor. Concurrent neural recordings and eye tracking revealed that joint control affected cognitive processing across multiple stages. Joint control involved increases in both behavioural and neural coupling - both quantified as interpersonal correlations - peaking at action completion. Correspondingly, a neural offset response acted as a mechanism for and marker of interpersonal neural coupling, underpinning successful joint actions.

Adults Do Not Distinguish Action Intentions Based on Movement Kinematics Presented in Naturalistic Settings

Predicting others’ actions is an essential part of acting in the social world. Action kinematics have been proposed to be a cue about others’ intentions. It is still an open question as to whether adults can use kinematic information in naturalistic settings when presented as a part of a richer visual scene than previously examined. We investigated adults’ intention perceptions from kinematics using naturalistic stimuli in two experiments. In experiment 1, thirty participants watched grasp-to-drink and grasp-to-place movements and identified the movement intention (to drink or to place), whilst their mouth-opening muscle activity was measured with electromyography (EMG) to examine participants’ motor simulation of the observed actions. We found anecdotal evidence that participants could correctly identify the intentions from the action kinematics, although we found no evidence for increased activation of their mylohyoid muscle during the observation of grasp-to-drink compared to grasp-to-place actions. In pre-registered experiment 2, fifty participants completed the same task online. With the increased statistical power, we found strong evidence that participants were not able to discriminate intentions based on movement kinematics. Together, our findings suggest that the role of action kinematics in intention perception is more complex than previously assumed. Although previous research indicates that under certain circumstances observers can perceive and act upon intention-specific kinematic information, perceptual differences in everyday scenes or the observers’ ability to use kinematic information in more naturalistic scenes seems limited.

A Methodological Framework for Assessing Social Presence in Music Interactions in Virtual Reality

Virtual reality (VR) brings radical new possibilities to the empirical study of social music cognition and interaction. In the present article, we consider the role of VR as a research tool, based on its potential to create a sense of "social presence": the illusory feeling of being, and socially interacting, inside a virtual environment. This makes VR promising for bridging ecological validity ("research in the wild") and experimental control ("research in the lab") in empirical music research. A critical assumption however is the actual ability of VR to simulate real-life social interactions, either via human-embodied avatars or computer-controlled agents. The mediation of social musical interactions via VR is particularly challenging due to their embodied, complex, and emotionally delicate nature. In this article, we introduce a methodological framework to operationalize social presence by a combination of factors across interrelated layers, relating to the performance output, embodied co-regulation, and subjective experiences. This framework provides the basis for the proposal of a pragmatic approach to determine the level of social presence in virtual musical interactions, by comparing the outcomes across the multiple layers with the outcomes of corresponding real-life musical interactions. We applied and tested this pragmatic approach via a case-study of piano duet performances of the piece Piano Phase composed by Steve Reich. This case-study indicated that a piano duet performed in VR, in which the real-time interaction between pianists is mediated by embodied avatars, might lead to a strong feeling of social presence, as reflected in the measures of performance output, embodied co-regulation, and subjective experience. In contrast, although a piano duet in VR between an actual pianist and a computer-controlled agent led to a relatively successful performance output, it was inadequate in terms of both embodied co-regulation and subjective experience.

Interacting brains coming in sync through their minds: an interbrain neurofeedback study

Neurophysiological evidence shows that interpersonal action coordination is accompanied by interbrain synchronization (IBS). However, the functional significance of this association remains unclear. Using two experimental designs, we explored whether IBS is amenable to neurofeedback (NFB). Feedback was provided either as two balls approaching each other (so-called ball design), or as two pendula, each reflecting the oscillatory activity of one of the two participants (so-called pendulum design). The NFB was provided at delta (i.e., 2.5 Hz) and theta (i.e., 5 Hz) electroencephalography frequencies, and manipulated by enhanced and inverse feedback. We showed that the participants were able to increase IBS by using NFB, especially when it was fed back at the theta frequency. Apart from intra- and interbrain coupling, other oscillatory activities (e.g., power spectral density, peak amplitude, and peak frequency) also changed during the task compared with the rest. Moreover, all the measures showed specific correlations with the subjective postsurvey item scores, reflecting subjective feeling and appraisal. We conclude that the use of IBS for NFB might help in specifying the contribution of IBS to interpersonal action coordination and in providing important information about the neural mechanisms of social interaction and the causal dimension of IBS.

Adults, but not preschoolers or toddlers integrate situational constraints in their action anticipations: a developmental study on the flexibility of anticipatory gaze

Recent theories stress the role of situational information in understanding others' behaviour. For example, the predictive coding framework assumes that people take contextual information into account when anticipating other's actions. Likewise, the teleological stance theory assumes an early developing ability to consider situational constraints in action prediction. The current study investigates, over a wide age range, whether humans flexibly integrate situational constraints in their action anticipations. By means of an eye-tracking experiment, 2-year-olds, 5-year-olds, younger and older adults (together N = 181) observed an agent repeatedly taking one of two paths to reach a goal. Then, this path became blocked, and for test trials only the other path was passable. Results demonstrated that in test trials younger and older adults anticipated that the agent would take the continuous path, indicating that they took the situational constraints into account. In contrast, 2- and 5-year-olds anticipated that the agent would take the blocked path, indicating that they still relied on the agent's previous observed behaviour and-contrary to claims by the teleological stance theory-did not take the situational constraints into account. The results highlight developmental changes in human's ability to include situational constraints in their visual anticipations. Overall, the study contributes to theories on predictive coding and the development of action understanding.

Acting with shared intentions: A systematic review on joint action coordination in Autism Spectrum Disorder

BACKGROUND

Joint actions, described as a form of social interaction in which individuals coordinate their actions in space and time to bring about a change in the environment, rely on sensory-motor processes that play a role in the development of social skills. Two brain networks, associated with "mirroring" and "mentalizing", are engaged during these actions: the mirror neuron and the theory of mind systems. People with autism spectrum disorder (ASD) showed impairment in interpersonal coordination during joint actions. Studying joint action coordination in ASD will contribute to clarify the interplay between sensory-motor and social processes throughout development and the interactions between the brain and the behavior.

METHOD

This review focused on empirical studies that reported behavioral and kinematic findings related to joint action coordination in people with ASD.

RESULTS

Literature on mechanisms involved in the joint action coordination impairment in ASD is still limited. Data are controversial. Different key-components of joint action coordination may be impaired, such as cooperative behavior, temporal coordination, and motor planning.

CONCLUSIONS

Interpersonal coordination during joint actions relies on early sensory-motor processes that have a key role in guiding social development. Early intervention targeting the sensory-motor processes involved in the development of joint action coordination could positively support social skills.

Coordination and Consonance Between Interacting, Improvising Musicians

Joint action (JA) is ubiquitous in our cognitive lives. From basketball teams to teams of surgeons, humans often coordinate with one another to achieve some common goal. Idealized laboratory studies of group behavior have begun to elucidate basic JA mechanisms, but little is understood about how these mechanisms scale up in more sophisticated and open-ended JA that occurs in the wild. We address this gap by examining coordination in a paragon domain for creative joint expression: improvising jazz musicians. Coordination in jazz music subserves an aesthetic goal: the generation of a collective musical expression comprising coherent, highly nuanced musical structure (e.g., rhythm, harmony). In our study, dyads of professional jazz pianists improvised in a "coupled," mutually adaptive condition, and an "overdubbed" condition that precluded mutual adaptation, as occurs in common studio recording practices. Using a model of musical tonality, we quantify the flow of rhythmic and harmonic information between musicians as a function of interaction condition. Our analyses show that mutually adapting dyads achieve greater temporal alignment and produce more consonant harmonies. These musical signatures of coordination were preferred by independent improvisers and naive listeners, who gave higher quality ratings to coupled interactions despite being blind to condition. We present these results and discuss their implications for music technology and JA research more generally.

Effect of perceived interpersonal closeness on the joint Simon effect in adolescents and adults

Here, we explored the role of perceived interpersonal closeness in joint action using the joint Simon task in adolescents and adults. In a two-choice reaction time task, spatially assigned responses to non-spatial stimulus features are faster when the stimulus and response are in congruent locations than not. This phenomenon is called Simon effect and is absent or strongly attenuated when a participant responds to only one of the stimuli. However, the effect reappears when two participants carry out the same go/no-go tasks cooperatively. This re-emergence of the Simon effect in joint action is called the joint Simon effect (JSE). In this study, we first replicated the standard and joint Simon effects in adolescents (n = 43), as well as adults (n = 39) with similar magnitude of the effects in the two age groups. The magnitude of the JSE was positively correlated with the level of closeness as measured by Inclusion of Other in the Self scale. This correlation was not significantly different in adolescents (n = 73) compared to adults (n = 71). Our findings show that joint action is sensitive to the social factor such as interpersonal closeness, and the underlying mechanisms are already mature by adolescence.

Action prediction during real-time parent-infant interactions

Social interactions provide a crucial context for early learning and cognitive development during infancy. Action prediction-the ability to anticipate an observed action-facilitates successful, coordinated interaction and is an important social-cognitive skill in early development. However, current knowledge about infant action prediction comes largely from screen-based laboratory tasks. We know little about what infants' action prediction skills look like during real-time, free-flowing interactions with a social partner. In the current study, we used head-mounted eyetracking to quantify 9-month-old infants' visual anticipations of their parents' actions during free-flowing parent-child play. Our findings reveal that infants do anticipate their parents' actions during dynamic interactions at rates significantly higher than would be expected by chance. In addition, the frequency with which they do so is associated with child-led joint attention and hand-eye coordination. These findings are the first to reveal infants' action prediction behaviors in a more naturalistic context than prior screen-based studies, and they support the idea that action prediction is inherently linked to motor development and plays an important role in infants' social-cognitive development. A video abstract of this article can be viewed at https://www.youtube.com/watch?v=9HrmcicfiqE.

Berichterstattung zu Sicherheit und Gesundheit bei der Arbeit

Die Berichterstattung zu Sicherheit und Gesundheit bei der Arbeit in Deutschland hat eine lange Tradition, wobei im ersten „Unfallverhütungsbericht Arbeit“ aus den 1960er-Jahren diese zunächst auf Arbeitsunfälle und Berufskrankheiten fokussierte. Die im Arbeitsschutz geltende Maxime der menschengerechten Arbeitsgestaltung und der Wandel der Arbeitswelt forderten in der Vergangenheit und fordern in Gegenwart und Zukunft aber verschiedene inhaltliche Erweiterungen und Veränderungen. Nicht zuletzt, um diesen Anforderungen nachkommen zu können, werden neben den verfügbaren amtlichen und Prozessdaten inzwischen auch verstärkt Befragungen von Erwerbstätigen berücksichtigt, die Einblicke in die individuellen Arbeitstätigkeiten und Befindlichkeiten erlauben.

Bei der Berichterstattung zu Sicherheit und Gesundheit und der Gesundheitsberichterstattung gibt es einige Überschneidungen, z. B. nutzen sie zum Teil die gleichen Daten, aber auch substanzielle Unterschiede. Neben Unterschieden in der Berichterstattung liegt die wohl größte Differenz in der zu betrachtenden Gruppe, die in der Gesundheitsberichterstattung deutlich vielfältiger ist.

Die größte Herausforderung für die Arbeitsweltberichterstattung liegt im (stetigen) Wandel der Arbeitswelt, der durch technische Entwicklungen auch strukturelle Veränderungen in der betrieblichen Zusammenarbeit bewirken kann. So kann z. B. orts- und zeitflexibles Arbeiten dafür sorgen, dass nicht nur die Zusammenarbeit mit Kolleginnen und Kollegen virtuell stattfinden muss, sondern auch die Anforderungen an Führungskräfte sich fundamental verändern. Erhebungsmethoden und Berichterstattungen müssen sich hier den Gegebenheiten anpassen.

Using head-mounted eye-trackers to study sensory-motor dynamics of coordinated attention

In this chapter, we introduce recent research using head-mounted eye-trackers to record sensory-motor behaviors at a high resolution and examine parent-child interactions at a micro-level. We focus on one important research topic in early social and cognitive development: how young children and their parents coordinate their visual attention in social interactions. We start by introducing head-mounted eye-tracking and recent studies conducted using this method. We then present two sets of novel analysis techniques that examine how manual actions of parents and children with and without hearing loss contribute to their attention coordination. In the first set of analyses, we investigated different pathways parents and children used to coordinate their visual attention in toy play. After that, we used Sankey diagrams to represent the temporal dynamics of parents' and children's manual actions prior to and during coordinated attention. These two sets of analyses allowed us to explore how participants' sensory-motor behaviors contribute to the establishment and maintenance of coordinated attention. More generally, head-mounted eye-tracking allows us to ask new questions and conduct new analyses that were not previously possible. With this new sensing technology, the results here highlight the importance of understanding early social interaction from a multimodal, embodied view.

Becoming better together: The early development of interpersonal coordination

Crucial for interacting successfully with other people is the ability to coordinate one's actions with those of others. Interpersonal coordination can be planned or emergent (spontaneous). Although typically easy for adults, coordinating successfully and smoothly with others may be far from trivial for infants and toddlers. What do we know about the developmental trajectory of interpersonal coordination in the first years of life? Which processes play a role in successfully coordinating with others? And how does the development of interpersonal coordination impact other aspects of children's development? In this chapter, we review when and how infants and young children develop successful interpersonal coordination skills (planned and emergent) in early childhood. We argue that insights from the field of cognitive (neuro-) science have significantly advanced our knowledge on which social-cognitive processes underlie interpersonal coordination and its development. In particular, we discuss four important social-cognitive processes; monitoring and predicting others' actions as well as planning and controlling one's own actions. We then present findings on the impact of interpersonal coordination on young children's social understanding, their prosocial behavior and affiliation. Together, we conclude that for future research on the development of interpersonal coordination interdisciplinary exchanges between fields like cognitive (neuro-) science and developmental science offer promising avenues.

Audiovisual Modulation in Music Perception for Musicians and Non-musicians

In audiovisual music perception, visual information from a musical instrument being played is available prior to the onset of the corresponding musical sound and consequently allows a perceiver to form a prediction about the upcoming audio music. This prediction in audiovisual music perception, compared to auditory music perception, leads to lower N1 and P2 amplitudes and latencies. Although previous research suggests that audiovisual experience, such as previous musical experience may enhance this prediction, a remaining question is to what extent musical experience modifies N1 and P2 amplitudes and latencies. Furthermore, corresponding event-related phase modulations quantified as inter-trial phase coherence (ITPC) have not previously been reported for audiovisual music perception. In the current study, audio video recordings of a keyboard key being played were presented to musicians and non-musicians in audio only (AO), video only (VO), and audiovisual (AV) conditions. With predictive movements from playing the keyboard isolated from AV music perception (AV-VO), the current findings demonstrated that, compared to the AO condition, both groups had a similar decrease in N1 amplitude and latency, and P2 amplitude, along with correspondingly lower ITPC values in the delta, theta, and alpha frequency bands. However, while musicians showed lower ITPC values in the beta-band in AV-VO compared to the AO, non-musicians did not show this pattern. Findings indicate that AV perception may be broadly correlated with auditory perception, and differences between musicians and non-musicians further indicate musical experience to be a specific factor influencing AV perception. Predicting an upcoming sound in AV music perception may involve visual predictory processes, as well as beta-band oscillations, which may be influenced by years of musical training. This study highlights possible interconnectivity in AV perception as well as potential modulation with experience.

Early warning signals in motion inference

The ability to infer intention lies at the basis of many social interactions played out via motor actions. We consider a simple paradigm of this ability in humans using data from experiments simulating an antagonistic game between an Attacker and a Blocker. Evidence shows early inference of an Attacker move by as much as 100ms but the nature of the informational cues signaling the impending move remains unknown. We show that the transition to action has the hallmark of a critical transition that is accompanied by early warning signals. These early warning signals occur as much as 130 ms before motion ensues-showing a sharp rise in motion autocorrelation at lag-1 and a sharp rise in the autocorrelation decay time. The early warning signals further correlate strongly with Blocker response times. We analyze the variance of the motion near the point of transition and find that it diverges in a manner consistent with the dynamics of a fold-transition. To test if humans can recognize and act upon these early warning signals, we simulate the dynamics of fold-transition events and ask people to recognize the onset of directional motion: participants react faster to fold-transition dynamics than to its uncorrelated counterpart. Together, our findings suggest that people can recognize the intent and onset of motion by inferring its early warning signals.

Neural dynamics of grip and goal integration during the processing of others' actions with objects: An ERP study

Recent behavioural evidence suggests that when processing others’ actions, motor acts and goal-related information both contribute to action recognition. Yet the neuronal mechanisms underlying the dynamic integration of the two action dimensions remain unclear. This study aims to elucidate the ERP components underlying the processing and integration of grip and goal-related information. The electrophysiological activity of 28 adults was recorded during the processing of object-directed action photographs (e.g., writing with pencil) containing either grip violations (e.g. upright pencil grasped with atypical-grip), goal violations (e.g., upside-down pencil grasped with typical-grip), both grip and goal violations (e.g., upside-down pencil grasped with atypical-grip), or no violations. Participants judged whether actions were overall typical or not according to object typical use. Brain activity was sensitive to the congruency between grip and goal information on the N400, reflecting the semantic integration between the two dimensions. On earlier components, brain activity was affected by grip and goal typicality independently. Critically, goal typicality but not grip typicality affected brain activity on the N300, supporting an earlier role of goal-related representations in action recognition. Findings provide new insights on the neural temporal dynamics of the integration of motor acts and goal-related information during the processing of others’ actions.

Susceptibility to the fusion illusion is modulated during both action execution and action observation

Many researchers have proposed that when an individual observes the actions of another individual, the observer simulates the action using many of the same neural areas that are involved in action production. The present study was designed to test this simulation hypothesis by comparing the perception of multisensory stimuli during both the execution and observation of an aiming action. The present work used the fusion illusion - an audio-visual illusion in which two visual stimuli presented with one auditory stimulus are erroneously perceived as being one visual stimulus. Previous research has shown that, during action execution, susceptibly to this illusion is reduced early in the execution of the movement when visual information may be more highly weighted than other sensory information. We sought to determine whether or not a non-acting observer of an action showed a similar reduction in susceptibility to the fusion illusion. Participants fixated a target and either executed or observed a manual aiming movement to that target. Audiovisual stimuli were presented at 0, 100, or 200 ms relative to movement onset and participants reported the number of perceived flashes after the movement was completed. Analysis of perceived flashes revealed that participants were less susceptible to the fusion illusion when the stimuli were presented early (100 ms) relative to later in the movement (200 ms). Critically, this pattern emerged in both execution and observation tasks. These findings support the hypothesis that observers simulate the performance of the actor and experience comparable real-time alterations in multisensory processing.

Chimpanzee Coordination and Potential Communication in a Two-touchscreen Turn-taking Game

Recent years have seen a growing interest in the question of whether and how groups of nonhuman primates coordinate their behaviors for mutual benefit. On the one hand, it has been shown that chimpanzees in the wild and in captivity can solve various coordination problems. On the other hand, evidence of communication in the context of coordination problems is scarce. Here, we investigated how pairs of chimpanzees (Pan troglodytes) solved a problem of dynamically coordinating their actions for achieving a joint goal. We presented five pairs of chimpanzees with a turn-taking coordination game, where the task was to send a virtual target from one computer display to another using two touch-screens. During the joint practice of the game some subjects exhibited spontaneous gesturing. To address the question whether these gestures were produced to sustain coordination, we introduced a joint test condition in which we simulated a coordination break-down scenario: subjects appeared either unwilling or unable to return the target to their partner. The frequency of gesturing was significantly higher in these test trials than in the regular trials. Our results suggest that at least in some contexts chimpanzees can exhibit communicative behaviors to sustain coordination in joint action.

The role of transients in action observation

A large number of studies have now described the various ways in which the observation of another person's dynamic movement can influence the speed with which the observer is able to prepare a motor action themselves. The typical results are most often explained with reference to theories that link perception and action. Such theories argue that the cognitive structures associated with each share common representations. Consequently, action preparation and action observation are often said to be functionally equivalent. However, the dominance of these theories in explaining action observation effects has masked the potential contribution from processes associated with the detection of low-level "transients" resulting from observing a body movement, such as motion and sound. In the present review, we describe work undertaken in one particular action observation phenomenon ("social inhibition of return") and show that the transient account provides the best explanation of the effect. We argue that future work should consider attention capture and orienting as a potential contributing factor to action observation effects more broadly.

Understanding the visual perception of awkward body movements: How interactions go awry

Dyadic interactions can sometimes elicit a disconcerting response from viewers, generating a sense of "awkwardness." Despite the ubiquity of awkward social interactions in daily life, it remains unknown what visual cues signal the oddity of human interactions and yield the subjective impression of awkwardness. In the present experiments, we focused on a range of greeting behaviors (handshake, fist bump, high five) to examine both the inherent objectivity and impact of contextual and kinematic information in the social evaluation of awkwardness. In Experiment 1, participants were asked to discriminate whether greeting behaviors presented in raw videos were awkward or natural, and if judged as awkward, participants provided verbal descriptions regarding the awkward greeting behaviors. Participants showed consensus in judging awkwardness from raw videos, with a high proportion of congruent responses across a range of awkward greeting behaviors. We also found that people used social-related and motor-related words in their descriptions for awkward interactions. Experiment 2 employed advanced computer vision techniques to present the same greeting behaviors in three different display types. All display types preserved kinematic information, but varied contextual information: (1) patch displays presented blurred scenes composed of patches; (2) body displays presented human body figures on a black background; and (3) skeleton displays presented skeletal figures of moving bodies. Participants rated the degree of awkwardness of greeting behaviors. Across display types, participants consistently discriminated awkward and natural greetings, indicating that the kinematics of body movements plays an important role in guiding awkwardness judgments. Multidimensional scaling analysis based on the similarity of awkwardness ratings revealed two primary cues: motor coordination (which accounted for most of the variability in awkwardness judgments) and social coordination. We conclude that the perception of awkwardness, while primarily inferred on the basis of kinematic information, is additionally affected by the perceived social coordination underlying human greeting behaviors.

Action coordination during a real-world task: Evidence from children with and without autism spectrum disorder

"Joint action"-the ability to coordinate actions with others-is critical for achieving individual and interpersonal goals and for our collective success as a species. Joint actions require accurate and rapid inferences about others' goals, intentions, and focus of attention, skills that are thought to be impaired in individuals with autism spectrum disorder (ASD). Research to date has not investigated joint action abilities in individuals with ASD during real-world social interactions. We conducted an experimental study that required children with ASD and typically developing children to move tables by themselves or collaboratively through a maze. This involved developing innovative methodologies for measuring action coordination-a critical component of the joint action process. We found that children with ASD are less likely to benefit from the collaboration of a peer than are typically developing children, and they are less likely to synchronize their steps when moving the table. However, these differences were masked when scaffolded by an adult. There was no evidence that ASD differences were due to gross motor delays in the participants with ASD. We argue that action coordination is a highly adaptive social process that is intrinsic to successful human functioning that manifests as atypical synchronization of mind and body in children with ASD.

Sensorimotor cortex activation during anticipation of upcoming predictable but not unpredictable actions

The mirror neuron system (MNS) becomes active during action execution and action observation, which is presumably reflected by reductions in mu (8-13 Hz) activity in the electroencephalogram over the sensorimotor cortex. The function of the MNS is still fiercely debated. The current study aimed to investigate a role of the MNS in anticipating others' actions by examining whether the MNS was activated - indexed by mu power suppression - prior to the onset of observed actions when the onset and type of action could be predicted on the basis of environmental cues. Young adults performed and observed cued grasping and placing actions in a card game in a real-life setting, while the predictability of the observed actions was manipulated using rules. Significant mu suppression, relative to within-trial baseline activity, was found both prior to and during executed actions, but also during action observation, and, crucially, prior to observed actions provided they were predictable. No anticipatory mu reductions were found prior to unpredictable observed actions. These results suggest top-down modulation of MNS activity by conceptual knowledge. This is the first study to demonstrate mu suppression prior to action onset - possibly reflecting MNS anticipatory activity - by explicitly manipulating predictability.