Ears, heads, and eyes: When singers synchronise

Pleasantness makes a good time: musical consonance shapes interpersonal synchronization in dyadic joint action

Introduction

Music making is a process by which humans across cultures come together to create patterns of sounds that are aesthetically pleasing. What remains unclear is how this aesthetic outcome affects the sensorimotor interaction between participants.

Method

Here we approach this question using an interpersonal sensorimotor synchronization paradigm to test whether the quality of a jointly created chord (consonant vs. dissonant) affects movement coordination. We recruited non-musician participants in dyads to perform a dyadic synchronization-continuation task (dSCT): on each trial, participants first synchronized their movements to a metronome (synchronization phase) and then continued tapping together at the same tempo without the metronome (continuation phase). Each tap yielded a note and participants heard both their own and that of their partner, thus creating a chord that was varied to be either consonant (Perf5 or Maj6) or dissonant (Min2 or Maj2). For each trial, participants also rated the pleasure they felt in creating the sounds together. Additionally, they completed questionnaires about social closeness to the other participant, musical reward sensitivity and musical training.

Results

Results showed that participants' taps were closer in time when they jointly created consonant (high pleasure) vs. dissonant (low pleasure) chords, and that pleasure experienced by the dyad in each trial predicted interpersonal synchronization. However, consonance did not affect individual synchronization with the metronome or individual tapping when the metronome was discontinued. The effect of consonance on synchronization was greater in dyads who reported feeling less close prior to the task.

Discussion

Together, these results highlight the role of consonance in shaping the temporal coordination of our actions with others. More broadly, this work shows that the aesthetic outcome of what we create together affects joint behaviors.

A framework for joint music making: Behavioral findings, neural processes, and computational models

Across different epochs and societies, humans occasionally gather to jointly make music. This universal form of collective behavior is as fascinating as it is fragmentedly understood. As the interest in joint music making (JMM) rapidly grows, we review the state-of-the-art of this emerging science, blending behavioral, neural, and computational contributions. We present a conceptual framework synthesizing research on JMM within four components. The framework is centered upon interpersonal coordination, a crucial requirement for JMM. The other components imply the influence of individuals' (past) experience, (current) social factors, and (future) goals on real-time coordination. Our aim is to promote the development of JMM research by organizing existing work, inspiring new questions, and fostering accessibility for researchers belonging to other research communities.

Duet synchronization interventions affect social interactions

Humans' complex behavior, such as speech, music, or dance, requires us to coordinate our actions with external sounds as well as with social partners. The presence of a partner can influence individuals' synchronization, and, in turn, social connection with the partner may depend on the degree of synchronization. We manipulated the synchronization quality in intervention conditions to address the causal relationship between observed temporal synchrony and perceived social interaction. Pairs of musician and nonmusician participants first performed a turn-taking task consisting of alternating which partner tapped their melody in synchrony with a metronome (each tap generated the next tone in the melody). In two intervention conditions, participants attempted to synchronize their melodies simultaneously with their partner, either with normal auditory feedback (normal feedback) or randomly placed delayed feedback on 25% of melodic tones (delayed feedback). After each intervention, the turn-taking condition was repeated, and participants completed a questionnaire about connectedness, relationship, and feeling of synchronization with their partner. Results showed that partners' mean asynchronies were more negative following the delayed feedback intervention. In addition, nonmusician partners' tapping variability was larger following the delayed feedback intervention when they had the delayed feedback intervention first. Ratings of connectedness, relationship, and feeling of synchronization with their partner were reduced for all participants after the delayed feedback Intervention. We modeled participants' synchronization performance in the post-intervention turn-taking conditions using delay-coupling oscillator models. Reductions in synchronization performance after delayed feedback intervention were reflected in reduced coupling strength. These findings suggest that turn-taking synchronization performance and social connectedness are altered following short interventions that disrupt synchronization with a partner.

Leadership dynamics in musical groups: Quantifying effects of musical structure on directionality of influence in concert performance videos

Music ensemble performance provides an ecologically valid context for investigating leadership dynamics in small group interactions. Musical texture, specifically the relative salience of simultaneously sounding ensemble parts, is a feature that can potentially alter leadership dynamics by introducing hierarchical relationships between individual parts. The present study extended previous work on quantifying interpersonal coupling in musical ensembles by examining the relationship between musical texture and leader-follower relations, operationalised as directionality of influence between co-performers' body motion in concert video recordings. It was hypothesised that the directionality of influence, indexed by Granger Causality, would be greater for 'homophonic' textures with a clear distinction between melody and accompaniment parts than for 'polyphonic' textures with less distinction between melody and accompaniment. This hypothesis was tested by using pose estimation algorithms to track instrumentalists' body movements in a string quartet and a clarinet quintet, and then applying Granger Causality analysis to their head motion to estimate directional influence between instrumentalist pairs for sections of the pieces that varied in texture. It was found that Granger Causality values were generally higher (indicating greater directionality of influence) for homophonic than polyphonic textures. Furthermore, considering melody and accompaniment instrument roles revealed more evidence for the melody instrument influencing accompanying instruments than vice versa, plus a high degree of directionality among accompanying instruments, in homophonic textures. These observed patterns of directional information flow in co-performer body motion are consistent with changing leader-follower relations depending on hierarchical relations between ensemble parts in terms of the relative salience of melodic material in the musical texture. The finding that automatic pose estimation can detect modulations of leadership dynamics in standard video recordings under naturalistic performance conditions has implications for investigating interpersonal coordination in large-scale music video datasets representing different cultural traditions, and for exploring nonverbal communication in group activities more generally.

Body motion of choral singers

Recent investigations on music performances have shown the relevance of singers' body motion for pedagogical as well as performance purposes. However, little is known about how the perception of voice-matching or task complexity affects choristers' body motion during ensemble singing. This study focussed on the body motion of choral singers who perform in duo along with a pre-recorded tune presented over a loudspeaker. Specifically, we examined the effects of the perception of voice-matching, operationalized in terms of sound spectral envelope, and task complexity on choristers' body motion. Fifteen singers with advanced choral experience first manipulated the spectral components of a pre-recorded short tune composed for the study, by choosing the settings they felt most and least together with. Then, they performed the tune in unison (i.e., singing the same melody simultaneously) and in canon (i.e., singing the same melody but at a temporal delay) with the chosen filter settings. Motion data of the choristers' upper body and audio of the repeated performances were collected and analyzed. Results show that the settings perceived as least together relate to extreme differences between the spectral components of the sound. The singers' wrists and torso motion was more periodic, their upper body posture was more open, and their bodies were more distant from the music stand when singing in unison than in canon. These findings suggest that unison singing promotes an expressive-periodic motion of the upper body.

You, me, and us: Maintaining self-other distinction enhances coordination, agency, and affect

Coordinating our actions with others changes how we behave and feel. Here, we provide evidence that interacting with others rests on a balance between self-other integration and segregation. Using a group walking paradigm, participants were instructed to synchronize with a metronome while listening to the sounds of 8 virtual partners. By manipulating the similarity and synchronicity of the partners' steps to the participant's own, our novel auditory task disentangles the effects of synchrony and self-other similarity and examines their contribution to both collective and individual awareness. We measured temporal coordination (step timing regularity and synchrony with the metronome), gait patterns, and subjective reports about sense of self and group cohesion. The main findings show that coordination is best when participants hear distinct but synchronous virtual others, leading to greater subjective feelings of agency, strength, dominance, and happiness.

Interpersonal synchrony when singing in a choir

Singing in a choir has long been known to enhance well-being and protect mental health. Clearly, the experience of a uniquely harmonious social activity is very satisfying for the singers. How might this come about? One of the important factors positively associated with well-being is interpersonal action coordination allowing the choir to function as a whole. This review focuses on temporal coordination dynamics of physiological systems and/or subsystems forming part or the core of the functional substrate of choir singing. These coordination dynamics will be evaluated with respect to the concept of a superordinate system, or superorganism, based on the principles of self-organization and circular causality. We conclude that choral singing is a dynamic process requiring tight interpersonal action coordination that is characterized by coupled physiological systems and specific network topology dynamics, representing a potent biomarker for social interaction.

Endogenous rhythms influence musicians' and non-musicians' interpersonal synchrony

Individuals display considerable rate differences in the spontaneous production of rhythmic behaviors (such as speech, gait, dance). Temporal precision in rhythmic behavior tends to be highest at individuals’ spontaneous production rates; musically trained partners with similar spontaneous rates show increased synchrony in joint tasks, consistent with predictions based on intrinsic frequencies of coupled oscillators. We address whether partner-specific influences of intrinsic frequencies are evidenced in musically trained and untrained individuals who tapped a familiar melody at a spontaneous (uncued) rate individually. Each individual then synchronized with a partner from the same musicianship group at an initially cued rate that matched the partners’ spontaneous rates. Musically trained partners showed greater synchrony in joint tapping than musically untrained partners. Asynchrony increased in both groups as the partners’ difference in individual spontaneous rates increased, with greater impact for musically untrained pairs. Recurrence quantification analysis confirmed that musically untrained individuals demonstrated greater determinism (less flexibility) in their tapping than musically trained individuals. Furthermore, individuals with greater determinism in solo performances demonstrated reduced synchrony in joint performances. These findings suggest that musicians’ increased temporal flexibility is associated with decreased endogenous constraints on production rate and greater interpersonal synchrony in musical tasks.

Social Interaction and Rate Effects in Models of Musical Synchronization

Important individual differences are observed in people's abilities to synchronize their body movements with regular auditory rhythms. We investigate whether synchronizing with a regular auditory cue is affected by each person's spontaneous production rate (SPR) and by hearing a partner's synchronization in a social context. Musically trained and untrained participants synchronized their tapping with an auditory cue presented at different rates (their own SPR or their partner's SPR) and in a Solo or Joint (turn-taking) condition. Linear and nonlinear oscillator models were fit to participants' mean asynchronies (signed timing differences between the cued onsets and taps). In Joint turn-taking, participants' synchrony was increased when the auditory signal was cued at the participant's own SPR, compared with their partner's SPR; in contrast, synchronization did not differ across rates in the Solo condition. Asynchronies in the Joint task became larger as the difference between partners' spontaneous rates increased; the increased asynchronies were driven by the faster partner who did not slow down to match the rate of their slower partner. Nonlinear delay-coupled models (with time delay, coupling strength, and intrinsic frequency) outperformed linear models (intrinsic frequency only) in accounting for tappers' synchronization adjustments. The nonlinear model's coupling value increased for musically trained participants, relative to untrained participants. Overall, these findings suggest that both intrinsic differences in partners' spontaneous rates and the social turn-taking context contribute to the range of synchrony in the general population. Delay-coupled models are capable of capturing the wide range of individual differences in auditory-motor synchronization.

Collective Rhythm as an Emergent Property During Human Social Coordination

The literature on social interactions has shown that participants coordinate not only at the behavioral but also at the physiological and neural levels, and that this coordination gives a temporal structure to the individual and social dynamics. However, it has not been fully explored whether such temporal patterns emerge during interpersonal coordination beyond dyads, whether this phenomenon arises from complex cognitive mechanisms or from relatively simple rules of behavior, or which are the sociocultural processes that underlie this phenomenon. We review the evidence for the existence of group-level rhythmic patterns that result from social interactions and argue that the complexity of group dynamics can lead to temporal regularities that cannot be predicted from the individual periodicities: an emergent collective rhythm. Moreover, we use this interpretation of the literature to discuss how taking into account the sociocultural niche in which individuals develop can help explain the seemingly divergent results that have been reported on the social influences and consequences of interpersonal coordination. We make recommendations on further research to test these arguments and their relationship to the feeling of belonging and assimilation experienced during group dynamics.

Behavioral and Neural Dynamics of Interpersonal Synchrony Between Performing Musicians: A Wireless EEG Hyperscanning Study

Interpersonal synchrony refers to the temporal coordination of actions between individuals and is a common feature of social behaviors, from team sport to ensemble music performance. Interpersonal synchrony of many rhythmic (periodic) behaviors displays dynamics of coupled biological oscillators. The current study addresses oscillatory dynamics on the levels of brain and behavior between music duet partners performing at spontaneous (uncued) rates. Wireless EEG was measured from N = 20 pairs of pianists as they performed a melody first in Solo performance (at their spontaneous rate of performance), and then in Duet performances at each partner's spontaneous rate. Influences of partners' spontaneous rates on interpersonal synchrony were assessed by correlating differences in partners' spontaneous rates of Solo performance with Duet tone onset asynchronies. Coupling between partners' neural oscillations was assessed by correlating amplitude envelope fluctuations of cortical oscillations at the Duet performance frequency between observed partners and between surrogate (re-paired) partners, who performed the same melody but at different times. Duet synchronization was influenced by partners' spontaneous rates in Solo performance. The size and direction of the difference in partners' spontaneous rates were mirrored in the size and direction of the Duet asynchronies. Moreover, observed Duet partners showed greater inter-brain correlations of oscillatory amplitude fluctuations than did surrogate partners, suggesting that performing in synchrony with a musical partner is reflected in coupled cortical dynamics at the performance frequency. The current study provides evidence that dynamics of oscillator coupling are reflected in both behavioral and neural measures of temporal coordination during musical joint action.

The sound of silence: an EEG study of how musicians time pauses in individual and joint music performance

Pauses are an integral feature of social interaction. Conversation partners often pause between conversational turns, and musical co-performers often pause between musical phrases. How do humans coordinate the duration of pauses to ensure seamless interaction? A total of 40 trained pianists performed a simple melody containing fermatas (notated expressive pauses of unspecified duration) first alone (Solo) and then with a partner (Duet) while electroencephalography (EEG) was recorded. As predicted, Duet partners' tone onset synchrony was reduced for tones following pauses. Pauses were shorter in Duet relative to Solo performance, and synchrony of partners' Duet tone onsets was enhanced for tones following shorter pauses. EEG analysis revealed classic signatures of action preparation during pauses, namely decreases in the power of cortical beta oscillations (13-30 Hz, event-related desynchronization ERD). Beta ERD did not differ between pauses in Solo and Duet performance, but was enhanced for shorter relative to longer pauses, suggesting that reduced pause durations in Duet performance facilitated a neural state of enhanced action readiness. Together these findings provide novel insight into behavioural strategies by which musical partners resolve coordination challenges posed by expressive silence, and capture a clear neural signature of action planning during time-varying silences in natural music performance.

Physiological and Behavioral Factors in Musicians' Performance Tempo

Musicians display individual differences in their spontaneous performance rates (tempo) for simple melodies, but the factors responsible are unknown. Previous research suggests that musical tempo modulates listeners' cardiovascular activity. We report an investigation of musicians' melody performances measured over a 12-h day and subsequent changes in the musicians' physiological activity. Skilled pianists completed four testing sessions in a single day as cardiac activity was recorded during an initial 5 min of baseline rest and during performances of familiar and unfamiliar melodies. Results indicated slower tempi for familiar and unfamiliar melodies at early testing times. Performance rates at 09 h were predicted by differences in participants' alertness and musical training; these differences were not explained by sleep patterns, chronotype, or cardiac activity. Individual differences in pianists' performance tempo were consistent across testing sessions: participants with a faster tempo at 09 h maintained a faster tempo at later testing sessions. Cardiac measures at early testing times indicated increased heart rates and more predictable cardiac dynamics during music performance than baseline rest, and during performances of unfamiliar melodies than familiar melodies. These findings provide the first evidence of cardiac dynamics that are unique to music performance contexts.

Musical training enhances temporal adaptation of auditory-motor synchronization

To coordinate their actions successfully with auditory events, individuals must be able to adapt their behaviour flexibly to environmental changes. Previous work has shown that musical training enhances the flexibility to synchronize behaviour with a wide range of stimulus periods. The current experiment investigated whether musical training enhances temporal adaptation to period perturbations as listeners tapped with a metronome, and whether this enhancement is specific to individuals' Spontaneous Production Rates (SPRs; individuals' natural uncued rates). Both musicians and nonmusicians adapted more quickly to period perturbations that slowed down than to those that sped up. Importantly, musicians adapted more quickly to all period perturbations than nonmusicians. Fits of a damped harmonic oscillator model to the tapping measures confirmed musicians' faster adaptation and greater responsiveness to period perturbations. These results suggest that, even when the task is tailored to individual SPRs, musical training increases the flexibility with which individuals can adapt to changes in their environment during auditory-motor tasks.

Staying Together: A Bidirectional Delay-Coupled Approach to Joint Action

To understand how individuals adapt to and anticipate each other in joint tasks, we employ a bidirectional delay-coupled dynamical system that allows for mutual adaptation and anticipation. In delay-coupled systems, anticipation is achieved when one system compares its own time-delayed behavior, which implicitly includes past information about the other system's behavior, with the other system's instantaneous behavior. Applied to joint music performance, the model allows each system to adapt its behavior to the dynamics of the other. Model predictions of asynchrony between two simultaneously produced musical voices were compared with duet pianists' behavior; each partner performed one voice while auditory feedback perturbations occurred at unpredictable times during live performance. As the model predicted, when auditory feedback from one musical voice was removed, the asynchrony changed: The pianist's voice that was removed anticipated (preceded) the actions of their partner. When the auditory feedback returned and both musicians could hear each other, they rapidly returned to baseline levels of asynchrony. To understand how the pianists anticipated each other, their performances were fitted by the model to examine change in model parameters (coupling strength, time-delay). When auditory feedback for one or both voices was removed, the fits showed the expected decrease in coupling strength and time-delay between the systems. When feedback about the voice(s) returned, the coupling strength and time-delay returned to baseline. These findings support the idea that when people perform actions together, they do so as a coupled bidirectional anticipatory system.