Quantifying phase correction in sensorimotor synchronization: empirical comparison of three paradigms

Brain networks for temporal adaptation, anticipation, and sensory-motor integration in rhythmic human behavior

Human interaction often requires the precise yet flexible interpersonal coordination of rhythmic behavior, as in group music making. The present fMRI study investigates the functional brain networks that may facilitate such behavior by enabling temporal adaptation (error correction), prediction, and the monitoring and integration of information about 'self' and the external environment. Participants were required to synchronize finger taps with computer-controlled auditory sequences that were presented either at a globally steady tempo with local adaptations to the participants' tap timing (Virtual Partner task) or with gradual tempo accelerations and decelerations but without adaptation (Tempo Change task). Connectome-based predictive modelling was used to examine patterns of brain functional connectivity related to individual differences in behavioral performance and parameter estimates from the adaptation and anticipation model (ADAM) of sensorimotor synchronization for these two tasks under conditions of varying cognitive load. Results revealed distinct but overlapping brain networks associated with ADAM-derived estimates of temporal adaptation, anticipation, and the integration of self-controlled and externally controlled processes across task conditions. The partial overlap between ADAM networks suggests common hub regions that modulate functional connectivity within and between the brain's resting-state networks and additional sensory-motor regions and subcortical structures in a manner reflecting coordination skill. Such network reconfiguration might facilitate sensorimotor synchronization by enabling shifts in focus on internal and external information, and, in social contexts requiring interpersonal coordination, variations in the degree of simultaneous integration and segregation of these information sources in internal models that support self, other, and joint action planning and prediction.

Expectancy-based rhythmic entrainment as continuous Bayesian inference

When presented with complex rhythmic auditory stimuli, humans are able to track underlying temporal structure (e.g., a "beat"), both covertly and with their movements. This capacity goes far beyond that of a simple entrained oscillator, drawing on contextual and enculturated timing expectations and adjusting rapidly to perturbations in event timing, phase, and tempo. Previous modeling work has described how entrainment to rhythms may be shaped by event timing expectations, but sheds little light on any underlying computational principles that could unify the phenomenon of expectation-based entrainment with other brain processes. Inspired by the predictive processing framework, we propose that the problem of rhythm tracking is naturally characterized as a problem of continuously estimating an underlying phase and tempo based on precise event times and their correspondence to timing expectations. We present two inference problems formalizing this insight: PIPPET (Phase Inference from Point Process Event Timing) and PATIPPET (Phase and Tempo Inference). Variational solutions to these inference problems resemble previous "Dynamic Attending" models of perceptual entrainment, but introduce new terms representing the dynamics of uncertainty and the influence of expectations in the absence of sensory events. These terms allow us to model multiple characteristics of covert and motor human rhythm tracking not addressed by other models, including sensitivity of error corrections to inter-event interval and perceived tempo changes induced by event omissions. We show that positing these novel influences in human entrainment yields a range of testable behavioral predictions. Guided by recent neurophysiological observations, we attempt to align the phase inference framework with a specific brain implementation. We also explore the potential of this normative framework to guide the interpretation of experimental data and serve as building blocks for even richer predictive processing and active inference models of timing.

Effects of a partner’s tap intervals on an individual’s timing control increase in slow-tempo dyad synchronisation using finger-tapping

In musical ensembles, musicians synchronise their movements with other members of the ensemble at various tempos. This study aims to investigate the extent of tempo dependency of own and partner’s timing information on rhythm production. We conducted a dyad synchronisation-continuous finger-tapping task. First, two participants synchronised with the same auditory metronome at various tempos. Subsequently, after stopping the metronome, the participants maintained the tempo with the presentation of the partner’s tap timing via auditory signals. This task was conducted in six metronome tempo conditions at 700 to 3,200 ms in 500 ms step. It was found that the partner’s previous inter-tap intervals increased as the metronome tempo decreased. The effects of own previous inter-tap intervals and synchronisation errors between own and the partner’s tap timing did not depend on the metronome tempo. Therefore, timing control in dyad synchronisation was affected by the partner’s tempo more strongly in slow than fast tempos. This strong effect of the partner in slow-tempo rhythm synchronisation could be due to stronger attention to the partner’s movement timing in slower tempos than in fast tempos.

Timing and correction of stepping movements with a virtual reality avatar

Research into the ability to coordinate one's movements with external cues has focussed on the use of simple rhythmic, auditory and visual stimuli, or interpersonal coordination with another person. Coordinating movements with a virtual avatar has not been explored, in the context of responses to temporal cues. To determine whether cueing of movements using a virtual avatar is effective, people's ability to accurately coordinate with the stimuli needs to be investigated. Here we focus on temporal cues, as we know from timing studies that visual cues can be difficult to follow in the timing context. Real stepping movements were mapped onto an avatar using motion capture data. Healthy participants were then motion captured whilst stepping in time with the avatar's movements, as viewed through a virtual reality headset. The timing of one of the avatar step cycles was accelerated or decelerated by 15% to create a temporal perturbation, for which participants would need to correct to, in order to remain in time. Step onset times of participants relative to the corresponding step-onsets of the avatar were used to measure the timing errors (asynchronies) between them. Participants completed either a visual-only condition, or auditory-visual with footstep sounds included, at two stepping tempo conditions (Fast: 400ms interval, Slow: 800ms interval). Participants' asynchronies exhibited slow drift in the Visual-Only condition, but became stable in the Auditory-Visual condition. Moreover, we observed a clear corrective response to the phase perturbation in both the fast and slow tempo auditory-visual conditions. We conclude that an avatar's movements can be used to influence a person's own motion, but should include relevant auditory cues congruent with the movement to ensure a suitable level of entrainment is achieved. This approach has applications in physiotherapy, where virtual avatars present an opportunity to provide the guidance to assist patients in adhering to prescribed exercises.

Response to perturbations as a built-in feature in a mathematical model for paced finger tapping

Paced finger tapping is one of the simplest tasks to study sensorimotor synchronization. The subject is instructed to tap in synchrony with a periodic sequence of brief tones, and the time difference (called asynchrony) between each response and the corresponding stimulus is recorded. Despite its simplicity, this task helps to unveil interesting features of the underlying neural system and the error-correction mechanism responsible for synchronization. Perturbation experiments are usually performed to probe the subject's response, for example, in the form of a "step change," i.e., an unexpected change in tempo. The asynchrony is the usual observable in such experiments and it is chosen as the main variable in many mathematical models that attempt to describe the phenomenon. In this work we show that although asynchrony can be perfectly described in operational terms, it is not well defined as a model variable when tempo perturbations are considered. We introduce an alternative variable and a mathematical model that intrinsically takes into account the perturbation and make theoretical predictions about the response to novel perturbations based on the geometrical organization of the trajectories in phase space. Our proposal is relevant to understand interpersonal synchronization and the synchronization to nonperiodic stimuli.

Spatiotemporal perturbations in paced finger tapping suggest a common mechanism for the processing of time errors

Paced finger tapping is a sensorimotor synchronization task where a subject has to keep pace with a metronome while the time differences (asynchronies) between each stimulus and its response are recorded. A usual way to study the underlying error correction mechanism is to perform unexpected temporal perturbations to the stimuli sequence. An overlooked issue is that at the moment of a temporal perturbation two things change: the stimuli period (a parameter) and the asynchrony (a variable). In terms of experimental manipulation, it would be desirable to have separate, independent control of parameter and variable values. In this work we perform paced finger tapping experiments combining simple temporal perturbations (tempo step change) and spatial perturbations with temporal effect (raised or lowered point of contact). In this way we decouple the parameter-and-variable confounding, performing novel perturbations where either the parameter or the variable changes. Our results show nonlinear features like asymmetry and are compatible with a common error correction mechanism for all types of asynchronies. We suggest taking this confounding into account when analyzing perturbations of any kind in finger tapping tasks but also in other areas of sensorimotor synchronization, like music performance experiments and paced walking in gait coordination studies.

Tutorial and simulations with ADAM: an adaptation and anticipation model of sensorimotor synchronization

Interpersonal coordination of movements often involves precise synchronization of action timing, particularly in expert domains such as ensemble music performance. According to the adaptation and anticipation model (ADAM) of sensorimotor synchronization, precise yet flexible interpersonal coordination is supported by reactive error correction mechanisms and anticipatory mechanisms that exploit systematic patterns in stimulus timing to plan future actions. Here, we provide a tutorial introduction to the computational architecture of ADAM and present a series of single- and dual-virtual agent simulations that examine the model parameters that produce ideal synchronization performance in different tempo conditions. In the single-agent simulations, a virtual agent synchronized responses to steady tempo sequence or a sequence containing gradual tempo changes. Parameters controlling basic reactive error (phase) correction were sufficient for producing ideal synchronization performance at the steady tempo, whereas parameters controlling anticipatory mechanisms were necessary for ideal performance with a tempo-changing sequence. In the dual-agent simulations, two interacting virtual agents produced temporal sequences from either congruent or incongruent internal performance templates specifying a steady tempo or tempo changes. Ideal performance was achieved with reactive error correction alone when both agents implemented the same performance template (either steady tempo or tempo change). In contrast, anticipatory mechanisms played a key role when one agent implemented a steady tempo template and the other agent implemented a tempo change template. These findings have implications for understanding the interplay between reactive and anticipatory mechanisms when agents possess compatible versus incompatible representations of task goals during human-human and human-machine interaction.

A mechanism of timing variability underlying the association between the mean and SD of asynchrony

Sensorimotor timing behaviors typically exhibit an elusive phenomenon known as the negative asynchrony. When synchronizing movements (e.g. finger taps) with an external sequence (e.g. a metronome), people's taps precede event onsets by a few tens of milliseconds. We recently reported that asynchrony is less negative in participants with lower asynchrony variability. This indicates an association between negative asynchrony and variability of timing. Here, in 24 metronome-synchronization data sets, we modeled asynchrony series using a sensorimotor synchronization model that accounts for serial dependence of asynchronies. The results showed that the modeling well captured the negative correlation between the mean and SD of asynchrony. The finding suggests that serial dependence in asynchronies is an essential mechanism of timing variability underlying the association between the mean and SD of asynchrony.

Cross-feedback with Partner Contributes to Performance Accuracy in Finger-tapping Rhythm Synchronization between One Leader and Two Followers

As observed in musical ensembles, people synchronize with a leader together with other people. This study aimed to investigate whether interdependency with a partner improves performance accuracy in rhythm synchronization with the leader. Participants performed a synchronization task via auditory signal by finger tapping in which two followers simultaneously synchronized with a leader: an isochronous metronome or a human leader with or without feedback from the followers. This task was conducted with and without cross-feedback (CFB) between the followers. The followers’ weak mutual tempo tracking via the CFB and the followers’ strong tempo tracking to the leader improved the tempo stability. Additionally, because the interdependency between the followers was weaker than the followers’ dependency on the human leader, the CFB did not enlarge the synchronization error between the human leader and the followers, which occurred in synchronization with the metronome. Thus, the CFB between the followers contributed to accuracy in synchronization with the human leader. The results suggest that in ensembles, players should strongly attend to the leader and should attempt to be less conscious of partners to maintain the appropriate balance between influences from the leader and partners.

Intentionality of a co-actor influences sensorimotor synchronisation with a virtual partner

Interpersonal sensorimotor synchronisation requires individuals to anticipate and adapt to their partner's movement timing. Research has demonstrated that the intentionality of a co-actor affects joint action planning, however, less is known about whether co-actor intentionality affects sensorimotor synchronisation. Explicit and implicit knowledge of a synchronisation partner's intentionality may influence coordination by modulating temporal anticipation and adaptation processes. We used a computer-controlled virtual partner (VP) consisting of tempo-changing auditory pacing sequences to simulate either an intentional or unintentional synchronisation partner. The VP was programmed to respond to the participant with low or moderate degrees of error correction, simulating a slightly or moderately adaptive human, respectively. In addition, task instructions were manipulated so that participants were told they were synchronising with either another person or a computer. Results indicated that synchronisation performance improved with the more adaptive VP. In addition, there was an influence of the explicit partner instruction, but this was dependent upon the degree of VP adaptivity and was modulated by subjective preferences for either the human or the computer partner. Beliefs about the intentionality of a synchronisation partner may thus influence interpersonal sensorimotor synchronisation in a manner that is modulated by preferences for interacting with intentional agents.

Groups clapping in unison undergo size-dependent error-induced frequency increase

Humans clapping together in unison is a familiar and robust example of emergent synchrony. We find that in experiments, such groups (from two to a few hundred) always increase clapping frequency, and larger groups increase more quickly. Based on single-person experiments and modeling, an individual tendency to rush is ruled out as an explanation. Instead, an asymmetric sensitivity in aural interactions explains the frequency increase, whereby individuals correct more strongly to match neighbour claps that precede their own clap, than those that follow it. A simple conceptual coupled oscillator model based on this interaction recovers the main features observed in experiments, and shows that the collective frequency increase is driven by the small timing errors in individuals, and the resulting inter-individual interactions that occur to maintain unison.

Cue properties change timing strategies in group movement synchronisation

To maintain synchrony in group activities, each individual within the group must continuously correct their movements to remain in time with the temporal cues available. Cues might originate from one or more members of the group. Current research suggests that when synchronising movements, individuals optimise their performance in terms of minimising variability of timing errors (asynchronies) between external cues and their own movements. However, the cost of this is an increase in the timing variability of their own movements. Here we investigate whether an individual’s timing strategy changes according to the task, in a group scenario. To investigate this, we employed a novel paradigm that positioned six individuals to form two chains with common origin and termination on the circumference of a circle. We found that participants with access to timing cues from only one other member used a strategy to minimise their asynchrony variance. In contrast, the participant at the common termination of the two chains, who was required to integrate timing cues from two members, used a strategy that minimised movement variability. We conclude that humans are able to flexibly switch timekeeping strategies to maintain task demands and thus optimise the temporal performance of their movements.

Impaired movement timing in neurological disorders: rehabilitation and treatment strategies

Timing abnormalities have been reported in many neurological disorders, including Parkinson's disease (PD). In PD, motor-timing impairments are especially debilitating in gait. Despite impaired audiomotor synchronization, PD patients' gait improves when they walk with an auditory metronome or with music. Building on that research, we make recommendations for optimizing sensory cues to improve the efficacy of rhythmic cuing in gait rehabilitation. Adaptive rhythmic metronomes (that synchronize with the patient's walking) might be especially effective. In a recent study we showed that adaptive metronomes synchronized consistently with PD patients' footsteps without requiring attention; this improved stability and reinstated healthy gait dynamics. Other strategies could help optimize sensory cues for gait rehabilitation. Groove music strongly engages the motor system and induces movement; bass-frequency tones are associated with movement and provide strong timing cues. Thus, groove and bass-frequency pulses could deliver potent rhythmic cues. These strategies capitalize on the close neural connections between auditory and motor networks; and auditory cues are typically preferred. However, moving visual cues greatly improve visuomotor synchronization and could warrant examination in gait rehabilitation. Together, a treatment approach that employs groove, auditory, bass-frequency, and adaptive (GABA) cues could help optimize rhythmic sensory cues for treating motor and timing deficits.

Modeling effects of cerebellar and basal ganglia lesions on adaptation and anticipation during sensorimotor synchronization

This study addressed the role of subcortical brain structures in temporal adaptation and anticipation during sensorimotor synchronization. The performance of patients with cerebellar or basal ganglia lesions was compared with that of healthy control participants on tasks requiring the synchronization of drum strokes with adaptive and tempo-changing auditory pacing sequences. The precision of sensorimotor synchronization was generally lower in patients relative to controls (i.e., variability of asynchronies was higher in patients), although synchronization accuracy (mean asynchrony) was commensurate. A computational model of adaptation and anticipation (ADAM) was used to examine potential sources of individual differences in precision by estimating participants' use of error correction, temporal prediction, and the amount of variability associated with central timekeeping and peripheral motor processes. Parameter estimates based on ADAM indicate that impaired precision was attributable to increased variability of timekeeper and motor processes as well as to reduced temporal prediction in both patient groups. Adaptive processes related to continuously applied error correction were, by contrast, intact in patients. These findings highlight the importance of investigating how subcortical structures, including the cerebellum and basal ganglia, interact with a broader network of cortical regions to support temporal adaptation and anticipation during sensorimotor synchronization.

Lower Bound on the Accuracy of Parameter Estimation Methods for Linear Sensorimotor Synchronization Models

The mechanisms that support sensorimotor synchronization — that is, the temporal coordination of movement with an external rhythm — are often investigated using linear computational models. The main method used for estimating the parameters of this type of model was established in the seminal work of Vorberg and Schulze (2002), and is based on fitting the model to the observed auto-covariance function of asynchronies between movements and pacing events. Vorberg and Schulze also identified the problem of parameter interdependence, namely, that different sets of parameters might yield almost identical fits, and therefore the estimation method cannot determine the parameters uniquely. This problem results in a large estimation error and bias, thereby limiting the explanatory power of existing linear models of sensorimotor synchronization. We present a mathematical analysis of the parameter interdependence problem. By applying the Cramér–Rao lower bound, a general lower bound limiting the accuracy of any parameter estimation procedure, we prove that the mathematical structure of the linear models used in the literature determines that this problem cannot be resolved by any unbiased estimation method without adopting further assumptions. We then show that adding a simple and empirically justified constraint on the parameter space — assuming a relationship between the variances of the noise terms in the model — resolves the problem. In a follow-up paper in this volume, we present a novel estimation technique that uses this constraint in conjunction with matrix algebra to reliably estimate the parameters of almost all linear models used in the literature.

Individual Differences in Temporal Anticipation and Adaptation During Sensorimotor Synchronization

Interpersonal coordination during musical joint action (e.g., ensemble performance) requires individuals to anticipate and adapt to each other’s action timing. Individuals differ in their ability to both anticipate and adapt, however, little is known about the relationship between these skills. The present study used paced finger tapping tasks to examine the relationship between anticipatory skill and adaptive (error correction) processes. Based on a computational model, it was hypothesized that temporal anticipation and adaptation will act together to facilitate synchronization accuracy and precision. Adaptive ability was measured as the degree of temporal error correction that participants (N= 52) engaged in when synchronizing with a ‘virtual partner’, that is, an auditory pacing signal that modulated its timing based on the participant’s performance. Anticipation was measured through a prediction index that reflected the degree to which participants’ inter-tap intervals led or lagged behind inter-onset intervals in tempo-changing sequences. A correlational analysis revealed a significant positive relationship between the prediction index and temporal error correction estimates, suggesting that anticipation and adaptation interact to facilitate synchronization performance. Hierarchical regression analyses revealed that adaptation was the best predictor of synchronization accuracy, whereas both adaptation and anticipation predicted synchronization precision. Together these results demonstrate a relationship between anticipatory and adaptive mechanisms, and indicate that individual differences in these two abilities are predictive of synchronization performance.

Parameter Estimation of Linear Sensorimotor Synchronization Models: Phase Correction, Period Correction, and Ensemble Synchronization

Linear models have been used in several contexts to study the mechanisms that underpin sensorimotor synchronization. Given that their parameters are often linked to psychological processes such as phase correction and period correction, the fit of the parameters to experimental data is an important practical question. We present a unified method for parameter estimation of linear sensorimotor synchronization models that extends available techniques and enhances their usability. This method enables reliable and efficient analysis of experimental data for single subject and multi-person synchronization. In a previous paper (Jacoby et al., 2015), we showed how to significantly reduce the estimation error and eliminate the bias of parameter estimation methods by adding a simple and empirically justified constraint on the parameter space. By applying this constraint in conjunction with the tools of matrix algebra, we here develop a novel method for estimating the parameters of most linear models described in the literature. Through extensive simulations, we demonstrate that our method reliably and efficiently recovers the parameters of two influential linear models: Vorberg and Wing (1996), and Schulze et al. (2005), together with their multi-person generalization to ensemble synchronization. We discuss how our method can be applied to include the study of individual differences in sensorimotor synchronization ability, for example, in clinical populations and ensemble musicians.

When they listen and when they watch: Pianists' use of nonverbal audio and visual cues during duet performance

Nonverbal auditory and visual communication helps ensemble musicians predict each other's intentions and coordinate their actions. When structural characteristics of the music make predicting co-performers' intentions difficult (e.g., following long pauses or during ritardandi), reliance on incoming auditory and visual signals may change. This study tested whether attention to visual cues during piano-piano and piano-violin duet performance increases in such situations. Pianists performed the secondo part to three duets, synchronizing with recordings of violinists or pianists playing the primo parts. Secondos' access to incoming audio and visual signals and to their own auditory feedback was manipulated. Synchronization was most successful when primo audio was available, deteriorating when primo audio was removed and only cues from primo visual signals were available. Visual cues were used effectively following long pauses in the music, however, even in the absence of primo audio. Synchronization was unaffected by the removal of secondos' own auditory feedback. Differences were observed in how successfully piano-piano and piano-violin duos synchronized, but these effects of instrument pairing were not consistent across pieces. Pianists' success at synchronizing with violinists and other pianists is likely moderated by piece characteristics and individual differences in the clarity of cueing gestures used.

Sensorimotor synchronization with tempo-changing auditory sequences: Modeling temporal adaptation and anticipation

The current study investigated the human ability to synchronize movements with event sequences containing continuous tempo changes. This capacity is evident, for example, in ensemble musicians who maintain precise interpersonal coordination while modulating the performance tempo for expressive purposes. Here we tested an ADaptation and Anticipation Model (ADAM) that was developed to account for such behavior by combining error correction processes (adaptation) with a predictive temporal extrapolation process (anticipation). While previous computational models of synchronization incorporate error correction, they do not account for prediction during tempo-changing behavior. The fit between behavioral data and computer simulations based on four versions of ADAM was assessed. These versions included a model with adaptation only, one in which adaptation and anticipation act in combination (error correction is applied on the basis of predicted tempo changes), and two models in which adaptation and anticipation were linked in a joint module that corrects for predicted discrepancies between the outcomes of adaptive and anticipatory processes. The behavioral experiment required participants to tap their finger in time with three auditory pacing sequences containing tempo changes that differed in the rate of change and the number of turning points. Behavioral results indicated that sensorimotor synchronization accuracy and precision, while generally high, decreased with increases in the rate of tempo change and number of turning points. Simulations and model-based parameter estimates showed that adaptation mechanisms alone could not fully explain the observed precision of sensorimotor synchronization. Including anticipation in the model increased the precision of simulated sensorimotor synchronization and improved the fit of model to behavioral data, especially when adaptation and anticipation mechanisms were linked via a joint module based on the notion of joint internal models. Overall results suggest that adaptation and anticipation mechanisms both play an important role during sensorimotor synchronization with tempo-changing sequences. This article is part of a Special Issue entitled SI: Prediction and Attention.

Rhythm in joint action: psychological and neurophysiological mechanisms for real-time interpersonal coordination

Human interaction often requires simultaneous precision and flexibility in the coordination of rhythmic behaviour between individuals engaged in joint activity, for example, playing a musical duet or dancing with a partner. This review article addresses the psychological processes and brain mechanisms that enable such rhythmic interpersonal coordination. First, an overview is given of research on the cognitive-motor processes that enable individuals to represent joint action goals and to anticipate, attend and adapt to other's actions in real time. Second, the neurophysiological mechanisms that underpin rhythmic interpersonal coordination are sought in studies of sensorimotor and cognitive processes that play a role in the representation and integration of self- and other-related actions within and between individuals' brains. Finally, relationships between social-psychological factors and rhythmic interpersonal coordination are considered from two perspectives, one concerning how social-cognitive tendencies (e.g. empathy) affect coordination, and the other concerning how coordination affects interpersonal affiliation, trust and prosocial behaviour. Our review highlights musical ensemble performance as an ecologically valid yet readily controlled domain for investigating rhythm in joint action.

Basic timing abilities stay intact in patients with musician's dystonia

Task-specific focal dystonia is a movement disorder that is characterized by the loss of voluntary motor control in extensively trained movements. Musician's dystonia is a type of task-specific dystonia that is elicited in professional musicians during instrumental playing. The disorder has been associated with deficits in timing. In order to test the hypothesis that basic timing abilities are affected by musician's dystonia, we investigated a group of patients (N = 15) and a matched control group (N = 15) on a battery of sensory and sensorimotor synchronization tasks. Results did not show any deficits in auditory-motor processing for patients relative to controls. Both groups benefited from a pacing sequence that adapted to their timing (in a sensorimotor synchronization task at a stable tempo). In a purely perceptual task, both groups were able to detect a misaligned metronome when it was late rather than early relative to a musical beat. Overall, the results suggest that basic timing abilities stay intact in patients with musician's dystonia. This supports the idea that musician's dystonia is a highly task-specific movement disorder in which patients are mostly impaired in tasks closely related to the demands of actually playing their instrument.

Sensorimotor synchronization: a review of recent research (2006-2012)

Sensorimotor synchronization (SMS) is the coordination of rhythmic movement with an external rhythm, ranging from finger tapping in time with a metronome to musical ensemble performance. An earlier review (Repp, 2005) covered tapping studies; two additional reviews (Repp, 2006a, b) focused on music performance and on rate limits of SMS, respectively. The present article supplements and extends these earlier reviews by surveying more recent research in what appears to be a burgeoning field. The article comprises four parts, dealing with (1) conventional tapping studies, (2) other forms of moving in synchrony with external rhythms (including dance and nonhuman animals' synchronization abilities), (3) interpersonal synchronization (including musical ensemble performance), and (4) the neuroscience of SMS. It is evident that much new knowledge about SMS has been acquired in the last 7 years.

Optimal feedback correction in string quartet synchronization

Control of relative timing is critical in ensemble music performance. We hypothesize that players respond to and correct asynchronies in tone onsets that arise from fluctuations in their individual tempos. We propose a first-order linear phase correction model and demonstrate that optimal performance that minimizes asynchrony variance predicts a specific value for the correction gain. In two separate case studies, two internationally recognized string quartets repeatedly performed a short excerpt from the fourth movement of Haydn's quartet Op. 74 no. 1, with intentional, but unrehearsed, expressive variations in timing. Time series analysis of successive tone onset asynchronies was used to estimate correction gains for all pairs of players. On average, both quartets exhibited near-optimal gain. However, individual gains revealed contrasting patterns of adjustment between some pairs of players. In one quartet, the first violinist exhibited less adjustment to the others compared with their adjustment to her. In the second quartet, the levels of correction by the first violinist matched those exhibited by the others. These correction patterns may be seen as reflecting contrasting strategies of first-violin-led autocracy versus democracy. The time series approach we propose affords a sensitive method for investigating subtle contrasts in music ensemble synchronization.

Leading the follower: an fMRI investigation of dynamic cooperativity and leader-follower strategies in synchronization with an adaptive virtual partner

From everyday experience we know that it is generally easier to interact with someone who adapts to our behavior. Beyond this, achieving a common goal will very much depend on who adapts to whom and to what degree. Therefore, many joint action tasks such as musical performance prove to be more successful when defined leader-follower roles are established. In the present study, we present a novel approach to explore the mechanisms of how individuals lead and, using functional magnetic resonance imaging (fMRI), probe the neural correlates of leading. Specifically, we implemented an adaptive virtual partner (VP), an auditory pacing signal, with which individuals were instructed to tap in synchrony while maintaining a steady tempo. By varying the degree of temporal adaptation (period correction) implemented by the VP, we manipulated the objective control individuals had to exert to maintain the overall tempo of the pacing sequence (which was prone to tempo drift with high levels of period correction). Our imaging data revealed that perceiving greater influence and leading are correlated with right lateralized frontal activation of areas involved in cognitive control and self-related processing. Using participants' subjective ratings of influence and task difficulty, we classified a subgroup of our cohort as "leaders", individuals who found the task of synchronizing easier when they felt more in control. Behavioral tapping measures showed that leaders employed less error correction and focused more on self-tapping (prioritizing the instruction to maintain the given tempo) than on the stability of the interaction (prioritizing the instruction to synchronize with the VP), with correlated activity in areas involved in self-initiated action including the pre-supplementary motor area.

The ADaptation and Anticipation Model (ADAM) of sensorimotor synchronization

A constantly changing environment requires precise yet flexible timing of movements. Sensorimotor synchronization (SMS)—the temporal coordination of an action with events in a predictable external rhythm—is a fundamental human skill that contributes to optimal sensory-motor control in daily life. A large body of research related to SMS has focused on adaptive error correction mechanisms that support the synchronization of periodic movements (e.g., finger taps) with events in regular pacing sequences. The results of recent studies additionally highlight the importance of anticipatory mechanisms that support temporal prediction in the context of SMS with sequences that contain tempo changes. To investigate the role of adaptation and anticipatory mechanisms in SMS we introduce ADAM: an ADaptation and Anticipation Model. ADAM combines reactive error correction processes (adaptation) with predictive temporal extrapolation processes (anticipation) inspired by the computational neuroscience concept of internal models. The combination of simulations and experimental manipulations based on ADAM creates a novel and promising approach for exploring adaptation and anticipation in SMS. The current paper describes the conceptual basis and architecture of ADAM.

Small perturbations in a finger-tapping task reveal inherent nonlinearities of the underlying error correction mechanism

Time processing in the few hundred milliseconds range is involved in the human skill of sensorimotor synchronization, like playing music in an ensemble or finger tapping to an external beat. In finger tapping, a mechanistic explanation in biologically plausible terms of how the brain achieves synchronization is still missing despite considerable research. In this work we show that nonlinear effects are important for the recovery of synchronization following a perturbation (a step change in stimulus period), even for perturbation magnitudes smaller than 10% of the period, which is well below the amount of perturbation needed to evoke other nonlinear effects like saturation. We build a nonlinear mathematical model for the error correction mechanism and test its predictions, and further propose a framework that allows us to unify the description of the three common types of perturbations. While previous authors have used two different model mechanisms for fitting different perturbation types, or have fitted different parameter value sets for different perturbation magnitudes, we propose the first unified description of the behavior following all perturbation types and magnitudes as the dynamical response of a compound model with fixed terms and a single set of parameter values.

A general linear framework for the comparison and evaluation of models of sensorimotor synchronization

Sensorimotor synchronization (SMS), the temporal coordination of a rhythmic movement with an external rhythm, has been studied most often in tasks that require tapping along with a metronome. Models of SMS use information about the timing of preceding stimuli and responses to predict when the next response will be made. This article compares the theoretical structure and empirical predictions of four two-parameter models proposed in the literature: Michon (Timing in temporal tracking, Van Gorcum, Assen, 1967), Hary and Moore (Br J Math Stat Psychol 40:109-124, 1987b), Mates (Biol Cybern 70:463-473, 1994a; Biol Cybern 70:475-484, 1994b), and Schulze et al. (Mus Percept 22:461-467, 2005). By embedding these models within a general linear framework, the mathematical equivalence of the Michon, Hary and Moore, and Schulze et al. models is demonstrated. The Mates model, which differs from the other three, is then tested empirically with new data from a tapping experiment in which the metronome alternated between two tempi. The Mates model predictions are found to be invalid for about one-third of the trials, suggesting that at least one of the model's underlying assumptions is incorrect. The other models cannot be refuted as easily, but they do not predict some features of the data very accurately. Comparison of the models' predictions in a training/test procedure did not yield any significant differences. The general linear framework introduced here may help in the formulation of new models that make better predictions.