Binding of movement, sound and touch: multimodal coordination dynamics

Tactile cues are more intrinsically linked to motor timing than visual cues in visual-tactile sensorimotor synchronization

Many tasks require precise synchronization with external sensory stimuli, such as driving a car. This study investigates whether combined visual-tactile information provides additional benefits to movement synchrony over separate visual and tactile stimuli and explores the relationship with the temporal binding window for multisensory integration. In Experiment 1, participants completed a sensorimotor synchronization task to examine movement variability and a simultaneity judgment task to measure the temporal binding window. Results showed similar synchronization variability between visual-tactile and tactile-only stimuli, but significantly lower than visual only. In Experiment 2, participants completed a visual-tactile sensorimotor synchronization task with cross-modal stimuli presented inside (stimulus onset asynchrony 80 ms) and outside (stimulus-onset asynchrony 400 ms) the temporal binding window to examine temporal accuracy of movement execution. Participants synchronized their movement with the first stimulus in the cross-modal pair, either the visual or tactile stimulus. Results showed significantly greater temporal accuracy when only one stimulus was presented inside the window and the second stimulus was outside the window than when both stimuli were presented inside the window, with movement execution being more accurate when attending to the tactile stimulus. Overall, these findings indicate there may be a modality-specific benefit to sensorimotor synchronization performance, such that tactile cues are weighted more strongly than visual information as tactile information is more intrinsically linked to motor timing than visual information. Further, our findings indicate that the visual-tactile temporal binding window is related to the temporal accuracy of movement execution.

In touch: Cardiac and respiratory patterns synchronize during ensemble singing with physical contact

Musical ensemble performances provide an ideal environment to gain knowledge about complex human interactions. Network structures of synchronization can reflect specific roles of individual performers on the one hand and a higher level of organization of all performers as a superordinate system on the other. This study builds on research on joint singing, using hyperscanning of respiration and heart rate variability (HRV) from eight professional singers. Singers performed polyphonic music, distributing their breathing within the same voice and singing without and with physical contact: that is touching each other's shoulder or waist. The idea of singing with touch was motivated by historical depictions of ensemble performances that showed singers touching each other. It raises the question of the potential benefit of touch for group performances. From a psycho-physiological point of view, physical contact should increase the synchronization of singing coordination. The results confirm previous findings on synchronization of respiration and HRV during choir singing and extend those findings to a non-homophonic musical repertoire while also revealing an increase in synchronization in respiration during physical contact. These effects were significant across different frequency ranges. The effect of physical contact was stronger when all singers were singing in comparison to the partial ensemble. Importantly, the synchronization could not be fully explained by the singing action (i.e., singing the same voice, or singing vs. listening) or by the standing position or touch. This finding suggests a higher level of organization of all singers, forming a superordinate system.

The Haken-Kelso-Bunz (HKB) model: from matter to movement to mind

This article presents a brief retrospective on the Haken-Kelso-Bunz (HKB) model of certain dynamical properties of human movement. Though unanticipated, HKB introduced, and demonstrated the power of, a new vocabulary for understanding behavior, cognition and the brain, revealed through a visually compelling mathematical picture that accommodated highly reproducible experimental facts and predicted new ones. HKB stands as a harbinger of paradigm change in several scientific fields, the effects of which are still being felt. In particular, HKB constitutes the foundation of a mechanistic science of coordination called Coordination Dynamics that extends from matter to movement to mind, and beyond.

Unifying Large- and Small-Scale Theories of Coordination

Coordination is a ubiquitous feature of all living things. It occurs by virtue of informational coupling among component parts and processes and can be quite specific (as when cells in the brain resonate to signals in the environment) or nonspecific (as when simple diffusion creates a source–sink dynamic for gene networks). Existing theoretical models of coordination—from bacteria to brains to social groups—typically focus on systems with very large numbers of elements (N→∞) or systems with only a few elements coupled together (typically N = 2). Though sharing a common inspiration in Nature’s propensity to generate dynamic patterns, both approaches have proceeded largely independent of each other. Ideally, one would like a theory that applies to phenomena observed on all scales. Recent experimental research by Mengsen Zhang and colleagues on intermediate-sized ensembles (in between the few and the many) proves to be the key to uniting large- and small-scale theories of coordination. Disorder–order transitions, multistability, order–order phase transitions, and especially metastability are shown to figure prominently on multiple levels of description, suggestive of a basic Coordination Dynamics that operates on all scales. This unified coordination dynamics turns out to be a marriage of two well-known models of large- and small-scale coordination: the former based on statistical mechanics (Kuramoto) and the latter based on the concepts of Synergetics and nonlinear dynamics (extended Haken–Kelso–Bunz or HKB). We show that models of the many and the few, previously quite unconnected, are thereby unified in a single formulation. The research has led to novel topological methods to handle the higher-dimensional dynamics of coordination in complex systems and has implications not only for understanding coordination but also for the design of (biorhythm inspired) computers.

Embodying melody through a conducting baton: a pilot comparison between musicians and non-musicians

Finger-tapping tasks have been widely adopted to investigate auditory-motor synchronization, i.e., the coupling of movement with an external auditory rhythm. However, the discrete nature of these movements usually limits their application to the study of beat perception in the context of isochronous rhythms. The purpose of the present pilot study was to test an innovative task that allows investigating bodily responses to complex, non-isochronous rhythms. A conductor's baton was provided to 16 healthy subjects, divided into 2 different groups depending on the years of musical training they had received (musicians or non-musicians). Ad hoc-created melodies, including notes of different durations, were played to the subjects. Each subject was asked to move the baton up and down according to the changes in pitch contour. Software for video analysis and modelling (Tracker®) was used to track the movement of the baton tip. The main parameters used for the analysis were the velocity peaks in the vertical axis. In the musician group, the number of velocity peaks exactly matched the number of notes, while in the non-musician group, the number of velocity peaks exceeded the number of notes. An exploratory data analysis using Poincaré plots suggested a greater degree of coupling between hand-arm movements and melody in musicians both with isochronous and non-isochronous rhythms. The calculated root mean square error (RMSE) between the note onset times and the velocity peaks, and the analysis of the distribution of velocity peaks in relationship to note onset times confirmed the effect of musical training. Notwithstanding the small number of participants, these results suggest that this novel behavioural task could be used to investigate auditory-motor coupling in the context of music in an ecologically valid setting. Furthermore, the task may be used for rhythm training and rehabilitation in neurological patients with movement disorders.

Dynamic Functional Connectivity between order and randomness and its evolution across the human adult lifespan

Functional Connectivity (FC) during resting-state or task conditions is not static but inherently dynamic. Yet, there is no consensus on whether fluctuations in FC may resemble isolated transitions between discrete FC states rather than continuous changes. This quarrel hampers advancing the study of dynamic FC. This is unfortunate as the structure of fluctuations in FC can certainly provide more information about developmental changes, aging, and progression of pathologies. We merge the two perspectives and consider dynamic FC as an ongoing network reconfiguration, including a stochastic exploration of the space of possible steady FC states. The statistical properties of this random walk deviate both from a purely "order-driven" dynamics, in which the mean FC is preserved, and from a purely "randomness-driven" scenario, in which fluctuations of FC remain uncorrelated over time. Instead, dynamic FC has a complex structure endowed with long-range sequential correlations that give rise to transient slowing and acceleration epochs in the continuous flow of reconfiguration. Our analysis for fMRI data in healthy elderly revealed that dynamic FC tends to slow down and becomes less complex as well as more random with increasing age. These effects appear to be strongly associated with age-related changes in behavioural and cognitive performance.

Multisensory integration and behavioral stability

Information coming from multiple senses, as compared to a single one, typically enhances our performance. The multisensory improvement has been extensively examined in perception studies, as well as in tasks involving a motor response like a simple reaction time. However, how this effect extends to more complex behavior, typically involving the coordination of movements, such as bimanual coordination or walking, is still unclear. A critical element in achieving motor coordination in complex behavior is its stability. Reaching a stable state in the coordination pattern allows to sustain complex behavior over time (e.g., without interruption or negative consequences, like falling). This study focuses on the relation between stability in the coordination of movement patterns, like walking, and multisensory improvement. Participants walk with unimodal and audio-tactile metronomes presented either at their preferred rate or at a slower walking rate, the instruction being to synchronize their steps to the metronomes. Walking at a slower rate makes gait more variable than walking at the preferred rate. Interestingly however, the multimodal stimuli enhance the stability of motor coordination but only in the slower condition. Thus, the reduced stability of the coordination pattern (at a slower gait rate) prompts the sensorimotor system to capitalize on multimodal stimulation. These findings provide evidence of a new link between multisensory improvement and behavioral stability, in the context of ecological sensorimotor task.

Individual and dyadic rope turning as a window into social coordination

The spontaneous and intentional movement coordination between peoples is well understood. Less is known about such interactions when the coordination is subordinate to the task and when the task involves, next to vision, mechanically induced haptic and kinesthetic coupling between dyadic partners. We therefore investigated dyadic jump rope turning. Fifteen dyadic pairs conjointly turned a jump rope to which five markers were equidistantly attached, and whose movements were recorded in 3D. In addition, each participant turned one side of the rope while the other side was quasi-fixed in an individual baseline condition. The participants' goal was to turn the rope regularly and smoothly. Individual spontaneous turning frequencies differed substantially across participants. Yet, dyadic pairs spontaneously turned the rope at a common frequency, indicative of frequency entrainment. The dyadic rope rotations were less variable despite weaker between near-hand marker coordination than the individual rope rotations, and the degree of performance improvement was most pronounced for participants who were paired with a partner who performed better in the individual condition. The direction and relative strength of the coupling between partners varied substantially across dyads, but the degree of coupling asymmetry had no substantial effect on the rope tuning quality. The absolute degree in which dyadic partners adjusted to each other, however, scaled moderately with their turning performance. Although the individual performances did not predict the dyadic performances, the difference in individual performance between dyadic partners had some predictive value for the dyadic performance. In combination, these results indicate that the partners were functionally adapting to each other in order to satisfy the task goal and suggest that the relative performance differences rather than the individual performances has predictive value for conjoint action.

Walking to a multisensory beat

Living in a complex and multisensory environment demands constant interaction between perception and action. In everyday life it is common to combine efficiently simultaneous signals coming from different modalities. There is evidence of a multisensory benefit in a variety of laboratory tasks (temporal judgement, reaction time tasks). It is less clear if this effect extends to ecological tasks, such as walking. Furthermore, benefits of multimodal stimulation are linked to temporal properties such as the temporal window of integration and temporal recalibration. These properties have been examined in tasks involving single, non-repeating stimulus presentations. Here we investigate the same temporal properties in the context of a rhythmic task, namely audio-tactile stimulation during walking. The effect of audio-tactile rhythmic cues on gait variability and the ability to synchronize to the cues was studied in young adults. Participants walked with rhythmic cues presented at different stimulus-onset asynchronies. We observed a multisensory benefit by comparing audio-tactile to unimodal stimulation. Moreover, both the temporal window of integration and temporal recalibration mediated the response to multimodal stimulation. In sum, rhythmic behaviours obey the same principles as temporal discrimination and detection behaviours and thus can also benefit from multimodal stimulation.

Help me if I can't: Social interaction effects in adult contextual word learning

A major challenge in second language acquisition is to build up new vocabulary. How is it possible to identify the meaning of a new word among several possible referents? Adult learners typically use contextual information, which reduces the number of possible referents a new word can have. Alternatively, a social partner may facilitate word learning by directing the learner's attention toward the correct new word meaning. While much is known about the role of this form of 'joint attention' in first language acquisition, little is known about its efficacy in second language acquisition. Consequently, we introduce and validate a novel visual word learning game to evaluate how joint attention affects the contextual learning of new words in a second language. Adult learners either acquired new words in a constant or variable sentence context by playing the game with a knowledgeable partner, or by playing the game alone on a computer. Results clearly show that participants who learned new words in social interaction (i) are faster in identifying a correct new word referent in variable sentence contexts, and (ii) temporally coordinate their behavior with a social partner. Testing the learned words in a post-learning recall or recognition task showed that participants, who learned interactively, better recognized words originally learned in a variable context. While this result may suggest that interactive learning facilitates the allocation of attention to a target referent, the differences in the performance during recognition and recall call for further studies investigating the effect of social interaction on learning performance. In summary, we provide first evidence on the role joint attention in second language learning. Furthermore, the new interactive learning game offers itself to further testing in complex neuroimaging research, where the lack of appropriate experimental set-ups has so far limited the investigation of the neural basis of adult word learning in social interaction.

Play along: effects of music and social interaction on word learning

Learning new words is an increasingly common necessity in everyday life. External factors, among which music and social interaction are particularly debated, are claimed to facilitate this task. Due to their influence on the learner's temporal behavior, these stimuli are able to drive the learner's attention to the correct referent of new words at the correct point in time. However, do music and social interaction impact learning behavior in the same way? The current study aims to answer this question. Native German speakers (N = 80) were requested to learn new words (pseudo-words) during a contextual learning game. This learning task was performed alone with a computer or with a partner, with or without music. Results showed that music and social interaction had a different impact on the learner's behavior: Participants tended to temporally coordinate their behavior more with a partner than with music, and in both cases more than with a computer. However, when both music and social interaction were present, this temporal coordination was hindered. These results suggest that while music and social interaction do influence participants' learning behavior, they have a different impact. Moreover, impaired behavior when both music and a partner are present suggests that different mechanisms are employed to coordinate with the two types of stimuli. Whether one or the other approach is more efficient for word learning, however, is a question still requiring further investigation, as no differences were observed between conditions in a retrieval phase, which took place immediately after the learning session. This study contributes to the literature on word learning in adults by investigating two possible facilitating factors, and has important implications for situations such as music therapy, in which music and social interaction are present at the same time.

Sensorimotor synchronization: a review of recent research (2006-2012)

Sensorimotor synchronization (SMS) is the coordination of rhythmic movement with an external rhythm, ranging from finger tapping in time with a metronome to musical ensemble performance. An earlier review (Repp, 2005) covered tapping studies; two additional reviews (Repp, 2006a, b) focused on music performance and on rate limits of SMS, respectively. The present article supplements and extends these earlier reviews by surveying more recent research in what appears to be a burgeoning field. The article comprises four parts, dealing with (1) conventional tapping studies, (2) other forms of moving in synchrony with external rhythms (including dance and nonhuman animals' synchronization abilities), (3) interpersonal synchronization (including musical ensemble performance), and (4) the neuroscience of SMS. It is evident that much new knowledge about SMS has been acquired in the last 7 years.

The metastable brain

Neural ensembles oscillate across a broad range of frequencies and are transiently coupled or "bound" together when people attend to a stimulus, perceive, think, and act. This is a dynamic, self-assembling process, with parts of the brain engaging and disengaging in time. But how is it done? The theory of Coordination Dynamics proposes a mechanism called metastability, a subtle blend of integration and segregation. Tendencies for brain regions to express their individual autonomy and specialized functions (segregation, modularity) coexist with tendencies to couple and coordinate globally for multiple functions (integration). Although metastability has garnered increasing attention, it has yet to be demonstrated and treated within a fully spatiotemporal perspective. Here, we illustrate metastability in continuous neural and behavioral recordings, and we discuss theory and experiments at multiple scales, suggesting that metastable dynamics underlie the real-time coordination necessary for the brain's dynamic cognitive, behavioral, and social functions.

Challenges for the understanding of the dynamics of social coordination

The way people interact can be examined by looking at the way they move relative to each other. Seeking the principles behind those interactions have consequences potentially related to any type of interpersonal function, far beyond the so-called "motor" processes typically associated with the study of movements, be it perceptive, cognitive, affective, pragmatic, or epistemic. Here, we present the way the framework of coordination dynamics define and addresses the interactive actions in a dyad. We first introduce the basics of pattern formation as the roots of the theoretical approach of coordination dynamics, and then the way this framework may contribute to establish a solution to classify behaviors. Thereafter we review promising empirical results on the dynamics of interpersonal coordination, and finally discuss were to go next to decipher the way the coordination between two people and the way each individual contribute may be disentangled.

Action-perception coordination dynamics of whole-body rhythmic movement in stance: a comparison study of street dancers and non-dancers

This study investigated whether whole-body, rhythmic action-perception coordination in stance is organized in terms of dynamic principles. We observed whether phase transition and hysteresis occur during the execution of dancing movements. Nine skilled street dancers and 9 novice controls performed 2 types of rhythmic knee-bending movements to a metronome beat in the standing position. Participants performed down-on-the-beat (in which knee flexion coincides with the beat) and up-on-the-beat (in which knee extension coincides with the beat), which are both typical components of street dance. All participants were instructed not to intervene in the pattern change. The auditory stimulus beat rate increased or decreased between 60 and 220 beats per minute (bpm) in steps of 20 bpm. We calculated the phase angle of beat time that is superposed on knee movement trajectory on a phase plane. Under the up-on-the-beat condition, phase transition and hysteresis were observed. The bifurcation frequency at which phase transition occurred significantly differed between groups, indicating that dancers were able to perform up-on-the-beat at higher movement frequencies than non-dancers. This suggests that dynamical properties may differ between Dancers and Non-dancers. The present results provide additional evidence that whole-body action-perception pattern formation is governed by general and common dynamical principles.

Continuity of visual and auditory rhythms influences sensorimotor coordination

People often coordinate their movement with visual and auditory environmental rhythms. Previous research showed better performances when coordinating with auditory compared to visual stimuli, and with bimodal compared to unimodal stimuli. However, these results have been demonstrated with discrete rhythms and it is possible that such effects depend on the continuity of the stimulus rhythms (i.e., whether they are discrete or continuous). The aim of the current study was to investigate the influence of the continuity of visual and auditory rhythms on sensorimotor coordination. We examined the dynamics of synchronized oscillations of a wrist pendulum with auditory and visual rhythms at different frequencies, which were either unimodal or bimodal and discrete or continuous. Specifically, the stimuli used were a light flash, a fading light, a short tone and a frequency-modulated tone. The results demonstrate that the continuity of the stimulus rhythms strongly influences visual and auditory motor coordination. Participants' movement led continuous stimuli and followed discrete stimuli. Asymmetries between the half-cycles of the movement in term of duration and nonlinearity of the trajectory occurred with slower discrete rhythms. Furthermore, the results show that the differences of performance between visual and auditory modalities depend on the continuity of the stimulus rhythms as indicated by movements closer to the instructed coordination for the auditory modality when coordinating with discrete stimuli. The results also indicate that visual and auditory rhythms are integrated together in order to better coordinate irrespective of their continuity, as indicated by less variable coordination closer to the instructed pattern. Generally, the findings have important implications for understanding how we coordinate our movements with visual and auditory environmental rhythms in everyday life.

Beyond the blank slate: routes to learning new coordination patterns depend on the intrinsic dynamics of the learner-experimental evidence and theoretical model

Using an approach that combines experimental studies of bimanual movements to visual stimuli and theoretical modeling, the present paper develops a dynamical account of sensorimotor learning, that is, how new skills are acquired and old ones modified. A significant aspect of our approach is the focus on the individual learner as the basic unit of analysis, in particular the quantification of predispositions and capabilities that the individual learner brings to the learning environment. Such predispositions constitute the learner's behavioral repertoire, captured here theoretically as a dynamical landscape (“intrinsic dynamics”). The learning process is demonstrated to not only lead to a relatively permanent improvement of performance in the required task—the usual outcome—but also to alter the individual's entire repertoire. Changes in the dynamical landscape due to learning are shown to result from two basic mechanisms or “routes”: bifurcation and shift. Which mechanism is selected depends the initial individual repertoire before new learning begins. Both bifurcation and shift mechanisms are accommodated by a dynamical model, a relatively straightforward development of the well-established HKB model of movement coordination. Model simulations show that although environmental or task demands may be met equally well using either mechanism, the bifurcation route results in greater stabilization of the to-be-learned behavior. Thus, stability not (or not only) error is demonstrated to be the basis of selection, both of a new pattern of behavior and the path (smooth shift versus abrupt qualitative change) that learning takes. In line with these results, recent neurophysiological evidence indicates that stability is a relevant feature around which brain activity is organized while an individual performs a coordination task. Finally, we explore the consequences of the dynamical approach to learning for theories of biological change.

Somatosensory driven interpersonal synchrony during rhythmic sway

Spontaneous synchrony emerges between individuals performing together rhythmic activities while communicating by means of sensory feedback. In this study, we examined the nature of interpersonal synchrony mediated by light fingertip contact when individuals sway rhythmically in the sagittal plane. The effect of traditional dance expertise on interpersonal synchrony was investigated. Sixty participants (30 dancers, 30 novices) formed three types of couples (10 expert couples, 10 novice couples, 10 mixed couples) and performed a rhythmical sway task (40s) that was either self or metronome paced (frequency: 0.25Hz). Cross spectral analysis of the center of pressure (CoP) displacement signals revealed that during self-paced sway fingertip contact evoked a decrease of the dominant sway frequency difference between partners, an increase in the coherence between the sway signals and a concentration of relative phase angles towards the in-phase (0°-20°) region. In metronome paced sway however, only expert dancers were able to benefit from haptic contact to further improve interpersonal synchrony. These findings suggest that haptic contact can stabilize the spontaneous coordination dynamics of two persons performing rhythmic sway together. The strength of the emerged synchrony depends on the individuals' expertise to integrate tactile and auditory information about sway.

Multistability and metastability: understanding dynamic coordination in the brain

Multistable coordination dynamics exists at many levels, from multifunctional neural circuits in vertebrates and invertebrates to large-scale neural circuitry in humans. Moreover, multistability spans (at least) the domains of action and perception, and has been found to place constraints upon, even dictating the nature of, intentional change and the skill-learning process. This paper reviews some of the key evidence for multistability in the aforementioned areas, and illustrates how it has been measured, modelled and theoretically understood. It then suggests how multistability—when combined with essential aspects of coordination dynamics such as instability, transitions and (especially) metastability—provides a platform for understanding coupling and the creative dynamics of complex goal-directed systems, including the brain and the brain–behaviour relation.

The interactive brain hypothesis

Enactive approaches foreground the role of interpersonal interaction in explanations of social understanding. This motivates, in combination with a recent interest in neuroscientific studies involving actual interactions, the question of how interactive processes relate to neural mechanisms involved in social understanding. We introduce the Interactive Brain Hypothesis (IBH) in order to help map the spectrum of possible relations between social interaction and neural processes. The hypothesis states that interactive experience and skills play enabling roles in both the development and current function of social brain mechanisms, even in cases where social understanding happens in the absence of immediate interaction. We examine the plausibility of this hypothesis against developmental and neurobiological evidence and contrast it with the widespread assumption that mindreading is crucial to all social cognition. We describe the elements of social interaction that bear most directly on this hypothesis and discuss the empirical possibilities open to social neuroscience. We propose that the link between coordination dynamics and social understanding can be best grasped by studying transitions between states of coordination. These transitions form part of the self-organization of interaction processes that characterize the dynamics of social engagement. The patterns and synergies of this self-organization help explain how individuals understand each other. Various possibilities for role-taking emerge during interaction, determining a spectrum of participation. This view contrasts sharply with the observational stance that has guided research in social neuroscience until recently. We also introduce the concept of readiness to interact to describe the practices and dispositions that are summoned in situations of social significance (even if not interactive). This latter idea links interactive factors to more classical observational scenarios.

Coordination dynamics in a socially situated nervous system

Traditional theories of cognitive science have typically accounted for the organization of human behavior by detailing requisite computational/representational functions and identifying neurological mechanisms that might perform these functions. Put simply, such approaches hold that neural activity causes behavior. This same general framework has been extended to accounts of human social behavior via concepts such as “common-coding” and “co-representation” and much recent neurological research has been devoted to brain structures that might execute these social-cognitive functions. Although these neural processes are unquestionably involved in the organization and control of human social interactions, there is good reason to question whether they should be accorded explanatory primacy. Alternatively, we propose that a full appreciation of the role of neural processes in social interactions requires appropriately situating them in their context of embodied-embedded constraints. To this end, we introduce concepts from dynamical systems theory and review research demonstrating that the organization of human behavior, including social behavior, can be accounted for in terms of self-organizing processes and lawful dynamics of animal-environment systems. Ultimately, we hope that these alternative concepts can complement the recent advances in cognitive neuroscience and thereby provide opportunities to develop a complete and coherent account of human social interaction.

Behavioral impact of unisensory and multisensory audio-tactile events: pros and cons for interlimb coordination in juggling

Recent behavioral neuroscience research revealed that elementary reactive behavior can be improved in the case of cross-modal sensory interactions thanks to underlying multisensory integration mechanisms. Can this benefit be generalized to an ongoing coordination of movements under severe physical constraints? We choose a juggling task to examine this question. A central issue well-known in juggling lies in establishing and maintaining a specific temporal coordination among balls, hands, eyes and posture. Here, we tested whether providing additional timing information about the balls and hands motions by using external sound and tactile periodic stimulations, the later presented at the wrists, improved the behavior of jugglers. One specific combination of auditory and tactile metronome led to a decrease of the spatiotemporal variability of the juggler's performance: a simple sound associated to left and right tactile cues presented antiphase to each other, which corresponded to the temporal pattern of hands movement in the juggling task. A contrario, no improvements were obtained in the case of other auditory and tactile combinations. We even found a degraded performance when tactile events were presented alone. The nervous system thus appears able to integrate in efficient way environmental information brought by different sensory modalities, but only if the information specified matches specific features of the coordination pattern. We discuss the possible implications of these results for the understanding of the neuronal integration process implied in audio-tactile interaction in the context of complex voluntary movement, and considering the well-known gating effect of movement on vibrotactile perception.

Coordination dynamics of elliptic shape drawing: effects of orientation and eccentricity

The aim of the present study was to show the effect of orientation on producing ellipses of various eccentricities. Ten right-handed adults were required to reproduce eight series of 84 ellipses on a graphic tablet at spontaneous speed. The ellipses displayed seven eccentricities, corresponding to the Lissajous plot produced by two orthogonal harmonic oscillators whose relative phase (RP) varied from 0° (a line) to 90° (a circle), and twelve orientations, the long axis of the ellipses aligned in a direction ranging from three to nine o'clock with respect to the body transverse plane. Results confirmed the participant's preference for two eccentricities, 0° and 45°, that is, a line and an ellipse of intermediate eccentricity, respectively. Except for the circle, insensitive to orientation by definition, orientation became a cogent constraint: In the 10:30 o'clock direction, line drawing (0° RP) increased in variability and the intermediate ellipse (45° RP) was attracted towards a circle (90° RP). These findings suggest an interaction between orientation effects, putatively due to differences in the motion of the wrist and fingers, and eccentricity effects, which may pertain to preferences arising from the non-linear coupling between the effectors.

Neuroimaging coordination dynamics in the sport sciences

Key methodological issues for designing, analyzing, and interpreting neuroimaging experiments are presented from the perspective of the framework of Coordination Dynamics. To this end, a brief overview of Coordination Dynamics is introduced, including the main concepts of control parameters and collective variables, theoretical modeling, novel experimental paradigms, and cardinal empirical findings. Basic conceptual and methodological issues for the design and implementation of coordination experiments in the context of neuroimaging are discussed. The paper concludes with a presentation of neuroimaging findings central to understanding the neural basis of coordination and addresses their relevance for the sport sciences. The latter include but are not restricted to learning and practice-related issues, the role of mental imagery, and the recovery of function following brain injury.

Social coordination in toddler's word learning: interacting systems of perception and action

We measured turn-taking in terms of hand and head movements and asked if the global rhythm of the participants' body activity relates to word learning. Six dyads composed of parents and toddlers (M = 18 months) interacted in a tabletop task wearing motion-tracking sensors on their hands and head. Parents were instructed to teach the labels of 10 novel objects and the child was later tested on a name-comprehension task. Using dynamic time warping, we compared the motion data of all body-part pairs, within and between partners. For every dyad, we also computed an overall measure of the quality of the interaction, that takes into consideration the state of interaction when the parent uttered an object label and the overall smoothness of the turn-taking. The overall interaction quality measure was correlated with the total number of words learned.In particular, head movements were inversely related to other partner's hand movements, and the degree of bodily coupling of parent and toddler predicted the words that children learned during the interaction. The implications of joint body dynamics to understanding joint coordination of activity in a social interaction, its scaffolding effect on the child's learning and its use in the development of artificial systems are discussed.

Social coordination dynamics: measuring human bonding

Spontaneous social coordination has been extensively described in natural settings but so far no controlled methodological approaches have been employed that systematically advance investigations into the possible self-organized nature of bond formation and dissolution between humans. We hypothesized that, under certain contexts, spontaneous synchrony-a well-described phenomenon in biological and physical settings-could emerge spontaneously between humans as a result of information exchange. Here, a new way to quantify interpersonal interactions in real time is proposed. In a simple experimental paradigm, pairs of participants facing each other were required to actively produce actions, while provided (or not) with the vision of similar actions being performed by someone else. New indices of interpersonal coordination, inspired by the theoretical framework of coordination dynamics (based on relative phase and frequency overlap between movements of individuals forming a pair) were developed and used. Results revealed that spontaneous phase synchrony (i.e., unintentional in-phase coordinated behavior) between two people emerges as soon as they exchange visual information, even if they are not explicitly instructed to coordinate with each other. Using the same tools, we also quantified the degree to which the behavior of each individual remained influenced by the social encounter even after information exchange had been removed, apparently a kind of social memory.

Inter- versus intramodal integration in sensorimotor synchronization: a combined behavioral and magnetoencephalographic study

Although the temporal occurrence of the pacing signal is predictable in sensorimotor synchronization tasks, normal subjects perform on-the-beat-tapping to an isochronous auditory metronome with an anticipatory error. This error originates from an intermodal task, that is, subjects have to bring information from the auditory and tactile modality to coincide. The aim of the present study was to illuminate whether the synchronization error is a finding specific to an intermodal timing task and whether the underlying cortical mechanisms are modality-specific or supramodal. We collected behavioral data and cortical evoked responses by magneto-encephalography (MEG) during performance of cross- and unimodal tapping-tasks. As expected, subjects showed negative asynchrony in performing an auditorily paced tapping task. However, no asynchrony emerged during tactile pacing, neither during pacing at the opposite finger nor at the toe. Analysis of cortical signals resulted in a three dipole model best explaining tap-contingent activity in all three conditions. The temporal behavior of the sources was similar between the conditions and, thus, modality independent. The localization of the two earlier activated sources was modality-independent as well whereas location of the third source varied with modality. In the auditory pacing condition it was localized in contralateral primary somatosensory cortex, during tactile pacing it was localized in contralateral posterior parietal cortex. In previous studies with auditory pacing the functional role of this third source was contradictory: A special temporal coupling pattern argued for involvement of the source in evaluating the temporal distance between tap and click whereas subsequent data gave no evidence for such an interpretation. Present data shed new light on this question by demonstrating differences between modalities in the localization of the third source with similar temporal behavior.

The perception-action interaction comes first

Dijkerman & de Haan (D&dH) study perception and action as two independent processes. However, in all daily activities the processes are completely intertwined, so it is difficult to separate one from the other. Humans perceive in order to move and also move in order to perceive. Understanding first how perception and action are coordinated, leads us then to determine how each component works independently.

Multisensory integration for timing engages different brain networks

How does the brain integrate information from different senses into a unitary percept? What factors influence such multisensory integration? Using a rhythmic behavioral paradigm and functional magnetic resonance imaging, we identified networks of brain regions for perceptions of physically synchronous and asynchronous auditory-visual events. Measures of behavioral performance revealed the existence of three distinct perceptual states. Perception of asynchrony activated a network of the primary sensory, prefrontal, and inferior parietal cortices, perception of synchrony disengaged the inferior parietal cortex and further recruited the superior colliculus, and when no clear percept was established, only the residual areas comprised of prefrontal and sensory areas were active. These results indicate that distinct percepts arise within specific brain sub-networks, the components of which are differentially engaged and disengaged depending on the timing of environmental signals.