Sensorimotor Synchronization With Auditory and Visual Modalities: Behavioral and Neural Differences

Perceptual coupling in human dyads: Kinematics does not affect interpersonal synchronization

The minimal, essential condition for individuals to interact is that they exchange information via at least one sensory channel. Once informational coupling is established, it enables basic forms of coordinated behavior to spontaneously emerge from the interaction. Our previous study revealed different coordination dynamics in dyads engaged in a joint finger-tapping task based on visual versus auditory coupling. This observation led us to propose the 'modality-dependent hypothesis', which posits that coordination dynamics are influenced by the sensory modality mediating informational coupling. However, recognizing that different modalities have inherent differences in accessing spatiotemporal features of perceived movement, we formulated the alternative 'kinematic hypothesis'. This hypothesis posits that differences in dynamics would vanish given equivalent kinematic information across modalities. The study involved forty (N = 40) participants, grouped into twenty (N = 20) dyads, who engaged in a joint finger-tapping task. This task was conducted under varying conditions of visual and auditory coupling, with manipulations in the access to kinematic information, categorized as discrete and continuous. Contrary to our initial predictions, the results strongly supported the 'modality-dependent hypothesis'. We observed that visual and auditory coupling consistently yielded distinct attractor dynamics, regardless of the access to kinematic information. Furthermore, all conditions of auditory coupling resulted in higher levels of synchronization than their visual counterparts. These findings suggest that the differences in interpersonal synchronization are predominantly influenced by the sensory modality, rather than the continuity of kinematic information. Our study highlights the significance of sensorimotor interactions in interpersonal synchronization and addresses the potential of sonification strategies in supporting motor training and rehabilitation.

The contribution of auditory imagery and visual rhythm perception to sensorimotor synchronization with external and imagined rhythm

Sensorimotor synchronization (SMS) refers to the temporal coordination of an external stimulus with movement. Our previous work revealed that while SMS with visual flashing patterns was less consistent than with auditory or tactile patterns, it was still evident in a sample of nonmusicians. Although previous studies have speculated the potential role of auditory imagery, its contribution to visual SMS performance is not well quantified. Utilizing a synchronization-continuation finger-tapping task with a visual stimulus that included implied motion, we aimed to examine how participants' imagery ability, musicality, and rhythm perception affected SMS performance. We quantified participants' SMS consistency in synchronization (with visual cues) and continuation (without visual cues) phases. Participants also performed a perception task assessing their ability to detect temporal perturbations in the visual rhythm and completed musical ability and imagery questionnaires. Our linear regression model for SMS consistency included the trial phase, self-reported auditory imagery control and musicality, and visual rhythm perception as predictors. Significant effects of trial phase and auditory imagery scores on SMS consistency suggested that participants performed SMS more consistently while the guiding visual stimulus was present and that the higher one's self-reported auditory imagery ability, the better their SMS when continuing with unguided rhythm. One's visual rhythm perception accuracy significantly correlated with SMS consistency during the synchronization phase, and there was no correlation between rhythm perception and auditory imagery control. Overall, our results suggested relatively independent contributions of auditory imagery and visual rhythm perception to SMS with visual rhythm. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Application of individual brain connectome in chronic ischemia: mapping symptoms before and after reperfusion

How brain functions in the distorted ischemic state before and after reperfusion is unclear. It is also uncertain whether there are any indicators within ischemic brain that could predict surgical outcomes. To alleviate these issues, we applied individual brain connectome in chronic steno-occlusive vasculopathy (CSOV) to map both ischemic symptoms and their postbypass changes. A total of 499 bypasses in 455 CSOV patients were collected and followed up for 47.8 ± 20.5 months. Using multimodal parcellation with connectivity-based and pathological distortion-independent approach, areal MR features of brain connectome were generated with three measurements of functional connectivity (FC), structural connectivity, and PageRank centrality at the single-subject level. Thirty-three machine-learning models were then trained with clinical and areal MR features to obtain acceptable classifiers for both ischemic symptoms and their postbypass changes, among which, 11 were deemed acceptable (AUC > 0.7). Notably, the FC feature-based model for long-term neurological outcomes performed very well (AUC > 0.8). Finally, a Shapley additive explanations plot was adopted to extract important individual features in acceptable models to generate "fingerprints" of brain connectome. This study not only establishes brain connectomic fingerprint databases for brain ischemia with distortion, but also provides informative insights for how brain functions before and after reperfusion.

Embodying Time in the Brain: A Multi-Dimensional Neuroimaging Meta-Analysis of 95 Duration Processing Studies

Time is an omnipresent aspect of almost everything we experience internally or in the external world. The experience of time occurs through such an extensive set of contextual factors that, after decades of research, a unified understanding of its neural substrates is still elusive. In this study, following the recent best-practice guidelines, we conducted a coordinate-based meta-analysis of 95 carefully-selected neuroimaging papers of duration processing. We categorized the included papers into 14 classes of temporal features according to six categorical dimensions. Then, using the activation likelihood estimation (ALE) technique we investigated the convergent activation patterns of each class with a cluster-level family-wise error correction at p < 0.05. The regions most consistently activated across the various timing contexts were the pre-SMA and bilateral insula, consistent with an embodied theory of timing in which abstract representations of duration are rooted in sensorimotor and interoceptive experience, respectively. Moreover, class-specific patterns of activation could be roughly divided according to whether participants were timing auditory sequential stimuli, which additionally activated the dorsal striatum and SMA-proper, or visual single interval stimuli, which additionally activated the right middle frontal and inferior parietal cortices. We conclude that temporal cognition is so entangled with our everyday experience that timing stereotypically common combinations of stimulus characteristics reactivates the sensorimotor systems with which they were first experienced.

Neuroimaging Signatures of Metacognitive Improvement in Sensorimotor Timing

Error monitoring is an essential human ability underlying learning and metacognition. In the time domain, humans possess a remarkable ability to learn and adapt to temporal intervals, yet the neural mechanisms underlying this are not clear. Recently, we demonstrated that humans improve sensorimotor time estimates when given the chance to incorporate previous trial feedback ( Bader and Wiener, 2021), suggesting that humans are metacognitively aware of their own timing errors. To test the neural basis of this metacognitive ability, human participants of both sexes underwent fMRI while they performed a visual temporal reproduction task with randomized supra-second intervals (1.5-6 s). Crucially, each trial was repeated following feedback, allowing a "re-do" to learn from the successes or errors in the initial trial. Behaviorally, we replicated our previous finding of improved re-do trial performance despite temporally uninformative (i.e., early or late) feedback. For neuroimaging, we observed a dissociation between estimating and reproducing time intervals. Estimation engaged the default mode network (DMN), including the superior frontal gyri, precuneus, and posterior cingulate, whereas reproduction activated regions associated traditionally with the "timing network" (TN), including the supplementary motor area (SMA), precentral gyrus, and right supramarginal gyrus. Notably, greater and more extensive DMN involvement was observed in re-do trials, whereas for the TN, it was more constrained. Task-based connectivity between these networks demonstrated higher inter-network correlation primarily when estimating initial trials, while re-do trial communication was higher during reproduction. Overall, these results suggest that the DMN and TN jointly mediate subjective self-awareness to improve timing performance.

Augmented Visual Feedback for Complex Motor Skill Acquisition: A Demonstration with Healthy Young Adults

Mirror therapy (MT) is a treatment for improving motor function after stroke. Video therapy (VT) training combines observation and imitation of video clips, and it has been used to conduct efficient occupational therapy. We sought to determine the effects of MT and VT on tool-use with healthy young adults. We assigned participants to three different training groups in which they used their non-dominant left hands to move a ball with chopsticks: (a) a self-paced MT group (N = 14), (b) an MT group who moved the ball while looking at the mirror image of the right hand with a sound cue (4-second intervals) (N = 12), and (c) a group who imitated a video demonstration (4-second intervals) (N = 13). The ball-moving time was significantly reduced after training in all three groups: MT group (p < .001), MT-R group (p = .010), and VT group (p = .014), but the video group showed significantly greater relative improvements in motion variability (p = .030) and object prediction (p = .008).We concluded that MT was as effective as VT in improving movement speed, but, with these healthy young adults, MT was less effective than VT in learning speed control or hand pre-shaping to refine task movements for tool use.

Amodal population clock in the primate medial premotor system for rhythmic tapping

The neural substrate for beat extraction and response entrainment to rhythms is not fully understood. Here we analyze the activity of medial premotor neurons in monkeys performing isochronous tapping guided by brief flashing stimuli or auditory tones. The population dynamics shared the following properties across modalities: the circular dynamics of the neural trajectories form a regenerating loop for every produced interval; the trajectories converge in similar state space at tapping times resetting the clock; and the tempo of the synchronized tapping is encoded in the trajectories by a combination of amplitude modulation and temporal scaling. Notably, the modality induces displacement in the neural trajectories in the auditory and visual subspaces without greatly altering the time-keeping mechanism. These results suggest that the interaction between the medial premotor cortex's amodal internal representation of pulse and a modality-specific external input generates a neural rhythmic clock whose dynamics govern rhythmic tapping execution across senses.

Rhythmic tapping to a moving beat motion kinematics overrules natural gravity

Beat induction is the cognitive ability that allows humans to listen to a regular pulse in music and move in synchrony with it. Although auditory rhythmic cues induce more consistent synchronization than flashing visual metronomes, this auditory-visual asymmetry can be canceled by visual moving stimuli. Here, we investigated whether the naturalness of visual motion or its kinematics could provide a synchronization advantage over flashing metronomes. Subjects were asked to tap in sync with visual metronomes defined by vertically accelerating/decelerating motion, either congruent or not with natural gravity; horizontally accelerating/decelerating motion; or flashing stimuli. We found that motion kinematics was the predominant factor determining rhythm synchronization, as accelerating moving metronomes in any cardinal direction produced more precise and predictive tapping than decelerating or flashing conditions. Our results support the notion that accelerating visual metronomes convey a strong sense of beat, as seen in the cueing movements of an orchestra director.

Differentiated, rather than shared, strategies for time-coordinated action in social and non-social domains in autistic individuals

Autism spectrum disorder (ASD) is a neurodevelopmental condition with a highly heterogeneous adult phenotype that includes social and non-social behavioral characteristics. The link between the characteristics assignable to the different domains remains unresolved. One possibility is that social and non-social behaviors in autism are modulated by a common underlying deficit. However, here we report evidence supporting an alternative concept that is individual-centered rather than deficit-centered. Individuals are assumed to have a distinctive style in the strategies they adopt to perform social and non-social tasks with these styles presumably being structured differently between autistic individuals and typically-developed (TD) individuals. We tested this hypothesis for the execution of time-coordinated (synchronized) actions. Participants performed (i) a social task that required synchronized gaze and pointing actions to interact with another person, and (ii) a non-social task that required finger-tapping actions synchronized to periodic stimuli at different time-scales and sensory modalities. In both tasks, synchronization behavior differed between ASD and TD groups. However, a principal component analysis of individual behaviors across tasks revealed associations between social and non-social features for the TD persons but such cross-domain associations were strikingly absent for autistic individuals. The highly differentiated strategies between domains in ASD are inconsistent with a general synchronization deficit and instead highlight the individualized developmental heterogeneity in the acquisition of domain-specific behaviors. We propose a cognitive model to help disentangle individual-centered from deficit-centered effects in other domains. Our findings reinforce the importance to identify individually differentiated phenotypes to personalize autism therapies.

The same phase creates a unique visual rhythm unifying moving elements in time

Attention can be selectively tuned to particular features at different spatial locations or objects. The deployment of attention can be guided by properties, such as color, orientation, and so forth, as guiding features. What might be such guiding features for visual stimuli under dynamic rhythmic conditions? We asked specifically what might be the parameters that attract attention when perceiving a visual rhythm. We used a visual search paradigm, in which a dynamic search display consisted of vertically "bouncing balls" with regular rhythms. The search target was defined by a unique visual rhythm (i.e., with either a shorter or longer period) among rhythmic distractors sharing an identical period. We modulated amplitudes and phases of the distractor balls systematically. The results showed a crucial factor of the phase, not the amplitude. If the phase is violated, the target suddenly "pops out" as an "oddball," showing an efficient parallel search. The findings indicate in general the essential role of the phase in conjunction with amplitude and period for visual rhythm perception. Furthermore, a higher saliency of moving objects with a higher frequency component has also been disclosed.

Multiple levels of contextual influence on action-based timing behavior and cortical activation

Procedures used to elicit both behavioral and neurophysiological data to address a particular cognitive question can impact the nature of the data collected. We used functional near-infrared spectroscopy (fNIRS) to assess performance of a modified finger tapping task in which participants performed synchronized or syncopated tapping relative to a metronomic tone. Both versions of the tapping task included a pacing phase (tapping with the tone) followed by a continuation phase (tapping without the tone). Both behavioral and brain-based findings revealed two distinct timing mechanisms underlying the two forms of tapping. Here we investigate the impact of an additional-and extremely subtle-manipulation of the study's experimental design. We measured responses in 23 healthy adults as they performed the two versions of the finger-tapping tasks either blocked by tapping type or alternating from one to the other type during the course of the experiment. As in our previous study, behavioral tapping indices and cortical hemodynamics were monitored, allowing us to compare results across the two study designs. Consistent with previous findings, results reflected distinct, context-dependent parameters of the tapping. Moreover, our results demonstrated a significant impact of study design on rhythmic entrainment in the presence/absence of auditory stimuli. Tapping accuracy and hemodynamic responsivity collectively indicate that the block design context is preferable for studying action-based timing behavior.

Associations between abnormal spontaneous neural activity and clinical variables, eye movements, and event-related potential indicators in major depressive disorder

Background

This study aimed to investigate the correlations between abnormal spontaneous neural activity measured with fractional amplitude of low-frequency fluctuations (fALFF) and clinical variables, eye movements, and event-related potential indicators in patients with major depressive disorder (MDD).

Methods

We recruited 42 patients with MDD and 42 healthy controls (HCs) and collected their clinical variables, eye movement, event-related potential, and resting-state functional magnetic resonance imaging (rs-fMRI) data. The fALFF, support vector machine (SVM), and correlation analysis were used to analyze the data.

Results

The results of the study showed that the fALFF values of the sensorimotor network, including the right middle temporal gyrus, right cerebellar Crus2, left occipital gyrus, and left middle temporal gyrus, were significantly higher compared to HCs. Correlation analysis showed that the abnormal fALFF value of the right cerebellar Crus2 was inversely correlated with the active coping scores of the Simplified Coping Style Questionnaire in the patients (r = -0.307, p = 0.048). No correlation was observed between abnormal fALFF values and other clinical symptoms, neuropsychological tests, eye movements, and event-related potential-related indicators in patients with MDD. fALFF values in the left middle temporal gyrus could be used to distinguish patients with MDD from HCs with an accuracy of 78.57%.

Conclusions

Patients with MDD exhibited enhanced spontaneous neural activity in the sensorimotor network. No associations were found between abnormal spontaneous neural activity and clinical variables, eye movements, and event-related potential related indicators in MDD.

Why art? The role of arts in arts and health

This article is an answer to a report called "What is the evidence on the role of the arts in improving health and well-being?" The authors conclude that the arts have an impact on mental and physical health. Yet, the question of the role of the arts remains unanswered. What is and what is not an art effect? Recently, embodied theory has inspired articles on the perception of art. These articles have not yet received attention in the field of Arts and Health. Scholars in psychosomatic medicine have argued for an approach based on recent work in enactive embodied theory to investigate the connection between the body and the mind. The present article examines how key concepts in this theory relate to art. This leads to a discussion of art in terms of empathy-the relation between the internal state of the artist and the internal state of the beholder. I exemplify with a conceptual framework of musical empathy. Implications for health are addressed.

Parkinson’s disease deficits in time perception to auditory as well as visual stimuli – A large online study

Cognitive deficits are common in Parkinson’s disease (PD) and range from mild cognitive impairment to dementia, often dramatically reducing quality of life. Physiological models have shown that attention and memory are predicated on the brain’s ability to process time. Perception has been shown to be increased or decreased by activation or deactivation of dopaminergic neurons respectively. Here we investigate differences in time perception between patients with PD and healthy controls. We have measured differences in sub-second- and second-time intervals. Sensitivity and error in perception as well as the response times are calculated. Additionally, we investigated intra-individual response variability and the effect of participant devices on both reaction time and sensitivity. Patients with PD have impaired sensitivity in discriminating between durations of both visual and auditory stimuli compared to healthy controls. Though initially designed as an in-person study, because of the pandemic the experiment was adapted into an online study. This adaptation provided a unique opportunity to enroll a larger number of international participants and use this study to evaluate the feasibility of future virtual studies focused on cognitive impairment. To our knowledge this is the only time perception study, focusing on PD, which measures the differences in perception using both auditory and visual stimuli. The cohort involved is the largest to date, comprising over 800 participants.

Differential motor system entrainment to auditory and visual rhythms

Perception of, and synchronization to, auditory rhythms is known to be more accurate than with flashing visual rhythms. The motor system is known to play a role in the processing of timing information for auditory rhythm perception, but it is unclear if the motor system plays the same role for visual rhythm perception. One demonstrated component of auditory rhythm perception is neural entrainment at the frequency of the auditory rhythm. In this study, we use EEG to measure the entrainment of both auditory and visual rhythms from the motor cortex while subjects either tapped in synchrony with or passively attended to the presented rhythms. To isolate activity from motor cortex, we used independent component analysis to first separate out neural sources, then selected components using a combination of component topography, dipole location, mu activation, and beta modulation. This process took advantage of the fact that tapping activity results in reduced mu power, and characteristic beta modulation, which helped select motor components. Our findings suggest neural entrainment in motor components was stronger for visual rhythms than auditory rhythms and strongest during the tapping conditions for both modalities. We also find mu power increased in response to both auditory and visual rhythms. These findings indicate that the generally greater rhythm perception capabilities of the auditory system over the visual system may not depend entirely on neural entrainment in the motor system, but rather how the motor system is able to use the timing information made available to it. NEW & NOTEWORTHY We investigated neural entrainment in the motor system for both auditory and visual isochronous rhythms using electroencephalogram. Counter to expectations, our findings suggest stronger entrainment for visual rhythms than for auditory rhythms. Motor system activity was isolated with a novel procedure using independent component analysis as a means of blind source separation, along with known markers of mu activity from the motor system to identify motor components.

Identifying a brain network for musical rhythm: A functional neuroimaging meta-analysis and systematic review

We conducted a systematic review and meta-analysis of 30 functional magnetic resonance imaging studies investigating processing of musical rhythms in neurotypical adults. First, we identified a general network for musical rhythm, encompassing all relevant sensory and motor processes (Beat-based, rest baseline, 12 contrasts) which revealed a large network involving auditory and motor regions. This network included the bilateral superior temporal cortices, supplementary motor area (SMA), putamen, and cerebellum. Second, we identified more precise loci for beat-based musical rhythms (Beat-based, audio-motor control, 8 contrasts) in the bilateral putamen. Third, we identified regions modulated by beat based rhythmic complexity (Complexity, 16 contrasts) which included the bilateral SMA-proper/pre-SMA, cerebellum, inferior parietal regions, and right temporal areas. This meta-analysis suggests that musical rhythm is largely represented in a bilateral cortico-subcortical network. Our findings align with existing theoretical frameworks about auditory-motor coupling to a musical beat and provide a foundation for studying how the neural bases of musical rhythm may overlap with other cognitive domains.

When Visual Cues Do Not Help the Beat: Evidence for a Detrimental Effect of Moving Point-Light Figures on Rhythmic Priming

Rhythm perception involves strong auditory-motor connections that can be enhanced with movement. However, it is unclear whether just seeing someone moving to a rhythm can enhance auditory-motor coupling, resulting in stronger entrainment. Rhythmic priming studies show that presenting regular rhythms before naturally spoken sentences can enhance grammaticality judgments compared to irregular rhythms or other baseline conditions. The current study investigated whether introducing a point-light figure moving in time with regular rhythms could enhance the rhythmic priming effect. Three experiments revealed that the addition of a visual cue did not benefit rhythmic priming in comparison to auditory conditions with a static image. In Experiment 1 (27 7–8-year-old children), grammaticality judgments were poorer after audio-visual regular rhythms (with a bouncing point-light figure) compared to auditory-only regular rhythms. In Experiments 2 (31 adults) and 3 (31 different adults), there was no difference in grammaticality judgments after audio-visual regular rhythms compared to auditory-only irregular rhythms for either a bouncing point-light figure (Experiment 2) or a swaying point-light figure (Experiment 3). Comparison of the observed performance with previous data suggested that the audio-visual component removed the regular prime benefit. These findings suggest that the visual cues used in this study do not enhance rhythmic priming and could hinder the effect by potentially creating a dual-task situation. In addition, individual differences in sensory-motor and social scales of music reward influenced the effect of the visual cue. Implications for future audio-visual experiments aiming to enhance beat processing, and the importance of individual differences will be discussed.

Sleep deprivation affects gait control

Different levels of sleep restriction affect human performance in multiple aspects. However, it is unclear how sleep deprivation affects gait control. We applied a paced gait paradigm that included subliminal rhythm changes to analyze the effects of different sleep restriction levels (acute, chronic and control) on performance. Acute sleep deprivation (one night) group exhibited impaired performance in the sensorimotor synchronization gait protocol, such as a decrease in the Period Error between the footfalls and the auditory stimulus as well as missing more frequently the auditory cues. The group with chronic sleep restriction also underperformed when compared to the control group with a tendency to a late footfall with respect to the RAC sound. Our results suggest that partial or total sleep deprivation leads to a decrease in the performance in the sensorimotor control of gait. The superior performance of the chronic sleep group when compared to the acute group suggests that there is a compensatory mechanism that helps to improve motor performance.

An ecological approach to measuring synchronization abilities across the animal kingdom

In this perspective paper, we focus on the study of synchronization abilities across the animal kingdom. We propose an ecological approach to studying nonhuman animal synchronization that begins from observations about when, how and why an animal might synchronize spontaneously with natural environmental rhythms. We discuss what we consider to be the most important, but thus far largely understudied, temporal, physical, perceptual and motivational constraints that must be taken into account when designing experiments to test synchronization in nonhuman animals. First and foremost, different species are likely to be sensitive to and therefore capable of synchronizing at different timescales. We also argue that it is fruitful to consider the latent flexibility of animal synchronization. Finally, we discuss the importance of an animal's motivational state for showcasing synchronization abilities. We demonstrate that the likelihood that an animal can successfully synchronize with an environmental rhythm is context-dependent and suggest that the list of species capable of synchronization is likely to grow when tested with ecologically honest, species-tuned experiments.

This article is part of the theme issue ‘Synchrony and rhythm interaction: from the brain to behavioural ecology’.

Modality-specific attractor dynamics in dyadic entrainment

Rhythmic joint coordination is ubiquitous in daily-life human activities. In order to coordinate their actions towards shared goals, individuals need to co-regulate their timing and move together at the collective level of behavior. Remarkably, basic forms of coordinated behavior tend to emerge spontaneously as long as two individuals are exposed to each other's rhythmic movements. The present study investigated the dynamics of spontaneous dyadic entrainment, and more specifically how they depend on the sensory modalities mediating informational coupling. By means of a novel interactive paradigm, we showed that dyadic entrainment systematically takes place during a minimalistic rhythmic task despite explicit instructions to ignore the partner. Crucially, the interaction was organized by clear dynamics in a modality-dependent fashion. Our results showed highly consistent coordination patterns in visually-mediated entrainment, whereas we observed more chaotic and more variable profiles in the auditorily-mediated counterpart. The proposed experimental paradigm yields empirical evidence for the overwhelming tendency of dyads to behave as coupled rhythmic units. In the context of our experimental design, it showed that coordination dynamics differ according to availability and nature of perceptual information. Interventions aimed at rehabilitating, teaching or training sensorimotor functions can be ultimately informed and optimized by such fundamental knowledge.

The influence of auditory rhythms on the speed of inferred motion

The present research explored the influence of isochronous auditory rhythms on the timing of movement-related prediction in two experiments. In both experiments, participants observed a moving disc that was visible for a predetermined period before disappearing behind a small, medium, or large occluded area for the remainder of its movement. In Experiment 1, the disc was visible for 1 s. During this period, participants were exposed to either a fast or slow auditory rhythm, or they heard nothing. They were instructed to press a key to indicate when they believed the moving disc had reached a specified location on the other side of the occluded area. The procedure measured the (signed) error in participants' estimate of the time it would take for a moving object to contact a stationary one. The principal results of Experiment 1 were main effects of the rate of the auditory rhythm and of the size of the occlusion on participants' judgments. In Experiment 2, the period of visibility was varied with size of the occlusion area to keep the total movement time constant for all three levels of occlusion. The results replicated the main effect of rhythm found in Experiment 1 and showed a small, significant interaction, but indicated no main effect of occlusion size. Overall, the results indicate that exposure to fast isochronous auditory rhythms during an interval of inferred motion can influence the imagined rate of such motion and suggest a possible role of an internal rhythmicity in the maintenance of temporally accurate dynamic mental representations.

Spatial Connectivity and Temporal Dynamic Functional Network Connectivity of Musical Emotions Evoked by Dynamically Changing Tempo

Music tempo is closely connected to listeners' musical emotion and multifunctional neural activities. Music with increasing tempo evokes higher emotional responses and music with decreasing tempo enhances relaxation. However, the neural substrate of emotion evoked by dynamically changing tempo is still unclear. To investigate the spatial connectivity and temporal dynamic functional network connectivity (dFNC) of musical emotion evoked by dynamically changing tempo, we collected dynamic emotional ratings and conducted group independent component analysis (ICA), sliding time window correlations, and k-means clustering to assess the FNC of emotion evoked by music with decreasing tempo (180-65 bpm) and increasing tempo (60-180 bpm). Music with decreasing tempo (with more stable dynamic valences) evoked higher valence than increasing tempo both with stronger independent components (ICs) in the default mode network (DMN) and sensorimotor network (SMN). The dFNC analysis showed that with time-decreasing FNC across the whole brain, emotion evoked by decreasing music was associated with strong spatial connectivity within the DMN and SMN. Meanwhile, it was associated with strong FNC between the DMN-frontoparietal network (FPN) and DMN-cingulate-opercular network (CON). The paired t-test showed that music with a decreasing tempo evokes stronger activation of ICs within DMN and SMN than that with an increasing tempo, which indicated that faster music is more likely to enhance listeners' emotions with multifunctional brain activities even when the tempo is slowing down. With increasing FNC across the whole brain, music with an increasing tempo was associated with strong connectivity within FPN; time-decreasing connectivity was found within CON, SMN, VIS, and between CON and SMN, which explained its unstable valence during the dynamic valence rating. Overall, the FNC can help uncover the spatial and temporal neural substrates of musical emotions evoked by dynamically changing tempi.

The Effect of Rhythm Abilities on Metronome-Cued Walking with an Induced Temporal Gait Asymmetry in Neurotypical Adults

ABSRACT. Human gait is inherently rhythmical, thus walking to rhythmic auditory stimulation is a promising intervention to improve temporal gait asymmetry (TGA) following neurologic injury such as stroke. However, the degree of benefit may relate to an individual's underlying rhythmic ability. We conducted an initial investigation into the relationship between rhythm abilities and responsiveness of TGA when walking to a metronome. TGA was induced in neurotypical young adults with ankle and thigh cuff weights. Participants were grouped by strong or weak rhythm ability based on beat perception and production tests. TGA was induced using a unilateral load affixed to the non-dominant leg. Participants walked under three conditions: uncued baseline, metronome set to 100% of baseline cadence, and metronome set to 90% of baseline cadence. Repeated measures analysis using generalized estimating equations was conducted to determine how rhythm ability affected TGA response in each walking condition. Most participants improved TGA when walking to a metronome at either tempo compared to baseline; however, this improvement did not differ between strong and weak rhythm ability groups. Those who scored worse on the rhythm perception test also were poorer at synchronizing their steps to the beat. The induced TGA is smaller than what is commonly experienced after stroke. A larger induced TGA may be necessary to reveal subtle differences in responsiveness to rhythmical auditory stimulation between those with strong and weak rhythm abilities.

Expectancy-based rhythmic entrainment as continuous Bayesian inference

When presented with complex rhythmic auditory stimuli, humans are able to track underlying temporal structure (e.g., a "beat"), both covertly and with their movements. This capacity goes far beyond that of a simple entrained oscillator, drawing on contextual and enculturated timing expectations and adjusting rapidly to perturbations in event timing, phase, and tempo. Previous modeling work has described how entrainment to rhythms may be shaped by event timing expectations, but sheds little light on any underlying computational principles that could unify the phenomenon of expectation-based entrainment with other brain processes. Inspired by the predictive processing framework, we propose that the problem of rhythm tracking is naturally characterized as a problem of continuously estimating an underlying phase and tempo based on precise event times and their correspondence to timing expectations. We present two inference problems formalizing this insight: PIPPET (Phase Inference from Point Process Event Timing) and PATIPPET (Phase and Tempo Inference). Variational solutions to these inference problems resemble previous "Dynamic Attending" models of perceptual entrainment, but introduce new terms representing the dynamics of uncertainty and the influence of expectations in the absence of sensory events. These terms allow us to model multiple characteristics of covert and motor human rhythm tracking not addressed by other models, including sensitivity of error corrections to inter-event interval and perceived tempo changes induced by event omissions. We show that positing these novel influences in human entrainment yields a range of testable behavioral predictions. Guided by recent neurophysiological observations, we attempt to align the phase inference framework with a specific brain implementation. We also explore the potential of this normative framework to guide the interpretation of experimental data and serve as building blocks for even richer predictive processing and active inference models of timing.

Modality-specific frequency band activity during neural entrainment to auditory and visual rhythms

Rhythm perception depends on the ability to predict the onset of rhythmic events. Previous studies indicate beta band modulation is involved in predicting the onset of auditory rhythmic events (Fujioka et al., 2009, 2012; Snyder & Large, 2005). We sought to determine if similar processes are recruited for prediction of visual rhythms by investigating whether beta band activity plays a role in a modality-dependent manner for rhythm perception. We looked at electroencephalography time-frequency neural correlates of prediction using an omission paradigm with auditory and visual rhythms. By using omissions, we can separate out predictive timing activity from stimulus-driven activity. We hypothesized that there would be modality-independent markers of rhythm prediction in induced beta band oscillatory activity, and our results support this hypothesis. We find induced and evoked predictive timing in both auditory and visual modalities. Additionally, we performed an exploratory-independent components-based spatial clustering analysis, and describe all resulting clusters. This analysis reveals that there may be overlapping networks of predictive beta activity based on common activation in the parietal and right frontal regions, auditory-specific predictive beta in bilateral sensorimotor regions, and visually specific predictive beta in midline central, and bilateral temporal/parietal regions. This analysis also shows evoked predictive beta activity in the left sensorimotor region specific to auditory rhythms and implicates modality-dependent networks for auditory and visual rhythm perception.

Keeping in time with social and non-social stimuli: Synchronisation with auditory, visual, and audio-visual cues

Everyday social interactions require us to closely monitor, predict, and synchronise our movements with those of an interacting partner. Experimental studies of social synchrony typically examine the social-cognitive outcomes associated with synchrony, such as affiliation. On the other hand, research on the sensorimotor aspects of synchronisation generally uses non-social stimuli (e.g. a moving dot). To date, the differences in sensorimotor aspects of synchronisation to social compared to non-social stimuli remain largely unknown. The present study aims to address this gap using a verbal response paradigm where participants were asked to synchronise a 'ba' response in time with social and non-social stimuli, which were presented auditorily, visually, or audio-visually combined. For social stimuli a video/audio recording of an actor performing the same verbal 'ba' response was presented, whereas for non-social stimuli a moving dot, an auditory metronome or both combined were presented. The impact of autistic traits on participants' synchronisation performance was examined using the Autism Spectrum Quotient (AQ). Our results revealed more accurate synchronisation for social compared to non-social stimuli, suggesting that greater familiarity with and motivation in attending to social stimuli may enhance our ability to better predict and synchronise with them. Individuals with fewer autistic traits demonstrated greater social learning, as indexed through an improvement in synchronisation performance to social vs non-social stimuli across the experiment.

Spontaneous Interpersonal Synchronization of Gait: A Systematic Review

Objective

To systematically review the existing evidence of spontaneous synchronization in human gait.

Data Sources

EBSCO, PubMed, Google Scholar, and PsycINFO were searched from inception to July 2020 using all possible combinations of (1) “spontaneous interpersonal synchronization” or “spontaneous interpersonal coordination” or “unintentional interpersonal synchronization” or “unintentional interpersonal coordination” and (2) “human movement” or “movement” or “walking” or “ambulation” or “gait.”

Study Selection

Studies had to focus on spontaneous synchronization in human gait, be published in a peer-reviewed journal, present original data (no review articles were included), and be written in English. The search yielded 137 results, and the inclusion criteria were met by 16 studies.

Data Extraction

Participant demographics, study purpose, setup, procedure, biomechanical measurement, coordination analytical technique, and findings were extracted. Our synthesis focused on the context in which this phenomenon has been studied, the role of sensory information in the emergence of spontaneous interpersonal synchronization in human gait, and the metrics used to quantify this behavior.

Data Synthesis

The included 16 articles ranged from 2007-2019 and used healthy, primarily young subjects to investigate the role of spontaneous interpersonal synchronization on gait behavior, with the majority using a side-by-side walking/running paradigm. All articles reported data supporting spontaneous interpersonal synchronization, with the strength of the synchronization depending on the sensory information available to the participants.

Conclusions

Walking alongside an intact locomotor system may provide an effective and biologically variable attractor signal for rehabilitation of gait behavior. Future research should focus on the utility of spontaneous interpersonal synchronization in clinical populations as a noninvasive method to enhance gait rehabilitation.

Disentangling the effects of modality, interval length and task difficulty on the accuracy and precision of older adults in a rhythmic reproduction task

Studies on the functional quality of the internal clock that governs the temporal processing of older adults have demonstrated mixed results as to whether they perceive and produce time slower, faster, or equally well as younger adults. These mixed results are due to a multitude of methodologies applied to study temporal processing: many tasks demand different levels of cognitive ability. To investigate the temporal accuracy and precision of older adults, in Experiment 1, we explored the age-related differences in rhythmic continuation task taking into consideration the effects of attentional resources required by the stimulus (auditory vs. visual; length of intervals). In Experiment 2, we added a dual task to explore the effect of attentional resources required by the task. Our findings indicate that (1) even in an inherently automatic rhythmic task, where older and younger adult’s general accuracy is comparable, accuracy but not precision is altered by the stimulus properties and (2) an increase in task load can magnify age-related differences in both accuracy and precision.

Neural Encoding and Representation of Time for Sensorimotor Control and Learning

The ability to perceive and produce movements in the real world with precise timing is critical for survival in animals, including humans. However, research on sensorimotor timing has rarely considered the tight interrelation between perception, action, and cognition. In this review, we present new evidence from behavioral, computational, and neural studies in humans and nonhuman primates, suggesting a pivotal link between sensorimotor control and temporal processing, as well as describing new theoretical frameworks regarding timing in perception and action. We first discuss the link between movement coordination and interval-based timing by addressing how motor training develops accurate spatiotemporal patterns in behavior and influences the perception of temporal intervals. We then discuss how motor expertise results from establishing task-relevant neural manifolds in sensorimotor cortical areas and how the geometry and dynamics of these manifolds help reduce timing variability. We also highlight how neural dynamics in sensorimotor areas are involved in beat-based timing. These lines of research aim to extend our understanding of how timing arises from and contributes to perceptual-motor behaviors in complex environments to seamlessly interact with other cognitive processes.

Audiovisual Modulation in Music Perception for Musicians and Non-musicians

In audiovisual music perception, visual information from a musical instrument being played is available prior to the onset of the corresponding musical sound and consequently allows a perceiver to form a prediction about the upcoming audio music. This prediction in audiovisual music perception, compared to auditory music perception, leads to lower N1 and P2 amplitudes and latencies. Although previous research suggests that audiovisual experience, such as previous musical experience may enhance this prediction, a remaining question is to what extent musical experience modifies N1 and P2 amplitudes and latencies. Furthermore, corresponding event-related phase modulations quantified as inter-trial phase coherence (ITPC) have not previously been reported for audiovisual music perception. In the current study, audio video recordings of a keyboard key being played were presented to musicians and non-musicians in audio only (AO), video only (VO), and audiovisual (AV) conditions. With predictive movements from playing the keyboard isolated from AV music perception (AV-VO), the current findings demonstrated that, compared to the AO condition, both groups had a similar decrease in N1 amplitude and latency, and P2 amplitude, along with correspondingly lower ITPC values in the delta, theta, and alpha frequency bands. However, while musicians showed lower ITPC values in the beta-band in AV-VO compared to the AO, non-musicians did not show this pattern. Findings indicate that AV perception may be broadly correlated with auditory perception, and differences between musicians and non-musicians further indicate musical experience to be a specific factor influencing AV perception. Predicting an upcoming sound in AV music perception may involve visual predictory processes, as well as beta-band oscillations, which may be influenced by years of musical training. This study highlights possible interconnectivity in AV perception as well as potential modulation with experience.

Lowered Rhythm Tapping Ability in Patients With Constructional Apraxia After Stroke

Rhythm tapping tasks are often used to explore temporal reproduction abilities. Many studies utilizing rhythm tapping tasks are conducted to evaluate temporal processing abilities with neurological impairments and neurodegenerative disorders. Among sensorimotor and cognitive disorders, rhythm processing abilities in constructional apraxia, a deficit in achieving visuospatial constructional activities, has not been evaluated. This study aimed to examine the rhythm tapping ability of patients with constructional apraxia after a stroke. Twenty-four patients were divided into two groups: with and without constructional apraxia. There were 11 participants in the constructional apraxia group and 13 in the without constructional apraxia group. The synchronization-continuation paradigm was employed in which a person performs a synchronized tapping activity to a metronome beat and continues tapping after the beat has stopped. For statistical analysis, a three-way mixed analysis of variance (2 × 2 × 3) was conducted. The factors were groups (with and without constructional apraxia), tapping tasks (synchronization and continuation), and inter-stimulus intervals (600, 750, and 1000 ms). A significant effect of group factor was found (F[1,132] = 16.62; p < 0.001). Patients in the without constructional apraxia group were able to more accurately reproduce intervals than those in the constructional apraxia group. Moreover, a significant effect of tapping tasks was found (F[1,132] = 8.22; p < 0.01). Intervals were reproduced more accurately for synchronization tasks than continuation tasks. There was no significant inter-stimulus interval effect. Overall, these results suggest that there might be a relation between temporal and spatial reproductions in a wide spectrum of processing levels, from sensory perception to cognitive function.

Natural rhythms of periodic temporal attention

That attention is a fundamentally rhythmic process has recently received abundant empirical evidence. The essence of temporal attention, however, is to flexibly focus in time. Whether this function is constrained by an underlying rhythmic neural mechanism is unknown. In six interrelated experiments, we behaviourally quantify the sampling capacities of periodic temporal attention during auditory or visual perception. We reveal the presence of limited attentional capacities, with an optimal sampling rate of ~1.4 Hz in audition and ~0.7 Hz in vision. Investigating the motor contribution to temporal attention, we show that it scales with motor rhythmic precision, maximal at ~1.7 Hz. Critically, motor modulation is beneficial to auditory but detrimental to visual temporal attention. These results are captured by a computational model of coupled oscillators, that reveals the underlying structural constraints governing the temporal alignment between motor and attention fluctuations.

Staying Together: A Bidirectional Delay-Coupled Approach to Joint Action

To understand how individuals adapt to and anticipate each other in joint tasks, we employ a bidirectional delay-coupled dynamical system that allows for mutual adaptation and anticipation. In delay-coupled systems, anticipation is achieved when one system compares its own time-delayed behavior, which implicitly includes past information about the other system's behavior, with the other system's instantaneous behavior. Applied to joint music performance, the model allows each system to adapt its behavior to the dynamics of the other. Model predictions of asynchrony between two simultaneously produced musical voices were compared with duet pianists' behavior; each partner performed one voice while auditory feedback perturbations occurred at unpredictable times during live performance. As the model predicted, when auditory feedback from one musical voice was removed, the asynchrony changed: The pianist's voice that was removed anticipated (preceded) the actions of their partner. When the auditory feedback returned and both musicians could hear each other, they rapidly returned to baseline levels of asynchrony. To understand how the pianists anticipated each other, their performances were fitted by the model to examine change in model parameters (coupling strength, time-delay). When auditory feedback for one or both voices was removed, the fits showed the expected decrease in coupling strength and time-delay between the systems. When feedback about the voice(s) returned, the coupling strength and time-delay returned to baseline. These findings support the idea that when people perform actions together, they do so as a coupled bidirectional anticipatory system.

Development of sensorimotor synchronization abilities: Motor and cognitive components

The aim of the present study was to examine both the development of sensorimotor synchronization in children in the age range from 5 to 8 years and the involvement of motor and cognitive capacities. Children performed a spontaneous motor tempo task and a synchronization-continuation task using an external auditory stimulus presented at three different inter-stimulus intervals: 500, 700, and 900 ms. Their motor and cognitive abilities (short-term memory, working memory, and attention) were also assessed with various neuropsychological tests. The results showed some developmental changes in synchronization capacities, with the regularity of tapping and the ability to slow down the tapping rate improving with age. The age-related differences in tapping were nevertheless greater in the continuation phase than in the synchronization phase. In addition, the development of motor capacities explained the age-related changes in performance for the synchronization phase and the continuation phase, although working memory capacities were also involved in the interindividual differences in performance in the continuation phase. The continuation phase is thus more cognitively demanding than the synchronization phase. Consequently, the improvement in sensorimotor synchronization during childhood is related to motor development in the case of synchronization but also to cognitive development with regard to the reproduction and maintenance of the rhythm in memory.

Trajectory formation during sensorimotor synchronization and syncopation to auditory and visual metronomes

Previous work on sensorimotor synchronization has investigated the dynamics of finger tapping and how individual movement trajectories contribute to timing accuracy via asymmetry in movement velocities. The present study investigated sensorimotor synchronization (in-phase) and syncopation (anti-phase) to both an auditory metronome and a visual flashing light at multiple frequencies to understand how individual movement phases contribute to the variability of timekeeping and error correction in different sensory modalities and with different task constraints. Results demonstrate that the proportional time spent in both the upward phase of movement and the holding phase of movement (time spent on the surface of the table) remain relatively invariant across both stimulus modalities and across tapping styles (syncopation and synchronization), but changes with interval duration, increasing as interval duration increases. The time spent in the downward phase of movement did significantly differ across stimulus modality and tapping style, increasing during both visuomotor timing and syncopation, accompanied by a significant decrease in flexion velocity during syncopation. Extension velocity and flexion time were found to be the main contributors to differences between visual and auditory timing, while flexion velocity and flexion time were found to be the main contributors to differences between synchronization and syncopation. No correlations were found between asynchrony and the upward, downward, or holding phases of movement, suggesting the existence of multiple error correction strategies.