Moving Stimuli Facilitate Synchronization But Not Temporal Perception

Embodying Time in the Brain: A Multi-Dimensional Neuroimaging Meta-Analysis of 95 Duration Processing Studies

Time is an omnipresent aspect of almost everything we experience internally or in the external world. The experience of time occurs through such an extensive set of contextual factors that, after decades of research, a unified understanding of its neural substrates is still elusive. In this study, following the recent best-practice guidelines, we conducted a coordinate-based meta-analysis of 95 carefully-selected neuroimaging papers of duration processing. We categorized the included papers into 14 classes of temporal features according to six categorical dimensions. Then, using the activation likelihood estimation (ALE) technique we investigated the convergent activation patterns of each class with a cluster-level family-wise error correction at p < 0.05. The regions most consistently activated across the various timing contexts were the pre-SMA and bilateral insula, consistent with an embodied theory of timing in which abstract representations of duration are rooted in sensorimotor and interoceptive experience, respectively. Moreover, class-specific patterns of activation could be roughly divided according to whether participants were timing auditory sequential stimuli, which additionally activated the dorsal striatum and SMA-proper, or visual single interval stimuli, which additionally activated the right middle frontal and inferior parietal cortices. We conclude that temporal cognition is so entangled with our everyday experience that timing stereotypically common combinations of stimulus characteristics reactivates the sensorimotor systems with which they were first experienced.

Visual movement impairs duration discrimination at short intervals

The classic advantage of audition over vision in time processing has been recently challenged by studies using continuously moving visual stimuli such as bouncing balls. Bouncing balls drive beat-based synchronisation better than static visual stimuli (flashes) and as efficiently as auditory ones (beeps). It is yet unknown how bouncing balls modulate performance in duration perception. Our previous study addressing this was inconclusive: there were no differences among bouncing balls, flashes, and beeps, but this could have been due to the fact that intervals were too long to allow sensitivity to modality (visual vs auditory). In this study, we conducted a first experiment to determine whether shorter intervals elicit cross-stimulus differences. We found that short (mean 157 ms) but not medium (326 ms) intervals made duration perception worse for bouncing balls compared with flashes and beeps. In a second experiment, we investigated whether the lower efficiency of bouncing balls was due to experimental confounds, lack of realism, or movement. We ruled out the experimental confounds and found support for the hypothesis that visual movement-be it continuous or discontinuous-impairs duration perception at short interval lengths. Therefore, unlike beat-based synchronisation, duration perception does not benefit from continuous visual movement, which may even have a detrimental effect at short intervals.

The same phase creates a unique visual rhythm unifying moving elements in time

Attention can be selectively tuned to particular features at different spatial locations or objects. The deployment of attention can be guided by properties, such as color, orientation, and so forth, as guiding features. What might be such guiding features for visual stimuli under dynamic rhythmic conditions? We asked specifically what might be the parameters that attract attention when perceiving a visual rhythm. We used a visual search paradigm, in which a dynamic search display consisted of vertically "bouncing balls" with regular rhythms. The search target was defined by a unique visual rhythm (i.e., with either a shorter or longer period) among rhythmic distractors sharing an identical period. We modulated amplitudes and phases of the distractor balls systematically. The results showed a crucial factor of the phase, not the amplitude. If the phase is violated, the target suddenly "pops out" as an "oddball," showing an efficient parallel search. The findings indicate in general the essential role of the phase in conjunction with amplitude and period for visual rhythm perception. Furthermore, a higher saliency of moving objects with a higher frequency component has also been disclosed.

Visual over auditory superiority in sensorimotor timing under optimized condition

Auditory over visual advantage in temporal processing is generally appreciated, such as the well-established auditory superiority in sensorimotor timing. To test for a possible visual superiority in temporal processing, here, we present a data set composed of a large 60 subjects sample and a data set including eight smaller samples of approximately 15 subjects, showing that synchronization to a temporally regular sequence was more stable for a visual bouncing ball (VB) than for auditory tones (ATs). The results demonstrate that vision can be superior over audition in sensorimotor timing under optimized conditions, challenging the generally believed auditory superiority in temporal processing. In contrast to the auditory-specific biological substrates of timing in sensorimotor interaction, the present finding points to tight visual-motor cortical coupling in sensorimotor timing.

Differential motor system entrainment to auditory and visual rhythms

Perception of, and synchronization to, auditory rhythms is known to be more accurate than with flashing visual rhythms. The motor system is known to play a role in the processing of timing information for auditory rhythm perception, but it is unclear if the motor system plays the same role for visual rhythm perception. One demonstrated component of auditory rhythm perception is neural entrainment at the frequency of the auditory rhythm. In this study, we use EEG to measure the entrainment of both auditory and visual rhythms from the motor cortex while subjects either tapped in synchrony with or passively attended to the presented rhythms. To isolate activity from motor cortex, we used independent component analysis to first separate out neural sources, then selected components using a combination of component topography, dipole location, mu activation, and beta modulation. This process took advantage of the fact that tapping activity results in reduced mu power, and characteristic beta modulation, which helped select motor components. Our findings suggest neural entrainment in motor components was stronger for visual rhythms than auditory rhythms and strongest during the tapping conditions for both modalities. We also find mu power increased in response to both auditory and visual rhythms. These findings indicate that the generally greater rhythm perception capabilities of the auditory system over the visual system may not depend entirely on neural entrainment in the motor system, but rather how the motor system is able to use the timing information made available to it. NEW & NOTEWORTHY We investigated neural entrainment in the motor system for both auditory and visual isochronous rhythms using electroencephalogram. Counter to expectations, our findings suggest stronger entrainment for visual rhythms than for auditory rhythms. Motor system activity was isolated with a novel procedure using independent component analysis as a means of blind source separation, along with known markers of mu activity from the motor system to identify motor components.

When Visual Cues Do Not Help the Beat: Evidence for a Detrimental Effect of Moving Point-Light Figures on Rhythmic Priming

Rhythm perception involves strong auditory-motor connections that can be enhanced with movement. However, it is unclear whether just seeing someone moving to a rhythm can enhance auditory-motor coupling, resulting in stronger entrainment. Rhythmic priming studies show that presenting regular rhythms before naturally spoken sentences can enhance grammaticality judgments compared to irregular rhythms or other baseline conditions. The current study investigated whether introducing a point-light figure moving in time with regular rhythms could enhance the rhythmic priming effect. Three experiments revealed that the addition of a visual cue did not benefit rhythmic priming in comparison to auditory conditions with a static image. In Experiment 1 (27 7–8-year-old children), grammaticality judgments were poorer after audio-visual regular rhythms (with a bouncing point-light figure) compared to auditory-only regular rhythms. In Experiments 2 (31 adults) and 3 (31 different adults), there was no difference in grammaticality judgments after audio-visual regular rhythms compared to auditory-only irregular rhythms for either a bouncing point-light figure (Experiment 2) or a swaying point-light figure (Experiment 3). Comparison of the observed performance with previous data suggested that the audio-visual component removed the regular prime benefit. These findings suggest that the visual cues used in this study do not enhance rhythmic priming and could hinder the effect by potentially creating a dual-task situation. In addition, individual differences in sensory-motor and social scales of music reward influenced the effect of the visual cue. Implications for future audio-visual experiments aiming to enhance beat processing, and the importance of individual differences will be discussed.

Advantage of audition over vision in a perceptual timing task but not in a sensorimotor timing task

Timing is essential for various behaviors and relative to vision, audition is considered to be specialized for temporal processing. The present study conducted a sensorimotor timing task that required tapping in synchrony with a temporally regular sequence and a perceptual timing task that required detecting a timing deviation among a temporally regular sequence. The sequence was composed of auditory tones, visual flashes, or a visual bouncing ball. In the sensorimotor task, sensorimotor timing performance (synchronization stability) of the bouncing ball was much greater than that of flashes and was comparable to that of tones. In the perceptual task, where perceptual timing performance of the bouncing ball was greater than that of flashes, it was poorer than that of tones. These results suggest the facilitation of both perceptual and sensorimotor processing of temporal information by the bouncing ball. Given such facilitation of temporal processing, however, audition is still superior over vision in perceptual detection of timing.

Sensorimotor Synchronization With Auditory and Visual Modalities: Behavioral and Neural Differences

It has long been known that the auditory system is better suited to guide temporally precise behaviors like sensorimotor synchronization (SMS) than the visual system. Although this phenomenon has been studied for many years, the underlying neural and computational mechanisms remain unclear. Growing consensus suggests the existence of multiple, interacting, context-dependent systems, and that reduced precision in visuo-motor timing might be due to the way experimental tasks have been conceived. Indeed, the appropriateness of the stimulus for a given task greatly influences timing performance. In this review, we examine timing differences for sensorimotor synchronization and error correction with auditory and visual sequences, to inspect the underlying neural mechanisms that contribute to modality differences in timing. The disparity between auditory and visual timing likely relates to differences in the processing specialization between auditory and visual modalities (temporal vs. spatial). We propose this difference could offer potential explanation for the differing temporal abilities between modalities. We also offer suggestions as to how these sensory systems interface with motor and timing systems.