Advantage of audition over vision in a perceptual timing task but not in a sensorimotor timing task

Vigilance to Spatialized Auditory Displays: Initial Assessment of Performance and Workload

OBJECTIVE

The present study was designed to evaluate human performance and workload associated with an auditory vigilance task that required spatial discrimination of auditory stimuli.

BACKGROUND

Spatial auditory displays have been increasingly developed and implemented into settings that require vigilance toward auditory spatial discrimination and localization (e.g., collision avoidance warnings). Research has yet to determine whether a vigilance decrement could impede performance in such applications.

METHOD

Participants completed a 40-minute auditory vigilance task in either a spatial discrimination condition or a temporal discrimination condition. In the spatial discrimination condition, participants differentiated sounds based on differences in spatial location. In the temporal discrimination condition, participants differentiated sounds based on differences in stimulus duration.

RESULTS

Correct detections and false alarms declined during the vigilance task, and each did so at a similar rate in both conditions. The overall level of correct detections did not differ significantly between conditions, but false alarms occurred more frequently within the spatial discrimination condition than in the temporal discrimination condition. NASA-TLX ratings and pupil diameter measurements indicated no differences in workload.

CONCLUSION

Results indicated that tasks requiring auditory spatial discrimination can induce a vigilance decrement; and they may result in inferior vigilance performance, compared to tasks requiring discrimination of auditory duration.

APPLICATION

Vigilance decrements may impede performance and safety in settings that depend on sustained attention to spatial auditory displays. Display designers should also be aware that auditory displays that require users to discriminate differences in spatial location may result in poorer discrimination performance than non-spatial displays.

Embodying Time in the Brain: A Multi-Dimensional Neuroimaging Meta-Analysis of 95 Duration Processing Studies

Time is an omnipresent aspect of almost everything we experience internally or in the external world. The experience of time occurs through such an extensive set of contextual factors that, after decades of research, a unified understanding of its neural substrates is still elusive. In this study, following the recent best-practice guidelines, we conducted a coordinate-based meta-analysis of 95 carefully-selected neuroimaging papers of duration processing. We categorized the included papers into 14 classes of temporal features according to six categorical dimensions. Then, using the activation likelihood estimation (ALE) technique we investigated the convergent activation patterns of each class with a cluster-level family-wise error correction at p < 0.05. The regions most consistently activated across the various timing contexts were the pre-SMA and bilateral insula, consistent with an embodied theory of timing in which abstract representations of duration are rooted in sensorimotor and interoceptive experience, respectively. Moreover, class-specific patterns of activation could be roughly divided according to whether participants were timing auditory sequential stimuli, which additionally activated the dorsal striatum and SMA-proper, or visual single interval stimuli, which additionally activated the right middle frontal and inferior parietal cortices. We conclude that temporal cognition is so entangled with our everyday experience that timing stereotypically common combinations of stimulus characteristics reactivates the sensorimotor systems with which they were first experienced.

Visual movement impairs duration discrimination at short intervals

The classic advantage of audition over vision in time processing has been recently challenged by studies using continuously moving visual stimuli such as bouncing balls. Bouncing balls drive beat-based synchronisation better than static visual stimuli (flashes) and as efficiently as auditory ones (beeps). It is yet unknown how bouncing balls modulate performance in duration perception. Our previous study addressing this was inconclusive: there were no differences among bouncing balls, flashes, and beeps, but this could have been due to the fact that intervals were too long to allow sensitivity to modality (visual vs auditory). In this study, we conducted a first experiment to determine whether shorter intervals elicit cross-stimulus differences. We found that short (mean 157 ms) but not medium (326 ms) intervals made duration perception worse for bouncing balls compared with flashes and beeps. In a second experiment, we investigated whether the lower efficiency of bouncing balls was due to experimental confounds, lack of realism, or movement. We ruled out the experimental confounds and found support for the hypothesis that visual movement-be it continuous or discontinuous-impairs duration perception at short interval lengths. Therefore, unlike beat-based synchronisation, duration perception does not benefit from continuous visual movement, which may even have a detrimental effect at short intervals.

Higher synchronization stability with piano experience: relationship with finger and presentation modality

BACKGROUND

Synchronous finger tapping to external sensory stimuli is more stable for audiovisual combined stimuli than sole auditory or visual stimuli. In addition, piano players are superior in synchronous tapping and manipulating the ring and little fingers as compared to inexperienced individuals. However, it is currently unknown whether the ability to synchronize to external sensory stimuli with the ring finger is at the level of the index finger in piano players. The aim of this study was to compare the effect of piano experience on synchronization stability between the index and ring fingers using auditory, visual, and audiovisual combined stimuli.

METHODS

Thirteen piano players and thirteen novices participated in this study. They were instructed to tap with their index or ring finger synchronously to auditory, visual, and audiovisual combined stimuli. The stimuli were presented from an electronic metronome at 1 Hz, and the tapping was performed 30 times in each condition. We analyzed standard deviation of intervals between the stimulus onset and the tap onset as synchronization stability.

RESULTS

Synchronization stability for visual stimuli was lower during ring than index finger tapping in novices; however, this decline was absent in piano players. Also, piano players showed the higher synchronization stability for audiovisual combined stimuli than sole visual and auditory stimuli when tapping with the index finger. On the other hand, in novices, synchronization stability was higher for audiovisual combined stimuli than only visual stimuli.

CONCLUSIONS

These findings suggest that improvements of both sensorimotor processing and finger motor control by piano practice would contribute to superior synchronization stability.

Neural tracking of visual periodic motion

Periodicity is a fundamental property of biological systems, including human movement systems. Periodic movements support displacements of the body in the environment as well as interactions and communication between individuals. Here, we use electroencephalography (EEG) to investigate the neural tracking of visual periodic motion, and more specifically, the relevance of spatiotemporal information contained at and between their turning points. We compared EEG responses to visual sinusoidal oscillations versus nonlinear Rayleigh oscillations, which are both typical of human movements. These oscillations contain the same spatiotemporal information at their turning points but differ between turning points, with Rayleigh oscillations having an earlier peak velocity, shown to increase an individual's capacity to produce accurately synchronized movements. EEG analyses highlighted the relevance of spatiotemporal information between the turning points by showing that the brain precisely tracks subtle differences in velocity profiles, as indicated by earlier EEG responses for Rayleigh oscillations. The results suggest that the brain is particularly responsive to velocity peaks in visual periodic motion, supporting their role in conveying behaviorally relevant timing information at a neurophysiological level. The results also suggest key functions of neural oscillations in the Alpha and Beta frequency bands, particularly in the right hemisphere. Together, these findings provide insights into the neural mechanisms underpinning the processing of visual periodic motion and the critical role of velocity peaks in enabling proficient visuomotor synchronization.

Visual over auditory superiority in sensorimotor timing under optimized condition

Auditory over visual advantage in temporal processing is generally appreciated, such as the well-established auditory superiority in sensorimotor timing. To test for a possible visual superiority in temporal processing, here, we present a data set composed of a large 60 subjects sample and a data set including eight smaller samples of approximately 15 subjects, showing that synchronization to a temporally regular sequence was more stable for a visual bouncing ball (VB) than for auditory tones (ATs). The results demonstrate that vision can be superior over audition in sensorimotor timing under optimized conditions, challenging the generally believed auditory superiority in temporal processing. In contrast to the auditory-specific biological substrates of timing in sensorimotor interaction, the present finding points to tight visual-motor cortical coupling in sensorimotor timing.

Differential motor system entrainment to auditory and visual rhythms

Perception of, and synchronization to, auditory rhythms is known to be more accurate than with flashing visual rhythms. The motor system is known to play a role in the processing of timing information for auditory rhythm perception, but it is unclear if the motor system plays the same role for visual rhythm perception. One demonstrated component of auditory rhythm perception is neural entrainment at the frequency of the auditory rhythm. In this study, we use EEG to measure the entrainment of both auditory and visual rhythms from the motor cortex while subjects either tapped in synchrony with or passively attended to the presented rhythms. To isolate activity from motor cortex, we used independent component analysis to first separate out neural sources, then selected components using a combination of component topography, dipole location, mu activation, and beta modulation. This process took advantage of the fact that tapping activity results in reduced mu power, and characteristic beta modulation, which helped select motor components. Our findings suggest neural entrainment in motor components was stronger for visual rhythms than auditory rhythms and strongest during the tapping conditions for both modalities. We also find mu power increased in response to both auditory and visual rhythms. These findings indicate that the generally greater rhythm perception capabilities of the auditory system over the visual system may not depend entirely on neural entrainment in the motor system, but rather how the motor system is able to use the timing information made available to it. NEW & NOTEWORTHY We investigated neural entrainment in the motor system for both auditory and visual isochronous rhythms using electroencephalogram. Counter to expectations, our findings suggest stronger entrainment for visual rhythms than for auditory rhythms. Motor system activity was isolated with a novel procedure using independent component analysis as a means of blind source separation, along with known markers of mu activity from the motor system to identify motor components.

When Visual Cues Do Not Help the Beat: Evidence for a Detrimental Effect of Moving Point-Light Figures on Rhythmic Priming

Rhythm perception involves strong auditory-motor connections that can be enhanced with movement. However, it is unclear whether just seeing someone moving to a rhythm can enhance auditory-motor coupling, resulting in stronger entrainment. Rhythmic priming studies show that presenting regular rhythms before naturally spoken sentences can enhance grammaticality judgments compared to irregular rhythms or other baseline conditions. The current study investigated whether introducing a point-light figure moving in time with regular rhythms could enhance the rhythmic priming effect. Three experiments revealed that the addition of a visual cue did not benefit rhythmic priming in comparison to auditory conditions with a static image. In Experiment 1 (27 7–8-year-old children), grammaticality judgments were poorer after audio-visual regular rhythms (with a bouncing point-light figure) compared to auditory-only regular rhythms. In Experiments 2 (31 adults) and 3 (31 different adults), there was no difference in grammaticality judgments after audio-visual regular rhythms compared to auditory-only irregular rhythms for either a bouncing point-light figure (Experiment 2) or a swaying point-light figure (Experiment 3). Comparison of the observed performance with previous data suggested that the audio-visual component removed the regular prime benefit. These findings suggest that the visual cues used in this study do not enhance rhythmic priming and could hinder the effect by potentially creating a dual-task situation. In addition, individual differences in sensory-motor and social scales of music reward influenced the effect of the visual cue. Implications for future audio-visual experiments aiming to enhance beat processing, and the importance of individual differences will be discussed.

Sustained visual attention improves visuomotor timing

Relative to audition, vision is considered much less trustworthy in sensorimotor timing such as synchronizing finger movements with a temporally regular sequence. Visuomotor timing requires maintaining attention over time, whereas the sustained visual attention may not be well held in conventional visuomotor timing task settings where flashing visual stimuli consisted of a briefly presented flash and a long blank period. In the present study, the potential attentional lapses in time due to the disappearance of the flash were carefully controlled in Experiment 1 by changing the color of the flash instead of its disappearance, or in Experiment 2 by adding an additional continuously presented fixation point serving as an external attentional cue when the flash disappeared. Improvement of visuomotor timing performance was found in both experiments. The finding suggests a role of enhanced sustained visual attention in improving visuomotor timing, by which vision could also be a trustworthy modality for processing temporal information in sensorimotor interactions.