Cortical auditory distance representation based on direct-to-reverberant energy ratio

Comparing auditory distance perception in real and virtual environments and the role of the loudness cue: A study based on event-related potentials

The perception of the distance to a sound source is relevant in many everyday situations, not only in real spaces, but also in virtual reality (VR) environments. Where real rooms often reach their limits, VR offers far-reaching possibilities to simulate a wide range of acoustic scenarios. However, in virtual room acoustics a plausible reproduction of distance-related cues can be challenging. In the present study, we compared the detection of changes of the distance to a sound source and its neurocognitive correlates in a real and a virtual reverberant environment, using an active auditory oddball paradigm and EEG measures. The main goal was to test whether the experiments in the virtual and real environments produced equivalent behavioral and EEG results. Three loudspeakers were placed at ego-centric distances of 2 m (near), 4 m (center), and 8 m (far) in front of the participants (N = 20), each 66 cm below their ear level. Sequences of 500 ms noise stimuli were presented either from the center position (standards, 80 % of trials) or from the near or far position (targets, 10 % each). The participants had to indicate a target position via a joystick response ("near" or "far"). Sounds were emitted either by real loudspeakers in the real environment or rendered and played back for the corresponding positions via headphones in the virtual environment. In addition, within both environments, loudness of the auditory stimuli was either unaltered (natural loudness) or the loudness cue was manipulated, so that all three loudspeakers were perceived equally loud at the listener's position (matched loudness). The EEG analysis focused on the mismatch negativity (MMN), P3a, and P3b as correlates of deviance detection, attentional orientation, and context-updating/stimulus evaluation, respectively. Overall, behavioral data showed that detection of the target positions was reduced within the virtual environment, and especially when loudness was matched. Except for slight latency shifts in the virtual environment, EEG analysis indicated comparable patterns within both environments and independent of loudness settings. Thus, while the neurocognitive processing of changes in distance appears to be similar in virtual and real spaces, a proper representation of loudness appears to be crucial to achieve a good task performance in virtual acoustic environments.

Integration of somatosensory and motor-related information in the auditory system

An ability to integrate information provided by different sensory modalities is a fundamental feature of neurons in many brain areas. Because visual and auditory inputs often originate from the same external object, which may be located some distance away from the observer, the synthesis of these cues can improve localization accuracy and speed up behavioral responses. By contrast, multisensory interactions occurring close to the body typically involve a combination of tactile stimuli with other sensory modalities. Moreover, most activities involving active touch generate sound, indicating that stimuli in these modalities are frequently experienced together. In this review, we examine the basis for determining sound-source distance and the contribution of auditory inputs to the neural encoding of space around the body. We then consider the perceptual consequences of combining auditory and tactile inputs in humans and discuss recent evidence from animal studies demonstrating how cortical and subcortical areas work together to mediate communication between these senses. This research has shown that somatosensory inputs interface with and modulate sound processing at multiple levels of the auditory pathway, from the cochlear nucleus in the brainstem to the cortex. Circuits involving inputs from the primary somatosensory cortex to the auditory midbrain have been identified that mediate suppressive effects of whisker stimulation on auditory thalamocortical processing, providing a possible basis for prioritizing the processing of tactile cues from nearby objects. Close links also exist between audition and movement, and auditory responses are typically suppressed by locomotion and other actions. These movement-related signals are thought to cancel out self-generated sounds, but they may also affect auditory responses via the associated somatosensory stimulation or as a result of changes in brain state. Together, these studies highlight the importance of considering both multisensory context and movement-related activity in order to understand how the auditory cortex operates during natural behaviors, paving the way for future work to investigate auditory-somatosensory interactions in more ecological situations.

Auditory cues facilitate object movement processing in human extrastriate visual cortex during simulated self-motion: A pilot study

Visual segregation of moving objects is a considerable computational challenge when the observer moves through space. Recent psychophysical studies suggest that directionally congruent, moving auditory cues can substantially improve parsing object motion in such settings, but the exact brain mechanisms and visual processing stages that mediate these effects are still incompletely known. Here, we utilized multivariate pattern analyses (MVPA) of MRI-informed magnetoencephalography (MEG) source estimates to examine how crossmodal auditory cues facilitate motion detection during the observer's self-motion. During MEG recordings, participants identified a target object that moved either forward or backward within a visual scene that included nine identically textured objects simulating forward observer translation. Auditory motion cues 1) improved the behavioral accuracy of target localization, 2) significantly modulated the MEG source activity in the areas V2 and human middle temporal complex (hMT+), and 3) increased the accuracy at which the target movement direction could be decoded from hMT+ activity using MVPA. The increase of decoding accuracy by auditory cues in hMT+ was significant also when superior temporal activations in or near auditory cortices were regressed out from the hMT+ source activity to control for source estimation biases caused by point spread. Taken together, these results suggest that parsing object motion from self-motion-induced optic flow in the human extrastriate visual cortex can be facilitated by crossmodal influences from auditory system.

Sound Externalization: A Review of Recent Research

Sound externalization, or the perception that a sound source is outside of the head, is an intriguing phenomenon that has long interested psychoacousticians. While previous reviews are available, the past few decades have produced a substantial amount of new data.In this review, we aim to synthesize those data and to summarize advances in our understanding of the phenomenon. We also discuss issues related to the definition and measurement of sound externalization and describe quantitative approaches that have been taken to predict the outcomes of externalization experiments. Last, sound externalization is of practical importance for many kinds of hearing technologies. Here, we touch on two examples, discussing the role of sound externalization in augmented/virtual reality systems and bringing attention to the somewhat overlooked issue of sound externalization in wearers of hearing aids.

Do we need two ears to perceive the distance of a virtual frontal sound source?

The present study investigated whether the perception of virtual auditory distance is binaural, monaural, or both. Listeners evaluated the distance of a frontal source of pink noise simulated in a room via headphones. Experiment 1 was performed with eyes closed in a soundproof booth. Experiment 2 was performed with eyes open in the room used to create the stimuli. Individualized and non-individualized stimuli were compared. Different conditions for controlling sound level were tested. The amount of binaural information in the stimuli was varied by mixing the left and right ear signals in different proportions. Results showed that the use of non-individualized stimuli did not impair distance perception. Binaural information was not used by naive listeners to evaluate distance, both with and without visual information available. However, for some listeners, a complete absence of binaural information could disrupt distance evaluation with headphones. Sound level was a dominant cue used by listeners to judge for distance, and some listeners could also reliably use reverberation-related changes in spectral content. In the absence of specific training, artificial manipulation of sound level greatly altered distance judgments.

Do sounds near the hand facilitate tactile reaction times? Four experiments and a meta-analysis provide mixed support and suggest a small effect size

The brain represents the space immediately surrounding the body differently to more distant parts of space. Direct evidence for this ‘peripersonal space’ representation comes from neurophysiological studies in monkeys, which show distance-dependent responses to visual stimuli in neurons with spatially coincident tactile responses. Most evidence for peripersonal space in humans is indirect: spatial- and distance-dependent modulations of reaction times and error rates in behavioural tasks. In one task often used to assess peripersonal space, sounds near the body have been argued to speed reactions to tactile stimuli. We conducted four experiments attempting to measure this distance-dependent audiotactile interaction. We found no distance-dependent enhancement of tactile processing in error rates or task performance, but found some evidence for a general speeding of reaction times by 9.5 ms when sounds were presented near the hand. A systematic review revealed an overestimation of reported effect sizes, lack of control conditions, a wide variety of methods, post hoc removal of data, and flexible methods of data analysis. After correcting for the speed of sound, removing biased or inconclusive studies, correcting for temporal expectancy, and using the trim-and-fill method to correct for publication bias, meta-analysis revealed an overall benefit of 15.2 ms when tactile stimuli are accompanied by near sounds compared to sounds further away. While this effect may be due to peripersonal space, response probability and the number of trials per condition explained significant proportions of variance in this near versus far benefit. These confounds need to be addressed, and alternative explanations ruled out by future, ideally pre-registered, studies.