Visual and auditory facilitation in auditory spatial localization

Spontaneous head movements support accurate horizontal auditory localization in a virtual visual environment

This study investigates the relationship between auditory localization accuracy in the horizontal plane and the spontaneous translation and rotation of the head in response to an acoustic stimulus from an invisible sound source. Although a number of studies have suggested that localization ability improves with head movements, most of them measured the perceived source elevation and front-back disambiguation. We investigated the contribution of head movements to auditory localization in the anterior horizontal field in normal hearing subjects. A virtual reality scenario was used to conceal visual cues during the test through a head mounted display. In this condition, we found that an active search of the sound origin using head movements is not strictly necessary, yet sufficient for achieving greater sound source localization accuracy. This result may have important implications in the clinical assessment and training of adults and children affected by hearing and motor impairments.

Sound source localization with varying amount of visual information in virtual reality

To achieve accurate spatial auditory perception, subjects typically require personal head-related transfer functions (HRTFs) and the freedom for head movements. Loudspeaker-based virtual sound environments allow for realism without individualized measurements. To study audio-visual perception in realistic environments, the combination of spatially tracked head mounted displays (HMDs), also known as virtual reality glasses, and virtual sound environments may be valuable. However, HMDs were recently shown to affect the subjects’ HRTFs and thus might influence sound localization performance. Furthermore, due to limitations of the reproduction of visual information on the HMD, audio-visual perception might be influenced. Here, a sound localization experiment was conducted both with and without an HMD and with a varying amount of visual information provided to the subjects. Furthermore, interaural time and level difference errors (ITDs and ILDs) as well as spectral perturbations induced by the HMD were analyzed and compared to the perceptual localization data. The results showed a reduction of the localization accuracy when the subjects were wearing an HMD and when they were blindfolded. The HMD-induced error in azimuth localization was found to be larger in the left than in the right hemisphere. When visual information of the limited set of source locations was provided, the localization error induced by the HMD was found to be negligible. Presenting visual information of hand-location and room dimensions showed better sound localization performance compared to the condition with no visual information. The addition of possible source locations further improved the localization accuracy. Also adding pointing feedback in form of a virtual laser pointer improved the accuracy of elevation perception but not of azimuth perception.

Audiovisual integration in low vision individuals

Behavioral and neurophysiological studies have shown an enhancement of visual perception in crossmodal audiovisual stimulation conditions, both for sensitivity and reaction times, when the stimulation in the two sensory modalities occurs in condition of space and time congruency. The purpose of the present work is to verify whether congruent visual and acoustic stimulations can improve the detection of visual stimuli in people affected by low vision. Participants were asked to detect the presence of a visual stimulus (yes/no task) either presented in isolation (i.e., unimodal visual stimulation) or simultaneously with auditory stimuli, which could be placed in the same spatial position (i.e., crossmodal congruent conditions) or in different spatial positions (i.e., crossmodal incongruent conditions). The results show for the first time audiovisual integration effects in low vision individuals. In particular, it has been observed a significant visual detection benefit in the crossmodal congruent as compared to the unimodal visual condition. This effect is selective for visual stimulation that occurs in the portion of visual field that is impaired, and disappears in the region of space in which vision is spared. Surprisingly, there is a marginal crossmodal benefit when the sound is presented at 16 degrees far from the visual stimulus. The observed crossmodal effect seems to be determined by the contribution of both senses to a model of optimal combination, in which the most reliable provides the highest contribution. These results, indicating a significant beneficial effect of synchronous and spatially congruent sounds in a visual detection task, seem very promising for the development of a rehabilitation approach of low vision diseases based on the principles of multisensory integration.

Visual and proprioceptive representations in spatial memory

It has been shown that spatial information can be acquired from both visual and nonvisual modalities. The present study explored how spatial information from vision and proprioception was represented in memory, investigating orientation dependence of spatial memories acquired through visual and proprioceptive spatial learning. Experiment 1 examined whether visual learning alone and proprioceptive learning alone yielded orientation-dependent spatial memory. Results showed that spatial memories from both types of learning were orientation dependent. Experiment 2 explored how different orientations of the same environment were represented when they were learned visually and proprioceptively. Results showed that both visually and proprioceptively learned orientations were represented in spatial memory, suggesting that participants established two different reference systems based on each type of learning experience and interpreted the environment in terms of these two reference systems. The results provide some initial clues to how different modalities make unique contributions to spatial representations.

Confusing the mind by crossing the hands

Observers made temporal order judgements (TOJs) regarding which of two tactile stimuli presented to either hand (at stimulus onset asynchronies of up to 200 ms) occurred first. When the observers' hands were placed in an uncrossed posture (i.e., each hand in its own hemispace), performance was accurate, with a just noticeable difference (JND; the smallest interval which produces 75% correct performance) of 34 ms. By contrast, when the hands were crossed over the midline, performance declined such that 124 ms was needed for accurate performance. In a second experiment, we presented visual instead of tactile stimuli to evaluate the relative contribution of motor and perceptual confusions to the effect. While performance with crossed hands was significantly worse than with uncrossed hands (JND=36 vs. 31 ms, respectively), this difference was negligible compared to that with tactile stimuli. In a third experiment, experienced observers showed a robust crossed-hands deficit which was not improved by using different fingers on either hand. We argue that the decline in tactile discrimination performance when the hands are crossed reflects a failure to represent appropriately near simultaneous bimanual tactile stimuli, and stands in marked contrast to many recent observations of efficient remapping of singly-presented tactile stimuli.