Effects of increasing visual load on aurally and visually guided target acquisition in a virtual environment

Auditory and visual distractors disrupt multisensory temporal acuity in the crossmodal temporal order judgment task

The ability to synthesize information across multiple senses is known as multisensory integration and is essential to our understanding of the world around us. Sensory stimuli that occur close in time are likely to be integrated, and the accuracy of this integration is dependent on our ability to precisely discriminate the relative timing of unisensory stimuli (crossmodal temporal acuity). Previous research has shown that multisensory integration is modulated by both bottom-up stimulus features, such as the temporal structure of unisensory stimuli, and top-down processes such as attention. However, it is currently uncertain how attention alters crossmodal temporal acuity. The present study investigated whether increasing attentional load would decrease crossmodal temporal acuity by utilizing a dual-task paradigm. In this study, participants were asked to judge the temporal order of a flash and beep presented at various temporal offsets (crossmodal temporal order judgment (CTOJ) task) while also directing their attention to a secondary distractor task in which they detected a target stimulus within a stream visual or auditory distractors. We found decreased performance on the CTOJ task as well as increases in both the positive and negative just noticeable difference with increasing load for both the auditory and visual distractor tasks. This strongly suggests that attention promotes greater crossmodal temporal acuity and that reducing the attentional capacity to process multisensory stimuli results in detriments to multisensory temporal processing. Our study is the first to demonstrate changes in multisensory temporal processing with decreased attentional capacity using a dual task paradigm and has strong implications for developmental disorders such as autism spectrum disorders and developmental dyslexia which are associated with alterations in both multisensory temporal processing and attention.

Primary or secondary tasks? Dual-task interference between cyclist hazard perception and cadence control using cross-modal sensory aids with rider assistance bike computers

This research investigated the risks involved in bicycle riding while using various sensory modalities to deliver training information. To understand the risks associated with using bike computers, this study evaluated hazard perception performance through lab-based simulations of authentic riding conditions. Analysing hazard sensitivity (d') of signal detection theory, the rider's response time, and eye glances provided insights into the risks of using bike computers. In this study, 30 participants were tested with eight hazard perception tasks while they maintained a cadence of 60 ± 5 RPM and used bike computers with different sensory displays, namely visual, auditory, and tactile feedback signals. The results indicated that synchronously using different sense organs to receive cadence feedback significantly affects hazard perception performance; direct visual information leads to the worst rider distraction, with a mean sensitivity to hazards (d') of -1.03. For systems with multiple interacting sensory aids, auditory aids were found to result in the greatest reduction in sensitivity to hazards (d' mean = -0.57), whereas tactile sensory aids reduced the degree of rider distraction (d' mean = -0.23). Our work complements existing work in this domain by advancing the understanding of how to design devices that deliver information subtly, thereby preventing disruption of a rider's perception of road hazards.

Visual Distractors Disrupt Audiovisual Integration Regardless of Stimulus Complexity

The intricate relationship between multisensory integration and attention has been extensively researched in the multisensory field; however, the necessity of attention for the binding of multisensory stimuli remains contested. In the current study, we investigated whether diverting attention from well-known multisensory tasks would disrupt integration and whether the complexity of the stimulus and task modulated this interaction. A secondary objective of this study was to investigate individual differences in the interaction of attention and multisensory integration. Participants completed a simple audiovisual speeded detection task and McGurk task under various perceptual load conditions: no load (multisensory task while visual distractors present), low load (multisensory task while detecting the presence of a yellow letter in the visual distractors), and high load (multisensory task while detecting the presence of a number in the visual distractors). Consistent with prior studies, we found that increased perceptual load led to decreased reports of the McGurk illusion, thus confirming the necessity of attention for the integration of speech stimuli. Although increased perceptual load led to longer response times for all stimuli in the speeded detection task, participants responded faster on multisensory trials than unisensory trials. However, the increase in multisensory response times violated the race model for no and low perceptual load conditions only. Additionally, a geometric measure of Miller’s inequality showed a decrease in multisensory integration for the speeded detection task with increasing perceptual load. Surprisingly, we found diverging changes in multisensory integration with increasing load for participants who did not show integration for the no load condition: no changes in integration for the McGurk task with increasing load but increases in integration for the detection task. The results of this study indicate that attention plays a crucial role in multisensory integration for both highly complex and simple multisensory tasks and that attention may interact differently with multisensory processing in individuals who do not strongly integrate multisensory information.

Do whole-body vibrations affect spatial hearing?

To assist the human operator, modern auditory interfaces increasingly rely on sound spatialisation to display auditory information and warning signals. However, we often operate in environments that apply vibrations to the whole body, e.g. when driving a vehicle. Here, we report three experiments investigating the effect of sinusoidal vibrations along the vertical axis on spatial hearing. The first was a free-field, narrow-band noise localisation experiment with 5- Hz vibration at 0.88 ms(-2). The other experiments used headphone-based sound lateralisation tasks. Experiment 2 investigated the effect of vibration frequency (4 vs. 8 Hz) at two different magnitudes (0.83 vs. 1.65 ms(-2)) on a left-right discrimination one-interval forced-choice task. Experiment 3 assessed the effect on a two-interval forced-choice location discrimination task with respect to the central and two peripheral reference locations. In spite of the broad range of methods, none of the experiments show a reliable effect of whole-body vibrations on localisation performance.

PRACTITIONER SUMMARY

We report three experiments that used both free-field localisation and headphone lateralisation tasks to assess their sensitivity to whole-body vibrations at low frequencies. None of the experiments show a reliable effect of either frequency or magnitude of whole-body vibrations on localisation performance.

Effect of Visual-Verbal Load and Spatial Compatibility on Stimulus Response

This study examined the effects of visual-verbal load (as measured by a visually presented reading-memory task with three levels) on a visual/auditory stimulus-response task. The three levels of load were defined as follows: “No Load” meant no other stimuli were presented concurrently; “Free Load” meant that a letter (A, B, C, or D) appeared at the same time as the visual or auditory stimulus; and “Force Load” was the same as “Free Load,” but the participants were also instructed to count how many times the letter A appeared. The stimulus-response task also had three levels: “irrelevant,” “compatible,” and “incompatible” spatial conditions. These required different key-pressing responses. The visual stimulus was a red ball presented either to the left or to the right of the display screen, and the auditory stimulus was a tone delivered from a position similar to that of the visual stimulus. Participants also processed an irrelevant stimulus. The results indicated that participants perceived auditory stimuli earlier than visual stimuli and reacted faster under stimulus-response compatible conditions. These results held even under a high visual-verbal load. These findings suggest the following guidelines for systems used in driving: an auditory source, appropriately compatible signal and manual-response positions, and a visually simplified background.

Identity formation, learning styles and trust in virtual worlds

Avatars in virtual worlds exhibit many of the social behaviors found in real life. As such, we can learn much about our own behaviors from them based on prior theories. But social psychological theories that help researchers understand real life behaviors may not necessarily apply to virtual worlds. In fact, we know little about how human behaviors, as depicted by the actions of avatars, will be different than they are in real life. For insight into this, we look at how virtual worlds impact three behaviors: forming an identity, learning, and trusting. Each of these provides some insight into behavior in virtual worlds and suggests what we must work to understand in order to modify or adapt existing social psychological theories to apply to virtual worlds.