Multimodal Interaction of Contextual and Non-Contextual Sound and Haptics in Virtual Simulations

Individuals with and without Visual Impairments Use a Force Feedback Device to Identify the Friction and Hardness of Haptic Surfaces

The general purpose of this study is to promote access to haptic virtual environments. Using a haptic device, people with and without visual impairments (VI) are able to feel different textures and compare these textures based on different surface properties, i.e., friction and hardness. The objectives of this study were to examine the following: (a) whether the variables of friction and hardness were identifiable through the Touch device (Phantom Omni) and could therefore function as 3D haptic variables; (b) if there were differences between people with VI and sighted individuals in terms of their performance; (c) the differences that should exist between the values of each variable so that the virtual surfaces could be identified as different to each other; and (d) if the individual characteristics of participants have an impact on their performance. The results showed that it is necessary to use surfaces which are differentiated based on the degree of friction and hardness because the haptic properties of a virtual object are then better perceived. Individuals with VI need more time and more effort to understand friction and hardness, respectively. With the motivation of increasing access to object perception for people with VI in a virtual environment, accessibility advisors and experts can extract useful information for the development of functional and efficient 3D objects for haptic perception.

Effects of auditory feedback on gait behavior, gaze patterns and outcome performance in long jumping

In the current study, we conducted two experiments to investigate the impact of concurrent, action-induced auditory feedback on gait patterns, gaze behavior and outcome performance in long jumping. In Experiment 1, we examined the effects of present vs. absent auditory feedback on gait, gaze and performance outcome measures. Results revealed a significant interaction effect between condition (present vs. absent auditory feedback) and phase (acceleration vs. zeroing-in phase) on participants' step lengths indicating that the absence (rather than the presence) of auditory feedback led to facilitatory effects in terms of a more prototypical gait pattern (i.e., shorter steps in the acceleration phase and longer steps in the zeroing-in phase). Similarly, the absent auditory feedback led to a higher gaze stability in terms of less switches between areas of interest (AOIs). However, there was no effect on jumped distance. In Experiment 2, we scrutinized the influence of concurrent vs. delayed auditory feedback on all three performance parameters. In contrast to concurrent feedback, delayed auditory feedback negatively affected all three measures: participants showed (i) dysfunctional deviations from their prototypical gait pattern (i.e., shorter steps across both phases of the run-up), (ii) less stable, maladaptive gaze patterns (i.e., more switches between AOIs) and (iii) poorer jumping performance (i.e., shorter jumped distances). Together, the two experiments provide clear evidence for the impact of concurrent, action-induced auditory feedback on the coordination of complex, rhythmical motor tasks such as the long jump.

The Influence of Auditory Cues on Bodily and Movement Perception

The sounds that result from our movement and that mark the outcome of our actions typically convey useful information concerning the state of our body and its movement, as well as providing pertinent information about the stimuli with which we are interacting. Here we review the rapidly growing literature investigating the influence of non-veridical auditory cues (i.e., inaccurate in terms of their context, timing, and/or spectral distribution) on multisensory body and action perception, and on motor behavior. Inaccurate auditory cues provide a unique opportunity to study cross-modal processes: the ability to detect the impact of each sense when they provide a slightly different message is greater. Additionally, given that similar cross-modal processes likely occur regardless of the accuracy or inaccuracy of sensory input, studying incongruent interactions are likely to also help us predict interactions between congruent inputs. The available research convincingly demonstrates that perceptions of the body, of movement, and of surface contact features (e.g., roughness) are influenced by the addition of non-veridical auditory cues. Moreover, auditory cues impact both motor behavior and emotional valence, the latter showing that sounds that are highly incongruent with the performed movement induce feelings of unpleasantness (perhaps associated with lower processing fluency). Such findings are relevant to the design of auditory cues associated with product interaction, and the use of auditory cues in sport performance and therapeutic situations given the impact on motor behavior.

Audiohaptic Feedback Enhances Motor Performance in a Low-Fidelity Simulated Drilling Task

When used in educational settings, simulations utilizing virtual reality (VR) technologies can reduce training costs while providing a safe and effective learning environment. Tasks can be easily modified to maximize learning objectives of different levels of trainees (e.g., novice, intermediate, expert), and can be repeated for the development of psychomotor skills. VR offers a multisensory experience, providing visual, auditory, and haptic sensations with varying levels of fidelity. While simulating visual and auditory stimuli is relatively easy and cost-effective, similar representations of haptic sensation still require further development. Evidence suggests that mixing high- and low-fidelity realistic sensations (e.g., audition and haptic) can improve the overall perception of realism, however, whether this also leads to improved performance has not been examined. The current study examined whether audiohaptic stimuli presented in a virtual drilling task can lead to improved motor performance and subjective realism, compared to auditory stimuli alone. Right-handed participants (n = 16) completed 100 drilling trials of each stimulus type. Performance measures indicated that participants overshot the target during auditory trials, and undershot the target during audiohaptic trials. Undershooting is thought to be indicative of improved performance, optimizing both time and energy requirements.