Navigating without vision: basic and applied research

Review of Navigation Assistive Tools and Technologies for the Visually Impaired

The visually impaired suffer greatly while moving from one place to another. They face challenges in going outdoors and in protecting themselves from moving and stationary objects, and they also lack confidence due to restricted mobility. Due to the recent rapid rise in the number of visually impaired persons, the development of assistive devices has emerged as a significant research field. This review study introduces several techniques to help the visually impaired with their mobility and presents the state-of-the-art of recent assistive technologies that facilitate their everyday life. It also analyses comprehensive multiple mobility assistive technologies for indoor and outdoor environments and describes the different location and feedback methods for the visually impaired using assistive tools based on recent technologies. The navigation tools used for the visually impaired are discussed in detail in subsequent sections. Finally, a detailed analysis of various methods is also carried out, with future recommendations.

Evidence for cognitive impairment in patients with vestibular disorders

OBJECTIVE

 Extensive animal research has shown that vestibular damage can be associated with cognitive deficits. More recently, new evidence has emerged linking vestibular disorders to cognitive impairment in humans. Herein, we review contemporary research on the pathophysiology of cognitive-vestibular interactions and discuss its emerging clinical relevance.

DATA SOURCES

PubMed, Embase, and Cochrane databases.

REVIEW METHODS

A systematic literature search was performed with combinations of search terms: "cognition," "cognitive impairment," "chronic fatigue," "brain fog," "spatial navigation," "attention," "memory," "executive function," "processing speed," and "vestibular hypofunction." Relevant articles were considered for inclusion, including basic and clinical studies, case series, and major reviews.

CONCLUSIONS

Patients with vestibular disorders can demonstrate long-term deficits in both spatial and nonspatial cognitive domains. The underlying mechanism(s) linking the vestibular system to cognitive function is not well characterized, but several neuro-biologic correlates have been identified. Additional screening tools are required to identify individuals at risk for cognitive impairment, and further research is needed to determine whether treatment of vestibular dysfunction has the capacity to improve cognitive function.

IMPLICATIONS FOR PRACTICE

Physicians should be aware of emerging data supporting the presence of cognitive deficits in patients with vestibular disorders. Prevention and treatment of long-term cognitive deficits may be possible through screening and rehabilitation.

Cognitive map formation supported by auditory, haptic, and multimodal information in persons with blindness

For efficient navigation, the brain needs to adequately represent the environment in a cognitive map. In this review, we sought to give an overview of literature about cognitive map formation based on non-visual modalities in persons with blindness (PWBs) and sighted persons. The review is focused on the auditory and haptic modalities, including research that combines multiple modalities and real-world navigation. Furthermore, we addressed implications of route and survey representations. Taking together, PWBs as well as sighted persons can build up cognitive maps based on non-visual modalities, although the accuracy sometime somewhat differs between PWBs and sighted persons. We provide some speculations on how to deploy information from different modalities to support cognitive map formation. Furthermore, PWBs and sighted persons seem to be able to construct route as well as survey representations. PWBs can experience difficulties building up a survey representation, but this is not always the case, and research suggests that they can acquire this ability with sufficient spatial information or training. We discuss possible explanations of these inconsistencies.

Morphometry Difference of the Hippocampal Formation Between Blind and Sighted Individuals

The detailed morphometry alterations of the human hippocampal formation (HF) for blind individuals are still understudied. 50 subjects were recruited from Yantai Affiliated Hospital of Binzhou Medical University, including 16 congenital blindness, 14 late blindness, and 20 sighted controls. Volume and shape analysis were conducted between the blind (congenital or late) and sighted groups to observe the (sub)regional alterations of the HF. No significant difference of the hippocampal volume was observed between the blind and sighted subjects. Rightward asymmetry of the hippocampal volume was found for both congenital and late blind individuals, while no significant hemispheric difference was observed for the sighted controls. Shape analysis showed that the superior and inferior parts of both the hippocampal head and tail expanded, while the medial and lateral parts constrained for the blind individuals as compared to the sighted controls. The morphometry alterations for the congenital blind and late blind individuals are nearly the same. Significant expansion of the superior part of the hippocampal tail for both congenital and late blind groups were observed for the left hippocampi after FDR correction. Current results suggest that the cross-model plastic may occur in both hemispheres of the HF to improve the navigation ability without the stimuli of visual cues, and the alteration is more prominent for the left hemisphere.

The influence of yaw rotation on spatial navigation during development

Sensory cues enable navigation through space, as they inform us about movement properties, such as the amount of travelled distance and the heading direction. In this study, we focused on the ability to spatially update one's position when only proprioceptive and vestibular information is available. We aimed to investigate the effect of yaw rotation on path integration across development in the absence of visual feedback. To this end, we utilized the triangle completion task: participants were guided through two legs of a triangle and asked to close the shape by walking along its third imagined leg. To test the influence of yaw rotation across development, we tested children between 6 and 11 years old (y.o.) and adults on their perceptions of angles of different degrees. Our results demonstrated that the amount of turn while executing the angle influences performance at all ages, and in some aspects, also interacted with age. Indeed, whilst adults seemed to adjust their heading towards the end of their walked path, younger children took less advantage of this strategy. The amount of disorientation the path induced also affected participants' full maturational ability to spatially navigate with no visual feedback. Increasing induced disorientation required children to be older to reach adult-level performance. Overall, these results provide novel insights on the maturation of spatial navigation-related processes.

Balance, gait, and navigation performance are related to physical exercise in blind and visually impaired children and adolescents

Self-motion perception used for locomotion and navigation requires the integration of visual, vestibular, and proprioceptive input. In the absence of vision, postural stability and locomotor tasks become more difficult. Previous research has suggested that in visually deprived children, postural stability and levels of physical activity are overall lower than in sighted controls. Here we hypothesized that visually impaired and blind children and adolescents differ from sighted controls in postural stability and gait parameters, and that physically active individuals outperform sedentary peers in postural stability and gait parameters as well as in navigation performance. Fourteen blind and visually impaired children and adolescents (8-18 years of age) and 14 matched sighted individuals took part. Assessments included postural sway, single-leg stance time, parameters of gait variability and stability, self-reported physical activity, and navigation performance. Postural sway was larger and single-leg stance time was lower in blind and visually impaired participants than in blindfolded sighted individuals. Physical activity was higher in the sighted group. No differences between the group of blind and visually impaired and blindfolded sighted participants were observed for gait parameters and navigation performance. Higher levels of physical activity were related to lower postural sway, longer single-leg stance time, higher gait stability, and superior navigation performance in blind and visually impaired participants. The present data suggest that physical activity may enhance postural stability and gait parameters, and thereby promote navigation performance in blind and visually impaired children and adolescents.

Children five-to-nine years old can use path integration to build a cognitive map without vision

Although spatial navigation competence improves greatly from birth to adulthood, different spatial memory capacities emerge at different ages. Here, we characterized the capacity of 5-9-year-old children to use path integration to build egocentric and allocentric spatial representations to navigate in their environment, and compared their performance with that of young adults. First, blindfolded participants were tested on their ability to return to a starting point after being led on straight and two-legged paths. This egocentric homing task comprising angular and linear displacements allowed us to evaluate path integration capacities in absence of external landmarks. Second, we evaluated whether participants could use path integration, in absence of visual information, to create an allocentric spatial representation to navigate along novel paths between objects, and thus demonstrate the ability to build a cognitive map of their environment. Ninety percent of the 5-9-year-old children could use path integration to create an egocentric representation of their journey to return to a starting point, but they were overall less precise than adults. Sixty-four percent of 5-9-year-old children were capable of using path integration to build a cognitive map enabling them to take shortcuts, and task performance was not dependent on age. Imprecisions in novel paths made by the children who built a cognitive map could be explained by poorer integration of the experienced turns during the learning phase, as well as greater individual variability. In sum, these findings demonstrate that 5-9-year-old children can use path integration to build a cognitive map in absence of visual information.

High sensitivity to multisensory conflicts in agoraphobia exhibited by virtual reality

The primary aim of this study was to evaluate the effect of auditory feedback in a VR system planned for clinical use and to address the different factors that should be taken into account in building a bimodal virtual environment (VE). We conducted an experiment in which we assessed spatial performances in agoraphobic patients and normal subjects comparing two kinds of VEs, visual alone (Vis) and auditory–visual (AVis), during separate sessions. Subjects were equipped with a head-mounted display coupled with an electromagnetic sensor system and immersed in a virtual town. Their task was to locate different landmarks and become familiar with the town. In the AVis condition subjects were equipped with the head-mounted display and headphones, which delivered a soundscape updated in real-time according to their movement in the virtual town. While general performances remained comparable across the conditions, the reported feeling of immersion was more compelling in the AVis environment. However, patients exhibited more cybersickness symptoms in this condition. The result of this study points to the multisensory integration deficit of agoraphobic patients and underline the need for further research on multimodal VR systems for clinical use.

Head Anticipation During Locomotion With Auditory Instruction in the Presence and Absence of Visual Input

Head direction has been identified to anticipate trajectory direction during human locomotion. Head anticipation has also been shown to persist in darkness. Arguably, the purpose for this anticipatory behavior is related to motor control and trajectory planning, independently of the visual condition. This implies that anticipation remains in the absence of visual input. However, experiments so far have only explored this phenomenon with visual instructions which intrinsically primes a visual representation to follow. The primary objective of this study is to describe head anticipation in auditory instructed locomotion, in the presence and absence of visual input. Auditory instructed locomotion trajectories were performed in two visual conditions: eyes open and eyes closed. First, 10 sighted participants localized static sound sources to ensure they could understand the sound cues provided. Afterwards, they listened to a moving sound source while actively following it. Later, participants were asked to reproduce the trajectory of the moving sound source without sound. Anticipatory head behavior was observed during trajectory reproduction in both eyes open and closed conditions. The results suggest that head anticipation is related to motor anticipation rather than mental simulation of the trajectory.

Hungry Cat—A Serious Game for Conveying Spatial Information to the Visually Impaired

Navigation is done through obtaining spatial information from the environment and forming a spatial map about it. The visually impaired rely mainly on orientation and mobility training by a certified specialist to acquire spatial navigation skills. However, it is manpower intensive and costly. This research designed and developed a serious game, Hungry Cat. This game can convey spatial information of virtual rooms to children with visual impairment through game playing. An evaluation with 30 visually impaired participants was conducted by allowing them to explore each virtual room in Hungry Cat. After exploration, the food finding test, which is a game mode available in Hungry Cat, was conducted, followed by the physical wire net test to evaluate their ability in forming the spatial mental maps of the virtual rooms. The positive results of the evaluation obtained demonstrate the ability of Hungry Cat, in conveying spatial information about virtual rooms and aiding the development of spatial mental maps of these rooms through game playing.

Stereosonic vision: Exploring visual-to-auditory sensory substitution mappings in an immersive virtual reality navigation paradigm

Sighted people predominantly use vision to navigate spaces, and sight loss has negative consequences for independent navigation and mobility. The recent proliferation of devices that can extract 3D spatial information from visual scenes opens up the possibility of using such mobility-relevant information to assist blind and visually impaired people by presenting this information through modalities other than vision. In this work, we present two new methods for encoding visual scenes using spatial audio: simulated echolocation and distance-dependent hum volume modulation. We implemented both methods in a virtual reality (VR) environment and tested them using a 3D motion-tracking device. This allowed participants to physically walk through virtual mobility scenarios, generating data on real locomotion behaviour. Blindfolded sighted participants completed two tasks: maze navigation and obstacle avoidance. Results were measured against a visual baseline in which participants performed the same two tasks without blindfolds. Task completion time, speed and number of collisions were used as indicators of successful navigation, with additional metrics exploring detailed dynamics of performance. In both tasks, participants were able to navigate using only audio information after minimal instruction. While participants were 65% slower using audio compared to the visual baseline, they reduced their audio navigation time by an average 21% over just 6 trials. Hum volume modulation proved over 20% faster than simulated echolocation in both mobility scenarios, and participants also showed the greatest improvement with this sonification method. Nevertheless, we do speculate that simulated echolocation remains worth exploring as it provides more spatial detail and could therefore be more useful in more complex environments. The fact that participants were intuitively able to successfully navigate space with two new visual-to-audio mappings for conveying spatial information motivates the further exploration of these and other mappings with the goal of assisting blind and visually impaired individuals with independent mobility.

Spatial reorientation with non-visual cues: Failure to spontaneously use auditory information

Among the environmental stimuli that can guide navigation in space, most attention has been dedicated to visual information. The process of determining where you are and which direction you are facing (called reorientation) has been extensively examined by providing the navigator with two sources of information—typically the shape of the environment and its features—with an interest in the extent to which they are used. Similar questions with non-visual cues are lacking. Here, blindfolded sighted participants had to learn the location of a target in a real-world, circular search space. In Experiment 1, two ecologically relevant non-visual cues were provided: the slope of the floor and an array of two identical auditory landmarks. Slope successfully guided behaviour, suggesting that proprioceptive/kinesthetic access is sufficient to navigate on a slanted environment. However, despite the fact that participants could localise the auditory sources, this information was not encoded. In Experiment 2, the auditory cue was made more useful for the task because it had greater predictive value and there were no competing spatial cues. Nonetheless, again, the auditory landmark was not encoded. Finally, in Experiment 3, after being prompted, participants were able to reorient by using the auditory landmark. Overall, participants failed to spontaneously rely on the auditory cue, regardless of how informative it was.

Missing the egocentric spatial reference: a blank on the map

Egocentric (self-centered) and allocentric (viewpoint independent) representations of space are essential for spatial navigation and wayfinding. Deficits in spatial memory come with age-related cognitive decline, are marked in mild cognitive impairment (MCI) and Alzheimer’s disease (AD), and are associated with cognitive deficits in autism. In most of these disorders, a change in the brain areas engaged in the spatial reference system processing has been documented. However, the spatial memory deficits observed during physiological and pathological aging are quite different. While patients with AD and MCI have a general spatial navigation impairment in both allocentric and egocentric strategies, healthy older adults are particularly limited in the allocentric navigation, but they can still count on egocentric navigation strategy to solve spatial tasks. Therefore, specific navigational tests should be considered for differential diagnosis between healthy and pathological aging conditions. Finally, more research is still needed to better understand the spatial abilities of autistic individuals.

An Indoor Wayfinding System Based on Geometric Features Aided Graph SLAM for the Visually Impaired

This paper presents a 6-degree of freedom (DOF) pose estimation (PE) method and an indoor wayfinding system based on the method for the visually impaired. The PE method involves two-graph simultaneous localization and mapping (SLAM) processes to reduce the accumulative pose error of the device. In the first step, the floor plane is extracted from the 3-D camera's point cloud and added as a landmark node into the graph for 6-DOF SLAM to reduce roll, pitch, and Z errors. In the second step, the wall lines are extracted and incorporated into the graph for 3-DOF SLAM to reduce X , Y , and yaw errors. The method reduces the 6-DOF pose error and results in more accurate pose with less computational time than the state-of-the-art planar SLAM methods. Based on the PE method, a wayfinding system is developed for navigating a visually impaired person in an indoor environment. The system uses the estimated pose and floor plan to locate the device user in a building and guides the user by announcing the points of interest and navigational commands through a speech interface. Experimental results validate the effectiveness of the PE method and demonstrate that the system may substantially ease an indoor navigation task.

Shape Perception and Navigation in Blind Adults

Different sensory systems interact to generate a representation of space and to navigate. Vision plays a critical role in the representation of space development. During navigation, vision is integrated with auditory and mobility cues. In blind individuals, visual experience is not available and navigation therefore lacks this important sensory signal. In blind individuals, compensatory mechanisms can be adopted to improve spatial and navigation skills. On the other hand, the limitations of these compensatory mechanisms are not completely clear. Both enhanced and impaired reliance on auditory cues in blind individuals have been reported. Here, we develop a new paradigm to test both auditory perception and navigation skills in blind and sighted individuals and to investigate the effect that visual experience has on the ability to reproduce simple and complex paths. During the navigation task, early blind, late blind and sighted individuals were required first to listen to an audio shape and then to recognize and reproduce it by walking. After each audio shape was presented, a static sound was played and the participants were asked to reach it. Movements were recorded with a motion tracking system. Our results show three main impairments specific to early blind individuals. The first is the tendency to compress the shapes reproduced during navigation. The second is the difficulty to recognize complex audio stimuli, and finally, the third is the difficulty in reproducing the desired shape: early blind participants occasionally reported perceiving a square but they actually reproduced a circle during the navigation task. We discuss these results in terms of compromised spatial reference frames due to lack of visual input during the early period of development.

Outdoor difficulties experienced by a group of visually impaired Iranian people

Propose

A qualitative approach using semi-structured individual interviews was used to elicit common outdoor difficulties in individuals with visual impairment.

Methods

Interviews were recorded and then transcribed verbatim into text for thematic analysis. Twenty legally-blind individuals aged 34.25 ± 2.41 years with different etiologies were included in this study.

Results

All participants had experienced some sort of difficulty in outdoor environments. The most important problems as perceived by the participants are installation of tactile ground surface indicators, unsafe sidewalks, existence of obstacles on sidewalks, difficulty reading bus numbers, disorientation, fear of falling, recognition of faces, inability to read street names, the presence of spaces between platforms and buses, walking into glass doors, crossing streets, and the risk of Arial barriers.

Conclusions

As a visually impaired person might say, sidewalks can be the most dangerous of places. Appropriate urban modification can be very beneficial.

Indoor Spatial Updating with Reduced Visual Information

Purpose

Spatial updating refers to the ability to keep track of position and orientation while moving through an environment. People with impaired vision may be less accurate in spatial updating with adverse consequences for indoor navigation. In this study, we asked how artificial restrictions on visual acuity and field size affect spatial updating, and also judgments of the size of rooms.

Methods

Normally sighted young adults were tested with artificial restriction of acuity in Mild Blur (Snellen 20/135) and Severe Blur (Snellen 20/900) conditions, and a Narrow Field (8°) condition. The subjects estimated the dimensions of seven rectangular rooms with and without these visual restrictions. They were also guided along three-segment paths in the rooms. At the end of each path, they were asked to estimate the distance and direction to the starting location. In Experiment 1, the subjects walked along the path. In Experiment 2, they were pushed in a wheelchair to determine if reduced proprioceptive input would result in poorer spatial updating.

Results

With unrestricted vision, mean Weber fractions for room-size estimates were near 20%. Severe Blur but not Mild Blur yielded larger errors in room-size judgments. The Narrow Field was associated with increased error, but less than with Severe Blur. There was no effect of visual restriction on estimates of distance back to the starting location, and only Severe Blur yielded larger errors in the direction estimates. Contrary to expectation, the wheelchair subjects did not exhibit poorer updating performance than the walking subjects, nor did they show greater dependence on visual condition.

Discussion

If our results generalize to people with low vision, severe deficits in acuity or field will adversely affect the ability to judge the size of indoor spaces, but updating of position and orientation may be less affected by visual impairment.

Salient Feature of Haptic-Based Guidance of People in Low Visibility Environments Using Hard Reins

This paper presents salient features of human-human interaction where one person with limited auditory and visual perception of the environment (a follower) is guided by an agent with full perceptual capabilities (a guider) via a hard rein along a given path. We investigate several salient features of the interaction between the guider and follower such as: 1) the order of an autoregressive (AR) control policy that maps states of the follower to actions of the guider; 2) how the guider may modulate the pulling force in response to the trust level of the follower; and 3) how learning may successively apportion the responsibility of control across different muscles of the guider. Based on experimental systems identification on human demonstrations from ten pairs of naive subjects, we show that guiders tend to adopt a third-order AR predictive control policy and followers tend to adopt second-order reactive control policy. Moreover, the extracted guider's control policy was implemented and validated by human-robot interaction experiments. By modeling the follower's dynamics with a time varying virtual damped inertial system, we found that it is the coefficient of virtual damping which is most sensitive to the trust level of the follower. We used these experimental insights to derive a novel controller that integrates an optimal order control policy with a push/pull force modulator in response to the trust level of the follower monitored using a time varying virtual damped inertial model.

Impact of Vestibular Lesions on Allocentric Navigation and Interval Timing: The Role of Self-Initiated Motion in Spatial-Temporal Integration

Bilateral intratympanic sodium arsenate injections (100 mg/ml in isotonic saline) in adult male Long Evans rats produced impairments in allocentric navigation using a 12-arm radial maze procedure as well as a motor test battery designed to evaluate vestibular function. In contrast, no impairments in the accuracy or precision of duration reproduction using 20-s and 80-s peak-interval procedures were observed when both target durations were associated with the same lever response, but distinguished by signal modality (e.g., light or sound). In contrast, an ordinal-reproduction procedure with 800, 3200, and 12,800 ms standards requiring the timing of self-initiated movements during the production phase revealed large impairments in the accuracy and precision of timing for vestibular lesioned rats. These impairments were greater on trials in which self-initiated body movements (e.g., holding down the response lever for a fixed duration) were required without the support of external stimuli signaling the onset and offset of the reproduced duration in contrast to trials in which such external support was provided. The conclusion is that space and time are separable entities and not simply the product of a generalized system, but they can be integrated into a common metric using gravity and self-initiated movement as a reference.

Identification of Haptic Based Guiding Using Hard Reins

This paper presents identifications of human-human interaction in which one person with limited auditory and visual perception of the environment (a follower) is guided by an agent with full perceptual capabilities (a guider) via a hard rein along a given path. We investigate several identifications of the interaction between the guider and the follower such as computational models that map states of the follower to actions of the guider and the computational basis of the guider to modulate the force on the rein in response to the trust level of the follower. Based on experimental identification systems on human demonstrations show that the guider and the follower experience learning for an optimal stable state-dependent novel 3^rd and 2^nd order auto-regressive predictive and reactive control policies respectively. By modeling the follower’s dynamics using a time varying virtual damped inertial system, we found that the coefficient of virtual damping is most appropriate to explain the trust level of the follower at any given time. Moreover, we present the stability of the extracted guiding policy when it was implemented on a planar 1-DoF robotic arm. Our findings provide a theoretical basis to design advanced human-robot interaction algorithms applicable to a variety of situations where a human requires the assistance of a robot to perceive the environment.

Why vision is important to how we navigate

Place cells are a fundamental component of the rodent navigational system. One intriguing implication of place cells is that humans, by extension, have "map-like" (or GPS-like) knowledge that we use to represent space. Here, we review both behavioral and neural studies of human navigation, suggesting that how we process visual information forms a critical component of how we represent space. These include cellular and brain systems devoted to coding visual information during navigation in addition to a location coding system similar to that described in rodents. Together, these findings suggest that while it is highly useful to think of our navigation system involving internal "maps," we should not neglect the importance of high-resolution visual representations to how we navigate space.

Navigating from a Depth Image Converted into Sound

Background. Common manufactured depth sensors generate depth images that humans normally obtain from their eyes and hands. Various designs converting spatial data into sound have been recently proposed, speculating on their applicability as sensory substitution devices (SSDs). Objective. We tested such a design as a travel aid in a navigation task. Methods. Our portable device (MeloSee) converted 2D array of a depth image into melody in real-time. Distance from the sensor was translated into sound intensity, stereo-modulated laterally, and the pitch represented verticality. Twenty-one blindfolded young adults navigated along four different paths during two sessions separated by one-week interval. In some instances, a dual task required them to recognize a temporal pattern applied through a tactile vibrator while they navigated. Results. Participants learnt how to use the system on both new paths and on those they had already navigated from. Based on travel time and errors, performance improved from one week to the next. The dual task was achieved successfully, slightly affecting but not preventing effective navigation. Conclusions. The use of Kinect-type sensors to implement SSDs is promising, but it is restricted to indoor use and it is inefficient on too short range.

From ear to body: the auditory-motor loop in spatial cognition

Spatial memory is mainly studied through the visual sensory modality: navigation tasks in humans rarely integrate dynamic and spatial auditory information. In order to study how a spatial scene can be memorized on the basis of auditory and idiothetic cues only, we constructed an auditory equivalent of the Morris water maze, a task widely used to assess spatial learning and memory in rodents. Participants were equipped with wireless headphones, which delivered a soundscape updated in real time according to their movements in 3D space. A wireless tracking system (video infrared with passive markers) was used to send the coordinates of the subject's head to the sound rendering system. The rendering system used advanced HRTF-based synthesis of directional cues and room acoustic simulation for the auralization of a realistic acoustic environment. Participants were guided blindfolded in an experimental room. Their task was to explore a delimitated area in order to find a hidden auditory target, i.e., a sound that was only triggered when walking on a precise location of the area. The position of this target could be coded in relationship to auditory landmarks constantly rendered during the exploration of the area. The task was composed of a practice trial, 6 acquisition trials during which they had to memorize the localization of the target, and 4 test trials in which some aspects of the auditory scene were modified. The task ended with a probe trial in which the auditory target was removed. The configuration of searching paths allowed observing how auditory information was coded to memorize the position of the target. They suggested that space can be efficiently coded without visual information in normal sighted subjects. In conclusion, space representation can be based on sensorimotor and auditory cues only, providing another argument in favor of the hypothesis that the brain has access to a modality-invariant representation of external space.

Pointing to azimuths and elevations of targets: blind and blindfolded-sighted

Three groups of observers pointed to target circles in a path on the ground, in two parallel rows. Participants in one group viewed the circles and then pointed blindfolded. Those in a second group were blindfolded and then touched the circles with a stick while walking past them. Volunteers in the third group were blind adults, a diverse group, who also used a stick to detect the circles. For all three groups, as distance to the circles increased, pointing azimuths shrank and elevations increased. We suggest that directions to targets on major environmental surfaces may be appreciated similarly by the blind and sighted. We challenge the assumption that the principle of convergence to the horizon, available through vision because of the way in which visual angle decreases on the retina, is not available through touch.

Virtual environments for the transfer of navigation skills in the blind: a comparison of directed instruction vs. video game based learning approaches

For profoundly blind individuals, navigating in an unfamiliar building can represent a significant challenge. We investigated the use of an audio-based, virtual environment called Audio-based Environment Simulator (AbES) that can be explored for the purposes of learning the layout of an unfamiliar, complex indoor environment. Furthermore, we compared two modes of interaction with AbES. In one group, blind participants implicitly learned the layout of a target environment while playing an exploratory, goal-directed video game. By comparison, a second group was explicitly taught the same layout following a standard route and instructions provided by a sighted facilitator. As a control, a third group interacted with AbES while playing an exploratory, goal-directed video game however, the explored environment did not correspond to the target layout. Following interaction with AbES, a series of route navigation tasks were carried out in the virtual and physical building represented in the training environment to assess the transfer of acquired spatial information. We found that participants from both modes of interaction were able to transfer the spatial knowledge gained as indexed by their successful route navigation performance. This transfer was not apparent in the control participants. Most notably, the game-based learning strategy was also associated with enhanced performance when participants were required to find alternate routes and short cuts within the target building suggesting that a ludic-based training approach may provide for a more flexible mental representation of the environment. Furthermore, outcome comparisons between early and late blind individuals suggested that greater prior visual experience did not have a significant effect on overall navigation performance following training. Finally, performance did not appear to be associated with other factors of interest such as age, gender, and verbal memory recall. We conclude that the highly interactive and immersive exploration of the virtual environment greatly engages a blind user to develop skills akin to positive near transfer of learning. Learning through a game play strategy appears to confer certain behavioral advantages with respect to how spatial information is acquired and ultimately manipulated for navigation.

Real world navigation independence in the early blind correlates with differential brain activity associated with virtual navigation

Navigating is a complex cognitive task that places high demands on spatial abilities, particularly in the absence of sight. Significant advances have been made in identifying the neural correlates associated with various aspects of this skill; however, how the brain is able to navigate in the absence of visual experience remains poorly understood. Furthermore, how neural network activity relates to the wide variability in navigational independence and skill in the blind population is also unknown. Using functional magnetic resonance imaging, we investigated the neural correlates of audio-based navigation within a large scale, indoor virtual environment in early profoundly blind participants with differing levels of spatial navigation independence (assessed by the Santa Barbara Sense of Direction scale). Performing path integration tasks in the virtual environment was associated with activation within areas of a core network implicated in navigation. Furthermore, we found a positive relationship between Santa Barbara Sense of Direction scores and activation within right temporal parietal junction during the planning and execution phases of the task. These findings suggest that differential navigational ability in the blind may be related to the utilization of different brain network structures. Further characterization of the factors that influence network activity may have important implications regarding how this skill is taught in the blind community.

Development of an audio-based virtual gaming environment to assist with navigation skills in the blind

Audio-based Environment Simulator (AbES) is virtual environment software designed to improve real world navigation skills in the blind. Using only audio based cues and set within the context of a video game metaphor, users gather relevant spatial information regarding a building's layout. This allows the user to develop an accurate spatial cognitive map of a large-scale three-dimensional space that can be manipulated for the purposes of a real indoor navigation task. After game play, participants are then assessed on their ability to navigate within the target physical building represented in the game. Preliminary results suggest that early blind users were able to acquire relevant information regarding the spatial layout of a previously unfamiliar building as indexed by their performance on a series of navigation tasks. These tasks included path finding through the virtual and physical building, as well as a series of drop off tasks. We find that the immersive and highly interactive nature of the AbES software appears to greatly engage the blind user to actively explore the virtual environment. Applications of this approach may extend to larger populations of visually impaired individuals.

Path integration: effect of curved path complexity and sensory system on blindfolded walking

Path integration refers to the ability to integrate continuous information of the direction and distance traveled by the system relative to the origin. Previous studies have investigated path integration through blindfolded walking along simple paths such as straight line and triangles. However, limited knowledge exists regarding the role of path complexity in path integration. Moreover, little is known about how information from different sensory input systems (like vision and proprioception) contributes to accurate path integration. The purpose of the current study was to investigate how sensory information and curved path complexity affect path integration. Forty blindfolded participants had to accurately reproduce a curved path and return to the origin. They were divided into four groups that differed in the curved path, circle (simple) or figure-eight (complex), and received either visual (previously seen) or proprioceptive (previously guided) information about the path before they reproduced it. The dependent variables used were average trajectory error, walking speed, and distance traveled. The results indicated that (a) both groups that walked on a circular path and both groups that received visual information produced greater accuracy in reproducing the path. Moreover, the performance of the group that received proprioceptive information and later walked on a figure-eight path was less accurate than their corresponding circular group. The groups that had the visual information also walked faster compared to the group that had proprioceptive information. Results of the current study highlight the roles of different sensory inputs while performing blindfolded walking for path integration.

Teaching the blind to find their way by playing video games

Computer based video games are receiving great interest as a means to learn and acquire new skills. As a novel approach to teaching navigation skills in the blind, we have developed Audio-based Environment Simulator (AbES); a virtual reality environment set within the context of a video game metaphor. Despite the fact that participants were naïve to the overall purpose of the software, we found that early blind users were able to acquire relevant information regarding the spatial layout of a previously unfamiliar building using audio based cues alone. This was confirmed by a series of behavioral performance tests designed to assess the transfer of acquired spatial information to a large-scale, real-world indoor navigation task. Furthermore, learning the spatial layout through a goal directed gaming strategy allowed for the mental manipulation of spatial information as evidenced by enhanced navigation performance when compared to an explicit route learning strategy. We conclude that the immersive and highly interactive nature of the software greatly engages the blind user to actively explore the virtual environment. This in turn generates an accurate sense of a large-scale three-dimensional space and facilitates the learning and transfer of navigation skills to the physical world.

MPI CyberMotion Simulator: implementation of a novel motion simulator to investigate multisensory path integration in three dimensions

Path integration is a process in which self-motion is integrated over time to obtain an estimate of one's current position relative to a starting point ¹. Humans can do path integration based exclusively on visual ^2-3, auditory ⁴, or inertial cues ⁵. However, with multiple cues present, inertial cues - particularly kinaesthetic - seem to dominate ^6-7. In the absence of vision, humans tend to overestimate short distances (<5 m) and turning angles (<30°), but underestimate longer ones ⁵. Movement through physical space therefore does not seem to be accurately represented by the brain.

Extensive work has been done on evaluating path integration in the horizontal plane, but little is known about vertical movement (see ³ for virtual movement from vision alone). One reason for this is that traditional motion simulators have a small range of motion restricted mainly to the horizontal plane. Here we take advantage of a motion simulator ^8-9 with a large range of motion to assess whether path integration is similar between horizontal and vertical planes. The relative contributions of inertial and visual cues for path navigation were also assessed.

16 observers sat upright in a seat mounted to the flange of a modified KUKA anthropomorphic robot arm. Sensory information was manipulated by providing visual (optic flow, limited lifetime star field), vestibular-kinaesthetic (passive self motion with eyes closed), or visual and vestibular-kinaesthetic motion cues. Movement trajectories in the horizontal, sagittal and frontal planes consisted of two segment lengths (1st: 0.4 m, 2nd: 1 m; ±0.24 m/s² peak acceleration). The angle of the two segments was either 45° or 90°. Observers pointed back to their origin by moving an arrow that was superimposed on an avatar presented on the screen.

Observers were more likely to underestimate angle size for movement in the horizontal plane compared to the vertical planes. In the frontal plane observers were more likely to overestimate angle size while there was no such bias in the sagittal plane. Finally, observers responded slower when answering based on vestibular-kinaesthetic information alone. Human path integration based on vestibular-kinaesthetic information alone thus takes longer than when visual information is present. That pointing is consistent with underestimating and overestimating the angle one has moved through in the horizontal and vertical planes respectively, suggests that the neural representation of self-motion through space is non-symmetrical which may relate to the fact that humans experience movement mostly within the horizontal plane.

Divergent effects of age on performance in spatial associative learning and real idiothetic memory in humans

This study focuses on age-related differences concerning two kinds of spatial memory assessed by: (1) Paired Associates Learning (PAL) test from the CANTAB and (2) a test of Real Idiothetic Memory (RIM) using real-life settings. Despite a clear age-related drop in PAL that is reported in existing studies, age-related differences in idiothetic navigation still remain unclear. In our study we tested 80 healthy volunteers classified according to their age into two groups, i.e. young (aged from 20 to 29 years of life; n=40; 20M/20F) and elderly (from 64 to 77 years; n=40; 20M/20F) healthy volunteers. They were asked in the PAL test to remember the spatial location of visual patterns presented on a computer screen, and in the RIM test to walk on the arena in darkness in order to find a cue place and then to return to the start/exit point. A white noise was switched on at entering the cue place and switched off at leaving this place. Elderly subjects indicated poorer performance than their younger counterparts on the PAL test, as evidenced by all tested outcome measures. In contrast, for the RIM test no clear age effect was evidenced. In both tests no gender effect was observed. A dissociation in age-related changes for these two tests indicates that visuo-spatial associative learning and idiothetic navigation may have different cognitive control which is probably rooted in an interplay of different brain structures.

Locomotor body scheme

The concept of body schema has been introduced and widely discussed in the literature to explain various clinical observations and distortions in the body and space representation. Here we address the role of body schema related information in multi-joint limb motion. The processing of proprioceptive information may differ significantly in static and dynamic conditions since in the latter case the control system may employ specific dynamic rules and constraints. Accordingly, the perception of movement, e.g., estimation of step length and walking distance, may rely on a priori knowledge about intrinsic dynamics of limb segment motion and inherent relationships between gait parameters and body proportions. The findings are discussed in the general framework of space and body movement representation and suggest the existence of a dynamic locomotor body schema used for controlling step length and path estimation.

Crosswatch: a Camera Phone System for Orienting Visually Impaired Pedestrians at Traffic Intersections

Urban intersections are the most dangerous parts of a blind or visually impaired person's travel. To address this problem, this paper describes the novel "Crosswatch" system, which uses computer vision to provide information about the location and orientation of crosswalks to a blind or visually impaired pedestrian holding a camera cell phone. A prototype of the system runs on an off-the-shelf Nokia camera phone in real time, which automatically takes a few images per second, uses the cell phone's built-in computer to analyze each image in a fraction of a second and sounds an audio tone when it detects a crosswalk. Tests with blind subjects demonstrate the feasibility of the system and its ability to provide useful crosswalk alignment information under real-world conditions.

Vestibular influence on auditory metrical interpretation

When we move to music we feel the beat, and this feeling can shape the sound we hear. Previous studies have shown that when people listen to a metrically ambiguous rhythm pattern, moving the body on a certain beat--adults, by actively bouncing themselves in synchrony with the experimenter, and babies, by being bounced passively in the experimenter's arms--can bias their auditory metrical representation so that they interpret the pattern in a corresponding metrical form [Phillips-Silver, J., & Trainor, L. J. (2005). Feeling the beat: Movement influences infant rhythm perception. Science, 308, 1430; Phillips-Silver, J., & Trainor, L. J. (2007). Hearing what the body feels: Auditory encoding of rhythmic movement. Cognition, 105, 533-546]. The present studies show that in adults, as well as in infants, metrical encoding of rhythm can be biased by passive motion. Furthermore, because movement of the head alone affected auditory encoding whereas movement of the legs alone did not, we propose that vestibular input may play a key role in the effect of movement on auditory rhythm processing. We discuss possible cortical and subcortical sites for the integration of auditory and vestibular inputs that may underlie the interaction between movement and auditory metrical rhythm perception.

Straight after the turn: the role of the parietal lobes in egocentric space processing

Spatial information processing with respect to an egocentric reference frame has been shown to recruit a fronto-parietal network along the dorsal stream. The present study investigates how brain lesions in the relevant areas affect the ability to navigate through computer-simulated tunnels shown from a first person perspective. Our results suggest that parietal, but not frontal, patients are impaired in this task. They confused the direction of tunnel turns more frequently and made less accurate judgments about the location of the end position. Errors in map drawing suggest that the impairment may be linked to deficits in updating cognitive heading in the absence of corresponding perceptual information from the virtual environment.

Real and virtual mobility performance in simulated prosthetic vision

Wayfinding is an important activity that can be performed with limited visual resources, and thus may be an important application of early visual prostheses. In a pair of experiments we explored minimal visual resolution requirements of a simulated retinal electrode array for mobility in real and virtual environments, experienced by normally sighted subjects in video headsets. In experiment 1, inexperienced and experienced subjects traveled similar routes around a suite of offices with simulated implants of 4 x 4, 6 x 10 and 16 x 16 dots. In experiment 2, the effects of adding dynamic noise and removing a subset of 'phosphenes' from a 6 x 10 dot array on the mobility of experienced subjects through a series of different virtual 10-room buildings were determined. Performance was quantified in terms of time and navigation errors in both experiments, and wall contacts in the real environment; a compound score was also computed for trials in experiment 1. In experiment 1, inexperienced subjects required 16 x 16 dots for adequate performance, while experienced subjects reached similar levels with 6 x 10 dots. In experiment 2, dot removal up to 30% led to modest yet significant performance deterioration, and noise addition to slight but non-significant improvement, while practice led to a reduction in travel time by 50% over the 28-trial experiment. Error counts in experiment 2 were fairly high, but largely randomly distributed, and attributable to the high risk of becoming disoriented in the sparse visual environment. Substantial performance level differences were found between subjects, spanning a threefold range even after practice. The findings suggest that a retinal implant with as few as 60 electrodes may provide independent wayfinding abilities to the adventitiously blind, but that substantial practice and supervision will be required in learning this task.

Cognitive and brain mechanisms in sensory substitution of vision: a contribution to the study of human perception

Sensory substitution refers to the use of one sense to pick up information normally gathered by another sensory organ. With recent technological advances and scientific progress, sensory substitution appears as an interesting alternative for restoring some functions of a defective sensory organ (e.g., the sight in case of blindness). At the same time, our knowledge about cognitive and brain mechanisms involved in sensory substitution has grown considerably, bringing new insights into human perception and neural plasticity. From this perspective, sensory substitution can be considered as both a tool to investigate human cognition and brain functions, and a research topic in its own right. This paper addresses some of the major questions raised by sensory substitution, demonstrates how the study of sensory substitution enhances our understanding of human perception and brain plasticity and provides an overview of rehabilitation potentialities.

Navigation performance with a virtual auditory display: effects of beacon sound, capture radius, and practice

OBJECTIVE

We examined whether spatialized nonspeech beacons could guide navigation and how sound timbre, waypoint capture radius, and practice affect performance.

BACKGROUND

Auditory displays may assist mobility and wayfinding for those with temporary or permanent visual impairment, but they remain understudied. Previous systems have used speech-based interfaces.

METHOD

Participants (108 undergraduates) navigated three maps, guided by one of three beacons (pink noise, sonar ping, or 1000-Hz pure tone) spatialized by a virtual reality engine. Dependent measures were efficiency of time and path length.

RESULTS

Overall navigation was very successful, with significant effects of practice and capture radius, and interactions with beacon sound. Overshooting and subsequent hunting for waypoints was exacerbated for small radius conditions. A human-scale capture radius (1.5 m) and sonar-like beacon yielded the optimal combination for safety and efficiency.

CONCLUSION

The selection of beacon sound and capture radius depend on the specific application, including whether speed of travel or adherence to path are of primary concern. Extended use affects sound preferences and quickly leads to improvements in both speed and accuracy.

APPLICATION

These findings should lead to improved wayfinding systems for the visually impaired as well as for first responders (e.g., firefighters) and soldiers.

Geographical slant facilitates navigation and orientation in virtual environments

Theoretical considerations and earlier experimental findings indicate that traveling over slanted terrain can lead to an enrichment of the perceived spatial cues relevant for navigation. We investigated the proposed facilitation of a uniformly slanted environment on navigation and orientation performance with a virtual environment presented on a large 180 degrees screen, using as material a virtual town with eight places and twenty-four landmarks. In the control condition, this town was placed on a flat surface; in the two experimental conditions, the town was placed on a slope with a uniform angle of 4 degrees. Pedaling on a bicycle simulator, participants first explored the environment, then solved navigation tasks, pointed from various positions to distant landmarks, judged the relative elevation of pairs of distant landmarks from memory, and finally drew a sketch map of the environment. In comparison to the control condition, the number of navigation errors was significantly lower in the slanted conditions, and the deviations in the pointings to distant landmarks were massively reduced. Participants from the slant conditions also showed good knowledge of the relative elevations of pairs of distant locations. However, no differences in map-drawing quality were found. The results lend additional support to the proposition that our spatial knowledge, which is used in navigation and orientation, contains vertical information.

Audiological assessment, rehabilitation, and spatial hearing considerations associated with visual impairment in adults: an overview

The purpose of this paper is to increase awareness and interest among audiologists regarding the needs of adults who have both hearing loss and visual impairment, particularly people who are blind and travel independently. Case history, audiometric testing, and rehabilitation issues are considered, and extensive discussion of spatial hearing research as it relates to orientation and mobility is provided.

Does Psychology make a significant difference in our lives?

The intellectual tension between the virtues of basic versus applied research that characterized an earlier era of psychology is being replaced by an appreciation of creative applications of all research essential to improving the quality of human life. Psychologists are positioned to "give psychology away" to all those who can benefit from our wisdom. Psychologists were not there 35 years ago when American Psychological Association (APA) President George Miller first encouraged us to share our knowledge with the public. The author argues that psychology is indeed making a significant difference in people's lives; this article provides a sampling of evidence demonstrating how and why psychology matters, both in pervasive ways and specific applications. Readers are referred to a newly developed APA Web site that documents current operational uses of psychological research, theory, and methodology (its creation has been the author's primary presidential initiative): www.psychologymatters.org.

Lateralization of narrow-band noise by blind and sighted listeners

The effects of varying interaural time delay (ITD) and interaural intensity difference (IID) were measured in normal-hearing sighted and congenitally blind subjects as a function of eleven frequencies and at sound pressure levels of 70 and 90 dB, and at a sensation level of 25 dB (sensation level refers to the pressure level of the sound above its threshold for the individual subject). Using an 'acoustic' pointing paradigm, the subject varied the IID of a 500 Hz narrow-band (100 Hz) noise (the 'pointer') to coincide with the apparent lateral position of a 'target' ITD stimulus. ITDs of 0, +/-200, and +/-400 micros were obtained through total waveform delays of narrow-band noise, including envelope and fine structure. For both groups, the results of this experiment confirm the traditional view of binaural hearing for like stimuli: non-zero ITDs produce little perceived lateral displacement away from 0 IID at frequencies above 1250 Hz. To the extent that greater magnitude of lateralization for a given ITD, presentation level, and center frequency can be equated with superior localization abilities, blind listeners appear at least comparable and even somewhat better than sighted subjects, especially when attending to signals in the periphery. The present findings suggest that blind listeners are fully able to utilize the cues for spatial hearing, and that vision is not a mandatory prerequisite for the calibration of human spatial hearing.