VES: A Mixed-Reality System to Assist Multisensory Spatial Perception and Cognition for Blind and Visually Impaired People

EchoSee: An Assistive Mobile Application for Real-Time 3D Environment Reconstruction and Sonification Supporting Enhanced Navigation for People with Vision Impairments

Improving the quality of life for people with vision impairments has been an important goal in the research and design of assistive devices for several decades. This paper seeks to further that goal by introducing a novel assistive technology platform that leverages real-time 3D spatial audio to promote safe and efficient navigation for people who are blind or visually impaired (PVI). The presented platform, EchoSee, uses modern 3D scanning technology on a mobile device to construct a live, digital 3D map of a user's environment as they move about their surroundings. Spatialized, virtual audio sources (i.e., virtual speakers) are dynamically placed within the digital 3D scan of the world, providing the navigator with a real-time 3D stereo audio "soundscape." The digital 3D map, and its resultant soundscape, are continuously updated as the user moves about their environment. The generated soundscape is played back through headphones connected to the navigator's device. This paper details (1) the underlying technical components and how they were integrated to produce the mobile application that generates a dynamic soundscape on a consumer mobile device, (2) a methodology for analyzing navigation performance with the application, (3) the design and execution of a user study investigating the effectiveness of the presented system, and (4) a discussion of the results of that study along with a proposed future study and possible improvements. Altogether, this paper presents a novel software platform aimed at assisting individuals with vision impairments to navigate and understand spaces safely, efficiently, and independently and the results of a feasibility study analyzing the viability of the approach.

Survey and analysis of the current status of research in the field of outdoor navigation for the blind

PURPOSE

In this article, we comprehensively review the current situation and research on technology related to outdoor travel for blind and visually impaired people (BVIP), given the diverse types and incomplete functionality of navigation aids for the blind. This aims to provide a reference for related research in the fields of outdoor travel for BVIP and blind navigation.

MATERIALS AND METHODS

We compiled articles related to blind navigation, of which a total of 227 of them are included in the search criteria. One hundred and seventy-nine articles are selected from the initial set, from a technical point of view, to elaborate on five aspects of blind navigation: system equipment, data sources, guidance algorithms, optimization of related methods, and navigation maps.

RESULTS

The wearable form of assistive devices for the blind has the most research, followed by the handheld type of aids. The RGB data class based on vision sensor is the most common source of navigation environment information data. Object detection based on picture data is also particularly rich among navigation algorithms and associated methods, indicating that computer vision technology has become an important study content in the field of blind navigation. However, research on navigation maps is relatively less.

CONCLUSIONS

In the study and development of assistive equipment for BVIP, there will be an emphasis on prioritizing attributes, such as lightness, portability, and efficiency. In light of the upcoming driverless era, the research focus will be on the development of visual sensors and computer vision technologies that can aid in navigation for the blind.

Network QoS Impact on Spatial Perception through Sensory Substitution in Navigation Systems for Blind and Visually Impaired People

A navigation system for individuals suffering from blindness or visual impairment provides information useful to reach a destination. Although there are different approaches, traditional designs are evolving into distributed systems with low-cost, front-end devices. These devices act as a medium between the user and the environment, encoding the information gathered on the surroundings according to theories on human perceptual and cognitive processes. Ultimately, they are rooted in sensorimotor coupling. The present work searches for temporal constraints due to such human-machine interfaces, which in turn constitute a key design factor for networked solutions. To that end, three tests were conveyed to a group of 25 participants under different delay conditions between motor actions and triggered stimuli. The results show a trade-off between spatial information acquisition and delay degradation, and a learning curve even under impaired sensorimotor coupling.

A Design for Wayfinding: Developing a Mobile Application to Enhance Spatial Orientation at Taipei Metro

Taipei Metro, since its inception in 1996, has become the most important public transport option for commuters and travelers in the metropolitan Taipei area, delivering over two million daily rides. Nevertheless, the interior environment of Taipei Metro has a reputation for being disorienting, especially to the infrequent passengers. By incorporating the methods of behavioral mapping and visibility analysis, this study argues that the occurrence of disorientation is highly dependent on visual properties of Taipei Metro’s interior layout. Specifically, the number of decision-making stops and the visibility conditions of stairs and escalators are found to be particularly influential. To enhance the passengers’ wayfinding experience, a mobile application comprised of two components is proposed. The Route Planner is to advise the passengers to avoid the areas that cause disorientation, while the Navigator, by providing the panoramic views of certain locations, can help the passengers reach their destinations more easily.

VES: A Mixed-Reality Development Platform of Navigation Systems for Blind and Visually Impaired

Herein, we describe the Virtually Enhanced Senses (VES) system, a novel and highly configurable wireless sensor-actuator network conceived as a development and test-bench platform of navigation systems adapted for blind and visually impaired people. It allows to immerse its users into “walkable” purely virtual or mixed environments with simulated sensors and validate navigation system designs prior to prototype development. The haptic, acoustic, and proprioceptive feedback supports state-of-art sensory substitution devices (SSD). In this regard, three SSD were integrated in VES as examples, including the well-known “The vOICe”. Additionally, the data throughput, latency and packet loss of the wireless communication can be controlled to observe its impact in the provided spatial knowledge and resulting mobility and orientation performance. Finally, the system has been validated by testing a combination of two previous visual-acoustic and visual-haptic sensory substitution schemas with 23 normal-sighted subjects. The recorded data includes the output of a “gaze-tracking” utility adapted for SSD.

Inertial Measurement Unit Sensors in Assistive Technologies for Visually Impaired People, a Review

A diverse array of assistive technologies have been developed to help Visually Impaired People (VIP) face many basic daily autonomy challenges. Inertial measurement unit sensors, on the other hand, have been used for navigation, guidance, and localization but especially for full body motion tracking due to their low cost and miniaturization, which have allowed the estimation of kinematic parameters and biomechanical analysis for different field of applications. The aim of this work was to present a comprehensive approach of assistive technologies for VIP that include inertial sensors as input, producing results on the comprehension of technical characteristics of the inertial sensors, the methodologies applied, and their specific role in each developed system. The results show that there are just a few inertial sensor-based systems. However, these sensors provide essential information when combined with optical sensors and radio signals for navigation and special application fields. The discussion includes new avenues of research, missing elements, and usability analysis, since a limitation evidenced in the selected articles is the lack of user-centered designs. Finally, regarding application fields, it has been highlighted that a gap exists in the literature regarding aids for rehabilitation and biomechanical analysis of VIP. Most of the findings are focused on navigation and obstacle detection, and this should be considered for future applications.

Sounds That People with Visual Impairment Want to Experience

This article presents the expectations of visually impaired people with regard to the content of a set of sound exercises planned for implementation, which will mainly enable these people to become familiar with the sounds associated with specific life situations. Consultations were carried out with 20 people with visual impairment, which allowed for the recognition of the needs of these people regarding the sounds with which they wish to become acquainted. The 35 initially proposed sounds were assessed using a five-grade scale. These sounds included those that would be heard in a number of situations in which a person with a visual impairment could potentially be found, both at home and, for example, while in the street or at an office. During the consultations, people with visual impairment usually rated the sounds proposed for inclusion in the set of sound exercises as highly relevant or relevant. In most cases, the assessment was analogous regardless of whether the person had a visual impairment since birth or developed it relatively recently. There were also more than 100 sounds that were proposed for inclusion in the set. The results of the consultation demonstrate how important the information contained in sound is for people with visual impairment.