Voice Navigation Created by VIP Improves Spatial Performance in People with Impaired Vision

The difficulty associated with spatial navigation is one of the main obstacles to independent living for visually impaired people. With a lack of visual feedback, visually impaired people must identify information from the external environment through other sense organs. This study employed an observational survey to assess voice navigation version A, created by visually impaired people, and voice navigation version B, created by non-visually impaired people. Thirty-two simulated visually impaired people were assigned to conduct task assessments of voice navigation version A and version B. For mission 1, the mean completion rate is 0.988 ± 0.049 (version A); the mean error rate is 0.125 ± 0.182 (version A). For mission 2, the mean completion rate is 0.953 ± 0.148 (version A); the mean error rate is 0.094 ± 0.198 (version A). The assessment results concluded that version A has a higher completion rate (p = 0.001) and a lower error rate (p = 0.001). In the assessment of subjective satisfaction, all the indicators regarding the impression of navigation directives in version A were significantly superior to those indicators in version B. It appears that version A has a different logic of framing than version B. In future applications, a voice navigation version shall be built, according to the way visually impaired people think, because it will facilitate the direction guide when there is a lack of visual feedback.

Beyond the Cane: Describing Urban Scenes to Blind People for Mobility Tasks

Blind people face difficulties with independent mobility, impacting employment prospects, social inclusion, and quality of life. Given the advancements in computer vision, with more efficient and effective automated information extraction from visual scenes, it is important to determine what information is worth conveying to blind travelers, especially since people have a limited capacity to receive and process sensory information. We aimed to investigate which objects in a street scene are useful to describe and how those objects should be described. Thirteen cane-using participants, five of whom were early blind, took part in two urban walking experiments. In the first experiment, participants were asked to voice their information needs in the form of questions to the experimenter. In the second experiment, participants were asked to score scene descriptions and navigation instructions, provided by the experimenter, in terms of their usefulness. The descriptions included a variety of objects with various annotations per object. Additionally, we asked participants to rank order the objects and the different descriptions per object in terms of priority and explain why the provided information is or is not useful to them. The results reveal differences between early and late blind participants. Late blind participants requested information more frequently and prioritized information about objects’ locations. Our results illustrate how different factors, such as the level of detail, relative position, and what type of information is provided when describing an object, affected the usefulness of scene descriptions. Participants explained how they (indirectly) used information, but they were frequently unable to explain their ratings. The results distinguish between various types of travel information, underscore the importance of featuring these types at multiple levels of abstraction, and highlight gaps in current understanding of travel information needs. Elucidating the information needs of blind travelers is critical for the development of more useful assistive technologies.

Pharmaceutical mobile application for visually-impaired people in Thailand: development and implementation

BACKGROUND

Most mobile pharmaceutical applications produced for people with visual disabilities in Thailand fail to meet the required standard due to poor-quality regulations, defective design, lack of user support and impracticality; as a result, visually-impaired people are unable to use them. This research is motivated by the limited use of this technology in primary medical services and its aim is to enable people with disabilities to access effective digital health information. The research objective is to analyse, design and develop a mobile pharmaceutical application with functions that are appropriate for visually-impaired users, and test its usability.

RESULTS

Based on the design and development of the application, it contained five necessary functions. When testing the usability and users' satisfaction, it was found that the input or fill of information in the application was of low usability. According to the test results, the medicinal database function was missing 71 times and the voice command function was missing 34 times. Based on users' satisfaction results, users who had the highest level of usage gave higher average scores to users' attitude, users' confidence, user interface and system performance than those with lower levels of usage. The scores of both groups were found to be the same when discussing the implementation of the development.

CONCLUSIONS

This mobile application, which was developed based on the use of smart technology, will play an important role in supporting visually-impaired people in Thailand by enhancing the efficacy of self-care. The design and development of the application will ensure the suitability of many functions for visually-impaired users. However, despite the high functional capacity of the application, the gap in healthcare services between the general public and disabled groups will still exist if users have inadequate IT skills.

Experimental Analysis of a Spatialised Audio Interface for People with Visual Impairments

Sound perception is a fundamental skill for many people with severe sight impairments. The research presented in this article is part of an ongoing project with the aim to create a mobile guidance aid to help people with vision impairments find objects within an unknown indoor environment. This system requires an effective non-visual interface and uses bone-conduction headphones to transmit audio instructions to the user. It has been implemented and tested with spatialised audio cues, which convey the direction of a predefined target in 3D space. We present an in-depth evaluation of the audio interface with several experiments that involve a large number of participants, both blindfolded and with actual visual impairments, and analyse the pros and cons of our design choices. In addition to producing results comparable to the state-of-the-art, we found that Fitts’s Law (a predictive model for human movement) provides a suitable metric that can be used to improve and refine the quality of the audio interface in future mobile navigation aids.

Use of an Indoor Navigation System by Sighted and Blind Travelers

This article first reviews the pros and cons of current accessible indoor navigation systems and then describes a study using commercial smart devices to navigate routes through a complex building. Our interest was in comparing performance when using real-time narrative descriptions (system-aided condition) vs. a memory-based condition where the same narrative information was only provided to users from the route's origin. We tested two groups of blind and visually impaired (BVI) users, including people above and below 60 years of age, as well as a third sighted control group. Evaluating older BVI participants is important, as the majority of vision loss is age-related, yet navigation performance using access technology is rarely studied with this demographic. Behavioral results demonstrated that access to real-time (system-aided) information led to better navigation accuracy and greater confidence by blind users compared to the information-matched memory condition. Performance for blind participants over 60 years old was nearly identical with their younger peers—an important outcome supporting the efficacy of using navigational technologies by this fast-growing population. Route completion accuracy and requests for assistance did not reliably differ between blind and sighted participants when using the system, suggesting that access to narrative route information led to functionally equivalent navigation behavior, irrespective of visual status. Survey results revealed strong user support for real-time information and provided important guidance for future interface refinements.

Navigation Systems for the Blind and Visually Impaired: Past Work, Challenges, and Open Problems

Over the last decades, the development of navigation devices capable of guiding the blind through indoor and/or outdoor scenarios has remained a challenge. In this context, this paper’s objective is to provide an updated, holistic view of this research, in order to enable developers to exploit the different aspects of its multidisciplinary nature. To that end, previous solutions will be briefly described and analyzed from a historical perspective, from the first “Electronic Travel Aids” and early research on sensory substitution or indoor/outdoor positioning, to recent systems based on artificial vision. Thereafter, user-centered design fundamentals are addressed, including the main points of criticism of previous approaches. Finally, several technological achievements are highlighted as they could underpin future feasible designs. In line with this, smartphones and wearables with built-in cameras will then be indicated as potentially feasible options with which to support state-of-art computer vision solutions, thus allowing for both the positioning and monitoring of the user’s surrounding area. These functionalities could then be further boosted by means of remote resources, leading to cloud computing schemas or even remote sensing via urban infrastructure.

Getting Your Hands Dirty Outside the Lab: A Practical Primer for Conducting Wearable Vibrotactile Haptics Research

As haptics have become an ingrained part of our wearable experience, particularly through phones, smartwatches, and fitness trackers, significant research effort has been conducted to find new ways of using wearable haptics to convey information, especially while we are on-the-go. In this paper, instead of focusing on aspects of haptic information design, such as tacton encoding methods, actuators, and technical fabrication of devices, we address the more general recurring issues and "gotchas" that arise when moving from core haptic perceptual studies and in-lab wearable experiments to real world testing of wearable vibrotactile haptic systems. We summarize key issues for practitioners to take into account when designing and carrying out in-the-wild wearable haptic user studies, as well as for user studies in a lab environment that seek to simulate real-world conditions. We include not only examples from published work and commercial sources, but also hard-won illustrative examples derived from issues and failures from our own haptic studies. By providing a broad-based, accessible overview of recurring issues, we expect that both novice and experienced haptic researchers will find suggestions that will improve their own mobile wearable haptic studies.

Wearable Travel Aid for Environment Perception and Navigation of Visually Impaired People

Assistive devices for visually impaired people (VIP) which support daily traveling and improve social inclusion are developing fast. Most of them try to solve the problem of navigation or obstacle avoidance, and other works focus on helping VIP to recognize their surrounding objects. However, very few of them couple both capabilities (i.e., navigation and recognition). Aiming at the above needs, this paper presents a wearable assistive device that allows VIP to (i) navigate safely and quickly in unfamiliar environment, and (ii) to recognize the objects in both indoor and outdoor environments. The device consists of a consumer Red, Green, Blue and Depth (RGB-D) camera and an Inertial Measurement Unit (IMU), which are mounted on a pair of eyeglasses, and a smartphone. The device leverages the ground height continuity among adjacent image frames to segment the ground accurately and rapidly, and then search the moving direction according to the ground. A lightweight Convolutional Neural Network (CNN)-based object recognition system is developed and deployed on the smartphone to increase the perception ability of VIP and promote the navigation system. It can provide the semantic information of surroundings, such as the categories, locations, and orientations of objects. Human–machine interaction is performed through audio module (a beeping sound for obstacle alert, speech recognition for understanding the user commands, and speech synthesis for expressing semantic information of surroundings). We evaluated the performance of the proposed system through many experiments conducted in both indoor and outdoor scenarios, demonstrating the efficiency and safety of the proposed assistive system.

Personal Guidance System for People with Visual Impairment: A Comparison of Spatial Displays for Route Guidance

This article reports on a study of route guidance using a navigation system that receives location information from a Global Positioning System receiver. Fifteen visually impaired participants traveled along 50-meter (about 164-foot) paths in each of five conditions that were defined by the type of display interface used. One of the virtual displays—virtual speech—led to the shortest travel times and the highest subjective ratings, despite concerns about the use of headphones.

Vision-based Mobile Indoor Assistive Navigation Aid for Blind People

This paper presents a new holistic vision-based mobile assistive navigation system to help blind and visually impaired people with indoor independent travel. The system detects dynamic obstacles and adjusts path planning in real-time to improve navigation safety. First, we develop an indoor map editor to parse geometric information from architectural models and generate a semantic map consisting of a global 2D traversable grid map layer and context-aware layers. By leveraging the visual positioning service (VPS) within the Google Tango device, we design a map alignment algorithm to bridge the visual area description file (ADF) and semantic map to achieve semantic localization. Using the on-board RGB-D camera, we develop an efficient obstacle detection and avoidance approach based on a time-stamped map Kalman filter (TSM-KF) algorithm. A multi-modal human-machine interface (HMI) is designed with speech-audio interaction and robust haptic interaction through an electronic SmartCane. Finally, field experiments by blindfolded and blind subjects demonstrate that the proposed system provides an effective tool to help blind individuals with indoor navigation and wayfinding.

User Interface Preferences in the Design of a Camera-Based Navigation and Wayfinding Aid

Introduction

Development of a sensing device that can provide a sufficient perceptual substrate for persons with visual impairments to orient themselves and travel confidently has been a persistent rehabilitation technology goal, with the user interface posing a significant challenge. In the study presented here, we enlist the advice and ideas of individuals who are blind with respect to this challenge, for an envisioned camera-based aid to navigation and wayfinding.

Methods

We administered a short questionnaire about user preferences and needs for such a device to a sample of 10 well-educated, employed (or retired) visually impaired participants with light perception or less, who were familiar and comfortable with assistive technology. Generally, the items were rankings of relative priority.

Results

Participants preferred speech as a communications medium for navigating the environment; preferred controlling the auditory display by querying the system rather than interacting via a menu or receiving a stream of continuous speech; and preferred providing input to the system through a keypad rather than through a voice recognition system. Architectural features such as doors and stairs were ranked the most important environmental objects to be located with such a device (over furniture, persons, personal items, and even text signs).

Discussion

Our sample reported a desire for devices that can guide them to architectural features of their environment. They appear to prefer device interfaces that give them control, and would rather query a system than interact with a menu. They prefer unobtrusive input on a device via keypad rather than through voice recognition.

Implications for practitioners

Designers of camera-based navigation devices may wish to consider the preferences of our sample by incorporating a query-based interface with simple keypad input and speech output, and to include in their object recognition efforts the goal of identifying architectural features that are significant to users who are blind in navigation.

Finding a Target with an Accessible Global Positioning System

This article presents two target-location experiments. In the first experiment, 19 participants located a 25-foot chalk circle 93% of the time with a Global Positioning System (GPS) compared to 12% of the time without it. In a single-subject follow-up experiment, the participant came within 1 foot of the target on all GPS trials. Target-location techniques are described.

Accessible GPS: Reorientation and Target Location among users with Visual Impairments

This article presents the results of two single-subject experiments that were designed to determine consumers’ ability to use a BrailleNote GPS. The participants decreased their mean orientation time from 6 minutes to 45 seconds and increased their target- location efficiency fourfold with BGPS than without BGPS. Additional results and implications for the field are presented.

Users’ Evaluations of Four Electronic Travel Aids Aimed at Navigation for Persons who are Visually Impaired

Eighteen persons with visual impairments evaluated four systematically selected navigation systems. Their performance on 11 tasks, provided ratings, satisfaction on seven aspects of the assistive devices, and explanatory comments show generally valuable functionality and usability, as well as individual marked preferences for various features of different navigation systems.

Assessment of Indoor Route-finding Technology for People who are Visually Impaired

This study investigated navigation with route instructions generated by digital-map software and synthetic speech. The participants, either visually impaired or sighted wearing blindfolds, successfully located rooms in an unfamiliar building. Users with visual impairments demonstrated better route-finding performance when the technology provided distance information in the number of steps, rather than in the walking time or number of feet.

Nonvisual Route following with Guidance from a Simple Haptic or Auditory Display

A path-following experiment, using a global positioning system, was conducted with participants who were legally blind. On- and off-course confirmations were delivered by either a vibrotactile or an audio stimulus. These simple binary cues were sufficient for guidance and point to the need to offer output options for guidance systems for people who are visually impaired.

User-centered design of a depth data based obstacle detection and avoidance system for the visually impaired

The development of a novel depth-data based real-time obstacle detection and avoidance application for visually impaired (VI) individuals to assist them in navigating independently in indoors environments is presented in this paper. The application utilizes a mainstream, computationally efficient mobile device as the development platform in order to create a solution which not only is aesthetically appealing, cost-effective, lightweight and portable but also provides real-time performance and freedom from network connectivity constraints. To alleviate usability problems, a user-centered design approach has been adopted wherein semi-structured interviews with VI individuals in the local context were conducted to understand their micro-navigation practices, challenges and needs. The invaluable insights gained from these interviews have not only informed the design of our system but would also benefit other researchers developing similar applications. The resulting system design along with a detailed description of its obstacle detection and unique multimodal feedback generation modules has been provided. We plan to iteratively develop and test the initial prototype of the system with the end users to resolve any usability issues and better adapt it to their needs.

Transit apps for people with brain injury and other cognitive disabilities: the state of the art

Individuals with cognitive disability have difficulty using public transit, but little research is directed toward this issue. Recent studies suggest that smartphones may be useful assistive devices in this context. Current objectives were to (1) survey research into difficulties people with cognitive disabilities experience when using public transit, (2) survey the current state of the art of transit and personal navigation applications (apps) and features, (3) recommend best existing transit apps for people with cognitive disability, and (4) recommend the best designs and features of these apps to developers of future transit apps. Potentially useful features were found in four categories: Transit apps for (1) individuals with cognitive disabilities and (2) healthy individuals, and personal navigation apps for (3) individuals with cognitive disabilities and (4) healthy individuals. A total of 159 apps were examined, but only seven were found specific to public transit for cognitive disability. By comparing research recommendations and currently available features, we identified several unmet needs. We note that there appears to be a shortage of apps for this population-function but that there is good research in the area and it is well suited to inform app development.

Is there a future for sensory substitution outside academic laboratories?

Sensory substitution devices (SSDs) have been developed with the ultimate purpose of supporting sensory deprived individuals in their daily activities. However, more than forty years after their first appearance in the scientific literature, SSDs still remain more common in research laboratories than in the daily life of people with sensory deprivation. Here, we seek to identify the reasons behind the limited diffusion of SSDs among the blind community by discussing the ergonomic, neurocognitive and psychosocial issues potentially associated with the use of these systems. We stress that these issues should be considered together when developing future devices or improving existing ones. We provide some examples of how to achieve this by adopting a multidisciplinary and participatory approach. These efforts would contribute not solely to address fundamental theoretical research questions, but also to better understand the everyday needs of blind people and eventually promote the use of SSDs outside laboratories.

FingerSight: Fingertip Haptic Sensing of the Visual Environment

We present a novel device mounted on the fingertip for acquiring and transmitting visual information through haptic channels. In contrast to previous systems in which the user interrogates an intermediate representation of visual information, such as a tactile display representing a camera generated image, our device uses a fingertip-mounted camera and haptic stimulator to allow the user to feel visual features directly from the environment. Visual features ranging from simple intensity or oriented edges to more complex information identified automatically about objects in the environment may be translated in this manner into haptic stimulation of the finger. Experiments using an initial prototype to trace a continuous straight edge have quantified the user's ability to discriminate the angle of the edge, a potentially useful feature for higher levels analysis of the visual scene.

What verbal orientation information do blind and partially sighted people need to find their way around? A study of everyday navigation strategies in people with impaired vision

For wayfinding at museums, galleries, and heritage sites, the majority of people who are blind or partially sighted favour verbal directions over tactile maps. Those who provide such directions are usually fully sighted. Can we assume that details considered important by those with vision are equally helpful to those without? This article presents a study comparing descriptions of familiar routes recalled by fully sighted, partially sighted, and blind participants, highlighting the differences in navigation strategies. All participants cited landmarks, but those with impaired vision also detailed significantly more objects, or ‘milestones’, en route. In the absence of such milestones, they relied on metrics that were often environmental or body-based. Distances were precisely specified, rather than ‘vague’. Landmarks and milestones were accessed multimodally by blind and partially sighted participants, or a combination of landmarks and milestones were sought for corroboration. In contrast, fully sighted participants reported predominantly unimodal information, that is, visual. The findings may have practical implications for improving the type of verbal orientation information provided to assist blind or partially sighted people.

Representing Colour Through Hearing and Touch in Sensory Substitution Devices

Visual sensory substitution devices (SSDs) allow visually-deprived individuals to navigate and recognise the ‘visual world’; SSDs also provide opportunities for psychologists to study modality-independent theories of perception. At present most research has focused on encoding greyscale vision. However at the low spatial resolutions received by SSD users, colour information enhances object-ground segmentation, and provides more stable cues for scene and object recognition. Many attempts have been made to encode colour information in tactile or auditory modalities, but many of these studies exist in isolation. This review brings together a wide variety of tactile and auditory approaches to representing colour. We examine how each device constructs ‘colour’ relative to veridical human colour perception and report previous experiments using these devices. Theoretical approaches to encoding and transferring colour information through sound or touch are discussed for future devices, covering alternative stimulation approaches, perceptually distinct dimensions and intuitive cross-modal correspondences.

A wayfinding pilot study: the use of the Intelligent Public Vehicle by people with visual impairment

This pilot study on wayfinding for people with visual impairments concerns the viability of the Individual Public Vehicle (IPV). The results showed that the participants positively evaluated this new wayfinding aid because it requires little attention and little physical effort, and gives satisfaction. Moreover, as far as identifying the location of the IPV is concerned, an auditory message activated by users is judged less stressful. The IPV seems to be adapted to safe travel.

An oral tactile interface for blind navigation

An oral tactile interface was designed and evaluated to provide directional cues through the tactile channel, which may be utilized by a blind traveler to obtain directional guidance in outdoor navigation. The device was implemented as a mouthpiece with a microfabricated electrotactile display on top for tactile presentation onto the roof of the mouth and a tongue touch keypad at the bottom for simultaneous operation by the tongue. An experimental system allowed a user to communicate with a computer tactilely by using the oral interface. Directional cues were presented to the user as line or arrow patterns with four moving directions (leftward, rightward, forward, and backward). Electrotactile presentation on the roof of the mouth was evaluated in experiments of threshold measurement and identification of directional cues. Experimental results from six human subjects showed that the roof of the mouth required stimulation intensities around 15 V for threshold sensation, and around 25-30 V for comfortable and well-perceived stimulation. Furthermore, identification of leftward or rightward movements was highly accurate while performance on forward or backward moving patterns was mixed and varied considerably among subjects.