Preliminary Evaluation of a Wearable Camera-based Collision Warning Device for Blind Individuals

Design of Audio-Augmented-Reality-Based O&M Orientation Training for Visually Impaired Children

Orientation and Mobility training (O&M) is a specific program that teaches people with vision loss to orient themselves and travel safely within certain contexts. State-of-the-art research reveals that people with vision loss expect high-quality O&M training, especially at early ages, but the conventional O&M training methods involve tedious programs and require a high participation of professional trainers. However, there is an insufficient number of excellent trainers. In this work, we first interpret and discuss the relevant research in recent years. Then, we discuss the questionnaires and interviews we conducted with visually impaired people. On the basis of field investigation and related research, we propose the design of a training solution for children to operate and maintain direction based on audio augmented reality. We discuss how, within the perceptible scene created by EasyAR's map-aware framework, we created an AR audio source tracing training that simulates a social scene to strengthen the audiometric identification of the subjects, and then to verify the efficiency and feasibility of this scheme, we implemented the application prototype with the required hardware and software and conducted the subsequential experiments with blindfolded children. We confirm the high usability of the designed approach by analyzing the results of the pilot study. Compared with other orientation training studies, the method we propose makes the whole training process flexible and entertaining. At the same time, this training process does not involve excessive economic costs or require professional skills training, allowing users to undergo training at home or on the sports ground rather than having to go to rehabilitation sites or specified schools. Furthermore, according to the feedback from the experiments, the approach is promising in regard to gamification.

Beyond the Cane: Describing Urban Scenes to Blind People for Mobility Tasks

Blind people face difficulties with independent mobility, impacting employment prospects, social inclusion, and quality of life. Given the advancements in computer vision, with more efficient and effective automated information extraction from visual scenes, it is important to determine what information is worth conveying to blind travelers, especially since people have a limited capacity to receive and process sensory information. We aimed to investigate which objects in a street scene are useful to describe and how those objects should be described. Thirteen cane-using participants, five of whom were early blind, took part in two urban walking experiments. In the first experiment, participants were asked to voice their information needs in the form of questions to the experimenter. In the second experiment, participants were asked to score scene descriptions and navigation instructions, provided by the experimenter, in terms of their usefulness. The descriptions included a variety of objects with various annotations per object. Additionally, we asked participants to rank order the objects and the different descriptions per object in terms of priority and explain why the provided information is or is not useful to them. The results reveal differences between early and late blind participants. Late blind participants requested information more frequently and prioritized information about objects’ locations. Our results illustrate how different factors, such as the level of detail, relative position, and what type of information is provided when describing an object, affected the usefulness of scene descriptions. Participants explained how they (indirectly) used information, but they were frequently unable to explain their ratings. The results distinguish between various types of travel information, underscore the importance of featuring these types at multiple levels of abstraction, and highlight gaps in current understanding of travel information needs. Elucidating the information needs of blind travelers is critical for the development of more useful assistive technologies.

Home-Use Evaluation of a Wearable Collision Warning Device for Individuals With Severe Vision Impairments: A Randomized Clinical Trial

IMPORTANCE

There is scant rigorous evidence about the real-world mobility benefit of electronic mobility aids.

OBJECTIVE

To evaluate the effect of a collision warning device on the number of contacts experienced by blind and visually impaired people in their daily mobility.

DESIGN, SETTING, AND PARTICIPANTS

In this double-masked randomized clinical trial, participants used a collision warning device during their daily mobility over a period of 4 weeks. A volunteer sample of 31 independently mobile individuals with severe visual impairments, including total blindness and peripheral visual field restrictions, who used a long cane or guide dog as their habitual mobility aid completed the study. The study was conducted from January 2018 to December 2019.

INTERVENTIONS

The device automatically detected collision hazards using a chest-mounted video camera. It randomly switched between 2 modes: active mode (intervention condition), where it provided alerts for detected collision threats via 2 vibrotactile wristbands, and silent mode (control condition), where the device still detected collisions but did not provide any warnings to the user. Scene videos along with the collision warning information were recorded by the device. Potential collisions detected by the device were reviewed and scored, including contacts with the hazards, by 2 independent reviewers. Participants and reviewers were masked to the device operation mode.

MAIN OUTCOMES AND MEASURES

Rate of contacts per 100 hazards per hour, compared between the 2 device modes within each participant. Modified intention-to-treat analysis was used.

RESULTS

Of the 31 included participants, 18 (58%) were male, and the median (range) age was 61 (25-73) years. A total of 19 participants (61%) had a visual acuity (VA) of light perception or worse, and 28 (90%) reported a long cane as their habitual mobility aid. The median (interquartile range) number of contacts was lower in the active mode compared with silent mode (9.3 [6.6-14.9] vs 13.8 [6.9-24.3]; difference, 4.5; 95% CI, 1.5-10.7; P < .001). Controlling for demographic characteristics, presence of VA better than light perception, and fall history, the rate of contacts significantly reduced in the active mode compared with the silent mode (β = 0.63; 95% CI, 0.54-0.73; P < .001).

CONCLUSIONS AND RELEVANCE

In this study involving 31 visually impaired participants, the collision warnings were associated with a reduced rate of contacts with obstacles in daily mobility, indicating the potential of the device to augment habitual mobility aids.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT03057496.

Data Acquisition, Processing, and Reduction for Home-Use Trial of a Wearable Video Camera-Based Mobility Aid

Purpose

Evaluating mobility aids in naturalistic conditions across many days is challenging owing to the sheer amount of data and hard-to-control environments. For a wearable video camera-based collision warning device, we present the methodology for acquisition, reduction, review, and coding of video data for quantitative analyses of mobility outcomes in blind and visually impaired participants.

Methods

Scene videos along with collision detection information were obtained from a chest-mounted collision warning device during daily use of the device. The recorded data were analyzed after use. Collision risk events flagged by the device were manually reviewed and coded using a detailed annotation protocol by two independent masked reviewers. Data reduction was achieved by predicting agreements between reviewers based on a machine learning algorithm. Thus, only those events for which disagreements were predicted would be reviewed by the second reviewer. Finally, the ultimate disagreements were resolved via consensus, and mobility-related outcome measures such as percentage of body contacts were obtained.

Results

There were 38 hours of device use from 10 participants that were reviewed by both reviewers, with an agreement level of 0.66 for body contacts. The machine learning algorithm trained on 2714 events correctly predicted 90.5% of disagreements. For another 1943 events, the trained model successfully predicted 82% of disagreements, resulting in 81% data reduction.

Conclusions

The feasibility of mobility aid evaluation based on a large volume of naturalistic data is demonstrated. Machine learning–based disagreement prediction can lead to data reduction.

Translational Relevance

These methods provide a template for determining the real-world benefit of a mobility aid.

A Hardware-Friendly Optical Flow-Based Time-to-Collision Estimation Algorithm

This work proposes a hardware-friendly, dense optical flow-based Time-to-Collision (TTC) estimation algorithm intended to be deployed on smart video sensors for collision avoidance. The algorithm optimized for hardware first extracts biological visual motion features (motion energies), and then utilizes a Random Forests regressor to predict robust and dense optical flow. Finally, TTC is reliably estimated from the divergence of the optical flow field. This algorithm involves only feed-forward data flows with simple pixel-level operations, and hence has inherent parallelism for hardware acceleration. The algorithm offers good scalability, allowing for flexible tradeoffs among estimation accuracy, processing speed and hardware resource. Experimental evaluation shows that the accuracy of the optical flow estimation is improved due to the use of Random Forests compared to existing voting-based approaches. Furthermore, results show that estimated TTC values by the algorithm closely follow the ground truth. The specifics of the hardware design to implement the algorithm on a real-time embedded system are laid out.