Smartphone-based computer vision travelling aids for blind and visually impaired individuals: A systematic review

Range sensor-based assistive technology solutions for people with visual impairment: a review

PURPOSE

There are about 2.2 billion people with visual impairment all over the world. In order to improve the quality of life for visually impaired people, various devices have been developed that support the users to make them capable of performing everyday work. There have been considerable advancements in the development of assistive devices over the last few decades. This work aims to review the research work of past decade to explore the technologies used in the assistive devices for the mobility of the people with visual impairment. It focuses on range sensors based (RSB) solutions and to present a comprehensive comparison for researchers to improve the quality of the assistive devices.

METHODS

In order to improve life quality of about 2.2 billion people with visual impairment in the world, various assistive devices have been developed. This work aims to review the research work of past decade to explore the technologies used in the assistive devices for the mobility of the people with visual impairment. It focuses on RSB solutions and their comparison.

RESULTS

Various devices developed or the people with visual impairment, in last decade are described along with their workings. Authors also introduced, their own, newly developed assistive device for the people with visual impairment. The feedback from the people with visual impairment, about assistive technology, is also included in the paper.

CONCLUSIONS

This study will benefit researchers developing assistive devices to help for the mobility of the people with visual impairment. Through user feedback and evaluation of the assistive devices, the authors have come to the conclusion that performance, weight and cost of the assistive devices are always important considerations to make the assistive devices more popular among their users.Implication for rehabilitationAlthough assistive device cannot rehabilitate a visually impaired person, range sensor-based assistive device may have following implications:•Use of assistive device is growing, and performance, weight and cost of assistive devices are always important considerations so that proposed technology solution should widely be accepted and adopted by the visually impaired people.•The adaptability and acceptability of an assistive device by the visually impaired people must be considered during design phase.•Proposed assistive technological solutions should meet all the needed functions.•Performance of these devices should be assessed in application context so that these devices may help the visually impaired to perform their tasks independently.•At the end, a new light weight and low-cost device developed by authors, is also given, that could be used to assist the visually impaired people to move independently.

Applied Artificial Intelligence in Healthcare: A Review of Computer Vision Technology Application in Hospital Settings

Computer vision (CV), a type of artificial intelligence (AI) that uses digital videos or a sequence of images to recognize content, has been used extensively across industries in recent years. However, in the healthcare industry, its applications are limited by factors like privacy, safety, and ethical concerns. Despite this, CV has the potential to improve patient monitoring, and system efficiencies, while reducing workload. In contrast to previous reviews, we focus on the end-user applications of CV. First, we briefly review and categorize CV applications in other industries (job enhancement, surveillance and monitoring, automation, and augmented reality). We then review the developments of CV in the hospital setting, outpatient, and community settings. The recent advances in monitoring delirium, pain and sedation, patient deterioration, mechanical ventilation, mobility, patient safety, surgical applications, quantification of workload in the hospital, and monitoring for patient events outside the hospital are highlighted. To identify opportunities for future applications, we also completed journey mapping at different system levels. Lastly, we discuss the privacy, safety, and ethical considerations associated with CV and outline processes in algorithm development and testing that limit CV expansion in healthcare. This comprehensive review highlights CV applications and ideas for its expanded use in healthcare.

An exploration of smartphone use by, and support for people with vision impairment: a scoping review

PURPOSE

Smartphones have become a core piece of assistive technology (AT) for people with vision impairment (PVI) around the world. This scoping review sought to provide a comprehensive picture of the current evidence base of smartphones for PVI.

METHODS

Seven electronic databases (CINAHL, Cochrane Library, EMBASE, IEEE Xplore, Scopus, PubMed and Web of Science) were searched for papers published from 2007 to 2021. Peer-reviewed articles published in English which discussed smartphones use by PVI; smartphone technologies designed for PVI or training and learning support on the use of smartphones were included.

RESULTS

There were 16,899 records retrieved and 65 articles were included in this review. The majority (48%) of the papers focussed on developing better interfaces and Apps for PVI. Contrastingly, there was a paucity of papers (5%) discussing training or learning support for PVI to use smartphones and Apps effectively, even though it was highlighted to be important. Proper training will ensure that PVI can use this everyday technology as an AT to increase participation, enhance independence and improve quality of life overall.

CONCLUSIONS

The findings highlighted that smartphones and Apps can be used as effective and affordable AT by PVI. The many recent developments and research interest in smartphone technologies can further support its use. However, good training and learning support on the use of smartphones and Apps by PVI, is lacking. Future research should focus on the development, provision and evaluation of evidence based tailored training and support, especially in low- and middle-income countries. Implications for rehabilitationThere is a need for more training and learning support for people with vision impairment (PVI) on the use of smartphones and Apps.Individualized and a graded approach to training has been recommended for PVI to learn to use smartphones.When supporting or training people to use smartphones, the person's level of vision impairment as well as their age, are important considerations.Health professionals should be cognizant of the steep learning curve that some PVI may experience when using smartphones and Apps, especially when they switch from a phone with physical buttons to touchscreen.Certain smartphones features are useful to particular vision loss conditions. For example, zoom and magnification are helpful for those with low vision but text input and output, and commands using speech (e.g., Siri and TalkBack) are useful for those who are blind.

Impact of Apps as Assistive Devices for Visually Impaired Persons

The pervasiveness of mobile devices and other associated technologies has affected all aspects of our daily lives. People with visual impairments are no exception, as they increasingly tend to rely on mobile apps for assistance with various visual tasks in daily life. Compared to dedicated visual aids, mobile apps offer advantages such as affordability, versatility, portability, and ubiquity. We have surveyed hundreds of mobile apps of potential interest to people with vision impairments, either released as special assistive apps claiming to help in tasks such as text or object recognition (n = 68), digital accessibility (n = 84), navigation (n = 44), and remote sighted service (n = 4), among others, or marketed as general camera magnification apps that can be used for visual assistance (n = 77). While assistive apps as a whole received positive feedback from visually impaired users, as reported in various studies, evaluations of the usability of every app were typically limited to user reviews, which are often not scientifically informative. Rigorous evaluation studies on the effect of vision assistance apps on daily task performance and quality of life are relatively rare. Moreover, evaluation criteria are difficult to establish, given the heterogeneity of the visual tasks and visual needs of the users. In addition to surveying literature on vision assistance apps, this review discusses the feasibility and necessity of conducting scientific research to understand visual needs and methods to evaluate real-world benefits.

Vision Transformer Customized for Environment Detection and Collision Prediction to Assist the Visually Impaired

This paper presents a system that utilizes vision transformers and multimodal feedback modules to facilitate navigation and collision avoidance for the visually impaired. By implementing vision transformers, the system achieves accurate object detection, enabling the real-time identification of objects in front of the user. Semantic segmentation and the algorithms developed in this work provide a means to generate a trajectory vector of all identified objects from the vision transformer and to detect objects that are likely to intersect with the user's walking path. Audio and vibrotactile feedback modules are integrated to convey collision warning through multimodal feedback. The dataset used to create the model was captured from both indoor and outdoor settings under different weather conditions at different times across multiple days, resulting in 27,867 photos consisting of 24 different classes. Classification results showed good performance (95% accuracy), supporting the efficacy and reliability of the proposed model. The design and control methods of the multimodal feedback modules for collision warning are also presented, while the experimental validation concerning their usability and efficiency stands as an upcoming endeavor. The demonstrated performance of the vision transformer and the presented algorithms in conjunction with the multimodal feedback modules show promising prospects of its feasibility and applicability for the navigation assistance of individuals with vision impairment.

A Multi-Sensory Guidance System for the Visually Impaired Using YOLO and ORB-SLAM

Guidance systems for visually impaired persons have become a popular topic in recent years. Existing guidance systems on the market typically utilize auxiliary tools and methods such as GPS, UWB, or a simple white cane that exploits the user’s single tactile or auditory sense. These guidance methodologies can be inadequate in a complex indoor environment. This paper proposes a multi-sensory guidance system for the visually impaired that can provide tactile and auditory advice using ORB-SLAM and YOLO techniques. Based on an RGB-D camera, the local obstacle avoidance system is realized at the tactile level through point cloud filtering that can inform the user via a vibrating motor. Our proposed method can generate a dense navigation map to implement global obstacle avoidance and path planning for the user through the coordinate transformation. Real-time target detection and a voice-prompt system based on YOLO are also incorporated at the auditory level. We implemented the proposed system as a smart cane. Experiments are performed using four different test scenarios. Experimental results demonstrate that the impediments in the walking path can be reliably located and classified in real-time. Our proposed system can function as a capable auxiliary to help visually impaired people navigate securely by integrating YOLO with ORB-SLAM.

Ultrasonic Sound Guide System with Eyeglass Device for the Visually Impaired

The ultrasonic sound guide system presents the audio broadcasting system based on the inaudible ultrasonic sound to assist the indoor and outdoor navigation of the visually impaired. The transmitters are placed at the point of interest to propagate the frequency modulated voice signal in ultrasonic sound range. The dual channel receiver device is carried by the visually impaired person in the form of eyeglasses to receive the ultrasonic sound for the voice signal via demodulation. Since the ultrasonic sound demonstrates the acoustic properties, the velocity, directivity, attenuation, and superposition of ultrasonic sound provide the acoustic clue to the user for localizing the multiple transmitter positions by binaural localization capability. The visually impaired hear the designated voice signal and follow the signal attributions to arrive at the specific location. Due to the low microphone gain from side addressing, the time delay between the receiver channels demonstrates the high variance and high bias in end directions. However, the perception experiment shows the further prediction accuracy in end directions as compared to the center direction outcomes. The overall evaluations show the precise directional prediction for narrow- and wide-angle situations. The ultrasonic sound guide system is a useful device to localize places in the near field without touching braille.

Traveling more Independently: A Study on the Diverse Needs and Challenges of People with Visual or Mobility Impairments in Unfamiliar Indoor Environments

It is much more difficult for people with visual or mobility impairments to prepare for a trip or visit unfamiliar places than it is for people without disabilities. In addition to the usual travel arrangements, one needs to know if the various parts of the travel chain are accessible. To the best of our knowledge, there is no previous work that examines in depth travel behaviour for indoor environments for both trip planning and execution, highlighting the special needs of people with low vision, blindness or mobility impairments. In this paper, we present a survey with 125 participants with blindness, low vision and mobility impairments. We investigate how mobile they are, what strategies they use to prepare a journey to an unknown building, how they orient themselves there and what materials they use. For all three groups, our results provide insights into the problem space of the specific information needs when planning and executing a trip. We found that most of our participants have specific mobility problems depending on their disability. Feedback from the participants reveals there is a large information gap, especially for orientation in buildings, regarding availability of high-quality digital, tactile and printable indoor maps, accessibility of buildings and mobility supporting systems. In particular, there is a lack of available and high-quality indoor maps. Our analysis also points out that the specific needs differ for the three groups. Besides the expected between-group differences, also large in-group differences can be found. The current paper is an expanded version of [18] augmented by data of people with mobility impairments.

A Systematic Review of Urban Navigation Systems for Visually Impaired People

Blind and Visually impaired people (BVIP) face a range of practical difficulties when undertaking outdoor journeys as pedestrians. Over the past decade, a variety of assistive devices have been researched and developed to help BVIP navigate more safely and independently. In addition, research in overlapping domains are addressing the problem of automatic environment interpretation using computer vision and machine learning, particularly deep learning, approaches. Our aim in this article is to present a comprehensive review of research directly in, or relevant to, assistive outdoor navigation for BVIP. We breakdown the navigation area into a series of navigation phases and tasks. We then use this structure for our systematic review of research, analysing articles, methods, datasets and current limitations by task. We also provide an overview of commercial and non-commercial navigation applications targeted at BVIP. Our review contributes to the body of knowledge by providing a comprehensive, structured analysis of work in the domain, including the state of the art, and guidance on future directions. It will support both researchers and other stakeholders in the domain to establish an informed view of research progress.

Ability of Head-Mounted Display Technology to Improve Mobility in People With Low Vision: A Systematic Review

Purpose

The purpose of this study was to undertake a systematic literature review on how vision enhancements, implemented using head-mounted displays (HMDs), can improve mobility, orientation, and associated aspects of visual function in people with low vision.

Methods

The databases Medline, Chinl, Scopus, and Web of Science were searched for potentially relevant studies. Publications from all years until November 2018 were identified based on predefined inclusion and exclusion criteria. The data were tabulated and synthesized to produce a systematic review.

Results

The search identified 28 relevant papers describing the performance of vision enhancement techniques on mobility and associated visual tasks. Simplifying visual scenes improved obstacle detection and object recognition but decreased walking speed. Minification techniques increased the size of the visual field by 3 to 5 times and improved visual search performance. However, the impact of minification on mobility has not been studied extensively. Clinical trials with commercially available devices recorded poor results relative to conventional aids.

Conclusions

The effects of current vision enhancements using HMDs are mixed. They appear to reduce mobility efficiency but improved obstacle detection and object recognition. The review highlights the lack of controlled studies with robust study designs. To support the evidence base, well-designed trials with larger sample sizes that represent different types of impairments and real-life scenarios are required. Future work should focus on identifying the needs of people with different types of vision impairment and providing targeted enhancements.

Translational Relevance

This literature review examines the evidence regarding the ability of HMD technology to improve mobility in people with sight loss.