A Review of Technologies and Techniques for Indoor Navigation Systems for the Visually Impaired

A Comprehensive Survey on Emerging Assistive Technologies for Visually Impaired Persons: Lighting the Path with Visible Light Communications and Artificial Intelligence Innovations

In the context in which severe visual impairment significantly affects human life, this article emphasizes the potential of Artificial Intelligence (AI) and Visible Light Communications (VLC) in developing future assistive technologies. Toward this path, the article summarizes the features of some commercial assistance solutions, and debates the characteristics of VLC and AI, emphasizing their compatibility with blind individuals' needs. Additionally, this work highlights the AI potential in the efficient early detection of eye diseases. This article also reviews the existing work oriented toward VLC integration in blind persons' assistive applications, showing the existing progress and emphasizing the high potential associated with VLC use. In the end, this work provides a roadmap toward the development of an integrated AI-based VLC assistance solution for visually impaired people, pointing out the high potential and some of the steps to follow. As far as we know, this is the first comprehensive work which focuses on the integration of AI and VLC technologies in visually impaired persons' assistance domain.

Tape-Shaped, Multiscale, and Continuous-Readable Fiducial Marker for Indoor Navigation and Localization Systems

The present study proposes a fiducial marker for location systems that uses computer vision. The marker employs a set of tape-shaped markers that facilitate their positioning in the environment, allowing continuous reading to cover the entire perimeter of the environment and making it possible to minimize interruptions in the location service. Because the marker is present throughout the perimeter of the environment, it presents hierarchical coding patterns that allow it to be robust against multiple detection scales. We implemented an application to help the user generate the markers with a floor plan image. We conducted two types of tests, one in a 3D simulation environment and one in a real-life environment with a smartphone. The tests made it possible to measure the performance of the tape-shaped marker with readings at multiple distances compared to ArUco, QRCode, and STag with detections at distances of 10 to 0.5 m. The localization tests in the 3D environment analyzed the time of marker detection during the journey from one room to another in positioning conditions (A) with the markers positioned at the baseboard of the wall, (B) with the markers positioned at camera height, and (C) with the marker positioned on the floor. The localization tests in real conditions allowed us to measure the time of detections in favorable conditions of detections, demonstrating that the tape-shaped-marker-detection algorithm is not yet robust against blurring but is robust against lighting variations, difficult angle displays, and partial occlusions. In both test environments, the marker allowed for detection at multiple scales, confirming its functionality.

A review of sonification solutions in assistive systems for visually impaired people

PURPOSE

Visually impaired people (VIP) find it challenging to understand and gain awareness of their surroundings. Most activities require the use of the auditory or tactile senses. As such, assistive systems which are capable of aiding visually impaired people to understand, navigate and form a mental representation of their environment are extensively being studied and developed. The aim of this paper is to provide insight regarding the characteristics, as well as the advantages and drawbacks of different types of sonification strategies in assistive systems, to assess their suitability for certain use-cases.

MATERIALS AND METHODS

To this end, we reviewed a sizeable number of assistive solutions for VIP which provide a form of auditory feedback to the user, encountered in different scientific databases (Scopus, IEEE Xplore, ACM and Google Scholar) through direct searches and cross-referencing.

RESULTS

We classified these solutions based on the aural information they provide to the VIP - alerts, guidance and information about their environment, be it spatial or semantic. Our intention is not to provide an exhaustive review, but to select representative implementations from recent literature that highlight the particularities of each sonification approach.

CONCLUSIONS

Thus, anyone who is intent on developing an assistive solution will be able to choose the desired sonification class, being aware of the advantages/disadvantages and at the same time having a fairly wide selection of articles from the representative class.

A Smart Cane Based on 2D LiDAR and RGB-D Camera Sensor-Realizing Navigation and Obstacle Recognition

In this paper, an intelligent blind guide system based on 2D LiDAR and RGB-D camera sensing is proposed, and the system is mounted on a smart cane. The intelligent guide system relies on 2D LiDAR, an RGB-D camera, IMU, GPS, Jetson nano B01, STM32, and other hardware. The main advantage of the intelligent guide system proposed by us is that the distance between the smart cane and obstacles can be measured by 2D LiDAR based on the cartographer algorithm, thus achieving simultaneous localization and mapping (SLAM). At the same time, through the improved YOLOv5 algorithm, pedestrians, vehicles, pedestrian crosswalks, traffic lights, warning posts, stone piers, tactile paving, and other objects in front of the visually impaired can be quickly and effectively identified. Laser SLAM and improved YOLOv5 obstacle identification tests were carried out inside a teaching building on the campus of Hainan Normal University and on a pedestrian crossing on Longkun South Road in Haikou City, Hainan Province. The results show that the intelligent guide system developed by us can drive the omnidirectional wheels at the bottom of the smart cane and provide the smart cane with a self-leading blind guide function, like a "guide dog", which can effectively guide the visually impaired to avoid obstacles and reach their predetermined destination, and can quickly and effectively identify the obstacles on the way out. The mapping and positioning accuracy of the system's laser SLAM is 1 m ± 7 cm, and the laser SLAM speed of this system is 25~31 FPS, which can realize the short-distance obstacle avoidance and navigation function both in indoor and outdoor environments. The improved YOLOv5 helps to identify 86 types of objects. The recognition rates for pedestrian crosswalks and for vehicles are 84.6% and 71.8%, respectively; the overall recognition rate for 86 types of objects is 61.2%, and the obstacle recognition rate of the intelligent guide system is 25-26 FPS.

3D Visual Reconstruction as Prior Information for First Responder Localization and Visualization

In professional use cases like police or fire brigade missions, coordinated and systematic force management is crucial for achieving operational success during intervention by the emergency personnel. A real-time situation picture enhances the coordination of the team. This situation picture includes not only an overview of the environment but also the positions, i.e., localization, of the emergency forces. The overview of the environment can be obtained either from known situation pictures like floorplans or by scanning the environment with the aid of visual sensors. The self-localization problem can be solved outdoors using the Global Navigation Satellite System (GNSS), but it is not fully solved indoors, where the GNSS signal might not be received or might be degraded. In this paper, we propose a novel combination of an inertial localization technique based on simultaneous localization and mapping (SLAM) with 3D building scans, which are used as prior information, for geo-referencing the positions, obtaining a situation picture, and finally visualizing the results with an appropriate visualization tool. We developed a new method for converting point clouds into a hexagonal prism map specifically designed for our SLAM algorithm. With this combination, we could keep the equipment for first responders as lightweight as required. We showed that the positioning led to an average accuracy of less than 1m indoors, and the final visualization including the building layout obtained by the 3D building reconstruction will be advantageous for coordinating first responder operations.

Sound-Based Localization Using LSTM Networks for Visually Impaired Navigation

In this work, we developed a prototype that adopted sound-based systems for localization of visually impaired individuals. The system was implemented based on a wireless ultrasound network, which helped the blind and visually impaired to navigate and maneuver autonomously. Ultrasonic-based systems use high-frequency sound waves to detect obstacles in the environment and provide location information to the user. Voice recognition and long short-term memory (LSTM) techniques were used to design the algorithms. The Dijkstra algorithm was also used to determine the shortest distance between two places. Assistive hardware tools, which included an ultrasonic sensor network, a global positioning system (GPS), and a digital compass, were utilized to implement this method. For indoor evaluation, three nodes were localized on the doors of different rooms inside the house, including the kitchen, bathroom, and bedroom. The coordinates (interactive latitude and longitude points) of four outdoor areas (mosque, laundry, supermarket, and home) were identified and stored in a microcomputer's memory to evaluate the outdoor settings. The results showed that the root mean square error for indoor settings after 45 trials is about 0.192. In addition, the Dijkstra algorithm determined that the shortest distance between two places was within an accuracy of 97%.

Demonstration Laboratory of Industry 4.0 Retrofitting and Operator 4.0 Solutions: Education towards Industry 5.0

One of the main challenges of Industry 4.0 is how advanced sensors and sensing technologies can be applied through the Internet of Things layers of existing manufacturing. This is the so-called Brownfield Industry 4.0, where the different types and ages of machines and processes need to be digitalized. Smart retrofitting is the umbrella term for solutions to show how we can digitalize manufacturing machines. This problem is critical in the case of solutions to support human workers. The Operator 4.0 concept shows how we can efficiently support workers on the shop floor. The key indicator is the readiness level of a company, and the main bottleneck is the technical knowledge of the employees. This study proposes an education framework and a related Operator 4.0 laboratory that prepares students for the development and application of Industry 5.0 technologies. The concept of intelligent space is proposed as a basis of the educational framework, which can solve the problem of monitoring the stochastic nature of operators in production processes. The components of the intelligent space are detailed through the layers of the IoT in the form of a case study conducted at the laboratory. The applicability of indoor positioning systems is described with the integration of machine-, operator- and environment-based sensor data to obtain real-time information from the shop floor. The digital twin of the laboratory is developed in a discrete event simulator, which integrates the data from the shop floor and can control the production based on the simulation results. The presented framework can be utilized to design education for the generation of Industry 5.0.

Review of Navigation Assistive Tools and Technologies for the Visually Impaired

The visually impaired suffer greatly while moving from one place to another. They face challenges in going outdoors and in protecting themselves from moving and stationary objects, and they also lack confidence due to restricted mobility. Due to the recent rapid rise in the number of visually impaired persons, the development of assistive devices has emerged as a significant research field. This review study introduces several techniques to help the visually impaired with their mobility and presents the state-of-the-art of recent assistive technologies that facilitate their everyday life. It also analyses comprehensive multiple mobility assistive technologies for indoor and outdoor environments and describes the different location and feedback methods for the visually impaired using assistive tools based on recent technologies. The navigation tools used for the visually impaired are discussed in detail in subsequent sections. Finally, a detailed analysis of various methods is also carried out, with future recommendations.

A Multi-Sensory Guidance System for the Visually Impaired Using YOLO and ORB-SLAM

Guidance systems for visually impaired persons have become a popular topic in recent years. Existing guidance systems on the market typically utilize auxiliary tools and methods such as GPS, UWB, or a simple white cane that exploits the user’s single tactile or auditory sense. These guidance methodologies can be inadequate in a complex indoor environment. This paper proposes a multi-sensory guidance system for the visually impaired that can provide tactile and auditory advice using ORB-SLAM and YOLO techniques. Based on an RGB-D camera, the local obstacle avoidance system is realized at the tactile level through point cloud filtering that can inform the user via a vibrating motor. Our proposed method can generate a dense navigation map to implement global obstacle avoidance and path planning for the user through the coordinate transformation. Real-time target detection and a voice-prompt system based on YOLO are also incorporated at the auditory level. We implemented the proposed system as a smart cane. Experiments are performed using four different test scenarios. Experimental results demonstrate that the impediments in the walking path can be reliably located and classified in real-time. Our proposed system can function as a capable auxiliary to help visually impaired people navigate securely by integrating YOLO with ORB-SLAM.

Community- and Data-Driven Services for Multi-Policy Pedestrian Routing

Pedestrian routing is important in a multitude of public spaces, especially those characterized by a large number of newcomers. Their needs may be diverse, with priority for the shortest path, the less crowded or the less polluted one, the accessibility for reduced mobility, or the sheltering from unfavorable weather conditions. Hence, typical graph-based routing must be enriched to support multiple policies, at the choice of each person. The paper proposes a systemic approach and a set of services for orientation and accessibility, which are both community-driven and data-driven, for correctly perceiving the routing necessities and the surrounding situation. The response time to a pathfinding query depends on the types of policies applied and not only on their number, because each of them contributes to the customization of the weighted graph, although it refers to the same physical space traversed by pedestrians. The paper also presents results of loading tests for up to 5000 Virtual Users, inspired from real-life requirements and executed on a graph that models a real building in our university; different policies are applied to assess performance metrics, with simulated community feedback and sensor data.

Ultrasonic Sound Guide System with Eyeglass Device for the Visually Impaired

The ultrasonic sound guide system presents the audio broadcasting system based on the inaudible ultrasonic sound to assist the indoor and outdoor navigation of the visually impaired. The transmitters are placed at the point of interest to propagate the frequency modulated voice signal in ultrasonic sound range. The dual channel receiver device is carried by the visually impaired person in the form of eyeglasses to receive the ultrasonic sound for the voice signal via demodulation. Since the ultrasonic sound demonstrates the acoustic properties, the velocity, directivity, attenuation, and superposition of ultrasonic sound provide the acoustic clue to the user for localizing the multiple transmitter positions by binaural localization capability. The visually impaired hear the designated voice signal and follow the signal attributions to arrive at the specific location. Due to the low microphone gain from side addressing, the time delay between the receiver channels demonstrates the high variance and high bias in end directions. However, the perception experiment shows the further prediction accuracy in end directions as compared to the center direction outcomes. The overall evaluations show the precise directional prediction for narrow- and wide-angle situations. The ultrasonic sound guide system is a useful device to localize places in the near field without touching braille.

Digital Twin Applications: A Survey of Recent Advances and Challenges

Industry 4.0 integrates a series of emerging technologies, such as the Internet of Things (IoT), cyber-physical systems (CPS), cloud computing, and big data, and aims to improve operational efficiency and accelerate productivity inside the industrial environment. This article provides a series of information about the required structure to adopt Industry 4.0 approaches and a brief review of related concepts to finally identify challenges and research opportunities to envision the adoption of so-called digital twins. We want to pay attention to upgrading older systems aiming to provide the well-known advantages of Industry 4.0 to such legacy systems as reducing production costs, increasing efficiency, acquiring better robustness of equipment, and reaching advanced process connectivity.

Cloud Platforms for Context-Adaptive Positioning and Localisation in GNSS-Denied Scenarios-A Systematic Review

Cloud Computing and Cloud Platforms have become an essential resource for businesses, due to their advanced capabilities, performance, and functionalities. Data redundancy, scalability, and security, are among the key features offered by cloud platforms. Location-Based Services (LBS) often exploit cloud platforms to host positioning and localisation systems. This paper introduces a systematic review of current positioning platforms for GNSS-denied scenarios. We have undertaken a comprehensive analysis of each component of the positioning and localisation systems, including techniques, protocols, standards, and cloud services used in the state-of-the-art deployments. Furthermore, this paper identifies the limitations of existing solutions, outlining shortcomings in areas that are rarely subjected to scrutiny in existing reviews of indoor positioning, such as computing paradigms, privacy, and fault tolerance. We then examine contributions in the areas of efficient computation, interoperability, positioning, and localisation. Finally, we provide a brief discussion concerning the challenges for cloud platforms based on GNSS-denied scenarios.

Indoor Positioning Systems Can Revolutionise Digital Lean

The powerful combination of lean principles and digital technologies accelerates waste identification and mitigation faster than traditional lean methods. The new digital lean (also referred to as Lean 4.0) solutions incorporate sensors and digital equipment, yielding innovative solutions that extend the reach of traditional lean tools. The tracking of flexible and configurable production systems is not as straightforward as in a simple conveyor. This paper examines how the information provided by indoor positioning systems (IPS) can be utilised in the digital transformation of flexible manufacturing. The proposed IPS-based method enriches the information sources of value stream mapping and transforms positional data into key-performance indicators used in Lean Manufacturing. The challenges of flexible and reconfigurable manufacturing require a dynamic value stream mapping. To handle this problem, a process mining-based solution has been proposed. A case study is provided to show how the proposed method can be employed for monitoring and improving manufacturing efficiency.

Real-Time Locating System in Production Management

Real-time monitoring and optimization of production and logistics processes significantly improve the efficiency of production systems. Advanced production management solutions require real-time information about the status of products, production, and resources. As real-time locating systems (also referred to as indoor positioning systems) can enrich the available information, these systems started to gain attention in industrial environments in recent years. This paper provides a review of the possible technologies and applications related to production control and logistics, quality management, safety, and efficiency monitoring. This work also provides a workflow to clarify the steps of a typical real-time locating system project, including the cleaning, pre-processing, and analysis of the data to provide a guideline and reference for research and development of indoor positioning-based manufacturing solutions.

Effect Evaluation of Spatial Characteristics on Map Matching-Based Indoor Positioning

Map-matching is a popular method that uses spatial information to improve the accuracy of positioning methods. The performance of map matching methods is closely related to spatial characteristics. Although several studies have demonstrated that certain map matching algorithms are affected by some spatial structures (e.g., parallel paths), they focus on the analysis of single map matching method or few spatial structures. In this study, we explored how the most commonly-used four spatial characteristics (namely forks, open spaces, corners, and narrow corridors) affect three popular map matching methods, namely particle filtering (PF), hidden Markov model (HMM), and geometric methods. We first provide a theoretical analysis on how spatial characteristics affect the performance of map matching methods, and then evaluate these effects through experiments. We found that corners and narrow corridors are helpful in improving the positioning accuracy, while forks and open spaces often lead to a larger positioning error. We hope that our findings are helpful for future researchers in choosing proper map matching algorithms with considering the spatial characteristics.

A Low-Cost Calibration Method for Low-Cost MEMS Accelerometers Based on 3D Printing

A ubiquitous sensor in embedded systems is the accelerometer, as it enables a range of applications. However, accelerometers experience nonlinearities in their outputs caused by error terms and axes misalignment. These errors are a major concern because, in applications such as navigations systems, they accumulate over time, degrading the position accuracy. Through a calibration procedure, the errors can be modeled and compensated. Many methods have been proposed; however, they require sophisticated equipment available only in laboratories, which makes them complex and expensive. In this article, a simple, practical, and low-cost calibration method is proposed. It uses a 3D printed polyhedron, benefiting from the popularisation and low-cost of 3D printing in the present day. Additionally, each polyhedron could hold as much as 14 sensors, which can be calibrated simultaneously. The method was performed with a low-cost sensor and it significantly reduced the root-mean-square error (RMSE) of the sensor output. The RMSE was compared with the reported in similar proposals, and our method resulted in higher performance. The proposal enables accelerometer calibration at low-cost, and anywhere and anytime, not only by experts in laboratories. Compensating the sensor’s inherent errors thus increases the accuracy of its output.