Inertial Measurement Unit Sensors in Assistive Technologies for Visually Impaired People, a Review

Indoor Scene Recognition Mechanism Based on Direction-Driven Convolutional Neural Networks

Indoor location-based services constitute an important part of our daily lives, providing position and direction information about people or objects in indoor spaces. These systems can be useful in security and monitoring applications that target specific areas such as rooms. Vision-based scene recognition is the task of accurately identifying a room category from a given image. Despite years of research in this field, scene recognition remains an open problem due to the different and complex places in the real world. Indoor environments are relatively complicated because of layout variability, object and decoration complexity, and multiscale and viewpoint changes. In this paper, we propose a room-level indoor localization system based on deep learning and built-in smartphone sensors combining visual information with smartphone magnetic heading. The user can be room-level localized while simply capturing an image with a smartphone. The presented indoor scene recognition system is based on direction-driven convolutional neural networks (CNNs) and therefore contains multiple CNNs, each tailored for a particular range of indoor orientations. We present particular weighted fusion strategies that improve system performance by properly combining the outputs from different CNN models. To meet users' needs and overcome smartphone limitations, we propose a hybrid computing strategy based on mobile computation offloading compatible with the proposed system architecture. The implementation of the scene recognition system is split between the user's smartphone and a server, which aids in meeting the computational requirements of CNNs. Several experimental analysis were conducted, including to assess performance and provide a stability analysis. The results obtained on a real dataset show the relevance of the proposed approach for localization, as well as the interest in model partitioning in hybrid mobile computation offloading. Our extensive evaluation demonstrates an increase in accuracy compared to traditional CNN scene recognition, indicating the effectiveness and robustness of our approach.

Estimation of Spatio-Temporal Parameters of Gait and Posture of Visually Impaired People Using Wearable Sensors

In rehabilitating orientation and mobility (O&M) for visually impaired people (VIP), the measurement of spatio-temporal gait and postural parameters is of specific interest for rehabilitators to assess performance and improvements in independent mobility. In the current practice of rehabilitation worldwide, this assessment is carried out in people with estimates made visually. The objective of this research was to propose a simple architecture based on the use of wearable inertial sensors for quantitative estimation of distance traveled, step detection, gait velocity, step length and postural stability. These parameters were calculated using absolute orientation angles. Two different sensing architectures were tested for gait according to a selected biomechanical model. The validation tests included five different walking tasks. There were nine visually impaired volunteers in real-time acquisitions, where the volunteers walked indoor and outdoor distances at different gait velocities in their residences. The ground truth gait characteristics of the volunteers in five walking tasks and an assessment of the natural posture during the walking tasks are also presented in this article. One of the proposed methods was selected for presenting the lowest absolute error of the calculated parameters in all of the traveling experimentations: 45 walking tasks between 7 and 45 m representing a total of 1039 m walked and 2068 steps; the step length measurement was 4.6 ± 6.7 cm with a mean of 56 cm (11.59 Std) and 1.5 ± 1.6 relative error in step count, which compromised the distance traveled and gait velocity measurements, presenting an absolute error of 1.78 ± 1.80 m and 7.1 ± 7.2 cm/s, respectively. The results suggest that the proposed method and its architecture could be used as a tool for assistive technology designed for O&M training to assess gait parameters and/or navigation, and that a sensor placed in the dorsal area is sufficient to detect noticeable postural changes that compromise heading, inclinations and balancing in walking tasks.

Effects of sliding window variation in the performance of acceleration-based human activity recognition using deep learning models

Deep learning (DL) models are very useful for human activity recognition (HAR); these methods present better accuracy for HAR when compared to traditional, among other advantages. DL learns from unlabeled data and extracts features from raw data, as for the case of time-series acceleration. Sliding windows is a feature extraction technique. When used for preprocessing time-series data, it provides an improvement in accuracy, latency, and cost of processing. The time and cost of preprocessing can be beneficial especially if the window size is small, but how small can this window be to keep good accuracy? The objective of this research was to analyze the performance of four DL models: a simple deep neural network (DNN); a convolutional neural network (CNN); a long short-term memory network (LSTM); and a hybrid model (CNN-LSTM), when variating the sliding window size using fixed overlapped windows to identify an optimal window size for HAR. We compare the effects in two acceleration sources': wearable inertial measurement unit sensors (IMU) and motion caption systems (MOCAP). Moreover, short sliding windows of sizes 5, 10, 15, 20, and 25 frames to long ones of sizes 50, 75, 100, and 200 frames were compared. The models were fed using raw acceleration data acquired in experimental conditions for three activities: walking, sit-to-stand, and squatting. Results show that the most optimal window is from 20-25 frames (0.20-0.25s) for both sources, providing an accuracy of 99,07% and F1-score of 87,08% in the (CNN-LSTM) using the wearable sensors data, and accuracy of 98,8% and F1-score of 82,80% using MOCAP data; similar accurate results were obtained with the LSTM model. There is almost no difference in accuracy in larger frames (100, 200). However, smaller windows present a decrease in the F1-score. In regard to inference time, data with a sliding window of 20 frames can be preprocessed around 4x (LSTM) and 2x (CNN-LSTM) times faster than data using 100 frames.

Sensor-Based Prototype of a Smart Assistant for Visually Impaired People—Preliminary Results

People with visual impairment are the second largest affected category with limited access to assistive products. A complete, portable, and affordable smart assistant for helping visually impaired people to navigate indoors, outdoors, and interact with the environment is presented in this paper. The prototype of the smart assistant consists of a smart cane and a central unit; communication between user and the assistant is carried out through voice messages, making the system suitable for any user, regardless of their IT skills. The assistant is equipped with GPS, electronic compass, Wi-Fi, ultrasonic sensors, an optical sensor, and an RFID reader, to help the user navigate safely. Navigation functionalities work offline, which is especially important in areas where Internet coverage is weak or missing altogether. Physical condition monitoring, medication, shopping, and weather information, facilitate the interaction between the user and the environment, supporting daily activities. The proposed system uses different components for navigation, provides independent navigation systems for indoors and outdoors, both day and night, regardless of weather conditions. Preliminary tests provide encouraging results, indicating that the prototype has the potential to help visually impaired people to achieve a high level of independence in daily activities.