What Lies Beneath One’s Feet? Terrain Classification Using Inertial Data of Human Walk

Classification of human walking context using a single-point accelerometer

Real-world walking data offers rich insights into a person's mobility. Yet, daily life variations can alter these patterns, making the data challenging to interpret. As such, it is essential to integrate context for the extraction of meaningful information from real-world movement data. In this work, we leveraged the relationship between the characteristics of a walking bout and context to build a classification algorithm to distinguish between indoor and outdoor walks. We used data from 20 participants wearing an accelerometer on the thigh over a week. Their walking bouts were isolated and labeled using GPS and self-reporting data. We trained and validated two machine learning models, random forest and ensemble Support Vector Machine, using a leave-one-participant-out validation scheme on 15 subjects. The 5 remaining subjects were used as a testing set to choose a final model. The chosen model achieved an accuracy of 0.941, an F1-score of 0.963, and an AUROC of 0.931. This validated model was then used to label the walks from a different dataset with 15 participants wearing the same accelerometer. Finally, we characterized the differences between indoor and outdoor walks using the ensemble of the data. We found that participants walked significantly faster, longer, and more continuously when walking outdoors compared to indoors. These results demonstrate how movement data alone can be used to obtain accurate information on important contextual factors. These factors can then be leveraged to enhance our understanding and interpretation of real-world movement data, providing deeper insights into a person's health.

Towards Environment-Aware Fall Risk Assessment: Classifying Walking Surface Conditions Using IMU-Based Gait Data and Deep Learning

Fall risk assessment (FRA) helps clinicians make decisions about the best preventative measures to lower the risk of falls by identifying the different risks that are specific to an individual. With the development of wearable technologies such as inertial measurement units (IMUs), several free-living FRA methods based on fall predictors derived from IMU-based data have been introduced. The performance of such methods could be improved by increasing awareness of the individuals' walking environment. This study aims to introduce and analyze a 25-layer convolutional neural network model for classifying nine walking surface conditions using IMU-based gait data, providing a basis for environment-aware FRAs. A database containing data collected from thirty participants who wore six IMU sensors while walking on nine surface conditions was employed. A systematic analysis was conducted to determine the effects of gait signals (acceleration, magnetic field, and rate of turn), sensor placement, and signal segment size on the method's performance. Accuracies of 0.935 and 0.969 were achieved using a single and dual sensor, respectively, reaching an accuracy of 0.971 in the best-case scenario with optimal settings. The findings and analysis can help to develop more reliable and interpretable fall predictors, eventually leading to environment-aware FRA methods.

Egocentric vision-based detection of surfaces: towards context-aware free-living digital biomarkers for gait and fall risk assessment

Background

Falls in older adults are a critical public health problem. As a means to assess fall risks, free-living digital biomarkers (FLDBs), including spatiotemporal gait measures, drawn from wearable inertial measurement unit (IMU) data have been investigated to identify those at high risk. Although gait-related FLDBs can be impacted by intrinsic (e.g., gait impairment) and/or environmental (e.g., walking surfaces) factors, their respective impacts have not been differentiated by the majority of free-living fall risk assessment methods. This may lead to the ambiguous interpretation of the subsequent FLDBs, and therefore, less precise intervention strategies to prevent falls.

Methods

With the aim of improving the interpretability of gait-related FLDBs and investigating the impact of environment on older adults’ gait, a vision-based framework was proposed to automatically detect the most common level walking surfaces. Using a belt-mounted camera and IMUs worn by fallers and non-fallers (mean age 73.6 yrs), a unique dataset (i.e., Multimodal Ambulatory Gait and Fall Risk Assessment in the Wild (MAGFRA-W)) was acquired. The frames and image patches attributed to nine participants’ gait were annotated: (a) outdoor terrains: pavement (asphalt, cement, outdoor bricks/tiles), gravel, grass/foliage, soil, snow/slush; and (b) indoor terrains: high-friction materials (e.g., carpet, laminated floor), wood, and tiles. A series of ConvNets were developed: EgoPlaceNet categorizes frames into indoor and outdoor; and EgoTerrainNet (with outdoor and indoor versions) detects the enclosed terrain type in patches. To improve the framework’s generalizability, an independent training dataset with 9,424 samples was curated from different databases including GTOS and MINC-2500, and used for pretrained models’ (e.g., MobileNetV2) fine-tuning.

Results

EgoPlaceNet detected outdoor and indoor scenes in MAGFRA-W with 97.36$$\%$$ % and 95.59$$\%$$ % (leave-one-subject-out) accuracies, respectively. EgoTerrainNet-Indoor and -Outdoor achieved high detection accuracies for pavement (87.63$$\%$$ % ), foliage (91.24$$\%$$ % ), gravel (95.12$$\%$$ % ), and high-friction materials (95.02$$\%$$ % ), which indicate the models’ high generalizabiliy.

Conclusions

Encouraging results suggest that the integration of wearable cameras and deep learning approaches can provide objective contextual information in an automated manner, towards context-aware FLDBs for gait and fall risk assessment in the wild.

Human Activity Recognition of Individuals with Lower Limb Amputation in Free-Living Conditions: A Pilot Study

This pilot study aimed to investigate the implementation of supervised classifiers and a neural network for the recognition of activities carried out by Individuals with Lower Limb Amputation (ILLAs), as well as individuals without gait impairment, in free living conditions. Eight individuals with no gait impairments and four ILLAs wore a thigh-based accelerometer and walked on an improvised route in the vicinity of their homes across a variety of terrains. Various machine learning classifiers were trained and tested for recognition of walking activities. Additional investigations were made regarding the detail of the activity label versus classifier accuracy and whether the classifiers were capable of being trained exclusively on non-impaired individuals’ data and could recognize physical activities carried out by ILLAs. At a basic level of label detail, Support Vector Machines (SVM) and Long-Short Term Memory (LSTM) networks were able to acquire 77–78% mean classification accuracy, which fell with increased label detail. Classifiers trained on individuals without gait impairment could not recognize activities carried out by ILLAs. This investigation presents the groundwork for a HAR system capable of recognizing a variety of walking activities, both for individuals with no gait impairments and ILLAs.

Gait analysis in neurological populations: Progression in the use of wearables

Gait assessment is an essential tool for clinical applications not only to diagnose different neurological conditions but also to monitor disease progression as it contributes to the understanding of underlying deficits. There are established methods and models for data collection and interpretation of gait assessment within different pathologies. This narrative review aims to depict the evolution of gait assessment from observation and rating scales to wearable sensors and laboratory technologies and provide limitations and possible future directions in the field of gait assessment. In this context, we first present an extensive review of current clinical outcomes and gait models. Then, we demonstrate commercially available wearable technologies with their technical capabilities along with their use in gait assessment studies for various neurological conditions. In the next sections, a descriptive knowledge for existing inertial and EMG based algorithms and a sign based guide that shows the outcomes of previous neurological gait assessment studies are presented. Finally, we state a discussion for the use of wearables in gait assessment and speculate the possible research directions by revealing the limitations and knowledge gaps in the literature.

Person Re-Identification Using Deep Modeling of Temporally Correlated Inertial Motion Patterns

Person re-identification (re-ID) is among the essential components that play an integral role in constituting an automated surveillance environment. Majorly, the problem is tackled using data acquired from vision sensors using appearance-based features, which are strongly dependent on visual cues such as color, texture, etc., consequently limiting the precise re-identification of an individual. To overcome such strong dependence on visual features, many researchers have tackled the re-identification problem using human gait, which is believed to be unique and provide a distinctive biometric signature that is particularly suitable for re-ID in uncontrolled environments. However, image-based gait analysis often fails to extract quality measurements of an individual’s motion patterns owing to problems related to variations in viewpoint, illumination (daylight), clothing, worn accessories, etc. To this end, in contrast to relying on image-based motion measurement, this paper demonstrates the potential to re-identify an individual using inertial measurements units (IMU) based on two common sensors, namely gyroscope and accelerometer. The experiment was carried out over data acquired using smartphones and wearable IMUs from a total of 86 randomly selected individuals including 49 males and 37 females between the ages of 17 and 72 years. The data signals were first segmented into single steps and strides, which were separately fed to train a sequential deep recurrent neural network to capture implicit arbitrary long-term temporal dependencies. The experimental setup was devised in a fashion to train the network on all the subjects using data related to half of the step and stride sequences only while the inference was performed on the remaining half for the purpose of re-identification. The obtained experimental results demonstrate the potential to reliably and accurately re-identify an individual based on one’s inertial sensor data.