Gait Phase Estimation by Using LSTM in IMU-Based Gait Analysis-Proof of Concept

IMU-Based Real-Time Estimation of Gait Phase Using Multi-Resolution Neural Networks

This work presents a real-time gait phase estimator using thigh- and shank-mounted inertial measurement units (IMUs). A multi-rate convolutional neural network (CNN) was trained to estimate gait phase for a dataset of 16 participants walking on an instrumented treadmill with speeds varying between 0.1 to 1.9 m/s, and conditions such as asymmetric walking, stop-start, and sudden speed changes. One-subject-out cross-validation was used to assess the robustness of the estimator to the gait patterns of new individuals. The proposed model had a spatial root mean square error of 5.00±1.65%, and a temporal mean absolute error of 2.78±0.97% evaluated at the heel strike. A second cross-validation was performed to show that leaving out any of the walking conditions from the training dataset did not result in significant performance degradation. A 2-sample Kolmogorov-Smirnov test showed that there was no significant increase in spatial or temporal error when testing on the abnormal walking conditions left out of the training set. The results of the two cross-validations demonstrate that the proposed model generalizes well across new participants, various walking speeds, and gait patterns, showcasing its potential for use in investigating patient populations with pathological gaits and facilitating robot-assisted walking.

Bi-Directional Long Short-Term Memory-Based Gait Phase Recognition Method Robust to Directional Variations in Subject's Gait Progression Using Wearable Inertial Sensor

Inertial Measurement Unit (IMU) sensor-based gait phase recognition is widely used in medical and biomechanics fields requiring gait data analysis. However, there are several limitations due to the low reproducibility of IMU sensor attachment and the sensor outputs relative to a fixed reference frame. The prediction algorithm may malfunction when the user changes their walking direction. In this paper, we propose a gait phase recognition method robust to user body movements based on a floating body-fixed frame (FBF) and bi-directional long short-term memory (bi-LSTM). Data from four IMU sensors attached to the shanks and feet on both legs of three subjects, collected via the FBF method, are processed through preprocessing and the sliding window label overlapping method before inputting into the bi-LSTM for training. To improve the model's recognition accuracy, we selected parameters that influence both training and test accuracy. We conducted a sensitivity analysis using a level average analysis of the Taguchi method to identify the optimal combination of parameters. The model, trained with optimal parameters, was validated on a new subject, achieving a high test accuracy of 86.43%.

A Systematic Evaluation of Feature Encoding Techniques for Gait Analysis Using Multimodal Sensory Data

This paper addresses the problem of feature encoding for gait analysis using multimodal time series sensory data. In recent years, the dramatic increase in the use of numerous sensors, e.g., inertial measurement unit (IMU), in our daily wearable devices has gained the interest of the research community to collect kinematic and kinetic data to analyze the gait. The most crucial step for gait analysis is to find the set of appropriate features from continuous time series data to accurately represent human locomotion. This paper presents a systematic assessment of numerous feature extraction techniques. In particular, three different feature encoding techniques are presented to encode multimodal time series sensory data. In the first technique, we utilized eighteen different handcrafted features which are extracted directly from the raw sensory data. The second technique follows the Bag-of-Visual-Words model; the raw sensory data are encoded using a pre-computed codebook and a locality-constrained linear encoding (LLC)-based feature encoding technique. We evaluated two different machine learning algorithms to assess the effectiveness of the proposed features in the encoding of raw sensory data. In the third feature encoding technique, we proposed two end-to-end deep learning models to automatically extract the features from raw sensory data. A thorough experimental evaluation is conducted on four large sensory datasets and their outcomes are compared. A comparison of the recognition results with current state-of-the-art methods demonstrates the computational efficiency and high efficacy of the proposed feature encoding method. The robustness of the proposed feature encoding technique is also evaluated to recognize human daily activities. Additionally, this paper also presents a new dataset consisting of the gait patterns of 42 individuals, gathered using IMU sensors.

Walking-Speed-Adaptive Gait Phase Estimation for Wearable Robots

This paper introduces a Gait Phase Estimation Module (GPEM) and its real-time algorithm designed to estimate gait phases continuously and monotonically across a range of walking speeds and accelerations/decelerations. To address the challenges of real-world applications, we propose a speed-adaptive online gait phase estimation algorithm, which enables precise estimation of gait phases during both constant speed locomotion and dynamic speed changes. Experimental verification demonstrates that the proposed method offers smooth, continuous, and repetitive gait phase estimation when compared to conventional approaches such as the phase portrait method and time-based estimation. The proposed method achieved a 48% reduction in gait phase deviation compared to time-based estimation and a 48.29% reduction compared to the phase portrait method. The proposed algorithm is integrated within the GPEM, allowing for its versatile application in controlling gait assistive robots without incurring additional computational burden. The results of this study contribute to the development of robust and efficient gait phase estimation techniques for various robotic applications.

Adaptive Control Method for Gait Detection and Classification Devices with Inertial Measurement Unit

Cueing and feedback training can be effective in maintaining or improving gait in individuals with Parkinson's disease. We previously designed a rehabilitation assist device that can detect and classify a user's gait at only the swing phase of the gait cycle, for the ease of data processing. In this study, we analyzed the impact of various factors in a gait detection algorithm on the gait detection and classification rate (GDCR). We collected acceleration and angular velocity data from 25 participants (1 male and 24 females with an average age of 62 ± 6 years) using our device and analyzed the data using statistical methods. Based on these results, we developed an adaptive GDCR control algorithm using several equations and functions. We tested the algorithm under various virtual exercise scenarios using two control methods, based on acceleration and angular velocity, and found that the acceleration threshold was more effective in controlling the GDCR (average Spearman correlation -0.9996, p < 0.001) than the gyroscopic threshold. Our adaptive control algorithm was more effective in maintaining the target GDCR than the other algorithms (p < 0.001) with an average error of 0.10, while other tested methods showed average errors of 0.16 and 0.28. This algorithm has good scalability and can be adapted for future gait detection and classification applications.

A Real-Time Gait Phase Detection Method Based on BiLSTM-Attention Model

Real-time gait phase detection is essential to achieve accurate and stable walking assistance in intelligent rehabilitation training for patients with motor disorders. This study proposed an efficient real-time detection method to detect three gait phases (loading response, stance, and swing) based on a bidirectional long short-term memory network with an attention layer (BiLSTM-Attention). We validated our method on a public dataset where eight healthy subjects' data during treadmill walking were employed. A single inertial measurement unit (IMU) was attached to the shank to measure the sagittal plane acceleration of the lower leg and the angular velocity around the central lateral axis. These data were transposed and segmented into data sequences based on labels using a sliding window method. The data from 8 participants were divided into the training, validation, and test sets (5:1:2). Results showed the average recognition accuracy of the proposed model on new subjects was 97.40% with an average time delay of 15.7±10.1ms, showing the method's potential to be applied for practice use.

Estimation of Lower Extremity Muscle Activity in Gait Using the Wearable Inertial Measurement Units and Neural Network

The inertial measurement unit (IMU) has become more prevalent in gait analysis. However, it can only measure the kinematics of the body segment it is attached to. Muscle behaviour is an important part of gait analysis and provides a more comprehensive overview of gait quality. Muscle behaviour can be estimated using musculoskeletal modelling or measured using an electromyogram (EMG). However, both methods can be tasking and resource intensive. A combination of IMU and neural networks (NN) has the potential to overcome this limitation. Therefore, this study proposes using NN and IMU data to estimate nine lower extremity muscle activities. Two NN were developed and investigated, namely feedforward neural network (FNN) and long short-term memory neural network (LSTM). The results show that, although both networks were able to predict muscle activities well, LSTM outperformed the conventional FNN. This study confirms the feasibility of estimating muscle activity using IMU data and NN. It also indicates the possibility of this method enabling the gait analysis to be performed outside the laboratory environment with a limited number of devices.

Gait Events Prediction Using Hybrid CNN-RNN-Based Deep Learning Models through a Single Waist-Worn Wearable Sensor

Elderly gait is a source of rich information about their physical and mental health condition. As an alternative to the multiple sensors on the lower body parts, a single sensor on the pelvis has a positional advantage and an abundance of information acquirable. This study aimed to improve the accuracy of gait event detection in the elderly using a single sensor on the waist and deep learning models. Data were gathered from elderly subjects equipped with three IMU sensors while they walked. The input taken only from the waist sensor was used to train 16 deep-learning models including a CNN, RNN, and CNN-RNN hybrid with or without the Bidirectional and Attention mechanism. The groundtruth was extracted from foot IMU sensors. A fairly high accuracy of 99.73% and 93.89% was achieved by the CNN-BiGRU-Att model at the tolerance window of ±6 TS (±6 ms) and ±1 TS (±1 ms), respectively. Advancing from the previous studies exploring gait event detection, the model demonstrated a great improvement in terms of its prediction error having an MAE of 6.239 ms and 5.24 ms for HS and TO events, respectively, at the tolerance window of ±1 TS. The results demonstrated that the use of CNN-RNN hybrid models with Attention and Bidirectional mechanisms is promising for accurate gait event detection using a single waist sensor. The study can contribute to reducing the burden of gait detection and increase its applicability in future wearable devices that can be used for remote health monitoring (RHM) or diagnosis based thereon.

Computer Vision and Machine Learning-Based Gait Pattern Recognition for Flat Fall Prediction

BACKGROUND

Gait recognition has been applied in the prediction of the probability of elderly flat ground fall, functional evaluation during rehabilitation, and the training of patients with lower extremity motor dysfunction. Gait distinguishing between seemingly similar kinematic patterns associated with different pathological entities is a challenge for the clinician. How to realize automatic identification and judgment of abnormal gait is a significant challenge in clinical practice. The long-term goal of our study is to develop a gait recognition computer vision system using artificial intelligence (AI) and machine learning (ML) computing. This study aims to find an optimal ML algorithm using computer vision techniques and measure variables from lower limbs to classify gait patterns in healthy people. The purpose of this study is to determine the feasibility of computer vision and machine learning (ML) computing in discriminating different gait patterns associated with flat-ground falls.

METHODS

We used the Kinect® Motion system to capture the spatiotemporal gait data from seven healthy subjects in three walking trials, including normal gait, pelvic-obliquity-gait, and knee-hyperextension-gait walking. Four different classification methods including convolutional neural network (CNN), support vector machine (SVM), K-nearest neighbors (KNN), and long short-term memory (LSTM) neural networks were used to automatically classify three gait patterns. Overall, 750 sets of data were collected, and the dataset was divided into 80% for algorithm training and 20% for evaluation.

RESULTS

The SVM and KNN had a higher accuracy than CNN and LSTM. The SVM (94.9 ± 3.36%) had the highest accuracy in the classification of gait patterns, followed by KNN (94.0 ± 4.22%). The accuracy of CNN was 87.6 ± 7.50% and that of LSTM 83.6 ± 5.35%.

CONCLUSIONS

This study revealed that the proposed AI machine learning (ML) techniques can be used to design gait biometric systems and machine vision for gait pattern recognition. Potentially, this method can be used to remotely evaluate elderly patients and help clinicians make decisions regarding disposition, follow-up, and treatment.

Using Deep Learning to Predict Minimum Foot-Ground Clearance Event from Toe-Off Kinematics

Efficient, adaptive, locomotor function is critically important for maintaining our health and independence, but falls-related injuries when walking are a significant risk factor, particularly for more vulnerable populations such as older people and post-stroke individuals. Tripping is the leading cause of falls, and the swing-phase event Minimum Foot Clearance (MFC) is recognised as the key biomechanical determinant of tripping probability. MFC is defined as the minimum swing foot clearance, which is seen approximately mid-swing, and it is routinely measured in gait biomechanics laboratories using precise, high-speed, camera-based 3D motion capture systems. For practical intervention strategies designed to predict, and possibly assist, swing foot trajectory to prevent tripping, identification of the MFC event is essential; however, no technique is currently available to determine MFC timing in real-life settings outside the laboratory. One strategy has been to use wearable sensors, such as Inertial Measurement Units (IMUs), but these data are limited to primarily providing only tri-axial linear acceleration and angular velocity. The aim of this study was to develop Machine Learning (ML) algorithms to predict MFC timing based on the preceding toe-off gait event. The ML algorithms were trained using 13 young adults' foot trajectory data recorded from an Optotrak 3D motion capture system. A Deep Learning configuration was developed based on a Recurrent Neural Network with a Long Short-Term Memory (LSTM) architecture and Huber loss-functions to minimise MFC-timing prediction error. We succeeded in predicting MFC timing from toe-off characteristics with a mean absolute error of 0.07 s. Although further algorithm training using population-specific inputs are needed. The ML algorithms designed here can be used for real-time actuation of wearable active devices to increase foot clearance at critical MFC and reduce devastating tripping falls. Further developments in ML-guided actuation for active exoskeletons could prove highly effective in developing technologies to reduce tripping-related falls across a range of gait impaired populations.

Identification of gait patterns in walking with crutches through the selection of significant spatio-temporal parameters

Forearm crutches are one of the most accepted aids in rehabilitation for walking. The improper use of crutches may prolong the rehabilitation period and cause further limb damage or pain. However, it is possible to tackle this issue by using instrumented crutches that provide a quantitative gait analysis of the users. In addition, the study of different aspects of crutch walking could assist clinicians in choosing the optimum crutch gait pattern for individuals and instruct them to use the aids correctly. Measurements from the crutches are influenced by the performed gait pattern, determined by the legs and arms' sequence of movement. Since different parameters can describe gait, this paper aims to identify four gait patterns in walking with crutches through a reliable selection of gait parameters. In this study, we collected data from twenty healthy volunteers performing four gait patterns to reach this goal. First, we segmented the gait sequence in periodic cycles to detect two main phases, swing and stance. Then, we calculated different parameters for each gait walking pattern. Subsequently, we found a reduced set of parameters through some feature selection techniques. Selected parameters were validated employing three classification models. After evaluating the models' metrics, our findings indicated that the set of selected parameters could identify a crutch walking pattern.

Foot-to-Ground Phases Detection: A Comparison of Data Representation Formatting Methods with Respect to Adaption of Deep Learning Architectures

Identifying the foot stance and foot swing phases, also known as foot-to-ground (FTG) detection, is a branch of Human Activity Recognition (HAR). Our study aims to detect two main phases of the gait (i.e., foot-off and foot-contact) corresponding to the moments when each foot is in contact with the ground or not. This will allow the medical professionals to characterize and identify the different phases of the human gait and their respective patterns. This detection process is paramount for extracting gait features (e.g., step width, stride width, gait speed, cadence, etc.) used by medical experts to highlight gait anomalies, stance issues, or any other walking irregularities. It will be used to assist health practitioners with patient monitoring, in addition to developing a full pipeline for FTG detection that would help compute gait indicators. In this paper, a comparison of different training configurations, including model architectures, data formatting, and pre-processing, was conducted to select the parameters leading to the highest detection accuracy. This binary classification provides a label for each timestamp informing whether the foot is in contact with the ground or not. Models such as CNN, LSTM, and ConvLSTM were the best fits for this study. Yet, we did not exclude DNNs and Machine Learning models, such as Random Forest and XGBoost from our work in order to have a wide range of possible comparisons. As a result of our experiments, which included 27 senior participants who had a stroke in the past wearing IMU sensors on their ankles, the ConvLSTM model achieved a high accuracy of 97.01% for raw windowed data with a size of 3 frames per window, and each window was formatted to have two superimposed channels (accelerometer and gyroscope channels). The model was trained to have the best detection without any knowledge of the participants’ personal information including age, gender, health condition, the type of activity, or the used foot. In other words, the model’s input data only originated from IMU sensors. Overall, in terms of FTG detection, the combination of the ConvLSTM model and the data representation had an important impact in outperforming other start-of-the-art configurations; in addition, the compromise between the model’s complexity and its accuracy is a major asset for deploying this model and developing real-time solutions.

Human Gait Modeling, Prediction and Classification for Level Walking Using Harmonic Models Derived from a Single Thigh-Mounted IMU

The majority of human gait modeling is based on hip, foot or thigh acceleration. The regeneration accuracy of these modeling approaches is not very high. This paper presents a harmonic approach to modeling human gait during level walking based on gyroscopic signals for a single thigh-mounted Inertial Measurement Unit (IMU) and the flexion–extension derived from a single thigh-mounted IMU. The thigh angle can be modeled with five significant harmonics, with a regeneration accuracy of over 0.999 correlation and less than 0.5° RMSE per stride cycle. Comparable regeneration accuracies can be achieved with nine significant harmonics for the gyro signal. The fundamental frequency of the harmonic model can be estimated using the stride time, with an error level of 0.0479% (±0.0029%). Six commonly observed stride patterns, and harmonic models of thigh angle and gyro signal for those stride patterns, are presented in this paper. These harmonic models can be used to predict or classify the strides of walking trials, and the results are presented herein. Harmonic models may also be used for activity recognition. It has shown that human gait in level walking can be modeled with a harmonic model of thigh angle or gyro signal, using a single thigh-mounted IMU, to higher accuracies than existing techniques.

An Automatic Gait Analysis Pipeline for Wearable Sensors: A Pilot Study in Parkinson's Disease

The use of wearable sensors allows continuous recordings of physical activity from participants in free-living or at-home clinical studies. The large amount of data collected demands automatic analysis pipelines to extract gait parameters that can be used as clinical endpoints. We introduce a deep learning-based automatic pipeline for wearables that processes tri-axial accelerometry data and extracts gait events-bout segmentation, initial contact (IC), and final contact (FC)-from a single sensor located at either the lower back (near L5), shin or wrist. The gait events detected are posteriorly used for gait parameter estimation, such as step time, length, and symmetry. We report results from a leave-one-subject-out (LOSO) validation on a pilot study dataset of five participants clinically diagnosed with Parkinson's disease (PD) and six healthy controls (HC). Participants wore sensors at three body locations and walked on a pressure-sensing walkway to obtain reference gait data. Mean absolute errors (MAE) for the IC events ranged from 22.82 to 33.09 milliseconds (msecs) for the lower back sensor while for the shin and wrist sensors, MAE ranges were 28.56-64.66 and 40.19-72.50 msecs, respectively. For the FC-event detection, MAE ranges were 29.06-48.42, 40.19-72.70 and 36.06-60.18 msecs for the lumbar, wrist and shin sensors, respectively. Intraclass correlation coefficients, ICC(2,k), between the estimated parameters and the reference data resulted in good-to-excellent agreement (ICC ≥ 0.84) for the lumbar and shin sensors, excluding the double support time (ICC = 0.37 lumbar and 0.38 shin) and swing time (ICC = 0.55 lumbar and 0.59 shin). The wrist sensor also showed good agreements, but the ICCs were lower overall than for the other two sensors. Our proposed analysis pipeline has the potential to extract up to 100 gait-related parameters, and we expect our contribution will further support developments in the fields of wearable sensors, digital health, and remote monitoring in clinical trials.