Predicting Transitioning Walking Gaits: Hip and Knee Joint Trajectories From the Motion of Walking Canes

A Deep Learning Model with a Self-Attention Mechanism for Leg Joint Angle Estimation across Varied Locomotion Modes

Conventional trajectory planning for lower limb assistive devices usually relies on a finite-state strategy, which pre-defines fixed trajectory types for specific gait events and activities. The advancement of deep learning enables walking assistive devices to better adapt to varied terrains for diverse users by learning movement patterns from gait data. Using a self-attention mechanism, a temporal deep learning model is developed in this study to continuously generate lower limb joint angle trajectories for an ankle and knee across various activities. Additional analyses, including using Fast Fourier Transform and paired t-tests, are conducted to demonstrate the benefits of the proposed attention model architecture over the existing methods. Transfer learning has also been performed to prove the importance of data diversity. Under a 10-fold leave-one-out testing scheme, the observed attention model errors are 11.50% (±2.37%) and 9.31% (±1.56%) NRMSE for ankle and knee angle estimation, respectively, which are small in comparison to other studies. Statistical analysis using the paired t-test reveals that the proposed attention model appears superior to the baseline model in terms of reduced prediction error. The attention model also produces smoother outputs, which is crucial for safety and comfort. Transfer learning has been shown to effectively reduce model errors and noise, showing the importance of including diverse datasets. The suggested joint angle trajectory generator has the potential to seamlessly switch between different locomotion tasks, thereby mitigating the problem of detecting activity transitions encountered by the traditional finite-state strategy. This data-driven trajectory generation method can also reduce the burden on personalization, as traditional devices rely on prosthetists to experimentally tune many parameters for individuals with diverse gait patterns.

Motion Trajectories Prediction of Lower Limb Exoskeleton Based on Long Short-Term Memory (LSTM) Networks

A typical man–machine coupling system could provide the wearer a coordinated and assisted movement by the lower limb exoskeleton. The process of cooperative movement relies on the accurate perception of the wearer’s human movement information and the accurate planning and control of the joint movement of the lower limb exoskeleton. In this paper, a neural network and a Long-Short Term Memory (LSTM) machine learning model method is proposed to predict the actual movement trajectory of the human body’s lower limbs. Then a wearable joint angle measurement device was designed for gait trajectory prediction, which can be used for predictive control through machine learning methods. The experimental results show that the LSTM model can accurately predict the gait trajectory with an average mean square error. This method has practical significance for prediction the trajectory of the lower limb exoskeleton.

Real-Time Activity Recognition With Instantaneous Characteristic Features of Thigh Kinematics

Current supervised learning or deep learning-based activity recognition classifiers can achieve high accuracy in recognizing locomotion activities. Most available techniques use a high-dimensional space of features, e.g., combinations of EMG, kinematics and kinetics, and transformations over those signals. The associated classification rules are therefore complex; the machine tries to understand the human, but the human does not understand the machine. This paper presents an activity recognition system that uses signals from a thigh-mounted IMU and a force sensitive resistor to classify transitions between sitting, walking, stair ascending, and stair descending. The system uses the thigh's orientation and velocity with foot contact information at specific moments within a given activity as the features to classify transitions to other activities. We call these Instantaneous Characteristic Features (ICFs). Because these ICFs are biomechanically intuitive, they are easy for the user to understand and thus control the activity transitions of wearable robots. We assessed our classification algorithm offline using an existing dataset with 10 able-bodied subjects and online with another 10 able-bodied subjects wearing a real-time system. The offline study analyzed the effect of subject-dependency and ramp inclinations. The real-time classification accuracy was evaluated before and after training the subjects on the ICFs. The real-time system achieved overall pre-subject-training and post-subject-training error rates of 0.59% ± 0.24% and 0.56% ± 0.20%, respectively. We also evaluated the feasibility of our ICFs for amputee ambulation by analyzing a public dataset with the open-source bionic leg. The simplicity of these classification rules demonstrates a new paradigm for activity recognition where the human can understand the machine and vice-versa.

Effective recognition of human lower limb jump locomotion phases based on multi-sensor information fusion and machine learning

Jump locomotion is the basic movement of human. However, no thorough research on the recognition of jump sub-phases has been carried so far. This paper aims to use multi-sensor information fusion and machine learning to recognize the human jump phase, which is crucial to the development of exoskeleton that assists jumping. The method of information fusion for sensors including sEMG, IMU, and footswitch sensor is studied. The footswitch signals are filtered by median filter. A processing method of synthesizing Euler angles into phase angle is proposed, which is beneficial to data integration. The jump locomotion is creatively segmented into five phases. The onset and offset of active segment are detected by sample entropy of sEMG and standard deviation of acceleration signal. The features are extracted from analysis windows using multi-sensor information fusion, and the dimension of feature matrix is selected. By comparing the performances of state-of-the-art machine learning classifiers, feature subsets of sEMG, IMU, and footswitch signals are selected from time domain features in a series of analysis window parameters. The average recognition accuracy of sEMG and IMU is 91.76% and 97.68%, respectively. When using the combination of sEMG, IMU, and footswitch signals, the average accuracy is 98.70%, which outperforms the combination of sEMG and IMU (97.97%, p < 0.01). Graphical Abstract The sub-phases of human locomotion are recognized based on multi-sensor information fusion and machine learning method. The feature data of the sub-phases is visualized in 3-dimensional space. The predicted states and the true states in a complete jump are compared along the time axis.