Prediction of Limb Joint Angles Based on Multi-Source Signals by GS-GRNN for Exoskeleton Wearer

sEMG-Based End-to-End Continues Prediction of Human Knee Joint Angles Using the Tightly Coupled Convolutional Transformer Model

Wearable exoskeleton robots can promote the rehabilitation of patients with physical dysfunction. And improving human-computer interaction performance is a significant challenge for exoskeleton robots. The traditional feature extraction process based on surface Electromyography(sEMG) is complex and requires manual intervention, making real-time performance difficult to guarantee. In this study, we propose an end-to-end method to predict human knee joint angles based on sEMG signals using a tightly coupled convolutional transformer (TCCT) model. We first collected sEMG signals from 5 healthy subjects. Then, the envelope was extracted from the noise-removed sEMG signal and used as the input to the model. Finally, we developed the TCCT model to predict the knee joint angle after 100 ms. For the prediction performance, we used the Root Mean Square Error(RMSE), Pearson Correlation Coefficient(CC), and Adjustment R2 as metrics to evaluate the error between the actual knee angle and the predicted knee angle. The results show that the model can predict the human knee angle quickly and accurately. The mean RMSE, Adjustment R2, and (CC) values of the model are 3.79°, 0.96, and 0.98, respectively, which are better than traditional deep learning models such as Informer (4.14, 0.95, 0.98), CNN (5.56, 0.89, 0.96) and CNN-BiLSTM (3.97, 0.95, 0.98). In addition, the prediction time of our proposed model is only 11.67 ± 0.67 ms, which is less than 100 ms. Therefore, the real-time and accuracy of the model can meet the continuous prediction of human knee joint angle in practice.

Continuous online prediction of lower limb joints angles based on sEMG signals by deep learning approach

Continuous online prediction of human joints angles is a key point to improve the performance of man-machine cooperative control. In this study, a framework of online prediction method of joints angles by long short-term memory (LSTM) neural network only based on surface electromyography (sEMG) signals was proposed. The sEMG signals from eight muscles of five subjects' right leg and three joints angles and plantar pressure signals of subjects were collected simultaneously. Different inputs (only sEMG (unimodal), sEMG combined with plantar pressure (multimodal)) after online feature extraction and standardization were used for training the angle online prediction model by LSTM. The results indicate that there is no significant difference between the two kinds of inputs for LSTM model and the proposed method can make up for the shortage of using a single type of sensor. The range of mean values of root square mean error, mean absolute error and Pearson correlation coefficient of the three joints angles achieved by the proposed model only with the input of sEMG under four kinds of predicted time (50, 100, 150, and 200 ms) are [1.63°,3.20°],[1.27°, 2.36°] and [0.9747, 0.9935]. Three popular machine learning algorithms with different inputs were compared to the proposed model only based on sEMG. Experiment results demonstrate that the proposed method has the best prediction performance and there are highly significant differences between it and other methods. The difference of prediction results under different gait phases by the proposed method was also analyzed. The results indicate that the prediction effect of support phases is generally better than that of swing phases. Above experimental results show that the proposed method can realize accurate online joint angle prediction and has better performance to promote man-machine cooperation.

Continuous Motion Estimation of Knee Joint Based on a Parameter Self-Updating Mechanism Model

Estimation of continuous motion of human joints using surface electromyography (sEMG) signals has a critical part to play in intelligent rehabilitation. Traditional methods always use sEMG signals as inputs to build regression or biomechanical models to estimate continuous joint motion variables. However, it is challenging to accurately estimate continuous joint motion in new subjects due to the non-stationarity and individual differences in sEMG signals, which greatly limits the generalisability of the method. In this paper, a continuous motion estimation model for the human knee joint with a parameter self-updating mechanism based on the fusion of particle swarm optimization (PSO) and deep belief network (DBN) is proposed. According to the original sEMG signals of different subjects, the method adaptively optimized the parameters of the DBN model and completed the optimal reconstruction of signal feature structure in high-dimensional space to achieve the optimal estimation of continuous joint motion. Extensive experiments were conducted on knee joint motions. The results suggested that the average root mean square errors (RMSEs) of the proposed method were 9.42° and 7.36°, respectively, which was better than the results obtained by common neural networks. This finding lays a foundation for the human-robot interaction (HRI) of the exoskeleton robots based on the sEMG signals.

Effect of Gait Speed on Trajectory Prediction Using Deep Learning Models for Exoskeleton Applications

Gait speed is an important biomechanical determinant of gait patterns, with joint kinematics being influenced by it. This study aims to explore the effectiveness of fully connected neural networks (FCNNs), with a potential application for exoskeleton control, in predicting gait trajectories at varying speeds (specifically, hip, knee, and ankle angles in the sagittal plane for both limbs). This study is based on a dataset from 22 healthy adults walking at 28 different speeds ranging from 0.5 to 1.85 m/s. Four FCNNs (a generalised-speed model, a low-speed model, a high-speed model, and a low-high-speed model) are evaluated to assess their predictive performance on gait speeds included in the training speed range and on speeds that have been excluded from it. The evaluation involves short-term (one-step-ahead) predictions and long-term (200-time-step) recursive predictions. The results show that the performance of the low- and high-speed models, measured using the mean absolute error (MAE), decreased by approximately 43.7% to 90.7% when tested on the excluded speeds. Meanwhile, when tested on the excluded medium speeds, the performance of the low-high-speed model improved by 2.8% for short-term predictions and 9.8% for long-term predictions. These findings suggest that FCNNs are capable of interpolating to speeds within the maximum and minimum training speed ranges, even if not explicitly trained on those speeds. However, their predictive performance decreases for gaits at speeds beyond or below the maximum and minimum training speed ranges.

Lower body kinematics estimation during walking using an accelerometer

Measuring and predicting accurate joint angles are important to developing analytical tools to gauge users' progress. Such measurement is usually performed in laboratory settings, which is difficult and expensive. So, the aim of this study was continuous estimation of lower limb joint angles during walking using an accelerometer and random forest (RF). Thus, 73 subjects (26 women and 47 men) voluntarily participated in this study. The subjects walked at the slow, moderate, and fast speeds on a walkway, which was covered with 10 Vicon camera. Acceleration was used as input for a RF to estimate ankle, knee, and hip angles (in transverse, frontal, and sagittal planes). Pearson correlation coefficient (r) and Mean Square Error (MSE) were computed between the experimental and estimated data. Paired statistical parametric mapping (SPM) t-test was used to compare the experimental and estimated data throughout gait cycle. The results of this study showed that the MSE of joint angles between the experimental and estimated data ranged from 0.04 to 24.29 and r > 0.91. Moreover, the findings of SPM indicated that there was no significant difference between the experimental and estimated data of ankle, knee, and hip angles in all three planes throughout gait cycle. The results of our research developed a more accessible, portable procedure to quantifying lower limb joint angles by an accelerometer and RF. So, such wearable-based joint angles have the potential to be used in outside-laboratory settings to measure walking kinematics.

Lower limb exoskeleton robot and its cooperative control: A review, trends, and challenges for future research

Effective control of an exoskeleton robot (ER) using a human-robot interface is crucial for assessing the robot's movements and the force they produce to generate efficient control signals. Interestingly, certain surveys were done to show off cutting-edge exoskeleton robots. The review papers that were previously published have not thoroughly examined the control strategy, which is a crucial component of automating exoskeleton systems. As a result, this review focuses on examining the most recent developments and problems associated with exoskeleton control systems, particularly during the last few years (2017-2022). In addition, the trends and challenges of cooperative control, particularly multi-information fusion, are discussed.

Demand forecasting model for time-series pharmaceutical data using shallow and deep neural network model

Demand forecasting is a scientific and methodical assessment of future demand for a critical product.The effective Demand Forecast Model (DFM) enables pharmaceutical companies to be successful in the global market. The purpose of this research paper is to validate various shallow and deep neural network methods for demand forecasting, with the aim of recommending sales and marketing strategies based on the trend/seasonal effects of eight different groups of pharmaceutical products with different characteristics. The root mean squared error (RMSE) is used as the predictive accuracy of DFMs. This study also found that the mean RMSE value of the shallow neural network-based DFMs was 6.27 for all drug categories, which was lower than deep neural network models. According to the findings, DFMs based on shallow neural networks can effectively estimate future demand for pharmaceutical products.

Lower Body Joint Angle Prediction Using Machine Learning and Applied Biomechanical Inverse Dynamics

Extreme angles in lower body joints may adversely increase the risk of injury to joints. These injuries are common in the workplace and cause persistent pain and significant financial losses to people and companies. The purpose of this study was to predict lower body joint angles from the ankle to the lumbosacral joint (L5S1) by measuring plantar pressures in shoes. Joint angle prediction was aided by a designed footwear sensor consisting of six force-sensing resistors (FSR) and a microcontroller fitted with Bluetooth LE sensors. An Xsens motion capture system was utilized as a ground truth validation measuring 3D joint angles. Thirty-seven human subjects were tested squatting in an IRB-approved study. The Gaussian Process Regression (GPR) linear regression algorithm was used to create a progressive model that predicted the angles of ankle, knee, hip, and L5S1. The footwear sensor showed a promising root mean square error (RMSE) for each joint. The L5S1 angle was predicted to be RMSE of 0.21° for the X-axis and 0.22° for the Y-axis, respectively. This result confirmed that the proposed plantar sensor system had the capability to predict and monitor lower body joint angles for potential injury prevention and training of occupational workers.

A Hybrid Method Integrating A Musculoskeletal Model with Long Short-Term Memory (LSTM) for Human Motion Prediction

So far, it shows a growing interest in the biomechanics community in the development of wearable technologies and their clinical applications, which enables the diagnosis of movement disorders and design of the rehabilitation interventions. To provide reliable feedback in the human-machine interface for advanced rehabilitation devices, methods to predict motion intention was developed which aim to generate future human motion based on the measured motion. An inertial measurement unit (IMU) is a promising device for motion tracking, with the advantages of low cost and high convenience in sensor placement to measure motion in almost every environment. However, it reveals that few contributions have been devoted to human motion prediction with pure IMU data. Thus, we propose a hybrid method integrating a musculoskeletal (MSK) model and the long short-term memory (LSTM) artificial neural network (ANN) to predict human motion. The proposed method was capable to predict motion in the daily tasks (stand-to-sit-to-stand and walking) for healthy participants: the predicted knee joint angles had an RMSE of 2.93° when compared to measured knee joint angles from the IMU data. The proposed method outperformed the methods based on the ANN/MSK model (RMSE of 31.15°) and LSTM without the integration of the MSK model (RMSE of 31.26°) in the motion prediction. Clinical Relevance- This proposed model based on IMU data alone has the great potential to become a low-cost, easy-to-use alternative in motion prediction to interact with advanced rehabilitation devices in clinical practice.

Measurement, Evaluation, and Control of Active Intelligent Gait Training Systems—Analysis of the Current State of the Art

Gait recognition and rehabilitation has been a research hotspot in recent years due to its importance to medical care and elderly care. Active intelligent rehabilitation and assistance systems for lower limbs integrates mechanical design, sensing technology, intelligent control, and robotics technology, and is one of the effective ways to resolve the above problems. In this review, crucial technologies and typical prototypes of active intelligent rehabilitation and assistance systems for gait training are introduced. The limitations, challenges, and future directions in terms of gait measurement and intention recognition, gait rehabilitation evaluation, and gait training control strategies are discussed. To address the core problems of the sensing, evaluation and control technology of the active intelligent gait training systems, the possible future research directions are proposed. Firstly, different sensing methods need to be proposed for the decoding of human movement intention. Secondly, the human walking ability evaluation models will be developed by integrating the clinical knowledge and lower limb movement data. Lastly, the personalized gait training strategy for collaborative control of human–machine systems needs to be implemented in the clinical applications.

Human Joint Torque Estimation Based on Mechanomyography for Upper Extremity Exosuit

Human intention recognition belongs to the algorithm basis for exoskeleton robots to generate synergic movements and provide corresponding assistance. In this article, we acquire and analyze the mechanomyography (MMG) to estimate the current joint torque and apply this method to the rehabilitation training research of the upper extremity exosuit. In order to obtain relatively pure biological signals, a MMG processing method based on the Hilbert-Huang Transform (HHT) is proposed to eliminate the mixed noise and motion artifacts. After extracting features and forming the dataset, a random forest regression (RFR) model is designed to build the mapping relationship between MMG and human joint output through offline learning. In addition, an upper extremity exosuit is constructed for multi-joint assistance. Based on the above research, we develop a torque estimation-based control strategy and make it responsible for the intention understanding and motion servo of this customized system. Finally, an actual test verifies the accuracy and reliability of this recognition algorithm, and an efficiency evaluation experiment also proves the feasibility for power assistance.

Sensors and Actuation Technologies in Exoskeletons: A Review

Exoskeletons are robots that closely interact with humans and that are increasingly used for different purposes, such as rehabilitation, assistance in the activities of daily living (ADLs), performance augmentation or as haptic devices. In the last few decades, the research activity on these robots has grown exponentially, and sensors and actuation technologies are two fundamental research themes for their development. In this review, an in-depth study of the works related to exoskeletons and specifically to these two main aspects is carried out. A preliminary phase investigates the temporal distribution of scientific publications to capture the interest in studying and developing novel ideas, methods or solutions for exoskeleton design, actuation and sensors. The distribution of the works is also analyzed with respect to the device purpose, body part to which the device is dedicated, operation mode and design methods. Subsequently, actuation and sensing solutions for the exoskeletons described by the studies in literature are analyzed in detail, highlighting the main trends in their development and spread. The results are presented with a schematic approach, and cross analyses among taxonomies are also proposed to emphasize emerging peculiarities.

Enhancing the differentiation of walking and standing via the ratio of plantar pressures

Differentiation of standing and walking based on plantar pressures is helpful in developing strategies to reduce health risks in the workplace. In order to improve the differentiation ability, the paper proposes a new metric for posture differentiation, that is, the pressure ratio on the two anatomical plantar regions. The plantar pressures were collected from 30 persons during walking and standing. After verifying the normal distribution of the pressure ratio by the Monte Carlo method, two-way repeated-measures ANOVA was conducted for the pressure ratios. The advantage of the pressure ratio over two conventional pressure metrics (the average pressure and the peak pressure) is demonstrated by its much larger size effect. Furthermore, the pressure ratio permits to establish value ranges corresponding to walking and standing, which are less influenced by specific person factors, thus facilitating the design of a standardized posture recognition system. The underlying mechanism underlying the pressure ratio is discussed from the aspect of biomechanics of movement.

A Multi-Information Fusion Method for Gait Phase Classification in Lower Limb Rehabilitation Exoskeleton

Gait phase classification is important for rehabilitation training in patients with lower extremity motor dysfunction. Classification accuracy of the gait phase also directly affects the effect and rehabilitation training cycle. In this article, a multiple information (multi-information) fusion method for gait phase classification in lower limb rehabilitation exoskeleton is proposed to improve the classification accuracy. The advantage of this method is that a multi-information acquisition system is constructed, and a variety of information directly related to gait movement is synchronously collected. Multi-information includes the surface electromyography (sEMG) signals of the human lower limb during the gait movement, the angle information of the knee joints, and the plantar pressure information. The acquired multi-information is processed and input into a modified convolutional neural network (CNN) model to classify the gait phase. The experiment of gait phase classification with multi-information is carried out under different speed conditions, and the experiment is analyzed to obtain higher accuracy. At the same time, the gait phase classification results of multi-information and single information are compared. The experimental results verify the effectiveness of the multi-information fusion method. In addition, the delay time of each sensor and model classification time is measured, which shows that the system has tremendous real-time performance.

Path Planning and Impedance Control of a Soft Modular Exoskeleton for Coordinated Upper Limb Rehabilitation

The coordinated rehabilitation of the upper limb is important for the recovery of the daily living abilities of stroke patients. However, the guidance of the joint coordination model is generally lacking in the current robot-assisted rehabilitation. Modular robots with soft joints can assist patients to perform coordinated training with safety and compliance. In this study, a novel coordinated path planning and impedance control method is proposed for the modular exoskeleton elbow-wrist rehabilitation robot driven by pneumatic artificial muscles (PAMs). A convolutional neural network-long short-term memory (CNN-LSTM) model is established to describe the coordination relationship of the upper limb joints, so as to generate adaptive trajectories conformed to the coordination laws. Guided by the planned trajectory, an impedance adjustment strategy is proposed to realize active training within a virtual coordinated tunnel to achieve the robot-assisted upper limb coordinated training. The experimental results showed that the CNN-LSTM hybrid neural network can effectively quantify the coordinated relationship between the upper limb joints, and the impedance control method ensures that the robotic assistance path is always in the virtual coordination tunnel, which can improve the movement coordination of the patient and enhance the rehabilitation effectiveness.

Estimation of Knee Joint Angle Using Textile Capacitive Sensor and Artificial Neural Network Implementing with Three Shoe Types at Two Gait Speeds: A Preliminary Investigation

The lower limb joints might be affected by different shoe types and gait speeds. Monitoring joint angles might require skill and proper technique to obtain accurate data for analysis. We aimed to estimate the knee joint angle using a textile capacitive sensor and artificial neural network (ANN) implementing with three shoe types at two gait speeds. We developed a textile capacitive sensor with a simple structure design and less costly placing in insole shoes to measure the foot plantar pressure for building the deep learning models. The smartphone was used to video during walking at each condition, and Kinovea was applied to calibrate the knee joint angle. Six ANN models were created; three shoe-based ANN models, two speed-based ANN models, and one ANN model that used datasets from all experiment conditions to build a model. All ANN models at comfortable and fast gait provided a high correlation efficiency (0.75 to 0.97) with a mean relative error lower than 15% implement for three testing shoes. And compare the ANN with A convolution neural network contributes a similar result in predict the knee joint angle. A textile capacitive sensor is reliable for measuring foot plantar pressure, which could be used with the ANN algorithm to predict the knee joint angle even using high heel shoes.

Continuous Estimation of Knee Joint Angle Based on Surface Electromyography Using a Long Short-Term Memory Neural Network and Time-Advanced Feature

Continuous joint angle estimation based on a surface electromyography (sEMG) signal can be used to improve the man-machine coordination performance of the exoskeleton. In this study, we proposed a time-advanced feature and utilized long short-term memory (LSTM) with a root mean square (RMS) feature and its time-advanced feature (RMSTAF; collectively referred to as RRTAF) of sEMG to estimate the knee joint angle. To evaluate the effect of joint angle estimation, we used root mean square error (RMSE) and cross-correlation coefficient ρ between the estimated angle and actual angle. We also compared three methods (i.e., LSTM using RMS, BPNN (back propagation neural network) using RRTAF, and BPNN using RMS) with LSTM using RRTAF to highlight its good performance. Five healthy subjects participated in the experiment and their eight muscle (i.e., rectus femoris (RF), biceps femoris (BF), semitendinosus (ST), gracilis (GC), semimembranosus (SM), sartorius (SR), medial gastrocnemius (MG), and tibialis anterior (TA)) sEMG signals were taken as algorithm inputs. Moreover, the knee joint angles were used as target values. The experimental results showed that, compared with LSTM using RMS, BPNN using RRTAF, and BPNN using RMS, the average RMSE values of LSTM using RRTAF were respectively reduced by 8.57%, 46.62%, and 68.69%, whereas the average ρ values were respectively increased by 0.31%, 4.15%, and 18.35%. The results demonstrated that LSTM using RRTAF, which contained the time-advanced feature, had better performance for estimating the knee joint motion.