Evaluation of Synergy-Based Hand Gesture Recognition Method Against Force Variation for Robust Myoelectric Control

Concurrent and continuous estimation of multi-finger forces by synergy mapping and reconstruction: a pilot study

Objective.The absence of intuitive control in present myoelectric interfaces makes it a challenge for users to communicate with assistive devices efficiently in real-world conditions. This study aims to tackle this difficulty by incorporating neurophysiological entities, namely muscle and force synergies, onto multi-finger force estimation to allow intuitive myoelectric control.Approach. Eleven healthy subjects performed six isometric grasping tasks at three muscle contraction levels. The exerted fingertip forces were collected concurrently with the surface electromyographic (sEMG) signals from six extrinsic and intrinsic muscles of hand. Muscle synergies were then extracted from recorded sEMG signals, while force synergies were identified from measured force data. Afterwards, a linear regressor was trained to associate the two types of synergies. This would allow us to predict multi-finger forces simply by multiplying the activation signals derived from muscle synergies with the weighting matrix of initially identified force synergies. To mitigate the false activation of unintended fingers, the force predictions were finally corrected by a finger state recognition procedure.Main results. We found that five muscle synergies and four force synergies are able to make a tradeoff between the computation load and the prediction accuracy for the proposed model; When trained and tested on all six grasping tasks, our method (SYN-II) achieved better performance (R2= 0.80 ± 0.04, NRMSE = 0.19 ± 0.01) than conventional sEMG amplitude-based method; Interestingly, SYN-II performed better than all other methods when tested on two unknown tasks outside the four training tasks (R2= 0.74 ± 0.03, NRMSE = 0.22 ± 0.02), which indicated better generalization ability.Significance. This study shows the first attempt to link between muscle and force synergies to allow concurrent and continuous estimation of multi-finger forces from sEMG. The proposed approach may lay the foundation for high-performance myoelectric interfaces that allow users to control robotic hands in a more natural and intuitive manner.

Cross-user gesture recognition from sEMG signals using an optimal transport assisted student-teacher framework

The cross-user gesture recognition is a puzzle in the myoelectric control system, owing to great variability in muscle activities across different users. To address this problem, a novel optimal transport (OT) assisted student-teacher (ST) framework (termed OT-ST) was proposed in this paper to facilitate transfer across user domains in an unsupervised domain adaptation (UDA) manner. In this framework, the initial parameters of the ST models were trained with the labeled data from users in the source domain. In the model transfer stage for a new user in the target domain, the teacher model was utilized to generate pseudo labels for unlabeled testing samples, providing guidance to the adaptation of the student model. The OT algorithm was employed to optimize the pseudo labels generated from the teacher model, avoiding the model bias and further improving the effect of domain adaptation. The performance of the proposed OT-ST framework was evaluated via experiments of classifying seven hand gestures using high-density surface electromyogram (HD-sEMG) recordings from extensor digitorum muscles of eight intact-limbed subjects. The OT-ST framework yielded a high accuracy of 96.50 ± 2.88% for new users, and outperformed other common machine learning and UDA methods significantly (p < 0.01), demonstrating its effectiveness. The OT-ST framework does not require special repetitive training or any labeled data for calibration. In addition, it can incrementally learn from new testing samples and improve the recognition ability. This study provides a promising method for developing user-generic myoelectric pattern recognition, with wide applications in human-computer interaction, consumer electronics and prosthesis control.

A CNN-LSTM model for six human ankle movements classification on different loads

This study aims to address three problems in current studies in decoding the ankle movement intention for robot-assisted bilateral rehabilitation using surface electromyogram (sEMG) signals: (1) only up to four ankle movements could be identified while six ankle movements should be classified to provide better training; (2) feeding the raw sEMG signals directly into the neural network leads to high computational cost; and (3) load variation has large influence on classification accuracy. To achieve this, a convolutional neural network (CNN)-long short-term memory (LSTM) model, a time-domain feature selection method of the sEMG, and a two-step method are proposed. For the first time, the Boruta algorithm is used to select time-domain features of sEMG. The selected features, rather than raw sEMG signals are fed into the CNN-LSTM model. Hence, the number of model's parameters is reduced from 331,938 to 155,042, by half. Experiments are conducted to validate the proposed method. The results show that our method could classify six ankle movements with relatively good accuracy (95.73%). The accuracy of CNN-LSTM, CNN, and LSTM models with sEMG features as input are all higher than that of corresponding models with raw sEMG as input. The overall accuracy is improved from 73.23% to 93.50% using our two-step method for identifying the ankle movements with different loads. Our proposed CNN-LSTM model have the highest accuracy for ankle movements classification compared with CNN, LSTM, and Support Vector Machine (SVM).

User-Independent EMG Gesture Recognition Method Based on Adaptive Learning

In a gesture recognition system based on surface electromyogram (sEMG) signals, the recognition model established by existing users cannot directly generalize to the across-user scenarios due to the individual variability of sEMG signals. In this article, we propose an adaptive learning method to handle the problem. The muscle synergy is chosen as the feature vector because it can well-characterize the neural origin of movement. The initial train set is composed of representative samples extracted from the synergy matrix of the existing user. When the new users use the system, the label is obtained by the adaptive K nearest neighbor algorithm (KNN). The recognition process does not require the pre-experiment for new users due to the introduction of adaptive learning strategy, namely, the qualified data and the label of new user data evaluated by a risk evaluator are used to update the train set and KNN weights, so as to adapt to the new users. We have tested the algorithm in DB1 and DB5 of Ninapro databases. The average recognition accuracy is 68.04, 73.35, and 83.05% for different types of gestures, respectively, achieving the effects of the user-dependent method. Our study can avoid the re-training steps and the recognition performance will improve with the increased frequency of uses, which will further facilitate the widespread implementation of sEMG control systems using pattern recognition techniques.