Inter-Subject Domain Adaptation for CNN-Based Wrist Kinematics Estimation Using sEMG

Recurrent Neural Network Enabled Continuous Motion Estimation of Lower Limb Joints From Incomplete sEMG Signals

Decoding continuous human motion from surface electromyography (sEMG) in advance is crucial for improving the intelligence of exoskeleton robots. However, incomplete sEMG signals are prevalent on account of unstable data transmission, sensor malfunction, and electrode sheet detachment. These non-ideal factors severely compromise the accuracy of continuous motion recognition and the reliability of clinical applications. To tackle this challenge, this paper develops a multi-task parallel learning framework for continuous motion estimation with incomplete sEMG signals. Concretely, a residual network is incorporated into a recurrent neural network to integrate the information flow of hidden states and reconstruct random and consecutive missing sEMG signals. The attention mechanism is applied for redistributing the distribution of weights. A jointly optimized loss function is devised to enable training the model for simultaneously dealing with signal anomalies/absences and multi-joint continuous motion estimation. The proposed model is implemented for estimating hip, knee, and ankle joint angles of physically competent individuals and patients during diverse exercises. Experimental results indicate that the estimation root-mean-square errors with 60% missing sEMG signals steadily converges to below 5 degrees. Even with multi-channel electrode sheet shedding, our model still demonstrates cutting-edge estimation performance, errors only marginally increase 1 degree.

A Novel Active Learning Framework for Cross-Subject Human Activity Recognition from Surface Electromyography

Wearable sensor-based human activity recognition (HAR) methods hold considerable promise for upper-level control in exoskeleton systems. However, such methods tend to overlook the critical role of data quality and still encounter challenges in cross-subject adaptation. To address this, we propose an active learning framework that integrates the relation network architecture with data sampling techniques. Initially, target data are used to fine tune two auxiliary classifiers of the pre-trained model, thereby establishing subject-specific classification boundaries. Subsequently, we assess the significance of the target data based on classifier discrepancy and partition the data into sample and template sets. Finally, the sampled data and a category clustering algorithm are employed to tune model parameters and optimize template data distribution, respectively. This approach facilitates the adaptation of the model to the target subject, enhancing both accuracy and generalizability. To evaluate the effectiveness of the proposed adaptation framework, we conducted evaluation experiments on a public dataset and a self-constructed electromyography (EMG) dataset. Experimental results demonstrate that our method outperforms the compared methods across all three statistical metrics. Furthermore, ablation experiments highlight the necessity of data screening. Our work underscores the practical feasibility of implementing user-independent HAR methods in exoskeleton control systems.

Use of Artificial Intelligence Techniques to Assist Individuals with Physical Disabilities

Assistive technologies (AT) enable people with disabilities to perform activities of daily living more independently, have greater access to community and healthcare services, and be more productive performing educational and/or employment tasks. Integrating artificial intelligence (AI) with various agents, including electronics, robotics, and software, has revolutionized AT, resulting in groundbreaking technologies such as mind-controlled exoskeletons, bionic limbs, intelligent wheelchairs, and smart home assistants. This article provides a review of various AI techniques that have helped those with physical disabilities, including brain-computer interfaces, computer vision, natural language processing, and human-computer interaction. The current challenges and future directions for AI-powered advanced technologies are also addressed.

Across Sessions and Subjects Domain Adaptation for Building Robust Myoelectric Interface

Gesture interaction via surface electromyography (sEMG) signal is a promising approach for advanced human-computer interaction systems. However, improving the performance of the myoelectric interface is challenging due to the domain shift caused by the signal's inherent variability. To enhance the interface's robustness, we propose a novel adaptive information fusion neural network (AIFNN) framework, which could effectively reduce the effects of multiple scenarios. Specifically, domain adversarial training is established to inhibit the shared network's weights from exploiting domain-specific representation, thus allowing for the extraction of domain-invariant features. Effectively, classification loss, domain diversence loss and domain discrimination loss are employed, which improve classification performance while reduce distribution mismatches between the two domains. To simulate the application of myoelectric interface, experiments were carried out involving three scenarios (intra-session, inter-session and inter-subject scenarios). Ten non-disabled subjects were recruited to perform sixteen gestures for ten consecutive days. The experimental results indicated that the performance of AIFNN was better than two other state-of-the-art transfer learning approaches, namely fine-tuning (FT) and domain adversarial network (DANN). This study demonstrates the capability of AIFNN to maintain robustness over time and generalize across users in practical myoelectric interface implementations. These findings could serve as a foundation for future deployments.

Continuous Motion Intention Prediction Using sEMG for Upper-Limb Rehabilitation: A Systematic Review of Model-Based and Model-Free Approaches

Upper limb functional impairments persisting after stroke significantly affect patients' quality of life. Precise adjustment of robotic assistance levels based on patients' motion intentions using sEMG signals is crucial for active rehabilitation. This paper systematically reviews studies on continuous prediction of upper limb single joints and multi-joint combinations motion intention using Model-Based (MB) and Model-Free (MF) approaches over the past decade, based on 186 relevant studies screened from six major electronic databases. The findings indicate ongoing challenges in terms of subject composition, algorithm robustness and generalization, and algorithm feasibility for practical applications. Moreover, it suggests integrating the strengths of both MB and MF approaches to improve existing algorithms. Therefore, future research should further explore personalized MB-MF combination methods incorporating deep learning, attention mechanisms, muscle synergy features, motor unit features, and closed-loop feedback to achieve precise, real-time, and long-duration prediction of multi-joint complex movements, while further refining the transfer learning strategy for rapid algorithm deployment across days and subjects. Overall, this review summarizes the current research status, significant findings, and challenges, aiming to inspire future research on predicting upper limb motion intentions based on sEMG.

Random forest-based simultaneous and proportional myoelectric control system for finger movements

A classification scheme for myoelectric control systems (MCS) cannot mimic complex hand movements. This paper presents simultaneous and proportional MCS by estimating the angles of fourteen finger joints using time-domain feature extraction and random forest. The experimental results show that the best feature was the root mean square (RMS). Furthermore, the random forest attained an average coefficient of determination (R2) of 0.85 compared to other regressors which perform below 0.75. The ANOVA tests indicated that the performance of the proposed system was significantly different. Therefore, the proposed system will be the best option for real-time MCS applications in the future.

Continuous joint velocity estimation using CNN-based deep learning for multi-DoF prosthetic wrist for activities of daily living

Introduction

Myoelectric control of prostheses is a long-established technique, using surface electromyography (sEMG) to detect user intention and perform subsequent mechanical actions. Most machine learning models utilized in control systems are trained using isolated movements that do not reflect the natural movements occurring during daily activities. Moreover, movements are often affected by arm postures, the duration of activities, and personal habits. It is crucial to have a control system for multi-degree-of-freedom (DoF) prosthetic arms that is trained using sEMG data collected from activities of daily living (ADL) tasks.

Method

This work focuses on two major functional wrist movements: pronation-supination and dart-throwing movement (DTM), and introduces a new wrist control system that directly maps sEMG signals to the joint velocities of the multi-DoF wrist. Additionally, a specific training strategy (Quick training) is proposed that enables the controller to be applied to new subjects and handle situations where sensors may displace during daily living, muscles can become fatigued, or sensors can become contaminated (e.g., due to sweat). The prosthetic wrist controller is designed based on data from 24 participants and its performance is evaluated using the Root Mean Square Error (RMSE) and Pearson Correlation.

Result

The results are found to depend on the characteristics of the tasks. For example, tasks with dart-throwing motion show smaller RSME values (Hammer: 6.68 deg/s and Cup: 7.92 deg/s) compared to tasks with pronation-supination (Bulb: 43.98 deg/s and Screw: 53.64 deg/s). The proposed control technique utilizing Quick training demonstrates a decrease in the average root mean square error (RMSE) value by 35% and an increase in the average Pearson correlation value by 40% across all four ADL tasks.

Cross-subject EMG hand gesture recognition based on dynamic domain generalization

Electromyography (EMG) signal based cross-subject gesture recognition methods reduce the influence of individual differences using transfer learning technology. These methods generally require calibration data collected from new subjects to adapt the pre-trained model to existing subjects. However, collecting calibration data is usually trivial and inconvenient for new subjects. This is currently a major obstacle to the daily use of hand gesture recognition based on EMG signals. To tackle the problem, we propose a novel dynamic domain generalization (DDG) method which is able to achieve accurate recognition on the hand gesture of new subjects without any calibration data. In order to extract more robust and adaptable features, a meta-adjuster is leveraged to generate a series of template coefficients to dynamically adjust dynamic network parameters. Specifically, two different kinds of templates are designed, in which the first one is different kinds of features, such as temporal features, spatial features, and spatial-temporal features, and the second one is different normalization layers. Meanwhile, a mix-style data augmentation method is introduced to make the meta-adjuster's training data more diversified. Experimental results on a public dataset verify that the proposed DDG outperforms the counterpart methods.

Sensor-Based Motion Tracking System Evaluation for RULA in Assembly Task

Industries need a mechanism to monitor the workers' safety and to prevent Work-related Musculoskeletal Disorders (WMSDs). The development of ergonomics assessment tools helps the industry evaluate workplace design and worker posture. Many studies proposed the automated ergonomics assessment method to replace the manual; however, it only focused on calculating body angle and assessing the wrist section manually. This study aims to (a) propose a wrist kinematics measurement based on unobtrusive sensors, (b) detect potential WMSDs related to wrist posture, and (c) compare the wrist posture of subjects while performing assembly tasks to achieve a comprehensive and personalized ergonomic assessment. The wrist posture measurement is combined with the body posture measurement to provide a comprehensive ergonomics assessment based on RULA. Data were collected from subjects who performed the assembly process to evaluate our method. We compared the risk score assessed by the ergonomist and the risk score generated by our method. All body segments achieved more than an 80% similarity score, enhancing the scores for wrist position and wrist twist by 6.8% and 0.3%, respectively. A hypothesis analysis was conducted to evaluate the difference across the subjects. The results indicate that every subject performs tasks differently and has different potential risks regarding wrist posture.

Estimation of knee joint movement using single-channel sEMG signals with a feature-guided convolutional neural network

Estimating human motion intention, such as intent joint torque and movement, plays a crucial role in assistive robotics for ensuring efficient and safe human-robot interaction. For coupled human-robot systems, surface electromyography (sEMG) signal has been proven as an effective means for estimating human's intended movements. Usually, joint movement estimation uses sEMG signals measured from multiple muscles and needs many sEMG sensors placed on the human body, which may cause discomfort or result in mechanical/signal interference from wearable robots/environment during long-term routine use. Although the muscle synergy principle implies that it is possible to estimate human motion using sEMG signals from even one signal muscle, few studies investigated the feasibility of continuous motion estimation based on single-channel sEMG. In this study, a feature-guided convolutional neural network (FG-CNN) has been proposed to estimate human knee joint movement using single-channel sEMG. In the proposed FG-CNN, several handcrafted features have been fused into a CNN model to guide CNN feature extraction, and both handcrafted and CNN-extracted features were applied to a regression model, i.e., random forest regression, to estimate knee joint movements. Experiments with 8 healthy subjects were carried out, and sEMG signals measured from 6 muscles, i.e., vastus lateralis, vastus medialis, biceps femoris, semitendinosus, lateral or medial gastrocnemius (LG or MG), were separately evaluated for knee joint estimation using the proposed method. The experimental results demonstrated that the proposed FG-CNN method with single-channel sEMG signals from LG or MG can effectively estimate human knee joint movements. The average correlation coefficient between the measured and the estimated knee joint movements is 0.858 ± 0.085 for LG and 0.856 ± 0.057 for MG. Meanwhile, comparative studies showed that the combined handcrafted-CNN features outperform either the handcrafted features or the CNN features; the performance of the proposed signal-channel sEMG-based FG-CNN method is comparable to those of the traditional multi-channel sEMG-based methods. The outcomes of this study enable the possibility of developing a single-channel sEMG-based human-robot interface for knee joint movement estimation, which can facilitate the routine use of assistive robots.

SeNic: An Open Source Dataset for sEMG-Based Gesture Recognition in Non-ideal Conditions

In order to reduce the gap between the laboratory environment and actual use in daily life of human-machine interaction based on surface electromyogram (sEMG) intent recognition, this paper presents a benchmark dataset of sEMG in non-ideal conditions (SeNic). The dataset mainly consists of 8-channel sEMG signals, and electrode shifts from an 3D-printed annular ruler. A total of 36 subjects participate in our data acquisition experiments of 7 gestures in non-ideal conditions, where non-ideal factors of 1) electrode shifts, 2) individual difference, 3) muscle fatigue, 4) inter-day difference, and 5) arm postures are elaborately involved. Signals of sEMG are validated first in temporal and frequency domains. Results of recognizing gestures in ideal conditions indicate the high quality of the dataset. Adverse impacts in non-ideal conditions are further revealed in the amplitudes of these data and recognition accuracies. To be concluded, SeNic is a benchmark dataset that introduces several non-ideal factors which often degrade the robustness of sEMG-based systems. It could be used as a freely available dataset and a common platform for researchers in the sEMG-based recognition community. The benchmark dataset SeNic are available online via the website3.

Towards Robust, Adaptive and Reliable Upper-limb Motion Estimation Using Machine Learning and Deep Learning--A Survey in Myoelectric Control

To develop multi-functional human-machine interfaces that can help disabled people reconstruct lost functions of upper-limbs, machine learning (ML) and deep learning (DL) techniques have been widely implemented to decode human movement intentions from surface electromyography (sEMG) signals. However, due to the high complexity of upper-limb movements and the inherent non-stable characteristics of sEMG, the usability of ML/DL based control schemes is still greatly limited in practical scenarios. To this end, tremendous efforts have been made to improve model robustness, adaptation, and reliability. In this article, we provide a systematic review on recent achievements, mainly from three categories: multi-modal sensing fusion to gain additional information of the user, transfer learning (TL) methods to eliminate domain shift impacts on estimation models, and post-processing approaches to obtain more reliable outcomes. Special attention is given to fusion strategies, deep TL frameworks, and confidence estimation. \textcolor{red}{Research challenges and emerging opportunities, with respect to hardware development, public resources, and decoding strategies, are also analysed to provide perspectives for future developments.

Action Recognition of Lower Limbs Based on Surface Electromyography Weighted Feature Method

To improve the recognition rate of lower limb actions based on surface electromyography (sEMG), an effective weighted feature method is proposed, and an improved genetic algorithm support vector machine (IGA-SVM) is designed in this paper. First, for the problem of high feature redundancy and low discrimination in the surface electromyography feature extraction process, the weighted feature method is proposed based on the correlation between muscles and actions. Second, to solve the problem of the genetic algorithm selection operator easily falling into a local optimum solution, the improved genetic algorithm-support vector machine is designed by championship with sorting method. Finally, the proposed method is used to recognize six types of lower limb actions designed, and the average recognition rate reaches 94.75%. Experimental results indicate that the proposed method has definite potentiality in lower limb action recognition.

Multi-Joint Angles Estimation of Forearm Motion Using a Regression Model

To improve the life quality of forearm amputees, prosthetic hands with high accuracy, and robustness are necessary. The application of surface electromyography (sEMG) signals to control a prosthetic hand is challenging. In this study, we proposed a time-domain CNN model for the regression prediction of joint angles in three degrees of freedom (3-DOFs, include two wrist joint motion and one finger joint motion), and five-fold cross validation was used to evaluate the correlation coefficient (CC). The CC value results of wrist flexion/extension motion obtained from 10 participants was 0.87–0.92, pronation/supination motion was 0.72–0.95, and hand grip/open motion was 0.75–0.94. We backtracked the fully connected layer weights to create a geometry plot for analyzing the motion pattern to investigate the learning of the proposed model. In order to discuss the daily updateability of the model by transfer learning, we performed a second experiment on five of the participants in another day and conducted transfer learning based on smaller amount of dataset. The CC results improved (wrist flexion/extension was 0.90–0.97, pronation/supination was 0.84–0.96, hand grip/open was 0.85–0.92), suggesting the effectiveness of the transfer learning by incorporating the small amounts of sEMG data acquired in different days. We compared our CNN-based model with four conventional regression models, the result illustrates that proposed model significantly outperforms the four conventional models with and without transfer learning. The offline result suggests the reliability of the proposed model in real-time control in different days, it can be applied for real-time prosthetic control in the future.