Intra- and inter-channel deep convolutional neural network with dynamic label smoothing for multichannel biosignal analysis

Efficient processing of multichannel biosignals has significant application values in the fields of healthcare and human-machine interaction. Although previous research has achieved high recognition performance with deep convolutional neural networks, several key challenges still remain: (1) Effective extraction of spatial and temporal features from the multichannel biosignals. (2) Appropriate trade-off between performance and complexity for improving applicability in real-life situations given that traditional machine learning and 2D-based CNN approaches often involve excessive preprocessing steps or model parameters; and (3) Generalization ability of neural networks to compensate for domain difference and to reduce overfitting during training process. To address challenges 1 and 2, we propose a 1D-based deep intra and inter channel (I2C) convolution neural network. The I2C convolutional block is introduced to replace the standard convolutional layer, further extending it to several state-of-the-art modules, with the intent of extracting more effective features from multichannel biosignals with fewer parameters. To address challenge 3, we integrate a branch model into the main model to perform dynamic label smoothing, enabling the model to learn domain difference and improve its generalization ability. Experiments were conducted on three public multichannel biosignals databases, namely ISRUC-S3, HEF and Ninapro-DB1. The results suggest that the proposed method exhibits significant competitive advantages in accuracy, complexity, and generalization ability.

Research on Upper Limb Motion Intention Classification and Rehabilitation Robot Control Based on sEMG

sEMG is a non-invasive biomedical engineering technique that can detect and record electrical signals generated by muscles, reflecting both motor intentions and the degree of muscle contraction. This study aims to classify and recognize nine types of upper limb motor intentions based on surface electromyography (sEMG) and apply them to the interactive control of an end-effector rehabilitation robot. The research begins with selecting muscles and data preprocessing, incorporating the generation mechanism of sEMG along with the anatomical and kinesiological principles of upper limb muscles. Next, a musculoskeletal model of the upper limb is established and validated through simulations in OpenSim. To avoid the drawbacks of modeling methods, traditional machine learning and deep learning methods are employed to perform a nine-class classification task on the sEMG data, comparing the classification accuracy of different approaches. Finally, the motor intentions extracted using a multi-stream convolutional neural network (MLCNN) are utilized to control the iReMo® end-effector rehabilitation robot, with the system's motion smoothness and accuracy evaluated through tests involving different trajectories.

Multimodal optimal matching and augmentation method for small sample gesture recognition

In human-computer interaction, gesture recognition based on physiological signals offers advantages such as a more natural and fast interaction mode and less constrained by the environment than visual-based. Surface electromyography-based gesture recognition has significantly progressed. However, since individuals have physical differences, researchers must collect data multiple times from each user to train the deep learning model. This data acquisition process can be particularly burdensome for non-healthy users. Researchers are currently exploring transfer learning and data augmentation techniques to enhance the accuracy of small-sample gesture recognition models. However, challenges persist, such as negative transfer and limited diversity in training samples, leading to suboptimal recognition performance. Therefore, We introduce motion information into sEMG-based recognition and propose a multimodal optimal matching and augmentation method for small sample gesture recognition, achieving efficient gesture recognition with only one acquisition per gesture. Firstly, this method utilizes the optimal matching signal selection module to select the most similar signals from the existing data to the new user as the training set, reducing inter-domain differences. Secondly, the similarity calculation augmentation module enhances the diversity of the training set. Finally, the Modal-type embedding enhances the information interaction between each mode signal. We evaluated the effectiveness on Self-collected Stroke Patient, the Ninapro DB1 dataset and the Ninapro DB5 dataset and achieved accuracies of 93.69%, 91.65% and 98.56%, respectively. These results demonstrate that the method achieved performance comparable to traditional recognition models while significantly reducing the collected data.

TCNN-KAN: Optimized CNN by Kolmogorov-Arnold Network and Pruning Techniques for sEMG Gesture Recognition

Surface electromyography (sEMG) is a non-invasive technique that records the electrical signals generated by muscle activity. sEMG signals are widely used in the field of biomedical and health informatics for diagnosing and monitoring neuromuscular disorders, as well as in fields such as motor control, rehabilitation, and human-computer interaction. In this paper, we propose a novel model called the Triple Convolutional Neural Network and Kolmogorov-Arnold Network (TCNN-KAN) for recognizing gesture signals based on sEMG. Our approach replaces the commonly used fully connected layer with the KAN, parameterizing it as a spline function to improve classification accuracy. Specifically, when using a KAN instead, generate the TCNN-KAN-1 model. When using two KAN layers, generate the TCNN-KAN-2 model and generate the TCNN-KAN-3 model when KAN replaces all fully connected layers. Firstly, to ensure the model learns universal features, we fuse gesture signals from different individuals and segment them to create uniform window sizes. Then, the processed signal is input into the basic convolution layer of different depths for training. In order to improve the accuracy, we convert the standard fully connected layer in the convolutional layer to the KAN layer so that it has a learnable activation function in weight. Finally, we introduce unstructured pruning to reduce computational complexity and minimize overfitting by removing channels with lower feature importance. We use three datasets, NinaPro DB1, NinaPro DB5, and CSL, for evaluation. The results show that on the TCNN-KAN-2 model, each dataset has achieved the highest accuracy. Specifically, when the pruning rates were 0.2, 0.1, and 0.4, the accuracy rates reached 98.38%, 93.81%, and 75.56%, respectively.

Multi-scale attention patching encoder network: a deployable model for continuous estimation of hand kinematics from surface electromyographic signals

BACKGROUND

Simultaneous and proportional control (SPC) based on surface electromyographic (sEMG) signals has emerged as a research hotspot in the field of human-machine interaction (HMI). However, the existing continuous motion estimation methods mostly have an average Pearson coefficient (CC) of less than 0.85, while high-precision methods suffer from the problem of long inference time (> 200 ms) and can only estimate SPC of less than 15 hand movements, which limits their applications in HMI. To overcome these problems, we propose a smooth Multi-scale Attention Patching Encoder Network (sMAPEN).

METHODS

The sMAPEN consists of three modules, the Multi-scale Attention Fusion (MAF) module, the Patching Encoder (PE) module, and a smoothing layer. The MAF module adaptively captures the local spatiotemporal features at multiple scales, the PE module acquires the global spatiotemporal features of sEMG, and the smoothing layer further improves prediction stability.

RESULTS

To evaluate the performance of the model, we conducted continuous estimation of 40 subjects performing over 40 different hand movements on the Ninapro DB2. The results show that the average Pearson correlation coefficient (CC), normalized root mean square error (NRMSE), coefficient of determination (R2), and smoothness (SMOOTH) of the sMAPEN model are 0.9082, 0.0646°, 0.8163, and - 0.0017, respectively, which significantly outperforms that of the state-of-the-art methods in all metrics (p < 0.01). Furthermore, we tested the deployment performance of sMAPEN on the portable device, with a delay of only 97.93 ms.

CONCLUSIONS

Our model can predict up to 40 hand movements while achieving the highest predicting accuracy compared with other methods. Besides, the lightweight design strategy brings an improvement in inference speed, which enables the model to be deployed on wearable devices. All these promotions imply that sMAPEN holds great potential in HMI.

An efficient surface electromyography-based gesture recognition algorithm based on multiscale fusion convolution and channel attention

In the field of rehabilitation, although deep learning have been widely used in multitype gesture recognition via surface electromyography (sEMG), their higher algorithmic complexity often leads to low computationally inefficient, which compromise their practicality. To achieve more efficient multitype recognition, We propose the Residual-Inception-Efficient (RIE) model, which integrates Inception and efficient channel attention (ECA). The Inception, which is a multiscale fusion convolutional module, is adopted to enhance the ability to extract sEMG features. It uses fast dimensionality reduction, asymmetric convolution decomposition, and pooling to suppress the accumulation of parameters, then reducing the algorithmic complexity; The ECA is adopted to reweight the output features of Inception in different channels, enabling the RIE model to focus on information that is more relevant to gestures. 52-, 49-, and 52-class gesture recognition experiments are conducted on NinaPro DB1, DB3, and DB4 datasets, which contain 14,040, 3234, and 3120 gesture samples, respectively. RIE model proposed in this study achieves accuracies of 88.27%, 69.52%, and 84.55% on the three datasets, exhibiting excellent recognition accuracy and strong generalization ability. Moreover, this method reduces the algorithmic complexity from both spatial and temporal aspects, rendering it smaller in size and faster in computation compared to other lightweight algorithms. Therefore, the proposed RIE model possesses both lightweight computational requirements and reliable performance, providing an efficient deep learning method for gesture recognition based on sEMG.

Hand gesture recognition using sEMG signals with a multi-stream time-varying feature enhancement approach

Hand gesture recognition based on sparse multichannel surface electromyography (sEMG) still poses a significant challenge to deployment as a muscle-computer interface. Many researchers have been working to develop an sEMG-based hand gesture recognition system. However, the existing system still faces challenges in achieving satisfactory performance due to ineffective feature enhancement, so the prediction is erratic and unstable. To comprehensively tackle these challenges, we introduce a novel approach: a lightweight sEMG-based hand gesture recognition system using a 4-stream deep learning architecture. Each stream strategically combines Temporal Convolutional Network (TCN)-based time-varying features with Convolutional Neural Network (CNN)-based frame-wise features. In the first stream, we harness the power of the TCN module to extract nuanced time-varying temporal features. The second stream integrates a hybrid Long short-term memory (LSTM)-TCN module. This stream extracts temporal features using LSTM and seamlessly enhances them with TCN to effectively capture intricate long-range temporal relations. The third stream adopts a spatio-temporal strategy, merging the CNN and TCN modules. This integration facilitates concurrent comprehension of both spatial and temporal features, enriching the model's understanding of the underlying dynamics of the data. The fourth stream uses a skip connection mechanism to alleviate potential problems of data loss, ensuring a robust information flow throughout the network and concatenating the 4 stream features, yielding a comprehensive and effective final feature representation. We employ a channel attention-based feature selection module to select the most effective features, aiming to reduce the computational complexity and feed them into the classification module. The proposed model achieves an average accuracy of 94.31% and 98.96% on the Ninapro DB1 and DB9 datasets, respectively. This high-performance accuracy proves the superiority of the proposed model, and its implications extend to enhancing the quality of life for individuals using prosthetic limbs and advancing control systems in the field of robotic human-machine interfaces.

Big data in myoelectric control: large multi-user models enable robust zero-shot EMG-based discrete gesture recognition

Myoelectric control, the use of electromyogram (EMG) signals generated during muscle contractions to control a system or device, is a promising input, enabling always-available control for emerging ubiquitous computing applications. However, its widespread use has historically been limited by the need for user-specific machine learning models because of behavioural and physiological differences between users. Leveraging the publicly available 612-user EMG-EPN612 dataset, this work dispels this notion, showing that true zero-shot cross-user myoelectric control is achievable without user-specific training. By taking a discrete approach to classification (i.e., recognizing the entire dynamic gesture as a single event), a classification accuracy of 93.0% for six gestures was achieved on a set of 306 unseen users, showing that big data approaches can enable robust cross-user myoelectric control. By organizing the results into a series of mini-studies, this work provides an in-depth analysis of discrete cross-user models to answer unknown questions and uncover new research directions. In particular, this work explores the number of participants required to build cross-user models, the impact of transfer learning for fine-tuning these models, and the effects of under-represented end-user demographics in the training data, among other issues. Additionally, in order to further evaluate the performance of the developed cross-user models, a completely new dataset was created (using the same recording device) that includes known covariate factors such as cross-day use and limb-position variability. The results show that the large data models can effectively generalize to new datasets and mitigate the impact of common confounding factors that have historically limited the adoption of EMG-based inputs.

FPGA-based Implementation of Random Forest Classifier for sEMG Signal Classification

This paper proposes a method that combines a hardware description language and Random Forests theory, to develop a motion recognition embedded system. To validate this system, the Ninapro database DB2 was used as training and test data, resulting in an accuracy of 83.1% for the test base in movement prediction.

Unveiling EMG semantics: a prototype-learning approach to generalizable gesture classification

Objective.Upper limb loss can profoundly impact an individual's quality of life, posing challenges to both physical capabilities and emotional well-being. To restore limb function by decoding electromyography (EMG) signals, in this paper, we present a novel deep prototype learning method for accurate and generalizable EMG-based gesture classification. Existing methods suffer from limitations in generalization across subjects due to the diverse nature of individual muscle responses, impeding seamless applicability in broader populations.Approach.By leveraging deep prototype learning, we introduce a method that goes beyond direct output prediction. Instead, it matches new EMG inputs to a set of learned prototypes and predicts the corresponding labels.Main results.This novel methodology significantly enhances the model's classification performance and generalizability by discriminating subtle differences between gestures, making it more reliable and precise in real-world applications. Our experiments on four Ninapro datasets suggest that our deep prototype learning classifier outperforms state-of-the-art methods in terms of intra-subject and inter-subject classification accuracy in gesture prediction.Significance.The results from our experiments validate the effectiveness of the proposed method and pave the way for future advancements in the field of EMG gesture classification for upper limb prosthetics.

Dual Stream Long Short-Term Memory Feature Fusion Classifier for Surface Electromyography Gesture Recognition

Gesture recognition using electromyography (EMG) signals has prevailed recently in the field of human-computer interactions for controlling intelligent prosthetics. Currently, machine learning and deep learning are the two most commonly employed methods for classifying hand gestures. Despite traditional machine learning methods already achieving impressive performance, it is still a huge amount of work to carry out feature extraction manually. The existing deep learning methods utilize complex neural network architectures to achieve higher accuracy, which will suffer from overfitting, insufficient adaptability, and low recognition accuracy. To improve the existing phenomenon, a novel lightweight model named dual stream LSTM feature fusion classifier is proposed based on the concatenation of five time-domain features of EMG signals and raw data, which are both processed with one-dimensional convolutional neural networks and LSTM layers to carry out the classification. The proposed method can effectively capture global features of EMG signals using a simple architecture, which means less computational cost. An experiment is conducted on a public DB1 dataset with 52 gestures, and each of the 27 subjects repeats every gesture 10 times. The accuracy rate achieved by the model is 89.66%, which is comparable to that achieved by more complex deep learning neural networks, and the inference time for each gesture is 87.6 ms, which can also be implied in a real-time control system. The proposed model is validated using a subject-wise experiment on 10 out of the 40 subjects in the DB2 dataset, achieving a mean accuracy of 91.74%. This is illustrated by its ability to fuse time-domain features and raw data to extract more effective information from the sEMG signal and select an appropriate, efficient, lightweight network to enhance the recognition results.

Continuous Kalman Estimation Method for Finger Kinematics Tracking from Surface Electromyography

Deciphering hand motion intention from surface electromyography (sEMG) encounters challenges posed by the requisites of multiple degrees of freedom (DOFs) and adaptability. Unlike discrete action classification grounded in pattern recognition, the pursuit of continuous kinematics estimation is appreciated for its inherent naturalness and intuitiveness. However, prevailing estimation techniques contend with accuracy limitations and substantial computational demands. Kalman estimation technology, celebrated for its ease of implementation and real-time adaptability, finds extensive application across diverse domains. This study introduces a continuous Kalman estimation method, leveraging a system model with sEMG and joint angles as inputs and outputs. Facilitated by model parameter training methods, the approach deduces multiple DOF finger kinematics simultaneously. The method's efficacy is validated using a publicly accessible database, yielding a correlation coefficient (CC) of 0.73. With over 45,000 windows for training Kalman model parameters, the average computation time remains under 0.01 s. This pilot study amplifies its potential for further exploration and application within the realm of continuous finger motion estimation technology.

Fusion inception and transformer network for continuous estimation of finger kinematics from surface electromyography

Decoding surface electromyography (sEMG) to recognize human movement intentions enables us to achieve stable, natural and consistent control in the field of human computer interaction (HCI). In this paper, we present a novel deep learning (DL) model, named fusion inception and transformer network (FIT), which effectively models both local and global information on sequence data by fully leveraging the capabilities of Inception and Transformer networks. In the publicly available Ninapro dataset, we selected surface EMG signals from six typical hand grasping maneuvers in 10 subjects for predicting the values of the 10 most important joint angles in the hand. Our model's performance, assessed through Pearson's correlation coefficient (PCC), root mean square error (RMSE), and R-squared (R2) metrics, was compared with temporal convolutional network (TCN), long short-term memory network (LSTM), and bidirectional encoder representation from transformers model (BERT). Additionally, we also calculate the training time and the inference time of the models. The results show that FIT is the most performant, with excellent estimation accuracy and low computational cost. Our model contributes to the development of HCI technology and has significant practical value.

A rotary transformer cross-subject model for continuous estimation of finger joints kinematics and a transfer learning approach for new subjects

Introduction

Surface Electromyographic (sEMG) signals are widely utilized for estimating finger kinematics continuously in human-machine interfaces (HMI), and deep learning approaches are crucial in constructing the models. At present, most models are extracted on specific subjects and do not have cross-subject generalizability. Considering the erratic nature of sEMG signals, a model trained on a specific subject cannot be directly applied to other subjects. Therefore, in this study, we proposed a cross-subject model based on the Rotary Transformer (RoFormer) to extract features of multiple subjects for continuous estimation kinematics and extend it to new subjects by adversarial transfer learning (ATL) approach.

Methods

We utilized the new subject's training data and an ATL approach to calibrate the cross-subject model. To improve the performance of the classic transformer network, we compare the impact of different position embeddings on model performance, including learnable absolute position embedding, Sinusoidal absolute position embedding, and Rotary Position Embedding (RoPE), and eventually selected RoPE. We conducted experiments on 10 randomly selected subjects from the NinaproDB2 dataset, using Pearson correlation coefficient (CC), normalized root mean square error (NRMSE), and coefficient of determination (R2) as performance metrics.

Results

The proposed model was compared with four other models including LSTM, TCN, Transformer, and CNN-Attention. The results demonstrated that both in cross-subject and subject-specific cases the performance of RoFormer was significantly better than the other four models. Additionally, the ATL approach improves the generalization performance of the cross-subject model better than the fine-tuning (FT) transfer learning approach.

Discussion

The findings indicate that the proposed RoFormer-based method with an ATL approach has the potential for practical applications in robot hand control and other HMI settings. The model's superior performance suggests its suitability for continuous estimation of finger kinematics across different subjects, addressing the limitations of subject-specific models.

A Circular, Wireless Surface-Electromyography Array

Commercial, high-tech upper limb prostheses offer a lot of functionality and are equipped with high-grade control mechanisms. However, they are relatively expensive and are not accessible to the majority of amputees. Therefore, more affordable, accessible, open-source, and 3D-printable alternatives are being developed. A commonly proposed approach to control these prostheses is to use bio-potentials generated by skeletal muscles, which can be measured using surface electromyography (sEMG). However, this control mechanism either lacks accuracy when a single sEMG sensor is used or involves the use of wires to connect to an array of multiple nodes, which hinders patients' movements. In order to mitigate these issues, we have developed a circular, wireless s-EMG array that is able to collect sEMG potentials on an array of electrodes that can be spread (not) uniformly around the circumference of a patient's arm. The modular sEMG system is combined with a Bluetooth Low Energy System on Chip, motion sensors, and a battery. We have benchmarked this system with a commercial, wired, state-of-the-art alternative and found an r = 0.98 (p < 0.01) Spearman correlation between the root-mean-squared (RMS) amplitude of sEMG measurements measured by both devices for the same set of 20 reference gestures, demonstrating that the system is accurate in measuring sEMG. Additionally, we have demonstrated that the RMS amplitudes of sEMG measurements between the different nodes within the array are uncorrelated, indicating that they contain independent information that can be used for higher accuracy in gesture recognition. We show this by training a random forest classifier that can distinguish between 6 gestures with an accuracy of 97%. This work is important for a large and growing group of amputees whose quality of life could be improved using this technology.

Coming in handy: CeTI-Age - A comprehensive database of kinematic hand movements across the lifespan

The Tactile Internet aims to advance human-human and human-machine interactions that also utilize hand movements in real, digitized, and remote environments. Attention to elderly generations is necessary to make the Tactile Internet age inclusive. We present the first age-representative kinematic database consisting of various hand gesturing and grasping movements at individualized paces, thus capturing naturalistic movements. We make this comprehensive database of kinematic hand movements across the adult lifespan (CeTI-Age-Kinematic-Hand) publicly available to facilitate a deeper understanding of intra-individual-focusing especially on age-related differences-and inter-individual variability in hand kinematics. The core of the database contains participants' hand kinematics recorded with wearable resistive bend sensors, individual static 3D hand models, and all instructional videos used during the data acquisition. Sixty-three participants ranging from age 20 to 80 years performed six repetitions of 40 different naturalistic hand movements at individual paces. This unique database with data recorded from an adult lifespan sample can be used to advance machine-learning approaches in hand kinematic modeling and movement prediction for age-inclusive applications.

TraHGR: Transformer for Hand Gesture Recognition via Electromyography

Deep learning-based Hand Gesture Recognition (HGR) via surface Electromyogram (sEMG) signals have recently shown considerable potential for development of advanced myoelectric-controlled prosthesis. Although deep learning techniques can improve HGR accuracy compared to their classical counterparts, classifying hand movements based on sparse multichannel sEMG signals is still a challenging task. Furthermore, existing deep learning approaches, typically, include only one model as such can hardly extract representative features. In this paper, we aim to address this challenge by capitalizing on the recent advances in hybrid models and transformers. In other words, we propose a hybrid framework based on the transformer architecture, which is a relatively new and revolutionizing deep learning model. The proposed hybrid architecture, referred to as the Transformer for Hand Gesture Recognition (TraHGR), consists of two parallel paths followed by a linear layer that acts as a fusion center to integrate the advantage of each module. We evaluated the proposed architecture TraHGR based on the commonly used second Ninapro dataset, referred to as the DB2. The sEMG signals in the DB2 dataset are measured in real-life conditions from 40 healthy users, each performing 49 gestures. We have conducted an extensive set of experiments to test and validate the proposed TraHGR architecture, and compare its achievable accuracy with several recently proposed HGR classification algorithms over the same dataset. We have also compared the results of the proposed TraHGR architecture with each individual path and demonstrated the distinguishing power of the proposed hybrid architecture. The recognition accuracies of the proposed TraHGR architecture for the window of size 200ms and step size of 100ms are 86.00%, 88.72%, 81.27%, and 93.74%, which are 2.30%, 4.93%, 8.65%, and 4.20% higher than the state-of-the-art performance for DB2 (49 gestures), DB2-B (17 gestures), DB2-C (23 gestures), and DB2-D (9 gestures), respectively.

Movement recognition via channel-activation-wise sEMG attention

CONTEXT

Surface electromyography (sEMG) signals contain rich information recorded from muscle movements and therefore reflect the user's intention. sEMG has seen dominant applications in rehabilitation, clinical diagnosis as well as human engineering, etc. However, current feature extraction methods for sEMG signals have been seriously limited by their stochasticity, transiency, and non-stationarity.

OBJECTIVE

Our objective is to combat the difficulties induced by the aforementioned downsides of sEMG and thereby extract representative features for various downstream movement recognition.

METHOD

We propose a novel 3-axis view of sEMG features composed of temporal, spatial, and channel-wise summary. We leverage the state-of-the-art architecture Transformer to enforce efficient parallel search and to get rid of limitations imposed by previous work in gesture classification. The transformer model is designed on top of an attention-based module, which allows for the extraction of global contextual relevance among channels and the use of this relevance for sEMG recognition.

RESULTS

We compared the proposed method against existing methods on two Ninapro datasets consisting of data from both healthy people and amputees. Experimental results show the proposed method attains the state-of-the-art (SOTA) accuracy on both datasets. We further show that the proposed method enjoys strong generalization ability: a new SOTA is achieved by pretraining the model on a different dataset followed by fine-tuning it on the target dataset.

Surface electromyogram, kinematic, and kinetic dataset of lower limb walking for movement intent recognition

Surface electromyogram (sEMG) offers a rich set of motor information for decoding limb motion intention that serves as a control input to Intelligent human-machine synergy systems (IHMSS). Despite growing interest in IHMSS, the current publicly available datasets are limited and can hardly meet the growing demands of researchers. This study presents a novel lower limb motion dataset (designated as SIAT-LLMD), comprising sEMG, kinematic, and kinetic data with corresponding labels acquired from 40 healthy humans during 16 movements. The kinematic and kinetic data were collected using a motion capture system and six-dimensional force platforms and processed using OpenSim software. The sEMG data were recorded using nine wireless sensors placed on the subjects' thigh and calf muscles on the left limb. Besides, SIAT-LLMD provides labels to classify the different movements and different gait phases. Analysis of the dataset verified the synchronization and reproducibility, and codes for effective data processing are provided. The proposed dataset can serve as a new resource for exploring novel algorithms and models for characterizing lower limb movements.

Multi-modality deep forest for hand motion recognition via fusing sEMG and acceleration signals

Bio-signal based hand motion recognition plays a critical role in the tasks of human-machine interaction, such as the natural control of multifunctional prostheses. Although a large number of classification technologies have been taken to improve the motion recognition accuracy, it is still a challenge to achieve acceptable performance for multiple modality input. This study proposes a multi-modality deep forest (MMDF) framework to identify hand motions, in which surface electromyographic signals (sEMG) and acceleration signals (ACC) are fused at the input level. The proposed MMDF framework constitutes of three main stages, sEMG and ACC feature extraction, feature dimension reduction, and a cascade structure deep forest for classification. A public database "Ninapro DB7" is used to evaluate the performance of the proposed framework, and the experimental results show that it can achieve a significantly higher accuracy than that of competitors. Besides, our experimental results also show that MMDF outperforms other traditional classifiers with the input of the single modality of sEMG signals. In sum, this study verifies that ACC signals can be an excellent supplementary for sEMG, and MMDF is a plausible solution to fuse mulit-modality bio-signals for human motion recognition.

LST-EMG-Net: Long short-term transformer feature fusion network for sEMG gesture recognition

With the development of signal analysis technology and artificial intelligence, surface electromyography (sEMG) signal gesture recognition is widely used in rehabilitation therapy, human-computer interaction, and other fields. Deep learning has gradually become the mainstream technology for gesture recognition. It is necessary to consider the characteristics of the surface EMG signal when constructing the deep learning model. The surface electromyography signal is an information carrier that can reflect neuromuscular activity. Under the same circumstances, a longer signal segment contains more information about muscle activity, and a shorter segment contains less information about muscle activity. Thus, signals with longer segments are suitable for recognizing gestures that mobilize complex muscle activity, and signals with shorter segments are suitable for recognizing gestures that mobilize simple muscle activity. However, current deep learning models usually extract features from single-length signal segments. This can easily cause a mismatch between the amount of information in the features and the information needed to recognize gestures, which is not conducive to improving the accuracy and stability of recognition. Therefore, in this article, we develop a long short-term transformer feature fusion network (referred to as LST-EMG-Net) that considers the differences in the timing lengths of EMG segments required for the recognition of different gestures. LST-EMG-Net imports multichannel sEMG datasets into a long short-term encoder. The encoder extracts the sEMG signals' long short-term features. Finally, we successfully fuse the features using a feature cross-attention module and output the gesture category. We evaluated LST-EMG-Net on multiple datasets based on sparse channels and high density. It reached 81.47, 88.24, and 98.95% accuracy on Ninapro DB2E2, DB5E3 partial gesture, and CapgMyo DB-c, respectively. Following the experiment, we demonstrated that LST-EMG-Net could increase the accuracy and stability of various gesture identification and recognition tasks better than existing networks.

sEMG-based technology for silent voice recognition

Silent speech recognition (SSR) is a system that implements speech communication when a sound signal is not available using surface electromyography (sEMG)-based speech recognition. Researchers have used surface electrodes to record the electrically-activated potential of human articulation muscles to recognize speech content. SSR can be used for pilot-assisted speech recognition, communication of individuals with speech impairment, private communication, and other fields. In this feasibility study, we collected sEMG data for ten single Mandarin numeric words. After reducing power frequency interference and power supply noise from the sEMG signal, short-term energy (STE) was used for voice activity detection (VAD). The power spectrum features were extracted and fed into the classifier for final identification results. We used the Hold-out method to divide the data into training and test sets on a 7-3 scale, with an average accuracy of 92.3% and a maximum of 100% using a support vector machine (SVM) classifier. Experimental results showed that the proposed method has development potential, and is effective in identifying isolated words from the sEMG signal of the articulation muscles.

Performance improvement and complexity reduction in the classification of EMG signals with mRMR-based CNN-KNN combined model

For real-time applications, the performance in classifying the movement should be as high as possible, and the computational complexity should be low. This paper focuses on the classification of five upper arm movements which can be provided as a control for human-machine interface (HMI) based applications. The conventional machine learning algorithms are used for classification with both time and frequency domain features, and k-nearest neighbor (KNN) outplay others. To further improve the classification accuracy, pretrained CNN architectures are employed which leads to computational complexity and memory requirements. To overcome this, the deep convolutional neural network (CNN) model is introduced with three convolutional layers. To further improve the performance which is the key idea behind real-time applications, a hybrid CNN-KNN model is proposed. Even though the performance is high, the computation costs of the hybrid method are more. Minimum redundancy maximum relevance (mRMR), a feature selection method makes an effort to reduce feature dimensions. As a result, better performance is achieved by our proposed method CNN-KNN with mRMR which reduces computational complexity and memory requirement with a mean prediction accuracy of about 99.05±0.25% with 100 features.

Number of trials and data structure affect the number and components of muscle synergies in upper-limb reaching movements

Objective.Due to the variability of human movements, muscle activations vary among trials and subjects. However, few studies investigated how data organization methods for addressing variability impact the extracted muscle synergies.Approach.Fifteen healthy subjects performed a large set of upper limb multi-directional point-to-point reaching movements. Then, the study extracted muscle synergies under different data settings and investigated how data structure prior to synergy extraction, namely concatenation, averaging, and single trial, the number of considered trials, and the number of reaching directions affected the number and components of muscle synergies.Main results.The results showed that the number and components of synergies were significantly affected by the data structure. The concatenation method identified the highest number of synergies, and the averaging method usually found a smaller number of synergies. When the concatenated trials or reaching directions was lower than a minimum value, the number of synergies increased with the increase of the number of trials or reaching directions; however, when the number of trials or reaching directions reached a threshold, the number of synergies was usually constant or with less variation even when novel directions and trials were added. Similarity analysis also showed a slight increase when the number of trials or reaching directions was lower than a threshold. This study recommends that at least five trials and four reaching directions and the concatenation method are considered in muscle synergies analysis during upper limb tasks.Significance.This study makes the researchers focus on the variability analysis induced by the diseases rather than the techniques applied for synergies analysis and promotes applications of muscle synergies in clinical scenarios.

Biosignal-based transferable attention Bi-ConvGRU deep network for hand-gesture recognition towards online upper-limb prosthesis control

BACKGROUND AND OBJECTIVE

Upper-limb amputation can significantly affect a person's capabilities with a dramatic impact on their quality of life. As a biological signal, surface electromyogram (sEMG) provides a non-invasive means to measure underlying muscle activation patterns, corresponding to specific hand gestures. This project aims to develop a real-time deep learning based recognition model to automatically and reliably recognise these complex signals of a wide range of daily hand gestures from amputees and non-amputees.

METHODS

This paper proposes an attention bidirectional Convolutional Gated Recurrent Unit (Bi-ConvGRU) deep neural network for hand-gesture recognition. By training on sEMG data from both amputees and non-amputees, the model can learn to recognise a group of fine-grained hand movements. This is a significantly more challenging and underexplored area, compared to existing studies on coarse-control in lower limbs. One dimensional CNNs are initially used to extract intra-channel features. The novel use of a bidirectional sequential GRU (Bi-GRU) deep neural network allows the exploration of correlation of muscle activation among multi-channel sEMG signals from both prior and posterior time sequences. Importantly, the attention mechanism is employed following Bi-GRU layers. This enables the model to learn vital parts and feature weights, increasing robustness to bio-data noise and irregularity. Finally, we introduce the first of its kind transfer learning, demonstrating that a baseline model pre-trained with non-amputee data can be effectively refined with amputee data to build a personalised model for amputees.

RESULTS

The attention Bi-ConvGRU was evaluated on the benchmark database Ninapro, and achieved an average accuracy of 88.7%, outperforming the state-of-the-art on 18 gesture recognition by 6.7%.

CONCLUSIONS

To our knowledge, the developed end-to-end deep learning model is the first of its kind that enables reliable predictive decision making in short time windows (160ms). This reduced latency limits physiological awareness, enabling the potential for real-time, online and thus more intuitive bio-control of prosthetic devices for amputees.

Quantitative Assessment of Hand Motor Function for Post-Stroke Rehabilitation Based on HAGCN and Multimodality Fusion

Quantitative assessment of hand function can assist therapists in providing appropriate rehabilitation strategies, which plays an essential role in post-stroke rehabilitation. Conventionally, the assessment process relies heavily on clinical experience and lacks quantitative analysis. To quantitatively assess the hand motor function of patients with post-stroke hemiplegia, this study proposes a novel multi-modality fusion assessment framework. This framework includes three components: the kinematic feature extraction based on a graph convolutional network (HAGCN), the surface electromyography (sEMG) signal processing based on a multi-layer long short term memory (LSTM) network, and the quantitative assessment based on the multi-modality fusion. To the best of the authors' knowledge, this is the first study of applying a graph convolution network to assess the hand motor function. We also collect the kinematic data and sEMG data from 70 subjects who completed 28 types of hand movements. Therapists first graded patients using traditional motor assessment scales (Brunnstrom Scale and Fugl-Meyer Assessment Scale) and further refined the patient's motor assessment result by their experience. Then, we trained the HAGCN and LSTM networks and quantitatively assessed each patient based on the proposed assessment framework. Finally, the Spearman correlation coefficient (SC) between the assessment result of this study and the traditional scale are 0.908 and 0.967, demonstrating a significant correlation between the proposed assessment and the traditional scale scores. In addition, the SC value between the score of this study and the refined hand motor function is 0.997, indicating the "ceiling effect" of some traditional scales can be avoided.

A Time-Series Augmentation Method Based on Empirical Mode Decomposition and Integrated LSTM Neural Network

Adequate patients' data have always been critical for disease assessment. However, large amounts of patient data are often difficult to collect, especially when patients are required to complete a series of assessment movements. For example, assessing the hand motor function of stroke patients or Parkinson's patients requires patients to complete a series of evaluation movements, and it is often difficult for patients to complete each group of actions multiple times, resulting in a small amount of data. To solve the problem of insufficient data quantity, this study proposes a data augmentation method based on empirical mode decomposition and integrated long short-term memory neural network (EMD-ILSTM). The method mainly consists of two parts: one is to decompose the raw signal by the method of EMD, and the other is to use LSTM for data augmentation of the decomposed signal. Then, the method is tested on the public dataset named Ninaweb, and the test results show that the classification accuracy can be improved by 5.2% by using the augmented data for classification tasks. Finally, clinical trials are conducted to verify that after dimensionality reduction, the augmented data and raw data have smaller intra-class distances and larger inter-class distances, indicating that data augmentation is effective.

Selection of the Best Set of Features for sEMG-Based Hand Gesture Recognition Applying a CNN Architecture

The classification of surface myoelectric signals (sEMG) remains a great challenge when focused on its implementation in an electromechanical hand prosthesis, due to its nonlinear and stochastic nature, as well as the great difference between models applied offline and online. In this work, the selection of the set of the features that allowed us to obtain the best results for the classification of this type of signals is presented. In order to compare the results obtained, the Nina PRO DB2 and DB3 databases were used, which contain information on 50 different movements of 40 healthy subjects and 11 amputated subjects, respectively. The sEMG of each subject was acquired through 12 channels in a bipolar configuration. To carry out the classification, a convolutional neural network (CNN) was used and a comparison of four sets of features extracted in the time domain was made, three of which have shown good performance in previous works and one more that was used for the first time to train this type of network. Set one is composed of six features in the time domain (TD1), Set two has 10 features also in the time domain (TD2) including the autoregression model (AR), the third set has two features in the time domain derived from spectral moments (TD-PSD1), and finally, a set of five features also has information on the power spectrum of the signal obtained in the time domain (TD-PSD2). The selected features in each set were organized in four different ways for the formation of the training images. The results obtained show that the set of features TD-PSD2 obtained the best performance for all cases. With the set of features and the formation of images proposed, an increase in the accuracies of the models of 8.16% and 8.56% was obtained for the DB2 and DB3 databases, respectively, compared to the current state of the art that has used these databases.

Finger Movement Recognition via High-Density Electromyography of Intrinsic and Extrinsic Hand Muscles

Surface electromyography (sEMG) is commonly used to observe the motor neuronal activity within muscle fibers. However, decoding dexterous body movements from sEMG signals is still quite challenging. In this paper, we present a high-density sEMG (HD-sEMG) signal database that comprises simultaneously recorded sEMG signals of intrinsic and extrinsic hand muscles. Specifically, twenty able-bodied participants performed 12 finger movements under two paces and three arm postures. HD-sEMG signals were recorded with a 64-channel high-density grid placed on the back of hand and an 8-channel armband around the forearm. Also, a data-glove was used to record the finger joint angles. Synchronisation and reproducibility of the data collection from the HD-sEMG and glove sensors were ensured. The collected data samples were further employed for automated recognition of dexterous finger movements. The introduced dataset offers a new perspective to study the synergy between the intrinsic and extrinsic hand muscles during dynamic finger movements. As this dataset was collected from multiple participants, it also provides a resource for exploring generalized models for finger movement decoding.

Improved Multi-Stream Convolutional Block Attention Module for sEMG-Based Gesture Recognition

As a key technology for the non-invasive human-machine interface that has received much attention in the industry and academia, surface EMG (sEMG) signals display great potential and advantages in the field of human-machine collaboration. Currently, gesture recognition based on sEMG signals suffers from inadequate feature extraction, difficulty in distinguishing similar gestures, and low accuracy of multi-gesture recognition. To solve these problems a new sEMG gesture recognition network called Multi-stream Convolutional Block Attention Module-Gate Recurrent Unit (MCBAM-GRU) is proposed, which is based on sEMG signals. The network is a multi-stream attention network formed by embedding a GRU module based on CBAM. Fusing sEMG and ACC signals further improves the accuracy of gesture action recognition. The experimental results show that the proposed method obtains excellent performance on dataset collected in this paper with the recognition accuracies of 94.1%, achieving advanced performance with accuracy of 89.7% on the Ninapro DB1 dataset. The system has high accuracy in classifying 52 kinds of different gestures, and the delay is less than 300 ms, showing excellent performance in terms of real-time human-computer interaction and flexibility of manipulator control.

Evaluation of Methods for the Extraction of Spatial Muscle Synergies

Muscle synergies have been largely used in many application fields, including motor control studies, prosthesis control, movement classification, rehabilitation, and clinical studies. Due to the complexity of the motor control system, the full repertoire of the underlying synergies has been identified only for some classes of movements and scenarios. Several extraction methods have been used to extract muscle synergies. However, some of these methods may not effectively capture the nonlinear relationship between muscles and impose constraints on input signals or extracted synergies. Moreover, other approaches such as autoencoders (AEs), an unsupervised neural network, were recently introduced to study bioinspired control and movement classification. In this study, we evaluated the performance of five methods for the extraction of spatial muscle synergy, namely, principal component analysis (PCA), independent component analysis (ICA), factor analysis (FA), nonnegative matrix factorization (NMF), and AEs using simulated data and a publicly available database. To analyze the performance of the considered extraction methods with respect to several factors, we generated a comprehensive set of simulated data (ground truth), including spatial synergies and temporal coefficients. The signal-to-noise ratio (SNR) and the number of channels (NoC) varied when generating simulated data to evaluate their effects on ground truth reconstruction. This study also tested the efficacy of each synergy extraction method when coupled with standard classification methods, including K-nearest neighbors (KNN), linear discriminant analysis (LDA), support vector machines (SVM), and Random Forest (RF). The results showed that both SNR and NoC affected the outputs of the muscle synergy analysis. Although AEs showed better performance than FA in variance accounted for and PCA in synergy vector similarity and activation coefficient similarity, NMF and ICA outperformed the other three methods. Classification tasks showed that classification algorithms were sensitive to synergy extraction methods, while KNN and RF outperformed the other two methods for all extraction methods; in general, the classification accuracy of NMF and PCA was higher. Overall, the results suggest selecting suitable methods when performing muscle synergy-related analysis.

SeNic: An Open Source Dataset for sEMG-Based Gesture Recognition in Non-ideal Conditions

In order to reduce the gap between the laboratory environment and actual use in daily life of human-machine interaction based on surface electromyogram (sEMG) intent recognition, this paper presents a benchmark dataset of sEMG in non-ideal conditions (SeNic). The dataset mainly consists of 8-channel sEMG signals, and electrode shifts from an 3D-printed annular ruler. A total of 36 subjects participate in our data acquisition experiments of 7 gestures in non-ideal conditions, where non-ideal factors of 1) electrode shifts, 2) individual difference, 3) muscle fatigue, 4) inter-day difference, and 5) arm postures are elaborately involved. Signals of sEMG are validated first in temporal and frequency domains. Results of recognizing gestures in ideal conditions indicate the high quality of the dataset. Adverse impacts in non-ideal conditions are further revealed in the amplitudes of these data and recognition accuracies. To be concluded, SeNic is a benchmark dataset that introduces several non-ideal factors which often degrade the robustness of sEMG-based systems. It could be used as a freely available dataset and a common platform for researchers in the sEMG-based recognition community. The benchmark dataset SeNic are available online via the website3.

User-Independent EMG Gesture Recognition Method Based on Adaptive Learning

In a gesture recognition system based on surface electromyogram (sEMG) signals, the recognition model established by existing users cannot directly generalize to the across-user scenarios due to the individual variability of sEMG signals. In this article, we propose an adaptive learning method to handle the problem. The muscle synergy is chosen as the feature vector because it can well-characterize the neural origin of movement. The initial train set is composed of representative samples extracted from the synergy matrix of the existing user. When the new users use the system, the label is obtained by the adaptive K nearest neighbor algorithm (KNN). The recognition process does not require the pre-experiment for new users due to the introduction of adaptive learning strategy, namely, the qualified data and the label of new user data evaluated by a risk evaluator are used to update the train set and KNN weights, so as to adapt to the new users. We have tested the algorithm in DB1 and DB5 of Ninapro databases. The average recognition accuracy is 68.04, 73.35, and 83.05% for different types of gestures, respectively, achieving the effects of the user-dependent method. Our study can avoid the re-training steps and the recognition performance will improve with the increased frequency of uses, which will further facilitate the widespread implementation of sEMG control systems using pattern recognition techniques.

Toward Improving the Reliability of Discrete Movement Recognition of sEMG Signals

Currently, the classification accuracy of surface electromyography (sEMG) signals is high in literature, but the conventional recognition system may classify untrained movements or the trained movements of low reliability to one of its target classes by mistake. If such a system is used for prosthetic control, sometimes it may cause a disaster. A two-layer classifier that fuses the Gaussian mixture model (GMM) and k-nearest neighbor (kNN) in a sequential structure is proposed in this study. The proposed algorithm can reject the trained movements with low reliability and is efficient in rejecting the untrained movements, thus enhancing the reliability of the myoelectric control system. The results show that the proposed algorithm can produce 95.7% active accuracy in recognizing 12 trained movements and a 30.3% error rate for rejecting 12 untrained movements. When the movement number is six, the active accuracy for trained movements can reach 99.2%, and the error rate of untrained movement is only 17.4%, which is much better than previous studies. Therefore, the proposed classifier can accurately recognize the trained movements and reject untrained movement patterns effectively.

LSTM Network Classification of Dexterous Individual Finger Movements

Electrical activity is generated in the forearm muscles during muscular contractions that control dexterous movements of a human finger and thumb. Using this electrical activity as an input to train a neural network for the purposes of classifying finger movements is not straightforward. Low cost wearable sensors i.e., a Myo Gesture control armband (www.bynorth.com), generally have a lower sampling rate when compared with medical grade EMG detection systems e.g., 200 Hz vs 2000 Hz. Using sensors such as the Myo coupled with the lower amplitude generated by individual finger movements makes it difficult to achieve high classification accuracy. Low sampling rate makes it challenging to distinguish between large quantities of subtle finger movements when using a single network. This research uses two networks which enables for the reduction in the number of movements in each network that are being classified; in turn improving the classification. This is achieved by developing and training LSTM networks that focus on the extension and flexion signals of the fingers and a separate network that is trained using thumb movement signal data. By following this method, this research have increased classification of the individual finger movements to between 90 and 100%.

Kernel Density Estimation of Electromyographic Signals and Ensemble Learning for Highly Accurate Classification of a Large Set of Hand/Wrist Motions

The performance of myoelectric control highly depends on the features extracted from surface electromyographic (sEMG) signals. We propose three new sEMG features based on the kernel density estimation. The trimmed mean of density (TMD), the entropy of density, and the trimmed mean absolute value of derivative density were computed for each sEMG channel. These features were tested for the classification of single tasks as well as of two tasks concurrently performed. For single tasks, correlation-based feature selection was used, and the features were then classified using linear discriminant analysis (LDA), non-linear support vector machines, and multi-layer perceptron. The eXtreme gradient boosting (XGBoost) classifier was used for the classification of two movements simultaneously performed. The second and third versions of the Ninapro dataset (conventional control) and Ameri's movement dataset (simultaneous control) were used to test the proposed features. For the Ninapro dataset, the overall accuracy of LDA using the TMD feature was 98.99 ± 1.36% and 92.25 ± 9.48% for able-bodied and amputee subjects, respectively. Using ensemble learning of the three classifiers, the average macro and micro-F-score, macro recall, and precision on the validation sets were 98.23 ± 2.02, 98.32 ± 1.93, 98.32 ± 1.93, and 98.88 ± 1.31%, respectively, for the intact subjects. The movement misclassification percentage was 1.75 ± 1.73 and 3.44 ± 2.23 for the intact subjects and amputees. The proposed features were significantly correlated with the movement classes [Generalized Linear Model (GLM); P-value < 0.05]. An accurate online implementation of the proposed algorithm was also presented. For the simultaneous control, the overall accuracy was 99.71 ± 0.08 and 97.85 ± 0.10 for the XGBoost and LDA classifiers, respectively. The proposed features are thus promising for conventional and simultaneous myoelectric control.

A novel concatenate feature fusion RCNN architecture for sEMG-based hand gesture recognition

Hand gesture recognition tasks based on surface electromyography (sEMG) are vital in human-computer interaction, speech detection, robot control, and rehabilitation applications. However, existing models, whether traditional machine learnings (ML) or other state-of-the-arts, are limited in the number of movements. Targeting a large number of gesture classes, more data features such as temporal information should be persisted as much as possible. In the field of sEMG-based recognitions, the recurrent convolutional neural network (RCNN) is an advanced method due to the sequential characteristic of sEMG signals. However, the invariance of the pooling layer damages important temporal information. In the all convolutional neural network (ACNN), because of the feature-mixing convolution operation, a same output can be received from completely different inputs. This paper proposes a concatenate feature fusion (CFF) strategy and a novel concatenate feature fusion recurrent convolutional neural network (CFF-RCNN). In CFF-RCNN, a max-pooling layer and a 2-stride convolutional layer are concatenated together to replace the conventional simple dimensionality reduction layer. The featurewise pooling operation serves as a signal amplitude detector without using any parameter. The feature-mixing convolution operation calculates the contextual information. Complete evaluations are made on both the accuracy and convergence speed of the CFF-RCNN. Experiments are conducted using three sEMG benchmark databases named DB1, DB2 and DB4 from the NinaPro database. With more than 50 gestures, the classification accuracies of the CFF-RCNN are 88.87% on DB1, 99.51% on DB2, and 99.29% on DB4. These accuracies are the highest compared with reported accuracies of machine learnings and other state-of-the-art methods. To achieve accuracies of 86%, 99% and 98% for the RCNN, the training time are 2353.686 s, 816.173 s and 731.771 s, respectively. However, for the CFF-RCNN to reach the same accuracies, it needs only 1727.415 s, 542.245 s and 576.734 s, corresponding to a reduction of 26.61%, 33.56% and 21.19% in training time. We concluded that the CFF-RCNN is an improved method when classifying a large number of hand gestures. The CFF strategy significantly improved model performance with higher accuracy and faster convergence as compared to traditional RCNN.

Compression of EMG Signals Using Deep Convolutional Autoencoders

Efficient storage and transmission of electromyogram (EMG) data are important for emerging applications such as telemedicine and big data, as a vital tool for further advancement of the field. However, due to limitations in internet speed and hardware resources, transmission and storage of EMG data are challenging. As a solution, this work proposes a new method for EMG data compression using deep convolutional autoencoders (CAE). Eight-channel EMG data from 10 subjects, and high density EMG data from 18 subjects, were investigated for compression. The CAE architecture was designed to extract an abstract data representation that is heavily compressed, but from which the salient information for classification can be effectively reconstructed. The proposed method attained efficient compression; for CR=1600, the average PRDN (percentage RMS difference normalized) was 31.5% and the wrist motions classification accuracy (CA) reduced roughly 5%. The CAE substantially outperformed the state-of-the-art high-efficiency video coding and a well known wavelet-thresholding compression technique. Moreover, by reducing the bit-resolution of the CAEs compressed data from 24 bits to 6 bits, an additional 4-fold compression was achieved without significant degradation of the reconstruction performance. Furthermore, the CAEs inter subject performance was promising; e.g. for CR=1600, the PRDN for the inter-subject case was only 2.6% less than that of the within-subject performance. The powerful EMG compression performance with remarkable reconstruction results reflects the CAEs potential as an automatic end to end approach with the ability to learn the complete encoding and decoding process. Furthermore, the excellent inter-subject performance demonstrates the generalizability and usability of the proposed approach.

Spatio-temporal warping for myoelectric control: an offline, feasibility study

Objective.The efficacy of an adopted feature extraction method directly affects the classification of the electromyographic (EMG) signals in myoelectric control applications. Most methods attempt to extract the dynamics of the multi-channel EMG signals in the time domain and on a channel-by-channel, or at best pairs of channels, basis. However, considering multi-channel information to build a similarity matrix has not been taken into account.Approach.Combining methods of long and short-term memory (LSTM) and dynamic temporal warping, we developed a new feature, called spatio-temporal warping (STW), for myoelectric signals. This method captures the spatio-temporal relationships of multi-channels EMG signals.Main results. Across four online databases, we show that in terms of average classification error and standard deviation values, the STW feature outperforms traditional features by 5%-17%. In comparison to the more recent deep learning models, e.g. convolutional neural networks (CNNs), STW outperformed by 5%-18%. Also, STW showed enhanced performance when compared to the CNN + LSTM model by 2%-14%. All differences were statistically significant with a large effect size.Significance.This feasibility study provides evidence supporting the hypothesis that the STW feature of the EMG signals can enhance the classification accuracy in an explainable way when compared to recent deep learning methods. Future work includes real-time implementation of the method and testing for prosthesis control.

Gesture Classification of Surface Electromyography Signals Using Machine Learning Algorithms for Hand Prosthetics

The actions of humans executed by their hands play a remarkable part in controlling and handling variety of objects in their daily life activities. The effect of losing or degradation in the functioning of one hand has a greater influence in bringing down the regular activity. Hence the design of prosthetic hands which assists the individuals to enhance their regular activity seems a better remedy in this new era. This paper puts forward a classification framework using machine learning algorithms for classifying hand gesture signals. The surface electromyography (sEMG) dataset acquired for 9 wrist movements of publicly available database are utilized to identify the potential biomarkers for classification and in evaluating the efficacy of the proposed algorithm. The statistical and time domain features of the sEMG signals from 27 intact subjects and 11 trans-radial amputated subjects are extracted and the optimal features are determined implementing the feature selection approach based on correlation factor. The classifiers performance of machine learning algorithms namely support vector machine (SVM), Naïve bayes (NB) and Ensemble classifier are evaluated. The experimental results highlight that the SVM classifier can yield the maximum accuracy movement classification of 99.6% for intact and 97.56% for trans-amputee subjects. The proposed approach offers better accuracy and sensitivity compared to other approaches that have used the sEMG dataset for movement classification.

An Adaptive Multi-Modal Control Strategy to Attenuate the Limb Position Effect in Myoelectric Pattern Recognition

Over the last few decades, pattern recognition algorithms have shown promising results in the field of upper limb prostheses myoelectric control and are now gradually being incorporated in commercial devices. A widely used approach is based on a classifier which assigns a specific input value to a selected hand motion. While this method guarantees good performance and robustness within each class, it still shows limitations in adapting to different conditions encountered in real-world applications, such as changes in limb position or external loads. This paper proposes an adaptive method based on a pattern recognition classifier that takes advantage of an augmented dataset-i.e., representing variations in limb position or external loads-to selectively adapt to underrepresented variations. The proposed method was evaluated using a series of target achievement control tests with ten able-bodied volunteers. Results indicated a higher median completion rate >3.33% for the adapted algorithm compared to a classical pattern recognition classifier used as a baseline model. Subject-specific performance showed the potential for improved control after adaptation and a ≤13% completion rate; and in many instances, the adapted points were able to provide new information within classes. These preliminary results show the potential of the proposed method and encourage further development.

A Noble Classification Framework for Data Glove Classification of a Large Number of Hand Movements

The recognition of hand movements is an important method for human-computer interaction (HCI) technology, and it is widely used in virtual reality and other HCI areas. While many valuable efforts have been made, efficient ways to capture over 20 types of hand movements with high accuracy by one data glove are still lacking. This paper addresses a new classification framework for 52 hand movements. This classification framework includes the following two parts: the movement detection algorithm and the movement classification algorithm. The fine K-nearest neighbor (Fine KNN) is the core of the movement detection algorithm. The movement classification algorithm is composed of downsampling in data preparation and a new deep learning network named the DBDF network. Bidirectional Long Short-Term Memory (BiLSTM) is the main part of the DBDF network. The results of experiments using the Ninapro DB1 dataset demonstrate that our work can classify more types of hand movements than related algorithms with a precision of 93.15%.

A Novel EMG-Based Hand Gesture Recognition Framework Based on Multivariate Variational Mode Decomposition

Surface electromyography (sEMG) is a kind of biological signal that records muscle activity noninvasively, which is of great significance in advanced human-computer interaction, prosthetic control, clinical therapy, and biomechanics. However, the number of hand gestures that can be recognized is limited and the recognition accuracy needs to be further improved. These factors lead to the fact that sEMG products are not widely used in practice. The main contributions of this paper are as follows. Firstly, considering the increasing number of gestures to be recognized and the complexity of gestures, an extensible two-stage machine learning lightweight framework was innovatively proposed for multi-gesture task recognition. Secondly, the multivariate variational mode decomposition (MVMD) is applied to extract the spatial–temporal features from the multiple channels to the EMG signals, and the separable convolutional neural network is used for modelling. In this work, the experimental results for 52 hand gestures recognition task show that the average accuracy on each stage is about 90%. The potential movement information is mainly contained in the low-frequency oscillator of the sEMG signal, and the model performs better with the low-frequency oscillation from the MVMD algorithm on the second stage classification than that of other decomposition methods.