Classification of simultaneous movements using surface EMG pattern recognition

Simultaneous Three-Degrees-of-Freedom Prosthetic Control Based on Linear Regression and Closed-Loop Training Protocol

Machine learning-based controllers of prostheses using electromyographic signals have become very popular in the last decade. The regression approach allows a simultaneous and proportional control of the intended movement in a more natural way than the classification approach, where the number of movements is discrete by definition. However, it is not common to find regression-based controllers working for more than two degrees of freedom at the same time. In this paper, we present the application of the adaptive linear regressor in a relatively low-dimensional feature space with only eight sensors to the problem of a simultaneous and proportional control of three degrees of freedom (left-right, up-down and open-close hand movements). We show that a key element usually overlooked in the learning process of the regressor is the training paradigm. We propose a closed-loop procedure, where the human learns how to improve the quality of the generated EMG signals, helping also to obtain a better controller. We apply it to 10 healthy and 3 limb-deficient subjects. Results show that the combination of the multidimensional targets and the open-loop training protocol significantly improve the performance, increasing the average completion rate from 53% to 65% for the most complicated case of simultaneously controlling the three degrees of freedom.

Simple recognition of hand gestures using single-channel EMG signals

Electromyography (EMG) signals are used for many different purposes, such as recording and measuring the electrical activity generated by varying the body's skeletal muscles. Biosignals are different types of biomedical signals, like EMG signals, which can be used for the neural linkage with computers and are obtained from a particular part of the body such as tissue, muscle, organ, or cell system like the nervous system. Surface electromyography (SEMG) is a non-invasive method that can be used as an effective system for controlling upper arm prostheses. This study focused on classifying the five types of distinct finger movements investigated in four unique channels.We have used a classification technique, the k-nearest neighbors (KNN), to categorize the collected samples. Two time-domain features, (a) maximum (Max) and (b) minimum (Min), were used with one of these three features separately: mean absolute value (MAV), root mean square (RMS), and simple square integral (SSI) to classify gestures. We chose classification accuracy as a criterion for evaluating the effectiveness of every classification. We figured out that the first grouping, that is, (MAV, Max, Min), was the best choice for classification. The accuracy percentage in the four channels for the first group was 91.0%, 89.9%, 89.8%, and 96.0%, respectively.

Cross-User Electromyography Pattern Recognition Based on a Novel Spatial-Temporal Graph Convolutional Network

With the goal of promoting the development of myoelectric control technology, this paper focuses on exploring graph neural network (GNN) based robust electromyography (EMG) pattern recognition solutions. Given that high-density surface EMG (HD-sEMG) signal contains rich temporal and spatial information, the multi-view spatial-temporal graph convolutional network (MSTGCN)is adopted as the basic classifier, and a feature extraction convolutional neural network (CNN) module is designed and integrated into MSTGCN to generate a new model called CNN-MSTGCN. The EMG pattern recognition experiments are conducted on HD-sEMG data of 17 gestures from 11 subjects. The ablation experiments show that each functional module of the proposed CNN-MSTGCN network has played a more or less positive role in improving the performance of EMG pattern recognition. The user-independent recognition experiments and the transfer learning-based cross-user recognition experiments verify the advantages of the proposed CNN-MSTGCN network in improving recognition rate and reducing user training burden. In the user-independent recognition experiments, CNN-MSTGCN achieves the recognition rate of 68%, which is significantly better than those obtained by residual network-50 (ResNet50, 47.5%, p < 0.001) and long-short-term-memory (LSTM, 57.1%, p=0.045). In the transfer learning-based cross-user recognition experiments, TL-CMSTGCN achieves an impressive recognition rate of 92.3%, which is significantly superior to both TL-ResNet50 (84.6%, p = 0.003) and TL-LSTM (85.3%, p = 0.008). The research results of this paper indicate that GNN has certain advantages in overcoming the impact of individual differences, and can be used to provide possible solutions for achieving robust EMG pattern recognition technology.

Recent trends and challenges of surface electromyography in prosthetic applications

Surface electromyography (sEMG) meets extensive applications in the field of prosthesis in the current period. The effectiveness of sEMG in prosthesis applications has been verified by numerous revolutionary developments and extensive research attempts. A large volume of research and literature works have explored and validated the vast use of these signals in prostheses as an assistive technology. The objective of this paper is to conduct a systematic review and offer a detailed overview of the work record in the prosthesis and myoelectric interfaces framework. This review utilized a systematic search strategy to identify published articles discussing the state-of-the-art applications of sEMG in prostheses (including upper limb prosthesis and lower limb prostheses). Relevant studies were identified using electronic databases such as PubMed, IEEE Explore, SCOPUS, ScienceDirect, Google Scholar and Web of Science. Out of 3791 studies retrieved from the databases, 188 articles were found to be potentially relevant (after screening of abstracts and application of inclusion-exclusion criteria) and included in this review. This review presents an investigative analysis of sEMG-based prosthetic applications to assist the readers in making further advancements in this field. It also discusses the fundamental advantages and disadvantages of using sEMG in prosthetic applications. It also includes some important guidelines to follow in order to improve the performance of sEMG-based prosthesis. The findings of this study support the widespread use of sEMG in prosthetics. It is concluded that sEMG-based prosthesis technology, still in its sprouting phase, requires significant explorations for further development. Supplementary investigations are necessary in the direction of making a seamless mechanism of biomechatronics for sEMG-based prosthesis by cohesive efforts of robotic researchers and biomedical engineers.

A hierarchical dynamic Bayesian learning network for EMG-based early prediction of voluntary movement intention

Decoding human action intention prior to motion onset with surface electromyograms (sEMG) is an emerging neuroengineering topic with interesting clinical applications such as intelligent control of powered prosthesis/exoskeleton devices. Despite extensive prior works in the related fields, it remains a technical challenge due to considerable variability of complex multi-muscle activation patterns in terms of volatile spatio-temporal characteristics. To address this issue, we first hypothesize that the inherent variability of the idle state immediately preceding the motion initiation needs to be addressed explicitly. We therefore design a hierarchical dynamic Bayesian learning network model that integrates an array of Gaussian mixture model - hidden Markov models (GMM-HMMs), where each GMM-HMM learns the multi-sEMG processes either during the idle state, or during the motion initiation phase of a particular motion task. To test the hypothesis and evaluate the new learning network, we design and build a upper-limb sEMG-joystick motion study system, and collect data from 11 healthy volunteers. The data collection protocol adapted from the psychomotor vigilance task includes repeated and randomized binary hand motion tasks (push or pull) starting from either of two designated idle states: relaxed (with minimal muscle tones), or prepared (with muscle tones). We run a series of cross-validation tests to examine the performance of the method in comparison with the conventional techniques. The results suggest that the idle state recognition favors the dynamic Bayesian model over a static classification model. The results also show a statistically significant improvement in motion prediction accuracy by the proposed method (93.83±6.41%) in comparison with the conventional GMM-HMM method (89.71±8.98%) that does not explicitly account for the idle state. Moreover, we examine the progress of prediction accuracy over the course of motion initiation and identify the important hidden states that warrant future research.

Multichannel Sensorimotor Integration with a Dexterous Artificial Hand

Background

People use their hands to perform sophisticated tasks like playing a musical instrument by integrating manifold and diverse sensations of touch with motor control strategies. In contrast, prosthetic hands lack the capacity for multichannel haptic feedback and multitasking functionality remains rudimentary. There is a dearth of research exploring the potential of upper limb absent (ULA) people to integrate multiple channels of haptic feedback into dexterous prosthetic hand control strategies.

Methods

In this paper, we designed a novel experimental paradigm for three ULA people and nine additional subjects to investigate their ability to integrate two simultaneously activated channels of context-specific haptic feedback into their dexterous artificial hand control strategies. Artificial neural networks (ANN) were designed for pattern recognition of the array of efferent electromyogram signals that controlled the dexterous artificial hand. ANNs were also used to classify the directions that objects were sliding across two tactile sensor arrays on the index (I) and little (L) fingertips of the robotic hand. The direction of sliding contact at each robotic fingertip was encoded by different stimulation frequencies of wearable vibrotactile actuators for haptic feedback. The subjects were tasked with implementing different control strategies with each finger simultaneously depending upon the perceived directions of sliding contact. This required the 12 subjects to concurrently control individual fingers of the artificial hand by successfully interpreting two channels of simultaneously activated context-specific haptic feedback.

Results

Subjects were able to accomplish this complex feat of multichannel sensorimotor integration with an overall accuracy of 95.53% ± 0.23%. While there was no statistically significant difference in the classification accuracy between ULA people and the other subjects, the ULA people required more time to correctly respond to the simultaneous haptic feedback slip signals, suggesting a higher cognitive load required by the ULA people.

Conclusion

ULA people can integrate multiple channels of simultaneously activated and nuanced haptic feedback with their control of individual fingers of an artificial hand. These findings provide a step toward empowering amputees to multitask with dexterous prosthetic hands, which remains an ongoing challenge.

Research on Lower Limb Step Speed Recognition Method Based on Electromyography

Wearable exoskeletons play an important role in people's lives, such as helping stroke and amputation patients to carry out rehabilitation training and so on. How to make the exoskeleton accurately judge the human action intention is the basic requirement to ensure that it can complete the corresponding task. Traditional exoskeleton control signals include pressure values, joint angles and acceleration values, which can only reflect the current motion information of the human lower limbs and cannot be used to predict motion. The electromyography (EMG) signal always occurs before a certain movement; it can be used to predict the target's gait speed and movement as the input signal. In this study, the generalization ability of a BP neural network and the timing property of a hidden Markov chain are used to properly fuse the two, and are finally used in the research of this paper. Experiments show that, using the same training samples, the recognition accuracy of the three-layer BP neural network is only 91%, while the recognition accuracy of the fusion discriminant model proposed in this paper can reach 95.1%. The results show that the fusion of BP neural network and hidden Markov chain has a strong solving ability for the task of wearable exoskeleton recognition of target step speed.

SE-TCN network for continuous estimation of upper limb joint angles

The maturity of human-computer interaction technology has made it possible to use surface electromyographic signals (sEMG) to control exoskeleton robots and intelligent prostheses. However, the available upper limb rehabilitation robots controlled by sEMG have the shortcoming of inflexible joints. This paper proposes a method based on a temporal convolutional network (TCN) to predict upper limb joint angles by sEMG. The raw TCN depth was expanded to extract the temporal features and save the original information. The timing sequence characteristics of the muscle blocks that dominate the upper limb movement are not apparent, leading to low accuracy of the joint angle estimation. Therefore, this study squeeze-and-excitation networks (SE-Net) to improve the network model of the TCN. Finally, seven movements of the human upper limb were selected for ten human subjects, recording elbow angle (EA), shoulder vertical angle (SVA), and shoulder horizontal angle (SHA) values during their movements. The designed experiment compared the proposed SE-TCN model with the backpropagation (BP) and long short-term memory (LSTM) networks. The proposed SE-TCN systematically outperformed the BP network and LSTM model by the mean RMSE values: by 25.0 and 36.8% for EA, by 38.6 and 43.6% for SHA, and by 45.6 and 49.5% for SVA, respectively. Consequently, its R² values exceeded those of BP and LSTM by 13.6 and 39.20% for EA, 19.01 and 31.72% for SHA, and 29.22 and 31.89% for SVA, respectively. This indicates that the proposed SE-TCN model has good accuracy and can be used to estimate the angles of upper limb rehabilitation robots in the future.

Continuous Estimation of Human Knee Joint Angles by Fusing Kinematic and Myoelectric Signals

Exoskeleton robot is an essential tool in active rehabilitation training for patients with lower limb motor dysfunctions. Accurate and real-time recognition in human motion intention is a great challenge in exoskeleton robot, which can be implemented by continues estimation of human joint angles. In this study, we innovatively proposed a novel feature-based convolutional neural network-bi-directional long-short term memory network (CNN-BiLSTM) model to predict the knee joint angles more accurately and in real time. We validated our method on a public dataset, including surface electromyography(sEMG) and inertial measurement unit (IMU) data of 10 healthy subjects during normal walking. Initially, features extraction from each modal data achieved feature-level fusion. Then the importance of each sEMG and IMU signal feature for knee joint angle prediction was quantified by ensemble feature scorer (EFS) and the number of features required for prediction while ensuring accuracy was simplified by profile likelihood maximization (PLM) algorithm. Finally, the CNN-BiLSTM model was created by using the determined simplest features to further fuse the spatio-temporal correlation of signals. The results indicated that the EFS and PLM algorithm could remove the feature redundancy perfectly and estimation performance would become better when bi-modal gait data were fused. For the estimation performance, the average root mean square error (RMSE), adjusted [Formula: see text] and pearson correlation coefficient (CC) of our algorithm were 4.07, 0.95, and 0.98, respectively, which was better than CNN, BiLSTM and other three traditional machine learning methods. In addition, the model test time was 62.47 ± 0.29 ms, which was less than the predicted horizon of 100 ms. The real-time performance and accuracy are satisfactory. Compared with previous works, our method has great advantages in feature selection and model design, which further improves the prediction accuracy. These promising results demonstrate that the proposed method has considerable potential to be applied to exoskeleton robot control.

Simultaneous Prediction of Wrist and Hand Motions via Wearable Ultrasound Sensing for Natural Control of Hand Prostheses

Simultaneous prediction of wrist and hand motions is essential for the natural interaction with hand prostheses. In this paper, we propose a novel multi-out Gaussian process (MOGP) model and a multi-task deep learning (MTDL) algorithm to achieve simultaneous prediction of wrist rotation (pronation/ supination)¹ and finger gestures for transradial amputees via a wearable ultrasound array. We target six finger gestures with concurrent wrist rotation in four transradial amputees. Results show that MOGP outperforms previously reported subclass discriminant analysis for both predictions of discrete finger gestures and continuous wrist rotation. Moreover, we find that MTDL has the potential to improve the accuracy of finger gesture prediction compared to MOGP and classification-specific deep learning, albeit at the expense of reducing the accuracy of wrist rotation prediction. Extended comparative analysis shows the superiority of ultrasound over surface electromyography. This paper prioritizes exploring the performance of wearable ultrasound on the simultaneous prediction of wrist and hand motions for transradial amputees, demonstrating the potential of ultrasound in future prosthetic control. Our ultrasound-based adaptive prosthetic control dataset (UltraPro) will be released to promote the development of the prosthetic community.

Sequential Learning on sEMGs in Short- and Long-term Situations via Self-training Semi-supervised Support Vector Machine

The purpose of this study it to assess the effect of sequential learning of self-training support vector machine (ST-S3VM) on short- and long-term surface electromyogram (sEMG) datasets. A machine learning-based supervised classi-fier is enabling stable, complex, and high-performance motion control. Unlabeled sEMG measurements are easy by the devel-opment of wearable sensing technology. Thus, semi-supervised learning methods are attracted attention to utilize unlabeled sEMG data for supervised classifier with a small amount of labeled data. To evaluate robustness of ST-S3VM in realistic conditions, two public datasets which respectively contain a short- and long-term dataset were used. We compared the performance of ST-S3VM with four-kinds of SVM classifiers. In both short- and long-term situations, ST combined classifiers (ST-SVM and ST-S3VM) showed higher performances than the methods without ST (SVM and S3VM). In some cases, ST-S3VM had the best performance, but in other cases, ST-SVM had better performance than ST-S3VM. In order to make better use of unlabeled data, we will develop ST-S3VM to reduce the impact of harmful unlabeled data.

Simultaneous control of 2DOF upper-limb prosthesis with body compensations-based control: a multiple cases study

Controlling several joints simultaneously is a common feature of natural arm movements. Robotic prostheses shall offer this possibility to their wearer. Yet, existing approaches to control a robotic upper-limb prosthesis from myoelectric interfaces do not satisfactorily respond to this need: standard methods provide sequential joint-by-joint motion control only; advanced pattern recognition-based approaches allow the control of a limited subset of synchronized multi-joint movements and remain complex to set up. In this paper, we exploit a control method of an upper-limb prosthesis based on body motion measurement called Compensations Cancellation Control (CCC). It offers a straightforward simultaneous control of the intermediate joints, namely the wrist and the elbow. Four transhumeral amputated participants performed the Refined Rolyan Clothespin Test with an experimental prosthesis alternatively running CCC and conventional joint-by-joint myoelectric control. Task performance, joint motions, body compensations and cognitive load were assessed. This experiment shows that CCC restores simultaneity between prosthetic joints while maintaining the level of performance of conventional myoelectric control (used on a daily basis by three participants), without increasing compensatory motions nor cognitive load.

A Hierarchical Hand Gesture Recognition Framework for Sports Referee Training-Based EMG and Accelerometer Sensors

To cultivate professional sports referees, we develop a sports referee training system, which can recognize whether a trainee wearing the Myo armband makes correct judging signals while watching a prerecorded professional game. The system has to correctly recognize a set of gestures related to official referee's signals (ORSs) and another set of gestures used to intuitively interact with the system. These two gesture sets involve both large motion and subtle motion gestures, and the existing sensor-based methods using handcrafted features do not work well on recognizing all kinds of these gestures. In this work, deep belief networks (DBNs) are utilized to learn more representative features for hand gesture recognition, and selective handcrafted features are combined with the DBN features to achieve more robust recognition results. Moreover, a hierarchical recognition scheme is designed to first recognize the input gesture as a large or subtle motion gesture, and the corresponding classifiers for large motion gestures and subtle motion gestures are further used to obtain the final recognition result. Moreover, the Myo armband consists of eight-channel surface electromyography (sEMG) sensors and an inertial measurement unit (IMU), and these heterogeneous signals can be fused to achieve better recognition accuracy. We take basketball as an example to validate the proposed training system, and the experimental results show that the proposed hierarchical scheme considering DBN features of multimodality data outperforms other methods.

Age-dependent Upper Limb Myoelectric Control Capability in Typically Developing Children

Research in EMG-based control of prostheses has mainly utilized adult subjects who have fully developed neuromuscular control. Little is known about children's ability to generate consistent EMG signals necessary to control artificial limbs with multiple degrees of freedom. As a first step to address this gap, experiments were designed to validate and benchmark two experimental protocols that quantify the ability to coordinate forearm muscle contractions in typically developing children. Able-bodied, healthy adults and children participated in our experiments that aimed to measure an individual's ability to use myoelectric control interfaces. In the first experiment, participants performed 8 repetitions of 16 different hand/wrist movements. Using offline classification analysis based on Support Vector Machine, we quantified their ability to consistently produce distinguishable muscle contraction patterns. We demonstrated that children had a smaller number of highly independent movements (can be classified with >90% accuracy) than adults did. The second experiment measured participants' ability to control the position of a cursor on a 1-DoF virtual slide using proportional EMG control with three different visuomotor gain levels. We found that children had higher failure rates and slower average target acquisitions than adults did, primarily due to longer correction times that did not improve over repetitive practice. We also found that the performance in both experiments was age-dependent in children. The results of this study provide novel insights into the technical and empirical basis to better understand neuromuscular development in children with upper-limb loss.

Towards Robust, Adaptive and Reliable Upper-limb Motion Estimation Using Machine Learning and Deep Learning--A Survey in Myoelectric Control

To develop multi-functional human-machine interfaces that can help disabled people reconstruct lost functions of upper-limbs, machine learning (ML) and deep learning (DL) techniques have been widely implemented to decode human movement intentions from surface electromyography (sEMG) signals. However, due to the high complexity of upper-limb movements and the inherent non-stable characteristics of sEMG, the usability of ML/DL based control schemes is still greatly limited in practical scenarios. To this end, tremendous efforts have been made to improve model robustness, adaptation, and reliability. In this article, we provide a systematic review on recent achievements, mainly from three categories: multi-modal sensing fusion to gain additional information of the user, transfer learning (TL) methods to eliminate domain shift impacts on estimation models, and post-processing approaches to obtain more reliable outcomes. Special attention is given to fusion strategies, deep TL frameworks, and confidence estimation. \textcolor{red}{Research challenges and emerging opportunities, with respect to hardware development, public resources, and decoding strategies, are also analysed to provide perspectives for future developments.

Discrete action control for prosthetic digits

We aim to develop a paradigm for simultaneous and independent control of multiple degrees of freedom (DOFs) for upper-limb prostheses. To that end, we introduce action control, a novel method to operate prosthetic digits with surface electromyography (EMG) based on multi-output, multi-class classification. At each time step, the decoder classifies movement intent for each controllable DOF into one of three categories: open, close, or stall (i.e., no movement). We implemented a real-time myoelectric control system using this method and evaluated it by running experiments with one unilateral and two bilateral amputees. Participants controlled a six-DOF bar interface on a computer display, with each DOF corresponding to a motor function available in multi-articulated prostheses. We show that action control can significantly and systematically outperform the state-of-the-art method of position control via multi-output regression in both task- and non-task-related measures. Using the action control paradigm, improvements in median task performance over regression-based control ranged from 20.14% to 62.32% for individual participants. Analysis of a post-experimental survey revealed that all participants rated action higher than position control in a series of qualitative questions and expressed an overall preference for the former. Action control has the potential to improve the dexterity of upper-limb prostheses. In comparison with regression-based systems, it only requires discrete instead of real-valued ground truth labels, typically collected with motion tracking systems. This feature makes the system both practical in a clinical setting and also suitable for bilateral amputation. This work is the first demonstration of myoelectric digit control in bilateral upper-limb amputees. Further investigation and preclinical evaluation are required to assess the translational potential of the method.

Finger Force Estimation using Motor Unit Discharges Across Forearm Postures

BACKGROUND

Myoelectric-based decoding has gained popularity in upper-limb neural-machine interfaces. Motor unit (MU) firings decomposed from surface electromyographic (EMG) signals can represent motor intent, but EMG properties at different arm configurations can change due to electrode shift and differing neuromuscular states. This study investigated whether isometric fingertip force estimation using MU firings is robust to forearm rotations from a neutral to either a fully pronated or supinated posture.

METHODS

We extracted MU information from high-density EMG of the extensor digitorum communis in two ways: (1) Decomposed EMG in all three postures (MU-AllPost); and (2) Decomposed EMG in neutral posture (MU-Neu), and extracted MUs (separation matrix) were applied to other postures. Populational MU firing frequency estimated forces scaled to subjects' maximum voluntary contraction (MVC) using a regression analysis. The results were compared with the conventional EMG-amplitude method.

RESULTS

We found largely similar root-mean-square errors (RMSE) for the two MU-methods, indicating that MU decomposition was robust to postural differences. MU-methods demonstrated lower RMSE in the ring (EMG = 6.23, MU-AllPost = 5.72, MU-Neu = 5.64 %MVC) and pinky (EMG = 6.12, MU-AllPost = 4.95, MU-Neu = 5.36 %MVC) fingers, with mixed results in the middle finger (EMG = 5.47, MU-AllPost = 5.52, MU-Neu = 6.19% MVC).

CONCLUSION

Our results suggest that MU firings can be extracted reliably with little influence from forearm posture, highlighting its potential as an alternative decoding scheme for robust and continuous control of assistive devices.

A parallel classification strategy to simultaneous control elbow, wrist, and hand movements

Background

In the field of myoelectric control systems, pattern recognition (PR) algorithms have become always more interesting for predicting complex electromyography patterns involving movements with more than 2 Degrees of Freedom (DoFs). The majority of classification strategies, used for the prosthetic control, are based on single, hierarchical and parallel linear discriminant analysis (LDA) classifiers able to discriminate up to 19 wrist/hand gestures (in the 3-DoFs case), considering both combined and discrete motions. However, these strategies were introduced to simultaneously classify only 2 DoFs and their use is limited by the lack of online performance measures. This study introduces a novel classification strategy based on the Logistic Regression (LR) algorithm with regularization parameter to provide simultaneous classification of 3 DoFs motion classes.

Methods

The parallel PR-based strategy was tested on 15 healthy subjects, by using only six surface EMG sensors. Twenty-seven discrete and complex elbow, hand and wrist motions were classified by keeping the number of electromyographic (EMG) electrodes to a bare minimum and the classification error rate under 10 %. To this purpose, the parallel classification strategy was implemented by using three classifiers one for each DoF: the “Elbow classifier”, the “Wrist classifier”, and the “Hand classifier” provided the simultaneous control of the elbow, hand, and wrist joints, respectively.

Results

Both the offline and real-time performance metrics were evaluated and compared with the LDA parallel classification results. The real-time recognition results were statistically better with the LR classifier with respect to the LDA classifier, for all motion classes (elbow, hand and wrist).

Conclusions

In this paper, a novel parallel PR-based strategy was proposed for classifying up to 3 DoFs: three joint classifiers were employed simultaneously for classifying 27 motion classes related to the elbow, wrist, and hand and promising results were obtained.

A musculoskeletal model driven by muscle synergy-derived excitations for hand and wrist movements

OBJECTIVE

Musculoskeletal model (MM) driven by electromyography (EMG) signals has been identified as a promising approach to predicting human motions in the control of prostheses and robots. However, muscle excitations in MMs are generally derived from the EMG signals of the targeted sensor covering the muscle, inconsistent with the fact that signals of a sensor are from multiple muscles considering signal crosstalk in actual situation. To identify more accurate muscle excitations for MM in the presence of crosstalk, we proposed a novel excitation-extracting method inspired by muscle synergy for simultaneously estimating hand and wrist movements.

APPROACH

Muscle excitations were firstly extracted using a two-step muscle synergy-derived method. Specifically, we calculated subject-specific muscle weighting matrix and corresponding profiles according to contributions of different muscles for movements derived from synergistic motion relation. Then, the improved excitations were used to simultaneously estimate hand and wrist movements through musculoskeletal modeling. Moreover, the offline comparison among the proposed method, traditional MM and regression methods, and an online test of the proposed method were conducted.

MAIN RESULTS

The offline experiments demonstrated that the proposed approach outperformed the EMG envelope-driven MM and three regression models with higher R and lower NRMSE. Furthermore, the comparison of excitations of two MMs validated the effectiveness of the proposed approach in extracting muscle excitations in the presence of crosstalk. The online test further indicated the superior performance of the proposed method than the MM driven by EMG envelopes.

SIGNIFICANCE

The proposed excitation-extracting method identified more accurate neural commands for MMs, providing a promising approach in rehabilitation and robot control to model the transformation from surface EMG to joint kinematics.

Simultaneous control of natural and extra degrees of freedom by isometric force and electromyographic activity in the muscle-to-force null space

OBJECTIVE

Muscle activation patterns in the muscle-to-force null space, i.e., patterns that do not generate task-relevant forces, may provide an opportunity for motor augmentation by allowing to control additional end-effectors simultaneously to natural limbs. Here we tested the feasibility of muscular null space control for augmentation by assessing simultaneous control of natural and extra degrees of freedom.

APPROACH

We instructed eight participants to control translation and rotation of a virtual 3D end-effector by simultaneous generation of isometric force at the hand and null space activity extracted in real-time from the electromyographic signals recorded from 15 shoulder and arm muscles. First, we identified the null space components that each participant could control more naturally by voluntary co-contraction. Then, participants performed several blocks of a reaching and holding task. They displaced an ellipsoidal cursor to reach one of nine targets by generating force, and simultaneously rotated the cursor to match the target orientation by activating null space components. We developed an information-theoretic metric, an index of difficulty defined as the sum of a spatial and a temporal term, to assess individual null space control ability for both reaching and holding.

MAIN RESULTS

On average, participants could reach the targets in most trials already in the first block (72%) and they improved with practice (maximum 93%) but holding performance remained lower (maximum 43%). As there was a high inter-individual variability in performance, we performed a simulation with different spatial and temporal task conditions to estimate those for which each individual participants would have performed best.

SIGNIFICANCE

Muscular null space control is feasible and may be used to control additional virtual or robotics end-effectors. However, decoding of motor commands must be optimized according to individual null space control ability.

Single channel surface electromyogram deconvolution is a useful pre-processing for myoelectric control

Myoelectric control requires fast and stable identification of a movement from data recorded from a comfortable and straightforward system. Here we consider a new real-time pre-processing method applied to a single differential surface electromyogram (EMG): deconvolution, providing an estimation of the cumulative firings of motor units. A 2 channel-10 class finger movement problem has been investigated on 10 healthy subjects. We have compared raw EMG and deconvolution signals, as sources of information for two specific classifiers (based on either Support Vector Machines or k-Nearest Neighbours), with classical time-domain input features selected using Mutual Component Analysis. The overall results show that, using the proposed pre-processing technique, classification performances statistically improve. For example, the true positive rates of the best-tested configurations were 80.9% and 86.3% when using the EMG and its deconvoluted signal, respectively. Even considering the limited dataset and range of classification approaches investigated, these preliminary results indicate the potential usefulness of the deconvolution pre-processing, which could be easily embedded in different myoelectric control applications.

Comparison of Myoelectric Control Schemes for Simultaneous Hand and Wrist Movement using Chronically Implanted Electromyography: A Case Series. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021;2021:6224-6230. [PMID: 34892537 PMCID: PMC10964936 DOI: 10.1109/embc46164.2021.9630845]

OBJECTIVE

A current biomedical engineering challenge is the development of a system that allows fluid control of multi-functional prosthetic devices through a human-machine interface. Here we probe this challenge by studying two subjects with trans-radial limb loss as they control a virtual hand and wrist system using 6 or 8 chronically implanted intramuscular electromyographic (iEMG) signals. The subjects successfully controlled a 4, 5, and 6 Degrees of Freedom (DoF's) virtual hand and wrist systems to perform a target matching task.

APPROACH

Two control systems were evaluated where one tied EMG features directly to movement directions (Direct Control) and the other method determines user intent in the context of prior training data (Linear Interpolation).

MAIN RESULTS

Subjects successfully matched most targets with both controllers but differences were seen as the complexity of the virtual limb system increased. The Direct Control method encountered difficulty due to crosstalk at higher DoF's. The Linear Interpolation method reduced crosstalk effects and outperformed Direct Control at higher DoF's. This work also studied the use of the Postural Control Algorithm to control the hand postures simultaneously with wrist degrees of freedom.

SIGNIFICANCE

This work presents preliminary evidence that the PC algorithm can be used in conjunction with wrist control, that Direct Control with iEMG signals allows stable 4-DoF control, and that EMG pre-processing using the Linear Interpolation method can improve performance at 5 and 6-DoF's.

Upper-Limb Electromyogram Classification of Reaching-to-Grasping Tasks Based on Convolutional Neural Networks for Control of a Prosthetic Hand

In recent years, myoelectric interfaces using surface electromyogram (EMG) signals have been developed for assisting people with physical disabilities. Especially, in the myoelectric interfaces for robotic hands or arms, decoding the user’s upper-limb movement intentions is cardinal to properly control the prosthesis. However, because previous experiments were implemented with only healthy subjects, the possibility of classifying reaching-to-grasping based on the EMG signals from the residual limb without the below-elbow muscles was not investigated yet. Therefore, we aimed to investigate the possibility of classifying reaching-to-grasping tasks using the EMG from the upper arm and upper body without considering wrist muscles for prosthetic users. In our study, seven healthy subjects, one trans-radial amputee, and one wrist amputee were participated and performed 10 repeatable 12 reaching-to-grasping tasks based on the Southampton Hand Assessment Procedure (SHAP) with 12 different weighted (light and heavy) objects. The acquired EMG was processed using the principal component analysis (PCA) and convolutional neural network (CNN) to decode the tasks. The PCA–CNN method showed that the average accuracies of the healthy subjects were 69.4 ± 11.4%, using only the EMG signals by the upper arm and upper body. The result with the PCA–CNN method showed 8% significantly higher accuracies than the result with the widely used time domain and auto-regressive-support vector machine (TDAR–SVM) method as 61.6 ± 13.7%. However, in the cases of the amputees, the PCA–CNN showed slightly lower performance. In addition, in the aspects of assistant daily living, because grip force is also important when grasping an object after reaching, the possibility of classifying the two light and heavy objects in each reaching-to-grasping task was also investigated. Consequently, the PCA–CNN method showed higher accuracy at 70.1 ± 9.8%. Based on our results, the PCA–CNN method can help to improve the performance of classifying reaching-to-grasping tasks without wrist EMG signals. Our findings and decoding method can be implemented to further develop a practical human–machine interface using EMG signals.

MuscleNET: mapping electromyography to kinematic and dynamic biomechanical variables by machine learning

Objective.This paper proposes machine learning models for mapping surface electromyography (sEMG) signals to regression of joint angle, joint velocity, joint acceleration, joint torque, and activation torque.Approach.The regression models, collectively known as MuscleNET, take one of four forms: ANN (forward artificial neural network), RNN (recurrent neural network), CNN (convolutional neural network), and RCNN (recurrent convolutional neural network). Inspired by conventional biomechanical muscle models, delayed kinematic signals were used along with sEMG signals as the machine learning model's input; specifically, the CNN and RCNN were modeled with novel configurations for these input conditions. The models' inputs contain either raw or filtered sEMG signals, which allowed evaluation of the filtering capabilities of the models. The models were trained using human experimental data and evaluated with different individual data.Main results.Results were compared in terms of regression error (using the root-mean-square) and model computation delay. The results indicate that the RNN (with filtered sEMG signals) and RCNN (with raw sEMG signals) models, both with delayed kinematic data, can extract underlying motor control information (such as joint activation torque or joint angle) from sEMG signals in pick-and-place tasks. The CNNs and RCNNs were able to filter raw sEMG signals.Significance.All forms of MuscleNET were found to map sEMG signals within 2 ms, fast enough for real-time applications such as the control of exoskeletons or active prostheses. The RNN model with filtered sEMG and delayed kinematic signals is particularly appropriate for applications in musculoskeletal simulation and biomechatronic device control.

Hand Gesture Recognition Based on sEMG Signal and Convolutional Neural Network

Recently, deep learning has become a promising technique for constructing gesture recognition classifiers from surface electromyography (sEMG) signals in human–computer interaction. In this paper, we propose a gesture recognition method with sEMG signals based on a deep multi-parallel convolutional neural network (CNN), which solves the problem that traditional machine learning methods may lose too much useful information during feature extraction. CNNs provide an efficient way to constrain the complexity of feedforward neural networks by weight sharing and restriction to local connections. Sophisticated feature extraction is to be avoided and hand gestures are to be classified directly. A multi-parallel and multi-convolution layer convolution structure is proposed to classify hand gestures. Experiment results show that in comparison with five traditional machine learning methods, the proposed method could achieve higher accuracy.

Brain-Machine Interfaces: Lessons for Prosthetic Hand Control

Brain-machine interfaces (BMI) are being developed to restore upper limb function for persons with spinal cord injury or other motor degenerative conditions. BMI and implantable sensors for myoelectric prostheses directly extract information from the central or peripheral nervous system to provide users with high fidelity control of their prosthetic device. Control algorithms have been highly transferable between the 2 technologies but also face common issues. In this review of the current state of the art in each field, the authors point out similarities and differences between the 2 technologies that may guide the implementation of common solutions to these challenges.

The Need to Work Arm in Arm: Calling for Collaboration in Delivering Neuroprosthetic Limb Replacements

Over the last few decades there has been a push to enhance the use of advanced prosthetics within the fields of biomedical engineering, neuroscience, and surgery. Through the development of peripheral neural interfaces and invasive electrodes, an individual's own nervous system can be used to control a prosthesis. With novel improvements in neural recording and signal decoding, this intimate communication has paved the way for bidirectional and intuitive control of prostheses. While various collaborations between engineers and surgeons have led to considerable success with motor control and pain management, it has been significantly more challenging to restore sensation. Many of the existing peripheral neural interfaces have demonstrated success in one of these modalities; however, none are currently able to fully restore limb function. Though this is in part due to the complexity of the human somatosensory system and stability of bioelectronics, the fragmentary and as-yet uncoordinated nature of the neuroprosthetic industry further complicates this advancement. In this review, we provide a comprehensive overview of the current field of neuroprosthetics and explore potential strategies to address its unique challenges. These include exploration of electrodes, surgical techniques, control methods, and prosthetic technology. Additionally, we propose a new approach to optimizing prosthetic limb function and facilitating clinical application by capitalizing on available resources. It is incumbent upon academia and industry to encourage collaboration and utilization of different peripheral neural interfaces in combination with each other to create versatile limbs that not only improve function but quality of life. Despite the rapidly evolving technology, if the field continues to work in divided "silos," we will delay achieving the critical, valuable outcome: creating a prosthetic limb that is right for the patient and positively affects their life.

An Improved Performance of Deep Learning Based on Convolution Neural Network to Classify the Hand Motion by Evaluating Hyper Parameter

High accuracy in pattern recognition based on electromyography(EMG) contributes to the effectiveness of prosthetics hand development. This study aimed to improve performance and simplify the deep learning pre-processing based on the convolution neural network (CNN) algorithm for classifying ten hand motion from two raw EMG signals. The main contribution of this study is the simplicity of pre-processing stage in classifier machine. For instance, the feature extraction process is not required. Furthermore, the performance of the classifier was improved by evaluating the best hyperparameter in deep learning architecture. To validate the performance of deep learning, the public dataset from ten subjects was evaluated. The performance of the proposed method was compared to other conventional machine learning, specifically LDA, SVM, and KNN. The CNN can discriminate the ten hand-motion based on raw EMG signal without handcrafts feature extraction. The results of the evaluation showed that CNN outperformed other classifiers. The average accuracy for all motion ranges between 0.77 and 0.93. The statistical t-test between using two-channel(CH1 and CH2) and single-channel(CH2) shows that there is no significant difference in accuracy with p-value >0.05. The proposed method was useful in the study of prosthetic hand, which required the simple architecture of machine learning and high performance in the classification.

Classification of Walking Environments Using Deep Learning Approach Based on Surface EMG Sensors Only

Classification of terrain is a vital component in giving suitable control to a walking assistive device for the various walking conditions. Although surface electromyography (sEMG) signals have been combined with inputs from other sensors to detect walking intention, no study has yet classified walking environments using sEMG only. Therefore, the purpose of this study is to classify the current walking environment based on the entire sEMG profile gathered from selected muscles in the lower extremities. The muscle activations of selected muscles in the lower extremities were measured in 27 participants while they walked over flat-ground, upstairs, downstairs, uphill, and downhill. An artificial neural network (ANN) was employed to classify these walking environments using the entire sEMG profile recorded for all muscles during the stance phase. The result shows that the ANN was able to classify the current walking environment with high accuracy of 96.3% when using activation from all muscles. When muscle activation from flexor/extensor groups in the knee, ankle, and metatarsophalangeal joints were used individually to classify the environment, the triceps surae muscle activation showed the highest classification accuracy of 88.9%. In conclusion, a current walking environment was classified with high accuracy using an ANN based on only sEMG signals.

Stable, simultaneous and proportional 4-DoF prosthetic hand control via synergy-inspired linear interpolation: a case series

BACKGROUND

Current commercial prosthetic hand controllers limit patients' ability to fully engage high Degree-of-Freedom (DoF) prosthetic hands. Available feedforward controllers rely on large training data sets for controller setup and a need for recalibration upon prosthesis donning. Recently, an intuitive, proportional, simultaneous, regression-based 3-DoF controller remained stable for several months without retraining by combining chronically implanted electromyography (ciEMG) electrodes with a K-Nearest-Neighbor (KNN) mapping technique. The training dataset requirements for simultaneous KNN controllers increase exponentially with DoF, limiting the realistic development of KNN controllers in more than three DoF. We hypothesize that a controller combining linear interpolation, the muscle synergy framework, and a sufficient number of ciEMG channels (at least two per DoF), can allow stable, high-DoF control.

METHODS

Two trans-radial amputee subjects, S6 and S8, were implanted with percutaneously interfaced bipolar intramuscular electrodes. At the time of the study, S6 and S8 had 6 and 8 bipolar EMG electrodes, respectively. A Virtual Reality (VR) system guided users through single and paired training movements in one 3-DoF and four different 4-DoF cases. A linear model of user activity was built by partitioning EMG feature space into regions bounded by vectors of steady state movement EMG patterns. The controller evaluated online EMG signals by linearly interpolating the movement class labels for surrounding trained EMG movements. This yields a simultaneous, continuous, intuitive, and proportional controller. Controllers were evaluated in 3-DoF and 4-DoF through a target-matching task in which subjects controlled a virtual hand to match 80 targets spanning the available movement space. Match Percentage, Time-To-Target, and Path Efficiency were evaluated over a 10-month period based on subject availability.

RESULTS AND CONCLUSIONS

In 3-DoF, S6 and S8 matched most targets and demonstrated stable control after 8 and 10 months, respectively. In 4-DoF, both subjects initially found two of four 4-DoF controllers usable, matching most targets. S8 4-DoF controllers were stable, and showed improving trends over 7-9 months without retraining or at-home practice. S6 4-DoF controllers were unstable after 7 months without retraining. These results indicate that the performance of the controller proposed in this study may remain stable, or even improve, provided initial viability and a sufficient number of EMG channels. Overall, this study demonstrates a controller capable of stable, simultaneous, proportional, intuitive, and continuous control in 3-DoF for up to ten months and in 4-DoF for up to nine months without retraining or at-home use with minimal training times.

Control Strategies and Performance Assessment of Upper-Limb TMR Prostheses: A Review

The evolution of technological and surgical techniques has made it possible to obtain an even more intuitive control of multiple joints using advanced prosthetic systems. Targeted Muscle Reinnervation (TMR) is considered to be an innovative and relevant surgical technique for improving the prosthetic control for people with different amputation levels of the limb. Indeed, TMR surgery makes it possible to obtain reinnervated areas that act as biological amplifiers of the motor control. On the technological side, a great deal of research has been conducted in order to evaluate various types of myoelectric prosthetic control strategies, whether direct control or pattern recognition-based control. In the literature, different control performance metrics, which have been evaluated on TMR subjects, have been introduced, but no accepted reference standard defines the better strategy for evaluating the prosthetic control. Indeed, the presence of several evaluation tests that are based on different metrics makes it difficult the definition of standard guidelines for comprehending the potentiality of the proposed control systems. Additionally, there is a lack of evidence about the comparison of different evaluation approaches or the presence of guidelines on the most suitable test to proceed for a TMR patients case study. Thus, this review aims at identifying these limitations by examining the several studies in the literature on TMR subjects, with different amputation levels, and proposing a standard method for evaluating the control performance metrics.

Activities of daily living with bionic arm improved by combination training and latching filter in prosthesis control comparison

BACKGROUND

Advanced prostheses can restore function and improve quality of life for individuals with amputations. Unfortunately, most commercial control strategies do not fully utilize the rich control information from residual nerves and musculature. Continuous decoders can provide more intuitive prosthesis control using multi-channel neural or electromyographic recordings. Three components influence continuous decoder performance: the data used to train the algorithm, the algorithm, and smoothing filters on the algorithm's output. Individual groups often focus on a single decoder, so very few studies compare different decoders using otherwise similar experimental conditions.

METHODS

We completed a two-phase, head-to-head comparison of 12 continuous decoders using activities of daily living. In phase one, we compared two training types and a smoothing filter with three algorithms (modified Kalman filter, multi-layer perceptron, and convolutional neural network) in a clothespin relocation task. We compared training types that included only individual digit and wrist movements vs. combination movements (e.g., simultaneous grasp and wrist flexion). We also compared raw vs. nonlinearly smoothed algorithm outputs. In phase two, we compared the three algorithms in fragile egg, zipping, pouring, and folding tasks using the combination training and smoothing found beneficial in phase one. In both phases, we collected objective, performance-based (e.g., success rate), and subjective, user-focused (e.g., preference) measures.

RESULTS

Phase one showed that combination training improved prosthesis control accuracy and speed, and that the nonlinear smoothing improved accuracy but generally reduced speed. Phase one importantly showed simultaneous movements were used in the task, and that the modified Kalman filter and multi-layer perceptron predicted more simultaneous movements than the convolutional neural network. In phase two, user-focused metrics favored the convolutional neural network and modified Kalman filter, whereas performance-based metrics were generally similar among all algorithms.

CONCLUSIONS

These results confirm that state-of-the-art algorithms, whether linear or nonlinear in nature, functionally benefit from training on more complex data and from output smoothing. These studies will be used to select a decoder for a long-term take-home trial with implanted neuromyoelectric devices. Overall, clinical considerations may favor the mKF as it is similar in performance, faster to train, and computationally less expensive than neural networks.

Learning regularized representations of categorically labelled surface EMG enables simultaneous and proportional myoelectric control

Background

Processing the surface electromyogram (sEMG) to decode movement intent is a promising approach for natural control of upper extremity prostheses. To this end, this paper introduces and evaluates a new framework which allows for simultaneous and proportional myoelectric control over multiple degrees of freedom (DoFs) in real-time. The framework uses multitask neural networks and domain-informed regularization in order to automatically find nonlinear mappings from the forearm sEMG envelope to multivariate and continuous encodings of concurrent hand- and wrist kinematics, despite only requiring categorical movement instruction stimuli signals for calibration.

Methods

Forearm sEMG with 8 channels was collected from healthy human subjects (N = 20) and used to calibrate two myoelectric control interfaces, each with two output DoFs. The interfaces were built from (I) the proposed framework, termed Myoelectric Representation Learning (MRL), and, to allow for comparisons, from (II) a standard pattern recognition framework based on Linear Discriminant Analysis (LDA). The online performances of both interfaces were assessed with a Fitts’s law type test generating 5 quantitative performance metrics. The temporal stabilities of the interfaces were evaluated by conducting identical tests without recalibration 7 days after the initial experiment session.

Results

Metric-wise two-way repeated measures ANOVA with factors method (MRL vs LDA) and session (day 1 vs day 7) revealed a significant (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p<0.05$$\end{document}p<0.05) advantage for MRL over LDA in 5 out of 5 performance metrics, with metric-wise effect sizes (Cohen’s \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d$$\end{document}d) separating MRL from LDA ranging from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left|d\right|=0.62$$\end{document}d=0.62 to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left|d\right|=1.13$$\end{document}d=1.13. No significant effect on any metric was detected for neither session nor interaction between method and session, indicating that none of the methods deteriorated significantly in control efficacy during one week of intermission.

Conclusions

The results suggest that MRL is able to successfully generate stable mappings from EMG to kinematics, thereby enabling myoelectric control with real-time performance superior to that of the current commercial standard for pattern recognition (as represented by LDA). It is thus postulated that the presented MRL approach can be of practical utility for muscle-computer interfaces.

Simultaneous control of multiple functions of bionic hand prostheses: Performance and robustness in end users

Myoelectric hand prostheses are usually controlled with two bipolar electrodes located on the flexor and extensor muscles of the residual limb. With clinically established techniques, only one function can be controlled at a time. This is cumbersome and limits the benefit of additional functions offered by modern prostheses. Extensive research has been conducted on more advanced control techniques, but the clinical impact has been limited, mainly due to the lack of reliability in real-world conditions. We implemented a regression-based control approach that allows for simultaneous and proportional control of two degrees of freedom and evaluated it on five prosthetic end users. In the evaluation of tasks mimicking daily life activities, we included factors that limit reliability, such as tests in different arm positions and on different days. The regression approach was robust over multiple days and only slightly affected by changing in the arm position. Additionally, the regression approach outperformed two clinical control approaches in most conditions.

Dimensionality analysis of forearm muscle activation for myoelectric control in transradial amputees

Establishing a natural communication interface between the user and the terminal device is one of the central challenges of hand neuroprosthetics research. Surface electromyography (EMG) is the most common source of neural signals for interpreting a user’s intent in these interfaces. However, how the capacity of EMG generation is affected by various clinical parameters remains largely unknown. In this study, we examined the EMG activity of forearm muscles recorded from 11 transradially amputated subjects who performed a wide range of movements. EMG recordings from 40 able-bodied subjects were also analyzed to provide comparative benchmarks. By using non-negative matrix factorization, we extracted the synergistic EMG patterns for each subject to estimate the dimensionality of muscle control, under the framework of motor synergies. We found that amputees exhibited less than four synergies (with substantial variability related to the length of remaining limb and age), whereas able-bodied subjects commonly demonstrate five or more synergies. The results of this study provide novel insight into the muscle synergy framework and the design of natural myoelectric control interfaces.

Myoelectric digit action decoding with multi-output, multi-class classification: an offline analysis

The ultimate goal of machine learning-based myoelectric control is simultaneous and independent control of multiple degrees of freedom (DOFs), including wrist and digit artificial joints. For prosthetic finger control, regression-based methods are typically used to reconstruct position/velocity trajectories from surface electromyogram (EMG) signals. Unfortunately, such methods have thus far met with limited success. In this work, we propose action decoding, a paradigm-shifting approach for independent, multi-digit movement intent prediction based on multi-output, multi-class classification. At each moment in time, our algorithm decodes movement intent for each available DOF into one of three classes: open, close, or stall (i.e., no movement). Despite using a classifier as the decoder, arbitrary hand postures are possible with our approach. We analyse a public dataset previously recorded and published by us, comprising measurements from 10 able-bodied and two transradial amputee participants. We demonstrate the feasibility of using our proposed action decoding paradigm to predict movement action for all five digits as well as rotation of the thumb. We perform a systematic offline analysis by investigating the effect of various algorithmic parameters on decoding performance, such as feature selection and choice of classification algorithm and multi-output strategy. The outcomes of the offline analysis presented in this study will be used to inform the real-time implementation of our algorithm. In the future, we will further evaluate its efficacy with real-time control experiments involving upper-limb amputees.

Accurate recognition of lower limb ambulation mode based on surface electromyography and motion data using machine learning

Background and Objective The lower limb activity of recognition of the elderly, the weak, the disabled and the sick is an irreplaceable role in the caring of daily life. The main purpose of this study is to assess the feasibility of using the surface electromyography (sEMG) signal and inertial measurement units (IMUs) data to determine the optimal fusion features and classifier for the study of daily ambulation mode recognition. Methods We have carried out several steps of experiments to obtain and test the optimal combination of the sEMG data and the body motion data at the feature level and the most suitable machine learning classification algorithm. Firstly, the sEMG and IMUs signals of eighteen participants performing four different ambulatory activities have recorded using wearable sensors. Secondly, several features of the sEMG sensors and IMU data were extracted and tested by the Markov Random Field based Fisher-Markov feature selector. Finally, four ML classifiers with several feature combinations were estimated with sensitivity, precision and recognition accurate rate of ambulatory activity classification. Results The results of this work showed that all selected features were significantly statistical difference in four ambulation modes. The principal component analysis was used to reduce the dimension of selected sEMG features and IMU features to form a fusion feature input support vector machine classifier, which could predict ambulatory activities with good classification performance. Conclusions It is concluded that the results demonstrate the feasibility of the ML classification model, which could provide a more novel way to guarantee the recognition rate and effectiveness of monitor daily ambulatory activity.

Should Hands Be Restricted When Measuring Able-Bodied Participants to Evaluate Machine Learning Controlled Prosthetic Hands?

OBJECTIVE

When evaluating methods for machine-learning controlled prosthetic hands, able-bodied participants are often recruited, for practical reasons, instead of participants with upper limb absence (ULA). However, able-bodied participants have been shown to often perform myoelectric control tasks better than participants with ULA. It has been suggested that this performance difference can be reduced by restricting the wrist and hand movements of able-bodied participants. However, the effect of such restrictions on the consistency and separability of the electromyogram's (EMG) features remains unknown. The present work investigates whether the EMG separability and consistency between unaffected and affected arms differ and whether they change after restricting the unaffected limb in persons with ULA.

METHODS

Both arms of participants with unilateral ULA were compared in two conditions: with the unaffected hand and wrist restricted or not. Furthermore, it was tested if the effect of arm and restriction is influenced by arm posture (arm down, arm in front, or arm up).

RESULTS

Fourteen participants (two women, age = 53.4±4.05) with acquired transradial limb loss were recruited. We found that the unaffected limb generated more separated EMG than the affected limb. Furthermore, restricting the unaffected hand and wrist lowered the separability of the EMG when the arm was held down.

CONCLUSION

Limb restriction is a viable method to make the EMG of able-bodied participants more similar to that of participants with ULA.

SIGNIFICANCE

Future research that evaluates methods for machine learning controlled hands in able-bodied participants should restrict the participants' hand and wrist.

Intent Prediction Based on Biomechanical Coordination of EMG and Vision-Filtered Gaze for End-Point Control of an Arm Prosthesis

We propose a novel controller for powered prosthetic arms, where fused EMG and gaze data predict the desired end-point for a full arm prosthesis, which could drive the forward motion of individual joints. We recorded EMG, gaze, and motion-tracking during pick-and-place trials with 7 able-bodied subjects. Subjects positioned an object above a random target on a virtual interface, each completing around 600 trials. On average across all trials and subjects gaze preceded EMG and followed a repeatable pattern that allowed for prediction. A computer vision algorithm was used to extract the initial and target fixations and estimate the target position in 2D space. Two SVRs were trained with EMG data to predict the x- and y- position of the hand; results showed that the y-estimate was significantly better than the x-estimate. The EMG and gaze predictions were fused using a Kalman Filter-based approach, and the positional error from using EMG-only was significantly higher than the fusion of EMG and gaze. The final target position Root Mean Squared Error (RMSE) decreased from 9.28 cm with an EMG-only prediction to 6.94 cm when using a gaze-EMG fusion. This error also increased significantly when removing some or all arm muscle signals. However, using fused EMG and gaze, there were no significant difference between predictors that included all muscles, or only a subset of muscles.