Synergy matrices to estimate fluid wrist movements by surface electromyography

Exploring Muscle Synergies for Performance Enhancement and Learning in Myoelectric Control Maps

This work proposes two myoelectric control maps based on a DoF-wise synergy algorithm, inspired by human motor control studies. One map, called intuitive, matches control outputs with body movement directions. The second one, named non-intuitive, takes advantage of different synergies contribution to each DoF, without specific correlation to body movement directions. The effectiveness and learning process for the two maps is evaluated through performance metrics in ten able-bodied individuals. The analysis was conducted using a 2-DoFs center-reach-out task and a survey. Results showed equivalent performance and perception for both mappings. However, learning is only visible in subjects that performed better in non-intuitive mapping, that required some familiarization to then exploit its features. Most of the myoelectric control designs use intuitive mappings. Nevertheless, non-intuitive mapping could provide more design flexibility, which can be especially interesting for patients with motor disabilities.

Recent trends and challenges of surface electromyography in prosthetic applications

Surface electromyography (sEMG) meets extensive applications in the field of prosthesis in the current period. The effectiveness of sEMG in prosthesis applications has been verified by numerous revolutionary developments and extensive research attempts. A large volume of research and literature works have explored and validated the vast use of these signals in prostheses as an assistive technology. The objective of this paper is to conduct a systematic review and offer a detailed overview of the work record in the prosthesis and myoelectric interfaces framework. This review utilized a systematic search strategy to identify published articles discussing the state-of-the-art applications of sEMG in prostheses (including upper limb prosthesis and lower limb prostheses). Relevant studies were identified using electronic databases such as PubMed, IEEE Explore, SCOPUS, ScienceDirect, Google Scholar and Web of Science. Out of 3791 studies retrieved from the databases, 188 articles were found to be potentially relevant (after screening of abstracts and application of inclusion-exclusion criteria) and included in this review. This review presents an investigative analysis of sEMG-based prosthetic applications to assist the readers in making further advancements in this field. It also discusses the fundamental advantages and disadvantages of using sEMG in prosthetic applications. It also includes some important guidelines to follow in order to improve the performance of sEMG-based prosthesis. The findings of this study support the widespread use of sEMG in prosthetics. It is concluded that sEMG-based prosthesis technology, still in its sprouting phase, requires significant explorations for further development. Supplementary investigations are necessary in the direction of making a seamless mechanism of biomechatronics for sEMG-based prosthesis by cohesive efforts of robotic researchers and biomedical engineers.

Continuous Estimation of Finger and Wrist Joint Angles Using a Muscle Synergy Based Musculoskeletal Model

Recently, many muscle synergy-based human motion prediction models and algorithms have been proposed. In this study, the muscle synergies extracted from electromyography (EMG) data were used to construct a musculoskeletal model (MSM) to predict the joint angles of the wrist, thumb, index finger, and middle finger. EMG signals were analyzed using independent component analysis to reduce signal noise and task-irrelevant artifacts. The weights of each independent component (IC) were converted into a heat map related to the motion pattern and compared with human anatomy to find a different number of ICs matching the motion pattern. Based on the properties of the MSM, non-negative matrix factorization was used to extract muscle synergies from selected ICs that represent the extensor and flexor muscle groups. The effects of these choices on the prediction accuracy was also evaluated. The performance of the model was evaluated using the correlation coefficient (CC) and normalized root-mean-square error (NRMSE). The proposed method has a higher prediction accuracy than those of traditional methods, with an average CC of 92.0% and an average NRMSE of 10.7%.

Frequency-Domain sEMG Classification Using a Single Sensor

Working towards the development of robust motion recognition systems for assistive technology control, the widespread approach has been to use a plethora of, often times, multi-modal sensors. In this paper, we develop single-sensor motion recognition systems. Utilising the peripheral nature of surface electromyography (sEMG) data acquisition, we optimise the information extracted from sEMG sensors. This allows the reduction in sEMG sensors or provision of contingencies in a system with redundancies. In particular, we process the sEMG readings captured at the trapezius descendens and platysma muscles. We demonstrate that sEMG readings captured at one muscle contain distinct information on movements or contractions of other agonists. We used the trapezius and platysma muscle sEMG data captured in able-bodied participants and participants with tetraplegia to classify shoulder movements and platysma contractions using white-box supervised learning algorithms. Using the trapezius sensor, shoulder raise is classified with an accuracy of 99%. Implementing subject-specific multi-class classification, shoulder raise, shoulder forward and shoulder backward are classified with a 94% accuracy amongst object raise and shoulder raise-and-hold data in able bodied adults. A three-way classification of the platysma sensor data captured with participants with tetraplegia achieves a 95% accuracy on platysma contraction and shoulder raise detection.

A novel energy-motion model for continuous sEMG decoding: from muscle energy to motor pattern

At present, sEMG-based gesture recognition requires vast amounts of training data; otherwise it is limited to a few gestures. Objective. This paper presents a novel dynamic energy model that decodes continuous hand actions by training small amounts of sEMG data. Approach. The activation of forearm muscles can set the corresponding fingers in motion or state with movement trends. The moving fingers store kinetic energy, and the fingers with movement trends store potential energy. The kinetic energy and potential energy in each finger are dynamically allocated due to the adaptive-coupling mechanism of five-fingers in actual motion. Meanwhile, the sum of the two energies remains constant at a certain muscle activation. We regarded hand movements with the same direction of acceleration for five-finger as the same in energy mode and divided hand movements into ten energy modes. Independent component analysis and machine learning methods were used to model associations between sEMG signals and energy modes and expressed gestures by energy form adaptively. This theory imitates the self-adapting mechanism in actual tasks. Thus, ten healthy subjects were recruited, and three experiments mimicking activities of daily living were designed to evaluate the interface: (1) the expression of untrained gestures, (2) the decoding of the amount of single-finger energy, and (3) real-time control. Main results. (1) Participants completed the untrained hand movements (100/100, p < 0.0001). (2) The interface performed better than chance in the experiment where participants pricked balloons with a needle tip (779/1000, p < 0.0001). (3) In the experiment where participants punched a hole in the plasticine on the balloon, the success rate was over 95% (97.67 ± 5.04%, p < 0.01). Significance. The model can achieve continuous hand actions with speed or force information by training small amounts of sEMG data, which reduces learning task complexity.

Comparison of muscle synergies extracted from both legs during cycling at different mechanical conditions

Muscle synergies are the building blocks for generating movement by the central nervous system (CNS). According to this hypothesis, CNS decreases the complexity of motor control by combination of a small number of muscle synergies. The aim of this work is to investigate similarity of muscle synergies during cycling across various mechanical conditions. Twenty healthy subjects performed three 6- min cycling tasks at over a range of rotational speed (40, 50, and 60 rpm) and resistant torque (3, 5, and 7 N/m). Surface electromyography (sEMG) signals were recorded during pedaling from eight muscles of the right and left legs. We extracted four synchronous muscle synergies by using the non-negative matrix factorization (NMF) method. Mean and standard deviation of the goodness of the signal reconstruction (R²) for all subjects was obtained 0.9898 ± 0.0535. We investigated the functional roles of both leg muscles during cycling by synchronous muscle synergy extraction. We compared the muscle synergies extracted from all subjects in all mechanical conditions. The total mean and standard deviation of the similarity of synergy vectors for all subjects in all mechanical conditions was obtained 0.8788 ± 0.0709. We found the high degrees of similarity among the sets of synchronous muscle synergies across mechanical conditions and also across different subjects. Our results demonstrated that different subjects at different mechanical conditions use the same motor control strategies for cycling, despite inter-individual variability of muscle patterns.

Design and Assessment of Control Maps for Multi-Channel sEMG-Driven Prostheses and Supernumerary Limbs

Proportional and simultaneous control algorithms are considered as one of the most effective ways of mapping electromyographic signals to an artificial device. However, the applicability of these methods is limited by the high number of electromyographic features that they require to operate-typically twice as many the actuators to be controlled. Indeed, extracting many independent electromyographic signals is challenging for a number of reasons-ranging from technological to anatomical. On the contrary, the number of actively moving parts in classic prostheses or extra-limbs is often high. This paper faces this issue, by proposing and experimentally assessing a set of algorithms which are capable of proportionally and simultaneously control as many actuators as there are independent electromyographic signals available. Two sets of solutions are considered. The first uses as input electromyographic signals only, while the second adds postural measurements to the sources of information. At first, all the proposed algorithms are experimentally tested in terms of precision, efficiency, and usability on twelve able-bodied subjects, in a virtual environment. A state-of-the-art controller using twice the amount of electromyographic signals as input is adopted as benchmark. We then performed qualitative tests, where the maps are used to control a prototype of upper limb prosthesis. The device is composed of a robotic hand and a wrist implementing active prono-supination movement. Eight able-bodied subjects participated to this second round of testings. Finally, the proposed strategies were tested in exploratory experiments involving two subjects with limb loss. Results coming from the evaluations in virtual and realistic settings show encouraging results and suggest the effectiveness of the proposed approach.

Real-time, simultaneous myoelectric control using a convolutional neural network

The evolution of deep learning techniques has been transformative as they have allowed complex mappings to be trained between control inputs and outputs without the need for feature engineering. In this work, a myoelectric control system based on convolutional neural networks (CNN) is proposed as a possible alternative to traditional approaches that rely on specifically designed features. This CNN-based system is validated using a real-time Fitts' law style target acquisition test requiring single and combined wrist motions. The performance of the proposed system is then compared to that of a standard support vector machine (SVM) based myoelectric system using a set of time-domain features. Despite the prevalence and demonstrated performance of these well-known features, no significant difference (p>0.05) was found between the two methods for any of the computed control metrics. This demonstrates the potential for automated learning approaches to extract complex and rich information from stochastic biological signals. This first evaluation of the usability of a CNN in a real-time myoelectric control environment provides a basis for further exploration.

Evaluation of matrix factorisation approaches for muscle synergy extraction

The muscle synergy concept provides a widely-accepted paradigm to break down the complexity of motor control. In order to identify the synergies, different matrix factorisation techniques have been used in a repertoire of fields such as prosthesis control and biomechanical and clinical studies. However, the relevance of these matrix factorisation techniques is still open for discussion since there is no ground truth for the underlying synergies. Here, we evaluate factorisation techniques and investigate the factors that affect the quality of estimated synergies. We compared commonly used matrix factorisation methods: Principal component analysis (PCA), Independent component analysis (ICA), Non-negative matrix factorization (NMF) and second-order blind identification (SOBI). Publicly available real data were used to assess the synergies extracted by each factorisation method in the classification of wrist movements. Synthetic datasets were utilised to explore the effect of muscle synergy sparsity, level of noise and number of channels on the extracted synergies. Results suggest that the sparse synergy model and a higher number of channels would result in better estimated synergies. Without dimensionality reduction, SOBI showed better results than other factorisation methods. This suggests that SOBI would be an alternative when a limited number of electrodes is available but its performance was still poor in that case. Otherwise, NMF had the best performance when the number of channels was higher than the number of synergies. Therefore, NMF would be the best method for muscle synergy extraction.

Preliminary results toward a naturally controlled multi-synergistic prosthetic hand

Robotic hands embedding human motor control principles in their mechanical design are getting increasing interest thanks to their simplicity and robustness, combined with good performance. Another key aspect of these hands is that humans can use them very effectively thanks to the similarity of their behavior with real hands. Nevertheless, controlling more than one degree of actuation remains a challenging task. In this paper, we take advantage of these characteristics in a multi-synergistic prosthesis. We propose an integrated setup composed of Pisa/IIT SoftHand 2 and a control strategy which simultaneously and proportionally maps the human hand movements to the robotic hand. The control technique is based on a combination of non-negative matrix factorization and linear regression algorithms. It also features a real-time continuous posture compensation of the electromyographic signals based on an IMU. The algorithm is tested on five healthy subjects through an experiment in a virtual environment. In a separate experiment, the efficacy of the posture compensation strategy is evaluated on five healthy subjects and, finally, the whole setup is successfully tested in performing realistic daily life activities.

Robust extraction of basis functions for simultaneous and proportional myoelectric control via sparse non-negative matrix factorization

OBJECTIVE

This paper proposes a novel simultaneous and proportional multiple degree of freedom (DOF) myoelectric control method for active prostheses.

APPROACH

The approach is based on non-negative matrix factorization (NMF) of surface EMG signals with the inclusion of sparseness constraints. By applying a sparseness constraint to the control signal matrix, it is possible to extract the basis information from arbitrary movements (quasi-unsupervised approach) for multiple DOFs concurrently.

MAIN RESULTS

In online testing based on target hitting, able-bodied subjects reached a greater throughput (TP) when using sparse NMF (SNMF) than with classic NMF or with linear regression (LR). Accordingly, the completion time (CT) was shorter for SNMF than NMF or LR. The same observations were made in two patients with unilateral limb deficiencies.

SIGNIFICANCE

The addition of sparseness constraints to NMF allows for a quasi-unsupervised approach to myoelectric control with superior results with respect to previous methods for the simultaneous and proportional control of multi-DOF. The proposed factorization algorithm allows robust simultaneous and proportional control, is superior to previous supervised algorithms, and, because of minimal supervision, paves the way to online adaptation in myoelectric control.

On the use of higher-order tensors to model muscle synergies

The muscle synergy concept provides the best framework to understand motor control and it has been recently utilised in many applications such as prosthesis control. The current muscle synergy model relies on decomposing multi-channel surface Electromyography (EMG) signals into a synergy matrix (spatial mode) and its weighting function (temporal mode). This is done using several matrix factorisation techniques, with Non-negative matrix factorisation (NMF) being the most prominent method. Here, we introduce a 4^th-order tensor muscle synergy model that extends the current state of the art by taking spectral information and repetitions (movements) into account. This adds more depth to the model and provides more synergistic information. In particular, we illustrate a proof-of-concept study where the Tucker3 tensor decomposition model was applied to a subset of wrist movements from the Ninapro database. The results showed the potential of Tucker3 tensor factorisation in finding patterns of muscle synergies with information about the movements and highlights the differences between the current and proposed model.

Classification of hand and wrist tasks of unknown force levels using muscle synergies

Muscle synergies have been proposed as a way for the central nervous system (CNS) to simplify the generation of motor commands and they have been shown to explain a large portion of the variation in the muscle patterns across a variety of conditions. However, whether human subjects are able to control prostheses proportionally with a small set of synergies has not been tested directly. Here we investigated if muscle synergies can be used to identify different wrist and hand motions. We recorded electromyographic (EMG) activity from eight arm muscles while the subjects exerted seven different intensity levels during the motions when performing seven classes of hand and wrist motion. From these data we extracted the muscle synergies and classified the tasks associated to each contraction intensity profile by linear discriminant analysis (LDA). We compared the performance obtained using muscle synergies with the performance of using the mean absolute values (MAV) as a feature. Also, the consistency of extracted muscle synergies was studied across intensity variations. While the synergies showed relative consistency particularly across closer intensity levels, average classification results generated with the synergies were less accurate than MAVs. These results indicate that although the performance of muscle synergies was very close to MAVs, they do not provide additional information for task identification across different exerted intensity levels.

Real-Time Task Discrimination for Myoelectric Control Employing Task-Specific Muscle Synergies

We present a novel formulation that employs task-specific muscle synergies and state-space representation of neural signals to tackle the challenging myoelectric control problem for lower arm prostheses. The proposed framework incorporates information about muscle configurations, e.g., muscles acting synergistically or in agonist/antagonist pairs, using the hypothesis of muscle synergies. The synergy activation coefficients are modeled as the latent system state and are estimated using a constrained Kalman filter. These task-dependent synergy activation coefficients are estimated in real-time from the electromyogram (EMG) data and are used to discriminate between various tasks. The task discrimination is helped by a post-processing algorithm that uses posterior probabilities. The proposed algorithm is robust as well as computationally efficient, yielding a decision with > 90% discrimination accuracy in approximately 3 ms . The real-time performance and controllability of the algorithm were evaluated using the targeted achievement control (TAC) test. The proposed algorithm outperformed common machine learning algorithms for single- as well as multi-degree-of-freedom (DOF) tasks in both off-line discrimination accuracy and real-time controllability (p < 0.01).

The role of muscle synergies in myoelectric control: trends and challenges for simultaneous multifunction control

Myoelectric control is filled with potential to significantly change human-robot interaction due to the ability to non-invasively measure human motion intent. However, current control schemes have struggled to achieve the robust performance that is necessary for use in commercial applications. As demands in myoelectric control trend toward simultaneous multifunctional control, multi-muscle coordinations, or synergies, play larger roles in the success of the control scheme. Detecting and refining patterns in muscle activations robust to the high variance and transient changes associated with surface electromyography is essential for efficient, user-friendly control. This article reviews the role of muscle synergies in myoelectric control schemes by dissecting each component of the scheme with respect to associated challenges for achieving robust simultaneous control of myoelectric interfaces. Electromyography recording details, signal feature extraction, pattern recognition and motor learning based control schemes are considered, and future directions are proposed as steps toward fulfilling the potential of myoelectric control in clinically and commercially viable applications.

Synergy-Driven Myoelectric Control for EMG-Based Prosthetic Manipulation: A Case Study

The paper proposes a synergy-based myoelectric control strategy for prosthetic hands. Synergy is first reviewed in the context of hand movement, then postural synergy-based proportional and simultaneous control has been introduced to prosthetic manipulation via the principal component analysis (PCA) framework. Experiments have been comprehensively carried out on lab-developed prosthetic hand called SJU-5 to evaluate the proposed method. It is evident that the synergy driven myoelectric control achieves the targeted objectives and performs well on the SJU-5 prosthetic hand.

Support vector regression for improved real-time, simultaneous myoelectric control

This study describes the first application of a support vector machine (SVM) based scheme for real-time simultaneous and proportional myoelectric control of multiple degrees of freedom (DOFs). Three DOFs including wrist flexion-extension, abduction-adduction and forearm pronation-supination were investigated with 10 able-bodied subjects and two individuals with transradial limb deficiency (LD). A Fitts' law test involving real-time target acquisition tasks was conducted to compare the usability of the SVM-based control system to that of an artificial neural network (ANN) based method. Performance was assessed using the Fitts' law throughput value as well as additional metrics including completion rate, path efficiency and overshoot. The SVM-based approach outperformed the ANN-based system in every performance measure for able-bodied subjects. The SVM outperformed the ANN in path efficiency and throughput with the first LD subject and in throughput with the second LD subject. The superior performance of the SVM-based system appears to be due to its higher estimation accuracy of all DOFs during inactive and low amplitude segments (these periods were frequent during real-time control). Another advantage of the SVM-based method was that it substantially reduced the processing time for both training and real time control.

Preliminary study on proportional and simultaneous estimation of hand posture using surface EMG based on synergy concept

Most of current myoelectric prostheses are using sequential and on-off control strategy within pattern classification framework, which is of robustness. But it is not a natural neuromuscular control scheme. On the other hand, there are two difficulties to control the prosthesis proportionally and simultaneously. First, human hand is high dimensional with more than 20 degrees-of-freedom (DOFs); Second, extracting such control information from EMG is hard due to signal crosstalk and noises. This paper is aimed at proposing a new method for proportional and simultaneous myoelectric control, taking advantage of synergy concept. The hand motion and corresponding forearm EMG signals were collected simultaneously. Principal component analysis (PCA) is used to reduce hand motion dimension. And support vector regression (SVR) is adopted to build the connection between hand posture and EMG. Offline analysis validated the effectiveness of this method, and preliminary and positive results have been obtained.

Intuitive, online, simultaneous, and proportional myoelectric control over two degrees-of-freedom in upper limb amputees

We propose an approach for online simultaneous and proportional myoelectric control of two degrees-of-freedom (DoF) of the wrist, using surface electromyographic signals. The method is based on the nonnegative matrix factorization (NMF) of the wrist muscle activation to extract low-dimensional control signals translated by the user into kinematic variables. This procedure does not need a training set of signals for which the kinematics is known (labeled dataset) and is thus unsupervised (although it requires an initial calibration without labeled signals). The estimated control signals using NMF are used to directly control two DoFs of wrist. The method was tested on seven subjects with upper limb deficiency and on seven able-bodied subjects. The subjects performed online control of a virtual object with two DoFs to achieve goal-oriented tasks. The performance of the two subject groups, measured as the task completion rate, task completion time, and execution efficiency, was not statistically different. The approach was compared, and demonstrated to be superior to the online control by the industrial state-of-the-art approach. These results show that this new approach, which has several advantages over the previous myoelectric prosthetic control systems, has the potential of providing intuitive and dexterous control of artificial limbs for amputees.

Effect of vibrotactile feedback on an EMG-based proportional cursor control system

Surface electromyography (sEMG) has been introduced into the bio-mechatronics systems, however, most of them are lack of the sensory feedback. In this paper, the effect of vibrotactile feedback for a myoelectric cursor control system is investigated quantitatively. Simultaneous and proportional control signals are extracted from EMG using a muscle synergy model. Different types of feedback including vibrotactile feedback and visual feedback are added, assessed and compared with each other. The results show that vibrotactile feedback is capable of improving the performance of EMG-based human machine interface.