Prediction of finger kinematics from discharge timings of motor units: implications for intuitive control of myoelectric prostheses

Decomposition strategy for surface EMG with few channels: a simulation study

Objective.In the specific use of electromyogram (EMG) driven prosthetics, the user's disability reduces the space available for the electrode array. We propose a framework for EMG decomposition adapted to the condition of a few channels (less than 30 observations), which can elevate the potential of prosthetics in terms of cost and applicability.Approach.The new framework contains a peel-off approach, a refining strategy for motor unit (MU) spike train and MU action potential and a re-subtracting strategy to adapt the framework to few channels environments. Simulated EMG signals were generated to test the framework. In addition, we quantify and analyze the effect of strategies used in the framework.Main results.The results show that the new algorithm has an average improvement of 19.97% in the number of MUs identified compared to the control algorithm. Quantitative analysis of the usage strategies shows that the re-subtracting and refining strategies can effectively improve the performance of the framework under the condition of few channels.Significance.These prove that the new framework can be applied to few channel conditions, providing a optimization space for neural interface design in cost and user adaptation.

Accurate Identification of Motoneuron Discharges From Ultrasound Images Across the Full Muscle Cross-Section

OBJECTIVE

Non-invasive identification of motoneuron (MN) activity commonly uses electromyography (EMG). However, surface EMG (sEMG) detects only superficial sources, at less than approximately 10-mm depth. Intramuscular EMG can detect deep sources, but it is limited to sources within a few mm of the detection site. Conversely, ultrasound (US) images have high spatial resolution across the whole muscle cross-section. The activity of MNs can be extracted from US images due to the movements that MN activation generates in the innervated muscle fibers. Current US-based decomposition methods can accurately identify the location and average twitch induced by MN activity. However, they cannot accurately detect MN discharge times.

METHODS

Here, we present a method based on the convolutive blind source separation of US images to estimate MN discharge times with high accuracy. The method was validated across Ten participants using concomitant sEMG decomposition as the ground truth.

RESULTS

140 unique MN spike trains were identified from US images, with a rate of agreement (RoA) with sEMG decomposition of 87.4 ± 10.3%. Over 50% of these MN spike trains had a RoA greater than 90%. Furthermore, with US, we identified additional MUs well beyond the sEMG detection volume, at up to >30 mm below the skin.

CONCLUSION

The proposed method can identify discharges of MNs innervating muscle fibers in a large range of depths within the muscle from US images.

SIGNIFICANCE

The proposed methodology can non-invasively interface with the outer layers of the central nervous system innervating muscles across the full cross-section.

Does Exerting Grasps Involve a Finite Set of Muscle Patterns? A Study of Intra- and Intersubject Variability of Forearm sEMG Signals in Seven Grasp Types

Surface Electromyography (sEMG) signals are widely used as input to control robotic devices, prosthetic limbs, exoskeletons, among other devices, and provide information about someone's intention to perform a particular movement. However, the redundant action of 32 muscles in the forearm and hand means that the neuromotor system can select different combinations of muscular activities to perform the same grasp, and these combinations could differ among subjects, and even among the trials done by the same subject. In this work, 22 healthy subjects performed seven representative grasp types (the most commonly used). sEMG signals were recorded from seven representative forearm spots identified in a previous work. Intra- and intersubject variability are presented by using four sEMG characteristics: muscle activity, zero crossing, enhanced wavelength and enhanced mean absolute value. The results confirmed the presence of both intra- and intersubject variability, which evidences the existence of distinct, yet limited, muscle patterns while executing the same grasp. This work underscores the importance of utilizing diverse combinations of sEMG features or characteristics of various natures, such as time-domain or frequency-domain, and it is the first work to observe the effect of considering different muscular patterns during grasps execution. This approach is applicable for fine-tuning the control settings of current sEMG devices.

Stretchable, Flexible, Breathable, Self-Adhesive Epidermal Hand sEMG Sensor System

Hand function rehabilitation training typically requires monitoring the activation status of muscles directly related to hand function. However, due to factors such as the small surface area for hand-back electrode placement and significant skin deformation, the continuous real-time monitoring of high-quality surface electromyographic (sEMG) signals on the hand-back skin still poses significant challenges. We report a stretchable, flexible, breathable, and self-adhesive epidermal sEMG sensor system. The optimized serpentine structure exhibits a sufficient stretchability and filling ratio, enabling the high-quality monitoring of signals. The carving design minimizes the distribution of connecting wires, providing more space for electrode reservation. The low-cost fabrication design, combined with the cauterization design, facilitates large-scale production. Integrated with customized wireless data acquisition hardware, it demonstrates the real-time multi-channel sEMG monitoring capability for muscle activation during hand function rehabilitation actions. The sensor provides a new tool for monitoring hand function rehabilitation treatments, assessing rehabilitation outcomes, and researching areas such as prosthetic control.

Single-Channel sEMG-Based Estimation of Knee Joint Angle Using a Decomposition Algorithm With a State-Space Model

Accurate human motion estimation is crucial for effective and safe human-robot interaction when using robotic devices for rehabilitation or performance enhancement. Although surface electromyography (sEMG) signals have been widely used to estimate human movements, conventional sEMG-based methods, which need sEMG signals measured from multiple relevant muscles, are usually subject to some limitations, including interference between sEMG sensors and wearable robots/environment, complicated calibration, as well as discomfort during long-term routine use. Few methods have been proposed to deal with these limitations by using single-channel sEMG (i.e., reducing the sEMG sensors as much as possible). The main challenge for developing single-channel sEMG-based estimation methods is that high estimation accuracy is difficult to be guaranteed. To address this problem, we proposed an sEMG-driven state-space model combined with an sEMG decomposition algorithm to improve the accuracy of knee joint movement estimation based on single-channel sEMG signals measured from gastrocnemius. The effectiveness of the method was evaluated via both single- and multi-speed walking experiments with seven and four healthy subjects, respectively. The results showed that the normal root-mean-squared error of the estimated knee joint angle using the method could be limited to 15%. Moreover, this method is robust with respect to variations in walking speeds. The estimation performance of this method was basically comparable to that of state-of-the-art studies using multi-channel sEMG.

Sensing in Soft Robotics

Soft robotics is an exciting field of science and technology that enables robots to manipulate objects with human-like dexterity. Soft robots can handle delicate objects with care, access remote areas, and offer realistic feedback on their handling performance. However, increased dexterity and mechanical compliance of soft robots come with the need for accurate control of the position and shape of these robots. Therefore, soft robots must be equipped with sensors for better perception of their surroundings, location, force, temperature, shape, and other stimuli for effective usage. This review highlights recent progress in sensing feedback technologies for soft robotic applications. It begins with an introduction to actuation technologies and material selection in soft robotics, followed by an in-depth exploration of various types of sensors, their integration methods, and the benefits of multimodal sensing, signal processing, and control strategies. A short description of current market leaders in soft robotics is also included in the review to illustrate the growing demands of this technology. By examining the latest advancements in sensing feedback technologies for soft robots, this review aims to highlight the potential of soft robotics and inspire innovation in the field.

Soft Robotics Enables Neuroprosthetic Hand Design

Development and implementation of neuroprosthetic hands is a multidisciplinary field at the interface between humans and artificial robotic systems, which aims at replacing the sensorimotor function of the upper-limb amputees as their own. Although prosthetic hand devices with myoelectric control can be dated back to more than 70 years ago, their applications with anthropomorphic robotic mechanisms and sensory feedback functions are still at a relatively preliminary and laboratory stage. Nevertheless, a recent series of proof-of-concept studies suggest that soft robotics technology may be promising and useful in alleviating the design complexity of the dexterous mechanism and integration difficulty of multifunctional artificial skins, in particular, in the context of personalized applications. Here, we review the evolution of neuroprosthetic hands with the emerging and cutting-edge soft robotics, covering the soft and anthropomorphic prosthetic hand design and relating bidirectional neural interactions with myoelectric control and sensory feedback. We further discuss future opportunities on revolutionized mechanisms, high-performance soft sensors, and compliant neural-interaction interfaces for the next generation of neuroprosthetic hands.

Toward higher-performance bionic limbs for wider clinical use

Most prosthetic limbs can autonomously move with dexterity, yet they are not perceived by the user as belonging to their own body. Robotic limbs can convey information about the environment with higher precision than biological limbs, but their actual performance is substantially limited by current technologies for the interfacing of the robotic devices with the body and for transferring motor and sensory information bidirectionally between the prosthesis and the user. In this Perspective, we argue that direct skeletal attachment of bionic devices via osseointegration, the amplification of neural signals by targeted muscle innervation, improved prosthesis control via implanted muscle sensors and advanced algorithms, and the provision of sensory feedback by means of electrodes implanted in peripheral nerves, should all be leveraged towards the creation of a new generation of high-performance bionic limbs. These technologies have been clinically tested in humans, and alongside mechanical redesigns and adequate rehabilitation training should facilitate the wider clinical use of bionic limbs.

Toward a generalizable deep CNN for neural drive estimation across muscles and participants

Objective.High-density electromyography (HD-EMG) decomposition algorithms are used to identify individual motor unit (MU) spike trains, which collectively constitute the neural code of movements, to predict motor intent. This approach has advanced from offline to online decomposition, from isometric to dynamic contractions, leading to a wide range of neural-machine interface applications. However, current online methods need offline retraining when applied to the same muscle on a different day or to a different person, which limits their applications in a real-time neural-machine interface. We proposed a deep convolutional neural network (CNN) framework for neural drive estimation, which takes in frames of HD-EMG signals as input, extracts general spatiotemporal properties of MU action potentials, and outputs the number of spikes in each frame. The deep CNN can generalize its application without retraining to HD-EMG data recorded in separate sessions, muscles, or participants.Approach.We recorded HD-EMG signals from the vastus medialis and vastus lateralis muscles from five participants while they performed isometric contractions during two sessions separated by ∼20 months. We identified MU spike trains from HD-EMG signals using a convolutive blind source separation (BSS) method, and then used the cumulative spike train (CST) of these MUs and the HD-EMG signals to train and validate the deep CNN.Main results.On average, the correlation coefficients between CST from the BSS and that from deep CNN were0.983±0.006for leave-one-out across-sessions-and-muscles validation and0.989±0.002for leave-one-out across-participants validation. When trained with more than four datasets, the performance of deep CNN saturated at0.984±0.001for cross validations across muscles, sessions, and participants.Significance.We can conclude that the deep CNN is generalizable across the aforementioned conditions without retraining. We could potentially generate a robust deep CNN to estimate neural drive to muscles for neural-machine interfaces.

Mapping Individual Motor Unit Activity to Continuous Three-DoF Wrist Torques: Perspectives for Myoelectric Control

The surface electromyography (EMG) decomposition techniques provide access to motor neuron activities and have been applied to myoelectric control schemes. However, the current decomposition-based myoelectric control mainly focuses on discrete gestures or single-DoF continuous movements. In this study, we aimed to map the motor unit discharges, which were identified from high-density surface EMG, to the three degrees of freedom (DoFs) wrist movements. The 3-DoF wrist torques and high-density surface EMG signals were recorded concurrently from eight non-disabled subjects. The experimental protocol included single-DoF movements and their various combinations. We decoded the motor unit discharges from the EMG signals using a segment-wise decomposition algorithm. Then the neural features were extracted from motor unit discharges and projected to wrist torques with a multiple linear regression model. We compared the performance of two neural features (twitch model and spike counting) and two training schemes (single-DoF and multi-DoF training). On average, 145 ± 33 motor units were identified from each subject, with a pulse-to-noise ratio of 30.8 ± 4.2 dB. Both neural features exhibited high estimation accuracy of 3-DoF wrist torques, with an average [Formula: see text] of 0.76 ± 0.12 and normalized root mean square error of 11.4 ± 3.1%. These results demonstrated the efficiency of the proposed method in continuous estimation of 3-DoF wrist torques, which has the potential to advance dexterous myoelectric control based on neural information.

Electrotactile feedback improves grip force control and enables object stiffness recognition while using a myoelectric hand

Current myoelectric hands are limited in their ability to provide effective sensory feedback to the users, which highly affects their functionality and utility. Although the sensory information of a myoelectric hand can be acquired with equipped sensors, transforming the sensory signals into effective tactile sensations on users for functional tasks is a largely unsolved challenge. The purpose of this study aims to demonstrate that electrotactile feedback of the grip force improves the sensorimotor control of a myoelectric hand and enables object stiffness recognition. The grip force of a sensorized myoelectric hand was delivered to its users via electrotactile stimulation based on four kinds of typical encoding strategies, including graded (G), linear amplitude (LA), linear frequency (LF), and biomimetic (B) modulation. Object stiffness was encoded with the change of electrotactile sensations triggered by final grip force, as the prosthesis grasped the objects. Ten able-bodied subjects and two transradial amputees were recruited to participate in a dual-task virtual eggs test (VET) and an object stiffness discrimination test (OSDT) to quantify the prosthesis users' ability to handle fragile objects and recognize object stiffnesses, respectively. The quantified results showed that with electrotactile feedback enabled, the four kinds of encoding strategies allowed subjects to better able to handle fragile objects with similar performance, and the subjects were able to differentiate four levels of object stiffness at favorable accuracies (>86%) and high manual efficiency. Strategy LA presented the best stiffness discrimination performance, while strategy B was able to reduce the discrimination time but the discrimination accuracy was not better than the other three strategies. Electrotactile feedback also enhanced prosthesis embodiment and improved the users' confidence in prosthetic control. Outcomes indicate electrotactile feedback can be effectively exploited by the prosthesis users for grip force control and object stiffness recognition, proving the feasibility of functional sensory restoration of myoelectric prostheses equipped with electrotactile feedback.

Far-field electric potentials provide access to the output from the spinal cord from wrist-mounted sensors

OBJECTIVE

Neural interfaces need to become more unobtrusive and socially acceptable to appeal to general consumers outside rehabilitation settings.

APPROACH

We developed a non-invasive neural interface that provides access to spinal motor neuron activities from the wrist, which is the preferred location for a wearable. The interface decodes far-field potentials present at the tendon endings of the forearm muscles using blind source separation. First, we evaluated the reliability of the interface to detect motor neuron firings based on far-field potentials, and thereafter we used the decoded motor neuron activity for the prediction of finger contractions in offline and real-time conditions.

MAIN RESULTS

The results showed that motor neuron activity decoded from the far-field potentials at the wrist accurately predicted individual and combined finger commands and therefore allowed for highly accurate real-time task classification.

SIGNIFICANCE

These findings demonstrate the feasibility of a non-invasive, neural interface at the wrist for precise real-time control based on the output of the spinal cord.

Frequency-Domain sEMG Classification Using a Single Sensor

Working towards the development of robust motion recognition systems for assistive technology control, the widespread approach has been to use a plethora of, often times, multi-modal sensors. In this paper, we develop single-sensor motion recognition systems. Utilising the peripheral nature of surface electromyography (sEMG) data acquisition, we optimise the information extracted from sEMG sensors. This allows the reduction in sEMG sensors or provision of contingencies in a system with redundancies. In particular, we process the sEMG readings captured at the trapezius descendens and platysma muscles. We demonstrate that sEMG readings captured at one muscle contain distinct information on movements or contractions of other agonists. We used the trapezius and platysma muscle sEMG data captured in able-bodied participants and participants with tetraplegia to classify shoulder movements and platysma contractions using white-box supervised learning algorithms. Using the trapezius sensor, shoulder raise is classified with an accuracy of 99%. Implementing subject-specific multi-class classification, shoulder raise, shoulder forward and shoulder backward are classified with a 94% accuracy amongst object raise and shoulder raise-and-hold data in able bodied adults. A three-way classification of the platysma sensor data captured with participants with tetraplegia achieves a 95% accuracy on platysma contraction and shoulder raise detection.

Non-invasive analysis of motor unit activation during simultaneous and continuous wrist movements

Surface electromyography (EMG) signals have shown promising applications in human-machine interfacing (HMI) systems such as orthotics, prosthetics, and exoskeletons. Nevertheless, existing myoelectric control methods, generally based on time-domain or frequency-domain features, could not directly interpret neural commands. EMG decomposition techniques have become a prevailing solution to decode the motor neuron discharges from the spinal cord, whereas only single degree-of-freedom (DoF) movements are primarily involved in the current neural-based interfaces, resulting in limited intuitiveness and functionality. Here, we propose a non-invasive framework to analyze motor unit activities and estimate wrist torques during simultaneous contractions of multiple DoFs. Motor unit discharges were decoded from surface EMG signals and pooled into groups during sequential wrist movements. Then three neural features were extracted and linearly projected to the torques of multi-DoF tasks. On average, there were 4413 motor units identified for each motion with a PNR value of 25.82.9 dB. The neural features outperformed the classic EMG feature on the estimation accuracy with higher correlation coefficients and smoothness. These results demonstrate the feasibility and superiority of the proposed framework in kinetics estimation of simultaneous movements, extending the potential applications of surface EMG decomposition in human-machine interfaces.

Decoding Muscle Force From Motor Unit Firings Using Encoder-Decoder Networks

Appropriate interpretation of motor unit (MU) activities after surface EMG (sEMG) decomposition is a key factor to decode motor intentions in a noninvasive and physiologically meaningful way. However, there are great challenges due to the difficulty in cross-trial MU tracking and unavoidable loss of partial MU information resulting from incomplete decomposition. In light of these issues, this study presents a novel framework for interpreting MU activities and applies it to decode muscle force. The resulting MUs were clustered and classified into different categories by characterizing their spatially distributed firing waveforms. The process served as a general MU tracking method. On this basis, after transferring the MU firing trains to twitch force trains by a twitch force model, a deep network was designed to predict the normalized force. In addition, MU category distribution was examined to calibrate the actual force level, while functions of some unavailable MUs were compensated. To investigate the effectiveness of this framework, high-density sEMG signals were recorded using an 8×8 electrode array from the abductor pollicis brevis muscles of eight subjects, while thumb abduction force was measured. The proposed method outperformed three common methods ( ) yielding the lowest root mean square deviation of 6.68% ± 1.29% and the highest fitness ( R² ) of 0.94 ± 0.04 between the predicted force and the actual force. This study offers a valuable, computational solution for interpreting individual MU activities, and its effectiveness was confirmed in muscle force estimation.

Control of Newly-Designed Wearable Robotic Hand Exoskeleton Based on Surface Electromyographic Signals

The human hand plays a role in a variety of daily activities. This intricate instrument is vulnerable to trauma or neuromuscular disorders. Wearable robotic exoskeletons are an advanced technology with the potential to remarkably promote the recovery of hand function. However, the still face persistent challenges in mechanical and functional integration, with real-time control of the multiactuators in accordance with the motion intentions of the user being a particular sticking point. In this study, we demonstrated a newly-designed wearable robotic hand exoskeleton with multijoints, more degrees of freedom (DOFs), and a larger range of motion (ROM). The exoskeleton hand comprises six linear actuators (two for the thumb and the other four for the fingers) and can realize both independent movements of each digit and coordinative movement involving multiple fingers for grasp and pinch. The kinematic parameters of the hand exoskeleton were analyzed by a motion capture system. The exoskeleton showed higher ROM of the proximal interphalangeal and distal interphalangeal joints compared with the other exoskeletons. Five classifiers including support vector machine (SVM), K-near neighbor (KNN), decision tree (DT), multilayer perceptron (MLP), and multichannel convolutional neural networks (multichannel CNN) were compared for the offline classification. The SVM and KNN had a higher accuracy than the others, reaching up to 99%. For the online classification, three out of the five subjects showed an accuracy of about 80%, and one subject showed an accuracy over 90%. These results suggest that the new wearable exoskeleton could facilitate hand rehabilitation for a larger ROM and higher dexterity and could be controlled according to the motion intention of the subjects.

Wrist Torque Estimation via Electromyographic Motor Unit Decomposition and Image Reconstruction

Neural interface using decomposed motor units (MUs) from surface electromyography (sEMG) has allowed non-invasive access to the neural control signals, and provided a novel approach for intuitive human-machine interaction. However, most of the existing methods based on decomposed MUs merely adopted the discharge rate (DR) as the feature representations, which may lack local information around the discharge instant and ignore the subtle interactions of different MUs. In this study, we proposed an MU-specific image-based scheme for wrist torque estimation. Specifically, the high-density sEMG signals were decoded into motor unit spike trains (MUSTs), and then MU-specific images were reconstructed with MUSTs and corresponding motor unit action potential (MUAP). A convolutional neural network was used to learn representative features from MU-specific images automatically, and further to estimate wrist torques. The results demonstrated that the proposed method outperformed three conventional and a deep-learning regression approaches using DR features, with the estimation accuracy R² of 0.82 ± 0.09, 0.89 ± 0.06, and nRMSE of 12.6 ± 2.5%, 11.0 ± 3.1% for pronation/supination and flexion/extension, respectively. Further, the analysis of the extracted features from MU-specific images showed a higher correlation than DR for recorded torques, indicating the effectiveness of the proposed method. The outcomes of this study provide a novel and promising perspective for the intuitive control of neural interfacing.

Emerging Wearable Interfaces and Algorithms for Hand Gesture Recognition: A Survey

Hands are vital in a wide range of fundamental daily activities, and neurological diseases that impede hand function can significantly affect quality of life. Wearable hand gesture interfaces hold promise to restore and assist hand function and to enhance human-human and human-computer communication. The purpose of this review is to synthesize current novel sensing interfaces and algorithms for hand gesture recognition, and the scope of applications covers rehabilitation, prosthesis control, sign language recognition, and human-computer interaction. Results showed that electrical, dynamic, acoustical/vibratory, and optical sensing were the primary input modalities in gesture recognition interfaces. Two categories of algorithms were identified: 1) classification algorithms for predefined, fixed hand poses and 2) regression algorithms for continuous finger and wrist joint angles. Conventional machine learning algorithms, including linear discriminant analysis, support vector machines, random forests, and non-negative matrix factorization, have been widely used for a variety of gesture recognition applications, and deep learning algorithms have more recently been applied to further facilitate the complex relationship between sensor signals and multi-articulated hand postures. Future research should focus on increasing recognition accuracy with larger hand gesture datasets, improving reliability and robustness for daily use outside of the laboratory, and developing softer, less obtrusive interfaces.

In-silico development and assessment of a Kalman filter motor decoder for prosthetic hand control

Up to 50% of amputees abandon their prostheses, partly due to rapid degradation of the control systems, which require frequent recalibration. The goal of this study was to develop a Kalman filter-based approach to decoding motoneuron activity to identify movement kinematics and thereby provide stable, long-term, accurate, real-time decoding. The Kalman filter-based decoder was examined via biologically varied datasets generated from a high-fidelity computational model of the spinal motoneuron pool. The estimated movement kinematics controlled a simulated MuJoCo prosthetic hand. This clear-box approach showed successful estimation of hand movements under eight varied physiological conditions with no retraining. The mean correlation coefficient of 0.98 and mean normalized root mean square error of 0.06 over these eight datasets provide proof of concept that this decoder would improve long-term integrity of performance while performing new, untrained movements. Additionally, the decoder operated in real-time (~0.3 ms). Further results include robust performance of the Kalman filter when re-trained to more severe post-amputation limitations in the type and number of motoneurons remaining. An additional analysis shows that the decoder achieves better accuracy when using the firing of individual motoneurons as input, compared to using aggregate pool firing. Moreover, the decoder demonstrated robustness to noise affecting both the trained decoder parameters and the decoded motoneuron activity. These results demonstrate the utility of a proof of concept Kalman filter decoder that can support prosthetics' control systems to maintain accurate and stable real-time movement performance.

Simultaneous and proportional control of wrist and hand movements by decoding motor unit discharges in real time

Objective.Surface electromyography (EMG) decomposition techniques can be used to establish human-machine interfacing (HMI), but most investigations are implemented offline due to the computational load of the approach. Here, we generalize the offline decomposition algorithm to identify the motor unit (MU) activities in real time, and we propose a MU-based approach for online simultaneous and proportional control of multiple motor tasks.Approach.High-density surface EMG signals recorded from forearm muscles were decomposed into motor unit spike trains (MUST) with the proposed decomposition method. The MUSTs were first pooled into clusters in the calibration phase and the cumulative discharges of active MUs in each group were extracted as the control signal for each motor task. Then the subjects were instructed to control a virtual cursor with multiple motor tasks involving grasp and wrist movements. Fifteen able-bodied subjects and two patients with limb deficiency participated in the experiments to validate the proposed control scheme.Main results.On average, over 20 MUSTs were identified in real time with an estimated decomposition accuracy > 85%. The cumulative discharge in each pool was highly correlated with the activation of the specific motion (R=0.93±0.05). Moreover, the proposed MU-based method had superior performance in online tests than conventional myo-control methods based on global EMG features.Significance.These results indicate the feasibility of real-time neural decoding in a non-invasive way. Moreover, the superior performance in online tests proves the potential of the MU-based approach for the simultaneous and proportional control, promoting the application of EMG decomposition for HMI systems.

Learning regularized representations of categorically labelled surface EMG enables simultaneous and proportional myoelectric control

Background

Processing the surface electromyogram (sEMG) to decode movement intent is a promising approach for natural control of upper extremity prostheses. To this end, this paper introduces and evaluates a new framework which allows for simultaneous and proportional myoelectric control over multiple degrees of freedom (DoFs) in real-time. The framework uses multitask neural networks and domain-informed regularization in order to automatically find nonlinear mappings from the forearm sEMG envelope to multivariate and continuous encodings of concurrent hand- and wrist kinematics, despite only requiring categorical movement instruction stimuli signals for calibration.

Methods

Forearm sEMG with 8 channels was collected from healthy human subjects (N = 20) and used to calibrate two myoelectric control interfaces, each with two output DoFs. The interfaces were built from (I) the proposed framework, termed Myoelectric Representation Learning (MRL), and, to allow for comparisons, from (II) a standard pattern recognition framework based on Linear Discriminant Analysis (LDA). The online performances of both interfaces were assessed with a Fitts’s law type test generating 5 quantitative performance metrics. The temporal stabilities of the interfaces were evaluated by conducting identical tests without recalibration 7 days after the initial experiment session.

Results

Metric-wise two-way repeated measures ANOVA with factors method (MRL vs LDA) and session (day 1 vs day 7) revealed a significant (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p<0.05$$\end{document}p<0.05) advantage for MRL over LDA in 5 out of 5 performance metrics, with metric-wise effect sizes (Cohen’s \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d$$\end{document}d) separating MRL from LDA ranging from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left|d\right|=0.62$$\end{document}d=0.62 to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left|d\right|=1.13$$\end{document}d=1.13. No significant effect on any metric was detected for neither session nor interaction between method and session, indicating that none of the methods deteriorated significantly in control efficacy during one week of intermission.

Conclusions

The results suggest that MRL is able to successfully generate stable mappings from EMG to kinematics, thereby enabling myoelectric control with real-time performance superior to that of the current commercial standard for pattern recognition (as represented by LDA). It is thus postulated that the presented MRL approach can be of practical utility for muscle-computer interfaces.

Analysis of motor unit activities during multiple motor tasks by real-time EMG decomposition: perspective for myoelectric control

Surface electromyography (EMG) decomposition techniques have been applied for human-machine interfacing by decoding neural information, while most of decomposition approaches work offline. Here, we apply an online decomposition scheme to decode motor unit activities during three motor tasks, and measure the recognition accuracy of motor type and activation level using the decomposition results. High-density surface EMG signal were recorded from forearm muscles of six able-bodied subjects. The EMG signals were decomposed into motor unit spike trains (MUST) with a sliding window of 100 ms. The computation complexity had time consumption < 50 ms in each window. Most identified motor units discharged during only one motor task. On average, over 5 MUSTs were identified for each motion and the recognition accuracy based on motor unit activities was > 99%. The discharge rate of motor units was highly correlated with the activation level of each motion with an average correlation coefficient of 0.94 ± 0.04. These results indicate the feasibility of an online, multi-motion, and proportional control scheme based on neural decoding in a non-invasive way.

Single channel surface electromyogram deconvolution to explore motor unit discharges

Interference surface electromyogram (EMG) reflects many bioelectric properties of active motor units (MU), which are however difficult to estimate due to the asynchronous summation of their discharges. This paper introduces a deconvolution technique to estimate the cumulative firings of MUs. Tests in simulations show that the power spectral density of the estimated MU firings has a low-frequency peak corresponding to the mean firing rate of MUs in the detection volume of the recording system, weighted by the amplitudes of MU action potentials. The peak increases in amplitude and its centroid shifts to a higher frequency when MU synchronization is simulated (mainly due to the shift of discharges of large MUs). The peak is found even at high force levels, when such a contribution does not emerge from the EMG. This result is also confirmed in preliminary applications to experimental data. Moreover, the simulated cumulative firings of MUs are estimated with a correlation above 90% (considering frequency contributions up to 150 Hz), for all force levels. The method requires a single EMG channel, thus being feasible even in applied studies using simple recording systems. It may open many potential applications, e.g., in the study of the modulation of MU firing rate induced by either fatigue or pathology and in coherency analysis. Graphical Abstract Examples of application of the deconvolution (Deconv) algorithm and comparison with the cumulative firings and the cumulated weighted firings (CWF, i.e., each firing pattern is weighted by the root mean squared amplitude of the corresponding MU action potential). Portions of data are shown on the left, the power spectral densities (PSD) on the right (Welch method applied to 3 s of data, sub-epochs of 0.5 s, mean value removed from each of them, 50% of overlap). A) Simulated signal (50% of maximal voluntary contraction, MVC) with random MU firings. B) Simulated signal (50% MVC) with a level of synchronization equal to 10%. C) Experimental data from vastus medialis at 40% MVC (data decomposed by the algorithm of Holobar and Zazula, IEEE Trans. Sig. Proc. 2007; PSD of the cumulated firings almost identical to that of CWF, as few MUs were identified).