Hand gesture recognition based on motor unit spike trains decoded from high-density electromyography

Accurate Identification of Motoneuron Discharges From Ultrasound Images Across the Full Muscle Cross-Section

OBJECTIVE

Non-invasive identification of motoneuron (MN) activity commonly uses electromyography (EMG). However, surface EMG (sEMG) detects only superficial sources, at less than approximately 10-mm depth. Intramuscular EMG can detect deep sources, but it is limited to sources within a few mm of the detection site. Conversely, ultrasound (US) images have high spatial resolution across the whole muscle cross-section. The activity of MNs can be extracted from US images due to the movements that MN activation generates in the innervated muscle fibers. Current US-based decomposition methods can accurately identify the location and average twitch induced by MN activity. However, they cannot accurately detect MN discharge times.

METHODS

Here, we present a method based on the convolutive blind source separation of US images to estimate MN discharge times with high accuracy. The method was validated across Ten participants using concomitant sEMG decomposition as the ground truth.

RESULTS

140 unique MN spike trains were identified from US images, with a rate of agreement (RoA) with sEMG decomposition of 87.4 ± 10.3%. Over 50% of these MN spike trains had a RoA greater than 90%. Furthermore, with US, we identified additional MUs well beyond the sEMG detection volume, at up to >30 mm below the skin.

CONCLUSION

The proposed method can identify discharges of MNs innervating muscle fibers in a large range of depths within the muscle from US images.

SIGNIFICANCE

The proposed methodology can non-invasively interface with the outer layers of the central nervous system innervating muscles across the full cross-section.

Formation of Anisotropic Conducting Interlayer for High-Resolution Epidermal Electromyography Using Mixed-Conducting Particulate Composite

Epidermal electrophysiology is a non-invasive method used in research and clinical practices to study the electrical activity of the brain, heart, nerves, and muscles. However, electrode/tissue interlayer materials such as ionically conducting pastes can negatively affect recordings by introducing lateral electrode-to-electrode ionic crosstalk and reducing spatial resolution. To overcome this issue, biocompatible, anisotropic-conducting interlayer composites (ACI) that establish an electrically anisotropic interface with the skin are developed, enabling the application of dense cutaneous sensor arrays. High-density, conformable electrodes are also microfabricated that adhere to the ACI and follow the curvilinear surface of the skin. The results show that ACI significantly enhances the spatial resolution of epidermal electromyography (EMG) recording compared to conductive paste, permitting the acquisition of single muscle action potentials with distinct spatial profiles. The high-density EMG in developing mice, non-human primates, and humans is validated. Overall, high spatial-resolution epidermal electrophysiology enabled by ACI has the potential to advance clinical diagnostics of motor system disorders and enhance data quality for human-computer interface applications.

Optimal Motor Unit Subset Selection for Accurate Motor Intention Decoding: Towards Dexterous Real-Time Interfacing

OBJECTIVE

Motor unit (MU) discharge timings encode human motor intentions to the finest degree. Whilst tapping into such information can bring significant gains to a range of applications, current approaches to MU decoding from surface signals do not scale well with the demands of dexterous human-machine interfacing (HMI). To optimize the forward estimation accuracy and time-efficiency of such systems, we propose the inclusion of task-wise initialization and MU subset selection.

METHODS

Offline analyses were conducted on data recorded from 11 non-disabled subjects. Task-wise decomposition was applied to identify MUs from high-density surface electromyography (HD-sEMG) pertaining to 18 wrist/forearm motor tasks. The activities of a selected subset of MUs were extracted from test data and used for forward estimation of intended motor tasks and joint kinematics. To that end, various combinations of subset selection and estimation algorithms (both regression and classification-based) were tested for a range of subset sizes.

RESULTS

The mutual information-based minimum Redundancy Maximum Relevance (mRMR-MI) criterion retained MUs with the highest predicative power. When the portion of tracked MUs was reduced down to 25%, the regression performance decreased only by 3% (R2=0.79) while classification accuracy dropped by 2.7% (accuracy = 74%) when kernel-based estimators were considered.

CONCLUSION AND SIGNIFICANCE

Careful selection of tracked MUs can optimize the efficiency of MU-driven interfacing. In particular, prioritization of MUs exhibiting strong nonlinear relationships with target motions is best leveraged by kernel-based estimators. Hence, this frees resources for more robust and adaptive MU decoding techniques to be implemented in future.

Cross-user gesture recognition from sEMG signals using an optimal transport assisted student-teacher framework

The cross-user gesture recognition is a puzzle in the myoelectric control system, owing to great variability in muscle activities across different users. To address this problem, a novel optimal transport (OT) assisted student-teacher (ST) framework (termed OT-ST) was proposed in this paper to facilitate transfer across user domains in an unsupervised domain adaptation (UDA) manner. In this framework, the initial parameters of the ST models were trained with the labeled data from users in the source domain. In the model transfer stage for a new user in the target domain, the teacher model was utilized to generate pseudo labels for unlabeled testing samples, providing guidance to the adaptation of the student model. The OT algorithm was employed to optimize the pseudo labels generated from the teacher model, avoiding the model bias and further improving the effect of domain adaptation. The performance of the proposed OT-ST framework was evaluated via experiments of classifying seven hand gestures using high-density surface electromyogram (HD-sEMG) recordings from extensor digitorum muscles of eight intact-limbed subjects. The OT-ST framework yielded a high accuracy of 96.50 ± 2.88% for new users, and outperformed other common machine learning and UDA methods significantly (p < 0.01), demonstrating its effectiveness. The OT-ST framework does not require special repetitive training or any labeled data for calibration. In addition, it can incrementally learn from new testing samples and improve the recognition ability. This study provides a promising method for developing user-generic myoelectric pattern recognition, with wide applications in human-computer interaction, consumer electronics and prosthesis control.

Transformer-based hand gesture recognition from instantaneous to fused neural decomposition of high-density EMG signals

Designing efficient and labor-saving prosthetic hands requires powerful hand gesture recognition algorithms that can achieve high accuracy with limited complexity and latency. In this context, the paper proposes a Compact Transformer-based Hand Gesture Recognition framework referred to as [Formula: see text], which employs a vision transformer network to conduct hand gesture recognition using high-density surface EMG (HD-sEMG) signals. Taking advantage of the attention mechanism, which is incorporated into the transformer architectures, our proposed [Formula: see text] framework overcomes major constraints associated with most of the existing deep learning models such as model complexity; requiring feature engineering; inability to consider both temporal and spatial information of HD-sEMG signals, and requiring a large number of training samples. The attention mechanism in the proposed model identifies similarities among different data segments with a greater capacity for parallel computations and addresses the memory limitation problems while dealing with inputs of large sequence lengths. [Formula: see text] can be trained from scratch without any need for transfer learning and can simultaneously extract both temporal and spatial features of HD-sEMG data. Additionally, the [Formula: see text] framework can perform instantaneous recognition using sEMG image spatially composed from HD-sEMG signals. A variant of the [Formula: see text] is also designed to incorporate microscopic neural drive information in the form of Motor Unit Spike Trains (MUSTs) extracted from HD-sEMG signals using Blind Source Separation (BSS). This variant is combined with its baseline version via a hybrid architecture to evaluate potentials of fusing macroscopic and microscopic neural drive information. The utilized HD-sEMG dataset involves 128 electrodes that collect the signals related to 65 isometric hand gestures of 20 subjects. The proposed [Formula: see text] framework is applied to 31.25, 62.5, 125, 250 ms window sizes of the above-mentioned dataset utilizing 32, 64, 128 electrode channels. Our results are obtained via 5-fold cross-validation by first applying the proposed framework on the dataset of each subject separately and then, averaging the accuracies among all the subjects. The average accuracy over all the participants using 32 electrodes and a window size of 31.25 ms is 86.23%, which gradually increases till reaching 91.98% for 128 electrodes and a window size of 250 ms. The [Formula: see text] achieves accuracy of 89.13% for instantaneous recognition based on a single frame of HD-sEMG image. The proposed model is statistically compared with a 3D Convolutional Neural Network (CNN) and two different variants of Support Vector Machine (SVM) and Linear Discriminant Analysis (LDA) models. The accuracy results for each of the above-mentioned models are paired with their precision, recall, F1 score, required memory, and train/test times. The results corroborate effectiveness of the proposed [Formula: see text] framework compared to its counterparts.

Bio-robotics research for non-invasive myoelectric neural interfaces for upper-limb prosthetic control: a 10-year perspective review

A decade ago, a group of researchers from academia and industry identified a dichotomy between the industrial and academic state-of-the-art in upper-limb prosthesis control, a widely used bio-robotics application. They proposed that four key technical challenges, if addressed, could bridge this gap and translate academic research into clinically and commercially viable products. These challenges are unintuitive control schemes, lack of sensory feedback, poor robustness and single sensor modality. Here, we provide a perspective review on the research effort that occurred in the last decade, aiming at addressing these challenges. In addition, we discuss three research areas essential to the recent development in upper-limb prosthetic control research but were not envisioned in the review 10 years ago: deep learning methods, surface electromyogram decomposition and open-source databases. To conclude the review, we provide an outlook into the near future of the research and development in upper-limb prosthetic control and beyond.

Firing behavior of single motor units of the tibialis anterior in human walking as non-invasively revealed by HDsEMG decomposition

Objective.Investigation of the firing behavior of motor units (MUs) provides essential neuromuscular control information because MUs are the smallest organizational component of the neuromuscular system. The MUs activated during human infants' leg movements and rodent locomotion, mainly controlled by the spinal central pattern generator (CPG), show highly synchronous firing. In addition to spinal CPGs, the cerebral cortex is involved in neuromuscular control during walking in human adults. Based on the difference in the neural control mechanisms of locomotion between rodent, human infants and adults, MU firing behavior during adult walking probably has some different features from the other populations. However, so far, the firing activity of MUs in human adult walking has been largely unknown due to technical issues.Approach.Recent technical advances allow noninvasive investigation of MU firing by high-density surface electromyogram (HDsEMG) decomposition. We investigated the MU firing behavior of the tibialis anterior (TA) muscle during walking at a slow speed by HDsEMG decomposition.Main results.We found recruitment threshold modulation of MU between walking and steady isometric contractions. Doublet firings, and gait phase-specific firings were also observed during walking. We also found high MU synchronization during walking over a wide range of frequencies, probably including cortical and spinal CPG-related components. The amount of MU synchronization was modulated between the gait phases and motor tasks. These results suggest that the central nervous system flexibly controls MU firing to generate appropriate force of TA during human walking.Significance.This study revealed the MU behavior during walking at a slow speed and demonstrated the feasibility of noninvasive investigation of MUs during dynamic locomotor tasks, which will open new frontiers for the study of neuromuscular systems in the fields of neuroscience and biomedical engineering.

Estimation of the firing behaviour of a complete motoneuron pool by combining electromyography signal decomposition and realistic motoneuron modelling

Our understanding of the firing behaviour of motoneuron (MN) pools during human voluntary muscle contractions is currently limited to electrophysiological findings from animal experiments extrapolated to humans, mathematical models of MN pools not validated for human data, and experimental results obtained from decomposition of electromyographical (EMG) signals. These approaches are limited in accuracy or provide information on only small partitions of the MN population. Here, we propose a method based on the combination of high-density EMG (HDEMG) data and realistic modelling for predicting the behaviour of entire pools of motoneurons in humans. The method builds on a physiologically realistic model of a MN pool which predicts, from the experimental spike trains of a smaller number of individual MNs identified from decomposed HDEMG signals, the unknown recruitment and firing activity of the remaining unidentified MNs in the complete MN pool. The MN pool model is described as a cohort of single-compartment leaky fire-and-integrate (LIF) models of MNs scaled by a physiologically realistic distribution of MN electrophysiological properties and driven by a spinal synaptic input, both derived from decomposed HDEMG data. The MN spike trains and effective neural drive to muscle, predicted with this method, have been successfully validated experimentally. A representative application of the method in MN-driven neuromuscular modelling is also presented. The proposed approach provides a validated tool for neuroscientists, experimentalists, and modelers to infer the firing activity of MNs that cannot be observed experimentally, investigate the neuromechanics of human MN pools, support future experimental investigations, and advance neuromuscular modelling for investigating the neural strategies controlling human voluntary contractions.

Our experimental understanding of the firing behaviour of motoneuron (MN) pools during human voluntary muscle contractions is currently limited to the observation of small samples of active MNs obtained from EMG decomposition. EMG decomposition therefore provides an important but incomplete description of the role of individual MNs in the firing activity of the complete MN pool, which limits our understanding of the neural strategies of the whole MN pool and of how the firing activity of each MN contributes to the neural drive to muscle. Here, we combine decomposed high-density EMG (HDEMG) data and a physiologically realistic model of MN population to predict the unknown recruitment and firing activity of the remaining unidentified MNs in the complete MN pool. In brief, an experimental estimation of the synaptic current is input to a cohort of MN models, which are calibrated using the available decomposed HDEMG data, and predict the MN spike trains fired by the entire MN population. This novel approach is experimentally validated and applied to muscle force prediction from neuromuscular modelling.

Segment-Wise Decomposition of Surface Electromyography to Identify Discharges Across Motor Neuron Populations

OBJECTIVE

The surface electromyography (EMG) decomposition techniques have shown promising results in neurophysiologic investigations, clinical diagnosis, and human-machine interfacing. However, current decomposition methods could only decode a limited number of motor units (MUs) because of the local convergence. The number of identified MUs remains similar even though more muscles or movements are involved, where multiple motor neuron populations are activated. The objective of this study was to develop a segment-wise decomposition strategy to increase the number of MU decoded from multiple motor neuron populations.

METHODS

The EMG signals were divided into several segments depending on the number of involved movements. The motor neurons, activated during each movement, were regarded as a population. The convolution kernel compensation (CKC) method was applied individually for each segment to decode the motor unit discharges from each motor neuron population. The MU filters were obtained in each segment and filtrated to estimate the MU spike trains (MUSTs) from the global EMG signals. The decomposition performance was validated on synthetic and experimental EMG signals.

MAIN RESULTS

From synthetic EMG signals generated by two motor neuron populations, the proposed segment-wise CKC (swCKC) decoded significantly more MUs during low and medium excitation levels, with an increased rate of 16.3% to 75.4% compared with the conventional CKC. From experimental signals recorded during ten motor tasks, 133±24 MUs with the pulse-to-noise ratio of 36.6±6.5 dB were identified for each subject by swCKC, whereas the conventional CKC identified only 43±12 MUs.

CONCLUSION AND SIGNIFICANCE

These results indicate the feasibility and superiority of the proposed swCKC to decode MU activities across motor neuron populations, extending the potential applications of EMG decomposition for neural decoding during multiple motor tasks.

Actor-Critic Reinforcement Learning Based Algorithm for Contaminant Minimization in sEMG Movement Recognition

This paper aims to present an approach based on Reinforcement Learning (RL) concept to detect contaminants' type and minimize their effect on surface electromyography signal (sEMG) based movement recognition. The referred method was applied in the pre-processing stage of a sEMG based motion classification system using the Ninapro database 2 artificially contaminated with electrocardiography (ECG) interference, motion artifact (MOA), powerline interference (PLI) and additive white Gaussian noise (WGN). Support Vector Machine was the method for movement classification. The results showed an improvement of 8.9%, 16.7%, 15.9%, 16.5%, and 11.9% in the movement recognition accuracy with the application of the pre-processing algorithm to restore, respectively, one, three, six, nine, and 12 contaminated channels.

Far-field electric potentials provide access to the output from the spinal cord from wrist-mounted sensors

OBJECTIVE

Neural interfaces need to become more unobtrusive and socially acceptable to appeal to general consumers outside rehabilitation settings.

APPROACH

We developed a non-invasive neural interface that provides access to spinal motor neuron activities from the wrist, which is the preferred location for a wearable. The interface decodes far-field potentials present at the tendon endings of the forearm muscles using blind source separation. First, we evaluated the reliability of the interface to detect motor neuron firings based on far-field potentials, and thereafter we used the decoded motor neuron activity for the prediction of finger contractions in offline and real-time conditions.

MAIN RESULTS

The results showed that motor neuron activity decoded from the far-field potentials at the wrist accurately predicted individual and combined finger commands and therefore allowed for highly accurate real-time task classification.

SIGNIFICANCE

These findings demonstrate the feasibility of a non-invasive, neural interface at the wrist for precise real-time control based on the output of the spinal cord.

Enhanced Recognition of Amputated Wrist and Hand Movements by Deep Learning Method Using Multimodal Fusion of Electromyography and Electroencephalography

Motion classification can be performed using biometric signals recorded by electroencephalography (EEG) or electromyography (EMG) with noninvasive surface electrodes for the control of prosthetic arms. However, current single-modal EEG and EMG based motion classification techniques are limited owing to the complexity and noise of EEG signals, and the electrode placement bias, and low-resolution of EMG signals. We herein propose a novel system of two-dimensional (2D) input image feature multimodal fusion based on an EEG/EMG-signal transfer learning (TL) paradigm for detection of hand movements in transforearm amputees. A feature extraction method in the frequency domain of the EEG and EMG signals was adopted to establish a 2D image. The input images were used for training on a model based on the convolutional neural network algorithm and TL, which requires 2D images as input data. For the purpose of data acquisition, five transforearm amputees and nine healthy controls were recruited. Compared with the conventional single-modal EEG signal trained models, the proposed multimodal fusion method significantly improved classification accuracy in both the control and patient groups. When the two signals were combined and used in the pretrained model for EEG TL, the classification accuracy increased by 4.18-4.35% in the control group, and by 2.51-3.00% in the patient group.

A Way of Bionic Control Based on EI, EMG, and FMG Signals

Creating highly functional prosthetic, orthotic, and rehabilitation devices is a socially relevant scientific and engineering task. Currently, certain constraints hamper the development of such devices. The primary constraint is the lack of an intuitive and reliable control interface working between the organism and the actuator. The critical point in developing these devices and systems is determining the type and parameters of movements based on control signals recorded on an extremity. In the study, we investigate the simultaneous acquisition of electric impedance (EI), electromyography (EMG), and force myography (FMG) signals during basic wrist movements: grasping, flexion/extension, and rotation. For investigation, a laboratory instrumentation and software test setup were made for registering signals and collecting data. The analysis of the acquired signals revealed that the EI signals in conjunction with the analysis of EMG and FMG signals could potentially be highly informative in anthropomorphic control systems. The study results confirm that the comprehensive real-time analysis of EI, EMG, and FMG signals potentially allows implementing the method of anthropomorphic and proportional control with an acceptable delay.

Non-invasive analysis of motor unit activation during simultaneous and continuous wrist movements

Surface electromyography (EMG) signals have shown promising applications in human-machine interfacing (HMI) systems such as orthotics, prosthetics, and exoskeletons. Nevertheless, existing myoelectric control methods, generally based on time-domain or frequency-domain features, could not directly interpret neural commands. EMG decomposition techniques have become a prevailing solution to decode the motor neuron discharges from the spinal cord, whereas only single degree-of-freedom (DoF) movements are primarily involved in the current neural-based interfaces, resulting in limited intuitiveness and functionality. Here, we propose a non-invasive framework to analyze motor unit activities and estimate wrist torques during simultaneous contractions of multiple DoFs. Motor unit discharges were decoded from surface EMG signals and pooled into groups during sequential wrist movements. Then three neural features were extracted and linearly projected to the torques of multi-DoF tasks. On average, there were 4413 motor units identified for each motion with a PNR value of 25.82.9 dB. The neural features outperformed the classic EMG feature on the estimation accuracy with higher correlation coefficients and smoothness. These results demonstrate the feasibility and superiority of the proposed framework in kinetics estimation of simultaneous movements, extending the potential applications of surface EMG decomposition in human-machine interfaces.

Decoding Muscle Force From Motor Unit Firings Using Encoder-Decoder Networks

Appropriate interpretation of motor unit (MU) activities after surface EMG (sEMG) decomposition is a key factor to decode motor intentions in a noninvasive and physiologically meaningful way. However, there are great challenges due to the difficulty in cross-trial MU tracking and unavoidable loss of partial MU information resulting from incomplete decomposition. In light of these issues, this study presents a novel framework for interpreting MU activities and applies it to decode muscle force. The resulting MUs were clustered and classified into different categories by characterizing their spatially distributed firing waveforms. The process served as a general MU tracking method. On this basis, after transferring the MU firing trains to twitch force trains by a twitch force model, a deep network was designed to predict the normalized force. In addition, MU category distribution was examined to calibrate the actual force level, while functions of some unavailable MUs were compensated. To investigate the effectiveness of this framework, high-density sEMG signals were recorded using an 8×8 electrode array from the abductor pollicis brevis muscles of eight subjects, while thumb abduction force was measured. The proposed method outperformed three common methods ( ) yielding the lowest root mean square deviation of 6.68% ± 1.29% and the highest fitness ( R² ) of 0.94 ± 0.04 between the predicted force and the actual force. This study offers a valuable, computational solution for interpreting individual MU activities, and its effectiveness was confirmed in muscle force estimation.

Electrodes Adaptive Model in Estimating the Depth of Motor Unit: A Motor Unit Action Potential Based Approach

High-density surface electromyography (EMG) has been proposed to overcome the lower selectivity with respect to needle EMG and to provide information on a wide area over the considered muscle. Motor units decomposed from surface EMG signal of different depths differ in the distribution of action potentials detected in the skin surface. We propose a noninvasive model for estimating the depth of motor unit. We find that the depth of motor unit is linearly related to the Gaussian RMS width fitted by data points extracted from motor unit action potential. Simulated and experimental signals are used to evaluate the model performance. The correlation coefficient between reference depth and estimated depth is 0.92 ± 0.01 for simulated motor unit action potentials. Due to the symmetric nature of our model, no significant decrease is detected during the electrode selection procedure. We further checked the estimation results from decomposed motor units, the correlation coefficient between reference depth and estimated depth is 0.82 ± 0.07. For experimental signals, high discrimination of estimated depth vector is detected across gestures among trials. These results show the potential for a straightforward assessment of depth of motor units inside muscles. We discuss the potential of a non-invasive way for the location of decomposed motor units.

Wrist Torque Estimation via Electromyographic Motor Unit Decomposition and Image Reconstruction

Neural interface using decomposed motor units (MUs) from surface electromyography (sEMG) has allowed non-invasive access to the neural control signals, and provided a novel approach for intuitive human-machine interaction. However, most of the existing methods based on decomposed MUs merely adopted the discharge rate (DR) as the feature representations, which may lack local information around the discharge instant and ignore the subtle interactions of different MUs. In this study, we proposed an MU-specific image-based scheme for wrist torque estimation. Specifically, the high-density sEMG signals were decoded into motor unit spike trains (MUSTs), and then MU-specific images were reconstructed with MUSTs and corresponding motor unit action potential (MUAP). A convolutional neural network was used to learn representative features from MU-specific images automatically, and further to estimate wrist torques. The results demonstrated that the proposed method outperformed three conventional and a deep-learning regression approaches using DR features, with the estimation accuracy R² of 0.82 ± 0.09, 0.89 ± 0.06, and nRMSE of 12.6 ± 2.5%, 11.0 ± 3.1% for pronation/supination and flexion/extension, respectively. Further, the analysis of the extracted features from MU-specific images showed a higher correlation than DR for recorded torques, indicating the effectiveness of the proposed method. The outcomes of this study provide a novel and promising perspective for the intuitive control of neural interfacing.

Neural control of matched motor units during muscle shortening and lengthening at increasing velocities

Modulation of movement velocity is necessary during daily life tasks, work, and sports activities. However, assessing motor unit behavior during muscle shortening and lengthening at different velocities is challenging. High-density surface electromyography (HD-sEMG) is an established method to identify and track motor unit behavior in isometric contractions. Therefore, we used this methodology to unravel the behavior of the same motor units in dynamic contractions at low contraction velocities. Velocity-related changes in tibialis anterior motor unit behavior during concentric and eccentric contractions at 10% and 25% maximum voluntary isometric contraction were assessed by decomposing HD-sEMG signals recorded from the tibialis anterior muscle of eleven healthy participants at 5°/s, 10°/s, and 20°/s. Motor units extracted from the dynamic contractions were tracked across different velocities at the same load levels. On average, 14 motor units/participant were matched across different velocities, showing specific changes in discharge rate modulation. Specifically, increased velocity led to an increased rate of change in discharge rate (e.g., discharge rate slope, P = 0.025), recruitment and derecruitment discharge rates (P = 0.003 and P = 0.001), and decreased recruitment angles (P = 0.0001). Surprisingly, the application of the motor unit extraction filters calculated from 20°/s onto the recordings at 5°/s and 10°/s revealed that >92% of motor units recruited at the highest velocity were active on both lower velocities, indicating no additional recruitment of motor units. Our results suggest that motor unit rate coding rather than recruitment is responsible for controlling muscle shortening and lengthening contractions at increasing velocities against a constant load.NEW & NOTEWORTHY The control of movement velocity is accomplished by the modulation of the neural drive to muscle and its variation over time. In this study, we tracked motor units decomposed from HD-sEMG across shortening and lengthening contractions at increasing velocities in two submaximal load levels. We demonstrate that concentric and eccentric contractions of the tibialis anterior muscle at slow velocities are achieved by specific motor unit rate coding strategies rather than distinct recruitment schemes.

Analysis of motor unit spike trains estimated from high-density surface electromyography is highly reliable across operators

There is a growing interest in decomposing high-density surface electromyography (HDsEMG) into motor unit spike trains to improve knowledge on the neural control of muscle contraction. However, the reliability of decomposition approaches is sometimes questioned, especially because they require manual editing of the outputs. We aimed to assess the inter-operator reliability of the identification of motor unit spike trains. Eight operators with varying experience in HDsEMG decomposition were provided with the same data extracted using the convolutive kernel compensation method. They were asked to manually edit them following established procedures. Data included signals from three lower leg muscles and different submaximal intensities. After manual analysis, 126 ± 5 motor units were retained (range across operators: 119-134). A total of 3380 rate of agreement values were calculated (28 pairwise comparisons × 11 contractions/muscles × 4-28 motor units). The median rate of agreement value was 99.6%. Inter-operator reliability was excellent for both mean discharge rate and time at recruitment (intraclass correlation coefficient > 0.99). These results show that when provided with the same decomposed data and the same basic instructions, operators converge toward almost identical results. Our data have been made available so that they can be used for training new operators.

A convolutional neural network to identify motor units from high-density surface electromyography signals in real time

OBJECTIVES

This paper aims to investigate the feasibility and the validity of applying deep convolutional neural networks (CNN) to identify motor unit (MU) spike trains and estimate the neural drive to muscles from high-density electromyography (HD-EMG) signals in real time. Two distinct deep CNNs are compared with the convolution kernel compensation (CKC) algorithm using simulated and experimentally recorded signals. The effects of window size and step size of the input HD-EMG signals are also investigated.

APPROACH

The MU spike trains were first identified with the CKC algorithm. The HD-EMG signals and spike trains were used to train the deep CNN. Then, the deep CNN decomposed the HD-EMG signals into MU discharge times in real time. Two CNN approaches are compared with the CKC: 1) multiple single-output deep CNN (SO-DCNN) with one MU decomposed per network, and 2) one multiple-output deep CNN (MO-DCNN) to decompose all MUs (up to 23) with one network.

MAIN RESULTS

The MO-DCNN outperformed the SO-DCNN in terms of training time (3.2 to 21.4 s/epoch vs. 6.5 to 47.8 s/epoch, respectively) and prediction time (0.04 vs. 0.27 s/sample, respectively). The optimal window size and step size for MO-DCNN were 120 and 20 data points, respectively. It results in sensitivity of 98% and 85% with simulated and experimentally recorded HD-EMG signals, respectively. There is a high cross-correlation coefficient between the neural drive estimated with CKC and that estimated with MO-DCNN (range of r-value across conditions: 0.88-0.95).

SIGNIFICANCE

We demonstrate the feasibility and the validity of using deep CNN to accurately identify MU activity from HD-EMG with a latency lower than 80 ms, which falls within the lower bound of the human electromechanical delay. This method opens many opportunities for using the neural drive to interface humans with assistive devices.

Cut wires: The Electrophysiology of Regenerated Tissue

When nerves are damaged by trauma or disease, they are still capable of firing off electrical command signals that originate from the brain. Furthermore, those damaged nerves have an innate ability to partially regenerate, so they can heal from trauma and even reinnervate new muscle targets. For an amputee who has his/her damaged nerves surgically reconstructed, the electrical signals that are generated by the reinnervated muscle tissue can be sensed and interpreted with bioelectronics to control assistive devices or robotic prostheses. No two amputees will have identical physiologies because there are many surgical options for reconstructing residual limbs, which may in turn impact how well someone can interface with a robotic prosthesis later on. In this review, we aim to investigate what the literature has to say about different pathways for peripheral nerve regeneration and how each pathway can impact the neuromuscular tissue’s final electrophysiology. This information is important because it can guide us in planning the development of future bioelectronic devices, such as prosthetic limbs or neurostimulators. Future devices will primarily have to interface with tissue that has undergone some natural regeneration process, and so we have explored and reported here what is known about the bioelectrical features of neuromuscular tissue regeneration.

Materials, Devices, and Systems of On-Skin Electrodes for Electrophysiological Monitoring and Human-Machine Interfaces

On-skin electrodes function as an ideal platform for collecting high-quality electrophysiological (EP) signals due to their unique characteristics, such as stretchability, conformal interfaces with skin, biocompatibility, and wearable comfort. The past decade has witnessed great advancements in performance optimization and function extension of on-skin electrodes. With continuous development and great promise for practical applications, on-skin electrodes are playing an increasingly important role in EP monitoring and human-machine interfaces (HMI). In this review, the latest progress in the development of on-skin electrodes and their integrated system is summarized. Desirable features of on-skin electrodes are briefly discussed from the perspective of performances. Then, recent advances in the development of electrode materials, followed by the analysis of strategies and methods to enhance adhesion and breathability of on-skin electrodes are examined. In addition, representative integrated electrode systems and practical applications of on-skin electrodes in healthcare monitoring and HMI are introduced in detail. It is concluded with the discussion of key challenges and opportunities for on-skin electrodes and their integrated systems.

Generalized Finger Motion Classification Model Based on Motor Unit Voting

Surface electromyogram-based finger motion classification has shown its potential for prosthetic control. However, most current finger motion classification models are subject-specific, requiring calibration when applied to new subjects. Generalized subject-nonspecific models are essential for real-world applications. In this study, the authors developed a subject-nonspecific model based on motor unit (MU) voting. A high-density surface electromyogram was first decomposed into individual MUs. The features extracted from each MU were then fed into a random forest classifier to obtain the finger label (primary prediction). The final prediction was selected by voting for all primary predictions provided by the decomposed MUs. Experiments conducted on 14 subjects demonstrated that our method significantly outperformed traditional methods in the context of subject-nonspecific finger motion classification models.

On the Use of Fuzzy and Permutation Entropy in Hand Gesture Characterization from EMG Signals: Parameters Selection and Comparison

The surface electromyography signal (sEMG) is widely used for gesture characterization; its reliability is strongly connected to the features extracted from sEMG recordings. This study aimed to investigate the use of two complexity measures, i.e., fuzzy entropy (FEn) and permutation entropy (PEn) for hand gesture characterization. Fourteen upper limb movements, sorted into three sets, were collected on ten subjects and the performances of FEn and PEn for gesture descriptions were analyzed for different computational parameters. FEn and PEn were able to properly cluster the expected numbers of gestures, but computational parameters were crucial for ensuring clusters’ separability and proper gesture characterization. FEn and PEn were also compared with other eighteen classical time and frequency domain features through the minimum redundancy maximum relevance algorithm and showed the best predictive importance scores in two gesture sets; they also had scores within the subset of the best five features in the remaining one. Further, the classification accuracies of four different feature sets presented remarkable increases when FEn and PEn are included as additional features. Outcomes support the use of FEn and PEn for hand gesture description when computational parameters are properly selected, and they could be useful in supporting the development of robotic arms and prostheses myoelectric control.

Analysis of motor unit activities during multiple motor tasks by real-time EMG decomposition: perspective for myoelectric control

Surface electromyography (EMG) decomposition techniques have been applied for human-machine interfacing by decoding neural information, while most of decomposition approaches work offline. Here, we apply an online decomposition scheme to decode motor unit activities during three motor tasks, and measure the recognition accuracy of motor type and activation level using the decomposition results. High-density surface EMG signal were recorded from forearm muscles of six able-bodied subjects. The EMG signals were decomposed into motor unit spike trains (MUST) with a sliding window of 100 ms. The computation complexity had time consumption < 50 ms in each window. Most identified motor units discharged during only one motor task. On average, over 5 MUSTs were identified for each motion and the recognition accuracy based on motor unit activities was > 99%. The discharge rate of motor units was highly correlated with the activation level of each motion with an average correlation coefficient of 0.94 ± 0.04. These results indicate the feasibility of an online, multi-motion, and proportional control scheme based on neural decoding in a non-invasive way.

Hand-Gesture Recognition Based on EMG and Event-Based Camera Sensor Fusion: A Benchmark in Neuromorphic Computing

Hand gestures are a form of non-verbal communication used by individuals in conjunction with speech to communicate. Nowadays, with the increasing use of technology, hand-gesture recognition is considered to be an important aspect of Human-Machine Interaction (HMI), allowing the machine to capture and interpret the user's intent and to respond accordingly. The ability to discriminate between human gestures can help in several applications, such as assisted living, healthcare, neuro-rehabilitation, and sports. Recently, multi-sensor data fusion mechanisms have been investigated to improve discrimination accuracy. In this paper, we present a sensor fusion framework that integrates complementary systems: the electromyography (EMG) signal from muscles and visual information. This multi-sensor approach, while improving accuracy and robustness, introduces the disadvantage of high computational cost, which grows exponentially with the number of sensors and the number of measurements. Furthermore, this huge amount of data to process can affect the classification latency which can be crucial in real-case scenarios, such as prosthetic control. Neuromorphic technologies can be deployed to overcome these limitations since they allow real-time processing in parallel at low power consumption. In this paper, we present a fully neuromorphic sensor fusion approach for hand-gesture recognition comprised of an event-based vision sensor and three different neuromorphic processors. In particular, we used the event-based camera, called DVS, and two neuromorphic platforms, Loihi and ODIN + MorphIC. The EMG signals were recorded using traditional electrodes and then converted into spikes to be fed into the chips. We collected a dataset of five gestures from sign language where visual and electromyography signals are synchronized. We compared a fully neuromorphic approach to a baseline implemented using traditional machine learning approaches on a portable GPU system. According to the chip's constraints, we designed specific spiking neural networks (SNNs) for sensor fusion that showed classification accuracy comparable to the software baseline. These neuromorphic alternatives have increased inference time, between 20 and 40%, with respect to the GPU system but have a significantly smaller energy-delay product (EDP) which makes them between 30× and 600× more efficient. The proposed work represents a new benchmark that moves neuromorphic computing toward a real-world scenario.

Hybrid Encoder-Decoder Deep Networks for Decoding Neural Drive Information towards Precise Muscle Force Estimation

How to utilize and interpret microscopic motor unit (MU) activities after surface electromyogram (sEMG) decomposition towards accurate decoding of the neural control remains a great challenge. In this study, a novel framework of hybrid encoder-decoder deep networks is proposed to process the microscopic neural drive information and it is applied to precise muscle force estimation. After a high-density sEMG (HD-sEMG) decomposition was performed using the progressive FastICA peel-off algorithm, a muscle twitch force model was then applied to basically convert each channel's electric waveform (i.e., action potential) of each MU into a twitch force. Next, hybrid encoder-decoder deep networks were performed on every 50 ms of segment of the summation of twitch force trains from all decomposed MUs. The encoder network was designed to characterize spatial information of MU's force contribution over all channels, and the decoder network finally decoded the muscle force. This framework was validated on HD-sEMG recordings from the abductor pollicis brevis muscles of five subjects by a thumb abduction task using an 8 × 8 grid. The proposed framework yielded a mean root mean square error of 6.62% ± 1.26% and a mean coefficient of determination value of 0.95 ± 0.03 from a linear regression analysis between the estimated force and actual force over all data trials, and it outperformed three common methods with statistical significance (p < 0.001). This study offers a valuable solution for interpreting microscopic neural drive information and demonstrates its success in predicting muscle force.