Integration of Motor Unit Filters for Enhanced Surface Electromyogram Decomposition During Varying Force Isometric Contraction

Muscles generate varying levels of force by recruiting different numbers of motor units (MUs), and as the force increases, the number of recruited MUs gradually rises. However, current decoding methods encounter difficulties in maintaining a stable and consistent growth trend in MU numbers with increasing force. In some instances, an unexpected reduction in the number of MUs can even be observed as force intensifies. To address this issue, in this study, we propose an enhanced decoding method that adaptively reutilizes MU filters. Specifically, in addition to the normal decoding process, we introduced an additional procedure where MU filters are reused to initialize the algorithm. The MU filters are iterated and adapted to the new signals, aiming to decode motor units that were actually activated but cannot be identified due to heavy superimposition. We tested our method on both simulated and experimental surface electromyogram (sEMG) signals. We simulated isometric signals (10%-70%) with known MU firing patterns using experimentally recorded MU action potentials from forearm muscles and compared the decomposition results to two baseline approaches: convolution kernel compensation (CKC) and fast independent component analysis (fastICA). Our method increased the decoded MU number by a rate of 135.4% ± 62.5 % and 63.6% ± 20.2 % for CKC and fastICA, respectively, across different signal-to-noise ratios. The sensitivity and precision for MUs decomposed using the enhanced method remained at the same accuracy level (p <0.001) as those of normally decoded MUs. For the experimental signals, eight healthy subjects performed hand movements at five different force levels (10%-90%), during which sEMG signals were recorded and decomposed. The results indicate that the enhanced process increased the number of decoded MUs by 21.8% ± 10.9 % across all subjects. We discussed the possibility of fully capturing all activated motor units by appropriately reusing previously decoded MU filters and improving the balance of activated motor unit numbers across varying excitation levels.

Adaptive EMG decomposition in dynamic conditions based on online learning metrics with tunable hyperparameters

Objective. Developing neural decoders robust to non-stationary conditions is essential to ensure their long-term accuracy and stability. This is particularly important when decoding the neural drive to muscles during dynamic contractions, which pose significant challenges for stationary decoders.Approach. We propose a novel adaptive electromyography (EMG) decomposition algorithm that builds on blind source separation methods by leveraging the Kullback-Leibler divergence and kurtosis of the signals as metrics for online learning. The proposed approach provides a theoretical framework to tune the adaptation hyperparameters and compensate for non-stationarities in the mixing matrix, such as due to dynamic contractions, and to identify the underlying motor neuron (MN) discharges. The adaptation is performed in real-time (∼22 ms of computational time per 100 ms batches).Main results. The hyperparameters of the proposed adaptation captured anatomical differences between recording locations (forearm vs wrist) and generalised across subjects. Once optimised, the proposed adaptation algorithm significantly improved all decomposition performance metrics with respect to the absence of adaptation in a wide range of motion of the wrist (80∘). The rate of agreement, sensitivity, and precision were⩾90%in⩾80%of the cases in both simulated and experimentally recorded data, according to a two-source validation approach.Significance. The findings demonstrate the suitability of the proposed online learning metrics and hyperparameter optimisation to compensate the induced modulation and accurately decode MN discharges in dynamic conditions. Moreover, the study proposes an experimental validation method for EMG decomposition in dynamic tasks.

2CFastICA: A Novel Method for High Density Surface EMG Decomposition Based on Kernel Constrained FastICA and Correlation Constrained FastICA

This study presents a novel high density surface electromyography (EMG) decomposition method, named as 2CFastICA, because it incorporates two key algorithms: kernel constrained FastICA and correlation constrained FastICA. The former focuses on overcoming the local convergence of FastICA without requiring the peel-off strategy used in the progressive FastICA peel-off (PFP) framework. The latter further refines the output of kernel constrained FastICA by correcting possible erroneous or missed spikes. The two constrained FastICA algorithms supplement each other to warrant the decomposition performance. The 2CFastICA method was validated using simulated surface EMG signals with different motor unit numbers and signal to noise ratios (SNRs). Two source validation was also performed by simultaneous high density surface EMG and intramuscular EMG recordings, showing a matching rate (MR) of (97.2 ± 3.5)% for 170 common motor units. In addition, a different form of two source validation was also conducted taking advantages of the high density surface EMG characteristics of patients with amyotrophic lateral sclerosis, showing a MR of (99.4 ± 0.9)% for 34 common motor units from interference and sparse datasets. Both simulation and experimental results indicate that 2CFastICA can achieve similar decomposition performance to PFP. However, the efficiency of decomposition can be greatly improved by 2CFastICA since the complex signal processing procedures associated with the peel-off strategy are not required any more. Along with this paper, we also provide the MATLAB open source code of 2CFastICA for high density surface EMG decomposition.

NeuroMechanics: Electrophysiological and computational methods to accurately estimate the neural drive to muscles in humans in vivo

The ultimate neural signal for muscle control is the neural drive sent from the spinal cord to muscles. This neural signal comprises the ensemble of action potentials discharged by the active spinal motoneurons, which is transmitted to the innervated muscle fibres to generate forces. Accurately estimating the neural drive to muscles in humans in vivo is challenging since it requires the identification of the activity of a sample of motor units (MUs) that is representative of the active MU population. Current electrophysiological recordings usually fail in this task by identifying small MU samples with over-representation of higher-threshold with respect to lower-threshold MUs. Here, we describe recent advances in electrophysiological methods that allow the identification of more representative samples of greater numbers of MUs than previously possible. This is obtained with large and very dense arrays of electromyographic electrodes. Moreover, recently developed computational methods of data augmentation further extend experimental MU samples to infer the activity of the full MU pool. In conclusion, the combination of new electrode technologies and computational modelling allows for an accurate estimate of the neural drive to muscles and opens new perspectives in the study of the neural control of movement and in neural interfacing.

Improved multi-layer wavelet transform and blind source separation based ECG artifacts removal algorithm from the sEMG signal: in the case of upper limbs

Introduction: Surface electromyogram (sEMG) signals have been widely used in human upper limb force estimation and motion intention recognition. However, the electrocardiogram(ECG) artifact generated by the beating of the heart is a major factor that reduces the quality of the EMG signal when recording the sEMG signal from the muscle close to the heart. sEMG signals contaminated by ECG artifacts are difficult to be understood correctly. The objective of this paper is to effectively remove ECG artifacts from sEMG signals by a novel method. Methods: In this paper, sEMG and ECG signals of the biceps brachii, brachialis, and triceps muscle of the human upper limb will be collected respectively. Firstly, an improved multi-layer wavelet transform algorithm is used to preprocess the raw sEMG signal to remove the background noise and power frequency interference in the raw signal. Then, based on the theory of blind source separation analysis, an improved Fast-ICA algorithm was constructed to separate the denoising signals. Finally, an ECG discrimination algorithm was used to find and eliminate ECG signals in sEMG signals. This method consists of the following steps: 1) Acquisition of raw sEMG and ECG signals; 2) Decoupling the raw sEMG signal; 3) Fast-ICA-based signal component separation; 4) ECG artifact recognition and elimination. Results and discussion: The experimental results show that our method has a good effect on removing ECG artifacts from contaminated EMG signals. It can further improve the quality of EMG signals, which is of great significance for improving the accuracy of force estimation and motion intention recognition tasks. Compared with other state-of-the-art methods, our method can also provide the guiding significance for other biological signals.

Transferable non-invasive modal fusion-transformer (NIMFT) for end-to-end hand gesture recognition

Objective.Recent studies have shown that integrating inertial measurement unit (IMU) signals with surface electromyographic (sEMG) can greatly improve hand gesture recognition (HGR) performance in applications such as prosthetic control and rehabilitation training. However, current deep learning models for multimodal HGR encounter difficulties in invasive modal fusion, complex feature extraction from heterogeneous signals, and limited inter-subject model generalization. To address these challenges, this study aims to develop an end-to-end and inter-subject transferable model that utilizes non-invasively fused sEMG and acceleration (ACC) data.Approach.The proposed non-invasive modal fusion-transformer (NIMFT) model utilizes 1D-convolutional neural networks-based patch embedding for local information extraction and employs a multi-head cross-attention (MCA) mechanism to non-invasively integrate sEMG and ACC signals, stabilizing the variability induced by sEMG. The proposed architecture undergoes detailed ablation studies after hyperparameter tuning. Transfer learning is employed by fine-tuning a pre-trained model on new subject and a comparative analysis is performed between the fine-tuning and subject-specific model. Additionally, the performance of NIMFT is compared to state-of-the-art fusion models.Main results.The NIMFT model achieved recognition accuracies of 93.91%, 91.02%, and 95.56% on the three action sets in the Ninapro DB2 dataset. The proposed embedding method and MCA outperformed the traditional invasive modal fusion transformer by 2.01% (embedding) and 1.23% (fusion), respectively. In comparison to subject-specific models, the fine-tuning model exhibited the highest average accuracy improvement of 2.26%, achieving a final accuracy of 96.13%. Moreover, the NIMFT model demonstrated superiority in terms of accuracy, recall, precision, and F1-score compared to the latest modal fusion models with similar model scale.Significance.The NIMFT is a novel end-to-end HGR model, utilizes a non-invasive MCA mechanism to integrate long-range intermodal information effectively. Compared to recent modal fusion models, it demonstrates superior performance in inter-subject experiments and offers higher training efficiency and accuracy levels through transfer learning than subject-specific approaches.

Adaptive HD-sEMG decomposition: towards robust real-time decoding of neural drive

Objective. Neural interfacing via decomposition of high-density surface electromyography (HD-sEMG) should be robust to signal non-stationarities incurred by changes in joint pose and contraction intensity.Approach. We present an adaptive real-time motor unit decoding algorithm and test it on HD-sEMG collected from the extensor carpi radialis brevis during isometric contractions over a range of wrist angles and contraction intensities. The performance of the algorithm was verified using high-confidence benchmark decompositions derived from concurrently recorded intramuscular electromyography.Main results. In trials where contraction conditions between the initialization and testing data differed, the adaptive decoding algorithm maintained significantly higher decoding accuracies when compared to static decoding methods.Significance. Using "gold standard" verification techniques, we demonstrate the limitations of filter re-use decoding methods and show the necessity of parameter adaptation to achieve robust neural decoding.

Motoneuron-driven computational muscle modelling with motor unit resolution and subject-specific musculoskeletal anatomy

The computational simulation of human voluntary muscle contraction is possible with EMG-driven Hill-type models of whole muscles. Despite impactful applications in numerous fields, the neuromechanical information and the physiological accuracy such models provide remain limited because of multiscale simplifications that limit comprehensive description of muscle internal dynamics during contraction. We addressed this limitation by developing a novel motoneuron-driven neuromuscular model, that describes the force-generating dynamics of a population of individual motor units, each of which was described with a Hill-type actuator and controlled by a dedicated experimentally derived motoneuronal control. In forward simulation of human voluntary muscle contraction, the model transforms a vector of motoneuron spike trains decoded from high-density EMG signals into a vector of motor unit forces that sum into the predicted whole muscle force. The motoneuronal control provides comprehensive and separate descriptions of the dynamics of motor unit recruitment and discharge and decodes the subject's intention. The neuromuscular model is subject-specific, muscle-specific, includes an advanced and physiological description of motor unit activation dynamics, and is validated against an experimental muscle force. Accurate force predictions were obtained when the vector of experimental neural controls was representative of the discharge activity of the complete motor unit pool. This was achieved with large and dense grids of EMG electrodes during medium-force contractions or with computational methods that physiologically estimate the discharge activity of the motor units that were not identified experimentally. This neuromuscular model advances the state-of-the-art of neuromuscular modelling, bringing together the fields of motor control and musculoskeletal modelling, and finding applications in neuromuscular control and human-machine interfacing research.

A fully integrated, standalone stretchable device platform with in-sensor adaptive machine learning for rehabilitation

Post-surgical treatments of the human throat often require continuous monitoring of diverse vital and muscle activities. However, wireless, continuous monitoring and analysis of these activities directly from the throat skin have not been developed. Here, we report the design and validation of a fully integrated standalone stretchable device platform that provides wireless measurements and machine learning-based analysis of diverse vibrations and muscle electrical activities from the throat. We demonstrate that the modified composite hydrogel with low contact impedance and reduced adhesion provides high-quality long-term monitoring of local muscle electrical signals. We show that the integrated triaxial broad-band accelerometer also measures large body movements and subtle physiological activities/vibrations. We find that the combined data processed by a 2D-like sequential feature extractor with fully connected neurons facilitates the classification of various motion/speech features at a high accuracy of over 90%, which adapts to the data with noise from motion artifacts or the data from new human subjects. The resulting standalone stretchable device with wireless monitoring and machine learning-based processing capabilities paves the way to design and apply wearable skin-interfaced systems for the remote monitoring and treatment evaluation of various diseases.

Optimal Motor Unit Subset Selection for Accurate Motor Intention Decoding: Towards Dexterous Real-Time Interfacing

OBJECTIVE

Motor unit (MU) discharge timings encode human motor intentions to the finest degree. Whilst tapping into such information can bring significant gains to a range of applications, current approaches to MU decoding from surface signals do not scale well with the demands of dexterous human-machine interfacing (HMI). To optimize the forward estimation accuracy and time-efficiency of such systems, we propose the inclusion of task-wise initialization and MU subset selection.

METHODS

Offline analyses were conducted on data recorded from 11 non-disabled subjects. Task-wise decomposition was applied to identify MUs from high-density surface electromyography (HD-sEMG) pertaining to 18 wrist/forearm motor tasks. The activities of a selected subset of MUs were extracted from test data and used for forward estimation of intended motor tasks and joint kinematics. To that end, various combinations of subset selection and estimation algorithms (both regression and classification-based) were tested for a range of subset sizes.

RESULTS

The mutual information-based minimum Redundancy Maximum Relevance (mRMR-MI) criterion retained MUs with the highest predicative power. When the portion of tracked MUs was reduced down to 25%, the regression performance decreased only by 3% (R2=0.79) while classification accuracy dropped by 2.7% (accuracy = 74%) when kernel-based estimators were considered.

CONCLUSION AND SIGNIFICANCE

Careful selection of tracked MUs can optimize the efficiency of MU-driven interfacing. In particular, prioritization of MUs exhibiting strong nonlinear relationships with target motions is best leveraged by kernel-based estimators. Hence, this frees resources for more robust and adaptive MU decoding techniques to be implemented in future.

TraHGR: Transformer for Hand Gesture Recognition via Electromyography

Deep learning-based Hand Gesture Recognition (HGR) via surface Electromyogram (sEMG) signals have recently shown considerable potential for development of advanced myoelectric-controlled prosthesis. Although deep learning techniques can improve HGR accuracy compared to their classical counterparts, classifying hand movements based on sparse multichannel sEMG signals is still a challenging task. Furthermore, existing deep learning approaches, typically, include only one model as such can hardly extract representative features. In this paper, we aim to address this challenge by capitalizing on the recent advances in hybrid models and transformers. In other words, we propose a hybrid framework based on the transformer architecture, which is a relatively new and revolutionizing deep learning model. The proposed hybrid architecture, referred to as the Transformer for Hand Gesture Recognition (TraHGR), consists of two parallel paths followed by a linear layer that acts as a fusion center to integrate the advantage of each module. We evaluated the proposed architecture TraHGR based on the commonly used second Ninapro dataset, referred to as the DB2. The sEMG signals in the DB2 dataset are measured in real-life conditions from 40 healthy users, each performing 49 gestures. We have conducted an extensive set of experiments to test and validate the proposed TraHGR architecture, and compare its achievable accuracy with several recently proposed HGR classification algorithms over the same dataset. We have also compared the results of the proposed TraHGR architecture with each individual path and demonstrated the distinguishing power of the proposed hybrid architecture. The recognition accuracies of the proposed TraHGR architecture for the window of size 200ms and step size of 100ms are 86.00%, 88.72%, 81.27%, and 93.74%, which are 2.30%, 4.93%, 8.65%, and 4.20% higher than the state-of-the-art performance for DB2 (49 gestures), DB2-B (17 gestures), DB2-C (23 gestures), and DB2-D (9 gestures), respectively.

Movement recognition via channel-activation-wise sEMG attention

CONTEXT

Surface electromyography (sEMG) signals contain rich information recorded from muscle movements and therefore reflect the user's intention. sEMG has seen dominant applications in rehabilitation, clinical diagnosis as well as human engineering, etc. However, current feature extraction methods for sEMG signals have been seriously limited by their stochasticity, transiency, and non-stationarity.

OBJECTIVE

Our objective is to combat the difficulties induced by the aforementioned downsides of sEMG and thereby extract representative features for various downstream movement recognition.

METHOD

We propose a novel 3-axis view of sEMG features composed of temporal, spatial, and channel-wise summary. We leverage the state-of-the-art architecture Transformer to enforce efficient parallel search and to get rid of limitations imposed by previous work in gesture classification. The transformer model is designed on top of an attention-based module, which allows for the extraction of global contextual relevance among channels and the use of this relevance for sEMG recognition.

RESULTS

We compared the proposed method against existing methods on two Ninapro datasets consisting of data from both healthy people and amputees. Experimental results show the proposed method attains the state-of-the-art (SOTA) accuracy on both datasets. We further show that the proposed method enjoys strong generalization ability: a new SOTA is achieved by pretraining the model on a different dataset followed by fine-tuning it on the target dataset.

High-Density Surface EMG Decomposition by Combining Iterative Convolution Kernel Compensation With an Energy-Specific Peel-off Strategy

Objective- This study aims to develop a novel framework for high-density surface electromyography (HD-sEMG) signal decomposition with superior decomposition yield and accuracy, especially for low-energy MUs. Methods- An iterative convolution kernel compensation-peel off (ICKC-P) framework is proposed, which consists of three steps: decomposition of the motor units (MUs) with relatively large energy by using the iterative convolution kernel compensation (ICKC) method and extraction of low-energy MUs with a Post-Processor and novel 'peel-off' strategy. Results- The performance of the proposed framework was evaluated by both simulated and experimental HD-sEMG signals. Our simulation results demonstrated that, with 120 simulated MUs, the proposed framework extracts more MUs compared to K-means convolutional kernel compensation (KmCKC) approach across six noise levels. And the proposed 'peel-off' strategy estimates more accurate MUAP waveforms at six noise levels than the 'peel-off' strategy proposed in the progressive FastICA peel-off (PFP) framework. For the experimental sEMG signals recorded from biceps brachii, an average of 16.1 ±3.4 MUs were identified from each contraction, while only 10.0 ± 2.8 MUs were acquired by the KmCKC method. Conclusion- The high yield and accuracy of MUs decomposed from simulated and experimental HD-sEMG signals demonstrate the superiority of the proposed framework in decomposing low-energy MUs compared to existing methods for HD-sEMG signal decomposition. Significance- The proposed framework enables us to construct a more representative motor unit pool, consequently enhancing our understanding pertaining to various neuropathological conditions and providing invaluable information for the diagnosis and treatment of neuromuscular disorders and motor neuron diseases.

Transformer-based hand gesture recognition from instantaneous to fused neural decomposition of high-density EMG signals

Designing efficient and labor-saving prosthetic hands requires powerful hand gesture recognition algorithms that can achieve high accuracy with limited complexity and latency. In this context, the paper proposes a Compact Transformer-based Hand Gesture Recognition framework referred to as [Formula: see text], which employs a vision transformer network to conduct hand gesture recognition using high-density surface EMG (HD-sEMG) signals. Taking advantage of the attention mechanism, which is incorporated into the transformer architectures, our proposed [Formula: see text] framework overcomes major constraints associated with most of the existing deep learning models such as model complexity; requiring feature engineering; inability to consider both temporal and spatial information of HD-sEMG signals, and requiring a large number of training samples. The attention mechanism in the proposed model identifies similarities among different data segments with a greater capacity for parallel computations and addresses the memory limitation problems while dealing with inputs of large sequence lengths. [Formula: see text] can be trained from scratch without any need for transfer learning and can simultaneously extract both temporal and spatial features of HD-sEMG data. Additionally, the [Formula: see text] framework can perform instantaneous recognition using sEMG image spatially composed from HD-sEMG signals. A variant of the [Formula: see text] is also designed to incorporate microscopic neural drive information in the form of Motor Unit Spike Trains (MUSTs) extracted from HD-sEMG signals using Blind Source Separation (BSS). This variant is combined with its baseline version via a hybrid architecture to evaluate potentials of fusing macroscopic and microscopic neural drive information. The utilized HD-sEMG dataset involves 128 electrodes that collect the signals related to 65 isometric hand gestures of 20 subjects. The proposed [Formula: see text] framework is applied to 31.25, 62.5, 125, 250 ms window sizes of the above-mentioned dataset utilizing 32, 64, 128 electrode channels. Our results are obtained via 5-fold cross-validation by first applying the proposed framework on the dataset of each subject separately and then, averaging the accuracies among all the subjects. The average accuracy over all the participants using 32 electrodes and a window size of 31.25 ms is 86.23%, which gradually increases till reaching 91.98% for 128 electrodes and a window size of 250 ms. The [Formula: see text] achieves accuracy of 89.13% for instantaneous recognition based on a single frame of HD-sEMG image. The proposed model is statistically compared with a 3D Convolutional Neural Network (CNN) and two different variants of Support Vector Machine (SVM) and Linear Discriminant Analysis (LDA) models. The accuracy results for each of the above-mentioned models are paired with their precision, recall, F1 score, required memory, and train/test times. The results corroborate effectiveness of the proposed [Formula: see text] framework compared to its counterparts.

Robust Independent Component Analysis based EMG decomposition - a comparison study

High density surface Electromyography (HD-sEMG) provides a high fidelity measurement of the myoelectric activity that can be leveraged by EMG decomposition methods to estimate the motor neuron discharges. Independent Component Analysis (ICA) methods are used as basis for many EMG decomposition algorithms, for the estimation of motor unit action potential signals. Accurate source separation is a non-trivial task in EMG decomposition. While FastICA is widely used for this purpose, other methods with attractive characteristics, such as RobustICA, remain relatively unexplored. The purpose of the current work is to compare three different ICA-based EMG decomposition methods (FastICA, RobustICA and RobustICALCH) in terms of decomposition accuracy and computation time. The evaluation was performed on simulated data using a decomposition algorithm inspired by previous studies. Our results demonstrate that RobustICA outperforms the other methods in terms of number of correctly identified motor units, high decomposition accuracy, and low computation time, across different muscle contraction levels.

Semi-Automated Identification of Motor Units Concurrently Recorded in High-Density Surface and Intramuscular Electromyography

An increasing focus on extending automated surface electromyography (EMG) decomposition algorithms to operate under non-stationary conditions requires rigorous and robust validation. However, relevant benchmarks derived manually from iEMG are laborsome to obtain and this is further exacerbated by the need to consider multiple contraction conditions. This work demonstrates a semi-automatic technique for extracting motor units (MUs) whose activities are present in concurrently recorded high-density surface EMG (HD-sEMG) and intramuscular EMG (iEMG) during isometric contractions. We leverage existing automatic surface decomposition algorithms for initial identification of active MUs. Resulting spike times are then used to identify (trigger) the sources that are concurrently detectable in iEMG. We demonstrate this technique on recordings targeting the extensor carpi radialis brevis in five human subjects. This dataset consists of 117 trials across different force levels and wrist angles, from which the presented method yielded a set of 367 high-confidence decompositions. Thus, our approach effectively alleviates the overhead of manual decomposition as it efficiently produces reliable benchmarks under different conditions.Clinical Relevance- We present an efficient method for obtaining high-quality in-vivo decomposition particularly useful in the verification of new surface decomposition approaches.

A myoelectric digital twin for fast and realistic modelling in deep learning

Muscle electrophysiology has emerged as a powerful tool to drive human machine interfaces, with many new recent applications outside the traditional clinical domains, such as robotics and virtual reality. However, more sophisticated, functional, and robust decoding algorithms are required to meet the fine control requirements of these applications. Deep learning has shown high potential in meeting these demands, but requires a large amount of high-quality annotated data, which is expensive and time-consuming to acquire. Data augmentation using simulations, a strategy applied in other deep learning applications, has never been attempted in electromyography due to the absence of computationally efficient models. We introduce a concept of Myoelectric Digital Twin - highly realistic and fast computational model tailored for the training of deep learning algorithms. It enables simulation of arbitrary large and perfectly annotated datasets of realistic electromyography signals, allowing new approaches to muscular signal decoding, accelerating the development of human-machine interfaces.

Bio-robotics research for non-invasive myoelectric neural interfaces for upper-limb prosthetic control: a 10-year perspective review

A decade ago, a group of researchers from academia and industry identified a dichotomy between the industrial and academic state-of-the-art in upper-limb prosthesis control, a widely used bio-robotics application. They proposed that four key technical challenges, if addressed, could bridge this gap and translate academic research into clinically and commercially viable products. These challenges are unintuitive control schemes, lack of sensory feedback, poor robustness and single sensor modality. Here, we provide a perspective review on the research effort that occurred in the last decade, aiming at addressing these challenges. In addition, we discuss three research areas essential to the recent development in upper-limb prosthetic control research but were not envisioned in the review 10 years ago: deep learning methods, surface electromyogram decomposition and open-source databases. To conclude the review, we provide an outlook into the near future of the research and development in upper-limb prosthetic control and beyond.

Toward a generalizable deep CNN for neural drive estimation across muscles and participants

Objective.High-density electromyography (HD-EMG) decomposition algorithms are used to identify individual motor unit (MU) spike trains, which collectively constitute the neural code of movements, to predict motor intent. This approach has advanced from offline to online decomposition, from isometric to dynamic contractions, leading to a wide range of neural-machine interface applications. However, current online methods need offline retraining when applied to the same muscle on a different day or to a different person, which limits their applications in a real-time neural-machine interface. We proposed a deep convolutional neural network (CNN) framework for neural drive estimation, which takes in frames of HD-EMG signals as input, extracts general spatiotemporal properties of MU action potentials, and outputs the number of spikes in each frame. The deep CNN can generalize its application without retraining to HD-EMG data recorded in separate sessions, muscles, or participants.Approach.We recorded HD-EMG signals from the vastus medialis and vastus lateralis muscles from five participants while they performed isometric contractions during two sessions separated by ∼20 months. We identified MU spike trains from HD-EMG signals using a convolutive blind source separation (BSS) method, and then used the cumulative spike train (CST) of these MUs and the HD-EMG signals to train and validate the deep CNN.Main results.On average, the correlation coefficients between CST from the BSS and that from deep CNN were0.983±0.006for leave-one-out across-sessions-and-muscles validation and0.989±0.002for leave-one-out across-participants validation. When trained with more than four datasets, the performance of deep CNN saturated at0.984±0.001for cross validations across muscles, sessions, and participants.Significance.We can conclude that the deep CNN is generalizable across the aforementioned conditions without retraining. We could potentially generate a robust deep CNN to estimate neural drive to muscles for neural-machine interfaces.

Compressed spike-triggered averaging in iterative decomposition of surface EMG

BACKGROUND AND OBJECTIVE

Analysis of motor unit activity is important for assessing and treating diseases or injuries affecting natural movement. State-of-the-art decomposition translates high-density surface electromyography (HDsEMG) into motor unit activity. However, current decomposition methods offer far from complete separation of all motor units.

METHODS

This paper proposes a peel-off approach to automatic decomposition of HDsEMG into motor unit action potential (MUAP) trains, based on the Fast Independent Component Analysis algorithm (FastICA). The novel steps include utilizing compression by means of Principal Component Analysis and spike-triggered averaging, to estimate surface MUAP distributions with less noise, which are iteratively subtracted from the HDsEMG dataset. Furthermore, motor unit spike trains are estimated by high-dimensional density-based clustering of peaks in the FastICA source output. And finally, a new reliability measure is used to discard poor motor unit estimates by comparing the variance of the FastICA source output before and after the peel-off step. The method was validated using reconstructed synthetic data at three different signal-to-noise levels and was compared to an established deflationary FastICA approach.

RESULTS

Both algorithms had very high recall and precision, over 90%, for spikes from matching motor units, referred to as matched performance. However, the peel-off algorithm correctly identified more motor units for all noise levels. When accounting for unidentified motor units, total recall was up to 33 percentage points higher; and when accounting for duplicate estimates, total precision was up to 24 percentage points higher, compared to the state-of-the-art reference. In addition, a comparison was done using experimental data where the proposed algorithm had a matched recall of 97% and precision of 85% with respect to the reference algorithm.

CONCLUSION

These results show a substantial performance increase for decomposition of simulated HDsEMG data and serve to validate the proposed approach. This performance increase is an important step towards complete decomposition and extraction of information of motor unit activity.

Multi-day dataset of forearm and wrist electromyogram for hand gesture recognition and biometrics

Surface electromyography (sEMG) signals have been used for advanced prosthetics control, hand-gesture recognition (HGR), and more recently as a novel biometric trait. For these sEMG-based applications, the translation from laboratory research setting to real-life scenarios suffers from two major limitations: (1) a small subject pool, and (2) single-session data recordings, both of which prevents acceptable generalization ability. In this longitudinal database, forearm and wrist sEMG data were collected from 43 participants over three different days with long separation (Days 1, 8, and 29) while they performed static hand/wrist gestures. The objective of this dataset is to provide a comprehensive dataset for the development of robust machine learning algorithms of sEMG, for both HGR and biometric applications. We demonstrated the high quality of the current dataset by comparing with the Ninapro dataset. And we presented its usability for both HGR and biometric applications. Among other applications, the dataset can also be used for developing electrode-shift invariant generalized models, which can further bolster the development of wristband and forearm-bracelet sensors.

Segment-Wise Decomposition of Surface Electromyography to Identify Discharges Across Motor Neuron Populations

OBJECTIVE

The surface electromyography (EMG) decomposition techniques have shown promising results in neurophysiologic investigations, clinical diagnosis, and human-machine interfacing. However, current decomposition methods could only decode a limited number of motor units (MUs) because of the local convergence. The number of identified MUs remains similar even though more muscles or movements are involved, where multiple motor neuron populations are activated. The objective of this study was to develop a segment-wise decomposition strategy to increase the number of MU decoded from multiple motor neuron populations.

METHODS

The EMG signals were divided into several segments depending on the number of involved movements. The motor neurons, activated during each movement, were regarded as a population. The convolution kernel compensation (CKC) method was applied individually for each segment to decode the motor unit discharges from each motor neuron population. The MU filters were obtained in each segment and filtrated to estimate the MU spike trains (MUSTs) from the global EMG signals. The decomposition performance was validated on synthetic and experimental EMG signals.

MAIN RESULTS

From synthetic EMG signals generated by two motor neuron populations, the proposed segment-wise CKC (swCKC) decoded significantly more MUs during low and medium excitation levels, with an increased rate of 16.3% to 75.4% compared with the conventional CKC. From experimental signals recorded during ten motor tasks, 133±24 MUs with the pulse-to-noise ratio of 36.6±6.5 dB were identified for each subject by swCKC, whereas the conventional CKC identified only 43±12 MUs.

CONCLUSION AND SIGNIFICANCE

These results indicate the feasibility and superiority of the proposed swCKC to decode MU activities across motor neuron populations, extending the potential applications of EMG decomposition for neural decoding during multiple motor tasks.

Muscle Synergy-driven Motor Unit Clustering for Human-Machine Interfacing

Electromyographic signals (EMGs) can provide information on the overall activity of the innervating motor neuros in any given muscle but also globally reflect the underlying neuromechanics of human movement (e.g., muscle synergies). motor unit(MU) decomposition is a technique based on the deconvolution of high-density EMGs (HD-EMG) in order to derive the activities of the corresponding motor neurons. This powerful yet very sensitive tool has seen some traction in human-machine interfacing (HMI) for rehabilitation. Here, we propose combining the synergy-inspired channel clustering in order to isolate the most prominent regions of EMG activation in each targeted degree of freedom (DoF) and thus cater to decomposition's sensitivity demands. Our assumption is that this will lead to a higher number of extracted MUs and consequently better motion estimation in HMIs. Indeed, in four subjects, we have shown a 69% average increase in the number of MUs when decomposition was done using muscle-synergy channel clustering. Consequently, all three of our kinematic estimators benefited from an increased pool of units, with the linear regressor showing the greatest improvement once compared to, the artificial neural network and the gated recurrent unit, which had the overall best performance. Clinical Relevance- The results demonstrated in this work provide a new perspective on the online EMG-driven HMI systems that can be greatly beneficial in the rehabilitation of motor disorders.

Adaptation Strategies for Personalized Gait Neuroprosthetics

Personalization of gait neuroprosthetics is paramount to ensure their efficacy for users, who experience severe limitations in mobility without an assistive device. Our goal is to develop assistive devices that collaborate with and are tailored to their users, while allowing them to use as much of their existing capabilities as possible. Currently, personalization of devices is challenging, and technological advances are required to achieve this goal. Therefore, this paper presents an overview of challenges and research directions regarding an interface with the peripheral nervous system, an interface with the central nervous system, and the requirements of interface computing architectures. The interface should be modular and adaptable, such that it can provide assistance where it is needed. Novel data processing technology should be developed to allow for real-time processing while accounting for signal variations in the human. Personalized biomechanical models and simulation techniques should be developed to predict assisted walking motions and interactions between the user and the device. Furthermore, the advantages of interfacing with both the brain and the spinal cord or the periphery should be further explored. Technological advances of interface computing architecture should focus on learning on the chip to achieve further personalization. Furthermore, energy consumption should be low to allow for longer use of the neuroprosthesis. In-memory processing combined with resistive random access memory is a promising technology for both. This paper discusses the aforementioned aspects to highlight new directions for future research in gait neuroprosthetics.

Decoding Muscle Force From Motor Unit Firings Using Encoder-Decoder Networks

Appropriate interpretation of motor unit (MU) activities after surface EMG (sEMG) decomposition is a key factor to decode motor intentions in a noninvasive and physiologically meaningful way. However, there are great challenges due to the difficulty in cross-trial MU tracking and unavoidable loss of partial MU information resulting from incomplete decomposition. In light of these issues, this study presents a novel framework for interpreting MU activities and applies it to decode muscle force. The resulting MUs were clustered and classified into different categories by characterizing their spatially distributed firing waveforms. The process served as a general MU tracking method. On this basis, after transferring the MU firing trains to twitch force trains by a twitch force model, a deep network was designed to predict the normalized force. In addition, MU category distribution was examined to calibrate the actual force level, while functions of some unavailable MUs were compensated. To investigate the effectiveness of this framework, high-density sEMG signals were recorded using an 8×8 electrode array from the abductor pollicis brevis muscles of eight subjects, while thumb abduction force was measured. The proposed method outperformed three common methods ( ) yielding the lowest root mean square deviation of 6.68% ± 1.29% and the highest fitness ( R² ) of 0.94 ± 0.04 between the predicted force and the actual force. This study offers a valuable, computational solution for interpreting individual MU activities, and its effectiveness was confirmed in muscle force estimation.

Highly accurate real-time decomposition of single channel intramuscular EMG

Real-time intramuscular electromyography (iEMG) decomposition, as an identification procedure of individual motor neuron (MN) discharge timings from a streaming iEMG recording, has the potential to be used in human-machine interfacing. However, for these applications, the decomposition accuracy and speed of current approaches need to be improved.

METHODS

In our previous work, a real-time decomposition algorithm based on a Hidden Markov Model of EMG, using GPU-implemented Bayesian filter to estimate the spike trains of motor units (MU) and their action potentials (MUAPs), was proposed. In this paper, a substantially extended version of this algorithm that boosts the accuracy while maintaining real-time implementation, is introduced. Specifically, multiple heuristics that aim at resolving the problems leading to performance degradation, are applied to the original model. In addition, the recursive maximum likelihood (RML) estimator previously used to estimate the statistical parameters of the spike trains, is replaced by a linear regression (LR) estimator, which is computationally more efficient, in order to ensure real-time decomposition with the new heuristics.

RESULTS

The algorithm was validated using twenty-one experimental iEMG signals acquired from the tibialis anterior muscle of five subjects by fine wire electrodes. All signals were decomposed in real time. The decomposition accuracy depended on the level of muscle activation and was >90% when less than 10 MUs were identified, substantially exceeding previous real-time results.

CONCLUSION

Single channel iEMG signals can be very accurately decomposed in real time with the proposed algorithm.

SIGNIFICANCE

The proposed highly accurate algorithm for single-channel iEMG decomposition has the potential of providing neural information on motor tasks for human interfacing.

FS-HGR: Few-Shot Learning for Hand Gesture Recognition via Electromyography

This work is motivated by the recent advances in Deep Neural Networks (DNNs) and their widespread applications in human-machine interfaces. DNNs have been recently used for detecting the intended hand gesture through the processing of surface electromyogram (sEMG) signals. Objective: Although DNNs have shown superior accuracy compared to conventional methods when large amounts of data are available for training, their performance substantially decreases when data are limited. Collecting large datasets for training may be feasible in research laboratories, but it is not a practical approach for real-life applications. The main objective of this work is to design a modern DNN-based gesture detection model that relies on minimal training data while providing high accuracy. Methods: We propose the novel Few-Shot learning- Hand Gesture Recognition (FS-HGR) architecture. Few-shot learning is a variant of domain adaptation with the goal of inferring the required output based on just one or a few training observations. The proposed FS-HGR generalizes after seeing very few observations from each class by combining temporal convolutions with attention mechanisms. This allows the meta-learner to aggregate contextual information from experience and to pinpoint specific pieces of information within its available set of inputs. Data Source & Summary of Results: The performance of FS-HGR was tested on the second and fifth Ninapro databases, referred to as the DB2 and DB5, respectively. The DB2 consists of 50 gestures (rest included) from 40 healthy subjects. The Ninapro DB5 contains data from 10 healthy participants performing a total of 53 different gestures (rest included). The proposed approach for the Ninapro DB2 led to 85.94% classification accuracy on new repetitions with few-shot observation (5-way 5-shot), 81.29% accuracy on new subjects with few-shot observation (5-way 5-shot), and 73.36% accuracy on new gestures with few-shot observation (5-way 5-shot). Moreover, the proposed approach for the Ninapro DB5 led to 64.65% classification accuracy on new subjects with few-shot observation (5-way 5-shot).

A convolutional neural network to identify motor units from high-density surface electromyography signals in real time

OBJECTIVES

This paper aims to investigate the feasibility and the validity of applying deep convolutional neural networks (CNN) to identify motor unit (MU) spike trains and estimate the neural drive to muscles from high-density electromyography (HD-EMG) signals in real time. Two distinct deep CNNs are compared with the convolution kernel compensation (CKC) algorithm using simulated and experimentally recorded signals. The effects of window size and step size of the input HD-EMG signals are also investigated.

APPROACH

The MU spike trains were first identified with the CKC algorithm. The HD-EMG signals and spike trains were used to train the deep CNN. Then, the deep CNN decomposed the HD-EMG signals into MU discharge times in real time. Two CNN approaches are compared with the CKC: 1) multiple single-output deep CNN (SO-DCNN) with one MU decomposed per network, and 2) one multiple-output deep CNN (MO-DCNN) to decompose all MUs (up to 23) with one network.

MAIN RESULTS

The MO-DCNN outperformed the SO-DCNN in terms of training time (3.2 to 21.4 s/epoch vs. 6.5 to 47.8 s/epoch, respectively) and prediction time (0.04 vs. 0.27 s/sample, respectively). The optimal window size and step size for MO-DCNN were 120 and 20 data points, respectively. It results in sensitivity of 98% and 85% with simulated and experimentally recorded HD-EMG signals, respectively. There is a high cross-correlation coefficient between the neural drive estimated with CKC and that estimated with MO-DCNN (range of r-value across conditions: 0.88-0.95).

SIGNIFICANCE

We demonstrate the feasibility and the validity of using deep CNN to accurately identify MU activity from HD-EMG with a latency lower than 80 ms, which falls within the lower bound of the human electromechanical delay. This method opens many opportunities for using the neural drive to interface humans with assistive devices.