The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges

Revolutionizing motor dysfunction treatment: A novel closed-loop electrical stimulator guided by multiple motor tasks with predictive control

Functional electrical stimulation (FES) has been demonstrated as a viable method for addressing motor dysfunction in individuals affected by stroke, spinal cord injury, and other etiologies. By eliciting muscle contractions to facilitate joint movements, FES plays a crucial role in fostering the restoration of motor function compromised nervous system. In response to the challenge of muscle fatigue associated with conventional FES protocols, a novel biofeedback electrical stimulator incorporating multi-motor tasks and predictive control algorithms has been developed to enable adaptive modulation of stimulation parameters. The study initially establishes a Hammerstein model for the stimulated muscle group, representing a time-varying relationship between the stimulation pulse width and the root mean square (RMS) of the surface electromyography (sEMG). An online parameter identification algorithm utilizing recursive least squares is employed to estimate the time-varying parameters of the Hammerstein model. Predictive control is then implemented through feedback corrections based on the comparison between predicted and actual outputs, guided by an optimization objective function. The integration of predictive control and roll optimization enables closed-loop control of muscle stimulation. The motor training tasks of elbow flexion and extension, wrist flexion and extension, and five-finger grasping were selected for experimental validation. The results indicate that the model parameters were accurately identified, with a RMS error of 3.83 % between actual and predicted values. Furthermore, the predictive control algorithm, based on the motor tasks, effectively adjusted the stimulus parameters to ensure that the stimulated muscle groups can achieve the desired sEMG characteristic trajectory. The biofeedback electrical stimulator that was developed has the potential to assist patients experiencing motor dysfunction in achieving the appropriate joint movements. This research provides a foundation for a novel intelligent electrical stimulation model.

Targeted muscle reinnervation in bionic upper limb reconstruction: current status and future directions

Selective nerve transfers are used in the setting of upper limb amputation to improve myoelectric prosthesis control. This surgical concept is referred to as targeted muscle reinnervation (TMR) and describes the rerouting of the major nerves of the arm onto the motor branches of the residual limb musculature. Aside from providing additional myosignals for prosthetic control, TMR can treat and prevent neuroma pain and possibly also phantom limb pain. This article reviews the history and current applications of TMR in upper limb amputation, with a focus on practical considerations. It further explores and identifies technological innovations to improve the man-machine interface in amputation care, particularly regarding implantable interfaces, such as muscle electrodes and osseointegration. Finally, future clinical directions and possible scientific avenues in this field are presented and critically discussed.

Towards Efficient Neural Decoder for Dexterous Finger Force Predictions

OBJECTIVE

Dexterous control of robot hands requires a robust neural-machine interface capable of accurately decoding multiple finger movements. Existing studies primarily focus on single-finger movement or rely heavily on multi-finger data for decoder training, which requires large datasets and high computation demand. In this study, we investigated the feasibility of using limited single-finger surface electromyogram (sEMG) data to train a neural decoder capable of predicting the forces of unseen multi-finger combinations.

METHODS

We developed a deep forest-based neural decoder to concurrently predict the extension and flexion forces of three fingers (index, middle, and ring-pinky). We trained the model using varying amounts of high-density EMG data in a limited condition (i.e., single-finger data).

RESULTS

We showed that the deep forest decoder could achieve consistently commendable performance with 7.0% of force prediction errors and R2 value of 0.874, significantly surpassing the conventional EMG amplitude method and convolutional neural network approach. However, the deep forest decoder accuracy degraded when a smaller amount of data was used for training and when the testing data became noisy.

CONCLUSION

The deep forest decoder shows accurate performance in multi-finger force prediction tasks. The efficiency aspect of the deep forest lies in the short training time and small volume of training data, which are two critical factors in current neural decoding applications.

SIGNIFICANCE

This study offers insights into efficient and accurate neural decoder training for advanced robotic hand control, which has the potential for real-life applications during human-machine interactions.

Decomposition strategy for surface EMG with few channels: a simulation study

Objective.In the specific use of electromyogram (EMG) driven prosthetics, the user's disability reduces the space available for the electrode array. We propose a framework for EMG decomposition adapted to the condition of a few channels (less than 30 observations), which can elevate the potential of prosthetics in terms of cost and applicability.Approach.The new framework contains a peel-off approach, a refining strategy for motor unit (MU) spike train and MU action potential and a re-subtracting strategy to adapt the framework to few channels environments. Simulated EMG signals were generated to test the framework. In addition, we quantify and analyze the effect of strategies used in the framework.Main results.The results show that the new algorithm has an average improvement of 19.97% in the number of MUs identified compared to the control algorithm. Quantitative analysis of the usage strategies shows that the re-subtracting and refining strategies can effectively improve the performance of the framework under the condition of few channels.Significance.These prove that the new framework can be applied to few channel conditions, providing a optimization space for neural interface design in cost and user adaptation.

Across Sessions and Subjects Domain Adaptation for Building Robust Myoelectric Interface

Gesture interaction via surface electromyography (sEMG) signal is a promising approach for advanced human-computer interaction systems. However, improving the performance of the myoelectric interface is challenging due to the domain shift caused by the signal's inherent variability. To enhance the interface's robustness, we propose a novel adaptive information fusion neural network (AIFNN) framework, which could effectively reduce the effects of multiple scenarios. Specifically, domain adversarial training is established to inhibit the shared network's weights from exploiting domain-specific representation, thus allowing for the extraction of domain-invariant features. Effectively, classification loss, domain diversence loss and domain discrimination loss are employed, which improve classification performance while reduce distribution mismatches between the two domains. To simulate the application of myoelectric interface, experiments were carried out involving three scenarios (intra-session, inter-session and inter-subject scenarios). Ten non-disabled subjects were recruited to perform sixteen gestures for ten consecutive days. The experimental results indicated that the performance of AIFNN was better than two other state-of-the-art transfer learning approaches, namely fine-tuning (FT) and domain adversarial network (DANN). This study demonstrates the capability of AIFNN to maintain robustness over time and generalize across users in practical myoelectric interface implementations. These findings could serve as a foundation for future deployments.

Simultaneous Three-Degrees-of-Freedom Prosthetic Control Based on Linear Regression and Closed-Loop Training Protocol

Machine learning-based controllers of prostheses using electromyographic signals have become very popular in the last decade. The regression approach allows a simultaneous and proportional control of the intended movement in a more natural way than the classification approach, where the number of movements is discrete by definition. However, it is not common to find regression-based controllers working for more than two degrees of freedom at the same time. In this paper, we present the application of the adaptive linear regressor in a relatively low-dimensional feature space with only eight sensors to the problem of a simultaneous and proportional control of three degrees of freedom (left-right, up-down and open-close hand movements). We show that a key element usually overlooked in the learning process of the regressor is the training paradigm. We propose a closed-loop procedure, where the human learns how to improve the quality of the generated EMG signals, helping also to obtain a better controller. We apply it to 10 healthy and 3 limb-deficient subjects. Results show that the combination of the multidimensional targets and the open-loop training protocol significantly improve the performance, increasing the average completion rate from 53% to 65% for the most complicated case of simultaneously controlling the three degrees of freedom.

Online Muscle Activation Onset Detection Using Likelihood of Conditional Heteroskedasticity of Electromyography Signals

Surface electromyography (sEMG) signals are crucial in developing human-machine interfaces, as they contain rich information about human neuromuscular activities.

OBJECTIVE

The real-time, accurate detection of muscle activation onset (MAO) is significant for EMG-triggered control strategies in embedded applications like prostheses and exoskeletons.

METHODS

This paper investigates sEMG signals using the generalized autoregressive conditional heteroskedasticity (GARCH) model, focusing on variance. A novel feature, the likelihood of conditional heteroskedasticity (LCH) extracted from the maximum likelihood estimation of GARCH parameters, is proposed. This feature effectively distinguishes signal from noise based on heteroskedasticity, allowing for the detection of MAO through the LCH feature and a basic threshold classifier. For online calculation, the model parameter estimation is simplified, enabling direct calculation of the LCH value using fixed parameters.

RESULTS

The proposed method was validated on two open-source datasets and demonstrated superior performance over existing methods. The mean absolute error of onset detection, compared with visual detection results, is approximately 65 ms under online conditions, showcasing high accuracy, universality, and noise insensitivity.

CONCLUSION

The results indicate that the proposed method using the LCH feature from the GARCH model is highly effective for real-time detection of muscle activation onset in sEMG signals.

SIGNIFICANCE

This novel approach shows great potential and possibility for real-world applications, reflecting its superior performance in accuracy, universality, and insensitivity to noise.

A survey on the state of the art of force myography technique (FMG): analysis and assessment

Precise feedback assures precise control commands especially for assistive or rehabilitation devices. Biofeedback systems integrated with assistive or rehabilitative robotic exoskeletons tend to increase its performance and effectiveness. Therefore, there has been plenty of research in the field of biofeedback covering different aspects such as signal acquisition, conditioning, feature extraction and integration with the control system. Among several types of biofeedback systems, Force myography (FMG) technique is a promising one in terms of affordability, high classification accuracies, ease to use, and low computational cost. Compared to traditional biofeedback systems such as electromyography (EMG) which offers some invasive techniques, FMG offers a completely non-invasive solution with much less effort for preprocessing with high accuracies. This work covers the whole aspects of FMG technique in terms of signal acquisition, feature extraction, signal processing, developing the machine learning model, evaluating tools for the performance of the model. Stating the difference between real-time and offline assessment, also highlighting the main uncovered points for further study, and thus enhancing the development of this technique.

Study on Gesture Recognition Method with Two-Stream Residual Network Fusing sEMG Signals and Acceleration Signals

Currently, surface EMG signals have a wide range of applications in human-computer interaction systems. However, selecting features for gesture recognition models based on traditional machine learning can be challenging and may not yield satisfactory results. Considering the strong nonlinear generalization ability of neural networks, this paper proposes a two-stream residual network model with an attention mechanism for gesture recognition. One branch processes surface EMG signals, while the other processes hand acceleration signals. Segmented networks are utilized to fully extract the physiological and kinematic features of the hand. To enhance the model's capacity to learn crucial information, we introduce an attention mechanism after global average pooling. This mechanism strengthens relevant features and weakens irrelevant ones. Finally, the deep features obtained from the two branches of learning are fused to further improve the accuracy of multi-gesture recognition. The experiments conducted on the NinaPro DB2 public dataset resulted in a recognition accuracy of 88.25% for 49 gestures. This demonstrates that our network model can effectively capture gesture features, enhancing accuracy and robustness across various gestures. This approach to multi-source information fusion is expected to provide more accurate and real-time commands for exoskeleton robots and myoelectric prosthetic control systems, thereby enhancing the user experience and the naturalness of robot operation.

Myoelectric control and virtual reality to enhance motor rehabilitation after stroke

Effective upper-limb rehabilitation for severely impaired stroke survivors is still missing. Recent studies endorse novel motor rehabilitation approaches such as robotic exoskeletons and virtual reality systems to restore the function of the paretic limb of stroke survivors. However, the optimal way to promote the functional reorganization of the central nervous system after a stroke has yet to be uncovered. Electromyographic (EMG) signals have been employed for prosthetic control, but their application to rehabilitation has been limited. Here we propose a novel approach to promote the reorganization of pathological muscle activation patterns and enhance upper-limb motor recovery in stroke survivors by using an EMG-controlled interface to provide personalized assistance while performing movements in virtual reality (VR). We suggest that altering the visual feedback to improve motor performance in VR, thereby reducing the effect of deviations of the actual, dysfunctional muscle patterns from the functional ones, will actively engage patients in motor learning and facilitate the restoration of functional muscle patterns. An EMG-controlled VR interface may facilitate effective rehabilitation by targeting specific changes in the structure of muscle synergies and in their activations that emerged after a stroke-offering the possibility to provide rehabilitation therapies addressing specific individual impairments.

Classification of hand and wrist movements via surface electromyogram using the random convolutional kernels transform

Prosthetic devices are vital for enhancing personal autonomy and the quality of life for amputees. However, the rejection rate for electric upper-limb prostheses remains high at around 30%, often due to issues like functionality, control, reliability, and cost. Thus, developing reliable, robust, and cost-effective human-machine interfaces is crucial for user acceptance. Machine learning algorithms using Surface Electromyography (sEMG) signal classification hold promise for natural prosthetic control. This study aims to enhance hand and wrist movement classification using sEMG signals, treated as time series data. A novel approach is employed, combining a variation of the Random Convolutional Kernel Transform (ROCKET) for feature extraction with a cross-validation ridge classifier. Traditionally, achieving high accuracy in time series classification required complex, computationally intensive methods. However, recent advances show that simple linear classifiers combined with ROCKET can achieve state-of-the-art accuracy with reduced computational complexity. The algorithm was tested on the UCI sEMG hand movement dataset, as well as on the Ninapro DB5 and DB7 datasets. We demonstrate how the proposed approach delivers high discrimination accuracy with minimal parameter tuning requirements, offering a promising solution to improve prosthetic control and user satisfaction.

Optical Myography-Based Sensing Methodology of Application of Random Loads to Muscles during Hand-Gripping Training

Hand-gripping training is important for improving the fundamental functions of human physical activity. Bernstein's idea of "repetition without repetition" suggests that motor control function should be trained under changing states. The randomness level of load should be visualized for self-administered screening when repeating various training tasks under changing states. This study aims to develop a sensing methodology of random loads applied to both the agonist and antagonist skeletal muscles when performing physical tasks. We assumed that the time-variability and periodicity of the applied load appear in the time-series feature of muscle deformation data. In the experiment, 14 participants conducted the gripping tasks with a gripper, ball, balloon, Palm clenching, and paper. Crumpling pieces of paper (paper exercise) involves randomness because the resistance force of the paper changes depending on the shape and layers of the paper. Optical myography during gripping tasks was measured, and time-series features were analyzed. As a result, our system could detect the random movement of muscles during training.

Motor unit activity and synaptic inputs to motoneurons in the caudal part of the injured spinal cord

Spinal cord injury (SCI) disrupts neuronal function below the lesion epicenter, causing disuse muscle atrophy. We investigated motor unit (MU) activity and synaptic inputs to motoneurons in the caudal region of the injured spinal cord. Participants with C4-C7 cervical injuries were studied. The extensor digitorum communis (EDC) muscle, which is mainly innervated by C8, was assessed for disuse muscle atrophy. Using advanced electromyography and signal-processing techniques, we examined the concurrent activation of a substantial population of MUs during force-tracking tasks. We found that in participants with SCI (n = 9), both MU discharge rates and the amplitudes of MU action potentials were significantly lower than in controls (n = 9). After SCI, MUs were recruited in a limited force range as the strength of muscle contractions increased, implying a disruption in the orderly MU recruitment pattern. Coherence analysis revealed reduced synaptic inputs to motoneurons in the delta band (0.5-5 Hz) for participants with SCI, suggesting diminished common synaptic inputs to the EDC muscle. In addition, participants with SCI exhibited greater muscle force variability. Using principal component analysis on low-frequency MU discharge rates, we found that the first common component (FCC) captured the most discharge variability in participants with SCI. The coefficients of variation (CV) of the FCC correlated with force signal CVs, suggesting force variability mainly results from common synaptic inputs to the EDC muscle after SCI. These results advance our understanding of the neurophysiology of disuse muscle atrophy in human SCI, paving the way for therapeutic interventions to restore muscle function.NEW & NOTEWORTHY This study analyzed motor unit (MU) function below the lesion epicenter in patients with spinal cord injury (SCI). We found reduced MU discharge rates and action potential amplitudes in participants with SCI compared with controls. The strength of common synaptic inputs to motoneurons was reduced in patients with SCI, with increased force variability primarily due to low-frequency oscillations of common inputs. This study enhances understanding of neurophysiological and behavioral changes in disuse muscle atrophy post-SCI.

Neuromorphic hardware for somatosensory neuroprostheses

In individuals with sensory-motor impairments, missing limb functions can be restored using neuroprosthetic devices that directly interface with the nervous system. However, restoring the natural tactile experience through electrical neural stimulation requires complex encoding strategies. Indeed, they are presently limited in effectively conveying or restoring tactile sensations by bandwidth constraints. Neuromorphic technology, which mimics the natural behavior of neurons and synapses, holds promise for replicating the encoding of natural touch, potentially informing neurostimulation design. In this perspective, we propose that incorporating neuromorphic technologies into neuroprostheses could be an effective approach for developing more natural human-machine interfaces, potentially leading to advancements in device performance, acceptability, and embeddability. We also highlight ongoing challenges and the required actions to facilitate the future integration of these advanced technologies.

Training prosthesis users to switch between modes of a multi-articulating prosthetic hand

PURPOSE

Producing triggers to switch between modes of myoelectric prosthetic hands has proven to be difficult. We evaluated whether digital training methods were feasible in individuals with an upper limb defect (ULD), whether myosignals in these individuals differ from those of non-impaired individuals and whether acquired skills transfer to prosthesis use.

MATERIALS AND METHODS

Two groups participated in a 9-day pre-test-post-test design study with seven 45-minute training sessions. One group trained using a serious game, the other with their myosignals digitally displayed. Both groups also trained using a prosthesis. The pre- and post-tests consisted of an adapted Clothespin Relocation Test and the spherical subset of the Southampton Hand Assessment Procedure. After the post-test, the System Usability Scale (SUS) was administered. Clinically relevant performance measures and myosignal features were analysed.

RESULTS

Four individuals with a ULD participated. SUS-scores deemed both training methods feasible. Three participants produced only a few correct triggers. Myosignals features indicated larger variability for individuals with a ULD compared to non-impaired individuals (previously published data [1]). Three participants indicated transfer of skill.

CONCLUSIONS

Even though both training methods were deemed feasible and most participants showed transfer, seven training sessions were insufficient to learn reliable switching behaviour.Trial registration: The study was approved by the medical ethics committee of the University Medical Center Groningen (METc 2018.268).Implications for rehabilitationSwitching between pre-programmed modes of a myoelectric prosthetic hand can be learned, however it does require training.Serious games can be considered useful training tools for trigger production in early phases of myoelectric prosthesis control training.In order to evoke transfer of skill from training to daily life both task-specificity and focus of attention during training should be taken into account.

Single-Channel sEMG-Based Estimation of Knee Joint Angle Using a Decomposition Algorithm With a State-Space Model

Accurate human motion estimation is crucial for effective and safe human-robot interaction when using robotic devices for rehabilitation or performance enhancement. Although surface electromyography (sEMG) signals have been widely used to estimate human movements, conventional sEMG-based methods, which need sEMG signals measured from multiple relevant muscles, are usually subject to some limitations, including interference between sEMG sensors and wearable robots/environment, complicated calibration, as well as discomfort during long-term routine use. Few methods have been proposed to deal with these limitations by using single-channel sEMG (i.e., reducing the sEMG sensors as much as possible). The main challenge for developing single-channel sEMG-based estimation methods is that high estimation accuracy is difficult to be guaranteed. To address this problem, we proposed an sEMG-driven state-space model combined with an sEMG decomposition algorithm to improve the accuracy of knee joint movement estimation based on single-channel sEMG signals measured from gastrocnemius. The effectiveness of the method was evaluated via both single- and multi-speed walking experiments with seven and four healthy subjects, respectively. The results showed that the normal root-mean-squared error of the estimated knee joint angle using the method could be limited to 15%. Moreover, this method is robust with respect to variations in walking speeds. The estimation performance of this method was basically comparable to that of state-of-the-art studies using multi-channel sEMG.

TraHGR: Transformer for Hand Gesture Recognition via Electromyography

Deep learning-based Hand Gesture Recognition (HGR) via surface Electromyogram (sEMG) signals have recently shown considerable potential for development of advanced myoelectric-controlled prosthesis. Although deep learning techniques can improve HGR accuracy compared to their classical counterparts, classifying hand movements based on sparse multichannel sEMG signals is still a challenging task. Furthermore, existing deep learning approaches, typically, include only one model as such can hardly extract representative features. In this paper, we aim to address this challenge by capitalizing on the recent advances in hybrid models and transformers. In other words, we propose a hybrid framework based on the transformer architecture, which is a relatively new and revolutionizing deep learning model. The proposed hybrid architecture, referred to as the Transformer for Hand Gesture Recognition (TraHGR), consists of two parallel paths followed by a linear layer that acts as a fusion center to integrate the advantage of each module. We evaluated the proposed architecture TraHGR based on the commonly used second Ninapro dataset, referred to as the DB2. The sEMG signals in the DB2 dataset are measured in real-life conditions from 40 healthy users, each performing 49 gestures. We have conducted an extensive set of experiments to test and validate the proposed TraHGR architecture, and compare its achievable accuracy with several recently proposed HGR classification algorithms over the same dataset. We have also compared the results of the proposed TraHGR architecture with each individual path and demonstrated the distinguishing power of the proposed hybrid architecture. The recognition accuracies of the proposed TraHGR architecture for the window of size 200ms and step size of 100ms are 86.00%, 88.72%, 81.27%, and 93.74%, which are 2.30%, 4.93%, 8.65%, and 4.20% higher than the state-of-the-art performance for DB2 (49 gestures), DB2-B (17 gestures), DB2-C (23 gestures), and DB2-D (9 gestures), respectively.

The second clinical study investigating the surgical method for the kineticomyographic control implementation of the bionic hand

In 2018, during our first clinical study on the kineticomyographic (KMG)-controlled bionic hand, we implanted three magnetic tags inside the musculotendinous junction of three paired extensor-flexor transferred tendons. However, the post-operative tissue adhesions affected the independent movements of the implanted tags and consequently the distinct patterns of the obtained signals. To overcome this issue, we modified our surgical procedure from a one-stage tendon transfer to a two-stage. During the first surgery, we created three tunnels using silicon rods for the smooth tendon gliding. In the second stage, we transferred the same three pairs of the forearm agonist-antagonist tendons through the tunnels and implanted the magnetic tags inside the musculotendinous junction. Compared to our prior clinical investigation, fluoroscopy and ultrasound evaluations revealed that the surgical modification in the current study yielded more pronounced independent movements in two specific magnetic tags associated with fingers (maximum 5.7 mm in the first trial vs. 28 mm in the recent trial with grasp and release) and thumb (maximum 3.2 mm in the first trial vs. 9 mm in the current trial with thumb flexion-extension). Furthermore, we observed that utilizing the flexor digitorum superficialis (FDS) tendons for the flexor component in finger and thumb tendon transfer resulted in more independent movements of the implanted tags, compared with the flexor digitorum profundus (FDP) in the prior research. This study can help us plan for our future five-channel bionic limb design by identifying the gestures with the most significant independent tag displacement.

Extracting Individual Muscle Drive and Activity From High-Density Surface Electromyography Signals Based on the Center of Gravity of Motor Unit

Neural interfacing has played an essential role in advancing our understanding of fundamental movement neurophysiology and the development of human-machine interface. However, direct neural interfaces from brain and nerve recording are currently limited in clinical areas for their invasiveness and high selectivity. Here, we applied the surface electromyogram (EMG) in studying the neural control of movement and proposed a new non-invasive way of extracting neural drive to individual muscles. Sixteen subjects performed isometric contractions to complete six hand tasks. High-density surface EMG signals (256 channels in total) recorded from the forearm muscles were decomposed into motor unit firing trains. The location of each decomposed motor unit was represented by its center of gravity and was put into clustering for distinct muscle regions. All the motor units in the same cluster served as a muscle-specific motor pool from which individual muscle drive could be extracted directly. Moreover, we cross-validated the self-clustered muscle regions by magnetic resonance imaging (MRI) recorded from the subjects' forearms. All motor units that fall within the MRI region are considered correctly clustered. We achieved a clustering accuracy of 95.72% ± 4.01% for all subjects. We provided a new framework for collecting experimental muscle-specific drives and generalized the way of surface electrode placement without prior knowledge of the targeting muscle architecture.

Random forest-based simultaneous and proportional myoelectric control system for finger movements

A classification scheme for myoelectric control systems (MCS) cannot mimic complex hand movements. This paper presents simultaneous and proportional MCS by estimating the angles of fourteen finger joints using time-domain feature extraction and random forest. The experimental results show that the best feature was the root mean square (RMS). Furthermore, the random forest attained an average coefficient of determination (R2) of 0.85 compared to other regressors which perform below 0.75. The ANOVA tests indicated that the performance of the proposed system was significantly different. Therefore, the proposed system will be the best option for real-time MCS applications in the future.

Consistent control information driven musculoskeletal model for multiday myoelectric control

Objective.Musculoskeletal model (MM)-based myoelectric interface has aroused great interest in human-machine interaction. However, the performance of electromyography (EMG)-driven MM in long-term use would be degraded owing to the inherent non-stationary characteristics of EMG signals. Here, to improve the estimation performance without retraining, we proposed a consistent muscle excitation extraction approach based on an improved non-negative matrix factorization (NMF) algorithm for MM when applied to simultaneous hand and wrist movement prediction.Approach.We added constraints andL2-norm regularization terms to the objective function of classic NMF regarding muscle weighting matrix and time-varying profiles, through which stable muscle synergies across days were identified. The resultant profiles of these synergies were then used to drive the MM. Both offline and online experiments were conducted to evaluate the performance of the proposed method in inter-day scenarios.Main results.The results demonstrated significantly better and more robust performance over several competitive methods in inter-day experiments, including machine learning methods, EMG envelope-driven MM, and classic NMF-based MM. Furthermore, the analysis of control information on different days revealed the effectiveness of the proposed method in obtaining consistent muscle excitations.Significance.The outcomes potentially provide a novel and promising pathway for the robust and zero-retraining control of myoelectric interfaces.

Using principles of motor control to analyze performance of human machine interfaces

There have been significant advances in biosignal extraction techniques to drive external biomechatronic devices or to use as inputs to sophisticated human machine interfaces. The control signals are typically derived from biological signals such as myoelectric measurements made either from the surface of the skin or subcutaneously. Other biosignal sensing modalities are emerging. With improvements in sensing modalities and control algorithms, it is becoming possible to robustly control the target position of an end-effector. It remains largely unknown to what extent these improvements can lead to naturalistic human-like movement. In this paper, we sought to answer this question. We utilized a sensing paradigm called sonomyography based on continuous ultrasound imaging of forearm muscles. Unlike myoelectric control strategies which measure electrical activation and use the extracted signals to determine the velocity of an end-effector; sonomyography measures muscle deformation directly with ultrasound and uses the extracted signals to proportionally control the position of an end-effector. Previously, we showed that users were able to accurately and precisely perform a virtual target acquisition task using sonomyography. In this work, we investigate the time course of the control trajectories derived from sonomyography. We show that the time course of the sonomyography-derived trajectories that users take to reach virtual targets reflect the trajectories shown to be typical for kinematic characteristics observed in biological limbs. Specifically, during a target acquisition task, the velocity profiles followed a minimum jerk trajectory shown for point-to-point arm reaching movements, with similar time to target. In addition, the trajectories based on ultrasound imaging result in a systematic delay and scaling of peak movement velocity as the movement distance increased. We believe this is the first evaluation of similarities in control policies in coordinated movements in jointed limbs, and those based on position control signals extracted at the individual muscle level. These results have strong implications for the future development of control paradigms for assistive technologies.

Encoding force modulation in two electrotactile feedback parameters strengthens sensory integration according to maximum likelihood estimation

Bidirectional human-machine interfaces involve commands from the central nervous system to an external device and feedback characterizing device state. Such feedback may be elicited by electrical stimulation of somatosensory nerves, where a task-relevant variable is encoded in stimulation amplitude or frequency. Recently, concurrent modulation in amplitude and frequency (multimodal encoding) was proposed. We hypothesized that feedback with multimodal encoding may effectively be processed by the central nervous system as two independent inputs encoded in amplitude and frequency, respectively, thereby increasing state estimate quality in accordance with maximum-likelihood estimation. Using an adaptation paradigm, we tested this hypothesis during a grasp force matching task where subjects received electrotactile feedback encoding instantaneous force in amplitude, frequency, or both, in addition to their natural force feedback. The results showed that adaptations in grasp force with multimodal encoding could be accurately predicted as the integration of three independent inputs according to maximum-likelihood estimation: amplitude modulated electrotactile feedback, frequency modulated electrotactile feedback, and natural force feedback (r² = 0.73). These findings show that multimodal electrotactile feedback carries an intrinsic advantage for state estimation accuracy with respect to single-variable modulation and suggest that this scheme should be the preferred strategy for bidirectional human-machine interfaces with electrotactile feedback.

Recent trends and challenges of surface electromyography in prosthetic applications

Surface electromyography (sEMG) meets extensive applications in the field of prosthesis in the current period. The effectiveness of sEMG in prosthesis applications has been verified by numerous revolutionary developments and extensive research attempts. A large volume of research and literature works have explored and validated the vast use of these signals in prostheses as an assistive technology. The objective of this paper is to conduct a systematic review and offer a detailed overview of the work record in the prosthesis and myoelectric interfaces framework. This review utilized a systematic search strategy to identify published articles discussing the state-of-the-art applications of sEMG in prostheses (including upper limb prosthesis and lower limb prostheses). Relevant studies were identified using electronic databases such as PubMed, IEEE Explore, SCOPUS, ScienceDirect, Google Scholar and Web of Science. Out of 3791 studies retrieved from the databases, 188 articles were found to be potentially relevant (after screening of abstracts and application of inclusion-exclusion criteria) and included in this review. This review presents an investigative analysis of sEMG-based prosthetic applications to assist the readers in making further advancements in this field. It also discusses the fundamental advantages and disadvantages of using sEMG in prosthetic applications. It also includes some important guidelines to follow in order to improve the performance of sEMG-based prosthesis. The findings of this study support the widespread use of sEMG in prosthetics. It is concluded that sEMG-based prosthesis technology, still in its sprouting phase, requires significant explorations for further development. Supplementary investigations are necessary in the direction of making a seamless mechanism of biomechatronics for sEMG-based prosthesis by cohesive efforts of robotic researchers and biomedical engineers.

A Review of Myoelectric Control for Prosthetic Hand Manipulation

Myoelectric control for prosthetic hands is an important topic in the field of rehabilitation. Intuitive and intelligent myoelectric control can help amputees to regain upper limb function. However, current research efforts are primarily focused on developing rich myoelectric classifiers and biomimetic control methods, limiting prosthetic hand manipulation to simple grasping and releasing tasks, while rarely exploring complex daily tasks. In this article, we conduct a systematic review of recent achievements in two areas, namely, intention recognition research and control strategy research. Specifically, we focus on advanced methods for motion intention types, discrete motion classification, continuous motion estimation, unidirectional control, feedback control, and shared control. In addition, based on the above review, we analyze the challenges and opportunities for research directions of functionality-augmented prosthetic hands and user burden reduction, which can help overcome the limitations of current myoelectric control research and provide development prospects for future research.

Advances in EMG measurement techniques, analysis procedures, and the impact of muscle mechanics on future requirements for the methodology

Muscular coordination enables locomotion and interaction with the environment. For more than 50 years electromyography (EMG) has provided insights into the central nervous system control of individual muscles or muscle groups, enabling both fine and gross motor functions. This information is available either at individual motor units (Mus) level or on a more global level from the coordination of different muscles or muscle groups. In particular, non-invasive EMG methods such as surface EMG (sEMG) or, more recently, spatial mapping methods (High-Density EMG - HDsEMG) have found their place in research into biomechanics, sport and exercise, ergonomics, rehabilitation, diagnostics, and increasingly for the control of technical devices. With further technical advances and a growing understanding of the relationship between EMG and movement task execution, it is expected that with time, especially non-invasive EMG methods will become increasingly important in movement sciences. However, while the total number of publications per year on non-invasive EMG methods is growing exponentially, the number of publications on this topic in journals with a scope in movement sciences has stagnated in the last decade. This review paper contextualizes non-invasive EMG development over the last 50 years, highlighting methodological progress. Changes in research topics related to non-invasive EMG were identified. Today non-invasive EMG procedures are increasingly used to control technical devices, where muscle mechanics have a minor influence. In movement science, however, the effect of muscle mechanics on the EMG signal cannot be neglected. This explains why non-invasive EMG's relevance in movement sciences has not developed as expected.

Hand Force Estimation from Acoustic Myography Using Deep Wavelet Scattering Transform and Long Short-Term Memory

The ability to estimate user intention from surface electromyogram (sEMG) signals is a crucial aspect in the design of powered prosthetics. Recently, researchers have been using regression techniques to connect the user's intent, as expressed through sEMG signals, to the force applied at the fingertips in order to achieve a natural and accurate form of control. However, there are still challenges associated with processing sEMG signals that need to be overcome to allow for widespread and clinical implementation of upper limb prostheses. As a result, alternative modalities functioning as promising control signals have been proposed as source of control input rather than the sEMG, such as Acoustic Myography (AMG). In this study, six high sensitivity array microphones were used to acquire AMG signals, with custom-built 3D printed microphone housing. To tackle the challenge of extracting the relevant information from AMG signals, the Wavelet Scattering Transform (WST) was utilized. alongside a Long Short-Term Memory (LSTM) neural network model for predicting the force from the AMG. The subjects were asked to use a hand dynamometer to measure the changes in force and correlate that to the force predicted by using the AMG features. Seven subjects were recruited for data collection in this study, using hardware designed by the research team. the performance results showed that the WST-LSTM model can be robustly utilized across varying window sizes and testing schemes, to achieve average NRMSE results of approximately 8%. These pioneering results suggest that AMG signals can be utilized to reliably estimate the force levels that the muscles are applying.Clinical Relevance- This research presents a new method for controlling upper limb prostheses using Acoustic Myography (AMG) signals. A novel method mapping the AMG signals to force applied by the corresponding muscles is developed. The presented findings have the potential to lead to the development of more natural and accurate control of human-machine interfaces.

Efficient Self-Attention Model for Speech Recognition-Based Assistive Robots Control

Assistive robots are tools that people living with upper body disabilities can leverage to autonomously perform Activities of Daily Living (ADL). Unfortunately, conventional control methods still rely on low-dimensional, easy-to-implement interfaces such as joysticks that tend to be unintuitive and cumbersome to use. In contrast, vocal commands may represent a viable and intuitive alternative. This work represents an important step toward providing a viable vocal interface for people living with upper limb disabilities by proposing a novel lightweight vocal command recognition system. The proposed model leverages the MobileNet2 architecture, augmenting it with a novel approach to the self-attention mechanism, achieving a new state-of-the-art performance for Keyword Spotting (KWS) on the Google Speech Commands Dataset (GSCD). Moreover, this work presents a new dataset, referred to as the French Speech Commands Dataset (FSCD), comprising 4963 vocal command utterances. Using the GSCD as the source, we used Transfer Learning (TL) to adapt the model to this cross-language task. TL has been shown to significantly improve the model performance on the FSCD. The viability of the proposed approach is further demonstrated through real-life control of a robotic arm by four healthy participants using both the proposed vocal interface and a joystick.

The concepts of muscle activity generation driven by upper limb kinematics

BACKGROUND

The underlying motivation of this work is to demonstrate that artificial muscle activity of known and unknown motion can be generated based on motion parameters, such as angular position, acceleration, and velocity of each joint (or the end-effector instead), which are similarly represented in our brains. This model is motivated by the known motion planning process in the central nervous system. That process incorporates the current body state from sensory systems and previous experiences, which might be represented as pre-learned inverse dynamics that generate associated muscle activity.

METHODS

We develop a novel approach utilizing recurrent neural networks that are able to predict muscle activity of the upper limbs associated with complex 3D human arm motions. Therefore, motion parameters such as joint angle, velocity, acceleration, hand position, and orientation, serve as input for the models. In addition, these models are trained on multiple subjects (n=5 including , 3 male in the age of 26±2 years) and thus can generalize across individuals. In particular, we distinguish between a general model that has been trained on several subjects, a subject-specific model, and a specific fine-tuned model using a transfer learning approach to adapt the model to a new subject. Estimators such as mean square error MSE, correlation coefficient r, and coefficient of determination R² are used to evaluate the goodness of fit. We additionally assess performance by developing a new score called the zero-line score. The present approach was compared with multiple other architectures.

RESULTS

The presented approach predicts the muscle activity for previously through different subjects with remarkable high precision and generalizing nicely for new motions that have not been trained before. In an exhausting comparison, our recurrent network outperformed all other architectures. In addition, the high inter-subject variation of the recorded muscle activity was successfully handled using a transfer learning approach, resulting in a good fit for the muscle activity for a new subject.

CONCLUSIONS

The ability of this approach to efficiently predict muscle activity contributes to the fundamental understanding of motion control. Furthermore, this approach has great potential for use in rehabilitation contexts, both as a therapeutic approach and as an assistive device. The predicted muscle activity can be utilized to guide functional electrical stimulation, allowing specific muscles to be targeted and potentially improving overall rehabilitation outcomes.

Predicting Wrist Joint Angles from the Kinematics of the Arm: Application to the Control of Upper Limb Prostheses

Automation of wrist rotations in upper limb prostheses allows simplification of the human-machine interface, reducing the user's mental load and avoiding compensatory movements. This study explored the possibility of predicting wrist rotations in pick-and-place tasks based on kinematic information from the other arm joints. To do this, the position and orientation of the hand, forearm, arm, and back were recorded from five subjects during transport of a cylindrical and a spherical object between four different locations on a vertical shelf. The rotation angles in the arm joints were obtained from the records and used to train feed-forward neural networks (FFNNs) and time-delay neural networks (TDNNs) in order to predict wrist rotations (flexion/extension, abduction/adduction, and pronation/supination) based on the angles at the elbow and shoulder. Correlation coefficients between actual and predicted angles of 0.88 for the FFNN and 0.94 for the TDNN were obtained. These correlations improved when object information was added to the network or when it was trained separately for each object (0.94 for the FFNN, 0.96 for the TDNN). Similarly, it improved when the network was trained specifically for each subject. These results suggest that it would be feasible to reduce compensatory movements in prosthetic hands for specific tasks by using motorized wrists and automating their rotation based on kinematic information obtained with sensors appropriately positioned in the prosthesis and the subject's body.

Soft Robotics Enables Neuroprosthetic Hand Design

Development and implementation of neuroprosthetic hands is a multidisciplinary field at the interface between humans and artificial robotic systems, which aims at replacing the sensorimotor function of the upper-limb amputees as their own. Although prosthetic hand devices with myoelectric control can be dated back to more than 70 years ago, their applications with anthropomorphic robotic mechanisms and sensory feedback functions are still at a relatively preliminary and laboratory stage. Nevertheless, a recent series of proof-of-concept studies suggest that soft robotics technology may be promising and useful in alleviating the design complexity of the dexterous mechanism and integration difficulty of multifunctional artificial skins, in particular, in the context of personalized applications. Here, we review the evolution of neuroprosthetic hands with the emerging and cutting-edge soft robotics, covering the soft and anthropomorphic prosthetic hand design and relating bidirectional neural interactions with myoelectric control and sensory feedback. We further discuss future opportunities on revolutionized mechanisms, high-performance soft sensors, and compliant neural-interaction interfaces for the next generation of neuroprosthetic hands.

A myoelectric digital twin for fast and realistic modelling in deep learning

Muscle electrophysiology has emerged as a powerful tool to drive human machine interfaces, with many new recent applications outside the traditional clinical domains, such as robotics and virtual reality. However, more sophisticated, functional, and robust decoding algorithms are required to meet the fine control requirements of these applications. Deep learning has shown high potential in meeting these demands, but requires a large amount of high-quality annotated data, which is expensive and time-consuming to acquire. Data augmentation using simulations, a strategy applied in other deep learning applications, has never been attempted in electromyography due to the absence of computationally efficient models. We introduce a concept of Myoelectric Digital Twin - highly realistic and fast computational model tailored for the training of deep learning algorithms. It enables simulation of arbitrary large and perfectly annotated datasets of realistic electromyography signals, allowing new approaches to muscular signal decoding, accelerating the development of human-machine interfaces.

Human Arm Workout Classification by Arm Sleeve Device Based on Machine Learning Algorithms

Wearables have been applied in the field of fitness in recent years to monitor human muscles by recording electromyographic (EMG) signals. Understanding muscle activation during exercise routines allows strength athletes to achieve the best results. Hydrogels, which are widely used as wet electrodes in the fitness field, are not an option for wearable devices due to their characteristics of being disposable and skin-adhesion. Therefore, a lot of research has been conducted on the development of dry electrodes that can replace hydrogels. In this study, to make it wearable, neoprene was impregnated with high-purity SWCNTs to develop a dry electrode with less noise than hydrogel. Due to the impact of COVID-19, the demand for workouts to improve muscle strength, such as home gyms and personal trainers (PT), has increased. Although there are many studies related to aerobic exercise, there is a lack of wearable devices that can assist in improving muscle strength. This pilot study proposed the development of a wearable device in the form of an arm sleeve that can monitor muscle activity by recording EMG signals of the arm using nine textile-based sensors. In addition, some machine learning models were used to classify three arm target movements such as wrist curl, biceps curl, and dumbbell kickback from the EMG signals recorded by fiber-based sensors. The results obtained show that the EMG signal recorded by the proposed electrode contains less noise compared to that collected by the wet electrode. This was also evidenced by the high accuracy of the classification model used to classify the three arms workouts. This work classification device is an essential step towards wearable devices that can replace next-generation PT.

Clinical Features to Predict the Use of a sEMG Wearable Device (REMO®) for Hand Motor Training of Stroke Patients: A Cross-Sectional Cohort Study

After stroke, upper limb motor impairment is one of the most common consequences that compromises the level of the autonomy of patients. In a neurorehabilitation setting, the implementation of wearable sensors provides new possibilities for enhancing hand motor recovery. In our study, we tested an innovative wearable (REMO^®) that detected the residual surface-electromyography of forearm muscles to control a rehabilitative PC interface. The aim of this study was to define the clinical features of stroke survivors able to perform ten, five, or no hand movements for rehabilitation training. 117 stroke patients were tested: 65% of patients were able to control ten movements, 19% of patients could control nine to one movement, and 16% could control no movements. Results indicated that mild upper limb motor impairment (Fugl-Meyer Upper Extremity ≥ 18 points) predicted the control of ten movements and no flexor carpi muscle spasticity predicted the control of five movements. Finally, severe impairment of upper limb motor function (Fugl-Meyer Upper Extremity > 10 points) combined with no pain and no restrictions of upper limb joints predicted the control of at least one movement. In conclusion, the residual motor function, pain and joints restriction, and spasticity at the upper limb are the most important clinical features to use for a wearable REMO^® for hand rehabilitation training.

Bio-robotics research for non-invasive myoelectric neural interfaces for upper-limb prosthetic control: a 10-year perspective review

A decade ago, a group of researchers from academia and industry identified a dichotomy between the industrial and academic state-of-the-art in upper-limb prosthesis control, a widely used bio-robotics application. They proposed that four key technical challenges, if addressed, could bridge this gap and translate academic research into clinically and commercially viable products. These challenges are unintuitive control schemes, lack of sensory feedback, poor robustness and single sensor modality. Here, we provide a perspective review on the research effort that occurred in the last decade, aiming at addressing these challenges. In addition, we discuss three research areas essential to the recent development in upper-limb prosthetic control research but were not envisioned in the review 10 years ago: deep learning methods, surface electromyogram decomposition and open-source databases. To conclude the review, we provide an outlook into the near future of the research and development in upper-limb prosthetic control and beyond.

The multi-grip and standard myoelectric hand prosthesis compared: does the multi-grip hand live up to its promise?

BACKGROUND

Multi-grip myoelectric hand prostheses (MHPs), with five movable and jointed fingers, have been developed to increase functionality. However, literature comparing MHPs with standard myoelectric hand prostheses (SHPs) is limited and inconclusive. To establish whether MHPs increase functionality, we compared MHPs with SHPs on all categories of the International Classification of Functioning, Disability, and Health-model (ICF-model).

METHODS

MHP users (N = 14, 64.3% male, mean age = 48.6 years) performed physical measurements (i.e., Refined Clothespin Relocation Test (RCRT), Tray-test, Box and Blocks Test, Southampton Hand Assessment Procedure) with their MHP and an SHP to compare the joint angle coordination and functionality related to the ICF-categories 'Body Function' and 'Activities' (within-group comparisons). SHP users (N = 19, 68.4% male, mean age = 58.1 years) and MHP users completed questionnaires/scales (i.e., Orthotics and Prosthetics Users' Survey-The Upper Extremity Functional Status Survey /OPUS-UEFS, Trinity Amputation and Prosthesis Experience Scales for upper extremity/TAPES-Upper, Research and Development-36/RAND-36, EQ-5D-5L, visual analogue scale/VAS, the Dutch version of the Quebec User Evaluation of Satisfaction with assistive technology/D-Quest, patient-reported outcome measure to assess the preferred usage features of upper limb prostheses/PUF-ULP) to compare user experiences and quality of life in the ICF-categories 'Activities', 'Participation', and 'Environmental Factors' (between-group comparisons).

RESULTS

'Body Function' and 'Activities': nearly all users of MHPs had similar joint angle coordination patterns with an MHP as when they used an SHP. The RCRT in the upward direction was performed slower in the MHP condition compared to the SHP condition. No other differences in functionality were found. 'Participation': MHP users had a lower EQ-5D-5L utility score; experienced more pain or limitations due to pain (i.e., measured with the RAND-36). 'Environmental Factors': MHPs scored better than SHPs on the VAS-item holding/shaking hands. The SHP scored better than the MHP on five VAS-items (i.e., noise, grip force, vulnerability, putting clothes on, physical effort to control) and the PUF-ULP.

CONCLUSION

MHPs did not show relevant differences in outcomes compared to SHPs on any of the ICF-categories. This underlines the importance of carefully considering whether the MHP is the most suitable option for an individual taking into account the additional costs of MHPs.

Modulation of spinal circuits following phase-dependent electrical stimulation of afferent pathways

Objective.Peripheral electrical stimulation (PES) of afferent pathways is a tool commonly used to induce neural adaptations in some neural disorders such as pathological tremor or stroke. However, the neuromodulatory effects of stimulation interventions synchronized with physiological activity (closed-loop strategies) have been scarcely researched in the upper-limb. Here, the short-term spinal effects of a 20-minute stimulation protocol where afferent pathways were stimulated with a closed-loop strategy named selective and adaptive timely stimulation (SATS) were explored in 11 healthy subjects.Approach. SATS was applied to the radial nerve in-phase (INP) or out-of-phase (OOP) with respect to the muscle activity of the extensor carpi radialis (ECR). The neural adaptations at the spinal cord level were assessed for the flexor carpi radialis (FCR) by measuring disynaptic Group I inhibition, Ia presynaptic inhibition, Ib facilitation from the H-reflex and estimation of the neural drive before, immediately after, and 30 minutes after the intervention.Main results.SATS strategy delivered electrical stimulation synchronized with the real-time muscle activity measured, with an average delay of 17 ± 8 ms. SATS-INP induced increased disynaptic Group I inhibition (77 ± 23% of baseline conditioned FCR H-reflex), while SATS-OOP elicited the opposite effect (125 ± 46% of baseline conditioned FCR H-reflex). Some of the subjects maintained the changes after 30 minutes. No other significant changes were found for the rest of measurements.Significance.These results suggest that the short-term modulatory effects of phase-dependent PES occur at specific targeted spinal pathways for the wrist muscles in healthy individuals. Importantly, timely recruitment of afferent pathways synchronized with specific muscle activity is a fundamental principle that shall be considered when tailoring PES protocols to modulate specific neural circuits. (NCT number 04501133).

Surface EMG Statistical and Performance Analysis of Targeted-Muscle-Reinnervated (TMR) Transhumeral Prosthesis Users in Home and Laboratory Settings

A pattern-recognition (PR)-based myoelectric control system is the trend of future prostheses development. Compared with conventional prosthetic control systems, PR-based control systems provide high dexterity, with many studies achieving >95% accuracy in the last two decades. However, most research studies have been conducted in the laboratory. There is limited research investigating how EMG signals are acquired when users operate PR-based systems in their home and community environments. This study compares the statistical properties of surface electromyography (sEMG) signals used to calibrate prostheses and quantifies the quality of calibration sEMG data through separability indices, repeatability indices, and correlation coefficients in home and laboratory settings. The results demonstrate no significant differences in classification performance between home and laboratory environments in within-calibration classification error (home: 6.33 ± 2.13%, laboratory: 7.57 ± 3.44%). However, between-calibration classification errors (home: 40.61 ± 9.19%, laboratory: 44.98 ± 12.15%) were statistically different. Furthermore, the difference in all statistical properties of sEMG signals is significant (p < 0.05). Separability indices reveal that motion classes are more diverse in the home setting. In summary, differences in sEMG signals generated between home and laboratory only affect between-calibration performance.

EMG-driven shared human-robot compliant control for in-hand object manipulation in hand prostheses

Objective. The limited functionality of hand prostheses remains one of the main reasons behind the lack of its wide adoption by amputees. Indeed, while commercial prostheses can perform a reasonable number of grasps, they are often inadequate for manipulating the object once in hand. This lack of dexterity drastically restricts the utility of prosthetic hands. We aim at investigating a novel shared control strategy that combines autonomous control of forces exerted by a robotic hand with electromyographic (EMG) decoding to perform robust in-hand object manipulation.Approach. We conduct a three-day long longitudinal study with eight healthy subjects controlling a 16-degrees-of-freedom robotic hand to insert objects in boxes of various orientations. EMG decoding from forearm muscles enables subjects to move, proportionally and simultaneously, the fingers of the robotic hand. The desired object rotation is inferred using two EMG electrodes placed on the shoulder that record the activity of muscles responsible for elevation and depression. During the object interaction phase, the autonomous controller stabilizes and rotates the object to achieve the desired pose. In this study, we compare an incremental and a proportional shoulder-decoding method in combination with two state machine interfaces offering different levels of assistance.Main results. Results indicate that robotic assistance reduces the number of failures by41%and, when combined with an incremental shoulder EMG decoding, leads to faster task completion time (median = 16.9 s), compared to other control conditions. Training to use the assistive device is fast. After one session of practice, all subjects managed to achieve tasks with50%less failures.Significance. Shared control approaches that give some authority to an autonomous controller on-board the prosthesis are an alternative to control schemes relying on EMG decoding alone. This may improve the dexterity and versatility of robotic prosthetic hands for people with trans-radial amputation. By delegating control of forces to the prosthesis' on-board control, one speeds up reaction time and improves the precision of force control. Such a shared control mechanism may enable amputees to perform fine insertion tasks solely using their prosthetic hands. This may restore some of the functionality of the disabled arm.

Applying Self-Supervised Representation Learning for Emotion Recognition Using Physiological Signals

The use of machine learning (ML) techniques in affective computing applications focuses on improving the user experience in emotion recognition. The collection of input data (e.g., physiological signals), together with expert annotations are part of the established standard supervised learning methodology used to train human emotion recognition models. However, these models generally require large amounts of labeled data, which is expensive and impractical in the healthcare context, in which data annotation requires even more expert knowledge. To address this problem, this paper explores the use of the self-supervised learning (SSL) paradigm in the development of emotion recognition methods. This approach makes it possible to learn representations directly from unlabeled signals and subsequently use them to classify affective states. This paper presents the key concepts of emotions and how SSL methods can be applied to recognize affective states. We experimentally analyze and compare self-supervised and fully supervised training of a convolutional neural network designed to recognize emotions. The experimental results using three emotion datasets demonstrate that self-supervised representations can learn widely useful features that improve data efficiency, are widely transferable, are competitive when compared to their fully supervised counterparts, and do not require the data to be labeled for learning.

Kinematic analysis of impairments and compensatory motor behavior during prosthetic grasping in below-elbow amputees

After a major upper limb amputation, the use of myoelectric prosthesis as assistive devices is possible. However, these prostheses remain quite difficult to control for grasping and manipulation of daily life objects. The aim of the present observational case study is to document the kinematics of grasping in a group of 10 below-elbow amputated patients fitted with a myoelectric prosthesis in order to describe and better understand their compensatory strategies. They performed a grasping to lift task toward 3 objects (a mug, a cylinder and a cone) placed at two distances within the reaching area in front of the patients. The kinematics of the trunk and upper-limb on the non-amputated and prosthetic sides were recorded with 3 electromagnetic Polhemus sensors placed on the hand, the forearm (or the corresponding site on the prosthesis) and the ipsilateral acromion. The 3D position of the elbow joint and the shoulder and elbow angles were calculated thanks to a preliminary calibration of the sensor position. We examined first the effect of side, distance and objects with non-parametric statistics. Prosthetic grasping was characterized by severe temporo-spatial impairments consistent with previous clinical or kinematic observations. The grasping phase was prolonged and the reaching and grasping components uncoupled. The 3D hand displacement was symmetrical in average, but with some differences according to the objects. Compensatory strategies involved the trunk and the proximal part of the upper-limb, as shown by a greater 3D displacement of the elbow for close target and a greater forward displacement of the acromion, particularly for far targets. The hand orientation at the time of grasping showed marked side differences with a more frontal azimuth, and a more "thumb-up" roll. The variation of hand orientation with the object on the prosthetic side, suggested that the lack of finger and wrist mobility imposed some adaptation of hand pose relative to the object. The detailed kinematic analysis allows more insight into the mechanisms of the compensatory strategies that could be due to both increased distal or proximal kinematic constraints. A better knowledge of those compensatory strategies is important for the prevention of musculoskeletal disorders and the development of innovative prosthetics.

Wearable super-resolution muscle-machine interfacing

Muscles are the actuators of all human actions, from daily work and life to communication and expression of emotions. Myography records the signals from muscle activities as an interface between machine hardware and human wetware, granting direct and natural control of our electronic peripherals. Regardless of the significant progression as of late, the conventional myographic sensors are still incapable of achieving the desired high-resolution and non-invasive recording. This paper presents a critical review of state-of-the-art wearable sensing technologies that measure deeper muscle activity with high spatial resolution, so-called super-resolution. This paper classifies these myographic sensors according to the different signal types (i.e., biomechanical, biochemical, and bioelectrical) they record during measuring muscle activity. By describing the characteristics and current developments with advantages and limitations of each myographic sensor, their capabilities are investigated as a super-resolution myography technique, including: (i) non-invasive and high-density designs of the sensing units and their vulnerability to interferences, (ii) limit-of-detection to register the activity of deep muscles. Finally, this paper concludes with new opportunities in this fast-growing super-resolution myography field and proposes promising future research directions. These advances will enable next-generation muscle-machine interfaces to meet the practical design needs in real-life for healthcare technologies, assistive/rehabilitation robotics, and human augmentation with extended reality.

Use of regenerative peripheral nerve interfaces and intramuscular electrodes to improve prosthetic grasp selection: a case study

Objective.Advanced myoelectric hands enable users to select from multiple functional grasps. Current methods for controlling these hands are unintuitive and require frequent recalibration. This case study assessed the performance of tasks involving grasp selection, object interaction, and dynamic postural changes using intramuscular electrodes with regenerative peripheral nerve interfaces (RPNIs) and residual muscles.Approach.One female with unilateral transradial amputation participated in a series of experiments to compare the performance of grasp selection controllers with RPNIs and intramuscular control signals with controllers using surface electrodes. These experiments included a virtual grasp-matching task with and without a concurrent cognitive task and physical tasks with a prosthesis including standardized functional assessments and a functional assessment where the individual made a cup of coffee ('Coffee Task') that required grasp transitions.Main results.In the virtual environment, the participant was able to select between four functional grasps with higher accuracy using the RPNI controller (92.5%) compared to surface controllers (81.9%). With the concurrent cognitive task, performance of the virtual task was more consistent with RPNI controllers (reduced accuracy by 1.1%) compared to with surface controllers (4.8%). When RPNI signals were excluded from the controller with intramuscular electromyography (i.e. residual muscles only), grasp selection accuracy decreased by up to 24%. The participant completed the Coffee Task with 11.7% longer completion time with the surface controller than with the RPNI controller. She also completed the Coffee Task with 11 fewer transition errors out of a maximum of 25 total errors when using the RPNI controller compared to surface controller.Significance.The use of RPNI signals in concert with residual muscles and intramuscular electrodes can improve grasp selection accuracy in both virtual and physical environments. This approach yielded consistent performance without recalibration needs while reducing cognitive load associated with pattern recognition for myoelectric control (clinical trial registration number NCT03260400).

Natural grasping movement recognition and force estimation using electromyography

Electromyography (EMG) generated by human hand movements is usually used to decode different action types with high accuracy. However, the classifications of the gestures rarely consider the impact of force, and the estimation of the grasp force when performing natural grasping movements is so far overlooked. Decoding natural grasping movements and estimating the force generated by the associated movements can help patients to improve the accuracy of prosthesis control. This study mainly focused on two aspects: the classification of four natural grasping movements and the force estimation of these actions. For this purpose, we designed an experimental platform where subjects could perform four common natural grasping movements in daily life, including pinch, palmar, twist, and plug grasp, to complete target profiles. On the one hand, the results showed that, for natural grasping movements with different levels of force (three levels at 20, 50, and 80%), the average accuracy could reach from 91.43 to 97.33% under five classification schemes. On the other hand, the feasibility of force estimation for natural grasping movements was demonstrated. Furthermore, in the process of force estimation, we confirmed that the regression performance about plug grasp was the best, and the average R2 could reach 0.9082. Besides, we found that the regression results were affected by the speed of force application. These findings contribute to the natural control of myoelectric prosthesis and the EMG-based rehabilitation training system, improving the user’s experience and acceptance.

Surgically Implanted Electrodes Enable Real-Time Finger and Grasp Pattern Recognition for Prosthetic Hands

Currently available prosthetic hands are capable of actuating anywhere from five to 30 degrees of freedom (DOF). However, grasp control of these devices remains unintuitive and cumbersome. To address this issue, we propose directly extracting finger commands from the neuromuscular system. Two persons with transradial amputations had bipolar electrodes implanted into regenerative peripheral nerve interfaces (RPNIs) and residual innervated muscles. The implanted electrodes recorded local electromyography with large signal amplitudes. In a series of single-day experiments, participants used a high speed movement classifier to control a virtual prosthetic hand in real-time. Both participants transitioned between 10 pseudo-randomly cued individual finger and wrist postures with an average success rate of 94.7% and trial latency of 255 ms. When the set was reduced to five grasp postures, metrics improved to 100% success and 135 ms trial latency. Performance remained stable across untrained static arm positions while supporting the weight of the prosthesis. Participants also used the high speed classifier to switch between robotic prosthetic grips and complete a functional performance assessment. These results demonstrate that pattern recognition systems can use intramuscular electrodes and RPNIs for fast and accurate prosthetic grasp control.

Integrating computer vision to prosthetic hand control with sEMG: Preliminary results in grasp classification

The myoelectric prosthesis is a promising tool to restore the hand abilities of amputees, but the classification accuracy of surface electromyography (sEMG) is not high enough for real-time application. Researchers proposed integrating sEMG signals with another feature that is not affected by amputation. The strong coordination between vision and hand manipulation makes us consider including visual information in prosthetic hand control. In this study, we identified a sweet period during the early reaching phase in which the vision data could yield a higher accuracy in classifying the grasp patterns. Moreover, the visual classification results from the sweet period could be naturally integrated with sEMG data collected during the grasp phase. After the integration, the accuracy of grasp classification increased from 85.5% (only sEMG) to 90.06% (integrated). Knowledge gained from this study encourages us to further explore the methods for incorporating computer vision into myoelectric data to enhance the movement control of prosthetic hands.