Neuro-Musculoskeletal Mapping for Man-Machine Interfacing

Springs vs. motors: Ideal assistance in the lower limbs during walking at different speeds

Recent years have witnessed breakthroughs in assistive exoskeletons; both passive and active devices have reduced metabolic costs near preferred walking speed by assisting muscle actions. Metabolic reductions at multiple speeds should thus also be attainable. Musculoskeletal simulation can potentially predict the interaction between assistive moments, muscle-tendon mechanics, and walking energetics. In this study, we simulated devices' optimal assistive moments based on minimal muscle activations during walking with prescribed kinematics and dynamics. We used a generic musculoskeletal model with tuned muscle-tendon parameters and computed metabolic rates from muscle actions. We then simulated walking across multiple speeds and with two ideal actuation modes-motor-based and spring-based-to assist ankle plantarflexion, knee extension, hip flexion, and hip abduction and compared computed metabolic rates. We found that both actuation modes considerably reduced physiological joint moments but did not always reduce metabolic rates. Compared to unassisted conditions, motor-based ankle plantarflexion and hip flexion assistance reduced metabolic rates, and this effect was more pronounced as walking speed increased. Spring-based hip flexion and abduction assistance increased metabolic rates at some walking speeds despite a moderate decrease in some muscle activations. Both modes of knee extension assistance reduced metabolic rates to a small extent, even though the actuation contributed with practically the entire net knee extension moment during stance. Motor-based hip abduction assistance reduced metabolic rates more than spring-based assistance, though this reduction was relatively small. Our study also suggests that an assistive strategy based on minimal muscle activations might result in a suboptimal reduction of metabolic rates. Future work should experimentally validate the effects of assistive moments and refine modeling assumptions accordingly. Our computational workflow is freely available online.

Feature evaluation for myoelectric pattern recognition of multiple nearby reaching targets

Intention detection of the reaching movement is considerable for myoelectric human and machine collaboration applications. A comprehensive set of handcrafted features was mined from windows of electromyogram (EMG) of the upper-limb muscles while reaching nine nearby targets like activities of daily living. The feature selection-based scoring method, neighborhood component analysis (NCA), selected the relevant feature subset. Finally, the target was recognized by the support vector machine (SVM) model. The classification performance was generalized by a nested cross-validation structure that selected the optimal feature subset in the inner loop. According to the low spatial resolution of the target location on display and following the slight discrimination of signals between targets, the best classification accuracy of 77.11 % was achieved for concatenating the features of two segments with a length of 2 and 0.25 s. Due to the lack of subtle variation in EMG, while reaching different targets, a wide range of features was applied to consider additional aspects of the knowledge contained in EMG signals. Furthermore, since NCA selected features that provided more discriminant power, it became achievable to employ various combinations of features and even concatenated features extracted from different movement parts to improve classification performance.

Alternative muscle synergy patterns of upper limb amputees

Myoelectric hand prostheses are effective tools for upper limb amputees to regain hand functions. Much progress has been made with pattern recognition algorithms to recognize surface electromyography (sEMG) patterns, but few attentions was placed on the amputees' motor learning process. Many potential myoelectric prostheses users could not fully master the control or had declined performance over time. It is possible that learning to produce distinct and consistent muscle activation patterns with the residual limb could help amputees better control the myoelectric prosthesis. In this study, we observed longitudinal effect of motor skill learning with 2 amputees who have developed alternative muscle activation patterns in response to the same set of target prosthetic actions. During a 10-week program, amputee participants were trained to produce distinct and constant muscle activations with visual feedback of live sEMG and without interaction with prosthesis. At the end, their sEMG patterns were different from each other and from non-amputee control groups. For certain intended hand motion, gradually reducing root mean square (RMS) variance was observed. The learning effect was also assessed with a CNN-LSTM mixture classifier designed for mobile sEMG pattern recognition. The classification accuracy had a rising trend over time, implicating potential performance improvement of myoelectric prosthesis control. A follow-up session took place 6 months after the program and showed lasting effect of the motor skill learning in terms of sEMG pattern classification accuracy. The results indicated that with proper feedback training, amputees could learn unique muscle activation patterns that allow them to trigger intended prosthesis functions, and the original motor control scheme is updated. The effect of such motor skill learning could help to improve myoelectric prosthetic control performance.

NeuroMechanics: Electrophysiological and computational methods to accurately estimate the neural drive to muscles in humans in vivo

The ultimate neural signal for muscle control is the neural drive sent from the spinal cord to muscles. This neural signal comprises the ensemble of action potentials discharged by the active spinal motoneurons, which is transmitted to the innervated muscle fibres to generate forces. Accurately estimating the neural drive to muscles in humans in vivo is challenging since it requires the identification of the activity of a sample of motor units (MUs) that is representative of the active MU population. Current electrophysiological recordings usually fail in this task by identifying small MU samples with over-representation of higher-threshold with respect to lower-threshold MUs. Here, we describe recent advances in electrophysiological methods that allow the identification of more representative samples of greater numbers of MUs than previously possible. This is obtained with large and very dense arrays of electromyographic electrodes. Moreover, recently developed computational methods of data augmentation further extend experimental MU samples to infer the activity of the full MU pool. In conclusion, the combination of new electrode technologies and computational modelling allows for an accurate estimate of the neural drive to muscles and opens new perspectives in the study of the neural control of movement and in neural interfacing.

Accurate Identification of Motoneuron Discharges From Ultrasound Images Across the Full Muscle Cross-Section

OBJECTIVE

Non-invasive identification of motoneuron (MN) activity commonly uses electromyography (EMG). However, surface EMG (sEMG) detects only superficial sources, at less than approximately 10-mm depth. Intramuscular EMG can detect deep sources, but it is limited to sources within a few mm of the detection site. Conversely, ultrasound (US) images have high spatial resolution across the whole muscle cross-section. The activity of MNs can be extracted from US images due to the movements that MN activation generates in the innervated muscle fibers. Current US-based decomposition methods can accurately identify the location and average twitch induced by MN activity. However, they cannot accurately detect MN discharge times.

METHODS

Here, we present a method based on the convolutive blind source separation of US images to estimate MN discharge times with high accuracy. The method was validated across Ten participants using concomitant sEMG decomposition as the ground truth.

RESULTS

140 unique MN spike trains were identified from US images, with a rate of agreement (RoA) with sEMG decomposition of 87.4 ± 10.3%. Over 50% of these MN spike trains had a RoA greater than 90%. Furthermore, with US, we identified additional MUs well beyond the sEMG detection volume, at up to >30 mm below the skin.

CONCLUSION

The proposed method can identify discharges of MNs innervating muscle fibers in a large range of depths within the muscle from US images.

SIGNIFICANCE

The proposed methodology can non-invasively interface with the outer layers of the central nervous system innervating muscles across the full cross-section.

Multi-layer CNN-LSTM network with self-attention mechanism for robust estimation of nonlinear uncertain systems

Introduction

With the help of robot technology, intelligent rehabilitation of patients with lower limb motor dysfunction caused by stroke can be realized. A key factor constraining the clinical application of rehabilitation robots is how to realize pattern recognition of human movement intentions by using the surface electromyography (sEMG) sensors to ensure unhindered human-robot interaction.

Methods

A multilayer CNN-LSTM prediction network incorporating the self-attention mechanism (SAM) is proposed, in this paper, which can extract and learn the periodic and trend characteristics of the sEMG signals, and realize the accurate autoregressive prediction of the human motion information. Firstly, the multilayer CNN-LSTM network utilizes the CNN layer for initial feature extraction of data, and the LSTM network is used to improve the enhancement of the historical time-series features. Then, the SAM is used to improve the global feature extraction performance and parallel computation speed of the network.

Results

In comparison with existing test is carried out using actual data from five healthy subjects as well as a clinical hemiplegic patient to verify the superiority and practicality of the proposed algorithm. The results show that most of the model's prediction R > 0.9 for different motion states of healthy subjects; in the experiments oriented to the motion characteristics of patient subjects, the angle prediction results of R > 0.99 for the untrained data on the affected side, which proves that our proposed model also has a better effect on the angle prediction of the affected side.

Discussion

The main contribution of this paper is to realize continuous motion estimation of ankle joint for healthy and hemiplegic individuals under non-ideal conditions (weak sEMG signals, muscle fatigue, high muscle tension, etc.), which improves the pattern recognition accuracy and robustness of the sEMG sensor-based system.

The assistance of BAZAR robot promotes improved upper limb motor coordination in workers performing an actual use-case manual material handling

This study aims at evaluating upper limb muscle coordination and activation in workers performing an actual use-case manual material handling (MMH). The study relies on the comparison of the workers' muscular activity while they perform the task, with and without the help of a dual-arm cobot (BAZAR). Eleven participants performed the task and the flexors and extensors muscles of the shoulder, elbow, wrist, and trunk joints were recorded using bipolar electromyography. The results showed that, when the particular MMH was carried out with BAZAR, both upper limb and trunk muscular co-activation and activation were decreased. Therefore, technologies that enable human-robot collaboration (HRC), which share a workspace with employees, relieve employees of external loads and enhance the effectiveness and calibre of task completion. Additionally, these technologies improve the worker's coordination, lessen the physical effort required to interact with the robot, and have a favourable impact on his or her physiological motor strategy. Practitioner summary: Upper limb and trunk muscle co-activation and activation is reduced when a specific manual material handling was performed with a cobot than without it. By improving coordination, reducing physical effort, and changing motor strategy, cobots could be proposed as an ergonomic intervention to lower workers' biomechanical risk in industry.

Motoneuron-driven computational muscle modelling with motor unit resolution and subject-specific musculoskeletal anatomy

The computational simulation of human voluntary muscle contraction is possible with EMG-driven Hill-type models of whole muscles. Despite impactful applications in numerous fields, the neuromechanical information and the physiological accuracy such models provide remain limited because of multiscale simplifications that limit comprehensive description of muscle internal dynamics during contraction. We addressed this limitation by developing a novel motoneuron-driven neuromuscular model, that describes the force-generating dynamics of a population of individual motor units, each of which was described with a Hill-type actuator and controlled by a dedicated experimentally derived motoneuronal control. In forward simulation of human voluntary muscle contraction, the model transforms a vector of motoneuron spike trains decoded from high-density EMG signals into a vector of motor unit forces that sum into the predicted whole muscle force. The motoneuronal control provides comprehensive and separate descriptions of the dynamics of motor unit recruitment and discharge and decodes the subject's intention. The neuromuscular model is subject-specific, muscle-specific, includes an advanced and physiological description of motor unit activation dynamics, and is validated against an experimental muscle force. Accurate force predictions were obtained when the vector of experimental neural controls was representative of the discharge activity of the complete motor unit pool. This was achieved with large and dense grids of EMG electrodes during medium-force contractions or with computational methods that physiologically estimate the discharge activity of the motor units that were not identified experimentally. This neuromuscular model advances the state-of-the-art of neuromuscular modelling, bringing together the fields of motor control and musculoskeletal modelling, and finding applications in neuromuscular control and human-machine interfacing research.

Optimal Motor Unit Subset Selection for Accurate Motor Intention Decoding: Towards Dexterous Real-Time Interfacing

OBJECTIVE

Motor unit (MU) discharge timings encode human motor intentions to the finest degree. Whilst tapping into such information can bring significant gains to a range of applications, current approaches to MU decoding from surface signals do not scale well with the demands of dexterous human-machine interfacing (HMI). To optimize the forward estimation accuracy and time-efficiency of such systems, we propose the inclusion of task-wise initialization and MU subset selection.

METHODS

Offline analyses were conducted on data recorded from 11 non-disabled subjects. Task-wise decomposition was applied to identify MUs from high-density surface electromyography (HD-sEMG) pertaining to 18 wrist/forearm motor tasks. The activities of a selected subset of MUs were extracted from test data and used for forward estimation of intended motor tasks and joint kinematics. To that end, various combinations of subset selection and estimation algorithms (both regression and classification-based) were tested for a range of subset sizes.

RESULTS

The mutual information-based minimum Redundancy Maximum Relevance (mRMR-MI) criterion retained MUs with the highest predicative power. When the portion of tracked MUs was reduced down to 25%, the regression performance decreased only by 3% (R2=0.79) while classification accuracy dropped by 2.7% (accuracy = 74%) when kernel-based estimators were considered.

CONCLUSION AND SIGNIFICANCE

Careful selection of tracked MUs can optimize the efficiency of MU-driven interfacing. In particular, prioritization of MUs exhibiting strong nonlinear relationships with target motions is best leveraged by kernel-based estimators. Hence, this frees resources for more robust and adaptive MU decoding techniques to be implemented in future.

Modeling Human Suboptimal Control: A Review

This review paper provides an overview of the approaches to model neuromuscular control, focusing on methods to identify nonoptimal control strategies typical of populations with neuromuscular disorders or children. Where possible, the authors tightened the description of the methods to the mechanisms behind the underlying biomechanical and physiological rationale. They start by describing the first and most simplified approach, the reductionist approach, which splits the role of the nervous and musculoskeletal systems. Static optimization and dynamic optimization methods and electromyography-based approaches are summarized to highlight their limitations and understand (the need for) their developments over time. Then, the authors look at the more recent stochastic approach, introduced to explore the space of plausible neural solutions, thus implementing the uncontrolled manifold theory, according to which the central nervous system only controls specific motions and tasks to limit energy consumption while allowing for some degree of adaptability to perturbations. Finally, they explore the literature covering the explicit modeling of the coupling between the nervous system (acting as controller) and the musculoskeletal system (the actuator), which may be employed to overcome the split characterizing the reductionist approach.

Bio-robotics research for non-invasive myoelectric neural interfaces for upper-limb prosthetic control: a 10-year perspective review

A decade ago, a group of researchers from academia and industry identified a dichotomy between the industrial and academic state-of-the-art in upper-limb prosthesis control, a widely used bio-robotics application. They proposed that four key technical challenges, if addressed, could bridge this gap and translate academic research into clinically and commercially viable products. These challenges are unintuitive control schemes, lack of sensory feedback, poor robustness and single sensor modality. Here, we provide a perspective review on the research effort that occurred in the last decade, aiming at addressing these challenges. In addition, we discuss three research areas essential to the recent development in upper-limb prosthetic control research but were not envisioned in the review 10 years ago: deep learning methods, surface electromyogram decomposition and open-source databases. To conclude the review, we provide an outlook into the near future of the research and development in upper-limb prosthetic control and beyond.

Toward a generalizable deep CNN for neural drive estimation across muscles and participants

Objective.High-density electromyography (HD-EMG) decomposition algorithms are used to identify individual motor unit (MU) spike trains, which collectively constitute the neural code of movements, to predict motor intent. This approach has advanced from offline to online decomposition, from isometric to dynamic contractions, leading to a wide range of neural-machine interface applications. However, current online methods need offline retraining when applied to the same muscle on a different day or to a different person, which limits their applications in a real-time neural-machine interface. We proposed a deep convolutional neural network (CNN) framework for neural drive estimation, which takes in frames of HD-EMG signals as input, extracts general spatiotemporal properties of MU action potentials, and outputs the number of spikes in each frame. The deep CNN can generalize its application without retraining to HD-EMG data recorded in separate sessions, muscles, or participants.Approach.We recorded HD-EMG signals from the vastus medialis and vastus lateralis muscles from five participants while they performed isometric contractions during two sessions separated by ∼20 months. We identified MU spike trains from HD-EMG signals using a convolutive blind source separation (BSS) method, and then used the cumulative spike train (CST) of these MUs and the HD-EMG signals to train and validate the deep CNN.Main results.On average, the correlation coefficients between CST from the BSS and that from deep CNN were0.983±0.006for leave-one-out across-sessions-and-muscles validation and0.989±0.002for leave-one-out across-participants validation. When trained with more than four datasets, the performance of deep CNN saturated at0.984±0.001for cross validations across muscles, sessions, and participants.Significance.We can conclude that the deep CNN is generalizable across the aforementioned conditions without retraining. We could potentially generate a robust deep CNN to estimate neural drive to muscles for neural-machine interfaces.

Mapping Individual Motor Unit Activity to Continuous Three-DoF Wrist Torques: Perspectives for Myoelectric Control

The surface electromyography (EMG) decomposition techniques provide access to motor neuron activities and have been applied to myoelectric control schemes. However, the current decomposition-based myoelectric control mainly focuses on discrete gestures or single-DoF continuous movements. In this study, we aimed to map the motor unit discharges, which were identified from high-density surface EMG, to the three degrees of freedom (DoFs) wrist movements. The 3-DoF wrist torques and high-density surface EMG signals were recorded concurrently from eight non-disabled subjects. The experimental protocol included single-DoF movements and their various combinations. We decoded the motor unit discharges from the EMG signals using a segment-wise decomposition algorithm. Then the neural features were extracted from motor unit discharges and projected to wrist torques with a multiple linear regression model. We compared the performance of two neural features (twitch model and spike counting) and two training schemes (single-DoF and multi-DoF training). On average, 145 ± 33 motor units were identified from each subject, with a pulse-to-noise ratio of 30.8 ± 4.2 dB. Both neural features exhibited high estimation accuracy of 3-DoF wrist torques, with an average [Formula: see text] of 0.76 ± 0.12 and normalized root mean square error of 11.4 ± 3.1%. These results demonstrated the efficiency of the proposed method in continuous estimation of 3-DoF wrist torques, which has the potential to advance dexterous myoelectric control based on neural information.

Surgically Implanted Electrodes Enable Real-Time Finger and Grasp Pattern Recognition for Prosthetic Hands

Currently available prosthetic hands are capable of actuating anywhere from five to 30 degrees of freedom (DOF). However, grasp control of these devices remains unintuitive and cumbersome. To address this issue, we propose directly extracting finger commands from the neuromuscular system. Two persons with transradial amputations had bipolar electrodes implanted into regenerative peripheral nerve interfaces (RPNIs) and residual innervated muscles. The implanted electrodes recorded local electromyography with large signal amplitudes. In a series of single-day experiments, participants used a high speed movement classifier to control a virtual prosthetic hand in real-time. Both participants transitioned between 10 pseudo-randomly cued individual finger and wrist postures with an average success rate of 94.7% and trial latency of 255 ms. When the set was reduced to five grasp postures, metrics improved to 100% success and 135 ms trial latency. Performance remained stable across untrained static arm positions while supporting the weight of the prosthesis. Participants also used the high speed classifier to switch between robotic prosthetic grips and complete a functional performance assessment. These results demonstrate that pattern recognition systems can use intramuscular electrodes and RPNIs for fast and accurate prosthetic grasp control.

Estimation of the firing behaviour of a complete motoneuron pool by combining electromyography signal decomposition and realistic motoneuron modelling

Our understanding of the firing behaviour of motoneuron (MN) pools during human voluntary muscle contractions is currently limited to electrophysiological findings from animal experiments extrapolated to humans, mathematical models of MN pools not validated for human data, and experimental results obtained from decomposition of electromyographical (EMG) signals. These approaches are limited in accuracy or provide information on only small partitions of the MN population. Here, we propose a method based on the combination of high-density EMG (HDEMG) data and realistic modelling for predicting the behaviour of entire pools of motoneurons in humans. The method builds on a physiologically realistic model of a MN pool which predicts, from the experimental spike trains of a smaller number of individual MNs identified from decomposed HDEMG signals, the unknown recruitment and firing activity of the remaining unidentified MNs in the complete MN pool. The MN pool model is described as a cohort of single-compartment leaky fire-and-integrate (LIF) models of MNs scaled by a physiologically realistic distribution of MN electrophysiological properties and driven by a spinal synaptic input, both derived from decomposed HDEMG data. The MN spike trains and effective neural drive to muscle, predicted with this method, have been successfully validated experimentally. A representative application of the method in MN-driven neuromuscular modelling is also presented. The proposed approach provides a validated tool for neuroscientists, experimentalists, and modelers to infer the firing activity of MNs that cannot be observed experimentally, investigate the neuromechanics of human MN pools, support future experimental investigations, and advance neuromuscular modelling for investigating the neural strategies controlling human voluntary contractions.

Reducing the number of EMG electrodes during online hand gesture classification with changing wrist positions

Background

Myoelectric control based on hand gesture classification can be used for effective, contactless human–machine interfacing in general applications (e.g., consumer market) as well as in the clinical context. However, the accuracy of hand gesture classification can be impacted by several factors including changing wrist position. The present study aimed at investigating how channel configuration (number and placement of electrode pads) affects performance in hand gesture recognition across wrist positions, with the overall goal of reducing the number of channels without the loss of performance with respect to the benchmark (all channels).

Methods

Matrix electrodes (256 channels) were used to record high-density EMG from the forearm of 13 healthy subjects performing a set of 8 gestures in 3 wrist positions and 2 force levels (low and moderate). A reduced set of channels was chosen by applying sequential forward selection (SFS) and simple circumferential placement (CIRC) and used for gesture classification with linear discriminant analysis. The classification success rate and task completion rate were the main outcome measures for offline analysis across the different number of channels and online control using 8 selected channels, respectively.

Results

The offline analysis demonstrated that good accuracy (> 90%) can be achieved with only a few channels. However, using data from all wrist positions required more channels to reach the same performance. Despite the targeted placement (SFS) performing similarly to CIRC in the offline analysis, the task completion rate [median (lower–upper quartile)] in the online control was significantly higher for SFS [71.4% (64.8–76.2%)] compared to CIRC [57.1% (51.8–64.8%), p < 0.01], especially for low contraction levels [76.2% (66.7–84.5%) for SFS vs. 57.1% (47.6–60.7%) for CIRC, p < 0.01]. For the reduced number of electrodes, the performance with SFS was comparable to that obtained when using the full matrix, while the selected electrodes were highly subject-specific.

Conclusions

The present study demonstrated that the number of channels required for gesture classification with changing wrist positions could be decreased substantially without loss of performance, if those channels are placed strategically along the forearm and individually for each subject. The results also emphasize the importance of online assessment and motivate the development of configurable matrix electrodes with integrated channel selection.

Multi-Day EMG-Based Knee Joint Torque Estimation Using Hybrid Neuromusculoskeletal Modelling and Convolutional Neural Networks

Proportional control using surface electromyography (EMG) enables more intuitive control of a transfemoral prosthesis. However, EMG is a noisy signal which can vary over time, giving rise to the question what approach for knee torque estimation is most suitable for multi-day control. In this study we compared three different modelling frameworks to estimate knee torque in non-weight-bearing situations. The first model contained a convolutional neural network (CNN) which mapped EMG to knee torque directly. The second used a neuromusculoskeletal model (NMS) which used EMG, muscle tendon unit lengths and moment arms to compute knee torque. The third model (Hybrid) used a CNN to map EMG to specific muscle activation, which was used together with NMS components to compute knee torque. Multi-day measurements were conducted on ten able-bodied participants who performed non-weight bearing activities. CNN had the best performance in general and on each day (Normalized Root Mean Squared Error (NRMSE) 9.2 ± 4.4%). The Hybrid model (NRMSE 12.4 ± 3.4%) was able to outperform NMS (NRMSE 14.3 ± 4.2%). The NMS model showed no significant difference between measurement days. The CNN model and Hybrid models had significant performance differences between the first day and all other days. CNNs are suited for multi-day torque estimation in terms of error rate, outperforming the other two model types. NMS was the only model type which was robust over all days. This study investigated the behavior of three model types over multiple days, giving insight in the most suited modelling approach for multi-day torque estimation to be used in prosthetic control.

Bipolar versus high-density surface electromyography for evaluating risk in fatiguing frequency-dependent lifting activities

Workers often develop low back pain due to manually lifting heavy loads. Instrumental-based assessment tools are used to quantitatively assess the biomechanical risk in lifting activities. This study aims to verify the hypothesis that high-density surface electromyography (HDsEMG) allows an optimized discrimination of risk levels associated with different fatiguing lifting conditions compared to traditional bipolar sEMG. 15 participants performed three lifting tasks with a progressively increasing lifting index (LI) each lasting 15 min. Erector spinae (ES) activity was recorded using both bipolar and HDsEMG systems. The amplitude of both bipolar and HDsEMG can significantly discriminate each pair of LI. HDsEMG data could discriminate across the different LIs starting from the fourth minute of the task while bipolar sEMG could only do so towards the end. The higher discriminative power of HDsEMG data across the lifting tasks makes such methodology a valuable tool to be used to monitor fatigue while lifting and could extend the possibilities offered by currently available instrumental-based tools.

Wrist Torque Estimation via Electromyographic Motor Unit Decomposition and Image Reconstruction

Neural interface using decomposed motor units (MUs) from surface electromyography (sEMG) has allowed non-invasive access to the neural control signals, and provided a novel approach for intuitive human-machine interaction. However, most of the existing methods based on decomposed MUs merely adopted the discharge rate (DR) as the feature representations, which may lack local information around the discharge instant and ignore the subtle interactions of different MUs. In this study, we proposed an MU-specific image-based scheme for wrist torque estimation. Specifically, the high-density sEMG signals were decoded into motor unit spike trains (MUSTs), and then MU-specific images were reconstructed with MUSTs and corresponding motor unit action potential (MUAP). A convolutional neural network was used to learn representative features from MU-specific images automatically, and further to estimate wrist torques. The results demonstrated that the proposed method outperformed three conventional and a deep-learning regression approaches using DR features, with the estimation accuracy R² of 0.82 ± 0.09, 0.89 ± 0.06, and nRMSE of 12.6 ± 2.5%, 11.0 ± 3.1% for pronation/supination and flexion/extension, respectively. Further, the analysis of the extracted features from MU-specific images showed a higher correlation than DR for recorded torques, indicating the effectiveness of the proposed method. The outcomes of this study provide a novel and promising perspective for the intuitive control of neural interfacing.

A segmented forearm model of hand pronation-supination approximates joint moments for real time applications

Musculoskeletal modeling is a new computational tool to reverse engineer human control systems, which require efficient algorithms running in real-time. Human hand pronation-supination movement is accomplished by movement of the radius and ulna bones relative to each other via the complex proximal and distal radioulnar joints, each with multiple degrees of freedom (DOFs). Here, we report two simplified models of this complex kinematic transformation implemented as a part of a 20 DOF model of the hand and forearm. The pronation/supination DOF was implemented as a single rotation joint either within the forearm segment or separating proximal and distal parts of the forearm segment. Torques produced by the inverse dynamic simulations with anatomical architecture of the forearm (OpenSim model) were used as the "gold standard" in the comparison of two simple models. Joint placement was iteratively optimized to achieve the closest representation of torques during realistic hand movements. The model with a split forearm segment performed better than the model with a solid forearm segment in simulating pronation/supination torques. We conclude that simplifying pronation/supination DOF as a single-axis rotation between arm segments is a viable strategy to reduce the complexity of multi-DOF dynamic simulations.

A convolutional neural network to identify motor units from high-density surface electromyography signals in real time

OBJECTIVES

This paper aims to investigate the feasibility and the validity of applying deep convolutional neural networks (CNN) to identify motor unit (MU) spike trains and estimate the neural drive to muscles from high-density electromyography (HD-EMG) signals in real time. Two distinct deep CNNs are compared with the convolution kernel compensation (CKC) algorithm using simulated and experimentally recorded signals. The effects of window size and step size of the input HD-EMG signals are also investigated.

APPROACH

The MU spike trains were first identified with the CKC algorithm. The HD-EMG signals and spike trains were used to train the deep CNN. Then, the deep CNN decomposed the HD-EMG signals into MU discharge times in real time. Two CNN approaches are compared with the CKC: 1) multiple single-output deep CNN (SO-DCNN) with one MU decomposed per network, and 2) one multiple-output deep CNN (MO-DCNN) to decompose all MUs (up to 23) with one network.

MAIN RESULTS

The MO-DCNN outperformed the SO-DCNN in terms of training time (3.2 to 21.4 s/epoch vs. 6.5 to 47.8 s/epoch, respectively) and prediction time (0.04 vs. 0.27 s/sample, respectively). The optimal window size and step size for MO-DCNN were 120 and 20 data points, respectively. It results in sensitivity of 98% and 85% with simulated and experimentally recorded HD-EMG signals, respectively. There is a high cross-correlation coefficient between the neural drive estimated with CKC and that estimated with MO-DCNN (range of r-value across conditions: 0.88-0.95).

SIGNIFICANCE

We demonstrate the feasibility and the validity of using deep CNN to accurately identify MU activity from HD-EMG with a latency lower than 80 ms, which falls within the lower bound of the human electromechanical delay. This method opens many opportunities for using the neural drive to interface humans with assistive devices.

Critical Issues and Imminent Challenges in the Use of sEMG in Return-To-Work Rehabilitation of Patients Affected by Neurological Disorders in the Epoch of Human-Robot Collaborative Technologies

Patients affected by neurological pathologies with motor disorders when they are of working age have to cope with problems related to employability, difficulties in working, and premature work interruption. It has been demonstrated that suitable job accommodation plans play a beneficial role in the overall quality of life of pathological subjects. A well-designed return-to-work program should consider several recent innovations in the clinical and ergonomic fields. One of the instrument-based methods used to monitor the effectiveness of ergonomic interventions is surface electromyography (sEMG), a multi-channel, non-invasive, wireless, wearable tool, which allows in-depth analysis of motor coordination mechanisms. Although the scientific literature in this field is extensive, its use remains significantly underexploited and the state-of-the-art technology lags expectations. This is mainly attributable to technical and methodological (electrode-skin impedance, noise, electrode location, size, configuration and distance, presence of crosstalk signals, comfort issues, selection of appropriate sensor setup, sEMG amplitude normalization, definition of correct sEMG-related outcomes and normative data) and cultural limitations. The technical and methodological problems are being resolved or minimized also thanks to the possibility of using reference books and tutorials. Cultural limitations are identified in the traditional use of qualitative approaches at the expense of quantitative measurement-based monitoring methods to design and assess ergonomic interventions and train operators. To bridge the gap between the return-to-work rehabilitation and other disciplines, several teaching courses, accompanied by further electrodes and instrumentations development, should be designed at all Bachelor, Master and PhD of Science levels to enhance the best skills available among physiotherapists, occupational health and safety technicians and ergonomists.

Surgically Implanted Electrodes Enable Real-Time Finger and Grasp Pattern Recognition for Prosthetic Hands

Currently available prosthetic hands are capable of actuating anywhere from five to 30 degrees of freedom (DOF). However, grasp control of these devices remains unintuitive and cumbersome. To address this issue, we propose directly extracting finger commands from the neuromuscular system via electrodes implanted in residual innervated muscles and regenerative peripheral nerve interfaces (RPNIs). Two persons with transradial amputations had RPNIs created by suturing autologous free muscle grafts to their transected median, ulnar, and dorsal radial sensory nerves. Bipolar electrodes were surgically implanted into their ulnar and median RPNIs and into their residual innervated muscles. The implanted electrodes recorded local electromyography (EMG) with Signal-to-Noise Ratios ranging from 23 to 350 measured across various movements. In a series of single-day experiments, participants used a high speed pattern recognition system to control a virtual prosthetic hand in real-time. Both participants were able to transition between 10 pseudo-randomly cued individual finger and wrist postures in the virtual environment with an average online accuracy of 86.5% and latency of 255 ms. When the set was reduced to five grasp postures, average metrics improved to 97.9% online accuracy and 135 ms latency. Virtual task performance remained stable across untrained static arm positions while supporting the weight of the prosthesis. Participants also used the high speed classifier to switch between robotic prosthetic grips and complete a functional performance assessment. These results demonstrate that pattern recognition systems can use the high-quality EMG afforded by intramuscular electrodes and RPNIs to provide users with fast and accurate grasp control.

SUMMARY

Surgically implanted electrodes recorded finger-specific electromyography enabling reliable finger and grasp control of an upper limb prosthesis.