Limb Position Tolerant Pattern Recognition for Myoelectric Prosthesis Control with Adaptive Sparse Representations From Extreme Learning

A multifaceted suite of metrics for comparative myoelectric prosthesis controller research

Upper limb robotic (myoelectric) prostheses are technologically advanced, but challenging to use. In response, substantial research is being done to develop person-specific prosthesis controllers that can predict a user's intended movements. Most studies that test and compare new controllers rely on simple assessment measures such as task scores (e.g., number of objects moved across a barrier) or duration-based measures (e.g., overall task completion time). These assessment measures, however, fail to capture valuable details about: the quality of device arm movements; whether these movements match users' intentions; the timing of specific wrist and hand control functions; and users' opinions regarding overall device reliability and controller training requirements. In this work, we present a comprehensive and novel suite of myoelectric prosthesis control evaluation metrics that better facilitates analysis of device movement details-spanning measures of task performance, control characteristics, and user experience. As a case example of their use and research viability, we applied these metrics in real-time control experimentation. Here, eight participants without upper limb impairment compared device control offered by a deep learning-based controller (recurrent convolutional neural network-based classification with transfer learning, or RCNN-TL) to that of a commonly used controller (linear discriminant analysis, or LDA). The participants wore a simulated prosthesis and performed complex functional tasks across multiple limb positions. Analysis resulting from our suite of metrics identified 16 instances of a user-facing problem known as the "limb position effect". We determined that RCNN-TL performed the same as or significantly better than LDA in four such problem instances. We also confirmed that transfer learning can minimize user training burden. Overall, this study contributes a multifaceted new suite of control evaluation metrics, along with a guide to their application, for use in research and testing of myoelectric controllers today, and potentially for use in broader rehabilitation technologies of the future.

Motor imagery classification using sparse representations: an exploratory study

The non-stationary nature of the EEG signal poses challenges for the classification of motor imagery. sparse representation classification (SRC) appears as an alternative for classification of untrained conditions and, therefore, useful in motor imagery. Empirical mode decomposition (EMD) deals with signals of this nature and appears at the rear of the classification, supporting the generation of features. In this work we evaluate the combination of these methods in a multiclass classification problem, comparing them with a conventional method in order to determine if their performance is regular. For comparison with SRC we use multilayer perceptron (MLP). We also evaluated a hybrid approach for classification of sparse representations with MLP (RSMLP). For comparison with EMD we used filtering by frequency bands. Feature selection methods were used to select the most significant ones, specifically Random Forest and Particle Swarm Optimization. Finally, we used data augmentation to get a more voluminous base. Regarding the first dataset, we observed that the classifiers that use sparse representation have results equivalent to each other, but they outperform the conventional MLP model. SRC and SRMLP achieve an average accuracy of [Formula: see text] and [Formula: see text] respectively while the MLP is [Formula: see text], representing a gain between [Formula: see text] and [Formula: see text]. The use of EMD in relation to other feature processing techniques is not superior. However, EMD does not influence negatively, there is an opportunity for improvement. Finally, the use of data augmentation proved to be important to obtain relevant results. In the second dataset, we did not observe the same results. Models based on sparse representation (SRC, SRMLP, etc.) have on average a performance close to other conventional models, but without surpassing them. The best sparse models achieve an average accuracy of [Formula: see text] among the subjects in the base, while other model reach [Formula: see text]. The improvement of self-adaptive mechanisms that respond efficiently to the user's context is a good way to achieve improvements in motor imagery applications. However, other scenarios should be investigated, since the advantage of these methods was not proven in all datasets studied. There is still room for improvement, such as optimizing the dictionary of sparse representation in the context of motor imagery. Investing efforts in synthetically increasing the training base has also proved important to reduce the costs of this group of applications.

sEMG-Based Gesture Recognition Using Temporal History

Surface electromyography (sEMG) patterns have been decoded using learning-based methods that determine complicated nonlinear decision boundaries. However, overlapping classes in sEMG pattern recognition still degrade the classification accuracy because they cannot be separated by the decision boundaries. We hypothesized that certain overlapping classes can be separated while tracing the temporal history of sEMG patterns. Therefore, a novel post-processing method is proposed to adjust classification errors using the separated patterns from the temporal history of overlapping classes. The proposed method confirms the confidence of the prediction result by calculating the instantaneous pattern separability for the sequential sEMG input. The prediction result with high separability pattern is considered to have a high confidence of being correct (reliable). This result is stored for adjusting the next sEMG input. When the subsequent prediction is identified as having low confidence (unreliable), the predicted result is adjusted using the stored reliable predicted results. The proposed method adds an adjustment step to an existing classifier (maximum likelihood classifier (MLC), k-nearest neighbor (KNN), and support vector machine (SVM)), such that it can be attached to the back-end regardless of the type of classifier. Ten subjects performed 13 types of hand gestures, including overlapping patterns. The overall classification accuracy was enhanced to 88.93%(+8.12%p, MLC), 91.31%(+7.68%p, KNN), and 99.65%(+11.63%p, SVM) after the implementation of the proposed post-processing. Additionally, a faster and more accurate gesture classification was achieved with accuracy enhancement before gesture completion as 85.62%(+4.23%p, MLC), 89.77%(+4.23%p, KNN), and 97.62%(+11.12%p, SVM).

Deep extreme learning machine with knowledge augmentation for EEG seizure signal recognition

Introduction

Intelligent recognition of electroencephalogram (EEG) signals can remarkably improve the accuracy of epileptic seizure prediction, which is essential for epileptic diagnosis. Extreme learning machine (ELM) has been applied to EEG signals recognition, however, the artifacts and noises in EEG signals have a serious effect on recognition efficiency. Deep learning is capable of noise resistance, contributing to removing the noise in raw EEG signals. But traditional deep networks suffer from time-consuming training and slow convergence.

Methods

Therefore, a novel deep learning based ELM (denoted as DELM) motivated by stacking generalization principle is proposed in this paper. Deep extreme learning machine (DELM) is a hierarchical network composed of several independent ELM modules. Augmented EEG knowledge is taken as complementary component, which will then be mapped into next module. This learning process is so simple and fast, meanwhile, it can excavate the implicit knowledge in raw data to a greater extent. Additionally, the proposed method is operated in a single-direction manner, so there is no need to perform parameters fine-tuning, which saves the expense of time.

Results

Extensive experiments are conducted on the public Bonn EEG dataset. The experimental results demonstrate that compared with the commonly-used seizure prediction methods, the proposed DELM wins the best average accuracies in 13 out of the 22 data and the best average F-measure scores in 10 out of the 22 data. And the running time of DELM is more than two times quickly than deep learning methods.

Discussion

Therefore, DELM is superior to traditional and some state-of-the-art machine learning methods. The proposed architecture demonstrates its feasibility and superiority in epileptic EEG signal recognition. The proposed less computationally intensive deep classifier enables faster seizure onset detection, which is showing great potential on the application of real-time EEG signal classification.

Two-stage classification of electromyogram signals from hand grasps in the transverse plane

This paper presents a two-stage classification to resolve the effect of arm position changes on electromyogram (EMG) classification for hand grasps in the transverse plane. The proposed method combines the EMG signals with the signals from an inertial measurement unit in both the position and motion classification stages. To improve accuracy, we incorporate EMG data from the upper arm and shoulder with the forearm EMG signals. When evaluated on the five alternative object grasps placed on the nine positions, the proposed technique yields an average total classification error of 0.9%, which is a substantial improvement over the single-stage classification (4.3%).

sEMG-Based Hand Posture Recognition and Visual Feedback Training for the Forearm Amputee

sEMG-based gesture recognition is useful for human-computer interactions, especially for technology supporting rehabilitation training and the control of electric prostheses. However, high variability in the sEMG signals of untrained users degrades the performance of gesture recognition algorithms. In this study, the hand posture recognition algorithm and radar plot-based visual feedback training were developed using multichannel sEMG sensors. Ten healthy adults and one bilateral forearm amputee participated by repeating twelve hand postures ten times. The visual feedback training was performed for two days and five days in healthy adults and a forearm amputee, respectively. Artificial neural network classifiers were trained with two types of feature vectors: a single feature vector and a combination of feature vectors. The classification accuracy of the forearm amputee increased significantly after three days of hand posture training. These results indicate that the visual feedback training efficiently improved the performance of sEMG-based hand posture recognition by reducing variability in the sEMG signal. Furthermore, a bilateral forearm amputee was able to participate in the rehabilitation training by using a radar plot, and the radar plot-based visual feedback training would help the amputees to control various electric prostheses.

Surgically Implanted Electrodes Enable Real-Time Finger and Grasp Pattern Recognition for Prosthetic Hands

Currently available prosthetic hands are capable of actuating anywhere from five to 30 degrees of freedom (DOF). However, grasp control of these devices remains unintuitive and cumbersome. To address this issue, we propose directly extracting finger commands from the neuromuscular system. Two persons with transradial amputations had bipolar electrodes implanted into regenerative peripheral nerve interfaces (RPNIs) and residual innervated muscles. The implanted electrodes recorded local electromyography with large signal amplitudes. In a series of single-day experiments, participants used a high speed movement classifier to control a virtual prosthetic hand in real-time. Both participants transitioned between 10 pseudo-randomly cued individual finger and wrist postures with an average success rate of 94.7% and trial latency of 255 ms. When the set was reduced to five grasp postures, metrics improved to 100% success and 135 ms trial latency. Performance remained stable across untrained static arm positions while supporting the weight of the prosthesis. Participants also used the high speed classifier to switch between robotic prosthetic grips and complete a functional performance assessment. These results demonstrate that pattern recognition systems can use intramuscular electrodes and RPNIs for fast and accurate prosthetic grasp control.

A review of upper limb pediatric prostheses and perspectives on future advancements

Many complex factors affect whether a child with a congenital upper limb deficiency will wear a prosthetic limb. Ultimately, for a child to wear and use their prosthesis, it must facilitate the effective performance of daily tasks and promote healthy social interactions. Although numerous pediatric devices are available, most provide a single open-close grasp (if a grasping function is available at all) and often offer nonanthropomorphic appearances, falling short of meeting these criteria. In this narrative review, we provide a critical assessment of the state of upper limb prostheses for children. We summarize literature using quality of life measures and categorize driving factors affecting prosthesis use into two main groupings: psychosocial and physical functioning. We define psychosocial functioning as factors related to social inclusion/exclusion, emotional function, independence, and school functioning. Physical functioning is defined as factors associated with the physical use of a prosthesis. The reviewed literature suggests that psychosocial domains of quality of life may be influenced by a congenital limb deficiency, and currently available prostheses provide little benefit in the physical functioning domains. Finally, we discuss technological advancements in adult prostheses that have yet to be leveraged for pediatric devices, including describing recently developed adult electric hands that may improve physical functioning through multiple grasping configurations and provide more hand-like cosmesis. We outline actions necessary to translate similar technologies for children and discuss further strategies to begin removing barriers to pediatric device adoption.

Monitoring at-home prosthesis control improvements through real-time data logging

OBJECTIVE

Validating the ability for advanced prostheses to improve function beyond the laboratory remains a critical step in enabling long-term benefits for prosthetic limb users.

APPROACH

A nine week take-home case study was completed with a single participant with upper limb amputation and osseointegration (OI) to better understand how an advanced prosthesis is used during daily activities. The participant was already an expert prosthesis user and used the Modular Prosthetic Limb (MPL) at home during the study. The MPL was controlled using wireless electromyography (EMG) pattern recognition-based movement decoding. Clinical assessments were performed before and after the take-home portion of the study. Data was recorded using an onboard data log in order to measure daily prosthesis usage, sensor data, and EMG data.

MAIN RESULT

The participant's continuous prosthesis usage steadily increased (p = 0.04, max = 5.5 hr) over time and over 30% of the total time was spent actively controlling the prosthesis. The duration of prosthesis usage after each pattern recognition training session also increased over time (p = 0.04), resulting in up to 5.4 hr of usage before retraining the movement decoding algorithm. Pattern recognition control accuracy improved (1.2% per week, p < 0.001) with a maximum number of 10 classes trained at once and the transitions between different degrees of freedom increased as the study progressed, indicating smooth and efficient control of the advanced prosthesis. Variability of decoding accuracy also decreased with prosthesis usage (p < 0.001) and 30% of the time was spent performing a prosthesis movement. During clinical evaluations, Box and Blocks and the Assessment of the Capacity for Myoelectric Control (ACMC) scores increased by 43% and 6.2%, respectively, demonstrating prosthesis functionality and the NASA Task Load Index (NASA-TLX) scores decreased, on average, by 25% across assessments, indicating reduced cognitive workload while using the MPL, over the nine week study.

SIGNIFICANCE

In this case study, we demonstrate that an onboard system to monitor prosthesis usage enables better understanding of how prostheses are incorporated into daily life. That knowledge can support the long-term goal of completely restoring independence and quality of life to individuals living with upper limb amputation.

Recurrent Convolutional Neural Networks as an Approach to Position-Aware Myoelectric Prosthesis Control

OBJECTIVE

Persons with normal arm function can perform complex wrist and hand movements over a wide range of limb positions. However, for those with transradial amputation who use myoelectric prostheses, control across multiple limb positions can be challenging, frustrating, and can increase the likelihood of device abandonment. In response, the goal of this research was to investigate recurrent convolutional neural network (RCNN)-based position-aware myoelectric prosthesis control strategies.

METHODS

Surface electromyographic (EMG) and inertial measurement unit (IMU) signals, obtained from 16 non-disabled participants wearing two Myo armbands, served as inputs to RCNN classification and regression models. Such models predicted movements (wrist flexion/extension and forearm pronation/supination), based on a multi-limb-position training routine. RCNN classifiers and RCNN regressors were compared to linear discriminant analysis (LDA) classifiers and support vector regression (SVR) regressors, respectively. Outcomes were examined to determine whether RCNN-based control strategies could yield accurate movement predictions, while using the fewest number of available Myo armband data streams.

RESULTS

An RCNN classifier (trained with forearm EMG data, and forearm and upper arm IMU data) predicted movements with 99.00% accuracy (versus the LDAs 97.67%). An RCNN regressor (trained with forearm EMG and IMU data) predicted movements with R2 values of 84.93% for wrist flexion/extension and 84.97% for forearm pronation/supination (versus the SVRs 77.26% and 60.73%, respectively). The control strategies that employed these models required fewer than all available data streams.

CONCLUSION

RCNN-based control strategies offer novel means of mitigating limb position challenges.

SIGNIFICANCE

This research furthers the development of improved position-aware myoelectric prosthesis control.

An Adaptive Multi-Modal Control Strategy to Attenuate the Limb Position Effect in Myoelectric Pattern Recognition

Over the last few decades, pattern recognition algorithms have shown promising results in the field of upper limb prostheses myoelectric control and are now gradually being incorporated in commercial devices. A widely used approach is based on a classifier which assigns a specific input value to a selected hand motion. While this method guarantees good performance and robustness within each class, it still shows limitations in adapting to different conditions encountered in real-world applications, such as changes in limb position or external loads. This paper proposes an adaptive method based on a pattern recognition classifier that takes advantage of an augmented dataset-i.e., representing variations in limb position or external loads-to selectively adapt to underrepresented variations. The proposed method was evaluated using a series of target achievement control tests with ten able-bodied volunteers. Results indicated a higher median completion rate >3.33% for the adapted algorithm compared to a classical pattern recognition classifier used as a baseline model. Subject-specific performance showed the potential for improved control after adaptation and a ≤13% completion rate; and in many instances, the adapted points were able to provide new information within classes. These preliminary results show the potential of the proposed method and encourage further development.

Design, Construction and Tests of a Low-Cost Myoelectric Thumb

Myoelectric signals can be used to control prostheses or exoskeletons as well as robots, i.e., devices assisting the user or replacing a missing part of the body. A typical application of myoelectric prostheses is the human hand. Here, the development of a low-cost myoelectric thumb is described, which can either be used as an additional finger or as prosthesis. Combining 3D printing with inexpensive sensors, electrodes, and electronics, the recent project offers the possibility to produce an individualized myoelectric thumb at significantly lower costs than commercial myoelectric prostheses. Alternatively, a second thumb may be supportive for people with special manual tasks. These possibilities are discussed together with disadvantages of a second thumb and drawbacks of the low-cost solution in terms of mechanical properties and wearing comfort. The study shows that a low-cost customized myoelectric thumb can be produced in this way, but further research on controlling the thumb as well as improving motorization are necessarily to make it fully usable for daily tasks.

Brain-Machine Interfaces: Lessons for Prosthetic Hand Control

Brain-machine interfaces (BMI) are being developed to restore upper limb function for persons with spinal cord injury or other motor degenerative conditions. BMI and implantable sensors for myoelectric prostheses directly extract information from the central or peripheral nervous system to provide users with high fidelity control of their prosthetic device. Control algorithms have been highly transferable between the 2 technologies but also face common issues. In this review of the current state of the art in each field, the authors point out similarities and differences between the 2 technologies that may guide the implementation of common solutions to these challenges.

Electrode Shifts Estimation and Adaptive Correction for Improving Robustness of sEMG-Based Recognition

In sEMG-based recognition systems, accuracy is severely worsened by disturbances, such as electrode shifts by doffing/donning. Traditional recognition models are fixed or static, with limited abilities to work in the presence of the disturbances. In this paper, a transfer learning method is proposed to reduce the impact of electrode shifts. In the proposed method, a novel activation angle is introduced to locate electrodes within a polar coordinate system. An adaptive transformation is utilized to correct electrode-shifted sEMG samples. The transformation is based on estimated shifts relative to the initial position. The experiments acquisition data from ten subjects consist of sEMG signals under eight gestures in seven or nine arbitrary positions, and recorded shifts from a 3D-printed annular ruler. In our extensive experiments, the errors between recorded shifts (as the reference) and estimated shifts is about -0.017±0.13 radians. Eight gestures recognition results have shown an average accuracy around 79.32%, which represents a significant improvement over the 35.72% ( ) average accuracy of results obtained using nonadaptive models, and 60.99% ( ) results of the other method iGLCM (an improved gray-level co-occurrence matrix). More importantly, by only using one-label samples, the proposed method updates the pre-trained model in an initial position. As a result, the pre-trained model can be adaptively corrected to recognize eight-label gestures in arbitrarily rotary positions. It is proven a highly efficient way to relieve subjects' re-training burden of sEMG-based rehabilitation systems.

Activities of daily living with bionic arm improved by combination training and latching filter in prosthesis control comparison

BACKGROUND

Advanced prostheses can restore function and improve quality of life for individuals with amputations. Unfortunately, most commercial control strategies do not fully utilize the rich control information from residual nerves and musculature. Continuous decoders can provide more intuitive prosthesis control using multi-channel neural or electromyographic recordings. Three components influence continuous decoder performance: the data used to train the algorithm, the algorithm, and smoothing filters on the algorithm's output. Individual groups often focus on a single decoder, so very few studies compare different decoders using otherwise similar experimental conditions.

METHODS

We completed a two-phase, head-to-head comparison of 12 continuous decoders using activities of daily living. In phase one, we compared two training types and a smoothing filter with three algorithms (modified Kalman filter, multi-layer perceptron, and convolutional neural network) in a clothespin relocation task. We compared training types that included only individual digit and wrist movements vs. combination movements (e.g., simultaneous grasp and wrist flexion). We also compared raw vs. nonlinearly smoothed algorithm outputs. In phase two, we compared the three algorithms in fragile egg, zipping, pouring, and folding tasks using the combination training and smoothing found beneficial in phase one. In both phases, we collected objective, performance-based (e.g., success rate), and subjective, user-focused (e.g., preference) measures.

RESULTS

Phase one showed that combination training improved prosthesis control accuracy and speed, and that the nonlinear smoothing improved accuracy but generally reduced speed. Phase one importantly showed simultaneous movements were used in the task, and that the modified Kalman filter and multi-layer perceptron predicted more simultaneous movements than the convolutional neural network. In phase two, user-focused metrics favored the convolutional neural network and modified Kalman filter, whereas performance-based metrics were generally similar among all algorithms.

CONCLUSIONS

These results confirm that state-of-the-art algorithms, whether linear or nonlinear in nature, functionally benefit from training on more complex data and from output smoothing. These studies will be used to select a decoder for a long-term take-home trial with implanted neuromyoelectric devices. Overall, clinical considerations may favor the mKF as it is similar in performance, faster to train, and computationally less expensive than neural networks.

A Transferable Adaptive Domain Adversarial Neural Network for Virtual Reality Augmented EMG-Based Gesture Recognition

Within the field of electromyography-based (EMG) gesture recognition, disparities exist between the offline accuracy reported in the literature and the real-time usability of a classifier. This gap mainly stems from two factors: 1) The absence of a controller, making the data collected dissimilar to actual control. 2) The difficulty of including the four main dynamic factors (gesture intensity, limb position, electrode shift, and transient changes in the signal), as including their permutations drastically increases the amount of data to be recorded. Contrarily, online datasets are limited to the exact EMG-based controller used to record them, necessitating the recording of a new dataset for each control method or variant to be tested. Consequently, this paper proposes a new type of dataset to serve as an intermediate between offline and online datasets, by recording the data using a real-time experimental protocol. The protocol, performed in virtual reality, includes the four main dynamic factors and uses an EMG-independent controller to guide movements. This EMG-independent feedback ensures that the user is in-the-loop during recording, while enabling the resulting dynamic dataset to be used as an EMG-based benchmark. The dataset is comprised of 20 able-bodied participants completing three to four sessions over a period of 14 to 21 days. The ability of the dynamic dataset to serve as a benchmark is leveraged to evaluate the impact of different recalibration techniques for long-term (across-day) gesture recognition, including a novel algorithm, named TADANN. TADANN consistently and significantly ( [Formula: see text]) outperforms using fine-tuning as the recalibration technique.

Feedback-aided data acquisition improves myoelectric control of a prosthetic hand

OBJECTIVE

Pattern-recognition-based myocontrol can be unreliable, which may limit its use in the clinical practice and everyday activities. One cause for this is the poor generalization of the underlying machine learning models to untrained conditions. Acquiring the training data and building the model more interactively can reduce this problem. For example, the user could be encouraged to target the model's instabilities during the data acquisition supported by automatic feedback guidance. Interactivity is an emerging trend in myocontrol of upper-limb electric prostheses: the user should be actively involved throughout the training and usage of the device.

APPROACH

In this study, 18 non-disabled participants tested two novel feedback-aided acquisition protocols against a standard one that did not provide any guidance. All the protocols acquired data dynamically in multiple arm positions to counteract the limb position effect. During feedback-aided acquisition, an acoustic signal urged the participant to hover with the arm in specific regions of her peri-personal space, de facto acquiring more data where needed. The three protocols were compared on everyday manipulation tasks performed with a prosthetic hand.

MAIN RESULTS

Our results showed that feedback-aided data acquisition outperformed the acquisition routine without guidance, both objectively and subjectively.

SIGNIFICANCE

This indicates that the interaction with the user during the data acquisition is fundamental to improve myocontrol.

Surgically Implanted Electrodes Enable Real-Time Finger and Grasp Pattern Recognition for Prosthetic Hands

Currently available prosthetic hands are capable of actuating anywhere from five to 30 degrees of freedom (DOF). However, grasp control of these devices remains unintuitive and cumbersome. To address this issue, we propose directly extracting finger commands from the neuromuscular system via electrodes implanted in residual innervated muscles and regenerative peripheral nerve interfaces (RPNIs). Two persons with transradial amputations had RPNIs created by suturing autologous free muscle grafts to their transected median, ulnar, and dorsal radial sensory nerves. Bipolar electrodes were surgically implanted into their ulnar and median RPNIs and into their residual innervated muscles. The implanted electrodes recorded local electromyography (EMG) with Signal-to-Noise Ratios ranging from 23 to 350 measured across various movements. In a series of single-day experiments, participants used a high speed pattern recognition system to control a virtual prosthetic hand in real-time. Both participants were able to transition between 10 pseudo-randomly cued individual finger and wrist postures in the virtual environment with an average online accuracy of 86.5% and latency of 255 ms. When the set was reduced to five grasp postures, average metrics improved to 97.9% online accuracy and 135 ms latency. Virtual task performance remained stable across untrained static arm positions while supporting the weight of the prosthesis. Participants also used the high speed classifier to switch between robotic prosthetic grips and complete a functional performance assessment. These results demonstrate that pattern recognition systems can use the high-quality EMG afforded by intramuscular electrodes and RPNIs to provide users with fast and accurate grasp control.

SUMMARY

Surgically implanted electrodes recorded finger-specific electromyography enabling reliable finger and grasp control of an upper limb prosthesis.

Sensory stimulation enhances phantom limb perception and movement decoding

OBJECTIVE

A major challenge for controlling a prosthetic arm is communication between the device and the user's phantom limb. We show the ability to enhance phantom limb perception and improve movement decoding through targeted transcutaneous electrical nerve stimulation in individuals with an arm amputation.

APPROACH

Transcutaneous nerve stimulation experiments were performed with four participants with arm amputation to map phantom limb perception. We measured myoelectric signals during phantom hand movements before and after participants received sensory stimulation. Using electroencephalogram (EEG) monitoring, we measured the neural activity in sensorimotor regions during phantom movements and stimulation. In one participant, we also tracked sensory mapping over 2 years and movement decoding performance over 1 year.

MAIN RESULTS

Results show improvements in the participants' ability to perceive and move the phantom hand as a result of sensory stimulation, which leads to improved movement decoding. In the extended study with one participant, we found that sensory mapping remains stable over 2 years. Sensory stimulation improves within-day movement decoding while performance remains stable over 1 year. From the EEG, we observed cortical correlates of sensorimotor integration and increased motor-related neural activity as a result of enhanced phantom limb perception.

SIGNIFICANCE

This work demonstrates that phantom limb perception influences prosthesis control and can benefit from targeted nerve stimulation. These findings have implications for improving prosthesis usability and function due to a heightened sense of the phantom hand.

Hybrid Impedance-Admittance Control for Upper Limb Exoskeleton Using Electromyography

Exoskeletons are wearable mobile robots that combine various technologies to enable limb movement with greater strength and endurance, being used in several application areas, such as industry and medicine. In this context, this paper presents the development of a hybrid control method for exoskeletons, combining admission and impedance control based on electromyographic input signals. A proof of concept of a robotic arm with two degrees of freedom, mimicking the functions of a human’s upper limb, was built to evaluate the proposed control system. Through tests that measured the discrepancy between the angles of the human joint and the joint of the exoskeleton, it was possible to determine that the system remained within an acceptable error range. The average error is lower than 4.3%, and the robotic arm manages to mimic the movements of the upper limbs of a human in real-time.

Resilient EMG Classification to Enable Reliable Upper-Limb Movement Intent Detection

Reliable control of assistive devices using surface electromyography (sEMG) remains an unsolved task due to the signal's stochastic behavior that prevents robust pattern recognition for real-time control. Non-representative samples lead to inherent class overlaps that generate classification ripples for which the most common alternatives rely on post-processing and sample discard methods that insert additional delays and often do not offer substantial improvements. In this paper, a resilient classification pipeline based on Extreme Learning Machines (ELM) was used to classify 17 different upper-limb movements through sEMG signals from a total of 99 trials derived from three different databases. The method was compared to a baseline ELM and a sample discarding (DISC) method and proved to generate more stable and consistent classifications. The average accuracy boost of ≈ 10% in all databases lead to average weighted accuracy rates higher as 53,4% for amputees and 89,0% for non-amputee volunteers. The results match or outperform related works even without sample discards.

Should Hands Be Restricted When Measuring Able-Bodied Participants to Evaluate Machine Learning Controlled Prosthetic Hands?

OBJECTIVE

When evaluating methods for machine-learning controlled prosthetic hands, able-bodied participants are often recruited, for practical reasons, instead of participants with upper limb absence (ULA). However, able-bodied participants have been shown to often perform myoelectric control tasks better than participants with ULA. It has been suggested that this performance difference can be reduced by restricting the wrist and hand movements of able-bodied participants. However, the effect of such restrictions on the consistency and separability of the electromyogram's (EMG) features remains unknown. The present work investigates whether the EMG separability and consistency between unaffected and affected arms differ and whether they change after restricting the unaffected limb in persons with ULA.

METHODS

Both arms of participants with unilateral ULA were compared in two conditions: with the unaffected hand and wrist restricted or not. Furthermore, it was tested if the effect of arm and restriction is influenced by arm posture (arm down, arm in front, or arm up).

RESULTS

Fourteen participants (two women, age = 53.4±4.05) with acquired transradial limb loss were recruited. We found that the unaffected limb generated more separated EMG than the affected limb. Furthermore, restricting the unaffected hand and wrist lowered the separability of the EMG when the arm was held down.

CONCLUSION

Limb restriction is a viable method to make the EMG of able-bodied participants more similar to that of participants with ULA.

SIGNIFICANCE

Future research that evaluates methods for machine learning controlled hands in able-bodied participants should restrict the participants' hand and wrist.

Robustness of Muscle Synergies under Variant Muscle Contraction Force during Forearm Movements

The Electromyography-based Pattern-Recognition (EMG-PR) framework has been investigated for almost three decades towards developing an intuitive myoelectric prosthesis. To utilize the knowledge of the underlying neurophysiological processes of natural movements, the concept of muscle synergy has been applied in prosthesis control and proved to be of great potential recently. For a muscle-synergy-based myoelectric system, the variation of muscle contraction force is also a confounding factor. This study evaluates the robustness of muscle synergies under a variant force level for forearm movements. Six channels of forearm surface EMG were recorded from six healthy subjects when they performed 4 movements (hand open, hand close, wrist flexion, and wrist extension) using low, moderate, and high force, respectively. Muscle synergies were extracted from the EMG using the alternating nonnegativity constrained least squares and active set (NNLS) algorithm. Three analytic strategies were adopted to examine whether muscle synergies were conserved under different force levels. Our results consistently showed that there exists fixed, robust muscle synergies across force levels. This outcome would provide valuable insights to the implementation of muscle- synergy-based assistive technology for the upper extremity.

Understanding Limb Position and External Load Effects on Real-Time Pattern Recognition Control in Amputees

Limb position is a factor that negatively affects myoelectric pattern recognition classification accuracy. However, prior studies evaluating impact on real-time control for upper-limb amputees have done so without a physical prosthesis on the residual limb. It remains unclear how limb position affects real-time pattern recognition control in amputees when their residual limb is supporting various weights. We used a virtual reality target achievement control test to evaluate the effects of limb position and external load on real-time pattern recognition control in fourteen intact limb subjects and six major upper limb amputee subjects. We also investigated how these effects changed based on different control system training methods. In a static training method, subjects kept their unloaded arm by their side with the elbow bent whereas in the dynamic training method, subjects moved their arm throughout a workspace while supporting a load. When static training was used, limb position significantly affected real-time control in all subjects. However, amputee subjects were still able to adequately complete tasks in all conditions, even in untrained limb positions. Moreover, increasing external loads decreased controller performance, albeit to a lesser extent in amputee subjects. The effects of limb position did not change as load increased, and vice versa. In intact limb subjects, dynamic training significantly reduced the limb position effect but did not completely remove them. In contrast, in amputee subjects, dynamic training eliminated the limb position effect in three out of four outcome measures. However, it did not reduce the effects of load for either subject population. These findings suggest that results obtained from intact limb subjects may not generalize to amputee subjects and that advanced training methods can substantially improve controller robustness to different limb positions regardless of limb loading.

Smoothed arg max Extreme Learning Machine: An Alternative to Avoid Classification Ripple in sEMG Signals

Despite all the recent developments of using the surface electromyography (sEMG) as a control signal, reliable classifications still remain an arduous task due to overlapping classes and classification ripples. In this paper, we present a straightforward approach to avoid classification ripple based on smoothing the arg max value of an Extreme Learning Machine (ELM) classifier. We compare the baseline accuracy of the classifier with an arg max filtered by a traditional Exponential Smoothing Filter (ESF) and our adaptation of Antonyan Vardan Transform (AVT). The classifiers were evaluated using sEMG data acquired through 12 channels from four subjects performing 17 different movements of forearm and fingers with three repetitions each. In the best scenario, our methods reached results higher than 96% and 82% of overall and weighted accuracy, respectively. Those results match or outperform similar papers of the literature using a simpler model, which may help the application of the techniques on embedded platforms and make the practical use of such devices more feasible.

Stable Responsive EMG Sequence Prediction and Adaptive Reinforcement With Temporal Convolutional Networks

Prediction of movement intentions from electromyographic (EMG) signals is typically performed with a pattern recognition approach, wherein a short dataframe of raw EMG is compressed into an instantaneous feature-encoding that is meaningful for classification. However, EMG signals are time-varying, implying that a frame-wise approach may not sufficiently incorporate temporal context into predictions, leading to erratic and unstable prediction behavior.

OBJECTIVE

We demonstrate that sequential prediction models and, specifically, temporal convolutional networks are able to leverage useful temporal information from EMG to achieve superior predictive performance.

METHODS

We compare this approach to other sequential and frame-wise models predicting 3 simultaneous hand and wrist degrees-of-freedom from 2 amputee and 13 non-amputee human subjects in a minimally constrained experiment. We also compare these models on the publicly available Ninapro and CapgMyo amputee and non-amputee datasets.

RESULTS

Temporal convolutional networks yield predictions that are more accurate and stable than frame-wise models, especially during inter-class transitions, with an average response delay of 4.6 ms and simpler feature-encoding. Their performance can be further improved with adaptive reinforcement training.

SIGNIFICANCE

Sequential models that incorporate temporal information from EMG achieve superior movement prediction performance and these models allow for novel types of interactive training.

CONCLUSIONS

Addressing EMG decoding as a sequential modeling problem will lead to enhancements in the reliability, responsiveness, and movement complexity available from prosthesis control systems.

The Merits of Dynamic Data Acquisition for Realistic Myocontrol

Natural myocontrol is the intuitive control of a prosthetic limb via the user's voluntary muscular activations. This type of control is usually implemented by means of pattern recognition, which uses a set of training data to create a model that can decipher these muscular activations. A consequence of this approach is that the reliability of a myocontrol system depends on how representative this training data is for all types of signal variability that may be encountered when the amputee puts the prosthesis into real use. Myoelectric signals are indeed known to vary according to the position and orientation of the limb, among other factors, which is why it has become common practice to take this variability into account by acquiring training data in multiple body postures. To shed further light on this problem, we compare two ways of collecting data: while the subjects hold their limb statically in several positions one at a time, which is the traditional way, or while they dynamically move their limb at a constant pace through those same positions. Since our interest is to investigate any differences when controlling an actual prosthetic device, we defined an evaluation protocol that consisted of a series of complex, bimanual daily-living tasks. Fourteen intact participants performed these tasks while wearing prosthetic hands mounted on splints, which were controlled via either a statically or dynamically built myocontrol model. In both cases all subjects managed to complete all tasks and participants without previous experience in myoelectric control manifested a significant learning effect; moreover, there was no significant difference in the task completion times achieved with either model. When evaluated in a simulated scenario with traditional offline performance evaluation, on the other hand, the dynamically-trained system showed significantly better accuracy. Regardless of the setting, the dynamic data acquisition was faster, less tiresome, and better accepted by the users. We conclude that dynamic data acquisition is advantageous and confirm the limited relevance of offline analyses for online myocontrol performance.

Modulation of muscle synergies for multiple forearm movements under variant force and arm position constraints

OBJECTIVE

To promote clinical applications of muscle-synergy-based neurorehabilitation techniques, this study aims to clarify any potential modulations of both the muscular compositions and temporal activations of forearm muscle synergies for multiple movements under variant force levels and arm positions.

APPROACH

Two groups of healthy subjects participated in this study. Electromyography (EMG) signals were collected when they performed four hand and wrist movements under variant constraints-three different force levels for one group and five arm positions for the other. Muscle synergies were extracted from the EMGs, and their robustness across variant force levels and arm positions was separately assessed by evaluating their across-condition structure similarity, cross-validation, and cluster analysis. The synergies' activation coefficients across the variant constraints were also compared, and the coefficients were used to discriminate the different force levels and the arm positions, respectively.

MAIN RESULTS

Overall, the muscle synergies were relatively fixed across variant constraints, but they were more robust to variant forces than to changing arm positions. The activations of muscle synergies depended largely on the level of contraction force and could discriminate the force levels very well, but the coefficients corresponding to different arm positions discriminated the positions with lower accuracy. Similar results were found for all types of forearm movement analyzed.

SIGNIFICANCE

With our experiment and subject-specific analysis, only slight modulation of the muscular compositions of forearm muscle synergies was found under variant force and arm position constraints. Our results may shed valuable insights to future design of both muscle-synergy-based assistive robots and motor-function assessments.

Current Trends and Confounding Factors in Myoelectric Control: Limb Position and Contraction Intensity

This manuscript presents a hybrid study of a comprehensive review and a systematic(research) analysis. Myoelectric control is the cornerstone ofmany assistive technologies used in clinicalpractice, such as prosthetics and orthoses, and human-computer interaction, such as virtual reality control.Although the classification accuracy of such devices exceeds 90% in a controlled laboratory setting,myoelectric devices still face challenges in robustness to variability of daily living conditions.The intrinsic physiological mechanisms limiting practical implementations of myoelectric deviceswere explored: the limb position effect and the contraction intensity effect. The degradationof electromyography (EMG) pattern recognition in the presence of these factors was demonstratedon six datasets, where classification performance was 13% and 20% lower than the controlledsetting for the limb position and contraction intensity effect, respectively. The experimental designsof limb position and contraction intensity literature were surveyed. Current state-of-the-art trainingstrategies and robust algorithms for both effects were compiled and presented. Recommendationsfor future limb position effect studies include: the collection protocol providing exemplars of at least 6positions (four limb positions and three forearm orientations), three-dimensional space experimentaldesigns, transfer learning approaches, and multi-modal sensor configurations. Recommendationsfor future contraction intensity effect studies include: the collection of dynamic contractions, nonlinearcomplexity features, and proportional control.

A Myoelectric Postural Control Algorithm for Persons With Transradial Amputations: A Consideration of Clinical Readiness

BACKGROUND

The bottleneck in upper limb prosthetic design is the myoelectric control algorithm. Here we studied the clinical readiness of the myoelectric postural control algorithm in a laboratory setting with two trans-radial amputees using a commercially available prosthetic limb system.

TECHNIQUE

The postural control algorithm was integrated into prosthetic limb systems using standard of care components. A comparison between a commercial state of the art system (the i-limb revolution state-based myoelectric controller) and the postural controller was performed with two people with trans-radial amputation using a self-contained prosthesis system.

DISCUSSION

The performance using the i-limb revolution state-based controller versus the postural controller was mixed based on the Southampton Hand Assessment Procedure. The SHAP scores indicate that the postural controller with i-limb revolution provided an average of 66% of hand function compared to an intact limb. Future work will study the advantages of the postural control algorithm in everyday use.

Online Natural Myocontrol of Combined Hand and Wrist Actions Using Tactile Myography and the Biomechanics of Grasping

Objective: Despite numerous recent advances in the field of rehabilitation robotics, simultaneous, and proportional control of hand and/or wrist prostheses is still unsolved. In this work we concentrate on myocontrol of combined actions, for instance power grasping while rotating the wrist, by only using training data gathered from single actions. This is highly desirable since gathering data for all possible combined actions would be unfeasibly long and demanding for the amputee. Approach: We first investigated physiologically feasible limits for muscle activation during combined actions. Using these limits we involved 12 intact participants and one amputee in a Target Achievement Control test, showing that tactile myography, i.e., high-density force myography, solves the problem of combined actions to a remarkable extent using simple linear regression. Since real-time usage of many sensors can be computationally demanding, we compare this approach with another one using a reduced feature set. These reduced features are obtained using a fast, spatial first-order approximation of the sensor values. Main results: By using the training data of single actions only, i.e., power grasp or wrist movements, subjects achieved an average success rate of 70.0% in the target achievement test using ridge regression. When combining wrist actions, e.g., pronating and flexing the wrist simultaneously, similar results were obtained with an average of 68.1%. If a power grasp is added to the pool of actions, combined actions are much more difficult to achieve (36.1%). Significance: To the best of our knowledge, for the first time, the effectiveness of tactile myography on single and combined actions is evaluated in a target achievement test. The present study includes 3 DoFs control instead of the two generally used in the literature. Additionally, we define a set of physiologically plausible muscle activation limits valid for most experiments of this kind.

Biceps Brachii Muscle Synergy and Target Reaching in a Virtual Environment

A muscular synergy is a theory suggesting that the central nervous system uses few commands to activate a group of muscles to produce a given movement. Here, we investigate how a muscle synergy extracted from a single muscle can be at the origin of different signals which could facilitate the control of modern upper limb myoelectric prostheses with many degrees of freedom. Five pairs of surface electrodes were positioned across the biceps of 12 normal subjects and electromyographic (EMG) signals were collected while their upper limbs were in eight different static postures. Those signals were used to move, within a virtual cube, a small red sphere toward different targets. With three muscular synergies extracted from the five EMG signals, a classifier was trained to identify which synergy pattern was associated with a given static posture. Later, when a posture was recognized, the result was a displacement of a red sphere toward a corner of a virtual cube presented on a computer screen. The axes of the cube were assigned to the shoulder, elbow and wrist joint while each of its the corners was associated with a static posture. The goal for subjects was to reach, one at a time, the four targets positioned at different locations and heights in the virtual cube with different sequences of postures. The results of 12 normal subjects indicate that with the muscular synergies of the biceps brachii, it was possible, but not easy for an untrained person, to reach a target on each trial. Thus, as a proof of concept, we show that features of the biceps muscular synergy have the potential to facilitate the control of upper limb myoelectric prostheses. To our knowledge, this has never been shown before.

Myoelectric Control for Upper Limb Prostheses

State-of-the-art high-end prostheses are electro-mechanically able to provide a great variety of movements. Nevertheless, in order to functionally replace a human limb, it is essential that each movement is properly controlled. This is the goal of prosthesis control, which has become a growing research field in the last decades, with the ultimate goal of reproducing biological limb control. Therefore, exploration and development of prosthesis control are crucial to improve many aspects of an amputee’s life. Nowadays, a large divergence between academia and industry has become evident in commercial systems. Although several studies propose more natural control systems with promising results, basic one degree of freedom (DoF), a control switching system is the most widely used option in industry because of simplicity, robustness and inertia. A few classification controlled prostheses have emerged in the last years but they are still a low percentage of the used ones. One of the factors that generate this situation is the lack of robustness of more advanced control algorithms in daily life activities outside of laboratory conditions. Because of this, research has shifted towards more functional prosthesis control. This work reviews the most recent literature in upper limb prosthetic control. It covers commonly used variants of possible biological inputs, its processing and translation to actual control, mostly focusing on electromyograms as well as the problems it will have to overcome in near future.

Real-Time EMG Based Pattern Recognition Control for Hand Prostheses: A Review on Existing Methods, Challenges and Future Implementation

Upper limb amputation is a condition that significantly restricts the amputees from performing their daily activities. The myoelectric prosthesis, using signals from residual stump muscles, is aimed at restoring the function of such lost limbs seamlessly. Unfortunately, the acquisition and use of such myosignals are cumbersome and complicated. Furthermore, once acquired, it usually requires heavy computational power to turn it into a user control signal. Its transition to a practical prosthesis solution is still being challenged by various factors particularly those related to the fact that each amputee has different mobility, muscle contraction forces, limb positional variations and electrode placements. Thus, a solution that can adapt or otherwise tailor itself to each individual is required for maximum utility across amputees. Modified machine learning schemes for pattern recognition have the potential to significantly reduce the factors (movement of users and contraction of the muscle) affecting the traditional electromyography (EMG)-pattern recognition methods. Although recent developments of intelligent pattern recognition techniques could discriminate multiple degrees of freedom with high-level accuracy, their efficiency level was less accessible and revealed in real-world (amputee) applications. This review paper examined the suitability of upper limb prosthesis (ULP) inventions in the healthcare sector from their technical control perspective. More focus was given to the review of real-world applications and the use of pattern recognition control on amputees. We first reviewed the overall structure of pattern recognition schemes for myo-control prosthetic systems and then discussed their real-time use on amputee upper limbs. Finally, we concluded the paper with a discussion of the existing challenges and future research recommendations.

Learn the Temporal-Spatial Feature of sEMG via Dual-Flow Network

Surface electromyography (sEMG) signals have been widely used in human–machine interaction, providing more nature control expedience for external devices. However, due to the instability of sEMG, it is hard to extract consistent sEMG patterns for motion recognition. This paper proposes a dual-flow network to extract the temporal-spatial feature of sEMG for gesture recognition. The proposed network model uses convolutional neural network (CNN) and long short-term memory methods (LSTM) to, respectively, extract the spatial feature and temporal feature of sEMG, simultaneously. These features extracted by CNN and LSTM are merged into temporal-spatial feature to form an end-to-end network. A dataset was constructed for testing the performance of the network. In this database, the average recognition accuracy by using our dual-flow model reached 78.31%, which was improved by 6.69% compared to the baseline CNN (71.67%). In addition, NinaPro DB1 is also used to evaluate the proposed methods, receiving 1.86% higher recognition accuracy than the baseline CNN classifier. It is believed that the proposed dual-flow network owns the merit in extracting stable sEMG feature for gesture recognition, and can be further applied into practical applications.

The Effect of Feedback During Training Sessions on Learning Pattern-Recognition-Based Prosthesis Control

Human-machine interfaces have not yet advanced to enable intuitive control of multiple degrees of freedom as offered by modern myoelectric prosthetic hands. Pattern Recognition (PR) control has been proposed to make human-machine interfaces in myoelectric prosthetic hands more intuitive, but it requires the user to generate high-quality, i.e., consistent and separable, electromyogram (EMG) patterns. To generate such patterns, user training is required and has shown promising results. However, how different levels of feedback affect effectivity in training differently, has not been established yet. Furthermore, a correlation between qualities of the EMG patterns (the focus of training) and user performance has not been shown yet. In this study, 37 able-bodied participants (mean age 21 years, 19 males) were recruited and trained PR control over five days. Three levels of feedback were tested for their effectiveness: no external feedback, visual feedback and visual feedback with coaching. Training resulted in improved performance from pre- to post-test with no interaction effect of feedback. Feedback did however affect the quality of the EMG patterns where people who did not receive external feedback generated higher amplitude patterns. A weak correlation was found between a principal component, composed of EMG amplitude and pattern variability, and performance. Our results show that training is highly effective in improving PR control regardless of feedback and that none of the quality metrics correlate with performance. We discuss how different levels of feedback can be leveraged to improve PR control training.

Enhanced Performance for Multi-Forearm Movement Decoding Using Hybrid IMU-sEMG Interface

Control of active prosthetic hands using surface electromyography (sEMG) signals is an active research area; despite the advances in sEMG pattern recognition and classification techniques, none of the commercially available prosthetic hands provide the user with an intuitive control. One of the major reasons for this disparity between academia and industry is the variation of sEMG signals in a dynamic environment as opposed to the controlled laboratory conditions. This research investigated the effects of sEMG signal variation on the performance of a hand motion classifier due to arm position variation and also explored the effect of static position and dynamic movement strategies for classifier training. A wearable system is used to measure the electrical activity of the muscles and the position of the forearm while performing six classes of hand motions. The system is made position aware (POS) using inertial measurement units (IMUs) for different arm movement gestures. The hand gestures are decoded under both static and dynamic forearm movements. Four time domain (TD) features are extracted from the sEMG signals along with IMU-based arm position information. The features are trained and tested using linear discriminant analysis (LDA) and support vector machine (SVM) for both TD and TD-POS features. The results for the SVM show a significant difference between the static and dynamic approaches, while the TD-POS features show enhanced classification performance in comparison to the TD-based classification. Results have shown the effectiveness of the dynamic training approach and sensor fusion techniques to improve the performance of existing stand-alone sEMG-based prosthetic control systems.

A myoelectric prosthetic hand with muscle synergy–based motion determination and impedance model–based biomimetic control

Prosthetic hands are prescribed to patients who have suffered an amputation of the upper limb due to an accident or a disease. This is done to allow patients to regain functionality of their lost hands. Myoelectric prosthetic hands were found to have the possibility of implementing intuitive controls based on operator’s electromyogram (EMG) signals. These controls have been extensively studied and developed. In recent years, development costs and maintainability of prosthetic hands have been improved through three-dimensional (3D) printing technology. However, no previous studies have realized the advantages of EMG-based classification of multiple finger movements in conjunction with the introduction of advanced control mechanisms based on human motion. This paper proposes a 3D-printed myoelectric prosthetic hand and an accompanying control system. The muscle synergy–based motion-determination method and biomimetic impedance control are introduced in the proposed system, enabling the classification of unlearned combined motions and smooth and intuitive finger movements of the prosthetic hand. We evaluate the proposed system through operational experiments performed on six healthy participants and an upper-limb amputee participant. The experimental results demonstrate that our prosthetic hand system can successfully classify both learned single motions and unlearned combined motions from EMG signals with a high degree of accuracy. Furthermore, applications to real-world uses of prosthetic hands are demonstrated through control tasks conducted by the amputee participant.

Differentiating Variations in Thumb Position From Recordings of the Surface Electromyogram in Adults Performing Static Grips, a Proof of Concept Study

Hand gesture and grip formations are produced by the muscle synergies arising between extrinsic and intrinsic hand muscles and many functional hand movements involve repositioning of the thumb relative to other digits. In this study we explored whether changes in thumb posture in able-body volunteers can be identified and classified from the modulation of forearm muscle surface-electromyography (sEMG) alone without reference to activity from the intrinsic musculature. In this proof-of-concept study, our goal was to determine if there is scope to develop prosthetic hand control systems that may incorporate myoelectric thumb-position control. Healthy volunteers performed a controlled-isometric grip task with their thumb held in four different opposing-postures. Grip force during task performance was maintained at 30% maximal-voluntary-force and sEMG signals from the forearm were recorded using 2D high-density sEMG (HD-sEMG arrays). Correlations between sEMG amplitude and root-mean squared estimates with variation in thumb-position were investigated using principal-component analysis and self-organizing feature maps. Results demonstrate that forearm muscle sEMG patterns possess classifiable parameters that correlate with variations in static thumb position (accuracy of 88.25 ± 0.5% anterior; 91.25 ± 2.5% posterior musculature of the forearm sites). Of importance, this suggests that in transradial amputees, despite the loss of access to the intrinsic muscles that control thumb action, an acceptable level of control over a thumb component within myoelectric devices may be achievable. Accordingly, further work exploring the potential to provide myoelectric control over the thumb within a prosthetic hand is warranted.

Natural Myocontrol in a Realistic Setting: a Comparison Between Static and Dynamic Data Acquisition

Natural myocontrol employs pattern recognition to allow users to control a robotic limb intuitively using their own voluntary muscular activations. The reliability of myocontrol strongly depends on the signals initially collected from the users, which must appropriately capture the variability encountered later on during operation. Since myoelectric signals can vary based on the position and orientation of the limb, it has become best practice to gather data in multiple body postures. We hereby concentrate on this acquisition protocol and investigate the relative merits of collecting data either statically or dynamically. In the static case, data for a desired hand configuration is collected while the users keep their hand still in certain positions, whereas in the dynamic case, data is collected while users move their limbs, passing through the required positions with a roughly constant velocity.Fourteen able-bodied subjects were asked to naturally control two dexterous hand prostheses mounted on splints, performing a set of complex, realistic bimanual activities of daily living. We could not find any significant difference between the protocols in terms of the total execution times, although the dynamic data acquisition was faster and less tiring. This would indicate that dynamic data acquisition should be preferred over the static one.

Adaptive Hybrid Classifier for Myoelectric Pattern Recognition Against the Interferences of Outlier Motion, Muscle Fatigue, and Electrode Doffing

Traditional myoelectric prostheses that employ a static pattern recognition model to identify human movement intention from surface electromyography (sEMG) signals hardly adapt to the changes in the sEMG characteristics caused by interferences from daily activities, which hinders the clinical applications of such prostheses. In this paper, we focus on methods to reduce or eliminate the impacts of three types of daily interferences on myoelectric pattern recognition (MPR), i.e., outlier motion, muscle fatigue, and electrode doffing/donning. We constructed an adaptive incremental hybrid classifier (AIHC) by combining one-class support vector data description and multi-class linear discriminant analysis in conjunction with two specific update schemes. We developed an AIHC-based MPR strategy to improve the robustness of MPR against the three interferences. Extensive experiments on hand-motion recognition were conducted to demonstrate the performance of the proposed method. Experimental results show that the AIHC has significant advantages over non-adaptive classifiers under various interferences, with improvements in the classification accuracy ranging from 7.1% to 39% ( ). The additional evaluations on data deviations demonstrate that the AIHC can accommodate large-scale changes in the sEMG characteristics, revealing the potential of the AIHC-based MPR strategy in the development of clinical myoelectric prostheses.

Neuromorphic vision and tactile fusion for upper limb prosthesis control

A major issue with upper limb prostheses is the disconnect between sensory information perceived by the user and the information perceived by the prosthesis. Advances in prosthetic technology introduced tactile information for monitoring grasping activity, but visual information, a vital component in the human sensory system, is still not fully utilized as a form of feedback to the prosthesis. For able-bodied individuals, many of the decisions for grasping or manipulating an object, such as hand orientation and aperture, are made based on visual information before contact with the object. We show that inclusion of neuromorphic visual information, combined with tactile feedback, improves the ability and efficiency of both able-bodied and amputee subjects to pick up and manipulate everyday objects. We discovered that combining both visual and tactile information in a real-time closed loop feedback strategy generally decreased the completion time of a task involving picking up and manipulating objects compared to using a single modality for feedback. While the full benefit of the combined feedback was partially obscured by experimental inaccuracies of the visual classification system, we demonstrate that this fusion of neuromorphic signals from visual and tactile sensors can provide valuable feedback to a prosthetic arm for enhancing real-time function and usability.

Improving the functionality, robustness, and adaptability of myoelectric control for dexterous motion restoration

The development of advanced and effective human-machine interfaces, especially for amputees to control their prostheses, is very high priority and a very active area of research. An intuitive control method should retain an adequate level of functionality for dexterous operation, provide robustness against confounding factors, and supply adaptability for diverse long-term usage, all of which are current problems being tackled by researchers. This paper reviews the state-of-the-art, as well as, the limitations of current myoelectric signal control (MSC) methods. To address the research topic on functionality, we review different approaches to prosthetic hand control (DOF configuration, discrete or simultaneous, etc.), and how well the control is performed (accuracy, response, intuitiveness, etc.). To address the research on robustness, we review the confounding factors (limb positions, electrode shift, force variance, and inadvertent activity) that affect the stability of the control performance. Lastly, to address adaptability, we review the strategies that can automatically adjust the classifier for different individuals and for long-term usage. This review provides a thorough overview of the current MSC methods and helps highlight the current areas of research focus and resulting clinic usability for the MSC methods for upper-limb prostheses.

Effects of Confidence-Based Rejection on Usability and Error in Pattern Recognition-Based Myoelectric Control

Rejection of movements based on the confidence in the classification decision has previously been demonstrated to improve the usability of pattern recognition based myoelectric control. To this point, however, the optimal rejection threshold has been determined heuristically, and it is not known how different thresholds affect the tradeoff between error mitigation and false rejections in real-time closed-loop control. To answer this question, 24 able-bodied subjects completed a real-time Fitts' law-style virtual cursor control task using a support vector machine classifier. It was found that rejection improved information throughput at all thresholds, with the best performance coming at thresholds between 0.60 and 0.75. Two fundamental types of error were defined and identified: operator error (identifiable, repeatable behaviors, directly attributable to the user), and systemic error (other errors attributable to misclassification or noise). The incidence of both operator and systemic errors were found to decrease as rejection threshold increased. Moreover, while the incidence of all error types correlated strongly with path efficiency, only systemic errors correlated strongly with throughput and trial completion rate. Interestingly, more experienced users were found to commit as many errors as novice users, despite performing better in the Fitts' task, suggesting that there is more to usability than error prevention alone. Nevertheless, these results demonstrate the usability gains possible with rejection across a range of thresholds for both novice and experienced users alike.

A Classification Method for Myoelectric Control of Hand Prostheses Inspired by Muscle Coordination

Dexterous upper limb myoelectric prostheses can, to some extent, restore the motor functions lost after an amputation. However, ensuring the reliability of myoelectric control is still an open challenge. In this paper, we propose a classification method that exploits the regularity in muscle activation patterns (uniform scaling) across different force levels within a given movement class. This assumption leads to a simple training procedure, using training data collected at single contraction intensity for each movement class. The proposed method was compared to the widely accepted benchmark [linear discriminant analysis (LDA) classifier] using off-line and online evaluation. The off-line classification errors obtained with the new method were either lower or higher than LDA depending upon the chosen feature set. In the online evaluation, the new classification method was operated using amplitude-EMG features and compared to the state-of-the-art LDA classifier combined with the time domain feature set. The online evaluation was performed in 11 able-bodied and one amputee subject using a set of four functional tasks mimicking daily-life activities. The tasks assessed the dexterity (e.g., switching between functions) and robustness of control (e.g., handling heavy objects). With the new classification scheme, the amputee performed better in all functional tasks, whereas the able-bodied subjects performed significantly better in three out of four functional tasks. Overall, the novel method outperformed the state-of-the-art approach (LDA) while utilizing less training data and a smaller feature set. The proposed method is, therefore, a simple but effective and robust classification scheme, convenient for online implementation and clinical use.

Designing Feedback Controllers for Human-Prosthetic Systems Using H∞ Model Matching

Prosthetic hands are important tools for improving the lives of upper limb amputees, yet most devices lack the ability to provide a sense of touch back to the user. Recent improvements have been made in electromyography (EMG) prosthesis control as well as in biologically relevant tactile sensors to provide sensory feedback to amputees through nerve stimulation. However, sensory feedback has been designed heuristically, which can lead to either unnatural sensations or to excessive feedback that bothers the user. In this study, we apply optimal control techniques to synthesize sensory feedback to the user, and to synthesize the conversion from EMG to an actuation command to the prosthesis. Specifically, we construct a feedback control system architecture and solve the $H_{\infty }$ model matching problem to make the closed-loop user-prosthetic system to behave like a pre-specified ideal system in response to elemental inputs (e.g. impulse, step, etc). We design feedback controllers assuming that human and prosthetic components behave in a linear fashion as a proof-of-concept, and the closed-loop system is able to match ideal systems that are slow, fast and that have both slow and fast dynamics (like healthy humans).