Surface EMG in advanced hand prosthetics

A Myoelectric Postural Control Algorithm for Persons With Transradial Amputations: A Consideration of Clinical Readiness

BACKGROUND

The bottleneck in upper limb prosthetic design is the myoelectric control algorithm. Here we studied the clinical readiness of the myoelectric postural control algorithm in a laboratory setting with two trans-radial amputees using a commercially available prosthetic limb system.

TECHNIQUE

The postural control algorithm was integrated into prosthetic limb systems using standard of care components. A comparison between a commercial state of the art system (the i-limb revolution state-based myoelectric controller) and the postural controller was performed with two people with trans-radial amputation using a self-contained prosthesis system.

DISCUSSION

The performance using the i-limb revolution state-based controller versus the postural controller was mixed based on the Southampton Hand Assessment Procedure. The SHAP scores indicate that the postural controller with i-limb revolution provided an average of 66% of hand function compared to an intact limb. Future work will study the advantages of the postural control algorithm in everyday use.

The current state of bionic limbs from the surgeon's viewpoint

Amputations have a devastating impact on patients' health with consequent psychological distress, economic loss, difficult reintegration into society, and often low embodiment of standard prosthetic replacement.The main characteristic of bionic limbs is that they establish an interface between the biological residuum and an electronic device, providing not only motor control of prosthesis but also sensitive feedback.Bionic limbs can be classified into three main groups, according to the type of the tissue interfaced: nerve-transferred muscle interfacing (targeted muscular reinnervation), direct muscle interfacing and direct nerve interfacing.Targeted muscular reinnervation (TMR) involves the transfer of the remaining nerves of the amputated stump to the available muscles.With direct muscle interfacing, direct intramuscular implants record muscular contractions which are then wirelessly captured through a coil integrated in the socket to actuate prosthesis movement.The third group is the direct interfacing of the residual nerves using implantable electrodes that enable reception of electric signals from the prosthetic sensors. This can improve sensation in the phantom limb.The surgical procedure for electrode implantation consists of targeting the proximal nerve area, competently introducing, placing, and fixing the electrodes and cables, while retaining movement of the arm/leg and nerve, and avoiding excessive neural damage.Advantages of bionic limbs are: the improvement of sensation, improved reintegration/embodiment of the artificial limb, and better controllability. Cite this article: EFORT Open Rev 2020;5:65-72. DOI: 10.1302/2058-5241.5.180038.

PCA-Based Channel Selection in High-Density EMG for Improving Force Estimation

In this paper, a method for selecting channels to improve estimated force using fast orthogonal search (FOS) has been proposed. Surface electromyogram (sEMG) signals acquired from linear surface electrode arrays, placed on the long head and short head of biceps brachii, and brachioradialis during isometric contractions are used to estimate force induced at the wrist using the FOS algorithm. The method utilizes principle component analysis (PCA) in the frequency domain to select the channels with the highest contribution to the first principal component (PC). Our analysis demonstrates that our proposed method is capable of reducing the dimensionality of the data (the number of channels was reduced from 21 to 9) while improving the accuracy of the estimated force.

EMG map image processing for recognition of fingers movement

Electromyography (EMG) is the conventional noninvasive method for the estimation of muscle activities. We developed a new image processing method for the recognition of individual finger movements based on EMG maps. The maps were formed from the EMG recordings via an array electrode with 24 contacts connected to a multichannel wireless miniature digital amplifier. The task was to detect and quantify the high activity regions in the EMG maps in persons with no known motor impairment. The results show the temporal and spatial patterns within the images during well-defined finger movements. The average accuracy of the automatic recognition compared with the recognition by an expert clinician in persons involved in the tests was 97.87 ± 0.92%. The application of the technique is foreseen for control for an assistive system (hand prosthesis and exoskeleton) since the interface is wearable and the processing can be implemented on a microcomputer.

Comprehensive Review on Current and Future Regulatory Requirements on Wearable Sensors in Preclinical and Clinical Testing

Medical devices are designed, tested, and placed on the market in a highly regulated environment. Wearable sensors are crucial components of various medical devices: design and validation of wearable sensors, if managed according to international standards, can foster innovation while respecting regulatory requirements. The purpose of this paper is to review the upcoming European Union (EU) Medical Device Regulations 2017/745 and 2017/746, the current and future International Electrotechnical Commission (IEC) and International Organization for Standardization (ISO) standards that set methods for design and validation of medical devices, with a focus on wearable sensors. Risk classification according to the regulation is described. The international standards IEC 62304, IEC 60601, ISO 14971, and ISO 13485 are reviewed to define regulatory restrictions during design, pre-clinical validation and clinical validation of devices that include wearable sensors as crucial components. This paper is not about any specific innovation but it is a toolbox for interpreting current and future regulatory restrictions; an integrated method for design planning, validation and clinical testing is proposed. Application of this method to design wearable sensors should be evaluated in the future in order to assess its potentially positive impact to fostering innovation and to ensure timely development.

Recurrent Neural Network Based Early Prediction of Future Hand Movements

This work focuses on a system for hand prostheses that can overcome the delay problem introduced by classical approaches while being reliable. The proposed approach based on a recurrent neural network enables us to incorporate the sequential nature of the surface electromyogram data and the proposed system can be used either for classification or early prediction of hand movements. Especially the latter is a key to a latency free steering of a prosthesis. The experiments conducted on the first three Ninapro databases reveal that the prediction up to 200 ms ahead in the future is possible without a significant drop in accuracy. Furthermore, for classification, our proposed approach outperforms the state of the art classifiers even though we used significantly shorter windows for feature extraction.

Myoelectric Control for Upper Limb Prostheses

State-of-the-art high-end prostheses are electro-mechanically able to provide a great variety of movements. Nevertheless, in order to functionally replace a human limb, it is essential that each movement is properly controlled. This is the goal of prosthesis control, which has become a growing research field in the last decades, with the ultimate goal of reproducing biological limb control. Therefore, exploration and development of prosthesis control are crucial to improve many aspects of an amputee’s life. Nowadays, a large divergence between academia and industry has become evident in commercial systems. Although several studies propose more natural control systems with promising results, basic one degree of freedom (DoF), a control switching system is the most widely used option in industry because of simplicity, robustness and inertia. A few classification controlled prostheses have emerged in the last years but they are still a low percentage of the used ones. One of the factors that generate this situation is the lack of robustness of more advanced control algorithms in daily life activities outside of laboratory conditions. Because of this, research has shifted towards more functional prosthesis control. This work reviews the most recent literature in upper limb prosthetic control. It covers commonly used variants of possible biological inputs, its processing and translation to actual control, mostly focusing on electromyograms as well as the problems it will have to overcome in near future.

Functional Tasks Performed by People with Severe Muscular Atrophy Using an sEMG Controlled Robotic Manipulator

For paralyzed people activities of daily living like eating or drinking are impossible without external assistance. Robotic assistance systems can give these people a part of their independence back. Especially if the operation with a joystick is not possible anymore due to a missing hand function, people need innovative interfaces to control assistive robots in 3D. Besides brain computer interfaces an approach based on surface electromyography (sEMG) can present an opportunity for people with a strong muscular atrophy. In this work we show that two people with proceeded spinal muscular atrophy can perform functional tasks using an sEMG controlled robotic manipulator. The interface provides a continuous control of three degrees of freedom of the endeffector of the robot. The performance was assessed with two clinical measures of upper limb functionality: the Box and Blocks Test and the Action Research Arm Test. Additionally, the participant could show that they can drink by themselves with the provided system.

Facilitate sEMG-Based Human–Machine Interaction Through Channel Optimization

Electromyography (EMG) has been widely accepted to interact with prosthetic hands, but still limited to using few channels for the control of few degrees of freedom. The use of more channels can improve the controllability, but it also increases system’s complexity and reduces its wearability. It is yet clear if optimizely placing the EMG channel could provide a feasible solution to this challenge. This study customized a genetic algorithm to optimize the number of channels and its position on the forearm in inter-day hand gesture recognition scenario. Our experimental results demonstrate that optimally selected 14 channels out of 16 can reach a peak inter-day hand gesture recognition accuracy at 72.3%, and optimally selecting 9 and 11 channels would reduce the performance by 3% and 10%. The cross-validation results also demonstrate that the optimally selected EMG channels from five subjects also work on the rest of the subjects, improving the accuracies by 3.09% and 4.5% in 9- and 11-channel combination, respectively. In sum, this study demonstrates the feasibility of channel reduction through genetic algorithm, and preliminary proves the significance of EMG channel optimization for human–machine interaction.

Strategies for neural control of prosthetic limbs: from electrode interfacing to 3D printing

Limb amputation is a major cause of disability in our community, for which motorised prosthetic devices offer a return to function and independence. With the commercialisation and increasing availability of advanced motorised prosthetic technologies, there is a consumer need and clinical drive for intuitive user control. In this context, rapid additive fabrication/prototyping capacities and biofabrication protocols embrace a highly-personalised medicine doctrine that marries specific patient biology and anatomy to high-end prosthetic design, manufacture and functionality. Commercially-available prosthetic models utilise surface electrodes that are limited by their disconnect between mind and device. As such, alternative strategies of mind-prosthetic interfacing have been explored to purposefully drive the prosthetic limb. This review investigates mind to machine interfacing strategies, with a focus on the biological challenges of long-term harnessing of the user's cerebral commands to drive actuation/movement in electronic prostheses. It covers the limitations of skin, peripheral nerve and brain interfacing electrodes, and in particular the challenges of minimising the foreign-body response, as well as a new strategy of grafting muscle onto residual peripheral nerves. In conjunction, this review also investigates the applicability of additive tissue engineering at the nerve-electrode boundary, which has led to pioneering work in neural regeneration and bioelectrode development for applications at the neuroprosthetic interface.

Sparsity Analysis of a Sonomyographic Muscle-Computer Interface

OBJECTIVE

Sonomyography has been shown to be a promising method for decoding volitional motor intent from analysis of ultrasound images of the forearm musculature. The objectives of this paper are to determine the optimal location for ultrasound transducer placement on the anterior forearm for imaging maximum muscle deformations during different hand motions, and to investigate the effect of using a sparse set of ultrasound scanlines for motion classification for ultrasound-based muscle-computer interfaces (MCIs).

METHODS

The optimal placement of the ultrasound transducer along the forearm was identified using freehand three-dimensional reconstructions of the muscle thickness during rest and motion completion. Based on the ultrasound images acquired from the optimally placed transducer, classification accuracy with equally spaced scanlines across the cross-sectional field of view was determined. Furthermore, the unique contribution of each scanline to class discrimination using Fisher criterion (FC) and mutual information (MI) with respect to motion discriminability was determined.

RESULTS

Experiments with five able-bodied subjects show that the maximum muscle deformation occurred between 40%-50% of the forearm length for multiple degrees-of-freedom. The average classification accuracy was 94% ± 6% with the entire 128-scanline image and 94% ± 5% with four equally spaced scanlines. However, no significant improvement in classification accuracy was observed with optimal scanline selection using FC and MI.

CONCLUSION

For an optimally placed transducer, a small subset of ultrasound scanlines can be used instead of a full imaging array without sacrificing performance in terms of classification accuracy for multiple degrees-of-freedom.

SIGNIFICANCE

The selection of a small subset of transducer elements can enable the reduction of computation, and simplification of the instrumentation and power consumption of wearable sonomyographic MCIs, particularly for rehabilitation and gesture recognition applications.

Online Learning and Classification of EMG-Based Gestures on a Parallel Ultra-Low Power Platform Using Hyperdimensional Computing

This paper presents a wearable electromyographic gesture recognition system based on the hyperdimensional computing paradigm, running on a programmable parallel ultra-low-power (PULP) platform. The processing chain includes efficient on-chip training, which leads to a fully embedded implementation with no need to perform any offline training on a personal computer. The proposed solution has been tested on 10 subjects in a typical gesture recognition scenario achieving 85% average accuracy on 11 gestures recognition, which is aligned with the state-of-the-art, with the unique capability of performing online learning. Furthermore, by virtue of the hardware friendly algorithm and of the efficient PULP system-on-chip (Mr. Wolf) used for prototyping and evaluation, the energy budget required to run the learning part with 11 gestures is 10.04 mJ, and 83.2  μJ per classification. The system works with a average power consumption of 10.4 mW in classification, ensuring around 29 h of autonomy with a 100 mAh battery. Finally, the scalability of the system is explored by increasing the number of channels (up to 256 electrodes), demonstrating the suitability of our approach as universal, energy-efficient biopotential wearable recognition framework.

Wearable and Flexible Textile Electrodes for Biopotential Signal Monitoring: A review

Wearable electronics is a rapidly growing field that recently started to introduce successful commercial products into the consumer electronics market. Employment of biopotential signals in wearable systems as either biofeedbacks or control commands are expected to revolutionize many technologies including point of care health monitoring systems, rehabilitation devices, human–computer/machine interfaces (HCI/HMIs), and brain–computer interfaces (BCIs). Since electrodes are regarded as a decisive part of such products, they have been studied for almost a decade now, resulting in the emergence of textile electrodes. This study presents a systematic review of wearable textile electrodes in physiological signal monitoring, with discussions on the manufacturing of conductive textiles, metrics to assess their performance as electrodes, and an investigation of their application in the acquisition of critical biopotential signals for routine monitoring, assessment, and exploitation of cardiac (electrocardiography, ECG), neural (electroencephalography, EEG), muscular (electromyography, EMG), and ocular (electrooculography, EOG) functions.

Synergistic Myoelectrical Activities of Forearm Muscles Improving Robust Recognition of Multi-Fingered Gestures

Currently, surface electromyography (sEMG) features of the forearm multi-tendon muscles are widely used in gesture recognition, however, there are few investigations on the inherent physiological mechanism of muscle synergies. We aimed to study whether the muscle synergies could be used for gesture recognition. Five healthy participants executed five gestures of daily life (pinch, fist, open hand, grip, and extension) and the sEMG activity was acquired from six forearm muscles. A non-negative matrix factorization (NMF) algorithm was employed to decompose the pre-treated six-channel sEMG data to obtain the muscle synergy matrixes, in which the weights of each muscle channel determined the feature set for hand gesture classification. The results showed that the synergistic features of forearm muscles could be successfully clustered in the feature space, which enabled hand gestures to be recognized with high efficiency. By augmenting the number of participants, the mean recognition rate remained at more than 96% and reflected high robustness. We showed that muscle synergies can be well applied to gesture recognition.

Adaptive Auto-Regressive Proportional Myoelectric Control

In proportional myographic control, one can control either position or velocity of movement. Here, we propose to use adaptive auto-regressive filters, so as to gradually adjust between the two. We implemented this in an adaptive system with closed-loop feedback, where both the user and the machine simultaneously attempt to follow a cursor on a 2-D arena. We tested this on 15 able-bodied and three limb-deficient participants using an eight-channel myoelectric armband. The human-machine pairs learn to perform smoother cursor movements with a larger range of motion when using the auto-regressive filters, as compared with our previous effortswithmoving-average filters. Importantly, the human-machine system converges to an approximate velocity control strategy resulting in faster and more accuratemovements with lessmuscle effort. The method is not specific tomyoelectriccontroland could be used equally well for motion control using high-dimensional signals from reinnervatedmuscles or direct brain recordings.

A Human-Machine Interface Using Electrical Impedance Tomography for Hand Prosthesis Control

This paper presents a human-machine interface that establishes a link between the user and a hand prosthesis. It successfully uses electrical impedance tomography, a conventional bio-impedance imaging technique, using an array of electrodes contained in a wristband on the user's forearm. Using a high-performance analog front-end application specific integrated circuit (ASIC), the user's forearm inner bio-impedance redistribution is accurately assessed. These bio-signatures are strongly related to hand motions and using artificial neural networks, they can be learned so as to recognize the user's intention in real time for prosthesis operation. In this work, eleven hand motions are designed for prosthesis operation with a gesture switching enabled sub-grouping method. Experiments with five subjects show that the system can achieve 98.5% accuracy with a grouping of three gestures and an accuracy of 94.4% with two sets of five gestures. The ASIC comprises a current driver with common-mode reduction capability and a current feedback instrumentation amplifier (that occupy an area of 0.07 mm²). The ASIC operates from ±1.65 V power supplies and has a minimum bio-impedance sensitivity of 12.7 mΩ_p-p.

Improving the functionality, robustness, and adaptability of myoelectric control for dexterous motion restoration

The development of advanced and effective human-machine interfaces, especially for amputees to control their prostheses, is very high priority and a very active area of research. An intuitive control method should retain an adequate level of functionality for dexterous operation, provide robustness against confounding factors, and supply adaptability for diverse long-term usage, all of which are current problems being tackled by researchers. This paper reviews the state-of-the-art, as well as, the limitations of current myoelectric signal control (MSC) methods. To address the research topic on functionality, we review different approaches to prosthetic hand control (DOF configuration, discrete or simultaneous, etc.), and how well the control is performed (accuracy, response, intuitiveness, etc.). To address the research on robustness, we review the confounding factors (limb positions, electrode shift, force variance, and inadvertent activity) that affect the stability of the control performance. Lastly, to address adaptability, we review the strategies that can automatically adjust the classifier for different individuals and for long-term usage. This review provides a thorough overview of the current MSC methods and helps highlight the current areas of research focus and resulting clinic usability for the MSC methods for upper-limb prostheses.

Identification of forearm skin zones with similar muscle activation patterns during activities of daily living

Background

A deeper knowledge of the activity of the forearm muscles during activities of daily living (ADL) could help to better understand the role of those muscles and allow clinicians to treat motor dysfunctions more effectively and thus improve patients’ ability to perform activities of daily living.

Methods

In this work, we recorded sEMG activity from 30 spots distributed over the skin of the whole forearm of six subjects during the performance of 21 representative simulated ADL from the Sollerman Hand Function Test. Functional principal component analysis and hierarchical cluster analysis (HCA) were used to identify forearm spots with similar muscle activation patterns.

Results

The best classification of spots with similar activity in simulated ADL consisted in seven muscular-anatomically coherent groups: (1) wrist flexion and ulnar deviation; (2) wrist flexion and radial deviation; (3) digit flexion; (4) thumb extension and abduction/adduction; (5) finger extension; (6) wrist extension and ulnar deviation; and (7) wrist extension and radial deviation.

Conclusion

The number of sEMG sensors could be reduced from 30 to 7 without losing any relevant information, using them as representative spots of the muscular activity of the forearm in simulated ADL. This may help to assess muscle function in rehabilitation while also simplifying the complexity of prosthesis control.

Pattern recognition and bionic manipulator driving by surface electromyography signals using convolutional neural network

With the development of robotics, intelligent neuroprosthesis for amputees is more concerned. Research of robot controlling based on electrocardiogram, electromyography, and electroencephalogram is a hot spot. In medical research, electrode arrays are commonly used as sensors for surface electromyograms. Although these sensors collect more accurate data and sampling at higher frequencies, they have no advantage in terms of portability and ease of use. In recent years, there are also some small surface electromyography sensors for research. The portability of the sensor and the calculation speed of the calculation method directly affect the development of the bionic prosthesis. A consumer-grade surface electromyography device is selected as surface electromyography sensor in this study. We first proposed a data structure to convert raw surface electromyography signals from an array structure into a matrix structure (we called it surface electromyography graph). Then, a convolutional neural network was used to classify it. Discrete surface electromyography signals recorded from three persons 14 gestures (widely used in other research to evaluate the performance of classifier) have been applied to train the classifier and we get an accuracy of 97.27%. The impacts of different components used in convolutional neural network were tested with this data, and subsequently, the best results were selected to build the classifier used in this article. The NinaPro database 5 (one of the biggest surface electromyography data sets) was also used to evaluate our method, which comprises of hand movement data of 10 intact subjects with two myo armbands as sensors, and the classification accuracy increased by 13.76% on average when using double myo armbands and increased by 18.92% on average when using single myo armband. In order to driving the robot hand (bionic manipulator), a group of continuous surface electromyography signals was recorded to train the classifier, and an accuracy of 91.72% was acquired. We also used the same method to collect a set of surface electromyography data from a disabled with hand lost, then classified it using the abovementioned network and achieved an accuracy of 89.37%. Finally, the classifier was deployed to the microcontroller to drive the bionic manipulator, and the full video URL is given in the conclusion, with both the healthy man and the disabled tested with the bionic manipulator. The abovementioned results suggest that this method will help to facilitate the development and application of surface electromyography neuroprosthesis.

A Fatigue Involved Modification Framework for Force Estimation in Fatiguing Contraction

To alleviate the negative impacts of muscle fatigue on a force estimation model, a modification framework taking use of fatigue index was put forward in this paper. Muscle force and surface electromyography were first collected using high-density electrode grid and dynamometer. Then, multi-step signal pre-processing and a nonnegative matrix factorization-based signal optimization were conducted, with fatigue indices being extracted in the same time. Next, a degree 4 polynomial fitting model was employed to undertake the training process, and the relationship between the generated model parameters and fatigue indices was built up. In the end, the parameter-index relationship was applied on different testing sets to complete fatigue-modified force estimation. Significant improvement was found in most testing cases across different sexes and ages. Relative decreases of 36.5%, 20.7%, and 20.4% in the percentage root mean square error were achieved by young males, young females, and elderly males. The proposed method can boost the performances of force estimation models, thereby contributing to the development of a variety of fields including biomechanical study, rehabilitation treatment, and prosthesis research.

Causes of Performance Degradation in Non-invasive Electromyographic Pattern Recognition in Upper Limb Prostheses

Surface Electromyography (EMG)-based pattern recognition methods have been investigated over the past years as a means of controlling upper limb prostheses. Despite the very good reported performance of myoelectric controlled prosthetic hands in lab conditions, real-time performance in everyday life conditions is not as robust and reliable, explaining the limited clinical use of pattern recognition control. The main reason behind the instability of myoelectric pattern recognition control is that EMG signals are non-stationary in real-life environments and present a lot of variability over time and across subjects, hence affecting the system's performance. This can be the result of one or many combined changes, such as muscle fatigue, electrode displacement, difference in arm posture, user adaptation on the device over time and inter-subject singularity. In this paper an extensive literature review is performed to present the causes of the drift of EMG signals, ways of detecting them and possible techniques to counteract for their effects in the application of upper limb prostheses. The suggested techniques are organized in a table that can be used to recognize possible problems in the clinical application of EMG-based pattern recognition methods for upper limb prosthesis applications and state-of-the-art methods to deal with such problems.

Real-time, simultaneous myoelectric control using a convolutional neural network

The evolution of deep learning techniques has been transformative as they have allowed complex mappings to be trained between control inputs and outputs without the need for feature engineering. In this work, a myoelectric control system based on convolutional neural networks (CNN) is proposed as a possible alternative to traditional approaches that rely on specifically designed features. This CNN-based system is validated using a real-time Fitts' law style target acquisition test requiring single and combined wrist motions. The performance of the proposed system is then compared to that of a standard support vector machine (SVM) based myoelectric system using a set of time-domain features. Despite the prevalence and demonstrated performance of these well-known features, no significant difference (p>0.05) was found between the two methods for any of the computed control metrics. This demonstrates the potential for automated learning approaches to extract complex and rich information from stochastic biological signals. This first evaluation of the usability of a CNN in a real-time myoelectric control environment provides a basis for further exploration.

A SEMG-Force Estimation Framework Based on a Fast Orthogonal Search Method Coupled with Factorization Algorithms

A novel framework based on the fast orthogonal search (FOS) method coupled with factorization algorithms was proposed and implemented to realize high-accuracy muscle force estimation via surface electromyogram (SEMG). During static isometric elbow flexion, high-density SEMG (HD-SEMG) signals were recorded from upper arm muscles, and the generated elbow force was measured at the wrist. HD-SEMG signals were decomposed into time-invariant activation patterns and time-varying activation curves using three typical factorization algorithms including principal component analysis (PCA), independent component analysis (ICA), and nonnegative matrix factorization (NMF). The activation signal of the target muscle was obtained by summing the activation curves, and the FOS algorithm was used to create basis functions with activation signals and establish the force estimation model. Static isometric elbow flexion experiments at three target levels were performed on seven male subjects, and the force estimation performances were compared among three typical factorization algorithms as well as a conventional method for extracting the average signal envelope of all HD-SEMG channels (AVG-ENVLP method). The overall root mean square difference (RMSD) values between the measured forces and the estimated forces obtained by different methods were 11.79 ± 4.29% for AVG-ENVLP, 9.74 ± 3.77% for PCA, 9.59 ± 3.81% for ICA, and 9.51 ± 4.82% for NMF. The results demonstrated that, compared to the conventional AVG-ENVLP method, factorization algorithms could substantially improve the performance of force estimation. The FOS method coupled with factorization algorithms provides an effective way to estimate the combined force of multiple muscles and has potential value in the fields of sports biomechanics, gait analysis, prosthesis control strategy, and exoskeleton devices for assisted rehabilitation.

An isometric muscle force estimation framework based on a high-density surface EMG array and an NMF algorithm

OBJECTIVE

To realize accurate muscle force estimation, a novel framework is proposed in this paper which can extract the input of the prediction model from the appropriate activation area of the skeletal muscle.

APPROACH

Surface electromyographic (sEMG) signals from the biceps brachii muscle during isometric elbow flexion were collected with a high-density (HD) electrode grid (128 channels) and the external force at three contraction levels was measured at the wrist synchronously. The sEMG envelope matrix was factorized into a matrix of basis vectors with each column representing an activation pattern and a matrix of time-varying coefficients by a nonnegative matrix factorization (NMF) algorithm. The activation pattern with the highest activation intensity, which was defined as the sum of the absolute values of the time-varying coefficient curve, was considered as the major activation pattern, and its channels with high weighting factors were selected to extract the input activation signal of a force estimation model based on the polynomial fitting technique.

MAIN RESULTS

Compared with conventional methods using the whole channels of the grid, the proposed method could significantly improve the quality of force estimation and reduce the electrode number.

SIGNIFICANCE

The proposed method provides a way to find proper electrode placement for force estimation, which can be further employed in muscle heterogeneity analysis, myoelectric prostheses and the control of exoskeleton devices.

Surface EMG Pattern Recognition Using Long Short-Term Memory Combined with Multilayer Perceptron

Motion classification based on pattern recognition of surface EMG (sEMG) signals is a promising approach for prosthetic control. We present a pattern recognition model that combines long short-term memory (LSTM) network with multiplayer perceptron (MLP) for sEMG signals feature learning and classification. The LSTM network captures temporal dependencies of the sEMG signals while the MLP has no inherent temporal dynamics but focuses on the static characteristics. The combination of the two networks would learn a feature space that contains both the dynamic and static information of the sEMG signals, which helps to improve the motion classification accuracy. The architecture of the proposed network was optimized by investigating the appropriate width and depth of the neural network as well as the dropout to achieve the best classification results. The performance of the proposed pattern recognition model was evaluated using Ninapro database. The results show that the proposed model can produce better classification accuracy than most of the well-known recognition techniques.

Intsy: a low-cost, open-source, wireless multi-channel bioamplifier system

OBJECTIVE

Multi-channel electrical recordings of physiologically generated signals are common to a wide range of biomedical fields. The aim of this work was to develop, validate, and demonstrate the practical utility of a high-quality, low-cost 32/64-channel bioamplifier system with real-time wireless data streaming capability.

APPROACH

The new 'Intsy' system integrates three main off-the-shelf hardware components: (1) Intan RHD2132 bioamplifier; (2) Teensy 3.2 microcontroller; and (3) RN-42 Bluetooth 2.1 module with a custom LabView interface for real-time data streaming and visualization. Practical utility was validated by measuring serosal gastric slow waves and surface EMG on the forearm with various contraction force levels. Quantitative comparisons were made to a gold-standard commercial system (Biosemi ActiveTwo).

MAIN RESULTS

Intsy signal quality was quantitatively comparable to that of the ActiveTwo. Recorded slow wave signals had high SNR (24  ±  2.7 dB) and wavefront propagation was accurately mapped. EMG spike bursts were characterized by high SNR (⩾10 dB) and activation timing was readily identified. Stable data streaming rates achieved were 3.5 kS s^-1 for wireless and 64 kS s^-1 for USB-wired transmission.

SIGNIFICANCE

Intsy has the highest channel count of any existing open-source, wireless-enabled module. The flexibility, portability and low cost ($1300 for the 32-channel version, or $2500 for 64 channels) of this new hardware module reduce the entry barrier for a range of electrophysiological experiments, as are typical in the gastrointestinal (EGG), cardiac (ECG), neural (EEG), and neuromuscular (EMG) domains.

Repeatability of grasp recognition for robotic hand prosthesis control based on sEMG data

Control methods based on sEMG obtained promising results for hand prosthetics. Control system robustness is still often inadequate and does not allow the amputees to perform a large number of movements useful for everyday life. Only few studies analyzed the repeatability of sEMG classification of hand grasps. The main goals of this paper are to explore repeatability in sEMG data and to release a repeatability database with the recorded experiments. The data are recorded from 10 intact subjects repeating 7 grasps 12 times, twice a day for 5 days. The data are publicly available on the Ninapro web page. The analysis for the repeatability is based on the comparison of movement classification accuracy in several data acquisitions and for different subjects. The analysis is performed using mean absolute value and waveform length features and a Random Forest classifier. The accuracy obtained by training and testing on acquisitions at different times is on average 27.03% lower than training and testing on the same acquisition. The results obtained by training and testing on different acquisitions suggest that previous acquisitions can be used to train the classification algorithms. The inter-subject variability is remarkable, suggesting that specific characteristics of the subjects can affect repeatibility and sEMG classification accuracy. In conclusion, the results of this paper can contribute to develop more robust control systems for hand prostheses, while the presented data allows researchers to test repeatability in further analyses.

Interface Prostheses With Classifier-Feedback-Based User Training

It is evident that user training significantly affects performance of pattern-recognition-based myoelectric prosthetic device control. Despite plausible classification accuracy on offline datasets, online accuracy usually suffers from the changes in physiological conditions and electrode displacement. The user ability in generating consistent electromyographic (EMG) patterns can be enhanced via proper user training strategies in order to improve online performance. This study proposes a clustering-feedback strategy that provides real-time feedback to users by means of a visualized online EMG signal input as well as the centroids of the training samples, whose dimensionality is reduced to minimal number by dimension reduction. Clustering feedback provides a criterion that guides users to adjust motion gestures and muscle contraction forces intentionally. The experiment results have demonstrated that hand motion recognition accuracy increases steadily along the progress of the clustering-feedback-based user training, while conventional classifier-feedback methods, i.e., label feedback, hardly achieve any improvement. The result concludes that the use of proper classifier feedback can accelerate the process of user training, and implies prosperous future for the amputees with limited or no experience in pattern-recognition-based prosthetic device manipulation.It is evident that user training significantly affects performance of pattern-recognition-based myoelectric prosthetic device control. Despite plausible classification accuracy on offline datasets, online accuracy usually suffers from the changes in physiological conditions and electrode displacement. The user ability in generating consistent electromyographic (EMG) patterns can be enhanced via proper user training strategies in order to improve online performance. This study proposes a clustering-feedback strategy that provides real-time feedback to users by means of a visualized online EMG signal input as well as the centroids of the training samples, whose dimensionality is reduced to minimal number by dimension reduction. Clustering feedback provides a criterion that guides users to adjust motion gestures and muscle contraction forces intentionally. The experiment results have demonstrated that hand motion recognition accuracy increases steadily along the progress of the clustering-feedback-based user training, while conventional classifier-feedback methods, i.e., label feedback, hardly achieve any improvement. The result concludes that the use of proper classifier feedback can accelerate the process of user training, and implies prosperous future for the amputees with limited or no experience in pattern-recognition-based prosthetic device manipulation.

Simultaneous Force Regression and Movement Classification of Fingers via Surface EMG within a Unified Bayesian Framework

This contribution presents a novel methodology for myolectric-based control using surface electromyographic (sEMG) signals recorded during finger movements. A multivariate Bayesian mixture of experts (MoE) model is introduced which provides a powerful method for modeling force regression at the fingertips, while also performing finger movement classification as a by-product of the modeling algorithm. Bayesian inference of the model allows uncertainties to be naturally incorporated into the model structure. This method is tested using data from the publicly released NinaPro database which consists of sEMG recordings for 6 degree-of-freedom force activations for 40 intact subjects. The results demonstrate that the MoE model achieves similar performance compared to the benchmark set by the authors of NinaPro for finger force regression. Additionally, inherent to the Bayesian framework is the inclusion of uncertainty in the model parameters, naturally providing confidence bounds on the force regression predictions. Furthermore, the integrated clustering step allows a detailed investigation into classification of the finger movements, without incurring any extra computational effort. Subsequently, a systematic approach to assessing the importance of the number of electrodes needed for accurate control is performed via sensitivity analysis techniques. A slight degradation in regression performance is observed for a reduced number of electrodes, while classification performance is unaffected.

Real-time robustness evaluation of regression based myoelectric control against arm position change and donning/doffing

There are some practical factors, such as arm position change and donning/doffing, which prevent robust myoelectric control. The objective of this study is to precisely characterize the impacts of the two representative factors on myoelectric controllability in practical control situations, thereby providing useful references that can be potentially used to find better solutions for clinically reliable myoelectric control. To this end, a real-time target acquisition task was performed by fourteen subjects including one individual with congenital upper-limb deficiency, where the impacts of arm position change, donning/doffing and a combination of both factors on control performance was systematically evaluated. The changes in online performance were examined with seven different performance metrics to comprehensively evaluate various aspects of myoelectric controllability. As a result, arm position change significantly affects offline prediction accuracy, but not online control performance due to real-time feedback, thereby showing no significant correlation between offline and online performance. Donning/doffing was still problematic in online control conditions. It was further observed that no benefit was attained when using a control model trained with multiple position data in terms of arm position change, and the degree of electrode shift caused by donning/doffing was not severely associated with the degree of performance loss under practical conditions (around 1 cm electrode shift). Since this study is the first to concurrently investigate the impacts of arm position change and donning/doffing in practical myoelectric control situations, all findings of this study provide new insights into robust myoelectric control with respect to arm position change and donning/doffing.

Comparison of six electromyography acquisition setups on hand movement classification tasks

Hand prostheses controlled by surface electromyography are promising due to the non-invasive approach and the control capabilities offered by machine learning. Nevertheless, dexterous prostheses are still scarcely spread due to control difficulties, low robustness and often prohibitive costs. Several sEMG acquisition setups are now available, ranging in terms of costs between a few hundred and several thousand dollars. The objective of this paper is the relative comparison of six acquisition setups on an identical hand movement classification task, in order to help the researchers to choose the proper acquisition setup for their requirements. The acquisition setups are based on four different sEMG electrodes (including Otto Bock, Delsys Trigno, Cometa Wave + Dormo ECG and two Thalmic Myo armbands) and they were used to record more than 50 hand movements from intact subjects with a standardized acquisition protocol. The relative performance of the six sEMG acquisition setups is compared on 41 identical hand movements with a standardized feature extraction and data analysis pipeline aimed at performing hand movement classification. Comparable classification results are obtained with three acquisition setups including the Delsys Trigno, the Cometa Wave and the affordable setup composed of two Myo armbands. The results suggest that practical sEMG tests can be performed even when costs are relevant (e.g. in small laboratories, developing countries or use by children). All the presented datasets can be used for offline tests and their quality can easily be compared as the data sets are publicly available.

Self-Recalibrating Surface EMG Pattern Recognition for Neuroprosthesis Control Based on Convolutional Neural Network

Hand movement classification based on surface electromyography (sEMG) pattern recognition is a promising approach for upper limb neuroprosthetic control. However, maintaining day-to-day performance is challenged by the non-stationary nature of sEMG in real-life operation. In this study, we propose a self-recalibrating classifier that can be automatically updated to maintain a stable performance over time without the need for user retraining. Our classifier is based on convolutional neural network (CNN) using short latency dimension-reduced sEMG spectrograms as inputs. The pretrained classifier is recalibrated routinely using a corrected version of the prediction results from recent testing sessions. Our proposed system was evaluated with the NinaPro database comprising of hand movement data of 40 intact and 11 amputee subjects. Our system was able to achieve ~10.18% (intact, 50 movement types) and ~2.99% (amputee, 10 movement types) increase in classification accuracy averaged over five testing sessions with respect to the unrecalibrated classifier. When compared with a support vector machine (SVM) classifier, our CNN-based system consistently showed higher absolute performance and larger improvement as well as more efficient training. These results suggest that the proposed system can be a useful tool to facilitate long-term adoption of prosthetics for amputees in real-life applications.

Early prediction of future hand movements using sEMG data

We study in this work the feasibility of early prediction of hand movement based on sEMG signals to overcome the time delay issue of the conventional classification. Opposed to the classification task, the objective of early prediction is to predict a hand movement that is going to occur in the future given the information up to the current time point. The ability of early prediction may allow a hand prosthesis control system to compensate for the time delay and, as a result, improve the usability. Experimental results on the Ninapro database show that we can predict up to 300 ms ahead in the future while the prediction accuracy remains very close to that of the standard classification, i.e. it is just marginally lower. Furthermore, historical data prior the current time window is shown to be very important to improve performance, not only for the prediction but also the classification task.

EMG pattern classification to control a hand orthosis for functional grasp assistance after stroke

Wearable orthoses can function both as assistive devices, which allow the user to live independently, and as rehabilitation devices, which allow the user to regain use of an impaired limb. To be fully wearable, such devices must have intuitive controls, and to improve quality of life, the device should enable the user to perform Activities of Daily Living. In this context, we explore the feasibility of using electromyography (EMG) signals to control a wearable exotendon device to enable pick and place tasks. We use an easy to don, commodity forearm EMG band with 8 sensors to create an EMG pattern classification control for an exotendon device. With this control, we are able to detect a user's intent to open, and can thus enable extension and pick and place tasks. In experiments with stroke survivors, we explore the accuracy of this control in both non-functional and functional tasks. Our results support the feasibility of developing wearable devices with intuitive controls which provide a functional context for rehabilitation.

User adaptation in Myoelectric Man-Machine Interfaces

State of the art clinical hand prostheses are controlled in a simple and limited way that allows the activation of one function at a time. More advanced laboratory approaches, based on machine learning, offer a significant increase in functionality, but their clinical impact is limited, mainly due to lack of reliability. In this study, we analyse two conceptually different machine learning approaches, focusing on their robustness and performance in a closed loop application. A classification (finite number of classes) and a regression (continuous mapping) based projection of EMG into external commands were applied while artificially introducing non-stationarities in the EMG signals. When tested on ten able-bodied individuals and one transradial amputee, the two methods were similarly influenced by non-stationarities when tested offline. However, in online tests, where the user could adapt his muscle activation patterns to the changed conditions, the regression-based approach was significantly less influenced by the changes in signal features than the classification approach. This observation demonstrates, on the one hand, the importance of online tests with users in the loop for assessing the performance of myocontrol approaches. On the other hand, it also demonstrates that regression allows for a better user correction of control commands than classification.

Deep learning-based artificial vision for grasp classification in myoelectric hands

OBJECTIVE

Computer vision-based assistive technology solutions can revolutionise the quality of care for people with sensorimotor disorders. The goal of this work was to enable trans-radial amputees to use a simple, yet efficient, computer vision system to grasp and move common household objects with a two-channel myoelectric prosthetic hand.

APPROACH

We developed a deep learning-based artificial vision system to augment the grasp functionality of a commercial prosthesis. Our main conceptual novelty is that we classify objects with regards to the grasp pattern without explicitly identifying them or measuring their dimensions. A convolutional neural network (CNN) structure was trained with images of over 500 graspable objects. For each object, 72 images, at [Formula: see text] intervals, were available. Objects were categorised into four grasp classes, namely: pinch, tripod, palmar wrist neutral and palmar wrist pronated. The CNN setting was first tuned and tested offline and then in realtime with objects or object views that were not included in the training set.

MAIN RESULTS

The classification accuracy in the offline tests reached [Formula: see text] for the seen and [Formula: see text] for the novel objects; reflecting the generalisability of grasp classification. We then implemented the proposed framework in realtime on a standard laptop computer and achieved an overall score of [Formula: see text] in classifying a set of novel as well as seen but randomly-rotated objects. Finally, the system was tested with two trans-radial amputee volunteers controlling an i-limb Ultra^TM prosthetic hand and a motion control^TM prosthetic wrist; augmented with a webcam. After training, subjects successfully picked up and moved the target objects with an overall success of up to [Formula: see text]. In addition, we show that with training, subjects' performance improved in terms of time required to accomplish a block of 24 trials despite a decreasing level of visual feedback.

SIGNIFICANCE

The proposed design constitutes a substantial conceptual improvement for the control of multi-functional prosthetic hands. We show for the first time that deep-learning based computer vision systems can enhance the grip functionality of myoelectric hands considerably.

A Master–Slave Control Method for Dexterous Hands with Shaking Elimination Strategy

With the limitations of the artificial intelligence, automatic control and sensor technologies, the dexterous hand in unstructured environments to achieve fully autonomous operations is still very difficult. This paper proposed a master–slave control method for dexterous hands with the combination of the data glove and the micro-stepper motor. The hardware of this method included CyberGloveII device, personal computer (PC), integrated control board (ICB), and YWZ dexterous hand (a multi-fingered robot hand with 20 active degrees of freedom (DOFs)). By the CyberGloveII device, we gained human finger joints motion data in real-time firstly, which were preprocessed by a shaking elimination algorithm to ensure the motion stability of the dexterous hand. Then, the motion data were mapped to the dexterous hand joints, respectively. A communication protocol was designed to transfer the motion data between the PC and the ICB. The motion data were transmitted into the ICB through a serial interface driving the corresponding dexterous hand joints. The experimental results showed that this method is feasible, can achieve the open-loop control of dexterous hands, and has excellent movement accuracy, real-time and stability.

Surface EMG-Based Inter-Session Gesture Recognition Enhanced by Deep Domain Adaptation

High-density surface electromyography (HD-sEMG) is to record muscles' electrical activity from a restricted area of the skin by using two dimensional arrays of closely spaced electrodes. This technique allows the analysis and modelling of sEMG signals in both the temporal and spatial domains, leading to new possibilities for studying next-generation muscle-computer interfaces (MCIs). sEMG-based gesture recognition has usually been investigated in an intra-session scenario, and the absence of a standard benchmark database limits the use of HD-sEMG in real-world MCI. To address these problems, we present a benchmark database of HD-sEMG recordings of hand gestures performed by 23 participants, based on an 8 × 16 electrode array, and propose a deep-learning-based domain adaptation framework to enhance sEMG-based inter-session gesture recognition. Experiments on NinaPro, CSL-HDEMG and our CapgMyo dataset validate that our approach outperforms state-of-the-arts methods on intra-session and effectively improved inter-session gesture recognition.

Continuous Recognition of Multifunctional Finger and Wrist Movements in Amputee Subjects Based on sEMG and Accelerometry

Background:

While the classification of multifunctional finger and wrist movement based on surface electromyography (sEMG) signals in intact subjects can reach remarkable recognition rates, the performance obtained from amputated subjects remained low.

Methods:

In this paper, we proposed and evaluated the myoelectric control scheme of upper-limb prostheses by the continuous recognition of 17 multifunctional finger and wrist movements on 5 amputated subjects. Experimental validation was applied to select optimal features and classifiers for identifying sEMG and accelerometry (ACC) modalities under the windows-based analysis scheme. The majority vote is adopted to eliminate transient jumps and produces smooth output for window-based analysis scheme. Furthermore, principle component analysis was employed to reduce the dimension of features and to eliminate redundancy for ACC signal. Then a novel metric, namely movement error rate, was also employed to evaluate the performance of the continuous recognition framework proposed herein.

Results:

The average accuracy rates of classification were up to 88.7 ± 2.6% over 5 amputated subjects, which was an outstanding result in comparison with the previous literature.

Conclusion:

The proposed technique was proven to be a potential candidate for intelligent prosthetic systems, which would increase quality of life for amputated subjects.

Support vector machines to detect physiological patterns for EEG and EMG-based human-computer interaction: a review

Support vector machines (SVMs) are widely used classifiers for detecting physiological patterns in human-computer interaction (HCI). Their success is due to their versatility, robustness and large availability of free dedicated toolboxes. Frequently in the literature, insufficient details about the SVM implementation and/or parameters selection are reported, making it impossible to reproduce study analysis and results. In order to perform an optimized classification and report a proper description of the results, it is necessary to have a comprehensive critical overview of the applications of SVM. The aim of this paper is to provide a review of the usage of SVM in the determination of brain and muscle patterns for HCI, by focusing on electroencephalography (EEG) and electromyography (EMG) techniques. In particular, an overview of the basic principles of SVM theory is outlined, together with a description of several relevant literature implementations. Furthermore, details concerning reviewed papers are listed in tables and statistics of SVM use in the literature are presented. Suitability of SVM for HCI is discussed and critical comparisons with other classifiers are reported.

Assessment of a Wearable Force- and Electromyography Device and Comparison of the Related Signals for Myocontrol

In the frame of assistive robotics, multi-finger prosthetic hand/wrists have recently appeared, offering an increasing level of dexterity; however, in practice their control is limited to a few hand grips and still unreliable, with the effect that pattern recognition has not yet appeared in the clinical environment. According to the scientific community, one of the keys to improve the situation is multi-modal sensing, i.e., using diverse sensor modalities to interpret the subject's intent and improve the reliability and safety of the control system in daily life activities. In this work, we first describe and test a novel wireless, wearable force- and electromyography device; through an experiment conducted on ten intact subjects, we then compare the obtained signals both qualitatively and quantitatively, highlighting their advantages and disadvantages. Our results indicate that force-myography yields signals which are more stable across time during whenever a pattern is held, than those obtained by electromyography. We speculate that fusion of the two modalities might be advantageous to improve the reliability of myocontrol in the near future.

On optimal electrode configuration to estimate hand movements from forearm surface electromyography

Understanding the movement of the hand from sEMG signals acquired on the forearm is key in the development of future prosthetics of the upper limb. Despite the technical advancement on this technique, state of the art of sEMG still relies strongly on optimal electrode placement which is typically performed by a specialist by mean of a heuristic search. Involving a specialist has few major disadvantages including high costs and relatively long schedules. This work searches an optimal electrode configuration which could reduce or avoid the intervention of a specialist. More than 200 different possible electrode configurations were assessed by means of the average recognition rate over 11 different movements of the hand, wrist, and fingers. It is shown that using two rows of 8 equally spaced electrodes around the circumference of the forearm could be an optimal trade-off solution to accomplish the task of recognizing hand movement (ARR = 92%) without the need for a specialist or very complex hardware.