A comparison of the effects of electrode implantation and targeting on pattern classification accuracy for prosthesis control

Multimodal fusion of EMG and vision for human grasp intent inference in prosthetic hand control

Objective: For transradial amputees, robotic prosthetic hands promise to regain the capability to perform daily living activities. Current control methods based on physiological signals such as electromyography (EMG) are prone to yielding poor inference outcomes due to motion artifacts, muscle fatigue, and many more. Vision sensors are a major source of information about the environment state and can play a vital role in inferring feasible and intended gestures. However, visual evidence is also susceptible to its own artifacts, most often due to object occlusion, lighting changes, etc. Multimodal evidence fusion using physiological and vision sensor measurements is a natural approach due to the complementary strengths of these modalities. Methods: In this paper, we present a Bayesian evidence fusion framework for grasp intent inference using eye-view video, eye-gaze, and EMG from the forearm processed by neural network models. We analyze individual and fused performance as a function of time as the hand approaches the object to grasp it. For this purpose, we have also developed novel data processing and augmentation techniques to train neural network components. Results: Our results indicate that, on average, fusion improves the instantaneous upcoming grasp type classification accuracy while in the reaching phase by 13.66% and 14.8%, relative to EMG (81.64% non-fused) and visual evidence (80.5% non-fused) individually, resulting in an overall fusion accuracy of 95.3%. Conclusion: Our experimental data analyses demonstrate that EMG and visual evidence show complementary strengths, and as a consequence, fusion of multimodal evidence can outperform each individual evidence modality at any given time.

Functionally Adaptive Myosite Selection Using High-Density sEMG for Upper Limb Myoelectric Prostheses

OBJECTIVE

Our study defines a novel electrode placement method called Functionally Adaptive Myosite Selection (FAMS), as a tool for rapid and effective electrode placement during prosthesis fitting. We demonstrate a method for determining electrode placement that is adaptable towards individual patient anatomy and desired functional outcomes, agnostic to the type of classification model used, and provides insight into expected classifier performance without training multiple models.

METHODS

FAMS relies on a separability metric to rapidly predict classifier performance during prosthesis fitting.

RESULTS

The results show a predictable relationship between the FAMS metric and classifier accuracy (3.45%SE), allowing estimation of control performance with any given set of electrodes. Electrode configurations selected using the FAMS metric show improved control performance ( ) for target electrode counts compared to established methods when using an ANN classifier, and equivalent performance ( R2 ≥ .96) to previous top-performing methods on an LDA classifier, with faster convergence ( ). We used the FAMS method to determine electrode placement for two amputee subjects by using the heuristic to search through possible sets, and checking for saturation in performance vs electrode count. The resulting configurations that averaged 95.8% of the highest possible classification performance using a mean 25 number of electrodes (19.5% of the available sites).

SIGNIFICANCE

FAMS can be used to rapidly approximate the tradeoffs between increased electrode count and classifier performance, a useful tool during prosthesis fitting.

Wearable bioelectronics fabricated in situ on skins

In recent years, wearable bioelectronics has rapidly expanded for diagnosing, monitoring, and treating various pathological conditions from the skin surface. Although the devices are typically prefabricated as soft patches for general usage, there is a growing need for devices that are customized in situ to provide accurate data and precise treatment. In this perspective, the state-of-the-art in situ fabricated wearable bioelectronics are summarized, focusing primarily on Drawn-on-Skin (DoS) bioelectronics and other in situ fabrication methods. The advantages and limitations of these technologies are evaluated and potential future directions are suggested for the widespread adoption of these technologies in everyday life.

Virtual regression-based myoelectric hand-wrist prosthesis control and electrode site selection using no force feedback

Most transradial prosthesis users with conventional "Sequential" myoelectric control have two electrode sites which control one degree of freedom (DoF) at a time. Rapid EMG co-activation toggles control between DoFs (e.g., hand and wrist), providing limited function. We implemented a regression-based EMG control method which achieved simultaneous and proportional control of two DoFs in a virtual task. We automated electrode site selection using a short-duration (90 s) calibration period, without force feedback. Backward stepwise selection located the best electrodes for either six or 12 electrodes (selected from a pool of 16). We additionally studied two, 2-DoF controllers: "Intuitive" control (hand open-close and wrist pronation-supination controlled virtual target size and rotation, respectively) and "Mapping" control (wrist flexion-extension and ulnar-radial deviation controlled virtual target left-right and up-down movement, respectively). In practice, a Mapping controller would be mapped to control prosthesis hand open-close and wrist pronation-supination. Eleven able-bodied subjects and 4 limb-absent subjects completed virtual target matching tasks (fixed target moves to a new location after being "matched," and subject immediately pursues) and fixed (static) target tasks. For all subjects, both 2-DoF controllers with 6 optimally-sited electrodes had statistically better target matching performance than Sequential control in number of matches (average of 4-7 vs. 2 matches, p< 0.001) and throughput (average of 0.75-1.25 vs. 0.4 bits/s, p< 0.001), but not overshoot rate and path efficiency. There were no statistical differences between 6 and 12 optimally-sited electrodes for both 2-DoF controllers. These results support the feasibility of 2-DoF simultaneous, proportional myoelectric control.

A flexible carbon nanotube electrode array for acute in vivo EMG recordings

Executing complex behaviors requires precise control of muscle activity. Our understanding of how the nervous system learns and controls motor skills relies on recording electromyographic (EMG) signals from multiple muscles that are engaged in the motor task. Despite recent advances in tools for monitoring and manipulating neural activity, methods for recording in situ spiking activity in muscle fibers have changed little in recent decades. Here, we introduce a novel experimental approach to recording high-resolution EMG signals using parylene-coated carbon nanotube fibers (CNTFs). These fibers are fabricated via a wet spinning process and twisted together to create a bipolar electrode. Single CNTFs are strong, extremely flexible, small in diameter (14-24 µm), and have low interface impedance. We present two designs to build bipolar electrode arrays that, due to the small size of CNTF, lead to high spatial resolution EMG recordings. To test the EMG arrays, we recorded the activity of small (4 mm length) vocal muscles in songbirds in an acute setting. CNTF arrays were more flexible and yielded multiunit/bulk EMG recordings with higher SNR compared with stainless steel wire electrodes. Furthermore, we were able to record single-unit recordings not previously reported in these small muscles. CNTF electrodes are therefore well-suited for high-resolution EMG recording in acute settings, and we present both opportunities and challenges for their application in long-term chronic recordings.NEW & NOTEWORTHY We introduce a novel approach to record high-resolution EMG signals in small muscles using extremely strong and flexible carbon nanotube fibers (CNTFs). We test their functionality in songbird vocal muscles. Acute EMG recordings successfully yielded multiunit recordings with high SNR. Furthermore, they successfully isolated single-unit spike trains from CNTF recordings. CNTF electrodes have great potential for chronic EMG studies of small, deep muscles that demand high electrode flexibility and strength.

Customizable, reconfigurable, and anatomically coordinated large-area, high-density electromyography from drawn-on-skin electrode arrays

Accurate anatomical matching for patient-specific electromyographic (EMG) mapping is crucial yet technically challenging in various medical disciplines. The fixed electrode construction of multielectrode arrays (MEAs) makes it nearly impossible to match an individual's unique muscle anatomy. This mismatch between the MEAs and target muscles leads to missing relevant muscle activity, highly redundant data, complicated electrode placement optimization, and inaccuracies in classification algorithms. Here, we present customizable and reconfigurable drawn-on-skin (DoS) MEAs as the first demonstration of high-density EMG mapping from in situ-fabricated electrodes with tunable configurations adapted to subject-specific muscle anatomy. The DoS MEAs show uniform electrical properties and can map EMG activity with high fidelity under skin deformation-induced motion, which stems from the unique and robust skin-electrode interface. They can be used to localize innervation zones (IZs), detect motor unit propagation, and capture EMG signals with consistent quality during large muscle movements. Reconfiguring the electrode arrangement of DoS MEAs to match and extend the coverage of the forearm flexors enables localization of the muscle activity and prevents missed information such as IZs. In addition, DoS MEAs customized to the specific anatomy of subjects produce highly informative data, leading to accurate finger gesture detection and prosthetic control compared with conventional technology.

Individuals with moderate to severe hand impairments may struggle to use EMG control for assistive devices

Neurological trauma, such as stroke, traumatic brain injury (TBI), spinal cord injury, and cerebral palsy can cause mild to severe upper limb impairments. Hand impairment makes it difficult for individuals to complete activities of daily living, especially bimanual tasks. A robotic hand orthosis or hand exoskeleton can be used to restore partial function of an intact but impaired hand. It is common for upper extremity prostheses and orthoses to use electromyography (EMG) sensing as a method for the user to control their device. However some individuals with an intact but impaired hand may struggle to use a myoelectrically controlled device due to potentially confounding muscle activity. This study was conducted to evaluate the application of conventional EMG control techniques as a robotic orthosis/exoskeleton user input method for individuals with mild to severe hand impairments. Nine impaired subjects and ten healthy subjects were asked to perform repeated contractions of muscles in their forearm and then onset analysis and feature classification were used to determine the accuracy of the employed EMG techniques. The average accuracy for contraction identification across employed EMG techniques was 95.4% ± 4.9 for the healthy subjects and 73.9% ± 13.1 for the impaired subjects with a range of 47.0% ± 19.1 - 91.6% ± 8.5. These preliminary results suggest that the conventional EMG control technologies employed in this paper may be difficult for some impaired individuals to use due to their unreliable muscle control.

ROS-Neuro: An Open-Source Platform for Neurorobotics

The growing interest in neurorobotics has led to a proliferation of heterogeneous neurophysiological-based applications controlling a variety of robotic devices. Although recent years have seen great advances in this technology, the integration between human neural interfaces and robotics is still limited, making evident the necessity of creating a standardized research framework bridging the gap between neuroscience and robotics. This perspective paper presents Robot Operating System (ROS)-Neuro, an open-source framework for neurorobotic applications based on ROS. ROS-Neuro aims to facilitate the software distribution, the repeatability of the experimental results, and support the birth of a new community focused on neuro-driven robotics. In addition, the exploitation of Robot Operating System (ROS) infrastructure guarantees stability, reliability, and robustness, which represent fundamental aspects to enhance the translational impact of this technology. We suggest that ROS-Neuro might be the future development platform for the flourishing of a new generation of neurorobots to promote the rehabilitation, the inclusion, and the independence of people with disabilities in their everyday life.

Frequency-Domain sEMG Classification Using a Single Sensor

Working towards the development of robust motion recognition systems for assistive technology control, the widespread approach has been to use a plethora of, often times, multi-modal sensors. In this paper, we develop single-sensor motion recognition systems. Utilising the peripheral nature of surface electromyography (sEMG) data acquisition, we optimise the information extracted from sEMG sensors. This allows the reduction in sEMG sensors or provision of contingencies in a system with redundancies. In particular, we process the sEMG readings captured at the trapezius descendens and platysma muscles. We demonstrate that sEMG readings captured at one muscle contain distinct information on movements or contractions of other agonists. We used the trapezius and platysma muscle sEMG data captured in able-bodied participants and participants with tetraplegia to classify shoulder movements and platysma contractions using white-box supervised learning algorithms. Using the trapezius sensor, shoulder raise is classified with an accuracy of 99%. Implementing subject-specific multi-class classification, shoulder raise, shoulder forward and shoulder backward are classified with a 94% accuracy amongst object raise and shoulder raise-and-hold data in able bodied adults. A three-way classification of the platysma sensor data captured with participants with tetraplegia achieves a 95% accuracy on platysma contraction and shoulder raise detection.

Inter-classifier comparison for upper extremity EMG signal at different hand postures and arm positions using pattern recognition

The utilization of surface EMG and intramuscular EMG signals has been observed to create significant improvement in pattern recognition approaches and myoelectric control. However, there is less data of different arm positions and hand postures available. Hand postures and arm positions tend to affect the combination of surface and intramuscular EMG signal acquisition in terms of classifier accuracy. Hence, this study aimed to find a robust classifier for two scenarios: (1) at fixed arm position (FAP) where classifiers classify different hand postures and (2) at fixed hand posture (FHP) where classifiers classify different arm positions. A total of 20 healthy male participants (30.62 ± 3.87 years old) were recruited for this study. They were asked to perform five motion classes including hand grasp, hand open, rest, hand extension, and hand flexion at four different arm positions at 0°, 45°, 90°, and 135°. SVM, KNN, and LDA classifier were deployed. Statistical analysis in the form of pairwise comparisons was carried out using SPSS. It is concluded that there is no significant difference among the three classifiers. SVM gave highest accuracy of 75.35% and 58.32% at FAP and FHP respectively for each motion classification. KNN yielded the highest accuracies of 69.11% and 79.04% when data was pooled and was classified at different arm positions and at different hand postures respectively. The results exhibited that there is no significant effect of changing arm position and hand posture on the classifier accuracy.

Evaluating User and Machine Learning in Short- and Long-Term Pattern Recognition-Based Myoelectric Control

Proper training is essential to achieve reliable pattern recognition (PR) based myoelectric control. The amount of training is commonly determined by experience. The purpose of this study is to provide an offline validation method that makes the offline performance transferable to online control and find the proper amount of training that achieves good online performance. In the offline experiment, eight able-bodied subjects and three amputees participated in a ten-day training. Repeatability index (RI) and classification error (CE) were used to evaluate user learning and machine learning, respectively. The performance of cross-validation (CV) and time serial related validation (TSV) was compared. Learning curves were established with different training trials by TSV. In the online experiment, sixteen able-bodied subjects were randomly divided into two groups with one- or five-trial training, respectively, followed by participating in the test with and without classifier-output feedback. The correlation between offline and online tests was analyzed. Results indicated that five-trial training was proper to train the user and the classifier. The long-term retention of skills could not shorten the learning process. The correlation between CEs of TSV and the online test was strong ( r=0.87 ) with five-trial training, while the correlation between CEs of CV and the online test was weak ( r=0.30 ). Outcomes demonstrate that offline performance evaluated by TSV is transferable to online performance and the learning process can guide the user to achieve good online myoelectric control with minimum training.

Evaluation of windowing techniques for intramuscular EMG-based diagnostic, rehabilitative and assistive devices

Objective.Intramuscular electromyography (iEMG) signals, invasively recorded, directly from the muscles are used to diagnose various neuromuscular disorders/diseases and to control rehabilitative and assistive robotic devices. iEMG signals are potentially being used in neurology, kinesiology, rehabilitation and ergonomics, to detect/diagnose various diseases/disorders. Electromyography-based classification and analysis systems are being designed and tested for the classification of various neuromuscular disorders and to control rehabilitative and assistive robotic devices. Many studies have explored parameters such as the pre-processing, feature extraction and selection of classifiers that can affect the performance and efficacy of iEMG-based classification systems. The pre-processing stage includes the removal of any unwanted noise from the original signal and windowing of the signal.Approach.This study investigated and presented the optimum windowing configurations for robust control and better performance results of an iEMG-based analysis system based on the stationarity rate (SR) and classification accuracy. Both disjoint and overlap, windowing techniques with varying window and overlap sizes have been investigated using a machine learning-based classification algorithm called linear discriminant analysis.Main results.The optimum window size ranges are from 200-300 ms for the disjoint and 225-300 ms for the overlap windowing technique, respectively. The inferred results show that for the overlap windowing technique the optimum range of overlap size is from 10%-30% of the length of window size. The mean classification accuracy (MCA) and mean stationarity rate (MSR) were found to be lower in the disjoint windowing technique compared to overlap windowing at all investigated overlap sizes. Statistical analysis (two-way analysis of variance test) showed that the MSR and MCA of the overlap windowing technique was significantly different at overlap sizes of 10%-30% (p-values < 0.05).Significance.The presented results can be used to achieve the best possible classification results and SR for any iEMG-based real-time diagnosis, detection and control system, which can enhance the performance of the system significantly.

Brain-machine and muscle-machine bio-sensing methods for gesture intent acquisition in upper-limb prosthesis control: a review

This paper presents a review of a number of bio-sensing methods for gesture intent signal acquisition in control tasks for upper-limb prosthesis. The paper specifically provides a breakdown of the control task in myoelectric prosthesis, and in addition, highlights and describes the importance of the acquisition of a high-quality bio-signal. The paper also describes commonly used invasive and non-invasive brain and muscle machine interfaces such as electroencephalography, electrocorticography, electroneurography, surface electromyography, sonomyography, mechanomyography, near infra-red, force sensitive resistance/pressure, and magnetoencephalography. Each modality is reviewed based on its operating principle and limitations in gesture recognition, followed by respective advantages and disadvantages. Also described within this paper, are multimodal sensing approaches, which involve data fusion of information from various sensing modalities for an enhanced neuromuscular bio-sensing source. Using a semi-systematic review methodology, we are able to derive a novel tabular approach towards contrasting the various strengths and weaknesses of the reviewed bio-sensing methods towards gesture recognition in a prosthesis interface. This would allow for a streamlined method of down selection of an appropriate bio-sensor given specific prosthesis design criteria and requirements. The paper concludes by highlighting a number of research areas that require more work for strides to be made towards improving and enhancing the connection between man and machine as it concerns upper-limb prosthesis. Such areas include classifier augmentation for gesture recognition, filtering techniques for sensor disturbance rejection, feeling of tactile sensations with an artificial limb.

Functional assessment of the gluteus medius, cranial part of the biceps femoris, and vastus lateralis in Beagle dogs based on a novel gait phase classification

In humans, walking analysis based on the gait phase classification has been used for interpretation of functional roles of different movements occurring at individual joints, and it is useful for establishing a rehabilitation plan. However, there have been few reports on canine gait phase classification, and this is one of the reasons for preventing progress in canine rehabilitation. In this study, we determined phases of the canine gait cycle (GC) on the basis of the phase classification for human gait. The canine GC was able to be divided into initial contact (IC) and the following 5 phases: loading response (LR), middle stance (MidSt), pre-swing (PSw), early swing (ESw), and late swing (LSw). Next, the hind limb joint angles of the hip, stifle and tarsal joints and results of surface electromyography of the gluteus medius (GM), cranial part of the biceps femoris (CBF) and vastus lateralis (VL) muscles in relation to the gait phases were analyzed. The activities of three muscles showed similar changes during walking. The muscle activities were high in the LR phase and then declined and reached a minimum in the PSw phase, but they increased and reached a peak in the LSw phase, which was followed by the LR phase. In conclusion, the multiphasic canine GC was developed by modification of the human model, and the GC phase-related changes in the muscle activity and joint angles suggested the functions of GM, CBF and VL muscles in walking.

Locomo-Net: A Low -Complex Deep Learning Framework for sEMG-Based Hand Movement Recognition for Prosthetic Control

Background: The enhancement in the performance of the myoelectric pattern recognition techniques based on deep learning algorithm possess computationally expensive and exhibit extensive memory behavior. Therefore, in this paper we report a deep learning framework named 'Low-Complex Movement recognition-Net' (LoCoMo-Net) built with convolution neural network (CNN) for recognition of wrist and finger flexion movements; grasping and functional movements; and force pattern from single channel surface electromyography (sEMG) recording. The network consists of a two-stage pipeline: 1) input data compression; 2) data-driven weight sharing. Methods: The proposed framework was validated on two different datasets- our own dataset (DS1) and publicly available NinaPro dataset (DS2) for 16 movements and 50 movements respectively. Further, we have prototyped the proposed LoCoMo-Net on Virtex-7 Xilinx field-programmable gate array (FPGA) platform and validated for 15 movements from DS1 to demonstrate its feasibility for real-time execution. Results: The effectiveness of the proposed LoCoMo-Net was verified by a comparative analysis against the benchmarked models using the same datasets wherein our proposed model outperformed Twin- Support Vector Machine (SVM) and existing CNN based model by an average classification accuracy of 8.5 % and 16.0 % respectively. In addition, hardware complexity analysis is done to reveal the advantages of the two-stage pipeline where approximately 27 %, 49 %, 50 %, 23 %, and 43 % savings achieved in lookup tables (LUT's), registers, memory, power consumption and computational time respectively. Conclusion: The clinical significance of such sEMG based accurate and low-complex movement recognition system can be favorable for the potential improvement in quality of life of an amputated persons.

Analysis of Influence of Segmentation, Features, and Classification in sEMG Processing: A Case Study of Recognition of Brazilian Sign Language Alphabet

Sign Language recognition systems aid communication among deaf people, hearing impaired people, and speakers. One of the types of signals that has seen increased studies and that can be used as input for these systems is surface electromyography (sEMG). This work presents the recognition of a set of alphabet gestures from Brazilian Sign Language (Libras) using sEMG acquired from an armband. Only sEMG signals were used as input. Signals from 12 subjects were acquired using a MyoTM armband for the 26 signs of the Libras alphabet. Additionally, as the sEMG has several signal processing parameters, the influence of segmentation, feature extraction, and classification was considered at each step of the pattern recognition. In segmentation, window length and the presence of four levels of overlap rates were analyzed, as well as the contribution of each feature, the literature feature sets, and new feature sets proposed for different classifiers. We found that the overlap rate had a high influence on this task. Accuracies in the order of 99% were achieved for the following factors: segments of 1.75 s with a 12.5% overlap rate; the proposed set of four features; and random forest (RF) classifiers.

EMG-Centered Multisensory Based Technologies for Pattern Recognition in Rehabilitation: State of the Art and Challenges

In the field of rehabilitation, the electromyography (EMG) signal plays an important role in interpreting patients' intentions and physical conditions. Nevertheless, utilizing merely the EMG signal suffers from difficulty in recognizing slight body movements, and the detection accuracy is strongly influenced by environmental factors. To address the above issues, multisensory integration-based EMG pattern recognition (PR) techniques have been developed in recent years, and fruitful results have been demonstrated in diverse rehabilitation scenarios, such as achieving high locomotion detection and prosthesis control accuracy. Owing to the importance and rapid development of the EMG centered multisensory fusion technologies in rehabilitation, this paper reviews both theories and applications in this emerging field. The principle of EMG signal generation and the current pattern recognition process are explained in detail, including signal preprocessing, feature extraction, classification algorithms, etc. Mechanisms of collaborations between two important multisensory fusion strategies (kinetic and kinematics) and EMG information are thoroughly explained; corresponding applications are studied, and the pros and cons are discussed. Finally, the main challenges in EMG centered multisensory pattern recognition are discussed, and a future research direction of this area is prospected.

Stable, three degree-of-freedom myoelectric prosthetic control via chronic bipolar intramuscular electrodes: a case study

Background

Modern prosthetic hands are typically controlled using skin surface electromyographic signals (EMG) from remaining muscles in the residual limb. However, surface electrode performance is limited by changes in skin impedance over time, day-to-day variations in electrode placement, and relative motion between the electrodes and underlying muscles during movement: these limitations require frequent retraining of controllers. In the presented study, we used chronically implanted intramuscular electrodes to minimize these effects and thus create a more robust prosthetic controller.

Methods

A study participant with a transradial amputation was chronically implanted with 8 intramuscular EMG electrodes. A K Nearest Neighbor (KNN) regression velocity controller was trained to predict intended joint movement direction using EMG data collected during a single training session. The resulting KNN was evaluated over 12 weeks and in multiple arm posture configurations, with the participant controlling a 3 Degree-of-Freedom (DOF) virtual reality (VR) hand to match target VR hand postures. The performance of this EMG-based controller was compared to a position-based controller that used movement measured from the participant’s opposite (intact) hand. Surface EMG was also collected for signal quality comparisons.

Results

Signals from the implanted intramuscular electrodes exhibited less crosstalk between the various channels and had a higher Signal-to-Noise Ratio than surface electrode signals. The performance of the intramuscular EMG-based KNN controller in the VR control task showed no degradation over time, and was stable over the 6 different arm postures. Both the EMG-based KNN controller and the intact hand-based controller had 100% hand posture matching success rates, but the intact hand-based controller was slightly superior in regards to speed (trial time used) and directness of the VR hand control (path efficiency).

Conclusions

Chronically implanted intramuscular electrodes provide negligible crosstalk, high SNR, and substantial VR control performance, including the ability to use a fixed controller over 12 weeks and under different arm positions. This approach can thus be a highly effective platform for advanced, multi-DOF prosthetic control.

Towards Wearable A-Mode Ultrasound Sensing for Real-Time Finger Motion Recognition

It is evident that surface electromyography (sEMG) based human-machine interfaces (HMI) have inherent difficulty in predicting dexterous musculoskeletal movements such as finger motions. This paper is an attempt to investigate a plausible alternative to sEMG, ultrasound-driven HMI, for dexterous motion recognition due to its characteristic of detecting morphological changes of deep muscles and tendons. A multi-channel A-mode ultrasound lightweight device is adopted to evaluate the performance of finger motion recognition; an experiment is designed for both widely acceptable offline and online algorithms with eight able-bodied subjects employed. The experiment result presents that the offline recognition accuracy is up to 98.83% ± 0.79%. The real-time motion completion rate is 95.4% ± 8.7% and online motion selection time is 0.243 ± 0.127 s. The outcomes confirm the feasibility of A-mode ultrasound based wearable HMI and its prosperous applications in prosthetic devices, virtual reality, and remote manipulation.

Myoelectric Signal Classification of Targeted Muscles Using Dictionary Learning

Surface electromyography (sEMG) signals comprise electrophysiological information related to muscle activity. As this signal is easy to record, it is utilized to control several myoelectric prostheses devices. Several studies have been conducted to process sEMG signals more efficiently. However, research on optimal algorithms and electrode placements for the processing of sEMG signals is still inconclusive. In addition, very few studies have focused on minimizing the number of electrodes. In this study, we investigated the most effective method for myoelectric signal classification with a small number of electrodes. A total of 23 subjects participated in the study, and the sEMG data of 14 different hand movements of the subjects were acquired from targeted muscles and untargeted muscles. Furthermore, the study compared the classification accuracy of the sEMG data using discriminative feature-oriented dictionary learning (DFDL) and other conventional classifiers. DFDL demonstrated the highest classification accuracy among the classifiers, and its higher quality performance became more apparent as the number of channels decreased. The targeted method was superior to the untargeted method, particularly when classifying sEMG signals with DFDL. Therefore, it was concluded that the combination of the targeted method and the DFDL algorithm could classify myoelectric signals more effectively with a minimal number of channels.

Grip control and motor coordination with implanted and surface electrodes while grasping with an osseointegrated prosthetic hand

Background

Replacement of a lost limb by an artificial substitute is not yet ideal. Resolution and coordination of motor control approximating that of a biological limb could dramatically improve the functionality of prosthetic devices, and thus reduce the gap towards a suitable limb replacement.

Methods

In this study, we investigated the control resolution and coordination exhibited by subjects with transhumeral amputation who were implanted with epimysial electrodes and an osseointegrated interface that provides bidirectional communication in addition to skeletal attachment (e-OPRA Implant System). We assessed control resolution and coordination in the context of routine and delicate grasping using the Pick and Lift and the Virtual Eggs Tests. Performance when utilizing implanted electrodes was compared with the standard-of-care technology for myoelectric prostheses, namely surface electrodes.

Results

Results showed that implanted electrodes provide superior controllability over the prosthetic terminal device compared to conventional surface electrodes. Significant improvements were found in the control of the grip force and its reliability during object transfer. However, these improvements failed to increase motor coordination, and surprisingly decreased the temporal correlation between grip and load forces observed with surface electrodes. We found that despite being more functional and reliable, prosthetic control via implanted electrodes still depended highly on visual feedback.

Conclusions

Our findings indicate that incidental sensory feedback (visual, auditory, and osseoperceptive in this case) is insufficient for restoring natural grasp behavior in amputees, and support the idea that supplemental tactile sensory feedback is needed to learn and maintain the motor tasks internal model, which could ultimately restore natural grasp behavior in subjects using prosthetic hands.

Performance of Combined Surface and Intramuscular EMG for Classification of Hand Movements

The surface EMG (sEMG) has been used as control source for upper limb prosthetics since decades. Previous studies suggested that intramuscular EMG showed promising results for upper limb prosthetics. This study investigates the strength of combined surface and intramuscular EMG (cEMG) for improved myoelectric control. Five able-bodied subjects and three transradial amputees were evaluated using offline classification error as performance metric. Six surface and intramuscular channels were recorded concurrently from each subject for seven consecutive days and Stacked sparse autoencoders (SSAE) and LDA classifiers were used for classification. As a control source, either sEMG channels were used or combined channels were used with reduced features using PCA. In the within session analysis, cEMG $( 2.21 \pm 1.19${%) outperformed the sEMG ($4.63 \pm 2.07${%) for both able-bodied and amputee subjects using SSAE. For between session analysis, cEMG outperformed the sEMG for both able-bodied and amputee subjects with percentage points difference of 7.93. These results imply cEMG can significantly improve the performance of pattern recognition based myoelectric control scheme for amputee subjects too and further improvement can be made by utilizing SSAE which show improved performance as compared to LDA.

Time to peak torque and acceleration time are altered in male patients following traumatic shoulder instability

BACKGROUND

Numerous authors have evaluated the strength of the rotator cuff muscles in patients with shoulder instability. However, only limited data are available with regard to neuromuscular control in patients with traumatic anterior shoulder instability, in particular at 90° of abduction. This study was designed to assess muscle strength and neuromuscular control ability using time to peak torque and acceleration time in nonathletic patients with traumatic anterior shoulder instability.

METHODS

Isokinetic muscle performance testing was performed in 20 male nonathletic anterior shoulder instability patients compared with 20 side-matched asymptomatic volunteers. Isokinetic muscle performance testing was performed at an angular velocity of 180°/s with 90° of shoulder abduction. Muscle strength and neuromuscular control (time to peak torque and acceleration time) of the internal rotators (IRs) and external rotators (ERs) were measured.

RESULTS

There were no significant differences in muscle strength of the IRs and ERs between the 2 groups. The injured shoulder showed delayed neuromuscular control in both the IRs and ERs in the instability patients compared with the normal control subjects (time to peak torque, P = .023 for IRs and P = .020 for ERs; acceleration time, P = .035 for IRs and P = .021 for ERs).

CONCLUSION

The neuromuscular control of both the IRs and ERs was decreased in male nonathletic patients with traumatic anterior shoulder instability even though muscle strength was not altered. Therefore, clinicians and therapists should implement exercises that aim to restore neuromuscular control in the rehabilitation of nonathletic patients with anterior shoulder instability.

An IoT-Enabled Stroke Rehabilitation System Based on Smart Wearable Armband and Machine Learning

Surface electromyography signal plays an important role in hand function recovery training. In this paper, an IoT-enabled stroke rehabilitation system was introduced which was based on a smart wearable armband (SWA), machine learning (ML) algorithms, and a 3-D printed dexterous robot hand. User comfort is one of the key issues which should be addressed for wearable devices. The SWA was developed by integrating a low-power and tiny-sized IoT sensing device with textile electrodes, which can measure, pre-process, and wirelessly transmit bio-potential signals. By evenly distributing surface electrodes over user’s forearm, drawbacks of classification accuracy poor performance can be mitigated. A new method was put forward to find the optimal feature set. ML algorithms were leveraged to analyze and discriminate features of different hand movements, and their performances were appraised by classification complexity estimating algorithms and principal components analysis. According to the verification results, all nine gestures can be successfully identified with an average accuracy up to 96.20%. In addition, a 3-D printed five-finger robot hand was implemented for hand rehabilitation training purpose. Correspondingly, user’s hand movement intentions were extracted and converted into a series of commands which were used to drive motors assembled inside the dexterous robot hand. As a result, the dexterous robot hand can mimic the user’s gesture in a real-time manner, which shows the proposed system can be used as a training tool to facilitate rehabilitation process for the patients after stroke.

Effect of clinical parameters on the control of myoelectric robotic prosthetic hands

Improving the functionality of prosthetic hands with noninvasive techniques is still a challenge. Surface electromyography (sEMG) currently gives limited control capabilities; however, the application of machine learning to the analysis of sEMG signals is promising and has recently been applied in practice, but many questions still remain. In this study, we recorded the sEMG activity of the forearm of 11 male subjects with transradial amputation who were mentally performing 40 hand and wrist movements. The classification performance and the number of independent movements (defined as the subset of movements that could be distinguished with >90% accuracy) were studied in relationship to clinical parameters related to the amputation. The analysis showed that classification accuracy and the number of independent movements increased significantly with phantom limb sensation intensity, remaining forearm percentage, and temporal distance to the amputation. The classification results suggest the possibility of naturally controlling up to 11 movements of a robotic prosthetic hand with almost no training. Knowledge of the relationship between classification accuracy and clinical parameters adds new information regarding the nature of phantom limb pain as well as other clinical parameters, and it can lay the foundations for future "functional amputation" procedures in surgery.

Interface Prostheses With Classifier-Feedback-Based User Training

It is evident that user training significantly affects performance of pattern-recognition-based myoelectric prosthetic device control. Despite plausible classification accuracy on offline datasets, online accuracy usually suffers from the changes in physiological conditions and electrode displacement. The user ability in generating consistent electromyographic (EMG) patterns can be enhanced via proper user training strategies in order to improve online performance. This study proposes a clustering-feedback strategy that provides real-time feedback to users by means of a visualized online EMG signal input as well as the centroids of the training samples, whose dimensionality is reduced to minimal number by dimension reduction. Clustering feedback provides a criterion that guides users to adjust motion gestures and muscle contraction forces intentionally. The experiment results have demonstrated that hand motion recognition accuracy increases steadily along the progress of the clustering-feedback-based user training, while conventional classifier-feedback methods, i.e., label feedback, hardly achieve any improvement. The result concludes that the use of proper classifier feedback can accelerate the process of user training, and implies prosperous future for the amputees with limited or no experience in pattern-recognition-based prosthetic device manipulation.It is evident that user training significantly affects performance of pattern-recognition-based myoelectric prosthetic device control. Despite plausible classification accuracy on offline datasets, online accuracy usually suffers from the changes in physiological conditions and electrode displacement. The user ability in generating consistent electromyographic (EMG) patterns can be enhanced via proper user training strategies in order to improve online performance. This study proposes a clustering-feedback strategy that provides real-time feedback to users by means of a visualized online EMG signal input as well as the centroids of the training samples, whose dimensionality is reduced to minimal number by dimension reduction. Clustering feedback provides a criterion that guides users to adjust motion gestures and muscle contraction forces intentionally. The experiment results have demonstrated that hand motion recognition accuracy increases steadily along the progress of the clustering-feedback-based user training, while conventional classifier-feedback methods, i.e., label feedback, hardly achieve any improvement. The result concludes that the use of proper classifier feedback can accelerate the process of user training, and implies prosperous future for the amputees with limited or no experience in pattern-recognition-based prosthetic device manipulation.

Simultaneous Force Regression and Movement Classification of Fingers via Surface EMG within a Unified Bayesian Framework

This contribution presents a novel methodology for myolectric-based control using surface electromyographic (sEMG) signals recorded during finger movements. A multivariate Bayesian mixture of experts (MoE) model is introduced which provides a powerful method for modeling force regression at the fingertips, while also performing finger movement classification as a by-product of the modeling algorithm. Bayesian inference of the model allows uncertainties to be naturally incorporated into the model structure. This method is tested using data from the publicly released NinaPro database which consists of sEMG recordings for 6 degree-of-freedom force activations for 40 intact subjects. The results demonstrate that the MoE model achieves similar performance compared to the benchmark set by the authors of NinaPro for finger force regression. Additionally, inherent to the Bayesian framework is the inclusion of uncertainty in the model parameters, naturally providing confidence bounds on the force regression predictions. Furthermore, the integrated clustering step allows a detailed investigation into classification of the finger movements, without incurring any extra computational effort. Subsequently, a systematic approach to assessing the importance of the number of electrodes needed for accurate control is performed via sensitivity analysis techniques. A slight degradation in regression performance is observed for a reduced number of electrodes, while classification performance is unaffected.

Electromyographic Bridge-a multi-movement volitional control method for functional electrical stimulation: prototype system design and experimental validation

The voluntary participation of the paralyzed patients is crucial for the functional electrical stimulation (FES) therapy. In this study, we developed a strategy called "EMG Bridge" (EMGB) for volitional control of multiple movements using FES technique. The surface electromyography (sEMG) signals of the agonist muscles were transformed to stimulation pulses with various pulse width and frequency to stimulate the target paralyzed muscles using MAV/NSS co-modulation (MNDC) algorithm we proposed recently. Motion pattern classification based on linear discriminant analysis (LDA) was included to recognize the motion status and mapping the sEMG detection channel to the corresponding stimulation channel. A prototype EMGB system was built for real-time control of four hand movements. The test results showed that the movements can be reproduced with a successful rate of 92.5±3.5%. The angle trajectory of wrist joint and metacarpal-phalangeal joint can be mimicked with a maximum cross-correlation coefficient > 0.84 and a latency less than 300 ms.

Characterization of In-Body to On-Body Wireless Radio Frequency Link for Upper Limb Prostheses

Wireless implanted devices can be used to interface patients with disabilities with the aim of restoring impaired motor functions. Implanted devices that record and transmit electromyographic (EMG) signals have been applied for the control of active prostheses. This simulation study investigates the propagation losses and the absorption rate of a wireless radio frequency link for in-to-on body communication in the medical implant communication service (MICS) frequency band to control myoelectric upper limb prostheses. The implanted antenna is selected and a suitable external antenna is designed. The characterization of both antennas is done by numerical simulations. A heterogeneous 3D body model and a 3D electromagnetic solver have been used to model the path loss and to characterize the specific absorption rate (SAR). The path loss parameters were extracted and the SAR was characterized, verifying the compliance with the guideline limits. The path loss model has been also used for a preliminary link budget analysis to determine the feasibility of such system compliant with the IEEE 802.15.6 standard. The resulting link margin of 11 dB confirms the feasibility of the system proposed.

A Novel Percutaneous Electrode Implant for Improving Robustness in Advanced Myoelectric Control

Despite several decades of research, electrically powered hand and arm prostheses are still controlled with very simple algorithms that process the surface electromyogram (EMG) of remnant muscles to achieve control of one prosthetic function at a time. More advanced machine learning methods have shown promising results under laboratory conditions. However, limited robustness has largely prevented the transfer of these laboratory advances to clinical applications. In this paper, we introduce a novel percutaneous EMG electrode to be implanted chronically with the aim of improving the reliability of EMG detection in myoelectric control. The proposed electrode requires a minimally invasive procedure for its implantation, similar to a cosmetic micro-dermal implant. Moreover, being percutaneous, it does not require power and data telemetry modules. Four of these electrodes were chronically implanted in the forearm of an able-bodied human volunteer for testing their characteristics. The implants showed significantly lower impedance and greater robustness against mechanical interference than traditional surface EMG electrodes used for myoelectric control. Moreover, the EMG signals detected by the proposed systems allowed more stable control performance across sessions in different days than that achieved with classic EMG electrodes. In conclusion, the proposed implants may be a promising interface for clinically available prostheses.

Residuals of autoregressive model providing additional information for feature extraction of pattern recognition-based myoelectric control

Myoelectric control based on pattern recognition has been studied for several decades. Autoregressive (AR) features are one of the mostly used feature extraction methods among myoelectric control studies. Almost all previous studies only used the AR coefficients without the residuals of AR model for classification. However, the residuals of AR model contain important amplitude information of the electromyography (EMG) signals. In this study, we added the residuals to the AR features (AR+re) and compared its performance with the classical sixth-order AR coefficients. We tested six unilateral transradial amputees and eight able-bodied subjects for eleven hand and wrist motions. The classification accuracy (CA) of the intact side for amputee subjects and the right hand for able-bodied subjects showed that the CA of AR+re features was slightly but significantly higher than that of classical AR features (p = 0.009), which meant that residuals could provide additional information to classical AR features for classification. Interestingly, the CA of the affected side for amputee subjects showed that there was no significant difference between the CA of AR+re features and classical AR features (p > 0.05). We attributed this to the fact that the amputee subjects could not use their affected side to produce consistent EMG patterns as their intact side or the dominant hand of the able-bodied subjects. Since the residuals were already available when the AR coefficients were computed, the results of this study suggested adding the residuals to classical AR features to potentially improve the performance of pattern recognition-based myoelectric control.

Wireless radio channel for intramuscular electrode implants in the control of upper limb prostheses

In the last few years the use of implanted devices has been considered also in the field of myoelectric hand prostheses. Wireless implanted EMG (Electromyogram) sensors can improve the functioning of the prosthesis, providing information without the disadvantage of the wires, and the usability by amputees. The solutions proposed in the literature are based on proprietary communication protocols between the implanted devices and the prosthesis controller, using frequency bands that are already assigned to other purposes. This study proposes the use of a standard communication protocol (IEEE 802.15.6), specific for wireless body area networks (WBANs), which assign a specific bandwidth to implanted devices. The propagation losses from in-to-on body were investigated by numerical simulation with a 3D human model and an electromagnetic solver. The channel model resulting from the study represents the first step towards the development of myoelectric prosthetic hands which are driven by signals acquired by implanted sensors. However these results can provide important information to researchers for further developments, and manufacturers, which can decrease the production costs for hand prostheses having a common standard of communication with assigned frequencies of operation.

Improving robustness against electrode shift of high density EMG for myoelectric control through common spatial patterns

Background

Most prosthetic myoelectric control studies have concentrated on low density (less than 16 electrodes, LD) electromyography (EMG) signals, due to its better clinical applicability and low computation complexity compared with high density (more than 16 electrodes, HD) EMG signals. Since HD EMG electrodes have been developed more conveniently to wear with respect to the previous versions recently, HD EMG signals become an alternative for myoelectric prostheses. The electrode shift, which may occur during repositioning or donning/doffing of the prosthetic socket, is one of the main reasons for degradation in classification accuracy (CA).

Methods

HD EMG signals acquired from the forearm of the subjects were used for pattern recognition-based myoelectric control in this study. Multiclass common spatial patterns (CSP) with two types of schemes, namely one versus one (CSP-OvO) and one versus rest (CSP-OvR), were used for feature extraction to improve the robustness against electrode shift for myoelectric control. Shift transversal (ST1 and ST2) and longitudinal (SL1 and SL2) to the direction of the muscle fibers were taken into consideration. We tested nine intact-limb subjects for eleven hand and wrist motions. The CSP features (CSP-OvO and CSP-OvR) were compared with three commonly used features, namely time-domain (TD) features, time-domain autoregressive (TDAR) features and variogram (Variog) features.

Results

Compared with the TD features, the CSP features significantly improved the CA over 10 % in all shift configurations (ST1, ST2, SL1 and SL2). Compared with the TDAR features, a. the CSP-OvO feature significantly improved the average CA over 5 % in all shift configurations; b. the CSP-OvR feature significantly improved the average CA in shift configurations ST1, SL1 and SL2. Compared with the Variog features, the CSP features significantly improved the average CA in longitudinal shift configurations (SL1 and SL2).

Conclusion

The results demonstrated that the CSP features significantly improved the robustness against electrode shift for myoelectric control with respect to the commonly used features.

Comparison of surface and intramuscular EMG pattern recognition for simultaneous wrist/hand motion classification

The simultaneous control of multiple degrees of freedom (DOFs) is important for the intuitive, life-like control of artificial limbs. The objective of this study was to determine whether the use of intramuscular electromyogram (EMG) improved pattern classification of simultaneous wrist/hand movements compared to surface EMG. Two pattern classification methods were used in this analysis, and were trained to predict 1-DOF and 2-DOF movements involving wrist rotation, wrist flexion/extension, and hand open/close. The classification methods used were (1) a single pattern classifier discriminating between 1-DOF and 2-DOF motion classes, and (2) a parallel set of three classifiers to predict the activity of each of the 3 DOFs. We demonstrate that in this combined wrist/hand classification task, the use of intramuscular EMG significantly decreases classification error compared to surface EMG for the parallel configuration (p<0.01), but not for the single classifier. We also show that the use of intramuscular EMG mitigates the increase in errors produced when the parallel classifier method is trained without 2-DOF motion class data.

User adaptation in long-term, open-loop myoelectric training: implications for EMG pattern recognition in prosthesis control

OBJECTIVE

Recent studies have reported that the classification performance of electromyographic (EMG) signals degrades over time without proper classification retraining. This problem is relevant for the applications of EMG pattern recognition in the control of active prostheses.

APPROACH

In this study we investigated the changes in EMG classification performance over 11 consecutive days in eight able-bodied subjects and two amputees.

MAIN RESULTS

It was observed that, when the classifier was trained on data from one day and tested on data from the following day, the classification error decreased exponentially but plateaued after four days for able-bodied subjects and six to nine days for amputees. The between-day performance became gradually closer to the corresponding within-day performance.

SIGNIFICANCE

These results indicate that the relative changes in EMG signal features over time become progressively smaller when the number of days during which the subjects perform the pre-defined motions are increased. The performance of the motor tasks is thus more consistent over time, resulting in more repeatable EMG patterns, even if the subjects do not have any external feedback on their performance. The learning curves for both able-bodied subjects and subjects with limb deficiencies could be modeled as an exponential function. These results provide important insights into the user adaptation characteristics during practical long-term myoelectric control applications, with implications for the design of an adaptive pattern recognition system.

Real-Time Task Discrimination for Myoelectric Control Employing Task-Specific Muscle Synergies

We present a novel formulation that employs task-specific muscle synergies and state-space representation of neural signals to tackle the challenging myoelectric control problem for lower arm prostheses. The proposed framework incorporates information about muscle configurations, e.g., muscles acting synergistically or in agonist/antagonist pairs, using the hypothesis of muscle synergies. The synergy activation coefficients are modeled as the latent system state and are estimated using a constrained Kalman filter. These task-dependent synergy activation coefficients are estimated in real-time from the electromyogram (EMG) data and are used to discriminate between various tasks. The task discrimination is helped by a post-processing algorithm that uses posterior probabilities. The proposed algorithm is robust as well as computationally efficient, yielding a decision with > 90% discrimination accuracy in approximately 3 ms . The real-time performance and controllability of the algorithm were evaluated using the targeted achievement control (TAC) test. The proposed algorithm outperformed common machine learning algorithms for single- as well as multi-degree-of-freedom (DOF) tasks in both off-line discrimination accuracy and real-time controllability (p < 0.01).

Improving Myoelectric Control for Amputees through Transcranial Direct Current Stimulation

Most prosthetic myoelectric control studies have shown good performance for unimpaired subjects. However, performance is generally unacceptable for amputees. The primary problem is the poor quality of electromyography (EMG) signals of amputees compared with healthy individuals. To improve clinical performance of myoelectric control, this study explored transcranial direct current stimulation (tDCS) to modulate brain activity and enhance EMG quality. We tested six unilateral transradial amputees by applying active and sham anodal tDCS separately on two different days. Surface EMG signals were acquired from the affected and intact sides for 11 hand and wrist motions in the pre-tDCS and post-tDCS sessions. Autoregression coefficients and linear discriminant analysis classifiers were used to process the EMG data for pattern recognition of the 11 motions. For the affected side, active anodal tDCS significantly reduced the average classification error rate (CER) by 10.1%, while sham tDCS had no such effect. For the intact side, the average CER did not change on the day of sham tDCS but increased on the day of active tDCS. These results demonstrated that tDCS could modulate brain function and improve EMG-based classification performance for amputees. It has great potential in dramatically reducing the length of learning process of amputees for effectively using myoelectrically controlled multifunctional prostheses.

A real-time, practical sensor fault-tolerant module for robust EMG pattern recognition

BACKGROUND

Unreliability of surface EMG recordings over time is a challenge for applying the EMG pattern recognition (PR)-controlled prostheses in clinical practice. Our previous study proposed a sensor fault-tolerant module (SFTM) by utilizing redundant information in multiple EMG signals. The SFTM consists of multiple sensor fault detectors and a self-recovery mechanism that can identify anomaly in EMG signals and remove the recordings of the disturbed signals from the input of the pattern classifier to recover the PR performance. While the proposed SFTM has shown great promise, the previous design is impractical. A practical SFTM has to be fast enough, lightweight, automatic, and robust under different conditions with or without disturbances.

METHODS

This paper presented a real-time, practical SFTM towards robust EMG PR. A novel fast LDA retraining algorithm and a fully automatic sensor fault detector based on outlier detection were developed, which allowed the SFTM to promptly detect disturbances and recover the PR performance immediately. These components of SFTM were then integrated with the EMG PR module and tested on five able-bodied subjects and a transradial amputee in real-time for classifying multiple hand and wrist motions under different conditions with different disturbance types and levels.

RESULTS

The proposed fast LDA retraining algorithm significantly shortened the retraining time from nearly 1 s to less than 4 ms when tested on the embedded system prototype, which demonstrated the feasibility of a nearly "zero-delay" SFTM that is imperceptible to the users. The results of the real-time tests suggested that the SFTM was able to handle different types of disturbances investigated in this study and significantly improve the classification performance when one or multiple EMG signals were disturbed. In addition, the SFTM could also maintain the system's classification performance when there was no disturbance.

CONCLUSIONS

This paper presented a real-time, lightweight, and automatic SFTM, which paved the way for reliable and robust EMG PR for prosthesis control.