Towards resolving the co-existing impacts of multiple dynamic factors on the performance of EMG-pattern recognition based prostheses

Unravelling Influence Factors in Pattern Recognition Myoelectric Control Systems: The Impact of Limb Positions and Electrode Shifts

Pattern recognition (PR)-based myoelectric control systems can naturally provide multifunctional and intuitive control of upper limb prostheses and restore lost limb function, but understanding their robustness remains an open scientific question. This study investigates how limb positions and electrode shifts-two factors that have been suggested to cause classification deterioration-affect classifiers' performance by quantifying changes in the class distribution using each factor as a class and computing the repeatability and modified separability indices. Ten intact-limb participants took part in the study. Linear discriminant analysis (LDA) was used as the classifier. The results confirmed previous studies that limb positions and electrode shifts deteriorate classification performance (14-21% decrease) with no difference between factors (p > 0.05). When considering limb positions and electrode shifts as classes, we could classify them with an accuracy of 96.13 ± 1.44% and 65.40 ± 8.23% for single and all motions, respectively. Testing on five amputees corroborated the above findings. We have demonstrated that each factor introduces changes in the feature space that are statistically new class instances. Thus, the feature space contains two statistically classifiable clusters when the same motion is collected in two different limb positions or electrode shifts. Our results are a step forward in understanding PR schemes' challenges for myoelectric control of prostheses and further validation needs be conducted on more amputee-related datasets.

Use of Artificial Intelligence Techniques to Assist Individuals with Physical Disabilities

Assistive technologies (AT) enable people with disabilities to perform activities of daily living more independently, have greater access to community and healthcare services, and be more productive performing educational and/or employment tasks. Integrating artificial intelligence (AI) with various agents, including electronics, robotics, and software, has revolutionized AT, resulting in groundbreaking technologies such as mind-controlled exoskeletons, bionic limbs, intelligent wheelchairs, and smart home assistants. This article provides a review of various AI techniques that have helped those with physical disabilities, including brain-computer interfaces, computer vision, natural language processing, and human-computer interaction. The current challenges and future directions for AI-powered advanced technologies are also addressed.

A novel unsupervised dynamic feature domain adaptation strategy for cross-individual myoelectric gesture recognition

Objective.Surface electromyography pattern recognition (sEMG-PR) is considered as a promising control method for human-machine interaction systems. However, the performance of a trained classifier would greatly degrade for novel users since sEMG signals are user-dependent and largely affected by a number of individual factors such as the quantity of subcutaneous fat and the skin impedance.Approach.To solve this issue, we proposed a novel unsupervised cross-individual motion recognition method that aligned sEMG features from different individuals by self-adaptive dimensional dynamic distribution adaptation (SD-DDA) in this study. In the method, both the distances of marginal and conditional distributions between source and target features were minimized through automatically selecting the optimal feature domain dimension by using a small amount of unlabeled target data.Main results.The effectiveness of the proposed method was tested on four different feature sets, and results showed that the average classification accuracy was improved by above 10% on our collected dataset with the best accuracy reached 90.4%. Compared to six kinds of classic transfer learning methods, the proposed method showed an outstanding performance with improvements of 3.2%-13.8%. Additionally, the proposed method achieved an approximate 9% improvement on a publicly available dataset.Significance.These results suggested that the proposed SD-DDA method is feasible for cross-individual motion intention recognition, which would provide help for the application of sEMG-PR based system.

The LIBRA NeuroLimb: Hybrid Real-Time Control and Mechatronic Design for Affordable Prosthetics in Developing Regions

Globally, 2.5% of upper limb amputations are transhumeral, and both mechanical and electronic prosthetics are being developed for individuals with this condition. Mechanics often require compensatory movements that can lead to awkward gestures. Electronic types are mainly controlled by superficial electromyography (sEMG). However, in proximal amputations, the residual limb is utilized less frequently in daily activities. Muscle shortening increases with time and results in weakened sEMG readings. Therefore, sEMG-controlled models exhibit a low success rate in executing gestures. The LIBRA NeuroLimb prosthesis is introduced to address this problem. It features three active and four passive degrees of freedom (DOF), offers up to 8 h of operation, and employs a hybrid control system that combines sEMG and electroencephalography (EEG) signal classification. The sEMG and EEG classification models achieve up to 99% and 76% accuracy, respectively, enabling precise real-time control. The prosthesis can perform a grip within as little as 0.3 s, exerting up to 21.26 N of pinch force. Training and validation sessions were conducted with two volunteers. Assessed with the "AM-ULA" test, scores of 222 and 144 demonstrated the prosthesis's potential to improve the user's ability to perform daily activities. Future work will prioritize enhancing the mechanical strength, increasing active DOF, and refining real-world usability.

Transformer-based hand gesture recognition from instantaneous to fused neural decomposition of high-density EMG signals

Designing efficient and labor-saving prosthetic hands requires powerful hand gesture recognition algorithms that can achieve high accuracy with limited complexity and latency. In this context, the paper proposes a Compact Transformer-based Hand Gesture Recognition framework referred to as [Formula: see text], which employs a vision transformer network to conduct hand gesture recognition using high-density surface EMG (HD-sEMG) signals. Taking advantage of the attention mechanism, which is incorporated into the transformer architectures, our proposed [Formula: see text] framework overcomes major constraints associated with most of the existing deep learning models such as model complexity; requiring feature engineering; inability to consider both temporal and spatial information of HD-sEMG signals, and requiring a large number of training samples. The attention mechanism in the proposed model identifies similarities among different data segments with a greater capacity for parallel computations and addresses the memory limitation problems while dealing with inputs of large sequence lengths. [Formula: see text] can be trained from scratch without any need for transfer learning and can simultaneously extract both temporal and spatial features of HD-sEMG data. Additionally, the [Formula: see text] framework can perform instantaneous recognition using sEMG image spatially composed from HD-sEMG signals. A variant of the [Formula: see text] is also designed to incorporate microscopic neural drive information in the form of Motor Unit Spike Trains (MUSTs) extracted from HD-sEMG signals using Blind Source Separation (BSS). This variant is combined with its baseline version via a hybrid architecture to evaluate potentials of fusing macroscopic and microscopic neural drive information. The utilized HD-sEMG dataset involves 128 electrodes that collect the signals related to 65 isometric hand gestures of 20 subjects. The proposed [Formula: see text] framework is applied to 31.25, 62.5, 125, 250 ms window sizes of the above-mentioned dataset utilizing 32, 64, 128 electrode channels. Our results are obtained via 5-fold cross-validation by first applying the proposed framework on the dataset of each subject separately and then, averaging the accuracies among all the subjects. The average accuracy over all the participants using 32 electrodes and a window size of 31.25 ms is 86.23%, which gradually increases till reaching 91.98% for 128 electrodes and a window size of 250 ms. The [Formula: see text] achieves accuracy of 89.13% for instantaneous recognition based on a single frame of HD-sEMG image. The proposed model is statistically compared with a 3D Convolutional Neural Network (CNN) and two different variants of Support Vector Machine (SVM) and Linear Discriminant Analysis (LDA) models. The accuracy results for each of the above-mentioned models are paired with their precision, recall, F1 score, required memory, and train/test times. The results corroborate effectiveness of the proposed [Formula: see text] framework compared to its counterparts.

Enhanced Deep Transfer Learning Model based on Spatial-Temporal driven Scalograms for Precise Decoding of Motor Intent in Stroke Survivors

Intelligent rehabilitation robotics (RR) have been proposed in recent years to aid post-stroke survivors recover their lost limb functions. However, a large proportion of these robotic systems operate in a passive mode that restricts users to predefined trajectories that rarely align with their intended limb movements, precluding full functional recovery. To address this issue, an efficient Transfer Learning based Convolutional Neural Network (TL-CNN) model is proposed to decode post-stroke patients' motion intentions toward realizing dexterously active robotic training during rehabilitation. For the first time, we use Spatial-Temporal Descriptor based Continuous Wavelet Transform (STD-CWT) as input to TL-CNN to optimally decode limb movement intent patterns. We evaluated the STD-CWT method on three distinct wavelets including the Morse, Amor, and Bump, and compared their decoding outcomes with those of the commonly adopted CWT technique under similar experimental conditions. We then validated the method using electromyogram signals of five stroke survivors who performed twenty-one distinct motor tasks. The results showed that the proposed technique recorded a significantly higher (p<0.05) decoding accuracy and faster convergence compared to the common method. Our method equally recorded obvious class separability for individual motor tasks across subjects. The findings suggest that the STD-CWT Scalograms have the potential for robust decoding of motor intention and could facilitate intuitive and active motor training in stroke RR.Clinical Relevance- The study demonstrated the potential of Spatial Temporal based Scalograms in aiding precise and robust decoding of multi-class motor tasks, upon which dexterously active rehabilitation robotic training for full motor function restoration could be realized.

Ascertaining the optimal myoelectric signal recording duration for pattern recognition based prostheses control

Introduction

Electromyogram-based pattern recognition (EMG-PR) has been widely considered an essentially intuitive control method for multifunctional upper limb prostheses. A crucial aspect of the scheme is the EMG signal recording duration (SRD) from which requisite motor tasks are characterized per time, impacting the system's overall performance. For instance, lengthy SRD inevitably introduces fatigue (that alters the muscle contraction patterns of specific limb motions) and may incur high computational costs in building the motion intent decoder, resulting in inadequate prosthetic control and controller delay in practical usage. Conversely, relatively shorter SRD may lead to reduced data collection durations that, among other advantages, allow for more convenient prosthesis recalibration protocols. Therefore, determining the optimal SRD required to characterize limb motion intents adequately that will aid intuitive PR-based control remains an open research question.

Method

This study systematically investigated the impact and generalizability of varying lengths of myoelectric SRD on the characterization of multiple classes of finger gestures. The investigation involved characterizing fifteen classes of finger gestures performed by eight normally limb subjects using various groups of EMG SRD including 1, 5, 10, 15, and 20 s. Two different training strategies including Between SRD and Within-SRD were implemented across three popular machine learning classifiers and three time-domain features to investigate the impact of SRD on EMG-PR motion intent decoder.

Result

The between-SRD strategy results which is a reflection of the practical scenario showed that an SRD greater than 5 s but less than or equal to 10 s (>5 and < = 10 s) would be required to achieve decent average finger gesture decoding accuracy for all feature-classifier combinations. Notably, lengthier SRD would incur more acquisition and implementation time and vice-versa. In inclusion, the study's findings provide insight and guidance into selecting appropriate SRD that would aid inadequate characterization of multiple classes of limb motion tasks in PR-based control schemes for multifunctional prostheses.

Improving the Robustness of Human-Machine Interactive Control for Myoelectric Prosthetic Hand During Arm Position Changing

Robust classification of natural hand grasp type based on electromyography (EMG) still has some shortcomings in the practical prosthetic hand control, owing to the influence of dynamic arm position changing during hand actions. This study provided a framework for robust hand grasp type classification during dynamic arm position changes, improving both the “hardware” and “algorithm” components. In the hardware aspect, co-located synchronous EMG and force myography (FMG) signals are adopted as the multi-modal strategy. In the algorithm aspect, a sequential decision algorithm is proposed by combining the RNN-based deep learning model with a knowledge-based post-processing model. Experimental results showed that the classification accuracy of multi-modal EMG-FMG signals was increased by more than 10% compared with the EMG-only signal. Moreover, the classification accuracy of the proposed sequential decision algorithm improved the accuracy by more than 4% compared with other baseline models when using both EMG and FMG signals.

Identifying Oscillations under Multi-site Sensory Stimulation for High-level Peripheral Nerve Injured Patients：A Pilot Study

OBJECTIVE

For high-level peripheral nerve injured (PNI) patients with severe sensory dysfunction of upper extremities, identifying the multi-site tactile stimulation is of great importance to provide neurorehabilitation with sensory feedback. In this pilot study, we showed the feasibility of identifying multi-site and multi-intensity tactile stimulation in terms of electroencephalography (EEG).

APPROACH

Three high-level PNI patients and eight non-PNI participants were recruited in this study. Four different sites over the upper arm, forearm, thumb finger and little finger were randomly stimulated at two intensities (both sensory-level) based on the transcutaneous electrical nerve stimulation (TENS). Meanwhile, 64-channel EEG signals were recorded during the passive tactile sense stimulation on each side.

MAIN RESULTS

The spatial-spectral distribution of brain oscillations underlying multi-site sensory stimulation showed dominant power attenuation over the somatosensory and prefrontal cortices in both alpha-band (8-12 Hz) and beta-band (13-30 Hz). But there was no significant difference among different stimulation sites in terms of the averaged power spectral density over the region of interest (ROI). By further identifying different stimulation sites using temporal-spectral features, we found the classification accuracies were all above 89% for the affected arm of PNI patients, comparable to that from their intact side and that from the non-PNI group. When the stimulation site-intensity combinations were treated as eight separate classes, the classification accuracies were ranging from 88.89% to 99.30% for the affected side of PNI subjects, similar to that from their non-affected side and that from the non-PNI group. Other performance metrics, including Specificity, Precision, and F1-Score, also showed a sound identification performance for both PNI patients and non-PNI subjects.

SIGNIFICANCE

These results suggest that reliable brain oscillations could be evoked and identified well, even though induced tactile sense could not be discerned by the PNI patients. This study have implication for facilitating bidirectional neurorehabilitation systems with sensory feedback.

Review on electromyography based intention for upper limb control using pattern recognition for human-machine interaction

Upper limb myoelectric prosthetic control is an essential topic in the field of rehabilitation. The technique controls prostheses using surface electromyogram (sEMG) and intramuscular EMG (iEMG) signals. EMG signals are extensively used in controlling prosthetic upper and lower limbs, virtual reality entertainment, and human-machine interface (HMI). EMG signals are vital parameters for machine learning and deep learning algorithms and help to give an insight into the human brain’s function and mechanisms. Pattern recognition techniques pertaining to support vector machine (SVM), k-nearest neighbor (KNN) and Bayesian classifiers have been utilized to classify EMG signals. This paper presents a review on current EMG signal techniques, including electrode array utilization, signal acquisition, signal preprocessing and post-processing, feature selection and extraction, data dimensionality reduction, classification, and ultimate application to the community. The paper also discusses using alternatives to EMG signals, such as force sensors, to measure muscle activity with reliable results. Future implications for EMG classification include employing deep learning techniques such as artificial neural networks (ANN) and recurrent neural networks (RNN) for achieving robust results.

Myoelectric Pattern Recognition Performance Enhancement Using Nonlinear Features

The multichannel electrode array used for electromyogram (EMG) pattern recognition provides good performance, but it has a high cost, is computationally expensive, and is inconvenient to wear. Therefore, researchers try to use as few channels as possible while maintaining improved pattern recognition performance. However, minimizing the number of channels affects the performance due to the least separable margin among the movements possessing weak signal strengths. To meet these challenges, two time-domain features based on nonlinear scaling, the log of the mean absolute value (LMAV) and the nonlinear scaled value (NSV), are proposed. In this study, we validate the proposed features on two datasets, the existing four feature extraction methods, variable window size, and various signal-to-noise ratios (SNR). In addition, we also propose a feature extraction method where the LMAV and NSV are grouped with the existing 11 time-domain features. The proposed feature extraction method enhances accuracy, sensitivity, specificity, precision, and F1 score by 1.00%, 5.01%, 0.55%, 4.71%, and 5.06% for dataset 1, and 1.18%, 5.90%, 0.66%, 5.63%, and 6.04% for dataset 2, respectively. Therefore, the experimental results strongly suggest the proposed feature extraction method, for taking a step forward with regard to improved myoelectric pattern recognition performance.

Force-Invariant Improved Feature Extraction Method for Upper-Limb Prostheses of Transradial Amputees

A force-invariant feature extraction method derives identical information for all force levels. However, the physiology of muscles makes it hard to extract this unique information. In this context, we propose an improved force-invariant feature extraction method based on nonlinear transformation of the power spectral moments, changes in amplitude, and the signal amplitude along with spatial correlation coefficients between channels. Nonlinear transformation balances the forces and increases the margin among the gestures. Additionally, the correlation coefficient between channels evaluates the amount of spatial correlation; however, it does not evaluate the strength of the electromyogram signal. To evaluate the robustness of the proposed method, we use the electromyogram dataset containing nine transradial amputees. In this study, the performance is evaluated using three classifiers with six existing feature extraction methods. The proposed feature extraction method yields a higher pattern recognition performance, and significant improvements in accuracy, sensitivity, specificity, precision, and F1 score are found. In addition, the proposed method requires comparatively less computational time and memory, which makes it more robust than other well-known feature extraction methods.

Feature Extraction of Surface Electromyography Using Wavelet Weighted Permutation Entropy for Hand Movement Recognition

The feature extraction of surface electromyography (sEMG) signals has been an important aspect of myoelectric prosthesis control. To improve the practicability of myoelectric prosthetic hands, we proposed a feature extraction method for sEMG signals that uses wavelet weighted permutation entropy (WWPE). First, wavelet transform was used to decompose and preprocess sEMG signals collected from the relevant muscles of the upper limbs to obtain the wavelet sub-bands in each frequency segment. Then, the weighted permutation entropies (WPEs) of the wavelet sub-bands were extracted to construct WWPE feature set. Lastly, the WWPE feature set was used as input to a support vector machine (SVM) classifier and a backpropagation neural network (BPNN) classifier to recognize seven hand movements. Experimental results show that the proposed method exhibits remarkable recognition accuracy that is superior to those of single sub-band feature set and commonly used time-domain feature set. The maximum recognition accuracy rate is 100% for hand movements, and the average recognition accuracy rates of SVM and BPNN are 100% and 98%, respectively.

Characterizing Multiple Patterns of Motor Intent Using Spatial-Temporal Information for Intuitively Active Motor Training in Stroke Survivors

Upper extremity motor function loss severely affects stroke survivors during daily life activities. Different rehabilitation robotic systems have been developed to allow stroke survivors regain their motor function. Meanwhile, most of the robots only operate in a passive mode and restrict the users to navigate predefined trajectories which may not align with their motion intent, thus limiting motor recovery. One way to resolve this issue would be to utilize a decoded movement intent to trigger intuitively active motor training for patients. In this direction, this study proposed and investigated the use of spatial-temporal neuromuscular descriptor (STD) for optimal decoding of multiple patterns of movement intents in patient to provide inputs for active motor training in the rehabilitation robotic systems. The STD performance was validated using High-Density surface electromyogram recordings from five stroke survivors who performed 21 limb movements. Experimental results show that the STD achieved a significant reduction in limb movement classification error (13.36%) even in the presence of the inevitable White Gaussian Noise compared to other methods (p<0.05). The STD also showed obvious class separability for individual movement. Findings from this study suggest that the STD may provide potential inputs for intuitively active motor training in stroke rehabilitation robotic systems.Clinical Relevance- This study showed that spatial-temporal neuromuscular information could aid adequate decoding of movement intents upon which intuitively active motor training could be achieved in stroke rehabilitation robotic systems.

Appropriate Feature Set and Window Parameters Selection for Efficient Motion Intent Characterization towards Intelligently Smart EMG-PR System

The constantly rising number of limb stroke survivors and amputees has motivated the development of intelligent prosthetic/rehabilitation devices for their arm function restoration. The device often integrates a pattern recognition (PR) algorithm that decodes amputees’ limb movement intent from electromyogram (EMG) signals, characterized by neural information and symmetric distribution. However, the control performance of the prostheses mostly rely on the interrelations among multiple dynamic factors of feature set, windowing parameters, and signal conditioning that have rarely been jointly investigated to date. This study systematically investigated the interaction effects of these dynamic factors on the performance of EMG-PR system towards constructing optimal parameters for accurately robust movement intent decoding in the context of prosthetic control. In this regard, the interaction effects of various features across window lengths (50 ms~300 ms), increments (50 ms~125 ms), robustness to external interferences and sensor channels (2 ch~6 ch), were examined using EMG signals obtained from twelve subjects through a symmetrical movement elicitation protocol. Compared to single features, multiple features consistently achieved minimum decoding error below 10% across optimal windowing parameters of 250 ms/100 ms. Also, the multiple features showed high robustness to additive noise with obvious trade-offs between accuracy and computation time. Consequently, our findings may provide proper insight for appropriate parameter selection in the context of robust PR-based control strategy for intelligent rehabilitation device.

Exploring Stiffness Modulation in Prosthetic Hands and Its Perceived Function in Manipulation and Social Interaction

To physically interact with a rich variety of environments and to match situation-dependent requirements, humans adapt both the force and stiffness of their limbs. Reflecting this behavior in prostheses may promote a more natural and intuitive control and, consequently, improve prostheses acceptance in everyday life. This pilot study proposes a method to control a prosthetic robot hand and its impedance, and explores the utility of variable stiffness when performing activities of daily living and physical social interactions. The proposed method is capable of a simultaneous and proportional decoding of position and stiffness intentions from two surface electro-myographic sensors placed over a pair of antagonistic muscles. The feasibility of our approach is validated and compared to existing control modalities in a preliminary study involving one prosthesis user. The algorithm is implemented in a soft under-actuated prosthetic hand (SoftHand Pro). Then, we evaluate the usability of the proposed approach while executing a variety of tasks. Among these tasks, the user interacts with other 12 able-bodied subjects, whose experiences were also assessed. Several statistically significant aspects from the System Usability Scale indicate user's preference of variable stiffness control over low or high constant stiffness due to its reactivity and adaptability. Feedback reported by able-bodied subjects reveal a general tendency to favor soft interaction, i.e., low stiffness, which is perceived more human-like and comfortable. These combined results suggest the use of variable stiffness as a viable compromise between firm control and safe interaction which is worth investigating further.