Evaluation of Computer-Based Target Achievement Tests for Myoelectric Control

Simultaneous Three-Degrees-of-Freedom Prosthetic Control Based on Linear Regression and Closed-Loop Training Protocol

Machine learning-based controllers of prostheses using electromyographic signals have become very popular in the last decade. The regression approach allows a simultaneous and proportional control of the intended movement in a more natural way than the classification approach, where the number of movements is discrete by definition. However, it is not common to find regression-based controllers working for more than two degrees of freedom at the same time. In this paper, we present the application of the adaptive linear regressor in a relatively low-dimensional feature space with only eight sensors to the problem of a simultaneous and proportional control of three degrees of freedom (left-right, up-down and open-close hand movements). We show that a key element usually overlooked in the learning process of the regressor is the training paradigm. We propose a closed-loop procedure, where the human learns how to improve the quality of the generated EMG signals, helping also to obtain a better controller. We apply it to 10 healthy and 3 limb-deficient subjects. Results show that the combination of the multidimensional targets and the open-loop training protocol significantly improve the performance, increasing the average completion rate from 53% to 65% for the most complicated case of simultaneously controlling the three degrees of freedom.

A Sonomyography-Based Muscle Computer Interface for Individuals With Spinal Cord Injury

Impairment of hand functions in individuals with spinal cord injury (SCI) severely disrupts activities of daily living. Recent advances have enabled rehabilitation assisted by robotic devices to augment the residual function of the muscles. Traditionally, electromyography-based muscle activity sensing interfaces have been utilized to sense volitional motor intent to drive robotic assistive devices. However, the dexterity and fidelity of control that can be achieved with electromyography-based control have been limited due to inherent limitations in signal quality. We have developed and tested a muscle-computer interface (MCI) utilizing sonomyography to provide control of a virtual cursor for individuals with motor-incomplete spinal cord injury. We demonstrate that individuals with SCI successfully gained control of a virtual cursor by utilizing contractions of muscles of the wrist joint. The sonomyography-based interface enabled control of the cursor at multiple graded levels demonstrating the ability to achieve accurate and stable endpoint control. Our sonomyography-based muscle-computer interface can enable dexterous control of upper-extremity assistive devices for individuals with motor-incomplete SCI.

Current status and clinical perspectives of extended reality for myoelectric prostheses: review

Training with "Extended Reality" or X-Reality (XR) systems can undoubtedly enhance the control of the myoelectric prostheses. However, there is no consensus on which factors improve the efficiency of skill transfer from virtual training to actual prosthesis abilities. This review examines the current status and clinical applications of XR in the field of myoelectric prosthesis training and analyses possible influences on skill migration. We have conducted a thorough search on databases in the field of prostheses using keywords such as extended reality, virtual reality and serious gaming. Our scoping review encompassed relevant applications, control methods, performance evaluation and assessment metrics. Our findings indicate that the implementation of XR technology for myoelectric rehabilitative training on prostheses provides considerable benefits. Additionally, there are numerous standardised methods available for evaluating training effectiveness. Recently, there has been a surge in the number of XR-based training tools for myoelectric prostheses, with an emphasis on user engagement and virtual training evaluation. Insufficient attention has been paid to significant limitations in the behaviour, functionality, and usage patterns of XR and myoelectric prostheses, potentially obstructing the transfer of skills and prospects for clinical application. Improvements are recommended in four critical areas: activities of daily living, training strategies, feedback, and the alignment of the virtual environment with the physical devices.

A CW-CNN regression model-based real-time system for virtual hand control

For upper limb amputees, wearing a myoelectric prosthetic hand is the only way for them to continue normal life. Even until now, the proposal of a high-precision and natural performance real-time control system based on surface electromyography (sEMG) signals is still challenging. Researchers have proposed many strategies for motion classification or regression prediction tasks based on sEMG signals. However, most of them have been limited to offline analysis only. There are even few papers on real-time control based on deep learning models, almost all of which are about motion classification. Rare studies tried to use deep learning-based regression models in real-time control systems for multi-joint angle estimation via sEMG signals. This paper proposed a CW-CNN regression model-based real-time control system for virtual hand control. We designed an Adaptive Kalman Filter to smooth the joint angles output before sending them as control commands to control a virtual hand. Eight healthy participants were invited, and three sessions experiments were conducted on two different days for all of them. During the real-time experiment, we analyzed the joint angles estimation accuracy and computational latency. Moreover, target achievement control (TAC) test was applied to emphasize motion regression in real-time. The experimental results show that the proposed control system has high precision for 3-DOFs motion regression in simultaneously, and the system remains stable and low computational latency. In the future, the proposed real-time control system can be applied to actual prosthetic hand.

The Influence of Training with Visual Biofeedback on the Predictability of Myoelectric Control Usability

Studies have shown that closed-loop myoelectric control schemes can lead to changes in user performance and behavior compared to open-loop systems. When users are placed within the control loop, such as during real-time use, they must correct for errors made by the controller and learn what behavior is necessary to produce desired outcomes. Augmented feedback, consequently, has been used to incorporate the user throughout the training process and to facilitate learning. This work explores the effect of visual feedback presented during user training on both the performance and predictability of a myoelectric classification-based control system. Our results suggest that properly designed feedback mechanisms and training tasks can influence the quality of the training data and the ability to predict usability using linear combinations of metrics derived from feature space. Furthermore, our results confirm that the most common in-lab training protocol, screen guided training, may yield training data that are less representative of online use than training protocols that incorporate the user in the loop. These results suggest that training protocols should be designed that better parallel the testing environment to more effectively prepare both the algorithms and users for real-time control.

An Adaptive Multi-Modal Control Strategy to Attenuate the Limb Position Effect in Myoelectric Pattern Recognition

Over the last few decades, pattern recognition algorithms have shown promising results in the field of upper limb prostheses myoelectric control and are now gradually being incorporated in commercial devices. A widely used approach is based on a classifier which assigns a specific input value to a selected hand motion. While this method guarantees good performance and robustness within each class, it still shows limitations in adapting to different conditions encountered in real-world applications, such as changes in limb position or external loads. This paper proposes an adaptive method based on a pattern recognition classifier that takes advantage of an augmented dataset-i.e., representing variations in limb position or external loads-to selectively adapt to underrepresented variations. The proposed method was evaluated using a series of target achievement control tests with ten able-bodied volunteers. Results indicated a higher median completion rate >3.33% for the adapted algorithm compared to a classical pattern recognition classifier used as a baseline model. Subject-specific performance showed the potential for improved control after adaptation and a ≤13% completion rate; and in many instances, the adapted points were able to provide new information within classes. These preliminary results show the potential of the proposed method and encourage further development.

A Multi-Variate Approach to Predicting Myoelectric Control Usability

Pattern recognition techniques leveraging the use of electromyography signals have become a popular approach to provide intuitive control of myoelectric devices. Performance of these control interfaces is commonly quantified using offline classification accuracy, despite studies having shown that this metric is a poor indicator of usability. Researchers have identified alternative offline metrics that better correlate with online performance; however, the relationship has yet to be fully defined in the literature. This has necessitated the continued trial-and-error-style online testing of algorithms developed using offline approaches. To bridge this information divide, we conducted an exploratory study where thirty-two different metrics from the offline training data were extracted. A correlation analysis and an ordinary least squares regression were implemented to investigate the relationship between the offline metrics and six aspects online use. The results indicate that the current offline standard, classification accuracy, is a poor indicator of usability and that other metrics may hold predictive power. The metrics identified in this work also may constitute more representative evaluation criteria when designing and reporting new control schemes. Furthermore, linear combinations of offline training metrics generate substantially more accurate predictions than using individual metrics. We found that the offline metric feature efficiency generated the best predictions for the usability metric throughput. A combination of two offline metrics (mean semi-principal axes and mean absolute value) significantly outperformed feature efficiency alone, with a 166% increase in the predicted R² value (i.e., VEcv). These findings suggest that combinations of metrics could provide a more robust framework for predicting usability.

Gesture Recognition Using Surface Electromyography and Deep Learning for Prostheses Hand: State-of-the-Art, Challenges, and Future

Amputation of the upper limb brings heavy burden to amputees, reduces their quality of life, and limits their performance in activities of daily life. The realization of natural control for prosthetic hands is crucial to improving the quality of life of amputees. Surface electromyography (sEMG) signal is one of the most widely used biological signals for the prediction of upper limb motor intention, which is an essential element of the control systems of prosthetic hands. The conversion of sEMG signals into effective control signals often requires a lot of computational power and complex process. Existing commercial prosthetic hands can only provide natural control for very few active degrees of freedom. Deep learning (DL) has performed surprisingly well in the development of intelligent systems in recent years. The significant improvement of hardware equipment and the continuous emergence of large data sets of sEMG have also boosted the DL research in sEMG signal processing. DL can effectively improve the accuracy of sEMG pattern recognition and reduce the influence of interference factors. This paper analyzes the applicability and efficiency of DL in sEMG-based gesture recognition and reviews the key techniques of DL-based sEMG pattern recognition for the prosthetic hand, including signal acquisition, signal preprocessing, feature extraction, classification of patterns, post-processing, and performance evaluation. Finally, the current challenges and future prospects in clinical application of these techniques are outlined and discussed.

Can spatial filtering separate voluntary and involuntary components in children with dyskinetic cerebral palsy?

The design of myocontrolled devices faces particular challenges in children with dyskinetic cerebral palsy because the electromyographic signal for control contains both voluntary and involuntary components. We hypothesized that voluntary and involuntary components of movements would be uncorrelated and thus detectable as different synergistic patterns of muscle activity, and that removal of the involuntary components would improve online EMG-based control. Therefore, we performed a synergy-based decomposition of EMG-guided movements, and evaluated which components were most controllable using a Fitts' Law task. Similarly, we also tested which muscles were most controllable. We then tested whether removing the uncontrollable components or muscles improved overall function in terms of movement time, success rate, and throughput. We found that removal of less controllable components or muscles did not improve EMG control performance, and in many cases worsened performance. These results suggest that abnormal movement in dyskinetic CP is consistent with a pervasive distortion of voluntary movement rather than a superposition of separable voluntary and involuntary components of movement.

A Multiday Evaluation of Real-Time Intramuscular EMG Usability with ANN

Recent developments in implantable technology, such as high-density recordings, wireless transmission of signals to a prosthetic hand, may pave the way for intramuscular electromyography (iEMG)-based myoelectric control in the future. This study aimed to investigate the real-time control performance of iEMG over time. A novel protocol was developed to quantify the robustness of the real-time performance parameters. Intramuscular wires were used to record EMG signals, which were kept inside the muscles for five consecutive days. Tests were performed on multiple days using Fitts' law. Throughput, completion rate, path efficiency and overshoot were evaluated as performance metrics using three train/test strategies. Each train/test scheme was categorized on the basis of data quantity and the time difference between training and testing data. An artificial neural network (ANN) classifier was trained and tested on (i) data from the same day (WDT), (ii) data collected from the previous day and tested on present-day (BDT) and (iii) trained on all previous days including the present day and tested on present-day (CDT). It was found that the completion rate (91.6 ± 3.6%) of CDT was significantly better (p < 0.01) than BDT (74.02 ± 5.8%) and WDT (88.16 ± 3.6%). For BDT, on average, the first session of each day was significantly better (p < 0.01) than the second and third sessions for completion rate (77.9 ± 14.0%) and path efficiency (88.9 ± 16.9%). Subjects demonstrated the ability to achieve targets successfully with wire electrodes. Results also suggest that time variations in the iEMG signal can be catered by concatenating the data over several days. This scheme can be helpful in attaining stable and robust performance.

Real-Time Hand Gesture Recognition Using Surface Electromyography and Machine Learning: A Systematic Literature Review

Today, daily life is composed of many computing systems, therefore interacting with them in a natural way makes the communication process more comfortable. Human-Computer Interaction (HCI) has been developed to overcome the communication barriers between humans and computers. One form of HCI is Hand Gesture Recognition (HGR), which predicts the class and the instant of execution of a given movement of the hand. One possible input for these models is surface electromyography (EMG), which records the electrical activity of skeletal muscles. EMG signals contain information about the intention of movement generated by the human brain. This systematic literature review analyses the state-of-the-art of real-time hand gesture recognition models using EMG data and machine learning. We selected and assessed 65 primary studies following the Kitchenham methodology. Based on a common structure of machine learning-based systems, we analyzed the structure of the proposed models and standardized concepts in regard to the types of models, data acquisition, segmentation, preprocessing, feature extraction, classification, postprocessing, real-time processing, types of gestures, and evaluation metrics. Finally, we also identified trends and gaps that could open new directions of work for future research in the area of gesture recognition using EMG.

Supervision of M.Sc. theses using the writing of a scientific article as a framework to increase efficiency and quality of research outcomes

The number of published peer-reviewed research articles has increased exponentially in the past decades, as has the degree of competitiveness in scientific publishing. Publication of scientific articles remains the gold standard for measuring research quality. In this context, quality is understood as to how rigorously the scientific method was applied. However, a critical disconnect exists between the continuous channel of projects fed to students by research laboratories, and the scientific quality of outcomes these students produce. Here, we present a process for the supervision of M.Sc. thesis projects conducted in research laboratories with the objective to increase productivity and quality. It is based on an iterative model of writing a scientific article naturally following the scientific method. This approach intends to maximize learning and development for the student, as well as productivity for the research laboratory by facilitating the publication of a peer-reviewed scientific article out of the thesis work.

Differential Activation of Biceps Brachii Muscle Compartments for Human-Machine Interfacing

A central challenge for myoelectric limb prostheses resides in the fact that, as the level of amputation becomes more proximal, the number of functions to be replaced increases, while the number of muscles available to collect input signals for control decreases. Differential activation of compartments from a single muscle could provide additional control sites. However, such feat is not naturally under voluntary control. In this study, we investigated the feasibility of learning to differentially activate the two heads of the bicep brachii muscle (BBM), by using biofeedback via high-density surface electromyography (HD-sEMG). Using a one degree of freedom Fitts' law test, we observed that eight subjects could learn to control the center of gravity of BBM's myoelectric activity. In addition, we examined the activations patterns of BBM that allow for the decoding of distal hand movements. These patterns were found highly individual, but different enough to allow for decoding of motor volition of distal joints. These findings represent promising venues to increase the functionality of myoelectrically controlled upper limb prostheses.

Effects of Confidence-Based Rejection on Usability and Error in Pattern Recognition-Based Myoelectric Control

Rejection of movements based on the confidence in the classification decision has previously been demonstrated to improve the usability of pattern recognition based myoelectric control. To this point, however, the optimal rejection threshold has been determined heuristically, and it is not known how different thresholds affect the tradeoff between error mitigation and false rejections in real-time closed-loop control. To answer this question, 24 able-bodied subjects completed a real-time Fitts' law-style virtual cursor control task using a support vector machine classifier. It was found that rejection improved information throughput at all thresholds, with the best performance coming at thresholds between 0.60 and 0.75. Two fundamental types of error were defined and identified: operator error (identifiable, repeatable behaviors, directly attributable to the user), and systemic error (other errors attributable to misclassification or noise). The incidence of both operator and systemic errors were found to decrease as rejection threshold increased. Moreover, while the incidence of all error types correlated strongly with path efficiency, only systemic errors correlated strongly with throughput and trial completion rate. Interestingly, more experienced users were found to commit as many errors as novice users, despite performing better in the Fitts' task, suggesting that there is more to usability than error prevention alone. Nevertheless, these results demonstrate the usability gains possible with rejection across a range of thresholds for both novice and experienced users alike.

On the robustness of real-time myoelectric control investigations: a multiday Fitts' law approach

OBJECTIVE

Real-time myoelectric experimental protocol is considered as a means to quantify usability of myoelectric control schemes. While usability should be considered over time to assure clinical robustness, all real-time studies reported thus far are limited to a single session or day and thus the influence of time on real-time performance is still unexplored. In this study, the aim was to develop a novel experimental protocol to quantify the effect of time on real-time performance measures over multiple days using a Fitts' law approach.

APPROACH

Four metrics: throughput, completion rate, path efficiency and overshoot, were assessed using three train-test strategies: (i) an artificial neural network (ANN) classifier was trained on data collected from the previous day and tested on present day (BDT) (ii) trained and tested on the same day (WDT) and (iii) trained on all previous days including present day and tested on present day (CDT) in a week-long experimental protocol.

MAIN RESULTS

It was found that on average, the completion rate (98.37%  ±  1.47%) of CDT was significantly better (P  <  0.01) than that of BDT (86.25%  ±  3.46%) and WDT (94.22%  ±  2.74%). The throughput (0.40  ±  0.03 bits s^-1) of CDT was significantly better (P  =  0.001) than that of BDT (0.38  ±  0.03 bits s^-1). Offline analysis showed a different trend due to the difference in the training strategies.

SIGNIFICANCE

Results suggest that increasing the size of the training set over time can be beneficial to assure robust performance of the system over time.