Myoelectric digit action decoding with multi-output, multi-class classification: an offline analysis

Neuromorphic hardware for somatosensory neuroprostheses

In individuals with sensory-motor impairments, missing limb functions can be restored using neuroprosthetic devices that directly interface with the nervous system. However, restoring the natural tactile experience through electrical neural stimulation requires complex encoding strategies. Indeed, they are presently limited in effectively conveying or restoring tactile sensations by bandwidth constraints. Neuromorphic technology, which mimics the natural behavior of neurons and synapses, holds promise for replicating the encoding of natural touch, potentially informing neurostimulation design. In this perspective, we propose that incorporating neuromorphic technologies into neuroprostheses could be an effective approach for developing more natural human-machine interfaces, potentially leading to advancements in device performance, acceptability, and embeddability. We also highlight ongoing challenges and the required actions to facilitate the future integration of these advanced technologies.

Discrete action control for prosthetic digits

We aim to develop a paradigm for simultaneous and independent control of multiple degrees of freedom (DOFs) for upper-limb prostheses. To that end, we introduce action control, a novel method to operate prosthetic digits with surface electromyography (EMG) based on multi-output, multi-class classification. At each time step, the decoder classifies movement intent for each controllable DOF into one of three categories: open, close, or stall (i.e., no movement). We implemented a real-time myoelectric control system using this method and evaluated it by running experiments with one unilateral and two bilateral amputees. Participants controlled a six-DOF bar interface on a computer display, with each DOF corresponding to a motor function available in multi-articulated prostheses. We show that action control can significantly and systematically outperform the state-of-the-art method of position control via multi-output regression in both task- and non-task-related measures. Using the action control paradigm, improvements in median task performance over regression-based control ranged from 20.14% to 62.32% for individual participants. Analysis of a post-experimental survey revealed that all participants rated action higher than position control in a series of qualitative questions and expressed an overall preference for the former. Action control has the potential to improve the dexterity of upper-limb prostheses. In comparison with regression-based systems, it only requires discrete instead of real-valued ground truth labels, typically collected with motion tracking systems. This feature makes the system both practical in a clinical setting and also suitable for bilateral amputation. This work is the first demonstration of myoelectric digit control in bilateral upper-limb amputees. Further investigation and preclinical evaluation are required to assess the translational potential of the method.

Can Deep Synthesis of EMG Overcome the Geometric Growth of Training Data Required to Recognize Multiarticulate Motions?. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021;2021:6380-6383. [PMID: 34892572 DOI: 10.1109/embc46164.2021.9630276] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

By being predicated on supervised machine learning, pattern recognition approaches to myoelectric prosthesis control require electromyography (EMG) training data collected concurrently with every detectable motion. Within this framework, calibration protocols for simultaneous control of multifunctional prosthetic hands rapidly become prohibitively long-the number of unique motions grows geometrically with the number of controllable degrees of freedom (DoFs). This paper proposes a technique intended to circumvent this combinatorial explosion. Using EMG windows from 1-DoF motions as input and EMG windows from 2-DoF motions as targets, we train generative deep learning models to synthesize EMG windows appertaining to multi-DoF motions. Once trained, such models can be used to complete datasets consisting of only 1-DoF motions, enabling simple calibration protocols with durations that scale linearly with the number of DoFs. We evaluated synthetic EMG produced in this way via a classification task using a database of forearm surface EMG collected during 1-DoF and 2-DoF motions. Multi-output classifiers were trained on either (I) real data from 1-DoF and 2-DoF motions, (II) real data from only 1-DoF motions, or (III) real data from 1-DoF motions appended with synthetic EMG from 2-DoF motions. When tested on data containing all possible motions, classifiers trained on synthetic-appended data (III) significantly outperformed classifiers trained on 1-DoF real data (II), although significantly underperformed classifiers trained on both 1- and 2-DoF real data (I) (I < 0.05). These findings suggest that it is feasible to model EMG concurrent with multiarticulate motions as nonlinear combinations of EMG from constituent 1-DoF motions, and that such modelling can be harnessed to synthesize realistic training data.

Combined Dynamic Time Warping and Spatiotemporal Attention for Myoelectric Control

The success of pattern recognition based upper-limb prostheses control is linked to their ability to extract appropriate features from the electromyogram (EMG) signals. Traditional EMG feature extraction (FE) algorithms fail to extract spatial and inter-temporal information from the raw data, as they consider the EMG channels individually across a set of sliding windows with some degree of overlapping. To tackle these limitations, this paper presents a method that considers the spatial information of multi-channel EMG signals by utilising dynamic time warping (DTW). To satisfy temporal considerations, inspired by Long Short-Term Memory (LSTM) neural networks, our algorithm evolves the DTW feature representation across long and short-term components to capture the temporal dynamics of the EMG signal. As such the contribution of this paper is the development of a recursive spatio-temporal FE method, denoted as Recursive Temporal Warping (RTW). To investigate the performance of the proposed method, an offline EMG pattern recognition study with 53 movement classes performed by 10 subjects wearing 8 to 16 EMG channels was considered with the results compared against several conventional as well as deep learning-based models. We show that the use of the RTW can reduce classification errors significantly, paving the way for future real-time implementation.

Learning regularized representations of categorically labelled surface EMG enables simultaneous and proportional myoelectric control

Background

Processing the surface electromyogram (sEMG) to decode movement intent is a promising approach for natural control of upper extremity prostheses. To this end, this paper introduces and evaluates a new framework which allows for simultaneous and proportional myoelectric control over multiple degrees of freedom (DoFs) in real-time. The framework uses multitask neural networks and domain-informed regularization in order to automatically find nonlinear mappings from the forearm sEMG envelope to multivariate and continuous encodings of concurrent hand- and wrist kinematics, despite only requiring categorical movement instruction stimuli signals for calibration.

Methods

Forearm sEMG with 8 channels was collected from healthy human subjects (N = 20) and used to calibrate two myoelectric control interfaces, each with two output DoFs. The interfaces were built from (I) the proposed framework, termed Myoelectric Representation Learning (MRL), and, to allow for comparisons, from (II) a standard pattern recognition framework based on Linear Discriminant Analysis (LDA). The online performances of both interfaces were assessed with a Fitts’s law type test generating 5 quantitative performance metrics. The temporal stabilities of the interfaces were evaluated by conducting identical tests without recalibration 7 days after the initial experiment session.

Results

Metric-wise two-way repeated measures ANOVA with factors method (MRL vs LDA) and session (day 1 vs day 7) revealed a significant (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p<0.05$$\end{document}p<0.05) advantage for MRL over LDA in 5 out of 5 performance metrics, with metric-wise effect sizes (Cohen’s \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d$$\end{document}d) separating MRL from LDA ranging from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left|d\right|=0.62$$\end{document}d=0.62 to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left|d\right|=1.13$$\end{document}d=1.13. No significant effect on any metric was detected for neither session nor interaction between method and session, indicating that none of the methods deteriorated significantly in control efficacy during one week of intermission.

Conclusions

The results suggest that MRL is able to successfully generate stable mappings from EMG to kinematics, thereby enabling myoelectric control with real-time performance superior to that of the current commercial standard for pattern recognition (as represented by LDA). It is thus postulated that the presented MRL approach can be of practical utility for muscle-computer interfaces.

Arduino-Based Myoelectric Control: Towards Longitudinal Study of Prosthesis Use

Understanding how upper-limb prostheses are used in daily life helps to improve the design and robustness of prosthesis control algorithms and prosthetic components. However, only a very small fraction of published research includes prosthesis use in community settings. The cost, limited battery life, and poor generalisation may be the main reasons limiting the implementation of home-based applications. In this work, we introduce the design of a cost-effective Arduino-based myoelectric control system with wearable electromyogram (EMG) sensors. The design considerations focused on home studies, so the robustness, user-friendly control adjustments, and user supports were the main concerns. Three control algorithms, namely, direct control, abstract control, and linear discriminant analysis (LDA) classification, were implemented in the system. In this paper, we will share our design principles and report the robustness of the system in continuous operation in the laboratory. In addition, we will show a first real-time implementation of the abstract decoder for prosthesis control with an able-bodied participant.