Offline and online myoelectric pattern recognition analysis and real-time control of a robotic hand after spinal cord injury

A Virtual Reality-Based Protocol to Determine the Preferred Control Strategy for Hand Neuroprostheses in People With Paralysis

Hand neuroprostheses restore voluntary movement in people with paralysis through neuromodulation protocols. There are a variety of strategies to control hand neuroprostheses, which can be based on residual body movements or brain activity. There is no universally superior solution, rather the best approach may vary from patient to patient. Here, we propose a protocol based on an immersive virtual reality (VR) environment that simulates the use of a hand neuroprosthesis to allow patients to experience and familiarize themselves with various control schemes in clinically relevant tasks and choose the preferred one. We used our VR environment to compare two alternative control strategies over 5 days of training in four patients with C6 spinal cord injury: (a) control via the ipsilateral wrist, (b) control via the contralateral shoulder. We did not find a one-fits-all solution but rather a subject-specific preference that could not be predicted based only on a general clinical assessment. The main results were that the VR simulation allowed participants to experience the pros and cons of the proposed strategies and make an educated choice, and that there was a longitudinal improvement. This shows that our VR-based protocol is a useful tool for personalization and training of the control strategy of hand neuroprostheses, which could help to promote user comfort and thus acceptance.

Neuromuscular implants: Interfacing with skeletal muscle for improved clinical translation of prosthetic limbs

After an amputation, advanced prosthetic limbs can be used to interface with the nervous system and restore motor function. Despite numerous breakthroughs in the field, many of the recent research advancements have not been widely integrated into clinical practice. This review highlights recent innovations in neuromuscular implants-specifically those that interface with skeletal muscle-which could improve the clinical translation of prosthetic technologies. Skeletal muscle provides a physiologic gateway to harness and amplify signals from the nervous system. Recent surgical advancements in muscle reinnervation surgeries leverage the "bio-amplification" capabilities of muscle, enabling more intuitive control over a greater number of degrees of freedom in prosthetic limbs than previously achieved. We anticipate that state-of-the-art implantable neuromuscular interfaces that integrate well with skeletal muscle and novel surgical interventions will provide a long-term solution for controlling advanced prostheses. Flexible electrodes are expected to play a crucial role in reducing foreign body responses and improving the longevity of the interface. Additionally, innovations in device miniaturization and ongoing exploration of shape memory polymers could simplify surgical procedures for implanting such interfaces. Once implanted, wireless strategies for powering and transferring data from the interface can eliminate bulky external wires, reduce infection risk, and enhance day-to-day usability. By outlining the current limitations of neuromuscular interfaces along with potential future directions, this review aims to guide continued research efforts and future collaborations between engineers and specialists in the field of neuromuscular and musculoskeletal medicine.

Feasibility of force myography for the direct control of an assistive robotic hand orthosis in non-impaired individuals

BACKGROUND

Assistive robotic hand orthoses can support people with sensorimotor hand impairment in many activities of daily living and therefore help to regain independence. However, in order for the users to fully benefit from the functionalities of such devices, a safe and reliable way to detect their movement intention for device control is crucial. Gesture recognition based on force myography measuring volumetric changes in the muscles during contraction has been previously shown to be a viable and easy to implement strategy to control hand prostheses. Whether this approach could be efficiently applied to intuitively control an assistive robotic hand orthosis remains to be investigated.

METHODS

In this work, we assessed the feasibility of using force myography measured from the forearm to control a robotic hand orthosis worn on the hand ipsilateral to the measurement site. In ten neurologically-intact participants wearing a robotic hand orthosis, we collected data for four gestures trained in nine arm configurations, i.e., seven static positions and two dynamic movements, corresponding to typical activities of daily living conditions. In an offline analysis, we determined classification accuracies for two binary classifiers (one for opening and one for closing) and further assessed the impact of individual training arm configurations on the overall performance.

RESULTS

We achieved an overall classification accuracy of 92.9% (averaged over two binary classifiers, individual accuracies 95.5% and 90.3%, respectively) but found a large variation in performance between participants, ranging from 75.4 up to 100%. Averaged inference times per sample were measured below 0.15 ms. Further, we found that the number of training arm configurations could be reduced from nine to six without notably decreasing classification performance.

CONCLUSION

The results of this work support the general feasibility of using force myography as an intuitive intention detection strategy for a robotic hand orthosis. Further, the findings also generated valuable insights into challenges and potential ways to overcome them in view of applying such technologies for assisting people with sensorimotor hand impairment during activities of daily living.

Robust neural decoding for dexterous control of robotic hand kinematics

BACKGROUND

Manual dexterity is a fundamental motor skill that allows us to perform complex daily tasks. Neuromuscular injuries, however, can lead to the loss of hand dexterity. Although numerous advanced assistive robotic hands have been developed, we still lack dexterous and continuous control of multiple degrees of freedom in real-time. In this study, we developed an efficient and robust neural decoding approach that can continuously decode intended finger dynamic movements for real-time control of a prosthetic hand.

METHODS

High-density electromyogram (HD-EMG) signals were obtained from the extrinsic finger flexor and extensor muscles, while participants performed either single-finger or multi-finger flexion-extension movements. We implemented a deep learning-based neural network approach to learn the mapping from HD-EMG features to finger-specific population motoneuron firing frequency (i.e., neural-drive signals). The neural-drive signals reflected motor commands specific to individual fingers. The predicted neural-drive signals were then used to continuously control the fingers (index, middle, and ring) of a prosthetic hand in real-time.

RESULTS

Our developed neural-drive decoder could consistently and accurately predict joint angles with significantly lower prediction errors across single-finger and multi-finger tasks, compared with a deep learning model directly trained on finger force signals and the conventional EMG-amplitude estimate. The decoder performance was stable over time and was robust to variations of the EMG signals. The decoder also demonstrated a substantially better finger separation with minimal predicted error of joint angle in the unintended fingers.

CONCLUSIONS

This neural decoding technique offers a novel and efficient neural-machine interface that can consistently predict robotic finger kinematics with high accuracy, which can enable dexterous control of assistive robotic hands.

A Novel Methodology for Classifying EMG Movements Based on SVM and Genetic Algorithms

Electromyography (EMG) processing is a fundamental part of medical research. It offers the possibility of developing new devices and techniques for the diagnosis, treatment, care, and rehabilitation of patients, in most cases non-invasively. However, EMG signals are random, non-stationary, and non-linear, making their classification difficult. Due to this, it is of vital importance to define which factors are helpful for the classification process. In order to improve this process, it is possible to apply algorithms capable of identifying which features are most important in the categorization process. Algorithms based on metaheuristic methods have demonstrated an ability to search for suitable subsets of features for optimization problems. Therefore, this work proposes a methodology based on genetic algorithms for feature selection to find the parameter space that offers the slightest classification error in 250 ms signal segments. For classification, a support vector machine is used. For this work, two databases were used, the first corresponding to the right upper extremity and the second formed by movements of the right lower extremity. For both databases, a feature space reduction of over 65% was obtained, with a higher average classification efficiency of 91% for the best subset of parameters. In addition, particle swarm optimization (PSO) was applied based on right upper extremity data, obtaining an 88% average error and a 46% reduction for the best subset of parameters. Finally, a sensitivity analysis was applied to the characteristics selected by PSO and genetic algorithms for the database of the right upper extremity, obtaining that the parameters determined by the genetic algorithms show greater sensitivity for the classification process.

Estimation of knee joint movement using single-channel sEMG signals with a feature-guided convolutional neural network

Estimating human motion intention, such as intent joint torque and movement, plays a crucial role in assistive robotics for ensuring efficient and safe human-robot interaction. For coupled human-robot systems, surface electromyography (sEMG) signal has been proven as an effective means for estimating human's intended movements. Usually, joint movement estimation uses sEMG signals measured from multiple muscles and needs many sEMG sensors placed on the human body, which may cause discomfort or result in mechanical/signal interference from wearable robots/environment during long-term routine use. Although the muscle synergy principle implies that it is possible to estimate human motion using sEMG signals from even one signal muscle, few studies investigated the feasibility of continuous motion estimation based on single-channel sEMG. In this study, a feature-guided convolutional neural network (FG-CNN) has been proposed to estimate human knee joint movement using single-channel sEMG. In the proposed FG-CNN, several handcrafted features have been fused into a CNN model to guide CNN feature extraction, and both handcrafted and CNN-extracted features were applied to a regression model, i.e., random forest regression, to estimate knee joint movements. Experiments with 8 healthy subjects were carried out, and sEMG signals measured from 6 muscles, i.e., vastus lateralis, vastus medialis, biceps femoris, semitendinosus, lateral or medial gastrocnemius (LG or MG), were separately evaluated for knee joint estimation using the proposed method. The experimental results demonstrated that the proposed FG-CNN method with single-channel sEMG signals from LG or MG can effectively estimate human knee joint movements. The average correlation coefficient between the measured and the estimated knee joint movements is 0.858 ± 0.085 for LG and 0.856 ± 0.057 for MG. Meanwhile, comparative studies showed that the combined handcrafted-CNN features outperform either the handcrafted features or the CNN features; the performance of the proposed signal-channel sEMG-based FG-CNN method is comparable to those of the traditional multi-channel sEMG-based methods. The outcomes of this study enable the possibility of developing a single-channel sEMG-based human-robot interface for knee joint movement estimation, which can facilitate the routine use of assistive robots.

Myoelectric Control Systems for Upper Limb Wearable Robotic Exoskeletons and Exosuits-A Systematic Review

In recent years, myoelectric control systems have emerged for upper limb wearable robotic exoskeletons to provide movement assistance and/or to restore motor functions in people with motor disabilities and to augment human performance in able-bodied individuals. In myoelectric control, electromyographic (EMG) signals from muscles are utilized to implement control strategies in exoskeletons and exosuits, improving adaptability and human-robot interactions during various motion tasks. This paper reviews the state-of-the-art myoelectric control systems designed for upper-limb wearable robotic exoskeletons and exosuits, and highlights the key focus areas for future research directions. Here, different modalities of existing myoelectric control systems were described in detail, and their advantages and disadvantages were summarized. Furthermore, key design aspects (i.e., supported degrees of freedom, portability, and intended application scenario) and the type of experiments conducted to validate the efficacy of the proposed myoelectric controllers were also discussed. Finally, the challenges and limitations of current myoelectric control systems were analyzed, and future research directions were suggested.

Single channel surface electromyogram deconvolution is a useful pre-processing for myoelectric control

Myoelectric control requires fast and stable identification of a movement from data recorded from a comfortable and straightforward system. Here we consider a new real-time pre-processing method applied to a single differential surface electromyogram (EMG): deconvolution, providing an estimation of the cumulative firings of motor units. A 2 channel-10 class finger movement problem has been investigated on 10 healthy subjects. We have compared raw EMG and deconvolution signals, as sources of information for two specific classifiers (based on either Support Vector Machines or k-Nearest Neighbours), with classical time-domain input features selected using Mutual Component Analysis. The overall results show that, using the proposed pre-processing technique, classification performances statistically improve. For example, the true positive rates of the best-tested configurations were 80.9% and 86.3% when using the EMG and its deconvoluted signal, respectively. Even considering the limited dataset and range of classification approaches investigated, these preliminary results indicate the potential usefulness of the deconvolution pre-processing, which could be easily embedded in different myoelectric control applications.

Sensing and decoding the neural drive to paralyzed muscles during attempted movements of a person with tetraplegia using a sleeve array

Motor neurons convey information about motor intent that can be extracted and interpreted to control assistive devices. However, most methods for measuring the firing activity of single neurons rely on implanted microelectrodes. Although intracortical brain-computer interfaces (BCIs) have been shown to be safe and effective, the requirement for surgery poses a barrier to widespread use that can be mitigated by instead using noninvasive interfaces. The objective of this study was to evaluate the feasibility of deriving motor control signals from a wearable sensor that can detect residual motor unit activity in paralyzed muscles after chronic cervical spinal cord injury (SCI). Despite generating no observable hand movement, volitional recruitment of motor units below the level of injury was observed across attempted movements of individual fingers and overt wrist and elbow movements. Subgroups of motor units were coactive during flexion or extension phases of the task. Single digit movement intentions were classified offline from the EMG power (RMS) or motor unit firing rates with median classification accuracies >75% in both cases. Simulated online control of a virtual hand was performed with a binary classifier to test feasibility of real-time extraction and decoding of motor units. The online decomposition algorithm extracted motor units in 1.2 ms, and the firing rates predicted the correct digit motion 88 ± 24% of the time. This study provides the first demonstration of a wearable interface for recording and decoding firing rates of motor units below the level of injury in a person with motor complete SCI.

MuscleNET: mapping electromyography to kinematic and dynamic biomechanical variables by machine learning

Objective.This paper proposes machine learning models for mapping surface electromyography (sEMG) signals to regression of joint angle, joint velocity, joint acceleration, joint torque, and activation torque.Approach.The regression models, collectively known as MuscleNET, take one of four forms: ANN (forward artificial neural network), RNN (recurrent neural network), CNN (convolutional neural network), and RCNN (recurrent convolutional neural network). Inspired by conventional biomechanical muscle models, delayed kinematic signals were used along with sEMG signals as the machine learning model's input; specifically, the CNN and RCNN were modeled with novel configurations for these input conditions. The models' inputs contain either raw or filtered sEMG signals, which allowed evaluation of the filtering capabilities of the models. The models were trained using human experimental data and evaluated with different individual data.Main results.Results were compared in terms of regression error (using the root-mean-square) and model computation delay. The results indicate that the RNN (with filtered sEMG signals) and RCNN (with raw sEMG signals) models, both with delayed kinematic data, can extract underlying motor control information (such as joint activation torque or joint angle) from sEMG signals in pick-and-place tasks. The CNNs and RCNNs were able to filter raw sEMG signals.Significance.All forms of MuscleNET were found to map sEMG signals within 2 ms, fast enough for real-time applications such as the control of exoskeletons or active prostheses. The RNN model with filtered sEMG and delayed kinematic signals is particularly appropriate for applications in musculoskeletal simulation and biomechatronic device control.

Properties of the surface electromyogram following traumatic spinal cord injury: a scoping review

Traumatic spinal cord injury (SCI) disrupts spinal and supraspinal pathways, and this process is reflected in changes in surface electromyography (sEMG). sEMG is an informative complement to current clinical testing and can capture the residual motor command in great detail-including in muscles below the level of injury with seemingly absent motor activities. In this comprehensive review, we sought to describe how the sEMG properties are changed after SCI. We conducted a systematic literature search followed by a narrative review focusing on sEMG analysis techniques and signal properties post-SCI. We found that early reports were mostly focused on the qualitative analysis of sEMG patterns and evolved to semi-quantitative scores and a more detailed amplitude-based quantification. Nonetheless, recent studies are still constrained to an amplitude-based analysis of the sEMG, and there are opportunities to more broadly characterize the time- and frequency-domain properties of the signal as well as to take fuller advantage of high-density EMG techniques. We recommend the incorporation of a broader range of signal properties into the neurophysiological assessment post-SCI and the development of a greater understanding of the relation between these sEMG properties and underlying physiology. Enhanced sEMG analysis could contribute to a more complete description of the effects of SCI on upper and lower motor neuron function and their interactions, and also assist in understanding the mechanisms of change following neuromodulation or exercise therapy.

Meeting Proceedings for SCI 2020: Launching a Decade of Disruption in Spinal Cord Injury Research

The spinal cord injury (SCI) research community has experienced great advances in discovery research, technology development, and promising clinical interventions in the past decade. To build upon these advances and maximize the benefit to persons with SCI, the National Institutes of Health (NIH) hosted a conference February 12-13, 2019 titled "SCI 2020: Launching a Decade of Disruption in Spinal Cord Injury Research." The purpose of the conference was to bring together a broad range of stakeholders, including researchers, clinicians and healthcare professionals, persons with SCI, industry partners, regulators, and funding agency representatives to break down existing communication silos. Invited speakers were asked to summarize the state of the science, assess areas of technological and community readiness, and build collaborations that could change the trajectory of research and clinical options for people with SCI. In this report, we summarize the state of the science in each of five key domains and identify the gaps in the scientific literature that need to be addressed to move the field forward.

Effective recognition of human lower limb jump locomotion phases based on multi-sensor information fusion and machine learning

Jump locomotion is the basic movement of human. However, no thorough research on the recognition of jump sub-phases has been carried so far. This paper aims to use multi-sensor information fusion and machine learning to recognize the human jump phase, which is crucial to the development of exoskeleton that assists jumping. The method of information fusion for sensors including sEMG, IMU, and footswitch sensor is studied. The footswitch signals are filtered by median filter. A processing method of synthesizing Euler angles into phase angle is proposed, which is beneficial to data integration. The jump locomotion is creatively segmented into five phases. The onset and offset of active segment are detected by sample entropy of sEMG and standard deviation of acceleration signal. The features are extracted from analysis windows using multi-sensor information fusion, and the dimension of feature matrix is selected. By comparing the performances of state-of-the-art machine learning classifiers, feature subsets of sEMG, IMU, and footswitch signals are selected from time domain features in a series of analysis window parameters. The average recognition accuracy of sEMG and IMU is 91.76% and 97.68%, respectively. When using the combination of sEMG, IMU, and footswitch signals, the average accuracy is 98.70%, which outperforms the combination of sEMG and IMU (97.97%, p < 0.01). Graphical Abstract The sub-phases of human locomotion are recognized based on multi-sensor information fusion and machine learning method. The feature data of the sub-phases is visualized in 3-dimensional space. The predicted states and the true states in a complete jump are compared along the time axis.

Human Gait Recognition Based on Multiple Feature Combination and Parameter Optimization Algorithms

Accuracy is a key index of human gait recognition. In this paper, we propose an improved gait recognition algorithm, which combines multiple feature combination and artificial bee colony for optimizing the support vector machine (ABC-SVM). Firstly, considering the complexity characteristics of surface electromyography (sEMG) signals, four types of features are extracted from the denoised sEMG signals, including the time-domain features of integral of absolute value (IAV), variance (VAR), and number of zero-crossing (ZC) points, frequency-domain features of mean power frequency (MPF) and median frequency (MF), and wavelet features and fuzzy entropy features. Secondly, the classifiers of SVM, linear discriminant analysis (LDA), and extreme learning machine (ELM) are employed to recognize the gait with obtained features, including singe-class features, multiple combination features, and optimized features of dimension reduction by principal component analysis (PCA). Thirdly, the penalty coefficient and kernel function parameter of the SVM classifier are optimized by the ABC algorithm, and the influence of different features and classifiers on the recognition results is studied. Finally, the feature samples selected to construct the SVM classifier are trained and recognized. Results show that the classification performance of the ABC-SVM classifier is significantly better than that of the nonoptimized SVM classifier, and the average recognition rate is increased by 3.18%. In addition, the combined feature samples (time-domain, frequency-domain, wavelet, and fuzzy entropy features) not only improve the gait classification accuracy but also enhance the recognition stability.

Improvement of hand functions of spinal cord injury patients with electromyography-driven hand exoskeleton: A feasibility study

We have developed a one-of-a-kind hand exoskeleton, called Maestro, which can power finger movements of those surviving severe disabilities to complete daily tasks using compliant joints. In this paper, we present results from an electromyography (EMG) control strategy conducted with spinal cord injury (SCI) patients (C5, C6, and C7) in which the subjects completed daily tasks controlling Maestro with EMG signals from their forearm muscles. With its compliant actuation and its degrees of freedom that match the natural finger movements, Maestro is capable of helping the subjects grasp and manipulate a variety of daily objects (more than 15 from a standardized set). To generate control commands for Maestro, an artificial neural network algorithm was implemented along with a probabilistic control approach to classify and deliver four hand poses robustly with three EMG signals measured from the forearm and palm. Increase in the scores of a standardized test, called the Sollerman hand function test, and enhancement in different aspects of grasping such as strength shows feasibility that Maestro can be capable of improving the hand function of SCI subjects.

Lw-CNN-Based Myoelectric Signal Recognition and Real-Time Control of Robotic Arm for Upper-Limb Rehabilitation

Deep-learning models can realize the feature extraction and advanced abstraction of raw myoelectric signals without necessitating manual selection. Raw surface myoelectric signals are processed with a deep model in this study to investigate the feasibility of recognizing upper-limb motion intents and real-time control of auxiliary equipment for upper-limb rehabilitation training. Surface myoelectric signals are collected on six motions of eight subjects' upper limbs. A light-weight convolutional neural network (Lw-CNN) and support vector machine (SVM) model are designed for myoelectric signal pattern recognition. The offline and online performance of the two models are then compared. The average accuracy is (90 ± 5)% for the Lw-CNN and (82.5 ± 3.5)% for the SVM in offline testing of all subjects, which prevails over (84 ± 6)% for the online Lw-CNN and (79 ± 4)% for SVM. The robotic arm control accuracy is (88.5 ± 5.5)%. Significance analysis shows no significant correlation (p = 0.056) among real-time control, offline testing, and online testing. The Lw-CNN model performs well in the recognition of upper-limb motion intents and can realize real-time control of a commercial robotic arm.

Effective Multi-Mode Grasping Assistance Control of a Soft Hand Exoskeleton Using Force Myography

Human intention detection is fundamental to the control of robotic devices in order to assist humans according to their needs. This paper presents a novel approach for detecting hand motion intention, i.e., rest, open, close, and grasp, and grasping force estimation using force myography (FMG). The output is further used to control a soft hand exoskeleton called an SEM Glove. In this method, two sensor bands constructed using force sensing resistor (FSR) sensors are utilized to detect hand motion states and muscle activities. Upon placing both bands on an arm, the sensors can measure normal forces caused by muscle contraction/relaxation. Afterwards, the sensor data is processed, and hand motions are identified through a threshold-based classification method. The developed method has been tested on human subjects for object-grasping tasks. The results show that the developed method can detect hand motions accurately and to provide assistance w.r.t to the task requirement.

High accurate lightweight deep learning method for gesture recognition based on surface electromyography

BACKGROUND AND OBJECTIVES

Surface Electromyography (sEMG) is used mostly for neuromuscular diagnosis, assistive technology, physical rehabilitation, and human-computer interactions. Achieving a precise and lightweight method along with low latency for gesture recognition is still a real-life challenge, especially for rehabilitation and assistive robots. This work aims to introduce a highly accurate and lightweight deep learning method for gesture recognition.

METHODS

High-density sEMG, unlike sparse sEMG, does not require accurate electrode placement and provides more physiological information. Then we apply high-density sEMG, which, according to previous studies, leads to sEMG images. In this study, we introduce the Sensor-Wise method, which has a higher capability to extract features compared to the sEMG image method due to its high compatibility with the nature of sEMG signals and the structure of convolutional networks.

RESULTS

The proposed method, because of its optimal structure with only two hidden layers and its high compatibility, has shown no sign of overfitting and was able to reach an accuracy of almost 100% (99.99%) when it was evaluated by CapgMyo DB-a database through 96 electrodes. Using this method, even with 16 electrodes, we were able to reach an accuracy of 99.8%, which was higher than the accuracies reported in the previous studies. Additionally, the method was evaluated by the CSL-HDEMG database, where the accuracy reached 99.55%. Previous studies either introduced expensive computational methods with overfitting or reported lower accuracies compared to this study.

CONCLUSIONS

The Sensor- Wise method has high compatibility with the nature of sEMG signals and the structure of convolutional networks. The high accuracy and lightweight structure of this method with only two hidden layers make it a proper option for hardware implementation.