1
|
Losanno E, Ceradini M, Agnesi F, Righi G, Del Popolo G, Shokur S, Micera S. A Virtual Reality-Based Protocol to Determine the Preferred Control Strategy for Hand Neuroprostheses in People With Paralysis. IEEE Trans Neural Syst Rehabil Eng 2024; 32:2261-2269. [PMID: 38865234 DOI: 10.1109/tnsre.2024.3413192] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/14/2024]
Abstract
Hand neuroprostheses restore voluntary movement in people with paralysis through neuromodulation protocols. There are a variety of strategies to control hand neuroprostheses, which can be based on residual body movements or brain activity. There is no universally superior solution, rather the best approach may vary from patient to patient. Here, we propose a protocol based on an immersive virtual reality (VR) environment that simulates the use of a hand neuroprosthesis to allow patients to experience and familiarize themselves with various control schemes in clinically relevant tasks and choose the preferred one. We used our VR environment to compare two alternative control strategies over 5 days of training in four patients with C6 spinal cord injury: (a) control via the ipsilateral wrist, (b) control via the contralateral shoulder. We did not find a one-fits-all solution but rather a subject-specific preference that could not be predicted based only on a general clinical assessment. The main results were that the VR simulation allowed participants to experience the pros and cons of the proposed strategies and make an educated choice, and that there was a longitudinal improvement. This shows that our VR-based protocol is a useful tool for personalization and training of the control strategy of hand neuroprostheses, which could help to promote user comfort and thus acceptance.
Collapse
|
2
|
Quinn KN, Tian Y, Budde R, Irazoqui PP, Tuffaha S, Thakor NV. Neuromuscular implants: Interfacing with skeletal muscle for improved clinical translation of prosthetic limbs. Muscle Nerve 2024; 69:134-147. [PMID: 38126120 DOI: 10.1002/mus.28029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 11/27/2023] [Accepted: 12/10/2023] [Indexed: 12/23/2023]
Abstract
After an amputation, advanced prosthetic limbs can be used to interface with the nervous system and restore motor function. Despite numerous breakthroughs in the field, many of the recent research advancements have not been widely integrated into clinical practice. This review highlights recent innovations in neuromuscular implants-specifically those that interface with skeletal muscle-which could improve the clinical translation of prosthetic technologies. Skeletal muscle provides a physiologic gateway to harness and amplify signals from the nervous system. Recent surgical advancements in muscle reinnervation surgeries leverage the "bio-amplification" capabilities of muscle, enabling more intuitive control over a greater number of degrees of freedom in prosthetic limbs than previously achieved. We anticipate that state-of-the-art implantable neuromuscular interfaces that integrate well with skeletal muscle and novel surgical interventions will provide a long-term solution for controlling advanced prostheses. Flexible electrodes are expected to play a crucial role in reducing foreign body responses and improving the longevity of the interface. Additionally, innovations in device miniaturization and ongoing exploration of shape memory polymers could simplify surgical procedures for implanting such interfaces. Once implanted, wireless strategies for powering and transferring data from the interface can eliminate bulky external wires, reduce infection risk, and enhance day-to-day usability. By outlining the current limitations of neuromuscular interfaces along with potential future directions, this review aims to guide continued research efforts and future collaborations between engineers and specialists in the field of neuromuscular and musculoskeletal medicine.
Collapse
Affiliation(s)
- Kiara N Quinn
- Department of Biomedical Engineering, Johns Hopkins School of Medicine, Baltimore, Maryland, USA
| | - Yucheng Tian
- Department of Biomedical Engineering, Johns Hopkins School of Medicine, Baltimore, Maryland, USA
| | - Ryan Budde
- Department of Biomedical Engineering, Johns Hopkins School of Medicine, Baltimore, Maryland, USA
| | - Pedro P Irazoqui
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, Maryland, USA
| | - Sami Tuffaha
- Department of Plastic and Reconstructive Surgery, Johns Hopkins School of Medicine, Baltimore, Maryland, USA
| | - Nitish V Thakor
- Department of Biomedical Engineering, Johns Hopkins School of Medicine, Baltimore, Maryland, USA
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, Maryland, USA
| |
Collapse
|
3
|
Gantenbein J, Ahmadizadeh C, Heeb O, Lambercy O, Menon C. Feasibility of force myography for the direct control of an assistive robotic hand orthosis in non-impaired individuals. J Neuroeng Rehabil 2023; 20:101. [PMID: 37537602 PMCID: PMC10399035 DOI: 10.1186/s12984-023-01222-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2023] [Accepted: 07/21/2023] [Indexed: 08/05/2023] Open
Abstract
BACKGROUND Assistive robotic hand orthoses can support people with sensorimotor hand impairment in many activities of daily living and therefore help to regain independence. However, in order for the users to fully benefit from the functionalities of such devices, a safe and reliable way to detect their movement intention for device control is crucial. Gesture recognition based on force myography measuring volumetric changes in the muscles during contraction has been previously shown to be a viable and easy to implement strategy to control hand prostheses. Whether this approach could be efficiently applied to intuitively control an assistive robotic hand orthosis remains to be investigated. METHODS In this work, we assessed the feasibility of using force myography measured from the forearm to control a robotic hand orthosis worn on the hand ipsilateral to the measurement site. In ten neurologically-intact participants wearing a robotic hand orthosis, we collected data for four gestures trained in nine arm configurations, i.e., seven static positions and two dynamic movements, corresponding to typical activities of daily living conditions. In an offline analysis, we determined classification accuracies for two binary classifiers (one for opening and one for closing) and further assessed the impact of individual training arm configurations on the overall performance. RESULTS We achieved an overall classification accuracy of 92.9% (averaged over two binary classifiers, individual accuracies 95.5% and 90.3%, respectively) but found a large variation in performance between participants, ranging from 75.4 up to 100%. Averaged inference times per sample were measured below 0.15 ms. Further, we found that the number of training arm configurations could be reduced from nine to six without notably decreasing classification performance. CONCLUSION The results of this work support the general feasibility of using force myography as an intuitive intention detection strategy for a robotic hand orthosis. Further, the findings also generated valuable insights into challenges and potential ways to overcome them in view of applying such technologies for assisting people with sensorimotor hand impairment during activities of daily living.
Collapse
Affiliation(s)
- Jessica Gantenbein
- Rehabilitation Engineering Laboratory, Department of Health Sciences and Technology, ETH Zurich, Lengghalde 5, 8008, Zurich, Switzerland
| | - Chakaveh Ahmadizadeh
- Biomedical and Mobile Health Technology Lab, Department of Health Sciences and Technology, ETH Zurich, Lengghalde 5, 8008, Zurich, Switzerland
| | - Oliver Heeb
- Rehabilitation Engineering Laboratory, Department of Health Sciences and Technology, ETH Zurich, Lengghalde 5, 8008, Zurich, Switzerland
- Biomedical and Mobile Health Technology Lab, Department of Health Sciences and Technology, ETH Zurich, Lengghalde 5, 8008, Zurich, Switzerland
| | - Olivier Lambercy
- Rehabilitation Engineering Laboratory, Department of Health Sciences and Technology, ETH Zurich, Lengghalde 5, 8008, Zurich, Switzerland
- Future Health Technologies, Singapore-ETH Centre, Campus for Research Excellence And Technological Enterprise (CREATE), 1 Create Way, Singapore, 138602, Singapore
| | - Carlo Menon
- Biomedical and Mobile Health Technology Lab, Department of Health Sciences and Technology, ETH Zurich, Lengghalde 5, 8008, Zurich, Switzerland.
| |
Collapse
|
4
|
Fan J, Vargas L, Kamper DG, Hu X. Robust neural decoding for dexterous control of robotic hand kinematics. Comput Biol Med 2023; 162:107139. [PMID: 37301095 DOI: 10.1016/j.compbiomed.2023.107139] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2022] [Revised: 05/22/2023] [Accepted: 06/04/2023] [Indexed: 06/12/2023]
Abstract
BACKGROUND Manual dexterity is a fundamental motor skill that allows us to perform complex daily tasks. Neuromuscular injuries, however, can lead to the loss of hand dexterity. Although numerous advanced assistive robotic hands have been developed, we still lack dexterous and continuous control of multiple degrees of freedom in real-time. In this study, we developed an efficient and robust neural decoding approach that can continuously decode intended finger dynamic movements for real-time control of a prosthetic hand. METHODS High-density electromyogram (HD-EMG) signals were obtained from the extrinsic finger flexor and extensor muscles, while participants performed either single-finger or multi-finger flexion-extension movements. We implemented a deep learning-based neural network approach to learn the mapping from HD-EMG features to finger-specific population motoneuron firing frequency (i.e., neural-drive signals). The neural-drive signals reflected motor commands specific to individual fingers. The predicted neural-drive signals were then used to continuously control the fingers (index, middle, and ring) of a prosthetic hand in real-time. RESULTS Our developed neural-drive decoder could consistently and accurately predict joint angles with significantly lower prediction errors across single-finger and multi-finger tasks, compared with a deep learning model directly trained on finger force signals and the conventional EMG-amplitude estimate. The decoder performance was stable over time and was robust to variations of the EMG signals. The decoder also demonstrated a substantially better finger separation with minimal predicted error of joint angle in the unintended fingers. CONCLUSIONS This neural decoding technique offers a novel and efficient neural-machine interface that can consistently predict robotic finger kinematics with high accuracy, which can enable dexterous control of assistive robotic hands.
Collapse
Affiliation(s)
- Jiahao Fan
- Department of Mechanical Engineering, Pennsylvania State University, University Park, USA
| | - Luis Vargas
- Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill and North Carolina State University, USA
| | - Derek G Kamper
- Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill and North Carolina State University, USA
| | - Xiaogang Hu
- Department of Mechanical Engineering, Pennsylvania State University, University Park, USA; Department of Kinesiology, Pennsylvania State University, University Park, USA; Department of Physical Medicine & Rehabilitation, Pennsylvania State Hershey College of Medicine, USA; Huck Institutes of the Life Sciences, Pennsylvania State University, University Park, USA; Center for Neural Engineering, Pennsylvania State University, University Park, USA.
| |
Collapse
|
5
|
Aviles M, Sánchez-Reyes LM, Fuentes-Aguilar RQ, Toledo-Pérez DC, Rodríguez-Reséndiz J. A Novel Methodology for Classifying EMG Movements Based on SVM and Genetic Algorithms. MICROMACHINES 2022; 13:mi13122108. [PMID: 36557408 PMCID: PMC9781991 DOI: 10.3390/mi13122108] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/22/2022] [Revised: 11/23/2022] [Accepted: 11/24/2022] [Indexed: 05/28/2023]
Abstract
Electromyography (EMG) processing is a fundamental part of medical research. It offers the possibility of developing new devices and techniques for the diagnosis, treatment, care, and rehabilitation of patients, in most cases non-invasively. However, EMG signals are random, non-stationary, and non-linear, making their classification difficult. Due to this, it is of vital importance to define which factors are helpful for the classification process. In order to improve this process, it is possible to apply algorithms capable of identifying which features are most important in the categorization process. Algorithms based on metaheuristic methods have demonstrated an ability to search for suitable subsets of features for optimization problems. Therefore, this work proposes a methodology based on genetic algorithms for feature selection to find the parameter space that offers the slightest classification error in 250 ms signal segments. For classification, a support vector machine is used. For this work, two databases were used, the first corresponding to the right upper extremity and the second formed by movements of the right lower extremity. For both databases, a feature space reduction of over 65% was obtained, with a higher average classification efficiency of 91% for the best subset of parameters. In addition, particle swarm optimization (PSO) was applied based on right upper extremity data, obtaining an 88% average error and a 46% reduction for the best subset of parameters. Finally, a sensitivity analysis was applied to the characteristics selected by PSO and genetic algorithms for the database of the right upper extremity, obtaining that the parameters determined by the genetic algorithms show greater sensitivity for the classification process.
Collapse
Affiliation(s)
- Marcos Aviles
- Faculty of Engineering, Universidad Autónoma de Querétaro, Querétaro 76010, Mexico
| | | | - Rita Q. Fuentes-Aguilar
- Tecnológico de Monterrey, Institute of Advanced Materials for Sustainable Manufacturing, Guadalajara 45201, Mexico
| | | | | |
Collapse
|
6
|
Zhang S, Lu J, Huo W, Yu N, Han J. Estimation of knee joint movement using single-channel sEMG signals with a feature-guided convolutional neural network. Front Neurorobot 2022; 16:978014. [PMID: 36386394 PMCID: PMC9640579 DOI: 10.3389/fnbot.2022.978014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2022] [Accepted: 09/28/2022] [Indexed: 11/24/2022] Open
Abstract
Estimating human motion intention, such as intent joint torque and movement, plays a crucial role in assistive robotics for ensuring efficient and safe human-robot interaction. For coupled human-robot systems, surface electromyography (sEMG) signal has been proven as an effective means for estimating human's intended movements. Usually, joint movement estimation uses sEMG signals measured from multiple muscles and needs many sEMG sensors placed on the human body, which may cause discomfort or result in mechanical/signal interference from wearable robots/environment during long-term routine use. Although the muscle synergy principle implies that it is possible to estimate human motion using sEMG signals from even one signal muscle, few studies investigated the feasibility of continuous motion estimation based on single-channel sEMG. In this study, a feature-guided convolutional neural network (FG-CNN) has been proposed to estimate human knee joint movement using single-channel sEMG. In the proposed FG-CNN, several handcrafted features have been fused into a CNN model to guide CNN feature extraction, and both handcrafted and CNN-extracted features were applied to a regression model, i.e., random forest regression, to estimate knee joint movements. Experiments with 8 healthy subjects were carried out, and sEMG signals measured from 6 muscles, i.e., vastus lateralis, vastus medialis, biceps femoris, semitendinosus, lateral or medial gastrocnemius (LG or MG), were separately evaluated for knee joint estimation using the proposed method. The experimental results demonstrated that the proposed FG-CNN method with single-channel sEMG signals from LG or MG can effectively estimate human knee joint movements. The average correlation coefficient between the measured and the estimated knee joint movements is 0.858 ± 0.085 for LG and 0.856 ± 0.057 for MG. Meanwhile, comparative studies showed that the combined handcrafted-CNN features outperform either the handcrafted features or the CNN features; the performance of the proposed signal-channel sEMG-based FG-CNN method is comparable to those of the traditional multi-channel sEMG-based methods. The outcomes of this study enable the possibility of developing a single-channel sEMG-based human-robot interface for knee joint movement estimation, which can facilitate the routine use of assistive robots.
Collapse
Affiliation(s)
- Song Zhang
- College of Artificial Intelligence, Nankai University, Tianjin, China
- Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin, China
| | - Jiewei Lu
- College of Artificial Intelligence, Nankai University, Tianjin, China
- Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin, China
| | - Weiguang Huo
- College of Artificial Intelligence, Nankai University, Tianjin, China
- Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin, China
- *Correspondence: Ningbo Yu
| | - Ningbo Yu
- College of Artificial Intelligence, Nankai University, Tianjin, China
- Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin, China
- Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen, China
- Weiguang Huo
| | - Jianda Han
- College of Artificial Intelligence, Nankai University, Tianjin, China
- Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin, China
- Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen, China
| |
Collapse
|
7
|
Fu J, Choudhury R, Hosseini SM, Simpson R, Park JH. Myoelectric Control Systems for Upper Limb Wearable Robotic Exoskeletons and Exosuits-A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:8134. [PMID: 36365832 PMCID: PMC9655258 DOI: 10.3390/s22218134] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Revised: 10/13/2022] [Accepted: 10/21/2022] [Indexed: 06/16/2023]
Abstract
In recent years, myoelectric control systems have emerged for upper limb wearable robotic exoskeletons to provide movement assistance and/or to restore motor functions in people with motor disabilities and to augment human performance in able-bodied individuals. In myoelectric control, electromyographic (EMG) signals from muscles are utilized to implement control strategies in exoskeletons and exosuits, improving adaptability and human-robot interactions during various motion tasks. This paper reviews the state-of-the-art myoelectric control systems designed for upper-limb wearable robotic exoskeletons and exosuits, and highlights the key focus areas for future research directions. Here, different modalities of existing myoelectric control systems were described in detail, and their advantages and disadvantages were summarized. Furthermore, key design aspects (i.e., supported degrees of freedom, portability, and intended application scenario) and the type of experiments conducted to validate the efficacy of the proposed myoelectric controllers were also discussed. Finally, the challenges and limitations of current myoelectric control systems were analyzed, and future research directions were suggested.
Collapse
Affiliation(s)
- Jirui Fu
- Department of Mechanical and Aerospace Engineering, University of Central Florida, Orlando, FL 32816, USA
| | - Renoa Choudhury
- Department of Mechanical and Aerospace Engineering, University of Central Florida, Orlando, FL 32816, USA
| | - Saba M. Hosseini
- Department of Electrical and Computer Engineering, University of Central Florida, Orlando, FL 32816, USA
| | - Rylan Simpson
- Department of Electrical and Computer Engineering, University of Central Florida, Orlando, FL 32816, USA
| | - Joon-Hyuk Park
- Department of Mechanical and Aerospace Engineering, University of Central Florida, Orlando, FL 32816, USA
| |
Collapse
|
8
|
Yang Y, Ren J, Duan F. The Spiking Rates Inspired Encoder and Decoder for Spiking Neural Networks: An Illustration of Hand Gesture Recognition. Cognit Comput 2022. [DOI: 10.1007/s12559-022-10027-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
9
|
Bourges M, Naik GR, Mesin L. Single channel surface electromyogram deconvolution is a useful pre-processing for myoelectric control. IEEE Trans Biomed Eng 2021; 69:1767-1775. [PMID: 34847017 DOI: 10.1109/tbme.2021.3131650] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Myoelectric control requires fast and stable identification of a movement from data recorded from a comfortable and straightforward system. Here we consider a new real-time pre-processing method applied to a single differential surface electromyogram (EMG): deconvolution, providing an estimation of the cumulative firings of motor units. A 2 channel-10 class finger movement problem has been investigated on 10 healthy subjects. We have compared raw EMG and deconvolution signals, as sources of information for two specific classifiers (based on either Support Vector Machines or k-Nearest Neighbours), with classical time-domain input features selected using Mutual Component Analysis. The overall results show that, using the proposed pre-processing technique, classification performances statistically improve. For example, the true positive rates of the best-tested configurations were 80.9% and 86.3% when using the EMG and its deconvoluted signal, respectively. Even considering the limited dataset and range of classification approaches investigated, these preliminary results indicate the potential usefulness of the deconvolution pre-processing, which could be easily embedded in different myoelectric control applications.
Collapse
|
10
|
Ting JE, Del Vecchio A, Sarma D, Verma N, Colachis SC, Annetta NV, Collinger JL, Farina D, Weber DJ. Sensing and decoding the neural drive to paralyzed muscles during attempted movements of a person with tetraplegia using a sleeve array. J Neurophysiol 2021; 126:2104-2118. [PMID: 34788156 DOI: 10.1152/jn.00220.2021] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Motor neurons convey information about motor intent that can be extracted and interpreted to control assistive devices. However, most methods for measuring the firing activity of single neurons rely on implanted microelectrodes. Although intracortical brain-computer interfaces (BCIs) have been shown to be safe and effective, the requirement for surgery poses a barrier to widespread use that can be mitigated by instead using noninvasive interfaces. The objective of this study was to evaluate the feasibility of deriving motor control signals from a wearable sensor that can detect residual motor unit activity in paralyzed muscles after chronic cervical spinal cord injury (SCI). Despite generating no observable hand movement, volitional recruitment of motor units below the level of injury was observed across attempted movements of individual fingers and overt wrist and elbow movements. Subgroups of motor units were coactive during flexion or extension phases of the task. Single digit movement intentions were classified offline from the EMG power (RMS) or motor unit firing rates with median classification accuracies >75% in both cases. Simulated online control of a virtual hand was performed with a binary classifier to test feasibility of real-time extraction and decoding of motor units. The online decomposition algorithm extracted motor units in 1.2 ms, and the firing rates predicted the correct digit motion 88 ± 24% of the time. This study provides the first demonstration of a wearable interface for recording and decoding firing rates of motor units below the level of injury in a person with motor complete SCI.
Collapse
Affiliation(s)
- Jordyn E Ting
- Rehab Neural Engineering Labs, University of Pittsburgh, Pittsburgh, PA, United States.,Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, United States.,Center for the Neural Basis of Cognition, Pittsburgh, PA, United States
| | - Alessandro Del Vecchio
- Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander University, Erlangen-Nürnberg, Erlangen, Germany
| | - Devapratim Sarma
- Rehab Neural Engineering Labs, University of Pittsburgh, Pittsburgh, PA, United States.,Center for the Neural Basis of Cognition, Pittsburgh, PA, United States.,Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander University, Erlangen-Nürnberg, Erlangen, Germany.,Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, United States
| | - Nikhil Verma
- Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, United States
| | - Samuel C Colachis
- Medical Devices and Neuromodulation Group, Battelle Memorial Institute, Columbus, OH, United States
| | - Nicholas V Annetta
- Medical Devices and Neuromodulation Group, Battelle Memorial Institute, Columbus, OH, United States
| | - Jennifer L Collinger
- Rehab Neural Engineering Labs, University of Pittsburgh, Pittsburgh, PA, United States.,Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, United States.,Center for the Neural Basis of Cognition, Pittsburgh, PA, United States.,Department of Physical Medicine and Rehabilitation, University of Pittsburgh, Pittsburgh, PA, United States.,Human Engineering Research Laboratories, VA Center of Excellence, Department of Veterans Affairs, Pittsburgh, PA, United States.,Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA, United States
| | - Dario Farina
- Department of Bioengineering, Imperial College London, London, United Kingdom
| | - Douglas J Weber
- Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, United States.,Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, United States
| |
Collapse
|
11
|
Nasr A, Bell S, He J, Whittaker RL, Jiang N, Dickerson CR, McPhee J. MuscleNET: mapping electromyography to kinematic and dynamic biomechanical variables by machine learning. J Neural Eng 2021; 18. [PMID: 34352741 DOI: 10.1088/1741-2552/ac1adc] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2021] [Accepted: 08/05/2021] [Indexed: 02/02/2023]
Abstract
Objective.This paper proposes machine learning models for mapping surface electromyography (sEMG) signals to regression of joint angle, joint velocity, joint acceleration, joint torque, and activation torque.Approach.The regression models, collectively known as MuscleNET, take one of four forms: ANN (forward artificial neural network), RNN (recurrent neural network), CNN (convolutional neural network), and RCNN (recurrent convolutional neural network). Inspired by conventional biomechanical muscle models, delayed kinematic signals were used along with sEMG signals as the machine learning model's input; specifically, the CNN and RCNN were modeled with novel configurations for these input conditions. The models' inputs contain either raw or filtered sEMG signals, which allowed evaluation of the filtering capabilities of the models. The models were trained using human experimental data and evaluated with different individual data.Main results.Results were compared in terms of regression error (using the root-mean-square) and model computation delay. The results indicate that the RNN (with filtered sEMG signals) and RCNN (with raw sEMG signals) models, both with delayed kinematic data, can extract underlying motor control information (such as joint activation torque or joint angle) from sEMG signals in pick-and-place tasks. The CNNs and RCNNs were able to filter raw sEMG signals.Significance.All forms of MuscleNET were found to map sEMG signals within 2 ms, fast enough for real-time applications such as the control of exoskeletons or active prostheses. The RNN model with filtered sEMG and delayed kinematic signals is particularly appropriate for applications in musculoskeletal simulation and biomechatronic device control.
Collapse
Affiliation(s)
- Ali Nasr
- University of Waterloo, Ontario N2L 1W2, Canada
| | - Sydney Bell
- University of Waterloo, Ontario N2L 1W2, Canada
| | - Jiayuan He
- University of Waterloo, Ontario N2L 1W2, Canada
| | | | - Ning Jiang
- University of Waterloo, Ontario N2L 1W2, Canada
| | | | - John McPhee
- University of Waterloo, Ontario N2L 1W2, Canada
| |
Collapse
|
12
|
Balbinot G, Li G, Wiest MJ, Pakosh M, Furlan JC, Kalsi-Ryan S, Zariffa J. Properties of the surface electromyogram following traumatic spinal cord injury: a scoping review. J Neuroeng Rehabil 2021; 18:105. [PMID: 34187509 PMCID: PMC8244234 DOI: 10.1186/s12984-021-00888-2] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2021] [Accepted: 05/27/2021] [Indexed: 12/23/2022] Open
Abstract
Traumatic spinal cord injury (SCI) disrupts spinal and supraspinal pathways, and this process is reflected in changes in surface electromyography (sEMG). sEMG is an informative complement to current clinical testing and can capture the residual motor command in great detail-including in muscles below the level of injury with seemingly absent motor activities. In this comprehensive review, we sought to describe how the sEMG properties are changed after SCI. We conducted a systematic literature search followed by a narrative review focusing on sEMG analysis techniques and signal properties post-SCI. We found that early reports were mostly focused on the qualitative analysis of sEMG patterns and evolved to semi-quantitative scores and a more detailed amplitude-based quantification. Nonetheless, recent studies are still constrained to an amplitude-based analysis of the sEMG, and there are opportunities to more broadly characterize the time- and frequency-domain properties of the signal as well as to take fuller advantage of high-density EMG techniques. We recommend the incorporation of a broader range of signal properties into the neurophysiological assessment post-SCI and the development of a greater understanding of the relation between these sEMG properties and underlying physiology. Enhanced sEMG analysis could contribute to a more complete description of the effects of SCI on upper and lower motor neuron function and their interactions, and also assist in understanding the mechanisms of change following neuromodulation or exercise therapy.
Collapse
Affiliation(s)
- Gustavo Balbinot
- KITE-Toronto Rehabilitation Institute, University Health Network, Toronto, ON, M5G 2A2, Canada.
| | - Guijin Li
- KITE-Toronto Rehabilitation Institute, University Health Network, Toronto, ON, M5G 2A2, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, Canada
| | - Matheus Joner Wiest
- KITE-Toronto Rehabilitation Institute, University Health Network, Toronto, ON, M5G 2A2, Canada
| | - Maureen Pakosh
- Library & Information Services, Toronto Rehabilitation Institute, University Health Network, Toronto, Canada
| | - Julio Cesar Furlan
- KITE-Toronto Rehabilitation Institute, University Health Network, Toronto, ON, M5G 2A2, Canada
- Rehabilitation Sciences Institute, University of Toronto, Toronto, Canada
- Department of Medicine, Division of Physical Medicine and Rehabilitation, University of Toronto, Toronto, Canada
- Division of Physical Medicine and Rehabilitation, Toronto Rehabilitation Institute, University Health Network, Toronto, Canada
- Institute of Medical Sciences, University of Toronto, Toronto, Canada
| | - Sukhvinder Kalsi-Ryan
- KITE-Toronto Rehabilitation Institute, University Health Network, Toronto, ON, M5G 2A2, Canada
- Rehabilitation Sciences Institute, University of Toronto, Toronto, Canada
- Department of Physical Therapy, University of Toronto, Toronto, Canada
| | - Jose Zariffa
- KITE-Toronto Rehabilitation Institute, University Health Network, Toronto, ON, M5G 2A2, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, Canada
- Rehabilitation Sciences Institute, University of Toronto, Toronto, Canada
- Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada
| |
Collapse
|
13
|
Morse LR, Field-Fote EC, Contreras-Vidal J, Noble-Haeusslein LJ, Rodreick M, Shields RK, Sofroniew M, Wudlick R, Zanca JM. Meeting Proceedings for SCI 2020: Launching a Decade of Disruption in Spinal Cord Injury Research. J Neurotrauma 2021; 38:1251-1266. [PMID: 33353467 DOI: 10.1089/neu.2020.7174] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/30/2023] Open
Abstract
The spinal cord injury (SCI) research community has experienced great advances in discovery research, technology development, and promising clinical interventions in the past decade. To build upon these advances and maximize the benefit to persons with SCI, the National Institutes of Health (NIH) hosted a conference February 12-13, 2019 titled "SCI 2020: Launching a Decade of Disruption in Spinal Cord Injury Research." The purpose of the conference was to bring together a broad range of stakeholders, including researchers, clinicians and healthcare professionals, persons with SCI, industry partners, regulators, and funding agency representatives to break down existing communication silos. Invited speakers were asked to summarize the state of the science, assess areas of technological and community readiness, and build collaborations that could change the trajectory of research and clinical options for people with SCI. In this report, we summarize the state of the science in each of five key domains and identify the gaps in the scientific literature that need to be addressed to move the field forward.
Collapse
Affiliation(s)
- Leslie R Morse
- Department of Rehabilitation Medicine, University of Minnesota School of Medicine, Minneapolis, Minnesota, USA
| | - Edelle C Field-Fote
- Shepherd Center, Atlanta, Georgia, USA.,Division of Physical Therapy, Department of Rehabilitation Medicine, Emory University School of Medicine, Atlanta, Georgia, USA
| | - Jose Contreras-Vidal
- Laboratory for Non-Invasive Brain Machine Interfaces, NSF IUCRC BRAIN, Cullen College of Engineering, University of Houston, Houston, Texas, USA
| | - Linda J Noble-Haeusslein
- Departments of Neurology and Psychology and the Institute of Neuroscience, University of Texas at Austin, Austin, Texas, USA
| | | | - Richard K Shields
- Department of Physical Therapy and Rehabilitation Science, Roy J. and Lucille A. Carver College of Medicine, University of Iowa, Iowa City, Iowa, USA
| | - Michael Sofroniew
- Department of Neurobiology, University of California, Los Angeles, California, USA
| | - Robert Wudlick
- Department of Rehabilitation Medicine, University of Minnesota School of Medicine, Minneapolis, Minnesota, USA
| | - Jeanne M Zanca
- Spinal Cord Injury Research, Kessler Foundation, West Orange, New Jersey, USA.,Physical Medicine and Rehabilitation, Rutgers New Jersey Medical School, Newark, New Jersey, USA
| | | |
Collapse
|
14
|
Lu Y, Wang H, Hu F, Zhou B, Xi H. Effective recognition of human lower limb jump locomotion phases based on multi-sensor information fusion and machine learning. Med Biol Eng Comput 2021; 59:883-899. [PMID: 33745104 DOI: 10.1007/s11517-021-02335-9] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2020] [Accepted: 02/04/2021] [Indexed: 11/28/2022]
Abstract
Jump locomotion is the basic movement of human. However, no thorough research on the recognition of jump sub-phases has been carried so far. This paper aims to use multi-sensor information fusion and machine learning to recognize the human jump phase, which is crucial to the development of exoskeleton that assists jumping. The method of information fusion for sensors including sEMG, IMU, and footswitch sensor is studied. The footswitch signals are filtered by median filter. A processing method of synthesizing Euler angles into phase angle is proposed, which is beneficial to data integration. The jump locomotion is creatively segmented into five phases. The onset and offset of active segment are detected by sample entropy of sEMG and standard deviation of acceleration signal. The features are extracted from analysis windows using multi-sensor information fusion, and the dimension of feature matrix is selected. By comparing the performances of state-of-the-art machine learning classifiers, feature subsets of sEMG, IMU, and footswitch signals are selected from time domain features in a series of analysis window parameters. The average recognition accuracy of sEMG and IMU is 91.76% and 97.68%, respectively. When using the combination of sEMG, IMU, and footswitch signals, the average accuracy is 98.70%, which outperforms the combination of sEMG and IMU (97.97%, p < 0.01). Graphical Abstract The sub-phases of human locomotion are recognized based on multi-sensor information fusion and machine learning method. The feature data of the sub-phases is visualized in 3-dimensional space. The predicted states and the true states in a complete jump are compared along the time axis.
Collapse
Affiliation(s)
- Yanzheng Lu
- School of Mechanical Engineering and Automation, Northeastern University, Shenyang, China
| | - Hong Wang
- School of Mechanical Engineering and Automation, Northeastern University, Shenyang, China.
| | - Fo Hu
- School of Mechanical Engineering and Automation, Northeastern University, Shenyang, China
| | - Bin Zhou
- School of Mechanical Engineering and Automation, Northeastern University, Shenyang, China
| | - Hailong Xi
- School of Mechanical Engineering and Automation, Northeastern University, Shenyang, China
| |
Collapse
|
15
|
Chen Y, Ma K, Yang L, Yu S, Cai S, Xie L. Trunk compensation electromyography features purification and classification model using generative adversarial network. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2020.102345] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
16
|
Human Gait Recognition Based on Multiple Feature Combination and Parameter Optimization Algorithms. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2021; 2021:6693206. [PMID: 33727913 PMCID: PMC7937488 DOI: 10.1155/2021/6693206] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/21/2020] [Revised: 02/06/2021] [Accepted: 02/18/2021] [Indexed: 11/17/2022]
Abstract
Accuracy is a key index of human gait recognition. In this paper, we propose an improved gait recognition algorithm, which combines multiple feature combination and artificial bee colony for optimizing the support vector machine (ABC-SVM). Firstly, considering the complexity characteristics of surface electromyography (sEMG) signals, four types of features are extracted from the denoised sEMG signals, including the time-domain features of integral of absolute value (IAV), variance (VAR), and number of zero-crossing (ZC) points, frequency-domain features of mean power frequency (MPF) and median frequency (MF), and wavelet features and fuzzy entropy features. Secondly, the classifiers of SVM, linear discriminant analysis (LDA), and extreme learning machine (ELM) are employed to recognize the gait with obtained features, including singe-class features, multiple combination features, and optimized features of dimension reduction by principal component analysis (PCA). Thirdly, the penalty coefficient and kernel function parameter of the SVM classifier are optimized by the ABC algorithm, and the influence of different features and classifiers on the recognition results is studied. Finally, the feature samples selected to construct the SVM classifier are trained and recognized. Results show that the classification performance of the ABC-SVM classifier is significantly better than that of the nonoptimized SVM classifier, and the average recognition rate is increased by 3.18%. In addition, the combined feature samples (time-domain, frequency-domain, wavelet, and fuzzy entropy features) not only improve the gait classification accuracy but also enhance the recognition stability.
Collapse
|
17
|
Yun Y, Na Y, Esmatloo P, Dancausse S, Serrato A, Merring CA, Agarwal P, Deshpande AD. Improvement of hand functions of spinal cord injury patients with electromyography-driven hand exoskeleton: A feasibility study. WEARABLE TECHNOLOGIES 2021; 1:e8. [PMID: 39050268 PMCID: PMC11265402 DOI: 10.1017/wtc.2020.9] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/13/2019] [Revised: 07/31/2020] [Accepted: 10/10/2020] [Indexed: 07/27/2024]
Abstract
We have developed a one-of-a-kind hand exoskeleton, called Maestro, which can power finger movements of those surviving severe disabilities to complete daily tasks using compliant joints. In this paper, we present results from an electromyography (EMG) control strategy conducted with spinal cord injury (SCI) patients (C5, C6, and C7) in which the subjects completed daily tasks controlling Maestro with EMG signals from their forearm muscles. With its compliant actuation and its degrees of freedom that match the natural finger movements, Maestro is capable of helping the subjects grasp and manipulate a variety of daily objects (more than 15 from a standardized set). To generate control commands for Maestro, an artificial neural network algorithm was implemented along with a probabilistic control approach to classify and deliver four hand poses robustly with three EMG signals measured from the forearm and palm. Increase in the scores of a standardized test, called the Sollerman hand function test, and enhancement in different aspects of grasping such as strength shows feasibility that Maestro can be capable of improving the hand function of SCI subjects.
Collapse
Affiliation(s)
- Youngmok Yun
- Department of Mechanical Engineering, The University of Texas at Austin, Austin, Texas, USA
| | - Youngjin Na
- Department of Mechanical Systems Engineering, Sookmyung Women’s University, Seoul, Republic of Korea
| | - Paria Esmatloo
- Department of Mechanical Engineering, The University of Texas at Austin, Austin, Texas, USA
| | - Sarah Dancausse
- Department of Mechanical Engineering, The University of Texas at Austin, Austin, Texas, USA
| | - Alfredo Serrato
- Department of Mechanical Engineering, The University of Texas at Austin, Austin, Texas, USA
| | - Curtis A. Merring
- Department of Mechanical Engineering, The University of Texas at Austin, Austin, Texas, USA
| | - Priyanshu Agarwal
- Department of Mechanical Engineering, The University of Texas at Austin, Austin, Texas, USA
| | - Ashish D. Deshpande
- Department of Mechanical Engineering, The University of Texas at Austin, Austin, Texas, USA
| |
Collapse
|
18
|
Lw-CNN-Based Myoelectric Signal Recognition and Real-Time Control of Robotic Arm for Upper-Limb Rehabilitation. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2020; 2020:8846021. [PMID: 33456452 PMCID: PMC7785339 DOI: 10.1155/2020/8846021] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/30/2020] [Revised: 11/22/2020] [Accepted: 12/15/2020] [Indexed: 11/18/2022]
Abstract
Deep-learning models can realize the feature extraction and advanced abstraction of raw myoelectric signals without necessitating manual selection. Raw surface myoelectric signals are processed with a deep model in this study to investigate the feasibility of recognizing upper-limb motion intents and real-time control of auxiliary equipment for upper-limb rehabilitation training. Surface myoelectric signals are collected on six motions of eight subjects' upper limbs. A light-weight convolutional neural network (Lw-CNN) and support vector machine (SVM) model are designed for myoelectric signal pattern recognition. The offline and online performance of the two models are then compared. The average accuracy is (90 ± 5)% for the Lw-CNN and (82.5 ± 3.5)% for the SVM in offline testing of all subjects, which prevails over (84 ± 6)% for the online Lw-CNN and (79 ± 4)% for SVM. The robotic arm control accuracy is (88.5 ± 5.5)%. Significance analysis shows no significant correlation (p = 0.056) among real-time control, offline testing, and online testing. The Lw-CNN model performs well in the recognition of upper-limb motion intents and can realize real-time control of a commercial robotic arm.
Collapse
|
19
|
Islam MRU, Bai S. Effective Multi-Mode Grasping Assistance Control of a Soft Hand Exoskeleton Using Force Myography. Front Robot AI 2020; 7:567491. [PMID: 33501329 PMCID: PMC7805723 DOI: 10.3389/frobt.2020.567491] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2020] [Accepted: 09/04/2020] [Indexed: 11/17/2022] Open
Abstract
Human intention detection is fundamental to the control of robotic devices in order to assist humans according to their needs. This paper presents a novel approach for detecting hand motion intention, i.e., rest, open, close, and grasp, and grasping force estimation using force myography (FMG). The output is further used to control a soft hand exoskeleton called an SEM Glove. In this method, two sensor bands constructed using force sensing resistor (FSR) sensors are utilized to detect hand motion states and muscle activities. Upon placing both bands on an arm, the sensors can measure normal forces caused by muscle contraction/relaxation. Afterwards, the sensor data is processed, and hand motions are identified through a threshold-based classification method. The developed method has been tested on human subjects for object-grasping tasks. The results show that the developed method can detect hand motions accurately and to provide assistance w.r.t to the task requirement.
Collapse
|
20
|
Bahador A, Yousefi M, Marashi M, Bahador O. High accurate lightweight deep learning method for gesture recognition based on surface electromyography. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2020; 195:105643. [PMID: 32650088 DOI: 10.1016/j.cmpb.2020.105643] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/24/2020] [Accepted: 06/29/2020] [Indexed: 06/11/2023]
Abstract
BACKGROUND AND OBJECTIVES Surface Electromyography (sEMG) is used mostly for neuromuscular diagnosis, assistive technology, physical rehabilitation, and human-computer interactions. Achieving a precise and lightweight method along with low latency for gesture recognition is still a real-life challenge, especially for rehabilitation and assistive robots. This work aims to introduce a highly accurate and lightweight deep learning method for gesture recognition. METHODS High-density sEMG, unlike sparse sEMG, does not require accurate electrode placement and provides more physiological information. Then we apply high-density sEMG, which, according to previous studies, leads to sEMG images. In this study, we introduce the Sensor-Wise method, which has a higher capability to extract features compared to the sEMG image method due to its high compatibility with the nature of sEMG signals and the structure of convolutional networks. RESULTS The proposed method, because of its optimal structure with only two hidden layers and its high compatibility, has shown no sign of overfitting and was able to reach an accuracy of almost 100% (99.99%) when it was evaluated by CapgMyo DB-a database through 96 electrodes. Using this method, even with 16 electrodes, we were able to reach an accuracy of 99.8%, which was higher than the accuracies reported in the previous studies. Additionally, the method was evaluated by the CSL-HDEMG database, where the accuracy reached 99.55%. Previous studies either introduced expensive computational methods with overfitting or reported lower accuracies compared to this study. CONCLUSIONS The Sensor- Wise method has high compatibility with the nature of sEMG signals and the structure of convolutional networks. The high accuracy and lightweight structure of this method with only two hidden layers make it a proper option for hardware implementation.
Collapse
Affiliation(s)
- Ali Bahador
- Faculty of Electrical and Computer Engineering, K.N. Toosi University of Technology, Tehran 1631714191, Iran
| | - Moslem Yousefi
- Department of Mechanical Engineering, Islamic Azad University, Roudehen Branch, Roudehen 3973188981, Iran.
| | - Mehdi Marashi
- Department of Mechanical Engineering, Islamic Azad University, Roudehen Branch, Roudehen 3973188981, Iran
| | - Omid Bahador
- Faculty of Mechanical Engineering, Islamic Azad University, South Tehran Branch, Tehran 1584743311, Iran
| |
Collapse
|