1
|
Shin J, Miah ASM, Konnai S, Takahashi I, Hirooka K. Hand gesture recognition using sEMG signals with a multi-stream time-varying feature enhancement approach. Sci Rep 2024; 14:22061. [PMID: 39333258 PMCID: PMC11436881 DOI: 10.1038/s41598-024-72996-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2024] [Accepted: 09/12/2024] [Indexed: 09/29/2024] Open
Abstract
Hand gesture recognition based on sparse multichannel surface electromyography (sEMG) still poses a significant challenge to deployment as a muscle-computer interface. Many researchers have been working to develop an sEMG-based hand gesture recognition system. However, the existing system still faces challenges in achieving satisfactory performance due to ineffective feature enhancement, so the prediction is erratic and unstable. To comprehensively tackle these challenges, we introduce a novel approach: a lightweight sEMG-based hand gesture recognition system using a 4-stream deep learning architecture. Each stream strategically combines Temporal Convolutional Network (TCN)-based time-varying features with Convolutional Neural Network (CNN)-based frame-wise features. In the first stream, we harness the power of the TCN module to extract nuanced time-varying temporal features. The second stream integrates a hybrid Long short-term memory (LSTM)-TCN module. This stream extracts temporal features using LSTM and seamlessly enhances them with TCN to effectively capture intricate long-range temporal relations. The third stream adopts a spatio-temporal strategy, merging the CNN and TCN modules. This integration facilitates concurrent comprehension of both spatial and temporal features, enriching the model's understanding of the underlying dynamics of the data. The fourth stream uses a skip connection mechanism to alleviate potential problems of data loss, ensuring a robust information flow throughout the network and concatenating the 4 stream features, yielding a comprehensive and effective final feature representation. We employ a channel attention-based feature selection module to select the most effective features, aiming to reduce the computational complexity and feed them into the classification module. The proposed model achieves an average accuracy of 94.31% and 98.96% on the Ninapro DB1 and DB9 datasets, respectively. This high-performance accuracy proves the superiority of the proposed model, and its implications extend to enhancing the quality of life for individuals using prosthetic limbs and advancing control systems in the field of robotic human-machine interfaces.
Collapse
Affiliation(s)
- Jungpil Shin
- School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, Fukushima, 965-0006, Japan.
| | - Abu Saleh Musa Miah
- School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, Fukushima, 965-0006, Japan
| | - Sota Konnai
- School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, Fukushima, 965-0006, Japan
| | - Itsuki Takahashi
- School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, Fukushima, 965-0006, Japan
| | - Koki Hirooka
- School of Computer Science and Engineering, The University of Aizu, Aizuwakamatsu, Fukushima, 965-0006, Japan
| |
Collapse
|
2
|
Eddy E, Campbell E, Bateman S, Scheme E. Big data in myoelectric control: large multi-user models enable robust zero-shot EMG-based discrete gesture recognition. Front Bioeng Biotechnol 2024; 12:1463377. [PMID: 39380895 PMCID: PMC11459555 DOI: 10.3389/fbioe.2024.1463377] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2024] [Accepted: 08/28/2024] [Indexed: 10/10/2024] Open
Abstract
Myoelectric control, the use of electromyogram (EMG) signals generated during muscle contractions to control a system or device, is a promising input, enabling always-available control for emerging ubiquitous computing applications. However, its widespread use has historically been limited by the need for user-specific machine learning models because of behavioural and physiological differences between users. Leveraging the publicly available 612-user EMG-EPN612 dataset, this work dispels this notion, showing that true zero-shot cross-user myoelectric control is achievable without user-specific training. By taking a discrete approach to classification (i.e., recognizing the entire dynamic gesture as a single event), a classification accuracy of 93.0% for six gestures was achieved on a set of 306 unseen users, showing that big data approaches can enable robust cross-user myoelectric control. By organizing the results into a series of mini-studies, this work provides an in-depth analysis of discrete cross-user models to answer unknown questions and uncover new research directions. In particular, this work explores the number of participants required to build cross-user models, the impact of transfer learning for fine-tuning these models, and the effects of under-represented end-user demographics in the training data, among other issues. Additionally, in order to further evaluate the performance of the developed cross-user models, a completely new dataset was created (using the same recording device) that includes known covariate factors such as cross-day use and limb-position variability. The results show that the large data models can effectively generalize to new datasets and mitigate the impact of common confounding factors that have historically limited the adoption of EMG-based inputs.
Collapse
Affiliation(s)
- Ethan Eddy
- Institute of Biomedical Engineering, University of New Brunswick, Fredericton, NB, Canada
| | - Evan Campbell
- Institute of Biomedical Engineering, University of New Brunswick, Fredericton, NB, Canada
| | - Scott Bateman
- Faculty of Computer Science, University of New Brunswick, Fredericton, NB, Canada
| | - Erik Scheme
- Institute of Biomedical Engineering, University of New Brunswick, Fredericton, NB, Canada
| |
Collapse
|
3
|
Lee H, Jiang M, Yang J, Yang Z, Zhao Q. Unveiling EMG semantics: a prototype-learning approach to generalizable gesture classification. J Neural Eng 2024; 21:036031. [PMID: 38754410 DOI: 10.1088/1741-2552/ad4c98] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Accepted: 05/16/2024] [Indexed: 05/18/2024]
Abstract
Objective.Upper limb loss can profoundly impact an individual's quality of life, posing challenges to both physical capabilities and emotional well-being. To restore limb function by decoding electromyography (EMG) signals, in this paper, we present a novel deep prototype learning method for accurate and generalizable EMG-based gesture classification. Existing methods suffer from limitations in generalization across subjects due to the diverse nature of individual muscle responses, impeding seamless applicability in broader populations.Approach.By leveraging deep prototype learning, we introduce a method that goes beyond direct output prediction. Instead, it matches new EMG inputs to a set of learned prototypes and predicts the corresponding labels.Main results.This novel methodology significantly enhances the model's classification performance and generalizability by discriminating subtle differences between gestures, making it more reliable and precise in real-world applications. Our experiments on four Ninapro datasets suggest that our deep prototype learning classifier outperforms state-of-the-art methods in terms of intra-subject and inter-subject classification accuracy in gesture prediction.Significance.The results from our experiments validate the effectiveness of the proposed method and pave the way for future advancements in the field of EMG gesture classification for upper limb prosthetics.
Collapse
Affiliation(s)
- Hunmin Lee
- Department of Computer Science and Engineering, University of Minnesota, Twin Cities, MN, United States of America
| | - Ming Jiang
- Department of Computer Science and Engineering, University of Minnesota, Twin Cities, MN, United States of America
| | - Jinhui Yang
- Department of Computer Science and Engineering, University of Minnesota, Twin Cities, MN, United States of America
| | - Zhi Yang
- Department of Biomedical and Engineering, University of Minnesota, Twin Cities, MN, United States of America
| | - Qi Zhao
- Department of Computer Science and Engineering, University of Minnesota, Twin Cities, MN, United States of America
| |
Collapse
|
4
|
Zhang K, Badesa FJ, Liu Y, Ferre Pérez M. Dual Stream Long Short-Term Memory Feature Fusion Classifier for Surface Electromyography Gesture Recognition. SENSORS (BASEL, SWITZERLAND) 2024; 24:3631. [PMID: 38894423 PMCID: PMC11175185 DOI: 10.3390/s24113631] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/26/2024] [Revised: 05/28/2024] [Accepted: 05/31/2024] [Indexed: 06/21/2024]
Abstract
Gesture recognition using electromyography (EMG) signals has prevailed recently in the field of human-computer interactions for controlling intelligent prosthetics. Currently, machine learning and deep learning are the two most commonly employed methods for classifying hand gestures. Despite traditional machine learning methods already achieving impressive performance, it is still a huge amount of work to carry out feature extraction manually. The existing deep learning methods utilize complex neural network architectures to achieve higher accuracy, which will suffer from overfitting, insufficient adaptability, and low recognition accuracy. To improve the existing phenomenon, a novel lightweight model named dual stream LSTM feature fusion classifier is proposed based on the concatenation of five time-domain features of EMG signals and raw data, which are both processed with one-dimensional convolutional neural networks and LSTM layers to carry out the classification. The proposed method can effectively capture global features of EMG signals using a simple architecture, which means less computational cost. An experiment is conducted on a public DB1 dataset with 52 gestures, and each of the 27 subjects repeats every gesture 10 times. The accuracy rate achieved by the model is 89.66%, which is comparable to that achieved by more complex deep learning neural networks, and the inference time for each gesture is 87.6 ms, which can also be implied in a real-time control system. The proposed model is validated using a subject-wise experiment on 10 out of the 40 subjects in the DB2 dataset, achieving a mean accuracy of 91.74%. This is illustrated by its ability to fuse time-domain features and raw data to extract more effective information from the sEMG signal and select an appropriate, efficient, lightweight network to enhance the recognition results.
Collapse
Affiliation(s)
- Kexin Zhang
- Centre for Automation and Robotics (CAR) UPM-CSIC, Universidad Politécnica de Madrid (UPM), 28006 Madrid, Spain; (K.Z.); (F.J.B.)
| | - Francisco J. Badesa
- Centre for Automation and Robotics (CAR) UPM-CSIC, Universidad Politécnica de Madrid (UPM), 28006 Madrid, Spain; (K.Z.); (F.J.B.)
| | - Yinlong Liu
- State Key Laboratory of Internet of Things for Smart City, University of Macao, Macao;
| | - Manuel Ferre Pérez
- Centre for Automation and Robotics (CAR) UPM-CSIC, Universidad Politécnica de Madrid (UPM), 28006 Madrid, Spain; (K.Z.); (F.J.B.)
| |
Collapse
|
5
|
Zhang H, Peng B, Tian L, Samuel OW, Li G. Continuous Kalman Estimation Method for Finger Kinematics Tracking from Surface Electromyography. CYBORG AND BIONIC SYSTEMS 2024; 5:0094. [PMID: 38751457 PMCID: PMC11093877 DOI: 10.34133/cbsystems.0094] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2023] [Accepted: 01/09/2024] [Indexed: 05/18/2024] Open
Abstract
Deciphering hand motion intention from surface electromyography (sEMG) encounters challenges posed by the requisites of multiple degrees of freedom (DOFs) and adaptability. Unlike discrete action classification grounded in pattern recognition, the pursuit of continuous kinematics estimation is appreciated for its inherent naturalness and intuitiveness. However, prevailing estimation techniques contend with accuracy limitations and substantial computational demands. Kalman estimation technology, celebrated for its ease of implementation and real-time adaptability, finds extensive application across diverse domains. This study introduces a continuous Kalman estimation method, leveraging a system model with sEMG and joint angles as inputs and outputs. Facilitated by model parameter training methods, the approach deduces multiple DOF finger kinematics simultaneously. The method's efficacy is validated using a publicly accessible database, yielding a correlation coefficient (CC) of 0.73. With over 45,000 windows for training Kalman model parameters, the average computation time remains under 0.01 s. This pilot study amplifies its potential for further exploration and application within the realm of continuous finger motion estimation technology.
Collapse
Affiliation(s)
- Haoshi Zhang
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), Shenzhen 518055, China
- Shenzhen College of Advanced Technology,
University of Chinese Academy of Sciences, Shenzhen 518055, China
| | - Boxing Peng
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), Shenzhen 518055, China
- Shenzhen College of Advanced Technology,
University of Chinese Academy of Sciences, Shenzhen 518055, China
| | - Lan Tian
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), Shenzhen 518055, China
| | - Oluwarotimi Williams Samuel
- Shandong Zhongke Advanced Technology Co. Ltd., Jinan 250000, China
- School of Computing,
University of Derby, Derby DE22 3AW, UK
| | - Guanglin Li
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), Shenzhen 518055, China
- Shenzhen College of Advanced Technology,
University of Chinese Academy of Sciences, Shenzhen 518055, China
- Shandong Zhongke Advanced Technology Co. Ltd., Jinan 250000, China
| |
Collapse
|
6
|
Lin C, Zhang X. Fusion inception and transformer network for continuous estimation of finger kinematics from surface electromyography. Front Neurorobot 2024; 18:1305605. [PMID: 38765870 PMCID: PMC11100415 DOI: 10.3389/fnbot.2024.1305605] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Accepted: 03/04/2024] [Indexed: 05/22/2024] Open
Abstract
Decoding surface electromyography (sEMG) to recognize human movement intentions enables us to achieve stable, natural and consistent control in the field of human computer interaction (HCI). In this paper, we present a novel deep learning (DL) model, named fusion inception and transformer network (FIT), which effectively models both local and global information on sequence data by fully leveraging the capabilities of Inception and Transformer networks. In the publicly available Ninapro dataset, we selected surface EMG signals from six typical hand grasping maneuvers in 10 subjects for predicting the values of the 10 most important joint angles in the hand. Our model's performance, assessed through Pearson's correlation coefficient (PCC), root mean square error (RMSE), and R-squared (R2) metrics, was compared with temporal convolutional network (TCN), long short-term memory network (LSTM), and bidirectional encoder representation from transformers model (BERT). Additionally, we also calculate the training time and the inference time of the models. The results show that FIT is the most performant, with excellent estimation accuracy and low computational cost. Our model contributes to the development of HCI technology and has significant practical value.
Collapse
Affiliation(s)
- Chuang Lin
- School of Information Science and Technology, Dalian Maritime University, Dalian, China
| | | |
Collapse
|
7
|
Lin C, He Z. A rotary transformer cross-subject model for continuous estimation of finger joints kinematics and a transfer learning approach for new subjects. Front Neurosci 2024; 18:1306050. [PMID: 38572147 PMCID: PMC10987947 DOI: 10.3389/fnins.2024.1306050] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2023] [Accepted: 02/27/2024] [Indexed: 04/05/2024] Open
Abstract
Introduction Surface Electromyographic (sEMG) signals are widely utilized for estimating finger kinematics continuously in human-machine interfaces (HMI), and deep learning approaches are crucial in constructing the models. At present, most models are extracted on specific subjects and do not have cross-subject generalizability. Considering the erratic nature of sEMG signals, a model trained on a specific subject cannot be directly applied to other subjects. Therefore, in this study, we proposed a cross-subject model based on the Rotary Transformer (RoFormer) to extract features of multiple subjects for continuous estimation kinematics and extend it to new subjects by adversarial transfer learning (ATL) approach. Methods We utilized the new subject's training data and an ATL approach to calibrate the cross-subject model. To improve the performance of the classic transformer network, we compare the impact of different position embeddings on model performance, including learnable absolute position embedding, Sinusoidal absolute position embedding, and Rotary Position Embedding (RoPE), and eventually selected RoPE. We conducted experiments on 10 randomly selected subjects from the NinaproDB2 dataset, using Pearson correlation coefficient (CC), normalized root mean square error (NRMSE), and coefficient of determination (R2) as performance metrics. Results The proposed model was compared with four other models including LSTM, TCN, Transformer, and CNN-Attention. The results demonstrated that both in cross-subject and subject-specific cases the performance of RoFormer was significantly better than the other four models. Additionally, the ATL approach improves the generalization performance of the cross-subject model better than the fine-tuning (FT) transfer learning approach. Discussion The findings indicate that the proposed RoFormer-based method with an ATL approach has the potential for practical applications in robot hand control and other HMI settings. The model's superior performance suggests its suitability for continuous estimation of finger kinematics across different subjects, addressing the limitations of subject-specific models.
Collapse
Affiliation(s)
- Chuang Lin
- School of Information Science and Technology, Dalian Maritime University, Dalian, China
| | | |
Collapse
|
8
|
Deprez K, De Baecke E, Tijskens M, Schoeters R, Velghe M, Thielens A. A Circular, Wireless Surface-Electromyography Array. SENSORS (BASEL, SWITZERLAND) 2024; 24:1119. [PMID: 38400278 PMCID: PMC10892791 DOI: 10.3390/s24041119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Revised: 01/29/2024] [Accepted: 02/05/2024] [Indexed: 02/25/2024]
Abstract
Commercial, high-tech upper limb prostheses offer a lot of functionality and are equipped with high-grade control mechanisms. However, they are relatively expensive and are not accessible to the majority of amputees. Therefore, more affordable, accessible, open-source, and 3D-printable alternatives are being developed. A commonly proposed approach to control these prostheses is to use bio-potentials generated by skeletal muscles, which can be measured using surface electromyography (sEMG). However, this control mechanism either lacks accuracy when a single sEMG sensor is used or involves the use of wires to connect to an array of multiple nodes, which hinders patients' movements. In order to mitigate these issues, we have developed a circular, wireless s-EMG array that is able to collect sEMG potentials on an array of electrodes that can be spread (not) uniformly around the circumference of a patient's arm. The modular sEMG system is combined with a Bluetooth Low Energy System on Chip, motion sensors, and a battery. We have benchmarked this system with a commercial, wired, state-of-the-art alternative and found an r = 0.98 (p < 0.01) Spearman correlation between the root-mean-squared (RMS) amplitude of sEMG measurements measured by both devices for the same set of 20 reference gestures, demonstrating that the system is accurate in measuring sEMG. Additionally, we have demonstrated that the RMS amplitudes of sEMG measurements between the different nodes within the array are uncorrelated, indicating that they contain independent information that can be used for higher accuracy in gesture recognition. We show this by training a random forest classifier that can distinguish between 6 gestures with an accuracy of 97%. This work is important for a large and growing group of amputees whose quality of life could be improved using this technology.
Collapse
Affiliation(s)
- Kenneth Deprez
- Department of Information Technology, imec, Ghent University, 9052 Ghent, Belgium; (K.D.); (E.D.B.); (M.T.); (R.S.); (M.V.)
| | - Eliah De Baecke
- Department of Information Technology, imec, Ghent University, 9052 Ghent, Belgium; (K.D.); (E.D.B.); (M.T.); (R.S.); (M.V.)
| | - Mauranne Tijskens
- Department of Information Technology, imec, Ghent University, 9052 Ghent, Belgium; (K.D.); (E.D.B.); (M.T.); (R.S.); (M.V.)
| | - Ruben Schoeters
- Department of Information Technology, imec, Ghent University, 9052 Ghent, Belgium; (K.D.); (E.D.B.); (M.T.); (R.S.); (M.V.)
| | - Maarten Velghe
- Department of Information Technology, imec, Ghent University, 9052 Ghent, Belgium; (K.D.); (E.D.B.); (M.T.); (R.S.); (M.V.)
- Centre for Sustainability, Environment and Health, National Institute for Public Health and the Environment (RIVM), 3720 BA Bilthoven, The Netherlands
| | - Arno Thielens
- Department of Information Technology, imec, Ghent University, 9052 Ghent, Belgium; (K.D.); (E.D.B.); (M.T.); (R.S.); (M.V.)
- Photonics Initiative, The Advanced Science Research Center, The Graduate Center of the City University of New York, New York, NY 10031, USA
| |
Collapse
|
9
|
Muschter E, Schulz J, Tömösközi M, Herbst L, Küssel L, Sefunç M, Holtzhausen S, Speidel S, Fitzek FHP, Li SC. Coming in handy: CeTI-Age - A comprehensive database of kinematic hand movements across the lifespan. Sci Data 2023; 10:826. [PMID: 38007482 PMCID: PMC10676381 DOI: 10.1038/s41597-023-02738-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Accepted: 11/09/2023] [Indexed: 11/27/2023] Open
Abstract
The Tactile Internet aims to advance human-human and human-machine interactions that also utilize hand movements in real, digitized, and remote environments. Attention to elderly generations is necessary to make the Tactile Internet age inclusive. We present the first age-representative kinematic database consisting of various hand gesturing and grasping movements at individualized paces, thus capturing naturalistic movements. We make this comprehensive database of kinematic hand movements across the adult lifespan (CeTI-Age-Kinematic-Hand) publicly available to facilitate a deeper understanding of intra-individual-focusing especially on age-related differences-and inter-individual variability in hand kinematics. The core of the database contains participants' hand kinematics recorded with wearable resistive bend sensors, individual static 3D hand models, and all instructional videos used during the data acquisition. Sixty-three participants ranging from age 20 to 80 years performed six repetitions of 40 different naturalistic hand movements at individual paces. This unique database with data recorded from an adult lifespan sample can be used to advance machine-learning approaches in hand kinematic modeling and movement prediction for age-inclusive applications.
Collapse
Affiliation(s)
- Evelyn Muschter
- Technische Universität Dresden, Centre for Tactile Internet with Human-in-the-Loop, Dresden, 01062, Germany.
- Technische Universität Dresden, Chair of Lifespan Developmental Neuroscience, Dresden, 01062, Germany.
| | - Jonas Schulz
- Technische Universität Dresden, Deutsche Telekom Chair of Communication Networks, Dresden, 01062, Germany
| | - Máté Tömösközi
- Technische Universität Dresden, Centre for Tactile Internet with Human-in-the-Loop, Dresden, 01062, Germany
- Technische Universität Dresden, Deutsche Telekom Chair of Communication Networks, Dresden, 01062, Germany
| | - Leonie Herbst
- Technische Universität Dresden, Centre for Tactile Internet with Human-in-the-Loop, Dresden, 01062, Germany
- Technische Universität Dresden, Chair of Lifespan Developmental Neuroscience, Dresden, 01062, Germany
| | - Lena Küssel
- Technische Universität Dresden, Chair of Lifespan Developmental Neuroscience, Dresden, 01062, Germany
| | | | - Stefan Holtzhausen
- Technische Universität Dresden, Chair of Virtual Product Development, Dresden, 01062, Germany
| | - Stefanie Speidel
- Technische Universität Dresden, Centre for Tactile Internet with Human-in-the-Loop, Dresden, 01062, Germany
- National Center for Tumor Diseases (NCT/UCC) Dresden, Department of Translational Surgical Oncology, Dresden, 01307, Germany
| | - Frank H P Fitzek
- Technische Universität Dresden, Centre for Tactile Internet with Human-in-the-Loop, Dresden, 01062, Germany
- Technische Universität Dresden, Deutsche Telekom Chair of Communication Networks, Dresden, 01062, Germany
| | - Shu-Chen Li
- Technische Universität Dresden, Centre for Tactile Internet with Human-in-the-Loop, Dresden, 01062, Germany.
- Technische Universität Dresden, Chair of Lifespan Developmental Neuroscience, Dresden, 01062, Germany.
| |
Collapse
|
10
|
Zabihi S, Rahimian E, Asif A, Mohammadi A. TraHGR: Transformer for Hand Gesture Recognition via Electromyography. IEEE Trans Neural Syst Rehabil Eng 2023; 31:4211-4224. [PMID: 37831560 DOI: 10.1109/tnsre.2023.3324252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/15/2023]
Abstract
Deep learning-based Hand Gesture Recognition (HGR) via surface Electromyogram (sEMG) signals have recently shown considerable potential for development of advanced myoelectric-controlled prosthesis. Although deep learning techniques can improve HGR accuracy compared to their classical counterparts, classifying hand movements based on sparse multichannel sEMG signals is still a challenging task. Furthermore, existing deep learning approaches, typically, include only one model as such can hardly extract representative features. In this paper, we aim to address this challenge by capitalizing on the recent advances in hybrid models and transformers. In other words, we propose a hybrid framework based on the transformer architecture, which is a relatively new and revolutionizing deep learning model. The proposed hybrid architecture, referred to as the Transformer for Hand Gesture Recognition (TraHGR), consists of two parallel paths followed by a linear layer that acts as a fusion center to integrate the advantage of each module. We evaluated the proposed architecture TraHGR based on the commonly used second Ninapro dataset, referred to as the DB2. The sEMG signals in the DB2 dataset are measured in real-life conditions from 40 healthy users, each performing 49 gestures. We have conducted an extensive set of experiments to test and validate the proposed TraHGR architecture, and compare its achievable accuracy with several recently proposed HGR classification algorithms over the same dataset. We have also compared the results of the proposed TraHGR architecture with each individual path and demonstrated the distinguishing power of the proposed hybrid architecture. The recognition accuracies of the proposed TraHGR architecture for the window of size 200ms and step size of 100ms are 86.00%, 88.72%, 81.27%, and 93.74%, which are 2.30%, 4.93%, 8.65%, and 4.20% higher than the state-of-the-art performance for DB2 (49 gestures), DB2-B (17 gestures), DB2-C (23 gestures), and DB2-D (9 gestures), respectively.
Collapse
|
11
|
Zhang J, Matsuda Y, Fujimoto M, Suwa H, Yasumoto K. Movement recognition via channel-activation-wise sEMG attention. Methods 2023; 218:39-47. [PMID: 37479003 DOI: 10.1016/j.ymeth.2023.06.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2022] [Revised: 04/06/2023] [Accepted: 06/28/2023] [Indexed: 07/23/2023] Open
Abstract
CONTEXT Surface electromyography (sEMG) signals contain rich information recorded from muscle movements and therefore reflect the user's intention. sEMG has seen dominant applications in rehabilitation, clinical diagnosis as well as human engineering, etc. However, current feature extraction methods for sEMG signals have been seriously limited by their stochasticity, transiency, and non-stationarity. OBJECTIVE Our objective is to combat the difficulties induced by the aforementioned downsides of sEMG and thereby extract representative features for various downstream movement recognition. METHOD We propose a novel 3-axis view of sEMG features composed of temporal, spatial, and channel-wise summary. We leverage the state-of-the-art architecture Transformer to enforce efficient parallel search and to get rid of limitations imposed by previous work in gesture classification. The transformer model is designed on top of an attention-based module, which allows for the extraction of global contextual relevance among channels and the use of this relevance for sEMG recognition. RESULTS We compared the proposed method against existing methods on two Ninapro datasets consisting of data from both healthy people and amputees. Experimental results show the proposed method attains the state-of-the-art (SOTA) accuracy on both datasets. We further show that the proposed method enjoys strong generalization ability: a new SOTA is achieved by pretraining the model on a different dataset followed by fine-tuning it on the target dataset.
Collapse
Affiliation(s)
- Jiaxuan Zhang
- Nara Institute of Science and Technology (NAIST), Ikoma, Nara 630-0192, Japan.
| | - Yuki Matsuda
- Nara Institute of Science and Technology (NAIST), Ikoma, Nara 630-0192, Japan
| | | | - Hirohiko Suwa
- Nara Institute of Science and Technology (NAIST), Ikoma, Nara 630-0192, Japan
| | - Keiichi Yasumoto
- Nara Institute of Science and Technology (NAIST), Ikoma, Nara 630-0192, Japan
| |
Collapse
|
12
|
Wei W, Tan F, Zhang H, Mao H, Fu M, Samuel OW, Li G. Surface electromyogram, kinematic, and kinetic dataset of lower limb walking for movement intent recognition. Sci Data 2023; 10:358. [PMID: 37280249 DOI: 10.1038/s41597-023-02263-3] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Accepted: 05/23/2023] [Indexed: 06/08/2023] Open
Abstract
Surface electromyogram (sEMG) offers a rich set of motor information for decoding limb motion intention that serves as a control input to Intelligent human-machine synergy systems (IHMSS). Despite growing interest in IHMSS, the current publicly available datasets are limited and can hardly meet the growing demands of researchers. This study presents a novel lower limb motion dataset (designated as SIAT-LLMD), comprising sEMG, kinematic, and kinetic data with corresponding labels acquired from 40 healthy humans during 16 movements. The kinematic and kinetic data were collected using a motion capture system and six-dimensional force platforms and processed using OpenSim software. The sEMG data were recorded using nine wireless sensors placed on the subjects' thigh and calf muscles on the left limb. Besides, SIAT-LLMD provides labels to classify the different movements and different gait phases. Analysis of the dataset verified the synchronization and reproducibility, and codes for effective data processing are provided. The proposed dataset can serve as a new resource for exploring novel algorithms and models for characterizing lower limb movements.
Collapse
Affiliation(s)
- Wenhao Wei
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), and the SIAT Branch, Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, 518055, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, Guangdong, 518055, China
| | - Fangning Tan
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), and the SIAT Branch, Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, 518055, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, Guangdong, 518055, China
| | - Hang Zhang
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, Guangdong, 518055, China
| | - He Mao
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), and the SIAT Branch, Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, 518055, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, Guangdong, 518055, China
| | - Menglong Fu
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, Guangdong, 518055, China
| | - Oluwarotimi Williams Samuel
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), and the SIAT Branch, Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, 518055, China.
- School of Computing and Engineering, University of Derby, Derby, DE22 3AW, UK.
- Data Science Research Center, University of Derby, Derby, DE22 3AW, UK.
| | - Guanglin Li
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), and the SIAT Branch, Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, 518055, China.
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, Guangdong, 518055, China.
| |
Collapse
|
13
|
Mendes Junior JJA, Pontim CE, Dias TS, Campos DP. How do sEMG segmentation parameters influence pattern recognition process? An approach based on wearable sEMG sensor. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104546] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
|
14
|
Fang Y, Lu H, Liu H. Multi-modality deep forest for hand motion recognition via fusing sEMG and acceleration signals. INT J MACH LEARN CYB 2023; 14:1119-1131. [PMID: 36339898 PMCID: PMC9628499 DOI: 10.1007/s13042-022-01687-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2022] [Accepted: 10/10/2022] [Indexed: 11/28/2022]
Abstract
Bio-signal based hand motion recognition plays a critical role in the tasks of human-machine interaction, such as the natural control of multifunctional prostheses. Although a large number of classification technologies have been taken to improve the motion recognition accuracy, it is still a challenge to achieve acceptable performance for multiple modality input. This study proposes a multi-modality deep forest (MMDF) framework to identify hand motions, in which surface electromyographic signals (sEMG) and acceleration signals (ACC) are fused at the input level. The proposed MMDF framework constitutes of three main stages, sEMG and ACC feature extraction, feature dimension reduction, and a cascade structure deep forest for classification. A public database "Ninapro DB7" is used to evaluate the performance of the proposed framework, and the experimental results show that it can achieve a significantly higher accuracy than that of competitors. Besides, our experimental results also show that MMDF outperforms other traditional classifiers with the input of the single modality of sEMG signals. In sum, this study verifies that ACC signals can be an excellent supplementary for sEMG, and MMDF is a plausible solution to fuse mulit-modality bio-signals for human motion recognition.
Collapse
Affiliation(s)
- Yinfeng Fang
- School of Communication Engineering, Hangzhou Dianzi University, Hangzhou, 310018 Zhejiang China
| | - Huiqiao Lu
- School of Communication Engineering, Hangzhou Dianzi University, Hangzhou, 310018 Zhejiang China
| | - Han Liu
- College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060 Guangdong China
| |
Collapse
|
15
|
Simultaneous estimation of grip force and wrist angles by surface electromyography and acceleration signals. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
16
|
Zhang W, Zhao T, Zhang J, Wang Y. LST-EMG-Net: Long short-term transformer feature fusion network for sEMG gesture recognition. Front Neurorobot 2023; 17:1127338. [PMID: 36925629 PMCID: PMC10011454 DOI: 10.3389/fnbot.2023.1127338] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2022] [Accepted: 02/14/2023] [Indexed: 03/08/2023] Open
Abstract
With the development of signal analysis technology and artificial intelligence, surface electromyography (sEMG) signal gesture recognition is widely used in rehabilitation therapy, human-computer interaction, and other fields. Deep learning has gradually become the mainstream technology for gesture recognition. It is necessary to consider the characteristics of the surface EMG signal when constructing the deep learning model. The surface electromyography signal is an information carrier that can reflect neuromuscular activity. Under the same circumstances, a longer signal segment contains more information about muscle activity, and a shorter segment contains less information about muscle activity. Thus, signals with longer segments are suitable for recognizing gestures that mobilize complex muscle activity, and signals with shorter segments are suitable for recognizing gestures that mobilize simple muscle activity. However, current deep learning models usually extract features from single-length signal segments. This can easily cause a mismatch between the amount of information in the features and the information needed to recognize gestures, which is not conducive to improving the accuracy and stability of recognition. Therefore, in this article, we develop a long short-term transformer feature fusion network (referred to as LST-EMG-Net) that considers the differences in the timing lengths of EMG segments required for the recognition of different gestures. LST-EMG-Net imports multichannel sEMG datasets into a long short-term encoder. The encoder extracts the sEMG signals' long short-term features. Finally, we successfully fuse the features using a feature cross-attention module and output the gesture category. We evaluated LST-EMG-Net on multiple datasets based on sparse channels and high density. It reached 81.47, 88.24, and 98.95% accuracy on Ninapro DB2E2, DB5E3 partial gesture, and CapgMyo DB-c, respectively. Following the experiment, we demonstrated that LST-EMG-Net could increase the accuracy and stability of various gesture identification and recognition tasks better than existing networks.
Collapse
Affiliation(s)
- Wenli Zhang
- Faculty of Information Technology, Beijing University of Technology, Beijing, China
| | - Tingsong Zhao
- Faculty of Information Technology, Beijing University of Technology, Beijing, China
| | - Jianyi Zhang
- College of Art and Design, Beijing University of Technology, Beijing, China
| | - Yufei Wang
- Faculty of Information Technology, Beijing University of Technology, Beijing, China
| |
Collapse
|
17
|
Li W, Yuan J, Zhang L, Cui J, Wang X, Li H. sEMG-based technology for silent voice recognition. Comput Biol Med 2023; 152:106336. [PMID: 36473341 DOI: 10.1016/j.compbiomed.2022.106336] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2022] [Revised: 10/27/2022] [Accepted: 11/15/2022] [Indexed: 11/19/2022]
Abstract
Silent speech recognition (SSR) is a system that implements speech communication when a sound signal is not available using surface electromyography (sEMG)-based speech recognition. Researchers have used surface electrodes to record the electrically-activated potential of human articulation muscles to recognize speech content. SSR can be used for pilot-assisted speech recognition, communication of individuals with speech impairment, private communication, and other fields. In this feasibility study, we collected sEMG data for ten single Mandarin numeric words. After reducing power frequency interference and power supply noise from the sEMG signal, short-term energy (STE) was used for voice activity detection (VAD). The power spectrum features were extracted and fed into the classifier for final identification results. We used the Hold-out method to divide the data into training and test sets on a 7-3 scale, with an average accuracy of 92.3% and a maximum of 100% using a support vector machine (SVM) classifier. Experimental results showed that the proposed method has development potential, and is effective in identifying isolated words from the sEMG signal of the articulation muscles.
Collapse
Affiliation(s)
- Wei Li
- Research Center for Ultrasonics and Technologies, Institute of Acoustics, Chinese Academy of Sciences, Beijing 100190, China; School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Jianping Yuan
- Research Center for Ultrasonics and Technologies, Institute of Acoustics, Chinese Academy of Sciences, Beijing 100190, China
| | - Lu Zhang
- Research Center for Ultrasonics and Technologies, Institute of Acoustics, Chinese Academy of Sciences, Beijing 100190, China; School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Jie Cui
- Research Center for Ultrasonics and Technologies, Institute of Acoustics, Chinese Academy of Sciences, Beijing 100190, China
| | - Xiaodong Wang
- Research Center for Ultrasonics and Technologies, Institute of Acoustics, Chinese Academy of Sciences, Beijing 100190, China
| | - Hua Li
- Research Center for Ultrasonics and Technologies, Institute of Acoustics, Chinese Academy of Sciences, Beijing 100190, China.
| |
Collapse
|
18
|
Flower XL, Poonguzhali S. Performance improvement and complexity reduction in the classification of EMG signals with mRMR-based CNN-KNN combined model. JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2022. [DOI: 10.3233/jifs-220811] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
For real-time applications, the performance in classifying the movement should be as high as possible, and the computational complexity should be low. This paper focuses on the classification of five upper arm movements which can be provided as a control for human-machine interface (HMI) based applications. The conventional machine learning algorithms are used for classification with both time and frequency domain features, and k-nearest neighbor (KNN) outplay others. To further improve the classification accuracy, pretrained CNN architectures are employed which leads to computational complexity and memory requirements. To overcome this, the deep convolutional neural network (CNN) model is introduced with three convolutional layers. To further improve the performance which is the key idea behind real-time applications, a hybrid CNN-KNN model is proposed. Even though the performance is high, the computation costs of the hybrid method are more. Minimum redundancy maximum relevance (mRMR), a feature selection method makes an effort to reduce feature dimensions. As a result, better performance is achieved by our proposed method CNN-KNN with mRMR which reduces computational complexity and memory requirement with a mean prediction accuracy of about 99.05±0.25% with 100 features.
Collapse
Affiliation(s)
- X. Little Flower
- Department of Electronics and Communication Engineering, College of Engineering Guindy (CEG), Anna University, Chennai, India
| | - S. Poonguzhali
- Department of Electronics and Communication Engineering, College of Engineering Guindy (CEG), Anna University, Chennai, India
| |
Collapse
|
19
|
Zhao K, Zhang Z, Wen H, Scano A. Number of trials and data structure affect the number and components of muscle synergies in upper-limb reaching movements. Physiol Meas 2022; 43. [PMID: 36195081 DOI: 10.1088/1361-6579/ac9773] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2022] [Accepted: 10/04/2022] [Indexed: 02/07/2023]
Abstract
Objective.Due to the variability of human movements, muscle activations vary among trials and subjects. However, few studies investigated how data organization methods for addressing variability impact the extracted muscle synergies.Approach.Fifteen healthy subjects performed a large set of upper limb multi-directional point-to-point reaching movements. Then, the study extracted muscle synergies under different data settings and investigated how data structure prior to synergy extraction, namely concatenation, averaging, and single trial, the number of considered trials, and the number of reaching directions affected the number and components of muscle synergies.Main results.The results showed that the number and components of synergies were significantly affected by the data structure. The concatenation method identified the highest number of synergies, and the averaging method usually found a smaller number of synergies. When the concatenated trials or reaching directions was lower than a minimum value, the number of synergies increased with the increase of the number of trials or reaching directions; however, when the number of trials or reaching directions reached a threshold, the number of synergies was usually constant or with less variation even when novel directions and trials were added. Similarity analysis also showed a slight increase when the number of trials or reaching directions was lower than a threshold. This study recommends that at least five trials and four reaching directions and the concatenation method are considered in muscle synergies analysis during upper limb tasks.Significance.This study makes the researchers focus on the variability analysis induced by the diseases rather than the techniques applied for synergies analysis and promotes applications of muscle synergies in clinical scenarios.
Collapse
Affiliation(s)
- Kunkun Zhao
- School of Biomedical Engineering and Informatics, Nanjing Medical University, Nanjing, People's Republic of China.,School of Mechanical Engineering, Southeast University, Nanjing, People's Republic of China
| | - Zhisheng Zhang
- School of Mechanical Engineering, Southeast University, Nanjing, People's Republic of China
| | - Haiying Wen
- School of Mechanical Engineering, Southeast University, Nanjing, People's Republic of China
| | - Alessandro Scano
- UOS STIIMA Lecco-Human-Centered, Smart & Safe, Living Environment, Italian National Research Council (CNR), Via Previati 1/E, 23900 Lecco, Italy
| |
Collapse
|
20
|
Xie B, Meng J, Li B, Harland A. Biosignal-based transferable attention Bi-ConvGRU deep network for hand-gesture recognition towards online upper-limb prosthesis control. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 224:106999. [PMID: 35841852 DOI: 10.1016/j.cmpb.2022.106999] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/21/2022] [Revised: 06/13/2022] [Accepted: 06/30/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND AND OBJECTIVE Upper-limb amputation can significantly affect a person's capabilities with a dramatic impact on their quality of life. As a biological signal, surface electromyogram (sEMG) provides a non-invasive means to measure underlying muscle activation patterns, corresponding to specific hand gestures. This project aims to develop a real-time deep learning based recognition model to automatically and reliably recognise these complex signals of a wide range of daily hand gestures from amputees and non-amputees. METHODS This paper proposes an attention bidirectional Convolutional Gated Recurrent Unit (Bi-ConvGRU) deep neural network for hand-gesture recognition. By training on sEMG data from both amputees and non-amputees, the model can learn to recognise a group of fine-grained hand movements. This is a significantly more challenging and underexplored area, compared to existing studies on coarse-control in lower limbs. One dimensional CNNs are initially used to extract intra-channel features. The novel use of a bidirectional sequential GRU (Bi-GRU) deep neural network allows the exploration of correlation of muscle activation among multi-channel sEMG signals from both prior and posterior time sequences. Importantly, the attention mechanism is employed following Bi-GRU layers. This enables the model to learn vital parts and feature weights, increasing robustness to bio-data noise and irregularity. Finally, we introduce the first of its kind transfer learning, demonstrating that a baseline model pre-trained with non-amputee data can be effectively refined with amputee data to build a personalised model for amputees. RESULTS The attention Bi-ConvGRU was evaluated on the benchmark database Ninapro, and achieved an average accuracy of 88.7%, outperforming the state-of-the-art on 18 gesture recognition by 6.7%. CONCLUSIONS To our knowledge, the developed end-to-end deep learning model is the first of its kind that enables reliable predictive decision making in short time windows (160ms). This reduced latency limits physiological awareness, enabling the potential for real-time, online and thus more intuitive bio-control of prosthetic devices for amputees.
Collapse
Affiliation(s)
- Baao Xie
- School of Electrical and Information Engineering, Tianjin University, China; Eastern Institute of Advanced Study, China
| | - James Meng
- Lancashire Teaching Hospitals, NHS Foundation Trust, PR2 9HT, UK
| | - Baihua Li
- Department of Computer Science, Loughborough University, LE11 3TU, UK.
| | - Andy Harland
- School of Mechanical, Electrical and Manufacturing Engineering, Loughborough University, LE11 3TU, UK
| |
Collapse
|
21
|
Li C, Yang H, Cheng L, Huang F, Zhao S, Li D, Yan R. Quantitative Assessment of Hand Motor Function for Post-Stroke Rehabilitation Based on HAGCN and Multimodality Fusion. IEEE Trans Neural Syst Rehabil Eng 2022; 30:2032-2041. [PMID: 35853069 DOI: 10.1109/tnsre.2022.3192479] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Quantitative assessment of hand function can assist therapists in providing appropriate rehabilitation strategies, which plays an essential role in post-stroke rehabilitation. Conventionally, the assessment process relies heavily on clinical experience and lacks quantitative analysis. To quantitatively assess the hand motor function of patients with post-stroke hemiplegia, this study proposes a novel multi-modality fusion assessment framework. This framework includes three components: the kinematic feature extraction based on a graph convolutional network (HAGCN), the surface electromyography (sEMG) signal processing based on a multi-layer long short term memory (LSTM) network, and the quantitative assessment based on the multi-modality fusion. To the best of the authors' knowledge, this is the first study of applying a graph convolution network to assess the hand motor function. We also collect the kinematic data and sEMG data from 70 subjects who completed 28 types of hand movements. Therapists first graded patients using traditional motor assessment scales (Brunnstrom Scale and Fugl-Meyer Assessment Scale) and further refined the patient's motor assessment result by their experience. Then, we trained the HAGCN and LSTM networks and quantitatively assessed each patient based on the proposed assessment framework. Finally, the Spearman correlation coefficient (SC) between the assessment result of this study and the traditional scale are 0.908 and 0.967, demonstrating a significant correlation between the proposed assessment and the traditional scale scores. In addition, the SC value between the score of this study and the refined hand motor function is 0.997, indicating the "ceiling effect" of some traditional scales can be avoided.
Collapse
|
22
|
Geng Y, Yu Z, Long Y, Qin L, Chen Z, Li Y, Guo X, Li G. A CNN-Attention Network for Continuous Estimation of Finger Kinematics from Surface Electromyography. IEEE Robot Autom Lett 2022. [DOI: 10.1109/lra.2022.3169448] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Affiliation(s)
- Yanjuan Geng
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Zhebin Yu
- Hebei University of Technology, Tianjin, China
| | - Yucheng Long
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Liuni Qin
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Ziyin Chen
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Yongcheng Li
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Xin Guo
- Hebei University of Technology, Tianjin, China
| | - Guanglin Li
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| |
Collapse
|
23
|
Li C, Yang H, Cheng L, Huang F. A Time-Series Augmentation Method Based on Empirical Mode Decomposition and Integrated LSTM Neural Network. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:333-336. [PMID: 36085680 DOI: 10.1109/embc48229.2022.9871795] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Adequate patients' data have always been critical for disease assessment. However, large amounts of patient data are often difficult to collect, especially when patients are required to complete a series of assessment movements. For example, assessing the hand motor function of stroke patients or Parkinson's patients requires patients to complete a series of evaluation movements, and it is often difficult for patients to complete each group of actions multiple times, resulting in a small amount of data. To solve the problem of insufficient data quantity, this study proposes a data augmentation method based on empirical mode decomposition and integrated long short-term memory neural network (EMD-ILSTM). The method mainly consists of two parts: one is to decompose the raw signal by the method of EMD, and the other is to use LSTM for data augmentation of the decomposed signal. Then, the method is tested on the public dataset named Ninaweb, and the test results show that the classification accuracy can be improved by 5.2% by using the augmented data for classification tasks. Finally, clinical trials are conducted to verify that after dimensionality reduction, the augmented data and raw data have smaller intra-class distances and larger inter-class distances, indicating that data augmentation is effective.
Collapse
|
24
|
Sandoval-Espino JA, Zamudio-Lara A, Marbán-Salgado JA, Escobedo-Alatorre JJ, Palillero-Sandoval O, Velásquez-Aguilar JG. Selection of the Best Set of Features for sEMG-Based Hand Gesture Recognition Applying a CNN Architecture. SENSORS 2022; 22:s22134972. [PMID: 35808467 PMCID: PMC9269838 DOI: 10.3390/s22134972] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/28/2022] [Revised: 06/23/2022] [Accepted: 06/27/2022] [Indexed: 12/10/2022]
Abstract
The classification of surface myoelectric signals (sEMG) remains a great challenge when focused on its implementation in an electromechanical hand prosthesis, due to its nonlinear and stochastic nature, as well as the great difference between models applied offline and online. In this work, the selection of the set of the features that allowed us to obtain the best results for the classification of this type of signals is presented. In order to compare the results obtained, the Nina PRO DB2 and DB3 databases were used, which contain information on 50 different movements of 40 healthy subjects and 11 amputated subjects, respectively. The sEMG of each subject was acquired through 12 channels in a bipolar configuration. To carry out the classification, a convolutional neural network (CNN) was used and a comparison of four sets of features extracted in the time domain was made, three of which have shown good performance in previous works and one more that was used for the first time to train this type of network. Set one is composed of six features in the time domain (TD1), Set two has 10 features also in the time domain (TD2) including the autoregression model (AR), the third set has two features in the time domain derived from spectral moments (TD-PSD1), and finally, a set of five features also has information on the power spectrum of the signal obtained in the time domain (TD-PSD2). The selected features in each set were organized in four different ways for the formation of the training images. The results obtained show that the set of features TD-PSD2 obtained the best performance for all cases. With the set of features and the formation of images proposed, an increase in the accuracies of the models of 8.16% and 8.56% was obtained for the DB2 and DB3 databases, respectively, compared to the current state of the art that has used these databases.
Collapse
Affiliation(s)
- Jorge Arturo Sandoval-Espino
- Centro de Investigación en Ingeniería y Ciencias Aplicadas (CIICAp), Universidad Autónoma del Estado de Morelos, Cuernavaca 62209, Morelos, Mexico; (J.A.S.-E.); (A.Z.-L.); (J.J.E.-A.); (O.P.-S.)
| | - Alvaro Zamudio-Lara
- Centro de Investigación en Ingeniería y Ciencias Aplicadas (CIICAp), Universidad Autónoma del Estado de Morelos, Cuernavaca 62209, Morelos, Mexico; (J.A.S.-E.); (A.Z.-L.); (J.J.E.-A.); (O.P.-S.)
| | - José Antonio Marbán-Salgado
- Centro de Investigación en Ingeniería y Ciencias Aplicadas (CIICAp), Universidad Autónoma del Estado de Morelos, Cuernavaca 62209, Morelos, Mexico; (J.A.S.-E.); (A.Z.-L.); (J.J.E.-A.); (O.P.-S.)
- Correspondence:
| | - J. Jesús Escobedo-Alatorre
- Centro de Investigación en Ingeniería y Ciencias Aplicadas (CIICAp), Universidad Autónoma del Estado de Morelos, Cuernavaca 62209, Morelos, Mexico; (J.A.S.-E.); (A.Z.-L.); (J.J.E.-A.); (O.P.-S.)
| | - Omar Palillero-Sandoval
- Centro de Investigación en Ingeniería y Ciencias Aplicadas (CIICAp), Universidad Autónoma del Estado de Morelos, Cuernavaca 62209, Morelos, Mexico; (J.A.S.-E.); (A.Z.-L.); (J.J.E.-A.); (O.P.-S.)
| | - J. Guadalupe Velásquez-Aguilar
- Facultad de Ciencias Químicas e Ingeniería (FCQeI), Universidad Autónoma del Estado de Morelos, Cuernavaca 62209, Morelos, Mexico;
| |
Collapse
|
25
|
Hu X, Song A, Wang J, Zeng H, Wei W. Finger Movement Recognition via High-Density Electromyography of Intrinsic and Extrinsic Hand Muscles. Sci Data 2022; 9:373. [PMID: 35768439 PMCID: PMC9243097 DOI: 10.1038/s41597-022-01484-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2021] [Accepted: 06/15/2022] [Indexed: 11/09/2022] Open
Abstract
Surface electromyography (sEMG) is commonly used to observe the motor neuronal activity within muscle fibers. However, decoding dexterous body movements from sEMG signals is still quite challenging. In this paper, we present a high-density sEMG (HD-sEMG) signal database that comprises simultaneously recorded sEMG signals of intrinsic and extrinsic hand muscles. Specifically, twenty able-bodied participants performed 12 finger movements under two paces and three arm postures. HD-sEMG signals were recorded with a 64-channel high-density grid placed on the back of hand and an 8-channel armband around the forearm. Also, a data-glove was used to record the finger joint angles. Synchronisation and reproducibility of the data collection from the HD-sEMG and glove sensors were ensured. The collected data samples were further employed for automated recognition of dexterous finger movements. The introduced dataset offers a new perspective to study the synergy between the intrinsic and extrinsic hand muscles during dynamic finger movements. As this dataset was collected from multiple participants, it also provides a resource for exploring generalized models for finger movement decoding.
Collapse
Affiliation(s)
- Xuhui Hu
- State Key Laboratory of Bioelectronics, Nanjing, China
- Jiangsu Key Laboratory of Remote Measurement and Control, Nanjing, China
- School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Aiguo Song
- State Key Laboratory of Bioelectronics, Nanjing, China.
- Jiangsu Key Laboratory of Remote Measurement and Control, Nanjing, China.
- School of Instrument Science and Engineering, Southeast University, Nanjing, China.
| | - Jianzhi Wang
- State Key Laboratory of Bioelectronics, Nanjing, China
- Jiangsu Key Laboratory of Remote Measurement and Control, Nanjing, China
- School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Hong Zeng
- State Key Laboratory of Bioelectronics, Nanjing, China
- Jiangsu Key Laboratory of Remote Measurement and Control, Nanjing, China
- School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Wentao Wei
- School of Design Arts and Media, Nanjing University of Science and Technology, Nanjing, China
| |
Collapse
|
26
|
Wang S, Huang L, Jiang D, Sun Y, Jiang G, Li J, Zou C, Fan H, Xie Y, Xiong H, Chen B. Improved Multi-Stream Convolutional Block Attention Module for sEMG-Based Gesture Recognition. Front Bioeng Biotechnol 2022; 10:909023. [PMID: 35747495 PMCID: PMC9209772 DOI: 10.3389/fbioe.2022.909023] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Accepted: 05/17/2022] [Indexed: 11/25/2022] Open
Abstract
As a key technology for the non-invasive human-machine interface that has received much attention in the industry and academia, surface EMG (sEMG) signals display great potential and advantages in the field of human-machine collaboration. Currently, gesture recognition based on sEMG signals suffers from inadequate feature extraction, difficulty in distinguishing similar gestures, and low accuracy of multi-gesture recognition. To solve these problems a new sEMG gesture recognition network called Multi-stream Convolutional Block Attention Module-Gate Recurrent Unit (MCBAM-GRU) is proposed, which is based on sEMG signals. The network is a multi-stream attention network formed by embedding a GRU module based on CBAM. Fusing sEMG and ACC signals further improves the accuracy of gesture action recognition. The experimental results show that the proposed method obtains excellent performance on dataset collected in this paper with the recognition accuracies of 94.1%, achieving advanced performance with accuracy of 89.7% on the Ninapro DB1 dataset. The system has high accuracy in classifying 52 kinds of different gestures, and the delay is less than 300 ms, showing excellent performance in terms of real-time human-computer interaction and flexibility of manipulator control.
Collapse
Affiliation(s)
- Shudi Wang
- Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Wuhan University of Science and Technology, Wuhan, China
- Research Center for Biomimetic Robot and Intelligent Measurement and Control, Wuhan University of Science and Technology, Wuhan, China
| | - Li Huang
- College of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, China
- Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System, Wuhan University of Science and Technology, Wuhan, China
| | - Du Jiang
- Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Wuhan University of Science and Technology, Wuhan, China
- Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, China
| | - Ying Sun
- Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Wuhan University of Science and Technology, Wuhan, China
- Research Center for Biomimetic Robot and Intelligent Measurement and Control, Wuhan University of Science and Technology, Wuhan, China
- Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, China
| | - Guozhang Jiang
- Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Wuhan University of Science and Technology, Wuhan, China
- Research Center for Biomimetic Robot and Intelligent Measurement and Control, Wuhan University of Science and Technology, Wuhan, China
- Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, China
| | - Jun Li
- Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Wuhan University of Science and Technology, Wuhan, China
- Research Center for Biomimetic Robot and Intelligent Measurement and Control, Wuhan University of Science and Technology, Wuhan, China
| | - Cejing Zou
- Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Wuhan University of Science and Technology, Wuhan, China
- Research Center for Biomimetic Robot and Intelligent Measurement and Control, Wuhan University of Science and Technology, Wuhan, China
| | - Hanwen Fan
- Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Wuhan University of Science and Technology, Wuhan, China
- Research Center for Biomimetic Robot and Intelligent Measurement and Control, Wuhan University of Science and Technology, Wuhan, China
| | - Yuanmin Xie
- Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, China
| | - Hegen Xiong
- Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, China
| | - Baojia Chen
- Hubei Key Laboratory of Hydroelectric Machinery Design and Maintenance, China Three Gorges University, Yichang, China
| |
Collapse
|
27
|
Zhao K, Wen H, Zhang Z, Atzori M, Müller H, Xie Z, Scano A. Evaluation of Methods for the Extraction of Spatial Muscle Synergies. Front Neurosci 2022; 16:732156. [PMID: 35720729 PMCID: PMC9202610 DOI: 10.3389/fnins.2022.732156] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2021] [Accepted: 05/04/2022] [Indexed: 11/18/2022] Open
Abstract
Muscle synergies have been largely used in many application fields, including motor control studies, prosthesis control, movement classification, rehabilitation, and clinical studies. Due to the complexity of the motor control system, the full repertoire of the underlying synergies has been identified only for some classes of movements and scenarios. Several extraction methods have been used to extract muscle synergies. However, some of these methods may not effectively capture the nonlinear relationship between muscles and impose constraints on input signals or extracted synergies. Moreover, other approaches such as autoencoders (AEs), an unsupervised neural network, were recently introduced to study bioinspired control and movement classification. In this study, we evaluated the performance of five methods for the extraction of spatial muscle synergy, namely, principal component analysis (PCA), independent component analysis (ICA), factor analysis (FA), nonnegative matrix factorization (NMF), and AEs using simulated data and a publicly available database. To analyze the performance of the considered extraction methods with respect to several factors, we generated a comprehensive set of simulated data (ground truth), including spatial synergies and temporal coefficients. The signal-to-noise ratio (SNR) and the number of channels (NoC) varied when generating simulated data to evaluate their effects on ground truth reconstruction. This study also tested the efficacy of each synergy extraction method when coupled with standard classification methods, including K-nearest neighbors (KNN), linear discriminant analysis (LDA), support vector machines (SVM), and Random Forest (RF). The results showed that both SNR and NoC affected the outputs of the muscle synergy analysis. Although AEs showed better performance than FA in variance accounted for and PCA in synergy vector similarity and activation coefficient similarity, NMF and ICA outperformed the other three methods. Classification tasks showed that classification algorithms were sensitive to synergy extraction methods, while KNN and RF outperformed the other two methods for all extraction methods; in general, the classification accuracy of NMF and PCA was higher. Overall, the results suggest selecting suitable methods when performing muscle synergy-related analysis.
Collapse
Affiliation(s)
- Kunkun Zhao
- School of Mechanical Engineering, Southeast University, Nanjing, China
| | - Haiying Wen
- School of Mechanical Engineering, Southeast University, Nanjing, China
- Engineering Research Center of New Light Sources Technology and Equipment, Ministry of Education, Nanjing, China
- *Correspondence: Zhisheng Zhang,
| | - Zhisheng Zhang
- School of Mechanical Engineering, Southeast University, Nanjing, China
- *Correspondence: Zhisheng Zhang,
| | - Manfredo Atzori
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO Valais), Sierre, Switzerland
- Department of Neuroscience, University of Padova, Padua, Italy
| | - Henning Müller
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO Valais), Sierre, Switzerland
- Medical Faculty, University of Geneva, Geneva, Switzerland
| | - Zhongqu Xie
- School of Mechanical Engineering, Southeast University, Nanjing, China
| | - Alessandro Scano
- UOS STIIMA Lecco – Human-Centered, Smart and Safe, Living Environment, Italian National Research Council (CNR), Lecco, Italy
| |
Collapse
|
28
|
Zhu B, Zhang D, Chu Y, Gu Y, Zhao X. SeNic: An Open Source Dataset for sEMG-Based Gesture Recognition in Non-ideal Conditions. IEEE Trans Neural Syst Rehabil Eng 2022; 30:1252-1260. [PMID: 35533170 DOI: 10.1109/tnsre.2022.3173708] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
In order to reduce the gap between the laboratory environment and actual use in daily life of human-machine interaction based on surface electromyogram (sEMG) intent recognition, this paper presents a benchmark dataset of sEMG in non-ideal conditions (SeNic). The dataset mainly consists of 8-channel sEMG signals, and electrode shifts from an 3D-printed annular ruler. A total of 36 subjects participate in our data acquisition experiments of 7 gestures in non-ideal conditions, where non-ideal factors of 1) electrode shifts, 2) individual difference, 3) muscle fatigue, 4) inter-day difference, and 5) arm postures are elaborately involved. Signals of sEMG are validated first in temporal and frequency domains. Results of recognizing gestures in ideal conditions indicate the high quality of the dataset. Adverse impacts in non-ideal conditions are further revealed in the amplitudes of these data and recognition accuracies. To be concluded, SeNic is a benchmark dataset that introduces several non-ideal factors which often degrade the robustness of sEMG-based systems. It could be used as a freely available dataset and a common platform for researchers in the sEMG-based recognition community. The benchmark dataset SeNic are available online via the website3.
Collapse
|
29
|
Zheng N, Li Y, Zhang W, Du M. User-Independent EMG Gesture Recognition Method Based on Adaptive Learning. Front Neurosci 2022; 16:847180. [PMID: 35431778 PMCID: PMC9008251 DOI: 10.3389/fnins.2022.847180] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2022] [Accepted: 01/13/2022] [Indexed: 11/13/2022] Open
Abstract
In a gesture recognition system based on surface electromyogram (sEMG) signals, the recognition model established by existing users cannot directly generalize to the across-user scenarios due to the individual variability of sEMG signals. In this article, we propose an adaptive learning method to handle the problem. The muscle synergy is chosen as the feature vector because it can well-characterize the neural origin of movement. The initial train set is composed of representative samples extracted from the synergy matrix of the existing user. When the new users use the system, the label is obtained by the adaptive K nearest neighbor algorithm (KNN). The recognition process does not require the pre-experiment for new users due to the introduction of adaptive learning strategy, namely, the qualified data and the label of new user data evaluated by a risk evaluator are used to update the train set and KNN weights, so as to adapt to the new users. We have tested the algorithm in DB1 and DB5 of Ninapro databases. The average recognition accuracy is 68.04, 73.35, and 83.05% for different types of gestures, respectively, achieving the effects of the user-dependent method. Our study can avoid the re-training steps and the recognition performance will improve with the increased frequency of uses, which will further facilitate the widespread implementation of sEMG control systems using pattern recognition techniques.
Collapse
Affiliation(s)
- Nan Zheng
- College of Electrical Engineering and Automation, Fuzhou University, Fuzhou, China
- Fujian Provincial Key Laboratory of Medical Instrument and Pharmaceutical Technology, Fuzhou, China
| | - Yurong Li
- College of Electrical Engineering and Automation, Fuzhou University, Fuzhou, China
- Fujian Provincial Key Laboratory of Medical Instrument and Pharmaceutical Technology, Fuzhou, China
- *Correspondence: Yurong Li
| | - Wenxuan Zhang
- College of Electrical Engineering and Automation, Fuzhou University, Fuzhou, China
- Fujian Provincial Key Laboratory of Medical Instrument and Pharmaceutical Technology, Fuzhou, China
| | - Min Du
- Fujian Provincial Key Laboratory of Eco-Industrial Green Technology, Wuyishan University, Wuyishan, China
| |
Collapse
|
30
|
Toward Improving the Reliability of Discrete Movement Recognition of sEMG Signals. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12073374] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/10/2022]
Abstract
Currently, the classification accuracy of surface electromyography (sEMG) signals is high in literature, but the conventional recognition system may classify untrained movements or the trained movements of low reliability to one of its target classes by mistake. If such a system is used for prosthetic control, sometimes it may cause a disaster. A two-layer classifier that fuses the Gaussian mixture model (GMM) and k-nearest neighbor (kNN) in a sequential structure is proposed in this study. The proposed algorithm can reject the trained movements with low reliability and is efficient in rejecting the untrained movements, thus enhancing the reliability of the myoelectric control system. The results show that the proposed algorithm can produce 95.7% active accuracy in recognizing 12 trained movements and a 30.3% error rate for rejecting 12 untrained movements. When the movement number is six, the active accuracy for trained movements can reach 99.2%, and the error rate of untrained movement is only 17.4%, which is much better than previous studies. Therefore, the proposed classifier can accurately recognize the trained movements and reject untrained movement patterns effectively.
Collapse
|
31
|
Millar C, Siddique N, Kerr E. LSTM Network Classification of Dexterous Individual Finger Movements. JOURNAL OF ADVANCED COMPUTATIONAL INTELLIGENCE AND INTELLIGENT INFORMATICS 2022. [DOI: 10.20965/jaciii.2022.p0113] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Electrical activity is generated in the forearm muscles during muscular contractions that control dexterous movements of a human finger and thumb. Using this electrical activity as an input to train a neural network for the purposes of classifying finger movements is not straightforward. Low cost wearable sensors i.e., a Myo Gesture control armband (www.bynorth.com), generally have a lower sampling rate when compared with medical grade EMG detection systems e.g., 200 Hz vs 2000 Hz. Using sensors such as the Myo coupled with the lower amplitude generated by individual finger movements makes it difficult to achieve high classification accuracy. Low sampling rate makes it challenging to distinguish between large quantities of subtle finger movements when using a single network. This research uses two networks which enables for the reduction in the number of movements in each network that are being classified; in turn improving the classification. This is achieved by developing and training LSTM networks that focus on the extension and flexion signals of the fingers and a separate network that is trained using thumb movement signal data. By following this method, this research have increased classification of the individual finger movements to between 90 and 100%.
Collapse
|
32
|
Ghaderi P, Nosouhi M, Jordanic M, Marateb HR, Mañanas MA, Farina D. Kernel Density Estimation of Electromyographic Signals and Ensemble Learning for Highly Accurate Classification of a Large Set of Hand/Wrist Motions. Front Neurosci 2022; 16:796711. [PMID: 35356057 PMCID: PMC8959430 DOI: 10.3389/fnins.2022.796711] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2021] [Accepted: 02/01/2022] [Indexed: 11/13/2022] Open
Abstract
The performance of myoelectric control highly depends on the features extracted from surface electromyographic (sEMG) signals. We propose three new sEMG features based on the kernel density estimation. The trimmed mean of density (TMD), the entropy of density, and the trimmed mean absolute value of derivative density were computed for each sEMG channel. These features were tested for the classification of single tasks as well as of two tasks concurrently performed. For single tasks, correlation-based feature selection was used, and the features were then classified using linear discriminant analysis (LDA), non-linear support vector machines, and multi-layer perceptron. The eXtreme gradient boosting (XGBoost) classifier was used for the classification of two movements simultaneously performed. The second and third versions of the Ninapro dataset (conventional control) and Ameri's movement dataset (simultaneous control) were used to test the proposed features. For the Ninapro dataset, the overall accuracy of LDA using the TMD feature was 98.99 ± 1.36% and 92.25 ± 9.48% for able-bodied and amputee subjects, respectively. Using ensemble learning of the three classifiers, the average macro and micro-F-score, macro recall, and precision on the validation sets were 98.23 ± 2.02, 98.32 ± 1.93, 98.32 ± 1.93, and 98.88 ± 1.31%, respectively, for the intact subjects. The movement misclassification percentage was 1.75 ± 1.73 and 3.44 ± 2.23 for the intact subjects and amputees. The proposed features were significantly correlated with the movement classes [Generalized Linear Model (GLM); P-value < 0.05]. An accurate online implementation of the proposed algorithm was also presented. For the simultaneous control, the overall accuracy was 99.71 ± 0.08 and 97.85 ± 0.10 for the XGBoost and LDA classifiers, respectively. The proposed features are thus promising for conventional and simultaneous myoelectric control.
Collapse
Affiliation(s)
- Parviz Ghaderi
- The Biomedical Engineering Department, Engineering Faculty, University of Isfahan, Isfahan, Iran
- Laboratory of Sensory Processing, Brain Mind Institute, Faculty of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Marjan Nosouhi
- The Biomedical Engineering Department, Engineering Faculty, University of Isfahan, Isfahan, Iran
| | - Mislav Jordanic
- Biomedical Engineering Research Centre (CREB), Automatic Control Department (ESAII), Universitat Politècnica de Catalunya-Barcelona Tech (UPC), Barcelona, Spain
| | - Hamid Reza Marateb
- The Biomedical Engineering Department, Engineering Faculty, University of Isfahan, Isfahan, Iran
- Biomedical Engineering Research Centre (CREB), Automatic Control Department (ESAII), Universitat Politècnica de Catalunya-Barcelona Tech (UPC), Barcelona, Spain
| | - Miguel Angel Mañanas
- Biomedical Engineering Research Centre (CREB), Automatic Control Department (ESAII), Universitat Politècnica de Catalunya-Barcelona Tech (UPC), Barcelona, Spain
- CIBER de Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Madrid, Spain
| | - Dario Farina
- Department of Bioengineering, Imperial College London, London, United Kingdom
| |
Collapse
|
33
|
Xu P, Li F, Wang H. A novel concatenate feature fusion RCNN architecture for sEMG-based hand gesture recognition. PLoS One 2022; 17:e0262810. [PMID: 35051235 PMCID: PMC8775254 DOI: 10.1371/journal.pone.0262810] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2021] [Accepted: 01/06/2022] [Indexed: 11/18/2022] Open
Abstract
Hand gesture recognition tasks based on surface electromyography (sEMG) are vital in human-computer interaction, speech detection, robot control, and rehabilitation applications. However, existing models, whether traditional machine learnings (ML) or other state-of-the-arts, are limited in the number of movements. Targeting a large number of gesture classes, more data features such as temporal information should be persisted as much as possible. In the field of sEMG-based recognitions, the recurrent convolutional neural network (RCNN) is an advanced method due to the sequential characteristic of sEMG signals. However, the invariance of the pooling layer damages important temporal information. In the all convolutional neural network (ACNN), because of the feature-mixing convolution operation, a same output can be received from completely different inputs. This paper proposes a concatenate feature fusion (CFF) strategy and a novel concatenate feature fusion recurrent convolutional neural network (CFF-RCNN). In CFF-RCNN, a max-pooling layer and a 2-stride convolutional layer are concatenated together to replace the conventional simple dimensionality reduction layer. The featurewise pooling operation serves as a signal amplitude detector without using any parameter. The feature-mixing convolution operation calculates the contextual information. Complete evaluations are made on both the accuracy and convergence speed of the CFF-RCNN. Experiments are conducted using three sEMG benchmark databases named DB1, DB2 and DB4 from the NinaPro database. With more than 50 gestures, the classification accuracies of the CFF-RCNN are 88.87% on DB1, 99.51% on DB2, and 99.29% on DB4. These accuracies are the highest compared with reported accuracies of machine learnings and other state-of-the-art methods. To achieve accuracies of 86%, 99% and 98% for the RCNN, the training time are 2353.686 s, 816.173 s and 731.771 s, respectively. However, for the CFF-RCNN to reach the same accuracies, it needs only 1727.415 s, 542.245 s and 576.734 s, corresponding to a reduction of 26.61%, 33.56% and 21.19% in training time. We concluded that the CFF-RCNN is an improved method when classifying a large number of hand gestures. The CFF strategy significantly improved model performance with higher accuracy and faster convergence as compared to traditional RCNN.
Collapse
Affiliation(s)
- Pufan Xu
- School of Electronic Science and Engineering, Southeast University, Nanjing, Jiangsu, China
| | - Fei Li
- Institute of RF- & OE-ICs, Southeast University, Nanjing, Jiangsu, China
| | - Haipeng Wang
- School of Electronic and Information Engineering, Sanjiang University, Nanjing, Jiangsu, China
| |
Collapse
|
34
|
Dinashi K, Ameri A, Akhaee MA, Englehart K, Scheme E. Compression of EMG Signals Using Deep Convolutional Autoencoders. IEEE J Biomed Health Inform 2022; 26:2888-2897. [PMID: 35015656 DOI: 10.1109/jbhi.2022.3142034] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
Efficient storage and transmission of electromyogram (EMG) data are important for emerging applications such as telemedicine and big data, as a vital tool for further advancement of the field. However, due to limitations in internet speed and hardware resources, transmission and storage of EMG data are challenging. As a solution, this work proposes a new method for EMG data compression using deep convolutional autoencoders (CAE). Eight-channel EMG data from 10 subjects, and high density EMG data from 18 subjects, were investigated for compression. The CAE architecture was designed to extract an abstract data representation that is heavily compressed, but from which the salient information for classification can be effectively reconstructed. The proposed method attained efficient compression; for CR=1600, the average PRDN (percentage RMS difference normalized) was 31.5% and the wrist motions classification accuracy (CA) reduced roughly 5%. The CAE substantially outperformed the state-of-the-art high-efficiency video coding and a well known wavelet-thresholding compression technique. Moreover, by reducing the bit-resolution of the CAEs compressed data from 24 bits to 6 bits, an additional 4-fold compression was achieved without significant degradation of the reconstruction performance. Furthermore, the CAEs inter subject performance was promising; e.g. for CR=1600, the PRDN for the inter-subject case was only 2.6% less than that of the within-subject performance. The powerful EMG compression performance with remarkable reconstruction results reflects the CAEs potential as an automatic end to end approach with the ability to learn the complete encoding and decoding process. Furthermore, the excellent inter-subject performance demonstrates the generalizability and usability of the proposed approach.
Collapse
|
35
|
|
36
|
Jabbari M, Khushaba R, Nazarpour K. Spatio-temporal warping for myoelectric control: an offline, feasibility study. J Neural Eng 2021; 18. [PMID: 34757954 DOI: 10.1088/1741-2552/ac387f] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2021] [Accepted: 11/10/2021] [Indexed: 11/12/2022]
Abstract
Objective.The efficacy of an adopted feature extraction method directly affects the classification of the electromyographic (EMG) signals in myoelectric control applications. Most methods attempt to extract the dynamics of the multi-channel EMG signals in the time domain and on a channel-by-channel, or at best pairs of channels, basis. However, considering multi-channel information to build a similarity matrix has not been taken into account.Approach.Combining methods of long and short-term memory (LSTM) and dynamic temporal warping, we developed a new feature, called spatio-temporal warping (STW), for myoelectric signals. This method captures the spatio-temporal relationships of multi-channels EMG signals.Main results. Across four online databases, we show that in terms of average classification error and standard deviation values, the STW feature outperforms traditional features by 5%-17%. In comparison to the more recent deep learning models, e.g. convolutional neural networks (CNNs), STW outperformed by 5%-18%. Also, STW showed enhanced performance when compared to the CNN + LSTM model by 2%-14%. All differences were statistically significant with a large effect size.Significance.This feasibility study provides evidence supporting the hypothesis that the STW feature of the EMG signals can enhance the classification accuracy in an explainable way when compared to recent deep learning methods. Future work includes real-time implementation of the method and testing for prosthesis control.
Collapse
Affiliation(s)
- Milad Jabbari
- Edinburgh Neuroprosthetics Laboratory, School of Informatics, The University of Edinburgh, Edinburgh EH8 9AB, United Kingdom
| | - Rami Khushaba
- Australian Centre for Field Robotics, the University of Sydney, 8 Little Queen Street, Chippendale, NSW 2008, Australia
| | - Kianoush Nazarpour
- Edinburgh Neuroprosthetics Laboratory, School of Informatics, The University of Edinburgh, Edinburgh EH8 9AB, United Kingdom
| |
Collapse
|
37
|
Subhashini N, Kandaswamy A. Gesture Classification of Surface Electromyography Signals Using Machine Learning Algorithms for Hand Prosthetics. JOURNAL OF MEDICAL IMAGING AND HEALTH INFORMATICS 2021. [DOI: 10.1166/jmihi.2021.3907] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Abstract
The actions of humans executed by their hands play a remarkable part in controlling and handling variety of objects in their daily life activities. The effect of losing or degradation in the functioning of one hand has a greater influence in bringing down the regular activity. Hence
the design of prosthetic hands which assists the individuals to enhance their regular activity seems a better remedy in this new era. This paper puts forward a classification framework using machine learning algorithms for classifying hand gesture signals. The surface electromyography (sEMG)
dataset acquired for 9 wrist movements of publicly available database are utilized to identify the potential biomarkers for classification and in evaluating the efficacy of the proposed algorithm. The statistical and time domain features of the sEMG signals from 27 intact subjects and 11 trans-radial
amputated subjects are extracted and the optimal features are determined implementing the feature selection approach based on correlation factor. The classifiers performance of machine learning algorithms namely support vector machine (SVM), Naïve bayes (NB) and Ensemble classifier are
evaluated. The experimental results highlight that the SVM classifier can yield the maximum accuracy movement classification of 99.6% for intact and 97.56% for trans-amputee subjects. The proposed approach offers better accuracy and sensitivity compared to other approaches that have used the
sEMG dataset for movement classification.
Collapse
Affiliation(s)
- N. Subhashini
- The Department of Biomedical Engineering, PSG College of Technology, Coimbatore 641004, Tamil Nadu, India
| | - A. Kandaswamy
- The Department of Biomedical Engineering, PSG College of Technology, Coimbatore 641004, Tamil Nadu, India
| |
Collapse
|
38
|
Spieker V, Ganguly A, Haddadin S, Piazza C. An Adaptive Multi-Modal Control Strategy to Attenuate the Limb Position Effect in Myoelectric Pattern Recognition. SENSORS (BASEL, SWITZERLAND) 2021; 21:7404. [PMID: 34770709 PMCID: PMC8587119 DOI: 10.3390/s21217404] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/01/2021] [Revised: 10/29/2021] [Accepted: 11/01/2021] [Indexed: 11/16/2022]
Abstract
Over the last few decades, pattern recognition algorithms have shown promising results in the field of upper limb prostheses myoelectric control and are now gradually being incorporated in commercial devices. A widely used approach is based on a classifier which assigns a specific input value to a selected hand motion. While this method guarantees good performance and robustness within each class, it still shows limitations in adapting to different conditions encountered in real-world applications, such as changes in limb position or external loads. This paper proposes an adaptive method based on a pattern recognition classifier that takes advantage of an augmented dataset-i.e., representing variations in limb position or external loads-to selectively adapt to underrepresented variations. The proposed method was evaluated using a series of target achievement control tests with ten able-bodied volunteers. Results indicated a higher median completion rate >3.33% for the adapted algorithm compared to a classical pattern recognition classifier used as a baseline model. Subject-specific performance showed the potential for improved control after adaptation and a ≤13% completion rate; and in many instances, the adapted points were able to provide new information within classes. These preliminary results show the potential of the proposed method and encourage further development.
Collapse
Affiliation(s)
- Veronika Spieker
- Munich Institute of Robotics and Machine Intelligence, Technical University of Munich, 80797 Munich, Germany; (V.S.); (S.H.); (C.P.)
| | - Amartya Ganguly
- Munich Institute of Robotics and Machine Intelligence, Technical University of Munich, 80797 Munich, Germany; (V.S.); (S.H.); (C.P.)
| | - Sami Haddadin
- Munich Institute of Robotics and Machine Intelligence, Technical University of Munich, 80797 Munich, Germany; (V.S.); (S.H.); (C.P.)
| | - Cristina Piazza
- Munich Institute of Robotics and Machine Intelligence, Technical University of Munich, 80797 Munich, Germany; (V.S.); (S.H.); (C.P.)
- Department of Informatics, Technical University of Munich, 85748 Garching bei München, Germany
| |
Collapse
|
39
|
A Noble Classification Framework for Data Glove Classification of a Large Number of Hand Movements. JOURNAL OF ELECTRICAL AND COMPUTER ENGINEERING 2021. [DOI: 10.1155/2021/9472053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
The recognition of hand movements is an important method for human-computer interaction (HCI) technology, and it is widely used in virtual reality and other HCI areas. While many valuable efforts have been made, efficient ways to capture over 20 types of hand movements with high accuracy by one data glove are still lacking. This paper addresses a new classification framework for 52 hand movements. This classification framework includes the following two parts: the movement detection algorithm and the movement classification algorithm. The fine K-nearest neighbor (Fine KNN) is the core of the movement detection algorithm. The movement classification algorithm is composed of downsampling in data preparation and a new deep learning network named the DBDF network. Bidirectional Long Short-Term Memory (BiLSTM) is the main part of the DBDF network. The results of experiments using the Ninapro DB1 dataset demonstrate that our work can classify more types of hand movements than related algorithms with a precision of 93.15%.
Collapse
|
40
|
A Novel EMG-Based Hand Gesture Recognition Framework Based on Multivariate Variational Mode Decomposition. SENSORS 2021; 21:s21217002. [PMID: 34770309 PMCID: PMC8588500 DOI: 10.3390/s21217002] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/02/2021] [Revised: 10/06/2021] [Accepted: 10/12/2021] [Indexed: 11/16/2022]
Abstract
Surface electromyography (sEMG) is a kind of biological signal that records muscle activity noninvasively, which is of great significance in advanced human-computer interaction, prosthetic control, clinical therapy, and biomechanics. However, the number of hand gestures that can be recognized is limited and the recognition accuracy needs to be further improved. These factors lead to the fact that sEMG products are not widely used in practice. The main contributions of this paper are as follows. Firstly, considering the increasing number of gestures to be recognized and the complexity of gestures, an extensible two-stage machine learning lightweight framework was innovatively proposed for multi-gesture task recognition. Secondly, the multivariate variational mode decomposition (MVMD) is applied to extract the spatial–temporal features from the multiple channels to the EMG signals, and the separable convolutional neural network is used for modelling. In this work, the experimental results for 52 hand gestures recognition task show that the average accuracy on each stage is about 90%. The potential movement information is mainly contained in the low-frequency oscillator of the sEMG signal, and the model performs better with the low-frequency oscillation from the MVMD algorithm on the second stage classification than that of other decomposition methods.
Collapse
|
41
|
Luan Y, Shi Y, Wu W, Liu Z, Chang H, Cheng J. HAR-sEMG: A Dataset for Human Activity Recognition on Lower-Limb sEMG. Knowl Inf Syst 2021. [DOI: 10.1007/s10115-021-01598-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
42
|
Mao H, Fang P, Li G. Simultaneous estimation of multi-finger forces by surface electromyography and accelerometry signals. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.103005] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
43
|
Buongiorno D, Cascarano GD, De Feudis I, Brunetti A, Carnimeo L, Dimauro G, Bevilacqua V. Deep learning for processing electromyographic signals: A taxonomy-based survey. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2020.06.139] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
44
|
Bittibssi TM, Zekry AH, Genedy MA, Maged SA. sEMG pattern recognition based on recurrent neural network. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.103048] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
45
|
A Real-Time Capable Linear Time Classifier Scheme for Anticipated Hand Movements Recognition from Amputee Subjects Using Surface EMG Signals. Ing Rech Biomed 2021. [DOI: 10.1016/j.irbm.2020.08.003] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
46
|
Wen R, Wang Q, Li Z. Human hand movement recognition using infinite hidden Markov model based sEMG classification. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102592] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
47
|
Rahimian E, Zabihi S, Asif A, Farina D, Atashzar SF, Mohammadi A. FS-HGR: Few-Shot Learning for Hand Gesture Recognition via Electromyography. IEEE Trans Neural Syst Rehabil Eng 2021; 29:1004-1015. [PMID: 33945480 DOI: 10.1109/tnsre.2021.3077413] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
This work is motivated by the recent advances in Deep Neural Networks (DNNs) and their widespread applications in human-machine interfaces. DNNs have been recently used for detecting the intended hand gesture through the processing of surface electromyogram (sEMG) signals. Objective: Although DNNs have shown superior accuracy compared to conventional methods when large amounts of data are available for training, their performance substantially decreases when data are limited. Collecting large datasets for training may be feasible in research laboratories, but it is not a practical approach for real-life applications. The main objective of this work is to design a modern DNN-based gesture detection model that relies on minimal training data while providing high accuracy. Methods: We propose the novel Few-Shot learning- Hand Gesture Recognition (FS-HGR) architecture. Few-shot learning is a variant of domain adaptation with the goal of inferring the required output based on just one or a few training observations. The proposed FS-HGR generalizes after seeing very few observations from each class by combining temporal convolutions with attention mechanisms. This allows the meta-learner to aggregate contextual information from experience and to pinpoint specific pieces of information within its available set of inputs. Data Source & Summary of Results: The performance of FS-HGR was tested on the second and fifth Ninapro databases, referred to as the DB2 and DB5, respectively. The DB2 consists of 50 gestures (rest included) from 40 healthy subjects. The Ninapro DB5 contains data from 10 healthy participants performing a total of 53 different gestures (rest included). The proposed approach for the Ninapro DB2 led to 85.94% classification accuracy on new repetitions with few-shot observation (5-way 5-shot), 81.29% accuracy on new subjects with few-shot observation (5-way 5-shot), and 73.36% accuracy on new gestures with few-shot observation (5-way 5-shot). Moreover, the proposed approach for the Ninapro DB5 led to 64.65% classification accuracy on new subjects with few-shot observation (5-way 5-shot).
Collapse
|
48
|
Ebied A, Kinney-Lang E, Escudero J. Higher order tensor decomposition for proportional myoelectric control based on muscle synergies. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102523] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
49
|
Li C, Yang H, Cheng L. Fugl-Meyer hand motor imagination recognition for brain–computer interfaces using only fNIRS. COMPLEX INTELL SYST 2021. [DOI: 10.1007/s40747-020-00266-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Abstract
AbstractAs a relatively new physiological signal of brain, functional near-infrared spectroscopy (fNIRS) is being used more and more in brain–computer interface field, especially in the task of motor imagery. However, the classification accuracy based on this signal is relatively low. To improve the accuracy of classification, this paper proposes a new experimental paradigm and only uses fNIRS signals to complete the classification task of six subjects. Notably, the experiment is carried out in a non-laboratory environment, and movements of motion imagination are properly designed. And when the subjects are imagining the motions, they are also subvocalizing the movements to prevent distraction. Therefore, according to the motor area theory of the cerebral cortex, the positions of the fNIRS probes have been slightly adjusted compared with other methods. Next, the signals are classified by nine classification methods, and the different features and classification methods are compared. The results show that under this new experimental paradigm, the classification accuracy of 89.12% and 88.47% can be achieved using the support vector machine method and the random forest method, respectively, which shows that the paradigm is effective. Finally, by selecting five channels with the largest variance after empirical mode decomposition of the original signal, similar classification results can be achieved.
Collapse
|
50
|
Gohel V, Mehendale N. Review on electromyography signal acquisition and processing. Biophys Rev 2020; 12:10.1007/s12551-020-00770-w. [PMID: 33169207 PMCID: PMC7755956 DOI: 10.1007/s12551-020-00770-w] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2020] [Accepted: 10/26/2020] [Indexed: 12/01/2022] Open
Abstract
Electromyography (EMG) is a technique for recording biomedical electrical signals obtained from the neuromuscular activities. These signals are used to monitor medical abnormalities and activation levels, and also to analyze the biomechanics of any animal movements. In this article, we provide a short review of EMG signal acquisition and processing techniques. The average efficiency of capture of EMG signals with current technologies is around 70%. Once the signal is captured, signal processing algorithms then determine the recognition accuracy, with which signals are decoded for their corresponding purpose (e.g., moving robotic arm, speech recognition, gait analysis). The recognition accuracy can go as high as 99.8%. The accuracy with which the EMG signal is decoded has already crossed 99%, and with improvements in deep learning technology, there is a large scope for improvement in the design hardware that can efficiently capture EMG signals.
Collapse
Affiliation(s)
- Vidhi Gohel
- K. J. Somaiya College of Engineering, Mumbai, India
| | | |
Collapse
|