1
|
M E, Hans WJ, T M I, Lindsay NM. Multi-scale EMG classification with spatial-temporal attention for prosthetic hands. Comput Methods Biomech Biomed Engin 2023:1-16. [PMID: 38037332 DOI: 10.1080/10255842.2023.2287419] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Accepted: 11/20/2023] [Indexed: 12/02/2023]
Abstract
A classification framework for hand gestures using Electromyography (EMG) signals in prosthetic hands is presented. Leveraging the multi-scale characteristics and temporal nature of EMG signals, a Convolutional Neural Network (CNN) is used to extract multi-scale features and classify them with spatial-temporal attention. A multi-scale coarse-grained layer introduced into the input of one-dimensional CNN (1D-CNN) facilitates multi-scale feature extraction. The multi-scale features are fed into the attention layer and subsequently given to the fully connected layer to perform classification. The proposed model achieves classification accuracies of 93.4%, 92.8%, 91.3%, and 94.1% for Ninapro DB1, DB2, DB5, and DB7 respectively, thereby enhancing the confidence of prosthetic hand users.
Collapse
Affiliation(s)
- Emimal M
- Department of Electronics and Communication Engineering, Sri Sivasubramaniya Nadar College of Engineering, Kalavakkam, Chennai, India
| | - W Jino Hans
- Department of Electronics and Communication Engineering, Sri Sivasubramaniya Nadar College of Engineering, Kalavakkam, Chennai, India
| | - Inbamalar T M
- Department of Electronics and Communication Engineering, RMK College of Engineering and Technology, Puduvoyal, Chennai, India
| | - N Mahiban Lindsay
- Department of Electrical and Electronics Engineering, Hindustan Institute of Technology and Science, Padur, Chennai, India
| |
Collapse
|
2
|
Xiong B, Chen W, Niu Y, Gan Z, Mao G, Xu Y. A Global and Local Feature fused CNN architecture for the sEMG-based hand gesture recognition. Comput Biol Med 2023; 166:107497. [PMID: 37783073 DOI: 10.1016/j.compbiomed.2023.107497] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Revised: 07/22/2023] [Accepted: 09/15/2023] [Indexed: 10/04/2023]
Abstract
Deep learning methods have been widely used for the classification of hand gestures using sEMG signals. Existing deep learning architectures only captures local spatial information and has limitations in extracting global temporal dependency to enhance the model's performance. In this paper, we propose a Global and Local Feature fused CNN (GLF-CNN) model that extracts features both globally and locally from sEMG signals to enhance the performance of hand gestures classification. The model contains two independent branches extracting local and global features each and fuses them to learn more diversified features and effectively improve the stability of gesture recognition. Besides, it also exhibits lower computational cost compared to the present approaches. We conduct experiments on five benchmark databases, including the NinaPro DB4, NinaPro DB5, BioPatRec DB1-DB3, and the Mendeley Data. The proposed model achieved the highest average accuracy of 88.34% on these databases, with a 9.96% average accuracy improvement and a 50% reduction in variance compared to the models with the same number of parameters. Moreover, the classification accuracies for the BioPatRec DB1, BioPatRec DB3 and Mendeley Data are 91.4%, 91.0% and 88.6% respectively, corresponding to an improvement of 13.2%, 41.5% and 12.2% over the respective state-of-the-art models. The experimental results demonstrate that the proposed model effectively enhances robustness, with improved gesture recognition performance and generalization ability. It contributes a new way for prosthetic control and human-machine interaction.
Collapse
Affiliation(s)
- Baoping Xiong
- Computer Science and Mathematics, Fujian University of Technology, Fujian 350116, China
| | - Wensheng Chen
- Computer Science and Mathematics, Fujian University of Technology, Fujian 350116, China
| | - Yinxi Niu
- Computer Science and Mathematics, Fujian University of Technology, Fujian 350116, China
| | - Zhenhua Gan
- Computer Science and Mathematics, Fujian University of Technology, Fujian 350116, China
| | - Guojun Mao
- Computer Science and Mathematics, Fujian University of Technology, Fujian 350116, China
| | - Yong Xu
- Computer Science and Mathematics, Fujian University of Technology, Fujian 350116, China.
| |
Collapse
|
3
|
Wang X, Tang L, Zheng Q, Yang X, Lu Z. IRDC-Net: An Inception Network with a Residual Module and Dilated Convolution for Sign Language Recognition Based on Surface Electromyography. SENSORS (BASEL, SWITZERLAND) 2023; 23:5775. [PMID: 37447625 DOI: 10.3390/s23135775] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/18/2023] [Revised: 06/06/2023] [Accepted: 06/15/2023] [Indexed: 07/15/2023]
Abstract
Deaf and hearing-impaired people always face communication barriers. Non-invasive surface electromyography (sEMG) sensor-based sign language recognition (SLR) technology can help them to better integrate into social life. Since the traditional tandem convolutional neural network (CNN) structure used in most CNN-based studies inadequately captures the features of the input data, we propose a novel inception architecture with a residual module and dilated convolution (IRDC-net) to enlarge the receptive fields and enrich the feature maps, applying it to SLR tasks for the first time. This work first transformed the time domain signal into a time-frequency domain using discrete Fourier transformation. Second, an IRDC-net was constructed to recognize ten Chinese sign language signs. Third, the tandem CNN networks VGG-net and ResNet-18 were compared with our proposed parallel structure network, IRDC-net. Finally, the public dataset Ninapro DB1 was utilized to verify the generalization performance of the IRDC-net. The results showed that after transforming the time domain sEMG signal into the time-frequency domain, the classification accuracy (acc) increased from 84.29% to 91.70% when using the IRDC-net on our sign language dataset. Furthermore, for the time-frequency information of the public dataset Ninapro DB1, the classification accuracy reached 89.82%; this value is higher than that achieved in other recent studies. As such, our findings contribute to research into SLR tasks and to improving deaf and hearing-impaired people's daily lives.
Collapse
Affiliation(s)
- Xiangrui Wang
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China
| | - Lu Tang
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China
| | - Qibin Zheng
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China
| | - Xilin Yang
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China
| | - Zhiyuan Lu
- School of Rehabilitation Science and Engineering, University of Health and Rehabilitation Sciences, Qingdao 266072, China
| |
Collapse
|
4
|
Hu X, Song A, Wang J, Zeng H, Wei W. Finger Movement Recognition via High-Density Electromyography of Intrinsic and Extrinsic Hand Muscles. Sci Data 2022; 9:373. [PMID: 35768439 PMCID: PMC9243097 DOI: 10.1038/s41597-022-01484-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2021] [Accepted: 06/15/2022] [Indexed: 11/09/2022] Open
Abstract
Surface electromyography (sEMG) is commonly used to observe the motor neuronal activity within muscle fibers. However, decoding dexterous body movements from sEMG signals is still quite challenging. In this paper, we present a high-density sEMG (HD-sEMG) signal database that comprises simultaneously recorded sEMG signals of intrinsic and extrinsic hand muscles. Specifically, twenty able-bodied participants performed 12 finger movements under two paces and three arm postures. HD-sEMG signals were recorded with a 64-channel high-density grid placed on the back of hand and an 8-channel armband around the forearm. Also, a data-glove was used to record the finger joint angles. Synchronisation and reproducibility of the data collection from the HD-sEMG and glove sensors were ensured. The collected data samples were further employed for automated recognition of dexterous finger movements. The introduced dataset offers a new perspective to study the synergy between the intrinsic and extrinsic hand muscles during dynamic finger movements. As this dataset was collected from multiple participants, it also provides a resource for exploring generalized models for finger movement decoding.
Collapse
Affiliation(s)
- Xuhui Hu
- State Key Laboratory of Bioelectronics, Nanjing, China.,Jiangsu Key Laboratory of Remote Measurement and Control, Nanjing, China.,School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Aiguo Song
- State Key Laboratory of Bioelectronics, Nanjing, China. .,Jiangsu Key Laboratory of Remote Measurement and Control, Nanjing, China. .,School of Instrument Science and Engineering, Southeast University, Nanjing, China.
| | - Jianzhi Wang
- State Key Laboratory of Bioelectronics, Nanjing, China.,Jiangsu Key Laboratory of Remote Measurement and Control, Nanjing, China.,School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Hong Zeng
- State Key Laboratory of Bioelectronics, Nanjing, China.,Jiangsu Key Laboratory of Remote Measurement and Control, Nanjing, China.,School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Wentao Wei
- School of Design Arts and Media, Nanjing University of Science and Technology, Nanjing, China
| |
Collapse
|
5
|
Myoelectric Pattern Recognition Performance Enhancement Using Nonlinear Features. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:6414664. [PMID: 35528339 PMCID: PMC9076314 DOI: 10.1155/2022/6414664] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/08/2022] [Revised: 04/07/2022] [Accepted: 04/11/2022] [Indexed: 11/17/2022]
Abstract
The multichannel electrode array used for electromyogram (EMG) pattern recognition provides good performance, but it has a high cost, is computationally expensive, and is inconvenient to wear. Therefore, researchers try to use as few channels as possible while maintaining improved pattern recognition performance. However, minimizing the number of channels affects the performance due to the least separable margin among the movements possessing weak signal strengths. To meet these challenges, two time-domain features based on nonlinear scaling, the log of the mean absolute value (LMAV) and the nonlinear scaled value (NSV), are proposed. In this study, we validate the proposed features on two datasets, the existing four feature extraction methods, variable window size, and various signal-to-noise ratios (SNR). In addition, we also propose a feature extraction method where the LMAV and NSV are grouped with the existing 11 time-domain features. The proposed feature extraction method enhances accuracy, sensitivity, specificity, precision, and F1 score by 1.00%, 5.01%, 0.55%, 4.71%, and 5.06% for dataset 1, and 1.18%, 5.90%, 0.66%, 5.63%, and 6.04% for dataset 2, respectively. Therefore, the experimental results strongly suggest the proposed feature extraction method, for taking a step forward with regard to improved myoelectric pattern recognition performance.
Collapse
|