1
|
Chen X, Yang H, Zhang D, Hu X, Xie P. Hand Gesture Recognition Based on High-Density Myoelectricity in Forearm Flexors in Humans. SENSORS (BASEL, SWITZERLAND) 2024; 24:3970. [PMID: 38931754 PMCID: PMC11207234 DOI: 10.3390/s24123970] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/06/2024] [Revised: 06/16/2024] [Accepted: 06/17/2024] [Indexed: 06/28/2024]
Abstract
Electromyography-based gesture recognition has become a challenging problem in the decoding of fine hand movements. Recent research has focused on improving the accuracy of gesture recognition by increasing the complexity of network models. However, training a complex model necessitates a significant amount of data, thereby escalating both user burden and computational costs. Moreover, owing to the considerable variability of surface electromyography (sEMG) signals across different users, conventional machine learning approaches reliant on a single feature fail to meet the demand for precise gesture recognition tailored to individual users. Therefore, to solve the problems of large computational cost and poor cross-user pattern recognition performance, we propose a feature selection method that combines mutual information, principal component analysis and the Pearson correlation coefficient (MPP). This method can filter out the optimal subset of features that match a specific user while combining with an SVM classifier to accurately and efficiently recognize the user's gesture movements. To validate the effectiveness of the above method, we designed an experiment including five gesture actions. The experimental results show that compared to the classification accuracy obtained using a single feature, we achieved an improvement of about 5% with the optimally selected feature as the input to any of the classifiers. This study provides an effective guarantee for user-specific fine hand movement decoding based on sEMG signals.
Collapse
Affiliation(s)
- Xiaoling Chen
- Institute of Electric Engineering, Yanshan University, Qinhuangdao 066004, China; (X.C.); (H.Y.); (D.Z.); (X.H.)
- Key Laboratory of Measurement Technology and Instrumentation of Hebei Province, Institute of Electric Engineering, Yanshan University, Qinhuangdao 066004, China
| | - Huaigang Yang
- Institute of Electric Engineering, Yanshan University, Qinhuangdao 066004, China; (X.C.); (H.Y.); (D.Z.); (X.H.)
| | - Dong Zhang
- Institute of Electric Engineering, Yanshan University, Qinhuangdao 066004, China; (X.C.); (H.Y.); (D.Z.); (X.H.)
| | - Xinfeng Hu
- Institute of Electric Engineering, Yanshan University, Qinhuangdao 066004, China; (X.C.); (H.Y.); (D.Z.); (X.H.)
| | - Ping Xie
- Institute of Electric Engineering, Yanshan University, Qinhuangdao 066004, China; (X.C.); (H.Y.); (D.Z.); (X.H.)
- Key Laboratory of Measurement Technology and Instrumentation of Hebei Province, Institute of Electric Engineering, Yanshan University, Qinhuangdao 066004, China
| |
Collapse
|
2
|
Wang X, Ao D, Li L. Robust myoelectric pattern recognition methods for reducing users' calibration burden: challenges and future. Front Bioeng Biotechnol 2024; 12:1329209. [PMID: 38318193 PMCID: PMC10839078 DOI: 10.3389/fbioe.2024.1329209] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2023] [Accepted: 01/08/2024] [Indexed: 02/07/2024] Open
Abstract
Myoelectric pattern recognition (MPR) has evolved into a sophisticated technology widely employed in controlling myoelectric interface (MI) devices like prosthetic and orthotic robots. Current MIs not only enable multi-degree-of-freedom control of prosthetic limbs but also demonstrate substantial potential in consumer electronics. However, the non-stationary random characteristics of myoelectric signals poses challenges, leading to performance degradation in practical scenarios such as electrode shifting and switching new users. Conventional MIs often necessitate meticulous calibration, imposing a significant burden on users. To address user frustration during the calibration process, researchers have focused on identifying MPR methods that alleviate this burden. This article categorizes common scenarios that incur calibration burdens as based on data distribution shift and based on dynamic data categories. Then further investigated and summarized the popular robust MPR algorithms used to reduce the user's calibration burden. We categorize these algorithms as based on data manipulate, feature manipulation and, model structure. And describes the scenarios to which each method is applicable and the conditions required for calibration. Finally, this review is concluded with the advantages of robust MPR and the remaining challenges and future opportunities.
Collapse
Affiliation(s)
- Xiang Wang
- Institute of Medical Research, Northwestern Polytechnical University, Xi’an, China
| | - Di Ao
- Institute of Medical Research, Northwestern Polytechnical University, Xi’an, China
| | - Le Li
- Institute of Medical Research, Northwestern Polytechnical University, Xi’an, China
- Research and Development Institute of Northwestern Polytechnical University in Shenzhen, Shenzhen, China
| |
Collapse
|
3
|
Wu S, Shu L, Song Z, Xu X. SFDA: Domain Adaptation With Source Subject Fusion Based on Multi-Source and Single-Target Fall Risk Assessment. IEEE Trans Neural Syst Rehabil Eng 2023; 31:4907-4920. [PMID: 38032785 DOI: 10.1109/tnsre.2023.3337861] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2023]
Abstract
In cross-subject fall risk classification based on plantar pressure, a challenge is that data from different subjects have significant individual information. Thus, the models with insufficient generalization ability can't perform well on new subjects, which limits their application in daily life. To solve this problem, domain adaptation methods are applied to reduce the gap between source and target domain. However, these methods focus on the distribution of the source and the target domain, but ignore the potential correlation among multiple source subjects, which deteriorates domain adaptation performance. In this paper, we proposed a novel method named domain adaptation with subject fusion (SFDA) for fall risk assessment, greatly improving the cross-subject assessment ability. Specifically, SFDA synchronously carries out source target adaptation and multiple source subject fusion by domain adversarial module to reduce source-target gap and distribution distance within source subjects of same class. Consequently, target samples can learn more task-specific features from source subjects to improve the generalization ability. Experiment results show that SFDA achieved mean accuracy of 79.17 % and 73.66 % based on two backbones in a cross-subject classification manner, outperforming the state-of-the-art methods on continuous plantar pressure dataset. This study proves the effectiveness of SFDA and provides a novel tool for implementing cross-subject and few-gait fall risk assessment.
Collapse
|
4
|
Han J, Wei X, Faisal AA. EEG decoding for datasets with heterogenous electrode configurations using transfer learning graph neural networks. J Neural Eng 2023; 20:066027. [PMID: 37931308 DOI: 10.1088/1741-2552/ad09ff] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Accepted: 11/06/2023] [Indexed: 11/08/2023]
Abstract
Objective. Brain-machine interfacing (BMI) has greatly benefited from adopting machine learning methods for feature learning that require extensive data for training, which are often unavailable from a single dataset. Yet, it is difficult to combine data across labs or even data within the same lab collected over the years due to the variation in recording equipment and electrode layouts resulting in shifts in data distribution, changes in data dimensionality, and altered identity of data dimensions. Our objective is to overcome this limitation and learn from many different and diverse datasets across labs with different experimental protocols.Approach. To tackle the domain adaptation problem, we developed a novel machine learning framework combining graph neural networks (GNNs) and transfer learning methodologies for non-invasive motor imagery (MI) EEG decoding, as an example of BMI. Empirically, we focus on the challenges of learning from EEG data with different electrode layouts and varying numbers of electrodes. We utilize three MI EEG databases collected using very different numbers of EEG sensors (from 22 channels to 64) and layouts (from custom layouts to 10-20).Main results. Our model achieved the highest accuracy with lower standard deviations on the testing datasets. This indicates that the GNN-based transfer learning framework can effectively aggregate knowledge from multiple datasets with different electrode layouts, leading to improved generalization in subject-independent MI EEG classification.Significance. The findings of this study have important implications for brain-computer-interface research, as they highlight a promising method for overcoming the limitations posed by non-unified experimental setups. By enabling the integration of diverse datasets with varying electrode layouts, our proposed approach can help advance the development and application of BMI technologies.
Collapse
Affiliation(s)
- Jinpei Han
- Brain & Behaviour Lab, Department of Computing, Imperial College London, London SW7 2AZ, United Kingdom
| | - Xiaoxi Wei
- Brain & Behaviour Lab, Department of Computing, Imperial College London, London SW7 2AZ, United Kingdom
| | - A Aldo Faisal
- Brain & Behaviour Lab, Department of Computing, Imperial College London, London SW7 2AZ, United Kingdom
- Chair in Digital Health & Data Science, University of Bayreuth, 95447 Bayreuth, Germany
| |
Collapse
|
5
|
Duan S, Wu L, Liu A, Chen X. Alignment-Enhanced Interactive Fusion Model for Complete and Incomplete Multimodal Hand Gesture Recognition. IEEE Trans Neural Syst Rehabil Eng 2023; 31:4661-4671. [PMID: 37983152 DOI: 10.1109/tnsre.2023.3335101] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2023]
Abstract
Hand gesture recognition (HGR) based on surface electromyogram (sEMG) and Accelerometer (ACC) signals is increasingly attractive where fusion strategies are crucial for performance and remain challenging. Currently, neural network-based fusion methods have gained superior performance. Nevertheless, these methods typically fuse sEMG and ACC either in the early or late stages, overlooking the integration of entire cross-modal hierarchical information within each individual hidden layer, thus inducing inefficient inter-modal fusion. To this end, we propose a novel Alignment-Enhanced Interactive Fusion (AiFusion) model, which achieves effective fusion via a progressive hierarchical fusion strategy. Notably, AiFusion can flexibly perform both complete and incomplete multimodal HGR. Specifically, AiFusion contains two unimodal branches and a cascaded transformer-based multimodal fusion branch. The fusion branch is first designed to adequately characterize modality-interactive knowledge by adaptively capturing inter-modal similarity and fusing hierarchical features from all branches layer by layer. Then, the modality-interactive knowledge is aligned with that of unimodality using cross-modal supervised contrastive learning and online distillation from embedding and probability spaces respectively. These alignments further promote fusion quality and refine modality-specific representations. Finally, the recognition outcomes are set to be determined by available modalities, thus contributing to handling the incomplete multimodal HGR problem, which is frequently encountered in real-world scenarios. Experimental results on five public datasets demonstrate that AiFusion outperforms most state-of-the-art benchmarks in complete multimodal HGR. Impressively, it also surpasses the unimodal baselines in the challenging incomplete multimodal HGR. The proposed AiFusion provides a promising solution to realize effective and robust multimodal HGR-based interfaces.
Collapse
|
6
|
Zhang X, Wu L, Zhang X, Chen X, Li C, Chen X. Multi-source domain generalization and adaptation toward cross-subject myoelectric pattern recognition. J Neural Eng 2023; 20. [PMID: 36720167 DOI: 10.1088/1741-2552/acb7a0] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2022] [Accepted: 01/31/2023] [Indexed: 02/02/2023]
Abstract
Objective.Myoelectric pattern recognition (MPR) has shown satisfactory performance under ideal laboratory conditions. Nevertheless, the individual variances lead to dramatic performance degradation in cross-user MPR applications. It is crucial to enable the myoelectric interface to adapt to multiple users' surface electromyography (sEMG) distributions in practical.Approach.Domain adaptation (DA) is a promising approach to tackle cross-user challenges due to its ability to diminish the divergence between individual users' EMG distributions and escalate model generalization performance. However, existing DA methods in sEMG control are based on single-source domain adaptation (SDA). SDA solely mixes multiple training users' data as a combined source domain and attempts to align with a novel user. This simple data mixing manner ignores the sEMG distribution variations between disparate training users, leading to an insufficient variance elimination and lower performance. To this end, this paper proposes a multi-source synchronize domain adaptation framework with both DA and domain generalization (DG) capability. This multi-source framework aligns each source user and the new user in individual feature spaces, which better transfers the knowledge of existing users to the new user. Moreover, we retain the source-combined data to preserve the effectiveness of SDA. The property was further confirmed by evaluating the performance of the proposed method on data from nine subjects performing six tasks.Main results.Experiment results prove that the proposed multi-source framework achieved both positive DG and DA performance in a cross-user classification manner.Significance.This work demonstrates the usability and feasibility of the proposed multi-source framework in cross-user myoelectric control.
Collapse
Affiliation(s)
- Xuan Zhang
- School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, People's Republic of China
| | - Le Wu
- School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, People's Republic of China
| | - Xu Zhang
- School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, People's Republic of China
| | - Xiang Chen
- School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, People's Republic of China
| | - Chang Li
- Department of Biomedical Engineering, Hefei University of Technology, Hefei, 230009, People's Republic of China
| | - Xun Chen
- School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, People's Republic of China.,Institute of Dataspace, Hefei Comprehensive National Science Center, Hefei 230088, People's Republic of China
| |
Collapse
|