1
|
Li W, Zhang X, Shi P, Li S, Li P, Yu H. Across Sessions and Subjects Domain Adaptation for Building Robust Myoelectric Interface. IEEE Trans Neural Syst Rehabil Eng 2024; 32:2005-2015. [PMID: 38147425 DOI: 10.1109/tnsre.2023.3347540] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2023]
Abstract
Gesture interaction via surface electromyography (sEMG) signal is a promising approach for advanced human-computer interaction systems. However, improving the performance of the myoelectric interface is challenging due to the domain shift caused by the signal's inherent variability. To enhance the interface's robustness, we propose a novel adaptive information fusion neural network (AIFNN) framework, which could effectively reduce the effects of multiple scenarios. Specifically, domain adversarial training is established to inhibit the shared network's weights from exploiting domain-specific representation, thus allowing for the extraction of domain-invariant features. Effectively, classification loss, domain diversence loss and domain discrimination loss are employed, which improve classification performance while reduce distribution mismatches between the two domains. To simulate the application of myoelectric interface, experiments were carried out involving three scenarios (intra-session, inter-session and inter-subject scenarios). Ten non-disabled subjects were recruited to perform sixteen gestures for ten consecutive days. The experimental results indicated that the performance of AIFNN was better than two other state-of-the-art transfer learning approaches, namely fine-tuning (FT) and domain adversarial network (DANN). This study demonstrates the capability of AIFNN to maintain robustness over time and generalize across users in practical myoelectric interface implementations. These findings could serve as a foundation for future deployments.
Collapse
|
2
|
Emimal M, Hans WJ, Inbamalar TM, Lindsay NM. Classification of EMG signals with CNN features and voting ensemble classifier. Comput Methods Biomech Biomed Engin 2024:1-15. [PMID: 38317414 DOI: 10.1080/10255842.2024.2310726] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2023] [Accepted: 01/20/2024] [Indexed: 02/07/2024]
Abstract
Electromyography (EMG) signals are primarily used to control prosthetic hands. Classifying hand gestures efficiently with EMG signals presents numerous challenges. In addition to overcoming these challenges, a successful combination of feature extraction and classification approaches will improve classification accuracy. In the current work, convolutional neural network (CNN) features are used to reduce the redundancy problems associated with time and frequency domain features to improve classification accuracy. The features from the EMG signal are extracted using a CNN and are fed to the 'k' nearest neighbor (KNN) classifier with a different number of neighbors ( 1 N N , 3 N N , 5 N N , and 7 N N ) . It results in an ensemble of classifiers that are combined using a hard voting-based classifier. Based on the benchmark Ninapro DB4 database and CapgMyo database, the proposed framework obtained 91.3 % classification accuracy on CapgMyo and 89.5 % on Ninapro DB4.
Collapse
Affiliation(s)
- M Emimal
- Department of ECE, Sri Sivasubramaniya Nadar College of Engineering, Chennai, TamilNadu, India
| | - W Jino Hans
- Department of ECE, Sri Sivasubramaniya Nadar College of Engineering, Chennai, TamilNadu, India
| | - T M Inbamalar
- Department of ECE, RMK College of Engineering and Technology, Chennai, TamilNadu, India
| | - N Mahiban Lindsay
- Department of EEE, Hindustan Institute of Technology and Science, Chennai, TamilNadu, India
| |
Collapse
|
3
|
Zhao J, Yu Y, Sheng X, Zhu X. Consistent control information driven musculoskeletal model for multiday myoelectric control. J Neural Eng 2023; 20:056007. [PMID: 37567218 DOI: 10.1088/1741-2552/acef93] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2023] [Accepted: 08/10/2023] [Indexed: 08/13/2023]
Abstract
Objective.Musculoskeletal model (MM)-based myoelectric interface has aroused great interest in human-machine interaction. However, the performance of electromyography (EMG)-driven MM in long-term use would be degraded owing to the inherent non-stationary characteristics of EMG signals. Here, to improve the estimation performance without retraining, we proposed a consistent muscle excitation extraction approach based on an improved non-negative matrix factorization (NMF) algorithm for MM when applied to simultaneous hand and wrist movement prediction.Approach.We added constraints andL2-norm regularization terms to the objective function of classic NMF regarding muscle weighting matrix and time-varying profiles, through which stable muscle synergies across days were identified. The resultant profiles of these synergies were then used to drive the MM. Both offline and online experiments were conducted to evaluate the performance of the proposed method in inter-day scenarios.Main results.The results demonstrated significantly better and more robust performance over several competitive methods in inter-day experiments, including machine learning methods, EMG envelope-driven MM, and classic NMF-based MM. Furthermore, the analysis of control information on different days revealed the effectiveness of the proposed method in obtaining consistent muscle excitations.Significance.The outcomes potentially provide a novel and promising pathway for the robust and zero-retraining control of myoelectric interfaces.
Collapse
Affiliation(s)
- Jiamin Zhao
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Yang Yu
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Xinjun Sheng
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
- Meta Robotics Institute, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Xiangyang Zhu
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
- Meta Robotics Institute, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| |
Collapse
|
4
|
Schmidt MD, Glasmachers T, Iossifidis I. The concepts of muscle activity generation driven by upper limb kinematics. Biomed Eng Online 2023; 22:63. [PMID: 37355651 DOI: 10.1186/s12938-023-01116-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Accepted: 05/16/2023] [Indexed: 06/26/2023] Open
Abstract
BACKGROUND The underlying motivation of this work is to demonstrate that artificial muscle activity of known and unknown motion can be generated based on motion parameters, such as angular position, acceleration, and velocity of each joint (or the end-effector instead), which are similarly represented in our brains. This model is motivated by the known motion planning process in the central nervous system. That process incorporates the current body state from sensory systems and previous experiences, which might be represented as pre-learned inverse dynamics that generate associated muscle activity. METHODS We develop a novel approach utilizing recurrent neural networks that are able to predict muscle activity of the upper limbs associated with complex 3D human arm motions. Therefore, motion parameters such as joint angle, velocity, acceleration, hand position, and orientation, serve as input for the models. In addition, these models are trained on multiple subjects (n=5 including , 3 male in the age of 26±2 years) and thus can generalize across individuals. In particular, we distinguish between a general model that has been trained on several subjects, a subject-specific model, and a specific fine-tuned model using a transfer learning approach to adapt the model to a new subject. Estimators such as mean square error MSE, correlation coefficient r, and coefficient of determination R2 are used to evaluate the goodness of fit. We additionally assess performance by developing a new score called the zero-line score. The present approach was compared with multiple other architectures. RESULTS The presented approach predicts the muscle activity for previously through different subjects with remarkable high precision and generalizing nicely for new motions that have not been trained before. In an exhausting comparison, our recurrent network outperformed all other architectures. In addition, the high inter-subject variation of the recorded muscle activity was successfully handled using a transfer learning approach, resulting in a good fit for the muscle activity for a new subject. CONCLUSIONS The ability of this approach to efficiently predict muscle activity contributes to the fundamental understanding of motion control. Furthermore, this approach has great potential for use in rehabilitation contexts, both as a therapeutic approach and as an assistive device. The predicted muscle activity can be utilized to guide functional electrical stimulation, allowing specific muscles to be targeted and potentially improving overall rehabilitation outcomes.
Collapse
Affiliation(s)
- Marie D Schmidt
- Faculty of Electrical Engineering and Information Technology, Ruhr-University Bochum, Bochum, Germany.
- Institute of Computer Science, University of Applied Science Ruhr West, Mülheim an der Ruhr, Germany.
| | | | - Ioannis Iossifidis
- Institute of Computer Science, University of Applied Science Ruhr West, Mülheim an der Ruhr, Germany
| |
Collapse
|
5
|
Wang B, Hargrove L, Bao X, Kamavuako EN. Surface EMG Statistical and Performance Analysis of Targeted-Muscle-Reinnervated (TMR) Transhumeral Prosthesis Users in Home and Laboratory Settings. SENSORS (BASEL, SWITZERLAND) 2022; 22:9849. [PMID: 36560218 PMCID: PMC9786766 DOI: 10.3390/s22249849] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Revised: 12/09/2022] [Accepted: 12/12/2022] [Indexed: 06/17/2023]
Abstract
A pattern-recognition (PR)-based myoelectric control system is the trend of future prostheses development. Compared with conventional prosthetic control systems, PR-based control systems provide high dexterity, with many studies achieving >95% accuracy in the last two decades. However, most research studies have been conducted in the laboratory. There is limited research investigating how EMG signals are acquired when users operate PR-based systems in their home and community environments. This study compares the statistical properties of surface electromyography (sEMG) signals used to calibrate prostheses and quantifies the quality of calibration sEMG data through separability indices, repeatability indices, and correlation coefficients in home and laboratory settings. The results demonstrate no significant differences in classification performance between home and laboratory environments in within-calibration classification error (home: 6.33 ± 2.13%, laboratory: 7.57 ± 3.44%). However, between-calibration classification errors (home: 40.61 ± 9.19%, laboratory: 44.98 ± 12.15%) were statistically different. Furthermore, the difference in all statistical properties of sEMG signals is significant (p < 0.05). Separability indices reveal that motion classes are more diverse in the home setting. In summary, differences in sEMG signals generated between home and laboratory only affect between-calibration performance.
Collapse
Affiliation(s)
- Bingbin Wang
- Department of Engineering, King’s College London, London WC2R 2LS, UK
| | - Levi Hargrove
- Center for Bionic Medicine, the Shirley Ryan Ability, Chicago, IL 60611, USA
- Department of Physical Medicine and Rehabilitation, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA
| | - Xinqi Bao
- Department of Engineering, King’s College London, London WC2R 2LS, UK
| | - Ernest N. Kamavuako
- Department of Engineering, King’s College London, London WC2R 2LS, UK
- Faculté de Médecine, Université de Kindu, Site de Lwama II, Kindu, Maniema, Congo
| |
Collapse
|
6
|
Barfi M, Karami H, Faridi F, Sohrabi Z, Hosseini M. Improving robotic hand control via adaptive Fuzzy-PI controller using classification of EMG signals. Heliyon 2022; 8:e11931. [DOI: 10.1016/j.heliyon.2022.e11931] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Revised: 05/16/2022] [Accepted: 11/18/2022] [Indexed: 12/03/2022] Open
|
7
|
Tanaka T, Nambu I, Maruyama Y, Wada Y. Sliding-Window Normalization to Improve the Performance of Machine-Learning Models for Real-Time Motion Prediction Using Electromyography. SENSORS 2022; 22:s22135005. [PMID: 35808500 PMCID: PMC9269700 DOI: 10.3390/s22135005] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/10/2022] [Revised: 06/24/2022] [Accepted: 06/29/2022] [Indexed: 11/16/2022]
Abstract
Many researchers have used machine learning models to control artificial hands, walking aids, assistance suits, etc., using the biological signal of electromyography (EMG). The use of such devices requires high classification accuracy. One method for improving the classification performance of machine learning models is normalization, such as z-score. However, normalization is not used in most EMG-based motion prediction studies because of the need for calibration and fluctuation of reference value for calibration (cannot re-use). Therefore, in this study, we proposed a normalization method that combines sliding-window and z-score normalization that can be implemented in real-time processing without need for calibration. The effectiveness of this normalization method was confirmed by conducting a single-joint movement experiment of the elbow and predicting its rest, flexion, and extension movements from the EMG signal. The proposed method achieved 77.7% accuracy, an improvement of 21.5% compared to the non-normalization (56.2%). Furthermore, when using a model trained by other people’s data for application without calibration, the proposed method achieved 63.1% accuracy, an improvement of 8.8% compared to the z-score (54.4%). These results showed the effectiveness of the simple and easy-to-implement method, and that the classification performance of the machine learning model could be improved.
Collapse
Affiliation(s)
- Taichi Tanaka
- Department of Science Technology of Innovation, Nagaoka University of Technology, Nagaoka 940-2188, Japan
- Correspondence:
| | - Isao Nambu
- Department of Electrical, Electronics and Information Engineering, Nagaoka University of Technology, Nagaoka 940-2188, Japan; (I.N.); (Y.W.)
| | - Yoshiko Maruyama
- Department of Production Systems Engineering, National Institute of Technology, Hakodate College, Hakodate 042-8501, Japan;
| | - Yasuhiro Wada
- Department of Electrical, Electronics and Information Engineering, Nagaoka University of Technology, Nagaoka 940-2188, Japan; (I.N.); (Y.W.)
| |
Collapse
|
8
|
Wang H, Huang P, Xu T, Li G, Hu Y. Towards zero retraining for multiday motion recognition via a fully unsupervised adaptive approach and fabric myoelectric armband. IEEE Trans Neural Syst Rehabil Eng 2022; 30:217-225. [PMID: 35041609 DOI: 10.1109/tnsre.2022.3144323] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Surface electromyogram pattern recognition (EMG-PR) requires time-consuming training and retraining procedures for long-term use, hindering the usability of myoelectric control. In this paper, we design a fabric myoelectric armband to reduce the electrode shifts. Furthermore, we propose a fully unsupervised adaptive approach called hybrid serial classifier (HSC) to eliminate the burden of retraining over multiply days. We investigated the performance of our approach with a dataset of ten types of forearm motion from ten male subjects over eight weeks (total ten days, including: from day 1 to day 7, day 14, day 28, day 56). The average inter-day classification accuracies of HSC without any new retraining data are 86.61% when trained exclusively with the first day's EMG data, and 94.77% when trained with other nine days' data. We compare our proposed HSC algorithm with linear discriminant analysis (LDA) without recalibration (BLDA) and supervised adaption LDA (ALDA) with just one trial of new retraining data. The inter-day classification accuracy of HSC is significantly higher than that of BLDA and ALDA. These results indicate that our novel armband sEMG device is feasible for long-term use in conjunction with the proposed HSC algorithm.
Collapse
|
9
|
Dyson M, Olsen J, Dupan S. A Network-Enabled Myoelectric Platform for Prototyping Research Outside of the Lab. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:7422-7425. [PMID: 34892812 DOI: 10.1109/embc46164.2021.9630318] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
We present a network-enabled myoelectric platform for performing research outside of the laboratory environment. A low-cost, flexible, modular design based on common Internet of Things connectivity technology allows home-based research to be piloted. An outline of the platform is presented followed by technical results obtained from ten days of home-based tests with three participants. Results show the system enabled collection of close to 12,000 trials during around 28 cumulative hours of use. Home-based testing of multiple participants in parallel offers efficiency gains and provides a intuitive route toward long-term testing of upper-limb prosthetic devices in more naturalistic settings.Clinical relevance- In-home myoelectric training reduces clinician time. Network-enabled systems with back-end dashboards allow clinicians to monitor patients myoelectric ability over time and will provide a new way of accessing information about how upper-limb prosthetics are commonly used.
Collapse
|
10
|
Pradhan A, He J, Jiang N. Score, Rank, and Decision-Level Fusion Strategies of Multicode Electromyogram-based Verification and Identification Biometrics. IEEE J Biomed Health Inform 2021; 26:1068-1079. [PMID: 34473636 DOI: 10.1109/jbhi.2021.3109595] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Recent advances in biometric research have established surface electromyogram (sEMG) as a potential spoof-free solution to address some key limitations in current biometric traits. The nature of sEMG signals provide a unique dual-mode security: sEMGs have individual-specific characteristics (biometrics), and users can customize and change gestures just like passcodes. Such security also facilitates the use of code sequences (multicode) to further enhance the security. In this study, three levels of fusion, score, rank, and decision were investigated for two biometric applications, verification and identification. This study involved 24 subjects performing 16 hand/finger gestures, and code sequences with varying codelengths were generated. The performance of the verification and identification system was analyzed for varying codelength (M: 16) and rank (K: 14) to determine the best fusion scheme and desirable parameter values for a multicode sEMG biometric system. The results showed that the decision-level fusion scheme using a weighted majority voting resulted in an average equal error rate of 0.6% for the verification system when M=4. For the identification system, the score-level fusion scheme with score normalization based on fitting a Weibull distribution resulted in a minimum false rejection rate of 0.01% and false acceptance rate of 4.7% using a combination of K=2 and M=4. The results also suggested that the parameters M and K could be adjusted based on the number of users in the database to facilitate optimal performance. In summary, a multicode sEMG biometric system was developed to provide improved dual-mode security based on the personalized codes and biometric traits of individuals, with the combination of enhanced security and flexibility.
Collapse
|
11
|
Bao T, Zaidi SAR, Xie S, Yang P, Zhang ZQ. Inter-Subject Domain Adaptation for CNN-Based Wrist Kinematics Estimation Using sEMG. IEEE Trans Neural Syst Rehabil Eng 2021; 29:1068-1078. [PMID: 34086574 DOI: 10.1109/tnsre.2021.3086401] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Recently, convolutional neural network (CNN) has been widely investigated to decode human intentions using surface Electromyography (sEMG) signals. However, a pre-trained CNN model usually suffers from severe degradation when testing on a new individual, and this is mainly due to domain shift where characteristics of training and testing sEMG data differ substantially. To enhance inter-subject performances of CNN in the wrist kinematics estimation, we propose a novel regression scheme for supervised domain adaptation (SDA), based on which domain shift effects can be effectively reduced. Specifically, a two-stream CNN with shared weights is established to exploit source and target sEMG data simultaneously, such that domain-invariant features can be extracted. To tune CNN weights, both regression losses and a domain discrepancy loss are employed, where the former enable supervised learning and the latter minimizes distribution divergences between two domains. In this study, eight healthy subjects were recruited to perform wrist flexion-extension movements. Experiment results illustrated that the proposed regression SDA outperformed fine-tuning, a state-of-the-art transfer learning method, in both single-single and multiple-single scenarios of kinematics estimation. Unlike fine-tuning which suffers from catastrophic forgetting, regression SDA can maintain much better performances in original domains, which boosts the model reusability among multiple subjects.
Collapse
|
12
|
Fajardo JM, Gomez O, Prieto F. EMG hand gesture classification using handcrafted and deep features. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2020.102210] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
13
|
A novel statistical decimal pattern-based surface electromyogram signal classification method using tunable q-factor wavelet transform. Soft comput 2021. [DOI: 10.1007/s00500-020-05205-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
14
|
Tateno S, Liu H, Ou J. Development of Sign Language Motion Recognition System for Hearing-Impaired People Using Electromyography Signal. SENSORS (BASEL, SWITZERLAND) 2020; 20:E5807. [PMID: 33066452 PMCID: PMC7602266 DOI: 10.3390/s20205807] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/22/2020] [Revised: 10/10/2020] [Accepted: 10/12/2020] [Indexed: 06/11/2023]
Abstract
Sign languages are developed around the world for hearing-impaired people to communicate with others who understand them. Different grammar and alphabets limit the usage of sign languages between different sign language users. Furthermore, training is required for hearing-intact people to communicate with them. Therefore, in this paper, a real-time motion recognition system based on an electromyography signal is proposed for recognizing actual American Sign Language (ASL) hand motions for helping hearing-impaired people communicate with others and training normal people to understand the sign languages. A bilinear model is applied to deal with the on electromyography (EMG) data for decreasing the individual difference among different people. A long short-term memory neural network is used in this paper as the classifier. Twenty sign language motions in the ASL library are selected for recognition in order to increase the practicability of the system. The results indicate that this system can recognize these twenty motions with high accuracy among twenty participants. Therefore, this system has the potential to be widely applied to help hearing-impaired people for daily communication and normal people to understand the sign languages.
Collapse
Affiliation(s)
- Shigeyuki Tateno
- Graduate School of Information, Production and Systems, Waseda University, Kitakyushu 808-0135, Japan; (H.L.); (J.O.)
| | | | | |
Collapse
|
15
|
Teramae T, Matsubara T, Noda T, Morimoto J. Quaternion-Based Trajectory Optimization of Human Postures for Inducing Target Muscle Activation Patterns. IEEE Robot Autom Lett 2020. [DOI: 10.1109/lra.2020.3015460] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|