1
|
Tacca N, Dunlap C, Donegan SP, Hardin JO, Meyers E, Darrow MJ, Colachis Iv S, Gillman A, Friedenberg DA. Wearable high-density EMG sleeve for complex hand gesture classification and continuous joint angle estimation. Sci Rep 2024; 14:18564. [PMID: 39122791 PMCID: PMC11316006 DOI: 10.1038/s41598-024-64458-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2024] [Accepted: 06/10/2024] [Indexed: 08/12/2024] Open
Abstract
High-density electromyography (HD-EMG) can provide a natural interface to enhance human-computer interaction (HCI). This study aims to demonstrate the capability of a novel HD-EMG forearm sleeve equipped with up to 150 electrodes to capture high-resolution muscle activity, decode complex hand gestures, and estimate continuous hand position via joint angle predictions. Ten able-bodied participants performed 37 hand movements and grasps while EMG was recorded using the HD-EMG sleeve. Simultaneously, an 18-sensor motion capture glove calculated 23 joint angles from the hand and fingers across all movements for training regression models. For classifying across the 37 gestures, our decoding algorithm was able to differentiate between sequential movements with 97.3 ± 0.3 % accuracy calculated on a 100 ms bin-by-bin basis. In a separate mixed dataset consisting of 19 movements randomly interspersed, decoding performance achieved an average bin-wise accuracy of 92.8 ± 0.8 % . When evaluating decoders for use in real-time scenarios, we found that decoders can reliably decode both movements and movement transitions, achieving an average accuracy of 93.3 ± 0.9 % on the sequential set and 88.5 ± 0.9 % on the mixed set. Furthermore, we estimated continuous joint angles from the EMG sleeve data, achieving a R 2 of 0.884 ± 0.003 in the sequential set and 0.750 ± 0.008 in the mixed set. Median absolute error (MAE) was kept below 10° across all joints, with a grand average MAE of 1.8 ± 0 . 04 ∘ and 3.4 ± 0 . 07 ∘ for the sequential and mixed datasets, respectively. We also assessed two algorithm modifications to address specific challenges for EMG-driven HCI applications. To minimize decoder latency, we used a method that accounts for reaction time by dynamically shifting cue labels in time. To reduce training requirements, we show that pretraining models with historical data provided an increase in decoding performance compared with models that were not pretrained when reducing the in-session training data to only one attempt of each movement. The HD-EMG sleeve, combined with sophisticated machine learning algorithms, can be a powerful tool for hand gesture recognition and joint angle estimation. This technology holds significant promise for applications in HCI, such as prosthetics, assistive technology, rehabilitation, and human-robot collaboration.
Collapse
Affiliation(s)
- Nicholas Tacca
- Battelle Memorial Institute, Neurotechnology, Columbus, OH, USA.
| | - Collin Dunlap
- Battelle Memorial Institute, Neurotechnology, Columbus, OH, USA
| | - Sean P Donegan
- Air Force Research Laboratory, Materials And Manufacturing Directorate, Wright-Patterson AFB, OH, USA
| | - James O Hardin
- Air Force Research Laboratory, Materials And Manufacturing Directorate, Wright-Patterson AFB, OH, USA
| | - Eric Meyers
- Battelle Memorial Institute, Neurotechnology, Columbus, OH, USA
| | | | | | - Andrew Gillman
- Air Force Research Laboratory, Materials And Manufacturing Directorate, Wright-Patterson AFB, OH, USA
| | | |
Collapse
|
2
|
Wang X, Wang J, Fei N, Duanmu D, Feng B, Li X, IP WY, Hu Y. Alternative muscle synergy patterns of upper limb amputees. Cogn Neurodyn 2024; 18:1119-1133. [PMID: 38826662 PMCID: PMC11143172 DOI: 10.1007/s11571-023-09969-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2023] [Revised: 03/29/2023] [Accepted: 04/11/2023] [Indexed: 06/04/2024] Open
Abstract
Myoelectric hand prostheses are effective tools for upper limb amputees to regain hand functions. Much progress has been made with pattern recognition algorithms to recognize surface electromyography (sEMG) patterns, but few attentions was placed on the amputees' motor learning process. Many potential myoelectric prostheses users could not fully master the control or had declined performance over time. It is possible that learning to produce distinct and consistent muscle activation patterns with the residual limb could help amputees better control the myoelectric prosthesis. In this study, we observed longitudinal effect of motor skill learning with 2 amputees who have developed alternative muscle activation patterns in response to the same set of target prosthetic actions. During a 10-week program, amputee participants were trained to produce distinct and constant muscle activations with visual feedback of live sEMG and without interaction with prosthesis. At the end, their sEMG patterns were different from each other and from non-amputee control groups. For certain intended hand motion, gradually reducing root mean square (RMS) variance was observed. The learning effect was also assessed with a CNN-LSTM mixture classifier designed for mobile sEMG pattern recognition. The classification accuracy had a rising trend over time, implicating potential performance improvement of myoelectric prosthesis control. A follow-up session took place 6 months after the program and showed lasting effect of the motor skill learning in terms of sEMG pattern classification accuracy. The results indicated that with proper feedback training, amputees could learn unique muscle activation patterns that allow them to trigger intended prosthesis functions, and the original motor control scheme is updated. The effect of such motor skill learning could help to improve myoelectric prosthetic control performance.
Collapse
Affiliation(s)
- Xiaojun Wang
- Department of Orthopedics and Traumatology, LKS Faculty of Medicine, The University of Hong Kong, 999077 Hong Kong, China
| | - Junlin Wang
- Department of Orthopedics and Traumatology, LKS Faculty of Medicine, The University of Hong Kong, 999077 Hong Kong, China
- Orthopedics Center, The University of Hong Kong-Shenzhen Hospital, Shenzhen, 518000 China
| | - Ningbo Fei
- Department of Orthopedics and Traumatology, LKS Faculty of Medicine, The University of Hong Kong, 999077 Hong Kong, China
| | - Dehao Duanmu
- Department of Orthopedics and Traumatology, LKS Faculty of Medicine, The University of Hong Kong, 999077 Hong Kong, China
| | - Beibei Feng
- Department of Orthopedics and Traumatology, LKS Faculty of Medicine, The University of Hong Kong, 999077 Hong Kong, China
| | - Xiaodong Li
- Department of Orthopedics and Traumatology, LKS Faculty of Medicine, The University of Hong Kong, 999077 Hong Kong, China
- Orthopedics Center, The University of Hong Kong-Shenzhen Hospital, Shenzhen, 518000 China
| | - Wing-Yuk IP
- Department of Orthopedics and Traumatology, LKS Faculty of Medicine, The University of Hong Kong, 999077 Hong Kong, China
| | - Yong Hu
- Department of Orthopedics and Traumatology, LKS Faculty of Medicine, The University of Hong Kong, 999077 Hong Kong, China
- Orthopedics Center, The University of Hong Kong-Shenzhen Hospital, Shenzhen, 518000 China
| |
Collapse
|
3
|
Zabihi S, Rahimian E, Asif A, Mohammadi A. TraHGR: Transformer for Hand Gesture Recognition via Electromyography. IEEE Trans Neural Syst Rehabil Eng 2023; 31:4211-4224. [PMID: 37831560 DOI: 10.1109/tnsre.2023.3324252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/15/2023]
Abstract
Deep learning-based Hand Gesture Recognition (HGR) via surface Electromyogram (sEMG) signals have recently shown considerable potential for development of advanced myoelectric-controlled prosthesis. Although deep learning techniques can improve HGR accuracy compared to their classical counterparts, classifying hand movements based on sparse multichannel sEMG signals is still a challenging task. Furthermore, existing deep learning approaches, typically, include only one model as such can hardly extract representative features. In this paper, we aim to address this challenge by capitalizing on the recent advances in hybrid models and transformers. In other words, we propose a hybrid framework based on the transformer architecture, which is a relatively new and revolutionizing deep learning model. The proposed hybrid architecture, referred to as the Transformer for Hand Gesture Recognition (TraHGR), consists of two parallel paths followed by a linear layer that acts as a fusion center to integrate the advantage of each module. We evaluated the proposed architecture TraHGR based on the commonly used second Ninapro dataset, referred to as the DB2. The sEMG signals in the DB2 dataset are measured in real-life conditions from 40 healthy users, each performing 49 gestures. We have conducted an extensive set of experiments to test and validate the proposed TraHGR architecture, and compare its achievable accuracy with several recently proposed HGR classification algorithms over the same dataset. We have also compared the results of the proposed TraHGR architecture with each individual path and demonstrated the distinguishing power of the proposed hybrid architecture. The recognition accuracies of the proposed TraHGR architecture for the window of size 200ms and step size of 100ms are 86.00%, 88.72%, 81.27%, and 93.74%, which are 2.30%, 4.93%, 8.65%, and 4.20% higher than the state-of-the-art performance for DB2 (49 gestures), DB2-B (17 gestures), DB2-C (23 gestures), and DB2-D (9 gestures), respectively.
Collapse
|
4
|
Jiang N, Chen C, He J, Meng J, Pan L, Su S, Zhu X. Bio-robotics research for non-invasive myoelectric neural interfaces for upper-limb prosthetic control: a 10-year perspective review. Natl Sci Rev 2023; 10:nwad048. [PMID: 37056442 PMCID: PMC10089583 DOI: 10.1093/nsr/nwad048] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Revised: 01/01/2023] [Accepted: 02/07/2023] [Indexed: 04/05/2023] Open
Abstract
ABSTRACT
A decade ago, a group of researchers from academia and industry identified a dichotomy between the industrial and academic state-of-the-art in upper-limb prosthesis control, a widely used bio-robotics application. They proposed that four key technical challenges, if addressed, could bridge this gap and translate academic research into clinically and commercially viable products. These challenges are unintuitive control schemes, lack of sensory feedback, poor robustness and single sensor modality. Here, we provide a perspective review on the research effort that occurred in the last decade, aiming at addressing these challenges. In addition, we discuss three research areas essential to the recent development in upper-limb prosthetic control research but were not envisioned in the review 10 years ago: deep learning methods, surface electromyogram decomposition and open-source databases. To conclude the review, we provide an outlook into the near future of the research and development in upper-limb prosthetic control and beyond.
Collapse
Affiliation(s)
| | - Chen Chen
- State Key Laboratory of Mechanical System and Vibration, and Institute of Robotics, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Jiayuan He
- National Clinical Research Center for Geriatrics, West China Hospital, and Med-X Center for Manufacturing, Sichuan University, Chengdu 610041, China
| | - Jianjun Meng
- State Key Laboratory of Mechanical System and Vibration, and Institute of Robotics, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Lizhi Pan
- Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, School of Mechanical Engineering, Tianjin University, Tianjin 300350, China
| | - Shiyong Su
- Institute of Neuroscience, Université Catholique Louvain, Brussel B-1348, Belgium
| | - Xiangyang Zhu
- State Key Laboratory of Mechanical System and Vibration, and Institute of Robotics, Shanghai Jiao Tong University, Shanghai 200240, China
| |
Collapse
|
5
|
Ershad F, Houston M, Patel S, Contreras L, Koirala B, Lu Y, Rao Z, Liu Y, Dias N, Haces-Garcia A, Zhu W, Zhang Y, Yu C. Customizable, reconfigurable, and anatomically coordinated large-area, high-density electromyography from drawn-on-skin electrode arrays. PNAS NEXUS 2023; 2:pgac291. [PMID: 36712933 PMCID: PMC9837666 DOI: 10.1093/pnasnexus/pgac291] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/22/2022] [Accepted: 12/09/2022] [Indexed: 06/18/2023]
Abstract
Accurate anatomical matching for patient-specific electromyographic (EMG) mapping is crucial yet technically challenging in various medical disciplines. The fixed electrode construction of multielectrode arrays (MEAs) makes it nearly impossible to match an individual's unique muscle anatomy. This mismatch between the MEAs and target muscles leads to missing relevant muscle activity, highly redundant data, complicated electrode placement optimization, and inaccuracies in classification algorithms. Here, we present customizable and reconfigurable drawn-on-skin (DoS) MEAs as the first demonstration of high-density EMG mapping from in situ-fabricated electrodes with tunable configurations adapted to subject-specific muscle anatomy. The DoS MEAs show uniform electrical properties and can map EMG activity with high fidelity under skin deformation-induced motion, which stems from the unique and robust skin-electrode interface. They can be used to localize innervation zones (IZs), detect motor unit propagation, and capture EMG signals with consistent quality during large muscle movements. Reconfiguring the electrode arrangement of DoS MEAs to match and extend the coverage of the forearm flexors enables localization of the muscle activity and prevents missed information such as IZs. In addition, DoS MEAs customized to the specific anatomy of subjects produce highly informative data, leading to accurate finger gesture detection and prosthetic control compared with conventional technology.
Collapse
Affiliation(s)
- Faheem Ershad
- Department of Biomedical Engineering, Pennsylvania State University, University Park, PA, 16801, USA
- Department of Biomedical Engineering, University of Houston, Houston, TX, 77204, USA
| | - Michael Houston
- Department of Biomedical Engineering, University of Houston, Houston, TX, 77204, USA
| | - Shubham Patel
- Department of Engineering Science and Mechanics, Pennsylvania State University, University Park, PA, 16801, USA
- Department of Mechanical Engineering, University of Houston, Houston, TX, 77204, USA
| | - Luis Contreras
- Department of Biomedical Engineering, University of Houston, Houston, TX, 77204, USA
| | - Bikram Koirala
- Department of Mechanical Engineering, University of Houston, Houston, TX, 77204, USA
- Department of Engineering Technology, University of Houston, Houston, TX, 77204, USA
| | - Yuntao Lu
- Department of Engineering Science and Mechanics, Pennsylvania State University, University Park, PA, 16801, USA
- Materials Science and Engineering Program, University of Houston, Houston, TX, 77204, USA
| | - Zhoulyu Rao
- Department of Engineering Science and Mechanics, Pennsylvania State University, University Park, PA, 16801, USA
- Materials Science and Engineering Program, University of Houston, Houston, TX, 77204, USA
| | - Yang Liu
- Department of Biomedical Engineering, University of Houston, Houston, TX, 77204, USA
| | - Nicholas Dias
- Department of Biomedical Engineering, University of Houston, Houston, TX, 77204, USA
| | - Arturo Haces-Garcia
- Department of Engineering Technology, University of Houston, Houston, TX, 77204, USA
- Department of Electrical and Computer Engineering, University of Houston, Houston, TX, 77204, USA
| | - Weihang Zhu
- Department of Mechanical Engineering, University of Houston, Houston, TX, 77204, USA
- Department of Engineering Technology, University of Houston, Houston, TX, 77204, USA
| | - Yingchun Zhang
- Department of Biomedical Engineering, University of Houston, Houston, TX, 77204, USA
| | | |
Collapse
|
6
|
Pelaez Murciego L, Henrich MC, Spaich EG, Dosen S. Reducing the number of EMG electrodes during online hand gesture classification with changing wrist positions. J Neuroeng Rehabil 2022; 19:78. [PMID: 35864513 PMCID: PMC9306156 DOI: 10.1186/s12984-022-01056-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2022] [Accepted: 07/06/2022] [Indexed: 11/18/2022] Open
Abstract
Background Myoelectric control based on hand gesture classification can be used for effective, contactless human–machine interfacing in general applications (e.g., consumer market) as well as in the clinical context. However, the accuracy of hand gesture classification can be impacted by several factors including changing wrist position. The present study aimed at investigating how channel configuration (number and placement of electrode pads) affects performance in hand gesture recognition across wrist positions, with the overall goal of reducing the number of channels without the loss of performance with respect to the benchmark (all channels). Methods Matrix electrodes (256 channels) were used to record high-density EMG from the forearm of 13 healthy subjects performing a set of 8 gestures in 3 wrist positions and 2 force levels (low and moderate). A reduced set of channels was chosen by applying sequential forward selection (SFS) and simple circumferential placement (CIRC) and used for gesture classification with linear discriminant analysis. The classification success rate and task completion rate were the main outcome measures for offline analysis across the different number of channels and online control using 8 selected channels, respectively. Results The offline analysis demonstrated that good accuracy (> 90%) can be achieved with only a few channels. However, using data from all wrist positions required more channels to reach the same performance. Despite the targeted placement (SFS) performing similarly to CIRC in the offline analysis, the task completion rate [median (lower–upper quartile)] in the online control was significantly higher for SFS [71.4% (64.8–76.2%)] compared to CIRC [57.1% (51.8–64.8%), p < 0.01], especially for low contraction levels [76.2% (66.7–84.5%) for SFS vs. 57.1% (47.6–60.7%) for CIRC, p < 0.01]. For the reduced number of electrodes, the performance with SFS was comparable to that obtained when using the full matrix, while the selected electrodes were highly subject-specific. Conclusions The present study demonstrated that the number of channels required for gesture classification with changing wrist positions could be decreased substantially without loss of performance, if those channels are placed strategically along the forearm and individually for each subject. The results also emphasize the importance of online assessment and motivate the development of configurable matrix electrodes with integrated channel selection.
Collapse
Affiliation(s)
- Luis Pelaez Murciego
- Neurorehabilitation Systems, Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | - Mauricio C Henrich
- Neurorehabilitation Systems, Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | - Erika G Spaich
- Neurorehabilitation Systems, Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | - Strahinja Dosen
- Neurorehabilitation Systems, Department of Health Science and Technology, Aalborg University, Aalborg, Denmark.
| |
Collapse
|
7
|
Asghar A, Jawaid Khan S, Azim F, Shakeel CS, Hussain A, Niazi IK. Review on electromyography based intention for upper limb control using pattern recognition for human-machine interaction. Proc Inst Mech Eng H 2022; 236:628-645. [DOI: 10.1177/09544119221074770] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Upper limb myoelectric prosthetic control is an essential topic in the field of rehabilitation. The technique controls prostheses using surface electromyogram (sEMG) and intramuscular EMG (iEMG) signals. EMG signals are extensively used in controlling prosthetic upper and lower limbs, virtual reality entertainment, and human-machine interface (HMI). EMG signals are vital parameters for machine learning and deep learning algorithms and help to give an insight into the human brain’s function and mechanisms. Pattern recognition techniques pertaining to support vector machine (SVM), k-nearest neighbor (KNN) and Bayesian classifiers have been utilized to classify EMG signals. This paper presents a review on current EMG signal techniques, including electrode array utilization, signal acquisition, signal preprocessing and post-processing, feature selection and extraction, data dimensionality reduction, classification, and ultimate application to the community. The paper also discusses using alternatives to EMG signals, such as force sensors, to measure muscle activity with reliable results. Future implications for EMG classification include employing deep learning techniques such as artificial neural networks (ANN) and recurrent neural networks (RNN) for achieving robust results.
Collapse
Affiliation(s)
- Ali Asghar
- Department of Biomedical Engineering, Faculty of Engineering, Science, Technology and Management, Ziauddin University, Karachi, Pakistan
- Department of Electrical Engineering, Faculty of Engineering, Science, Technology and Management, Ziauddin University, Karachi, Pakistan
| | - Saad Jawaid Khan
- Department of Biomedical Engineering, Faculty of Engineering, Science, Technology and Management, Ziauddin University, Karachi, Pakistan
| | - Fahad Azim
- Department of Electrical Engineering, Faculty of Engineering, Science, Technology and Management, Ziauddin University, Karachi, Pakistan
| | - Choudhary Sobhan Shakeel
- Department of Biomedical Engineering, Faculty of Engineering, Science, Technology and Management, Ziauddin University, Karachi, Pakistan
| | - Amatullah Hussain
- College of Rehabilitation Sciences, Ziauddin University, Karachi, Pakistan
| | - Imran Khan Niazi
- Centre for Chiropractic Research, New Zealand College of Chiropractic, New Zealand
- Faculty of Health & Environmental Sciences, Health & Rehabilitation Research Institute, AUT University, New Zealand
- Centre for Sensory-Motor Interactions, Department of Health Science and Technology, Aalborg University, Denmark
| |
Collapse
|
8
|
Bao T, Xie SQ, Yang P, Zhou P, Zhang ZQ. Towards Robust, Adaptive and Reliable Upper-limb Motion Estimation Using Machine Learning and Deep Learning--A Survey in Myoelectric Control. IEEE J Biomed Health Inform 2022; 26:3822-3835. [PMID: 35294368 DOI: 10.1109/jbhi.2022.3159792] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
To develop multi-functional human-machine interfaces that can help disabled people reconstruct lost functions of upper-limbs, machine learning (ML) and deep learning (DL) techniques have been widely implemented to decode human movement intentions from surface electromyography (sEMG) signals. However, due to the high complexity of upper-limb movements and the inherent non-stable characteristics of sEMG, the usability of ML/DL based control schemes is still greatly limited in practical scenarios. To this end, tremendous efforts have been made to improve model robustness, adaptation, and reliability. In this article, we provide a systematic review on recent achievements, mainly from three categories: multi-modal sensing fusion to gain additional information of the user, transfer learning (TL) methods to eliminate domain shift impacts on estimation models, and post-processing approaches to obtain more reliable outcomes. Special attention is given to fusion strategies, deep TL frameworks, and confidence estimation. \textcolor{red}{Research challenges and emerging opportunities, with respect to hardware development, public resources, and decoding strategies, are also analysed to provide perspectives for future developments.
Collapse
|
9
|
Wang H, Huang P, Xu T, Li G, Hu Y. Towards zero retraining for multiday motion recognition via a fully unsupervised adaptive approach and fabric myoelectric armband. IEEE Trans Neural Syst Rehabil Eng 2022; 30:217-225. [PMID: 35041609 DOI: 10.1109/tnsre.2022.3144323] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Surface electromyogram pattern recognition (EMG-PR) requires time-consuming training and retraining procedures for long-term use, hindering the usability of myoelectric control. In this paper, we design a fabric myoelectric armband to reduce the electrode shifts. Furthermore, we propose a fully unsupervised adaptive approach called hybrid serial classifier (HSC) to eliminate the burden of retraining over multiply days. We investigated the performance of our approach with a dataset of ten types of forearm motion from ten male subjects over eight weeks (total ten days, including: from day 1 to day 7, day 14, day 28, day 56). The average inter-day classification accuracies of HSC without any new retraining data are 86.61% when trained exclusively with the first day's EMG data, and 94.77% when trained with other nine days' data. We compare our proposed HSC algorithm with linear discriminant analysis (LDA) without recalibration (BLDA) and supervised adaption LDA (ALDA) with just one trial of new retraining data. The inter-day classification accuracy of HSC is significantly higher than that of BLDA and ALDA. These results indicate that our novel armband sEMG device is feasible for long-term use in conjunction with the proposed HSC algorithm.
Collapse
|
10
|
Li X, Zheng Y, Liu Y, Tian L, Fang P, Cao J, Li G. A Novel Motion Recognition Method Based on Force Myography of Dynamic Muscle Contractions. Front Neurosci 2022; 15:783539. [PMID: 35095397 PMCID: PMC8792837 DOI: 10.3389/fnins.2021.783539] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2021] [Accepted: 12/10/2021] [Indexed: 11/23/2022] Open
Abstract
Surface electromyogram-based pattern recognition (sEMG-PR) has been considered as the most promising method to control multifunctional prostheses for decades. However, the commercial applications of sEMG-PR in prosthetic control is still limited due to the ambient noise and impedance variation between electrodes and skin surface. In order to reduce these issues, a force-myography-based pattern recognition method was proposed. In this method, a type of polymer-based flexible film sensors, the piezoelectrets, were used to record the rate of stress change (RSC) signals on the muscle surface of eight able-bodied subjects for six hand motions. Thirteen time domain features and four classification algorithms of linear discriminant analysis (LDA), K-nearest neighbor (KNN), artificial neural network (ANN), and support vector machine (SVM) were adopted to decode the RSC signals of different motion classes. In addition, the optimal feature set, classifier, and analysis window length were investigated systematically. Results showed that the average classification accuracy was 95.5 ± 2.2% by using the feature combination of root mean square (RMS) and waveform length (WL) for the classifier of KNN, and the analysis window length of 300 ms was found to obtain the best classification performance. Moreover, the robustness of the proposed method was investigated, and the classification accuracies were observed above 90% even when the white noise ratio increased to 50%. The work of this study demonstrated the effectiveness of RSC-based pattern recognition method for motion classification, and it would provide an alternative approach for the control of multifunctional prostheses.
Collapse
Affiliation(s)
- Xiangxin Li
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences, Shenzhen, China
- Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, China
- Shenzhen Engineering Laboratory of Neural Rehabilitation Technology, Shenzhen, China
| | - Yue Zheng
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences, Shenzhen, China
- Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, China
- Shenzhen Engineering Laboratory of Neural Rehabilitation Technology, Shenzhen, China
| | - Yan Liu
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences, Shenzhen, China
- Shenzhen Engineering Laboratory of Neural Rehabilitation Technology, Shenzhen, China
| | - Lan Tian
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences, Shenzhen, China
- Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, China
- Shenzhen Engineering Laboratory of Neural Rehabilitation Technology, Shenzhen, China
| | - Peng Fang
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences, Shenzhen, China
- Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China
- Shenzhen Engineering Laboratory of Neural Rehabilitation Technology, Shenzhen, China
- *Correspondence: Peng Fang,
| | - Jianglang Cao
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences, Shenzhen, China
- Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China
- Shenzhen Engineering Laboratory of Neural Rehabilitation Technology, Shenzhen, China
| | - Guanglin Li
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences, Shenzhen, China
- Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, China
- Shenzhen Engineering Laboratory of Neural Rehabilitation Technology, Shenzhen, China
- Guanglin Li,
| |
Collapse
|
11
|
Kim H, Im CH. Influence of the Number of Channels and Classification Algorithm on the Performance Robustness to Electrode Shift in Steady-State Visual Evoked Potential-Based Brain-Computer Interfaces. Front Neuroinform 2021; 15:750839. [PMID: 34744677 PMCID: PMC8569408 DOI: 10.3389/fninf.2021.750839] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2021] [Accepted: 09/21/2021] [Indexed: 11/13/2022] Open
Abstract
There remains an active investigation on elevating the classification accuracy and information transfer rate of brain-computer interfaces based on steady-state visual evoked potential. However, it has often been ignored that the performance of steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) can be affected through the minor displacement of the electrodes from their optimal locations in practical applications because of the mislocation of electrodes and/or concurrent use of electroencephalography (EEG) devices with external devices, such as virtual reality headsets. In this study, we evaluated the performance robustness of SSVEP-based BCIs with respect to the changes in electrode locations for various channel configurations and classification algorithms. Our experiments involved 21 participants, where EEG signals were recorded from the scalp electrodes densely attached to the occipital area of the participants. The classification accuracies for all the possible cases of electrode location shifts for various channel configurations (1–3 channels) were calculated using five training-free SSVEP classification algorithms, i.e., the canonical correlation analysis (CCA), extended CCA, filter bank CCA, multivariate synchronization index (MSI), and extended MSI (EMSI). Then, the performances of the BCIs were evaluated using two measures, i.e., the average classification accuracy (ACA) across the electrode shifts and robustness to the electrode shift (RES). Our results showed that the ACA increased with an increase in the number of channels regardless of the algorithm. However, the RES was enhanced with an increase in the number of channels only when MSI and EMSI were employed. While both ACA and RES values for the five algorithms were similar under the single-channel condition, both ACA and RES values for MSI and EMSI were higher than those of the other algorithms under the multichannel (i.e., two or three electrodes) conditions. In addition, EMSI outperformed MSI when comparing the ACA and RES values under the multichannel conditions. In conclusion, our results suggested that the use of multichannel configuration and employment of EMSI could make the performance of SSVEP-based BCIs more robust to the electrode shift from the optimal locations.
Collapse
Affiliation(s)
- Hodam Kim
- Department of Biomedical Engineering, Hanyang University, Seoul, South Korea.,Department of Electronic Engineering, Hanyang University, Seoul, South Korea
| | - Chang-Hwan Im
- Department of Biomedical Engineering, Hanyang University, Seoul, South Korea.,Department of Electronic Engineering, Hanyang University, Seoul, South Korea.,Department of HY-KIST Bioconvergence, Hanyang University, Seoul, South Korea.,Department of Artificial Intelligence, Hanyang University, Seoul, South Korea
| |
Collapse
|
12
|
Reynolds DJ, Shazar A, Zhang X. Design and Validation of a Sensor Fault-Tolerant Module for Real-Time High-Density EMG Pattern Recognition. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:6738-6742. [PMID: 34892654 DOI: 10.1109/embc46164.2021.9629541] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
With the advancements in electronics technology, high-density (HD) EMG sensing systems have become available and have been investigated for their feasibility and performance in neural-machine interface (NMI) applications. Comparing to the traditional single channel-based targeted muscle sensing method, HD EMG sensing performs a sampling of the electrical activity over a larger surface area and has the promise of 1) providing richer neural information from one temporal and two spatial dimensions and 2) ease of wear in real life without the need of anatomically targeted electrode placement. To use HD EMG in real-time NMI applications, challenges including high computational burden and unreliability of EMG recordings over time need to be addressed. This paper presented an HD EMG PR based NMI which seamlessly integrates HD EMG PR with a Sensor Fault-Tolerant Module (SFTM) which aimed to provide robust PR in real time. Experimental results showed that the SFTM was able to recover the PR accuracies by 6%-22% from disturbances including contact artifacts and loose contacts. A Python-based implementation of the proposed HD EMG SFTM was developed and was demonstrated to be computationally efficient for real-time performance. These results have demonstrated the feasibility of a robust real-time HD EMG PR-based NMI.
Collapse
|
13
|
Li X, Tian L, Zheng Y, Samuel OW, Fang P, Wang L, Li G. A new strategy based on feature filtering technique for improving the real-time control performance of myoelectric prostheses. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102969] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
14
|
Bao T, Zaidi SAR, Xie S, Yang P, Zhang ZQ. Inter-Subject Domain Adaptation for CNN-Based Wrist Kinematics Estimation Using sEMG. IEEE Trans Neural Syst Rehabil Eng 2021; 29:1068-1078. [PMID: 34086574 DOI: 10.1109/tnsre.2021.3086401] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Recently, convolutional neural network (CNN) has been widely investigated to decode human intentions using surface Electromyography (sEMG) signals. However, a pre-trained CNN model usually suffers from severe degradation when testing on a new individual, and this is mainly due to domain shift where characteristics of training and testing sEMG data differ substantially. To enhance inter-subject performances of CNN in the wrist kinematics estimation, we propose a novel regression scheme for supervised domain adaptation (SDA), based on which domain shift effects can be effectively reduced. Specifically, a two-stream CNN with shared weights is established to exploit source and target sEMG data simultaneously, such that domain-invariant features can be extracted. To tune CNN weights, both regression losses and a domain discrepancy loss are employed, where the former enable supervised learning and the latter minimizes distribution divergences between two domains. In this study, eight healthy subjects were recruited to perform wrist flexion-extension movements. Experiment results illustrated that the proposed regression SDA outperformed fine-tuning, a state-of-the-art transfer learning method, in both single-single and multiple-single scenarios of kinematics estimation. Unlike fine-tuning which suffers from catastrophic forgetting, regression SDA can maintain much better performances in original domains, which boosts the model reusability among multiple subjects.
Collapse
|
15
|
Lv B, Chai G, Sheng X, Ding H, Zhu X. Evaluating User and Machine Learning in Short- and Long-Term Pattern Recognition-Based Myoelectric Control. IEEE Trans Neural Syst Rehabil Eng 2021; 29:777-785. [PMID: 33861704 DOI: 10.1109/tnsre.2021.3073751] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Proper training is essential to achieve reliable pattern recognition (PR) based myoelectric control. The amount of training is commonly determined by experience. The purpose of this study is to provide an offline validation method that makes the offline performance transferable to online control and find the proper amount of training that achieves good online performance. In the offline experiment, eight able-bodied subjects and three amputees participated in a ten-day training. Repeatability index (RI) and classification error (CE) were used to evaluate user learning and machine learning, respectively. The performance of cross-validation (CV) and time serial related validation (TSV) was compared. Learning curves were established with different training trials by TSV. In the online experiment, sixteen able-bodied subjects were randomly divided into two groups with one- or five-trial training, respectively, followed by participating in the test with and without classifier-output feedback. The correlation between offline and online tests was analyzed. Results indicated that five-trial training was proper to train the user and the classifier. The long-term retention of skills could not shorten the learning process. The correlation between CEs of TSV and the online test was strong ( r=0.87 ) with five-trial training, while the correlation between CEs of CV and the online test was weak ( r=0.30 ). Outcomes demonstrate that offline performance evaluated by TSV is transferable to online performance and the learning process can guide the user to achieve good online myoelectric control with minimum training.
Collapse
|
16
|
Chandra S, Li J, Afsharipour B, Cardona AF, Suresh NL, Tian L, Deng Y, Zhong Y, Xie Z, Shen H, Huang Y, Rogers JA, Rymer WZ. Performance Evaluation of a Wearable Tattoo Electrode Suitable for High-Resolution Surface Electromyogram Recording. IEEE Trans Biomed Eng 2021; 68:1389-1398. [PMID: 33079653 PMCID: PMC8015348 DOI: 10.1109/tbme.2020.3032354] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
OBJECTIVE High-density surface electromyography (HD-sEMG) has been utilized extensively in neuromuscular research. Despite its potential advantages, limitations in electrode design have largely prevented widespread acceptance of the technology. Commercial electrodes have limited spatial fidelity, because of a lack of sharpness of the signal, and variable signal stability. We demonstrate here a novel tattoo electrode that addresses these issues. Our dry HD electrode grid exhibits remarkable deformability which ensures superior conformity with the skin surface, while faithfully recording signals during different levels of muscle contraction. METHOD We fabricated a 4 cm×3 cm tattoo HD electrode grid on a stretchable electronics membrane for sEMG applications. The grid was placed on the skin overlying the biceps brachii of healthy subjects, and was used to record signals for several hours while tracking different isometric contractions. RESULTS The sEMG signals were recorded successfully from all 64 electrodes across the grid. These electrodes were able to faithfully record sEMG signals during repeated contractions while maintaining a stable baseline at rest. During voluntary contractions, broad EMG frequency content was preserved, with accurate reproduction of the EMG spectrum across the full signal bandwidth. CONCLUSION The tattoo grid electrode can potentially be used for recording high-density sEMG from skin overlying major limb muscles. Layout programmability, good signal quality, excellent baseline stability, and easy wearability make this electrode a potentially valuable component of future HD electrode grid applications. SIGNIFICANCE The tattoo electrode can facilitate high fidelity recording in clinical applications such as tracking the evolution and time-course of challenging neuromuscular degenerative disorders.
Collapse
|
17
|
Wu L, Zhang X, Wang K, Chen X, Chen X. Improved High-Density Myoelectric Pattern Recognition Control Against Electrode Shift Using Data Augmentation and Dilated Convolutional Neural Network. IEEE Trans Neural Syst Rehabil Eng 2021; 28:2637-2646. [PMID: 33052847 DOI: 10.1109/tnsre.2020.3030931] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
OBJECTIVE the objective of this work is to develop a robust method for myoelectric control towards alleviating the interference of electrode shift. M ethods: In the proposed method, a preprocessing approach was first performed to convert high-density surface electromyogram (HD-sEMG) signals into a series of images, and the electrode shift appeared as pixel shift in these images. Next, a data augmentation approach was applied to the training data from just one position (no shift), so as to simulate HD-sEMG images derived from fictitious shift positions. The dilated convolutional neural network (DCNN) was subsequently adopted for classification. Compared to common convolutional neural network, DCNN always contained a larger receptive field that was supposed to be adept at mining wider spatial contextual information in images. This property was further confirmed to facilitate the classification of myoelectric patterns using HD-sEMG. The performance of the proposed method was evaluated with HD-sEMG data recorded by a 10×10 electrode array placed over forearm extensors of ten subjects during their performance of six wrist and finger extension tasks. RESULTS Under a variety of actual electrode shift conditions, the proposed method achieved a mean classification accuracy of 95.34%, and it outperformed other common methods. CONCLUSION This work demonstrated feasibility and usability of combining data augmentation and DCNN in predicting myoelectric patterns in the context of electrode shifts. SIGNIFICANCE The proposed method is a practical solution for robust myoelectric control against electrode array shifts.
Collapse
|
18
|
He J, Sheng X, Zhu X, Jiang N. Position Identification for Robust Myoelectric Control Against Electrode Shift. IEEE Trans Neural Syst Rehabil Eng 2021; 28:3121-3128. [PMID: 33196444 DOI: 10.1109/tnsre.2020.3038374] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
The vulnerability to the electrode shift was one of the key barriers to the wide application of pattern recognition-based (PR-based) myoelectric control systems outside the controlled laboratory conditions. To overcome this challenge, a novel framework named position identification (PI) was proposed. In the PI framework, an anchor gesture performed by the user was first analyzed to identify the current electrode position from a pool of potential electrode shift positions. Next, the classifier calibrated by the data of the identified position would be selected for following myoelectric control tasks. The results of the amputee and able-bodied participants both demonstrated that the differential filter combined with majority voting improved the PI accuracy. With only one second contraction of the chosen anchor gesture (hand close), the subsequent PR-based myoelectric control performance was fully restored from eight different electrode shift scenarios, with 1 cm in either or both perpendicular and parallel directions. The classification accuracies with PI framework were not significant before and after the shift ( 0.001). The advantage of restoring performance fully in just one second made it a practical solution to improve the robustness of PR-based myoelectric control systems in a wide range of real-world applications.
Collapse
|
19
|
Kyeong S, Kim J. Enhanced torque estimation method from multi-channel surface electromyography compensating electrode location variation. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:3965-3968. [PMID: 33018868 DOI: 10.1109/embc44109.2020.9176498] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Recognizing human intentions from the human counterpart is very important in human-robot interaction applications. Surface electromyography(sEMG) has been considered as a potential source for motion intention because the signal represents the on-set timing and amplitude of muscle activation. It is also reported that sEMG has the advantage of knowing body movements ahead of actual movement. However, sEMG based applications suffer from electrode location variation because sEMG shows different characteristics whenever the skin condition is different. They need to recreate the estimation model if electrodes are attached to different locations or conditions. In this paper, we developed a sEMG torque estimation model for electrode location variation. A decomposition model of sEMG signals was developed to discriminate the muscle source signals for electrode location variation, and we verified this model without making a new torque estimation model. Torque estimation accuracy using the proposed method was increased by 24.8% and torque prediction accuracy was increased by 47.7% for the electrode location variation in comparison with the method without decomposition. Therefore, the proposed sEMG decomposition method showed an enhancement in torque estimation for electrode location variation.
Collapse
|
20
|
Kim M, Chung WK, Kim K. Subject-Independent sEMG Pattern Recognition by Using a Muscle Source Activation Model. IEEE Robot Autom Lett 2020. [DOI: 10.1109/lra.2020.3006824] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
21
|
He J, Joshi MV, Chang J, Jiang N. Efficient correction of armband rotation for myoelectric-based gesture control interface. J Neural Eng 2020; 17:036025. [PMID: 32244236 DOI: 10.1088/1741-2552/ab8682] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE The appearance of commercial myoelectric armbands has greatly increased the portability and convenience of myoelectric controlled interfaces (MCIs). However, one limitation of the current state-of-the-art myoelectric control algorithms is that they have poor robustness against armband displacements, especially rotation, leading to great algorithmic performance degradation. The traditional remedy, retraining the interface, requires the data collection of all gestures and is impractical in many applications. The recently proposed position verification (PV) framework focused on quickly identifying and correcting the electrode positions after the displacement, showing the potential to restore the performance of MCI in a faster way. However, its online effectiveness is still yet to be validated. APPROACH This work proposed a novel algorithm of identifying the rotation direction to improve the efficiency of the PV framework and demonstrated the real-time capability of the PV framework using a commercially available armband. MAIN RESULTS The results showed that with PV, a 1.5-cm rotation could be corrected with an average of 3.1 ± 1.5 interactive adjustments, equivalent to around 15.5 ± 7.5 s, which was greatly reduced compared to retraining. There was no significant difference in the real-time control performance between before the armband displacement and after the PV correction. SIGNIFICANCE To the best of our knowledge, this study was the first maintaining pattern recognition-based myoelectric control performance in the presence of electrode shifts without recollecting the entire training data. It suggested the feasibility of the PV framework used in the myoelectric armband and MCI for practical applications.
Collapse
Affiliation(s)
- Jiayuan He
- Department of Systems Design Engineering, Faculty of Engineering, University of Waterloo, Waterloo, Canada. Author to whom any correspondence should be addressed
| | | | | | | |
Collapse
|
22
|
Huang G, Xian Z, Tang F, Li L, Zhang L, Zhang Z. Low-density surface electromyographic patterns under electrode shift: Characterization and NMF-based classification. Biomed Signal Process Control 2020. [DOI: 10.1016/j.bspc.2020.101890] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
23
|
Wu L, Zhang X, Chen X, Chen X. Visualized Evidences for Detecting Novelty in Myoelectric Pattern Recognition using 3D Convolutional Neural Networks .. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2019:2641-2644. [PMID: 31946438 DOI: 10.1109/embc.2019.8857199] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Although myoelectric pattern recognition (MPR) has been considered as a milestone technique to enable dexterous control of multiple degrees of freedom, outlier data interference (i.e., novelty) is a big issue affecting stability of conventional MPR control and its wide applications. Inspired by video classification techniques, we propose a novel method using 3-dimensional (3D) convolutional neural network (CNN) for extracting sufficient spatial-temporal features from high-density surface electromyogram (EMG) recordings processed as a video stream where time-varying information of muscular activity was taken into account. Given the targeted task patterns accurately characterized by the proposed method, it is straightforward to discriminate and then reject outlier data interferences regarded as untargeted and unlearnt patterns. Meanwhile, the strength of the 3D CNN in discriminating targeted task patterns through spatial-temporal features is further visualized and confirmed by t-Distributed Stochastic Neighbor Embedding (t-SNE). The performance of the proposed method was evaluated with surface EMG recorded by two 6 × 8 electrode arrays placed over forearm flexors and extensors of 3 subjects performing 7 targeted and 4 outlier motions. The visualized results by t-SNE showed the targeted task patterns were separated with each concentrated into a small region, while the outlier patterns were scattered around. On this basis, a traditional Mahalanobis Distance (MD) based method was applied for novelty detection and targeted task classification. Finally, the proposed method yielded averaged error rate of 10.98% for the targeted task patterns and <; 5% for all outlier data, respectively, which outperformed a common baseline method. All these findings demonstrate the advantage of characterizing myoelectric patterns using the proposed method and the potential of applying it to reject outlier data interference.
Collapse
|
24
|
Lara J, Paskaranandavadivel N, Cheng LK. Effect of Segmentation Parameters on Classification Accuracy of High-Density EMG recordings. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2019:6229-6232. [PMID: 31947266 DOI: 10.1109/embc.2019.8856809] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Electromyography (EMG) based control systems rely on the accurate identification of patterns extracted from signal features to predict the corresponding movement. The selection of segmentation window parameters and their impact on overall accuracy of classifiers has been previously studied for systems with a low number of EMG channels (<; 16). In this study a High-density EMG electrode array was used to evaluate the impact of the parameters when a high number of channels (128) was recorded. Findings show that in combination with high channel counts the impact of window length and overlap were marginal (<; 2% and <; 1% respectively). The number of channels was found to have direct correlation with achieved accuracy, with an improvement of up to 19.5 ± 4.5% in classification accuracy when increasing from 4 to 128 channels.
Collapse
|
25
|
Zhang X, Wu L, Yu B, Chen X, Chen X. Adaptive Calibration of Electrode Array Shifts Enables Robust Myoelectric Control. IEEE Trans Biomed Eng 2019; 67:1947-1957. [PMID: 31715562 DOI: 10.1109/tbme.2019.2952890] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
OBJECTIVE The objective of this work is to develop a novel method for adaptive calibration of the electrode array shifts toward achieving robust myoelectric pattern-recognition control. METHODS This work is inspired by the idea of image object detection when high-density surface electromyogram signals recorded from a two-dimensional electrode array carry rich spatial information and can serve as a muscular activation image. A convolutional neural network involving transfer learning (from a network for image recognition) is used to learn muscular activity patterns at an original/baseline position of an electrode array. The shift of an electrode array can be estimated by identifying and matching partially overlapped regions between the training images of muscular activation and the testing images at any other position, in an unsupervised manner. Given the calibration of an electrode array shift, the identification of muscular activity patterns is implemented accordingly. The performance of the proposed method was evaluated with data recorded by a 10 × 10 electrode array placed over the forearm extensors of 10 subjects performing 6 wrist and finger extension tasks. RESULTS The proposed method achieved high task classification accuracies around 95% and outperformed five traditional methods under conditions with multiple designated shifts in offline experiments and a random shift in online testing. CONCLUSION The proposed method is demonstrated to be a promising solution for the automatic and adaptive calibration of electrode array shifts. SIGNIFICANCE This work will enhance the robustness of myoelectric control systems.
Collapse
|
26
|
Lv B, Sheng X, Zhu X. Improving Myoelectric Pattern Recognition Robustness to Electrode Shift by Autoencoder. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2019; 2018:5652-5655. [PMID: 30441618 DOI: 10.1109/embc.2018.8513525] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
It is evident that the electrode shift will result in a degradation of myoelectric pattern recognition classification accuracy, which is inevitable during the prosthetic socket donning and doffing. To cope with this limitation, we propose an unsupervised feature extraction method called sparse autoencoder (SAE) to extract the robust spatial structure and correlation of high density (HD) electromyography (EMG). The algorithm is evaluated on nine intact-limbed subjects and one amputee. The experimental results show that SAE achieves lower classification error without shift, and significantly decrease the sensitivity to electrode shift with ±1 cm compared with the timedomain and autoregressive features (TDAR). Furthermore, SAE is not sensitive to the shift direction that is perpendicular to the muscle fibers. The promising results of this study make great contribution to promoting the applications of pattern recognition based myoelectric control system in real-world condition.
Collapse
|
27
|
Pan L, Crouch DL, Huang H. Comparing EMG-Based Human-Machine Interfaces for Estimating Continuous, Coordinated Movements. IEEE Trans Neural Syst Rehabil Eng 2019; 27:2145-2154. [PMID: 31478862 DOI: 10.1109/tnsre.2019.2937929] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Electromyography (EMG)-based interfaces are trending toward continuous, simultaneous control with multiple degrees of freedom. Emerging methods range from data-driven approaches to biomechanical model-based methods. However, there has been no direct comparison between these two types of continuous EMG-based interfaces. The aim of this study was to compare a musculoskeletal model (MM) with two data-driven approaches, linear regression (LR) and artificial neural network (ANN), for predicting continuous wrist and hand motions for EMG-based interfaces. Six able-bodied subjects and one transradial amputee subject performed (missing) metacarpophalangeal (MCP) and wrist flexion/extension, simultaneously or independently, while four EMG signals were recorded from forearm muscles. To add variation to the EMG signals, the subjects repeated the MCP and wrist motions at various upper extremity postures. For each subject, the EMG signals collected from the neutral posture were used to build the EMG interfaces; the EMG signals collected from all postures were used to evaluate the interfaces. The performance of the interface was quantified by Pearson's correlation coefficient (r) and the normalized root mean square error (NRMSE) between measured and estimated joint angles. The results demonstrated that the MM predicted movements more accurately, with higher r values and lower NRMSE, than either LR or ANN. Similar results were observed in the transradial amputee. Additionally, the variation in r across postures, an indicator of reliability against posture changes, was significantly lower (better) for the MM than for either LR or ANN. Our findings suggest that incorporating musculoskeletal knowledge into EMG-based human-machine interfaces could improve the estimation of continuous, coordinated motion.
Collapse
|
28
|
Gu Y, Yang D, Osborn L, Candrea D, Liu H, Thakor N. An adaptive socket with auto-adjusting air bladders for interfacing transhumeral prosthesis: A pilot study. Proc Inst Mech Eng H 2019; 233:812-822. [PMID: 31165676 DOI: 10.1177/0954411919853960] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Comfort is a critical aspect in the application of wearable device, such as rehabilitation robots and upper limb prostheses. As a physical interface between human body and prosthetic limb, the socket and its comfort largely contribute to the user's acceptance. Traditional sockets are static, lacking dynamic adjustment mechanism for the contact pressure. To ensure a reliable suspension during daily activities, the socket is usually designed to be tightly attached, with a large stress, on the residual limb, which may introduce considerable discomfort during long-term use. In this article, we present a novel adaptive transhumeral socket, in which we employ four independent bladders contacting with the stump. Not only can these bladders provide a necessary suspension for the device but also form an air cushion (soft body) that helps relieve the pressure concentration between the biological body and physical prosthesis. In real time, this adaptive socket continuously monitors the limb posture, the operating load, and the contacting pressure between the socket and the limb, and then dynamically adjusts the clamping force to ensure a reliable attachment during various daily activities. Since well adapting to the contours of the stump, the bladders can effectively accommodate the volume change of the stump, making a balanced load distribution on load-tolerant areas. Through modeling and numerical analysis, we established a dynamic strategy for estimating the external load and an automatic scheme for adjusting the bladders' air pressure. Finally, a close-loop control was constructed based on the contact pressure measured by our self-developed force sensors. Our preliminary experiments on one normal (i.e. non-amputee) subject verified the effectiveness of the proposed method, showing that the adaptive socket can considerably reduce the socket-limb contact pressure while sustaining a secure suspension on the upper arm.
Collapse
Affiliation(s)
- Yikun Gu
- 1 State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, China
| | - Dapeng Yang
- 1 State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, China
| | - Luke Osborn
- 2 Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
| | - Daniel Candrea
- 3 Biomedical Engineering, Duke University, Durham, NC, USA
| | - Hong Liu
- 1 State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, China
| | - Nitish Thakor
- 2 Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|
29
|
Prahm C, Schulz A, Paaben B, Schoisswohl J, Kaniusas E, Dorffner G, Hammer B, Aszmann O. Counteracting Electrode Shifts in Upper-Limb Prosthesis Control via Transfer Learning. IEEE Trans Neural Syst Rehabil Eng 2019; 27:956-962. [PMID: 30908234 DOI: 10.1109/tnsre.2019.2907200] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Research on machine learning approaches for upper-limb prosthesis control has shown impressive progress. However, translating these results from the lab to patient's everyday lives remains a challenge because advanced control schemes tend to break down under everyday disturbances, such as electrode shifts. Recently, it has been suggested to apply adaptive transfer learning to counteract electrode shifts using as little newly recorded training data as possible. In this paper, we present a novel, simple version of transfer learning and provide the first user study demonstrating the effectiveness of transfer learning to counteract electrode shifts. For this purpose, we introduce the novel Box and Beans test to evaluate prosthesis proficiency and compare user performance with an initial simple pattern recognition system, the system under electrode shifts, and the system after transfer learning. Our results show that transfer learning could significantly alleviate the impact of electrode shifts on user performance in the Box and Beans test.
Collapse
|
30
|
Tian L, Zimmerman B, Akhtar A, Yu KJ, Moore M, Wu J, Larsen RJ, Lee JW, Li J, Liu Y, Metzger B, Qu S, Guo X, Mathewson KE, Fan JA, Cornman J, Fatina M, Xie Z, Ma Y, Zhang J, Zhang Y, Dolcos F, Fabiani M, Gratton G, Bretl T, Hargrove LJ, Braun PV, Huang Y, Rogers JA. Large-area MRI-compatible epidermal electronic interfaces for prosthetic control and cognitive monitoring. Nat Biomed Eng 2019; 3:194-205. [PMID: 30948811 DOI: 10.1038/s41551-019-0347-x] [Citation(s) in RCA: 145] [Impact Index Per Article: 29.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2018] [Accepted: 01/02/2019] [Indexed: 12/11/2022]
Abstract
Skin-interfaced medical devices are critically important for diagnosing disease, monitoring physiological health and establishing control interfaces with prosthetics, computer systems and wearable robotic devices. Skin-like epidermal electronic technologies can support these use cases in soft and ultrathin materials that conformally interface with the skin in a manner that is mechanically and thermally imperceptible. Nevertheless, schemes so far have limited the overall sizes of these devices to less than a few square centimetres. Here, we present materials, device structures, handling and mounting methods, and manufacturing approaches that enable epidermal electronic interfaces that are orders of magnitude larger than previously realized. As a proof-of-concept, we demonstrate devices for electrophysiological recordings that enable coverage of the full scalp and the full circumference of the forearm. Filamentary conductive architectures in open-network designs minimize radio frequency-induced eddy currents, forming the basis for structural and functional compatibility with magnetic resonance imaging. We demonstrate the use of the large-area interfaces for the multifunctional control of a transhumeral prosthesis by patients who have undergone targeted muscle-reinnervation surgery, in long-term electroencephalography, and in simultaneous electroencephalography and structural and functional magnetic resonance imaging.
Collapse
Affiliation(s)
- Limei Tian
- Department of Biomedical Engineering, Texas A&M University, College Station, TX, USA.,Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Benjamin Zimmerman
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Aadeel Akhtar
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Ki Jun Yu
- School of Electrical and Electronic Engineering, Yonsei University, Seoul, Republic of Korea
| | - Matthew Moore
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Jian Wu
- Applied Mechanics Laboratory, Department of Engineering Mechanics, Center for Mechanics and Materials and Center for Flexible Electronics Technology, Tsinghua University, Beijing, China
| | - Ryan J Larsen
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Jung Woo Lee
- Department of Materials Science and Engineering, Pusan National University, Busan, Republic of Korea
| | - Jinghua Li
- Department of Materials Science and Engineering, Frederick Seitz Materials Research Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Yuhao Liu
- Department of Materials Science and Engineering, Frederick Seitz Materials Research Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Brian Metzger
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Subing Qu
- Department of Materials Science and Engineering, Frederick Seitz Materials Research Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Xiaogang Guo
- Applied Mechanics Laboratory, Department of Engineering Mechanics, Center for Mechanics and Materials and Center for Flexible Electronics Technology, Tsinghua University, Beijing, China
| | - Kyle E Mathewson
- Department of Psychology, University of Alberta, Edmonton, Alberta, Canada
| | - Jonathan A Fan
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
| | - Jesse Cornman
- Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Michael Fatina
- Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Zhaoqian Xie
- Departments of Civil and Environmental Engineering, Mechanical Engineering, and Materials Science and Engineering, Northwestern University, Evanston, IL, USA
| | - Yinji Ma
- Applied Mechanics Laboratory, Department of Engineering Mechanics, Center for Mechanics and Materials and Center for Flexible Electronics Technology, Tsinghua University, Beijing, China
| | - Jue Zhang
- Department of Materials Science and Engineering, Frederick Seitz Materials Research Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Yihui Zhang
- Applied Mechanics Laboratory, Department of Engineering Mechanics, Center for Mechanics and Materials and Center for Flexible Electronics Technology, Tsinghua University, Beijing, China
| | - Florin Dolcos
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Monica Fabiani
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Gabriele Gratton
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Timothy Bretl
- Department of Aerospace Engineering, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Levi J Hargrove
- Feinberg School of Medicine, Northwestern University, Shirley Ryan AbilityLab, Chicago, IL, USA
| | - Paul V Braun
- Department of Materials Science and Engineering, Frederick Seitz Materials Research Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL, USA
| | - Yonggang Huang
- Departments of Civil and Environmental Engineering, Mechanical Engineering, and Materials Science and Engineering, Northwestern University, Evanston, IL, USA
| | - John A Rogers
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA. .,Department of Materials Science and Engineering, Frederick Seitz Materials Research Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL, USA. .,Departments of Materials Science and Engineering, Biomedical Engineering, Neurological Surgery, Chemistry, Mechanical Engineering, Electrical Engineering and Computer Science, Simpson Querrey Institute and Feinberg Medical School Center for Bio-Integrated Electronics, Northwestern University, Evanston, IL, USA.
| |
Collapse
|
31
|
Dai C, Hu X. Extracting and Classifying Spatial Muscle Activation Patterns in Forearm Flexor Muscles Using High-Density Electromyogram Recordings. Int J Neural Syst 2019; 29:1850025. [DOI: 10.1142/s0129065718500259] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
The human hand is capable of producing versatile yet precise movements largely owing to the complex neuromuscular systems that control our finger movement. This study seeks to quantify the spatial activation patterns of the forearm flexor muscles during individualized finger flexions. High-density (HD) surface electromyogram (sEMG) signals of forearm flexor muscles were obtained, and individual motor units were decomposed from the sEMG. Both macro-level spatial patterns of EMG activity and micro-level motor unit distributions were used to systematically characterize the forearm flexor activation patterns. Different features capturing the spatial patterns were extracted, and the unique patterns of forearm flexor activation were then quantified using pattern recognition approaches. We found that the forearm flexor spatial activation during the ring finger flexion was mostly distinct from other fingers, whereas the activation patterns of the middle finger were least distinguishable. However, all the different activation patterns can still be classified in high accuracy (94–100%) using pattern recognition. Our findings indicate that the partial overlapping of neural activation can limit accurate identification of specific finger movement based on limited recordings and sEMG features, and that HD sEMG recordings capturing detailed spatial activation patterns at both macro- and micro-levels are needed.
Collapse
Affiliation(s)
- Chenyun Dai
- Joint Department of Biomedical Engineering, University of North Carolina — Chapel Hill and North Carolina State University, Raleigh, NC, USA
| | - Xiaogang Hu
- Joint Department of Biomedical Engineering, University of North Carolina — Chapel Hill and North Carolina State University, Raleigh, NC, USA
| |
Collapse
|
32
|
He J, Sheng X, Zhu X, Jiang N. Electrode Density Affects the Robustness of Myoelectric Pattern Recognition System With and Without Electrode Shift. IEEE J Biomed Health Inform 2019; 23:156-163. [DOI: 10.1109/jbhi.2018.2805760] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
33
|
Improving the functionality, robustness, and adaptability of myoelectric control for dexterous motion restoration. Exp Brain Res 2018; 237:291-311. [PMID: 30506366 DOI: 10.1007/s00221-018-5441-x] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2017] [Accepted: 11/20/2018] [Indexed: 10/27/2022]
Abstract
The development of advanced and effective human-machine interfaces, especially for amputees to control their prostheses, is very high priority and a very active area of research. An intuitive control method should retain an adequate level of functionality for dexterous operation, provide robustness against confounding factors, and supply adaptability for diverse long-term usage, all of which are current problems being tackled by researchers. This paper reviews the state-of-the-art, as well as, the limitations of current myoelectric signal control (MSC) methods. To address the research topic on functionality, we review different approaches to prosthetic hand control (DOF configuration, discrete or simultaneous, etc.), and how well the control is performed (accuracy, response, intuitiveness, etc.). To address the research on robustness, we review the confounding factors (limb positions, electrode shift, force variance, and inadvertent activity) that affect the stability of the control performance. Lastly, to address adaptability, we review the strategies that can automatically adjust the classifier for different individuals and for long-term usage. This review provides a thorough overview of the current MSC methods and helps highlight the current areas of research focus and resulting clinic usability for the MSC methods for upper-limb prostheses.
Collapse
|
34
|
Kim M, Kim K, Chung WK. Simple and Fast Compensation of sEMG Interface Rotation for Robust Hand Motion Recognition. IEEE Trans Neural Syst Rehabil Eng 2018; 26:2397-2406. [PMID: 30371382 DOI: 10.1109/tnsre.2018.2878439] [Citation(s) in RCA: 26] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Surface electromyography (sEMG) measurements have demonstrated the potential to recognize complex hand motions. In addition, sEMG enables natural recognition without disturbing movements, and thus, can be used in various fields such as teleoperation, assistant robots, and prosthetic hands. However, sEMG signals highly depend on electrode placements due to the complex muscle structures. A shift of the electrode can lead to inconsistent signal measurement. Thus, sEMG-based recognition is not practical for applications that require long-term and repeated usage. This paper proposes compensation of sEMG interface rotation for robust motion recognition. Once the relationship between sEMG signals and motions is trained, additional training for different electrode configurations is not necessary for a band-type interface. The proposed process is simple and fast. The interface rotation can be compensated for by performing only a single motion for approximately 2 s. The single motion for compensation is dependent on the muscle properties of the user. Generally, ulnar deviation may work. To demonstrate the proposed compensation, recognition of five hand motions is conducted. The experimental results indicate that the proposed compensation can cover the overall range of rotation. In addition, the proposed compensation is validated with a transradial amputee.
Collapse
|
35
|
Kyranou I, Vijayakumar S, Erden MS. Causes of Performance Degradation in Non-invasive Electromyographic Pattern Recognition in Upper Limb Prostheses. Front Neurorobot 2018; 12:58. [PMID: 30297994 PMCID: PMC6160857 DOI: 10.3389/fnbot.2018.00058] [Citation(s) in RCA: 34] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2018] [Accepted: 08/27/2018] [Indexed: 11/29/2022] Open
Abstract
Surface Electromyography (EMG)-based pattern recognition methods have been investigated over the past years as a means of controlling upper limb prostheses. Despite the very good reported performance of myoelectric controlled prosthetic hands in lab conditions, real-time performance in everyday life conditions is not as robust and reliable, explaining the limited clinical use of pattern recognition control. The main reason behind the instability of myoelectric pattern recognition control is that EMG signals are non-stationary in real-life environments and present a lot of variability over time and across subjects, hence affecting the system's performance. This can be the result of one or many combined changes, such as muscle fatigue, electrode displacement, difference in arm posture, user adaptation on the device over time and inter-subject singularity. In this paper an extensive literature review is performed to present the causes of the drift of EMG signals, ways of detecting them and possible techniques to counteract for their effects in the application of upper limb prostheses. The suggested techniques are organized in a table that can be used to recognize possible problems in the clinical application of EMG-based pattern recognition methods for upper limb prosthesis applications and state-of-the-art methods to deal with such problems.
Collapse
Affiliation(s)
- Iris Kyranou
- Edinburgh Centre of Robotics, Edinburgh, United Kingdom
- School of Informatics, Institute of Perception, Action and Behaviour, University of Edinburgh, Edinburgh, United Kingdom
- School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh, United Kingdom
| | - Sethu Vijayakumar
- Edinburgh Centre of Robotics, Edinburgh, United Kingdom
- School of Informatics, Institute of Perception, Action and Behaviour, University of Edinburgh, Edinburgh, United Kingdom
| | - Mustafa Suphi Erden
- Edinburgh Centre of Robotics, Edinburgh, United Kingdom
- School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh, United Kingdom
| |
Collapse
|
36
|
Pan L, Harmody A, Huang H. A Reliable Multi-User EMG Interface Based on A Generic-Musculoskeletal Model against Loading Weight Changes . ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2018; 2018:2104-2107. [PMID: 30440818 DOI: 10.1109/embc.2018.8512685] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
The reliability of myoelectric control is important to ensure the performance of prostheses during daily use. Recently, we proposed a multi-user neural-machine interface based on a generic musculoskeletal model to simultaneously and continuously estimate flexion/extension movements at the metacarpophalangeal (MCP) and wrist joints from surface electromyography (EMG) signals. Our previous results demonstrated that the multi-user EMG interface was reliable against upper limb posture changes. However, the reliability of the interface against different loading weights, which is an important factor that would decrease the performance of myoelectric control and be tested during the occupational therapy for myoelectric prosthesis users, is still unclear. In this study, we aimed to evaluate the reliability of the generic model over different loading weights. Four able-bodied subjects were tested in this study. Subjects performed a virtual hand/wrist posture matching task with three different loading weights (no weight, 1.25 Lbs, and 2.5 Lbs). All subjects accomplished all the assigned virtual tasks. The on-line experimental results showed that performance with different loading weights was very close. The results demonstrated that the multi-user EMG interface was reliable against the different loading weights, indicating it has potential to promote the myoelectric control into clinical applications.
Collapse
|
37
|
Increasing the robustness against force variation in EMG motion classification by common spatial patterns. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2018; 2017:406-409. [PMID: 29059896 DOI: 10.1109/embc.2017.8036848] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
In the practical use of an electromyography (EMG) pattern-recognition based myoelectric prosthesis, the variation of force levels to do a motion would be inevitable, which will cause a change of EMG patterns. Therefore, the force variation will decay the performance of a trained classifier. In this study, the common spatial pattern (CSP) method was proposed with an attempt to improve the robustness of EMG-PR based classifier against force variation. The EMG signals were acquired from three able-bodied subjects when they were performing the motions at low, medium, and high force levels, respectively. And in the pattern recognition, CSP features were extracted from the EMG signals for motion classification. By comparing the classification accuracies between the CSP and the commonly used time-domain (TD) features, the CSP features showed a better robustness against force variation with an increment of 5.3% of the average classification accuracy. Especially, the classification accuracy of a classifier was 84.2% when tested at low force level by using CSP features, which was 18.5% higher than that of the TD features. These preliminary results suggest that using CSP features may increase the robustness of EMG-based myoelectric control.
Collapse
|
38
|
Pan L, Crouch DL, Huang HH. Myoelectric Control Based on A Generic Musculoskeletal Model: Towards A Multi-User Neural-Machine Interface. IEEE Trans Neural Syst Rehabil Eng 2018; 26:1435-1442. [PMID: 29994312 DOI: 10.1109/tnsre.2018.2838448] [Citation(s) in RCA: 29] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
This study aimed to develop a novel electromyography (EMG)-based neural-machine interface (NMI) that is user-generic for continuously predicting coordinated motion between metacarpophalangeal (MCP) and wrist flexion/extension. The NMI requires a minimum calibration procedure that only involves capturing maximal voluntary muscle contraction for the monitored muscles for individual users. At the center of the NMI is a user-generic musculoskeletal model based on the experimental data collected from 6 able-bodied (AB) subjects and 9 different upper limb postures. The generic model was evaluated on-line on both AB subjects and a transradial amputee. The subjects were instructed to perform a virtual hand/wrist posture matching task with different upper limb postures. The on-line performance of the generic model was also compared with that of the musculoskeletal model customized to each individual user (called "specific model"). All subjects accomplished the assigned virtual tasks while using the user-generic NMI, although the AB subjects produced better performance than the amputee subject. Interestingly, compared to the specific model, the generic model produced comparable completion time, a reduced number of overshoots, and improved path efficiency in the virtual hand/wrist posture matching task. The results suggested that it is possible to design an EMG-driven NMI based on a musculoskeletal model that could fit multiple users, including upper limb amputees, for predicting coordinated MCP and wrist motion. The present new method might address the challenges of existing advanced EMG-based NMI that require frequent and lengthy customization and calibration. Our future research will focus on evaluating the developed NMI for powered prosthetic arms.
Collapse
|
39
|
A canonical correlation analysis based EMG classification algorithm for eliminating electrode shift effect. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2017; 2016:867-870. [PMID: 28268461 DOI: 10.1109/embc.2016.7590838] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Motion classification system based on surface Electromyography (sEMG) pattern recognition has achieved good results in experimental condition. But it is still a challenge for clinical implement and practical application. Many factors contribute to the difficulty of clinical use of the EMG based dexterous control. The most obvious and important is the noise in the EMG signal caused by electrode shift, muscle fatigue, motion artifact, inherent instability of signal and biological signals such as Electrocardiogram. In this paper, a novel method based on Canonical Correlation Analysis (CCA) was developed to eliminate the reduction of classification accuracy caused by electrode shift. The average classification accuracy of our method were above 95% for the healthy subjects. In the process, we validated the influence of electrode shift on motion classification accuracy and discovered the strong correlation with correlation coefficient of >0.9 between shift position data and normal position data.
Collapse
|
40
|
Lu H, Zhang H, Wang Z, Wang R, Li G. Using spatial features for classification of combined motions based on common spatial pattern. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2017; 2017:2271-2274. [PMID: 29060350 DOI: 10.1109/embc.2017.8037308] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Motion recognition is an important application of electromyography (EMG) analysis. While discrete motions such as hand open, hand close and wrist pronation have been extensively investigated, studies on combined motions involving two or more degrees of freedom (DOFs) are relatively few and the classification accuracy of the combined motions reported in previous studies is barely satisfactory. To improve the accuracy of the combined motion recognition, common spatial pattern (CSP) was employed in this study to extract spatial features. 18 forearm motion classes, consisted of 8 discrete motions and 10 combined motions, were classified by the proposed method. Our results showed that the accuracy rate of CSP features was 96.3%, which outperformed the commonly used time-domain (TD) features by 2.4% and TD combined with auto-regression coefficients (TDAR) by 0.6%. Moreover, CSP features cost noticeable much less time than TDAR and quite less time than TD in testing. These results suggest that CSP features could be a better feature set for multi-DOF myoelectric control than conventional features.
Collapse
|
41
|
Improving the Robustness of Real-Time Myoelectric Pattern Recognition against Arm Position Changes in Transradial Amputees. BIOMED RESEARCH INTERNATIONAL 2017; 2017:5090454. [PMID: 28523276 PMCID: PMC5421097 DOI: 10.1155/2017/5090454] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/26/2016] [Revised: 03/13/2017] [Accepted: 04/02/2017] [Indexed: 11/24/2022]
Abstract
Previous studies have showed that arm position variations would significantly degrade the classification performance of myoelectric pattern-recognition-based prosthetic control, and the cascade classifier (CC) and multiposition classifier (MPC) have been proposed to minimize such degradation in offline scenarios. However, it remains unknown whether these proposed approaches could also perform well in the clinical use of a multifunctional prosthesis control. In this study, the online effect of arm position variation on motion identification was evaluated by using a motion-test environment (MTE) developed to mimic the real-time control of myoelectric prostheses. The performance of different classifier configurations in reducing the impact of arm position variation was investigated using four real-time metrics based on dataset obtained from transradial amputees. The results of this study showed that, compared to the commonly used motion classification method, the CC and MPC configurations improved the real-time performance across seven classes of movements in five different arm positions (8.7% and 12.7% increments of motion completion rate, resp.). The results also indicated that high offline classification accuracy might not ensure good real-time performance under variable arm positions, which necessitated the investigation of the real-time control performance to gain proper insight on the clinical implementation of EMG-pattern-recognition-based controllers for limb amputees.
Collapse
|
42
|
Jiang Y, Togane M, Lu B, Yokoi H. sEMG Sensor Using Polypyrrole-Coated Nonwoven Fabric Sheet for Practical Control of Prosthetic Hand. Front Neurosci 2017; 11:33. [PMID: 28220058 PMCID: PMC5292570 DOI: 10.3389/fnins.2017.00033] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2016] [Accepted: 01/17/2017] [Indexed: 11/29/2022] Open
Abstract
One of the greatest challenges of using a myoelectric prosthetic hand in daily life is to conveniently measure stable myoelectric signals. This study proposes a novel surface electromyography (sEMG) sensor using polypyrrole-coated nonwoven fabric sheet as electrodes (PPy electrodes) to allow people with disabilities to control prosthetic limbs. The PPy electrodes are sewn on an elastic band to guarantee close contact with the skin and thus reduce the contact electrical impedance between the electrodes and the skin. The sensor is highly customizable to fit the size and the shape of the stump so that people with disabilities can attach the sensor by themselves. The performance of the proposed sensor was investigated experimentally by comparing measurements of Ag/AgCl electrodes with electrolytic gel and the sEMG from the same muscle fibers. The high correlation coefficient (0.87) between the two types of sensors suggests the effectiveness of the proposed sensor. Another experiment of sEMG pattern recognition to control myoelectric prosthetic hands showed that the PPy electrodes are as effective as Ag/AgCl electrodes for measuring sEMG signals for practical myoelectric control. We also investigated the relation between the myoelectric signals' signal-to-noise ratio and the source impedances by simultaneously measuring the source impedances and the myoelectric signals with a switching circuit. The results showed that differences in both the norm and the phase of the source impedance greatly affect the common mode noise in the signal.
Collapse
Affiliation(s)
- Yinlai Jiang
- Brain Science Inspired Life Support Research Center, University of Electro-Communications Tokyo, Japan
| | - Masami Togane
- Department of Mechanical Engineering and Intelligent Systems, University of Electro-Communications Tokyo, Japan
| | - Baoliang Lu
- Department of Computer Science and Engineering, Shanghai Jiao Tong University Shanghai, China
| | - Hiroshi Yokoi
- Brain Science Inspired Life Support Research Center, University of Electro-CommunicationsTokyo, Japan; Department of Mechanical Engineering and Intelligent Systems, University of Electro-CommunicationsTokyo, Japan
| |
Collapse
|
43
|
He J, Zhu X. Combining Improved Gray-Level Co-Occurrence Matrix With High Density Grid for Myoelectric Control Robustness to Electrode Shift. IEEE Trans Neural Syst Rehabil Eng 2016; 25:1539-1548. [PMID: 28026779 DOI: 10.1109/tnsre.2016.2644264] [Citation(s) in RCA: 27] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Pattern recognition-based myoelectric control is greatly influenced by electrode shift, which is inevitable during prosthesis donning and doffing. This study used gray-level co-occurrence matrix (GLCM) to represent the spatial distribution among high density (HD) electrodes and improved its calculation based on the using condition of myoelectric system, proposing a new feature, iGLCM, to improve the robustness of the system. The effects of its two parameters, quantization level and input data, were first evaluated and it was found that improved discrete Fourier transform (iDFT) performed better than the other three (time-domain, autoregressive, root mean square) as the input data of iGLCM, and increasing quantization level did not significantly decrease the error rate of iGLCM when it was above 8. The performance of iGLCM with iDFT as input data and 8 as quantization level was subsequently compared with previous robust approaches (time domain autoregressive, variogram, common spatial pattern and optimal less channel configuration) and its input data, iDFT. It was showed that iGLCM achieved comparable classification accuracy without shift, and significantly decreased the sensitivity to electrode shift with 1 cm (p < 0.05). More importantly, it could reduce the perpendicular shift distance to half interelectrode distance with the electrodes worn as a band around the circumference of the forearm. Combined with the small interelectrode distance of HD electrodes, it provided a way to control the effect of perpendicular shifts fundamentally, which were the main source of performance degradation. Finally, the analysis of feature space revealed that the robustness was improved by discarding information sensitivity to shift and keeping as much as useful information. This study highlighted the importance of HD electrodes in robust myoelectric control, and the outcome would help the design of robust control system based on pattern recognition and promote its application in real-world condition.
Collapse
|
44
|
Betthauser JL, Hunt CL, Osborn LE, Kaliki RR, Thakor NV. Limb-position robust classification of myoelectric signals for prosthesis control using sparse representations. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2016; 2016:6373-6376. [PMID: 28325032 PMCID: PMC10947745 DOI: 10.1109/embc.2016.7592186] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
The fundamental objective in non-invasive myoelectric prosthesis control is to determine the user's intended movements from corresponding skin-surface recorded electromyographic (sEMG) activation signals as quickly and accurately as possible. Linear Discriminant Analysis (LDA) has emerged as the de facto standard for real-time movement classification due to its ease of use, calculation speed, and remarkable classification accuracy under controlled training conditions. However, performance of cluster-based methods like LDA for sEMG pattern recognition degrades significantly when real-world testing conditions do not resemble the trained conditions, limiting the utility of myoelectrically controlled prosthesis devices. We propose an enhanced classification method that is more robust to generic deviations from training conditions by constructing sparse representations of the input data dictionary comprised of sEMG time-frequency features. We apply our method in the context of upper-limb position changes to demonstrate pattern recognition robustness and improvement over LDA across discrete positions not explicitly trained. For single position training we report an accuracy improvement in untrained positions of 7.95%, p ≪ .001, in addition to significant accuracy improvements across all multiposition training conditions, p <; .001.
Collapse
|