1
|
Wang B, Li J, Hargrove L, Kamavuako EN. Unravelling Influence Factors in Pattern Recognition Myoelectric Control Systems: The Impact of Limb Positions and Electrode Shifts. SENSORS (BASEL, SWITZERLAND) 2024; 24:4840. [PMID: 39123885 PMCID: PMC11314973 DOI: 10.3390/s24154840] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/19/2024] [Revised: 07/24/2024] [Accepted: 07/25/2024] [Indexed: 08/12/2024]
Abstract
Pattern recognition (PR)-based myoelectric control systems can naturally provide multifunctional and intuitive control of upper limb prostheses and restore lost limb function, but understanding their robustness remains an open scientific question. This study investigates how limb positions and electrode shifts-two factors that have been suggested to cause classification deterioration-affect classifiers' performance by quantifying changes in the class distribution using each factor as a class and computing the repeatability and modified separability indices. Ten intact-limb participants took part in the study. Linear discriminant analysis (LDA) was used as the classifier. The results confirmed previous studies that limb positions and electrode shifts deteriorate classification performance (14-21% decrease) with no difference between factors (p > 0.05). When considering limb positions and electrode shifts as classes, we could classify them with an accuracy of 96.13 ± 1.44% and 65.40 ± 8.23% for single and all motions, respectively. Testing on five amputees corroborated the above findings. We have demonstrated that each factor introduces changes in the feature space that are statistically new class instances. Thus, the feature space contains two statistically classifiable clusters when the same motion is collected in two different limb positions or electrode shifts. Our results are a step forward in understanding PR schemes' challenges for myoelectric control of prostheses and further validation needs be conducted on more amputee-related datasets.
Collapse
Affiliation(s)
- Bingbin Wang
- Department of Engineering, King′s College London, London WC2R 2LS, UK; (B.W.)
| | - Jinglin Li
- Department of Engineering, King′s College London, London WC2R 2LS, UK; (B.W.)
| | - Levi Hargrove
- Center for Bionic Medicine, Shirley Ryan Ability, Chicago, IL 60611, USA;
- Department of Physical Medicine and Rehabilitation, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA
| | - Ernest Nlandu Kamavuako
- Department of Engineering, King′s College London, London WC2R 2LS, UK; (B.W.)
- Faculté de Médecine, Université de Kindu, Site de Lwama II, Kindu, Maniema, Congo
| |
Collapse
|
2
|
Pancholi S, Wachs JP, Duerstock BS. Use of Artificial Intelligence Techniques to Assist Individuals with Physical Disabilities. Annu Rev Biomed Eng 2024; 26:1-24. [PMID: 37832939 DOI: 10.1146/annurev-bioeng-082222-012531] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/15/2023]
Abstract
Assistive technologies (AT) enable people with disabilities to perform activities of daily living more independently, have greater access to community and healthcare services, and be more productive performing educational and/or employment tasks. Integrating artificial intelligence (AI) with various agents, including electronics, robotics, and software, has revolutionized AT, resulting in groundbreaking technologies such as mind-controlled exoskeletons, bionic limbs, intelligent wheelchairs, and smart home assistants. This article provides a review of various AI techniques that have helped those with physical disabilities, including brain-computer interfaces, computer vision, natural language processing, and human-computer interaction. The current challenges and future directions for AI-powered advanced technologies are also addressed.
Collapse
Affiliation(s)
- Sidharth Pancholi
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, Indiana, USA;
| | - Juan P Wachs
- School of Industrial Engineering, Purdue University, West Lafayette, Indiana, USA
| | - Bradley S Duerstock
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, Indiana, USA;
- School of Industrial Engineering, Purdue University, West Lafayette, Indiana, USA
| |
Collapse
|
3
|
Liu Y, Peng X, Tan Y, Oyemakinde TT, Wang M, Li G, Li X. A novel unsupervised dynamic feature domain adaptation strategy for cross-individual myoelectric gesture recognition. J Neural Eng 2024; 20:066044. [PMID: 38134446 DOI: 10.1088/1741-2552/ad184f] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2023] [Accepted: 12/22/2023] [Indexed: 12/24/2023]
Abstract
Objective.Surface electromyography pattern recognition (sEMG-PR) is considered as a promising control method for human-machine interaction systems. However, the performance of a trained classifier would greatly degrade for novel users since sEMG signals are user-dependent and largely affected by a number of individual factors such as the quantity of subcutaneous fat and the skin impedance.Approach.To solve this issue, we proposed a novel unsupervised cross-individual motion recognition method that aligned sEMG features from different individuals by self-adaptive dimensional dynamic distribution adaptation (SD-DDA) in this study. In the method, both the distances of marginal and conditional distributions between source and target features were minimized through automatically selecting the optimal feature domain dimension by using a small amount of unlabeled target data.Main results.The effectiveness of the proposed method was tested on four different feature sets, and results showed that the average classification accuracy was improved by above 10% on our collected dataset with the best accuracy reached 90.4%. Compared to six kinds of classic transfer learning methods, the proposed method showed an outstanding performance with improvements of 3.2%-13.8%. Additionally, the proposed method achieved an approximate 9% improvement on a publicly available dataset.Significance.These results suggested that the proposed SD-DDA method is feasible for cross-individual motion intention recognition, which would provide help for the application of sEMG-PR based system.
Collapse
Affiliation(s)
- Yan Liu
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
- Shenzhen Institute of Artificial Intelligence and Robotics for Society, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
| | - Xinhao Peng
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
- Shenzhen Institute of Artificial Intelligence and Robotics for Society, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
| | - Yingxiao Tan
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
- Shenzhen Institute of Artificial Intelligence and Robotics for Society, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
| | - Tolulope Tofunmi Oyemakinde
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
- Shenzhen Institute of Artificial Intelligence and Robotics for Society, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
| | - Mengtao Wang
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
- Shenzhen Institute of Artificial Intelligence and Robotics for Society, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
| | - Guanglin Li
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
- Shenzhen Institute of Artificial Intelligence and Robotics for Society, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
| | - Xiangxin Li
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
- Shenzhen Institute of Artificial Intelligence and Robotics for Society, Research Center for Neural Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, People's Republic of China
| |
Collapse
|
4
|
Cifuentes-Cuadros AA, Romero E, Caballa S, Vega-Centeno D, Elias DA. The LIBRA NeuroLimb: Hybrid Real-Time Control and Mechatronic Design for Affordable Prosthetics in Developing Regions. SENSORS (BASEL, SWITZERLAND) 2023; 24:70. [PMID: 38202932 PMCID: PMC10780857 DOI: 10.3390/s24010070] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Revised: 12/15/2023] [Accepted: 12/19/2023] [Indexed: 01/12/2024]
Abstract
Globally, 2.5% of upper limb amputations are transhumeral, and both mechanical and electronic prosthetics are being developed for individuals with this condition. Mechanics often require compensatory movements that can lead to awkward gestures. Electronic types are mainly controlled by superficial electromyography (sEMG). However, in proximal amputations, the residual limb is utilized less frequently in daily activities. Muscle shortening increases with time and results in weakened sEMG readings. Therefore, sEMG-controlled models exhibit a low success rate in executing gestures. The LIBRA NeuroLimb prosthesis is introduced to address this problem. It features three active and four passive degrees of freedom (DOF), offers up to 8 h of operation, and employs a hybrid control system that combines sEMG and electroencephalography (EEG) signal classification. The sEMG and EEG classification models achieve up to 99% and 76% accuracy, respectively, enabling precise real-time control. The prosthesis can perform a grip within as little as 0.3 s, exerting up to 21.26 N of pinch force. Training and validation sessions were conducted with two volunteers. Assessed with the "AM-ULA" test, scores of 222 and 144 demonstrated the prosthesis's potential to improve the user's ability to perform daily activities. Future work will prioritize enhancing the mechanical strength, increasing active DOF, and refining real-world usability.
Collapse
Affiliation(s)
| | | | | | | | - Dante A. Elias
- Biomechanics and Applied Robotics Research Laboratory, Pontificia Universidad Católica del Perú, Lima 15088, Peru; (A.A.C.-C.); (E.R.); (S.C.); (D.V.-C.)
| |
Collapse
|
5
|
Montazerin M, Rahimian E, Naderkhani F, Atashzar SF, Yanushkevich S, Mohammadi A. Transformer-based hand gesture recognition from instantaneous to fused neural decomposition of high-density EMG signals. Sci Rep 2023; 13:11000. [PMID: 37419881 PMCID: PMC10329032 DOI: 10.1038/s41598-023-36490-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2022] [Accepted: 06/05/2023] [Indexed: 07/09/2023] Open
Abstract
Designing efficient and labor-saving prosthetic hands requires powerful hand gesture recognition algorithms that can achieve high accuracy with limited complexity and latency. In this context, the paper proposes a Compact Transformer-based Hand Gesture Recognition framework referred to as [Formula: see text], which employs a vision transformer network to conduct hand gesture recognition using high-density surface EMG (HD-sEMG) signals. Taking advantage of the attention mechanism, which is incorporated into the transformer architectures, our proposed [Formula: see text] framework overcomes major constraints associated with most of the existing deep learning models such as model complexity; requiring feature engineering; inability to consider both temporal and spatial information of HD-sEMG signals, and requiring a large number of training samples. The attention mechanism in the proposed model identifies similarities among different data segments with a greater capacity for parallel computations and addresses the memory limitation problems while dealing with inputs of large sequence lengths. [Formula: see text] can be trained from scratch without any need for transfer learning and can simultaneously extract both temporal and spatial features of HD-sEMG data. Additionally, the [Formula: see text] framework can perform instantaneous recognition using sEMG image spatially composed from HD-sEMG signals. A variant of the [Formula: see text] is also designed to incorporate microscopic neural drive information in the form of Motor Unit Spike Trains (MUSTs) extracted from HD-sEMG signals using Blind Source Separation (BSS). This variant is combined with its baseline version via a hybrid architecture to evaluate potentials of fusing macroscopic and microscopic neural drive information. The utilized HD-sEMG dataset involves 128 electrodes that collect the signals related to 65 isometric hand gestures of 20 subjects. The proposed [Formula: see text] framework is applied to 31.25, 62.5, 125, 250 ms window sizes of the above-mentioned dataset utilizing 32, 64, 128 electrode channels. Our results are obtained via 5-fold cross-validation by first applying the proposed framework on the dataset of each subject separately and then, averaging the accuracies among all the subjects. The average accuracy over all the participants using 32 electrodes and a window size of 31.25 ms is 86.23%, which gradually increases till reaching 91.98% for 128 electrodes and a window size of 250 ms. The [Formula: see text] achieves accuracy of 89.13% for instantaneous recognition based on a single frame of HD-sEMG image. The proposed model is statistically compared with a 3D Convolutional Neural Network (CNN) and two different variants of Support Vector Machine (SVM) and Linear Discriminant Analysis (LDA) models. The accuracy results for each of the above-mentioned models are paired with their precision, recall, F1 score, required memory, and train/test times. The results corroborate effectiveness of the proposed [Formula: see text] framework compared to its counterparts.
Collapse
Affiliation(s)
- Mansooreh Montazerin
- Department of Electrical and Computer Engineering, Concordia University, Montreal, QC, Canada
| | - Elahe Rahimian
- Concordia Institute for Information Systems Engineering, Concordia University, Montreal, QC, Canada
| | - Farnoosh Naderkhani
- Concordia Institute for Information Systems Engineering, Concordia University, Montreal, QC, Canada
| | - S Farokh Atashzar
- Departments of Electrical and Computer Engineering, Mechanical and Aerospace Engineering, New York University (NYU), New York, 10003, NY, USA
- NYU Center for Urban Science and Progress (CUSP), NYU WIRELESS, New York University (NYU), New York, 10003, NY, USA
| | - Svetlana Yanushkevich
- Biometric Technologies Laboratory, Department of Electrical and Software Engineering, Schulich School of Engineering, University of Calgary, Calgary, AB, Canada
| | - Arash Mohammadi
- Department of Electrical and Computer Engineering, Concordia University, Montreal, QC, Canada.
- Concordia Institute for Information Systems Engineering, Concordia University, Montreal, QC, Canada.
| |
Collapse
|
6
|
Samuel OW, Asogbon MG, Kulwa F, Zangene AR, Oyemakinde TT, Igbe T, McEwan AA, Li Y, Li G. Enhanced Deep Transfer Learning Model based on Spatial-Temporal driven Scalograms for Precise Decoding of Motor Intent in Stroke Survivors. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38083417 DOI: 10.1109/embc40787.2023.10340683] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Intelligent rehabilitation robotics (RR) have been proposed in recent years to aid post-stroke survivors recover their lost limb functions. However, a large proportion of these robotic systems operate in a passive mode that restricts users to predefined trajectories that rarely align with their intended limb movements, precluding full functional recovery. To address this issue, an efficient Transfer Learning based Convolutional Neural Network (TL-CNN) model is proposed to decode post-stroke patients' motion intentions toward realizing dexterously active robotic training during rehabilitation. For the first time, we use Spatial-Temporal Descriptor based Continuous Wavelet Transform (STD-CWT) as input to TL-CNN to optimally decode limb movement intent patterns. We evaluated the STD-CWT method on three distinct wavelets including the Morse, Amor, and Bump, and compared their decoding outcomes with those of the commonly adopted CWT technique under similar experimental conditions. We then validated the method using electromyogram signals of five stroke survivors who performed twenty-one distinct motor tasks. The results showed that the proposed technique recorded a significantly higher (p<0.05) decoding accuracy and faster convergence compared to the common method. Our method equally recorded obvious class separability for individual motor tasks across subjects. The findings suggest that the STD-CWT Scalograms have the potential for robust decoding of motor intention and could facilitate intuitive and active motor training in stroke RR.Clinical Relevance- The study demonstrated the potential of Spatial Temporal based Scalograms in aiding precise and robust decoding of multi-class motor tasks, upon which dexterously active rehabilitation robotic training for full motor function restoration could be realized.
Collapse
|
7
|
Asogbon MG, Samuel OW, Nsugbe E, Li Y, Kulwa F, Mzurikwao D, Chen S, Li G. Ascertaining the optimal myoelectric signal recording duration for pattern recognition based prostheses control. Front Neurosci 2023; 17:1018037. [PMID: 36908798 PMCID: PMC9992216 DOI: 10.3389/fnins.2023.1018037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2022] [Accepted: 01/27/2023] [Indexed: 02/24/2023] Open
Abstract
Introduction Electromyogram-based pattern recognition (EMG-PR) has been widely considered an essentially intuitive control method for multifunctional upper limb prostheses. A crucial aspect of the scheme is the EMG signal recording duration (SRD) from which requisite motor tasks are characterized per time, impacting the system's overall performance. For instance, lengthy SRD inevitably introduces fatigue (that alters the muscle contraction patterns of specific limb motions) and may incur high computational costs in building the motion intent decoder, resulting in inadequate prosthetic control and controller delay in practical usage. Conversely, relatively shorter SRD may lead to reduced data collection durations that, among other advantages, allow for more convenient prosthesis recalibration protocols. Therefore, determining the optimal SRD required to characterize limb motion intents adequately that will aid intuitive PR-based control remains an open research question. Method This study systematically investigated the impact and generalizability of varying lengths of myoelectric SRD on the characterization of multiple classes of finger gestures. The investigation involved characterizing fifteen classes of finger gestures performed by eight normally limb subjects using various groups of EMG SRD including 1, 5, 10, 15, and 20 s. Two different training strategies including Between SRD and Within-SRD were implemented across three popular machine learning classifiers and three time-domain features to investigate the impact of SRD on EMG-PR motion intent decoder. Result The between-SRD strategy results which is a reflection of the practical scenario showed that an SRD greater than 5 s but less than or equal to 10 s (>5 and < = 10 s) would be required to achieve decent average finger gesture decoding accuracy for all feature-classifier combinations. Notably, lengthier SRD would incur more acquisition and implementation time and vice-versa. In inclusion, the study's findings provide insight and guidance into selecting appropriate SRD that would aid inadequate characterization of multiple classes of limb motion tasks in PR-based control schemes for multifunctional prostheses.
Collapse
Affiliation(s)
- Mojisola Grace Asogbon
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China
| | - Oluwarotimi Williams Samuel
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China
| | - Ejay Nsugbe
- Nsugbe Research Labs, Swindon, United Kingdom
| | - Yongcheng Li
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China
| | - Frank Kulwa
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China
| | - Deogratias Mzurikwao
- Unit of Biomedical Engineering, Department of Physiology, School of Engineering, Muhimbili University of Health and Allied Sciences, Dar es Salaam, Tanzania
| | - Shixiong Chen
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China
| | - Guanglin Li
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China
| |
Collapse
|
8
|
Ke A, Huang J, Wang J, He J. Improving the Robustness of Human-Machine Interactive Control for Myoelectric Prosthetic Hand During Arm Position Changing. Front Neurorobot 2022; 16:853773. [PMID: 35747073 PMCID: PMC9211066 DOI: 10.3389/fnbot.2022.853773] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2022] [Accepted: 05/16/2022] [Indexed: 12/01/2022] Open
Abstract
Robust classification of natural hand grasp type based on electromyography (EMG) still has some shortcomings in the practical prosthetic hand control, owing to the influence of dynamic arm position changing during hand actions. This study provided a framework for robust hand grasp type classification during dynamic arm position changes, improving both the “hardware” and “algorithm” components. In the hardware aspect, co-located synchronous EMG and force myography (FMG) signals are adopted as the multi-modal strategy. In the algorithm aspect, a sequential decision algorithm is proposed by combining the RNN-based deep learning model with a knowledge-based post-processing model. Experimental results showed that the classification accuracy of multi-modal EMG-FMG signals was increased by more than 10% compared with the EMG-only signal. Moreover, the classification accuracy of the proposed sequential decision algorithm improved the accuracy by more than 4% compared with other baseline models when using both EMG and FMG signals.
Collapse
Affiliation(s)
- Ang Ke
- Key Laboratory of Ministry of Education for Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China
| | - Jian Huang
- Key Laboratory of Ministry of Education for Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China
- Shenzhen Huazhong University of Science and Technology Research Institute, Shenzhen, China
- *Correspondence: Jian Huang
| | - Jing Wang
- Key Laboratory of Ministry of Education for Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China
| | - Jiping He
- Department of Intelligent Robots and Systems, Beijing Institute of Technology, Beijing, China
| |
Collapse
|
9
|
Geng Y, Qin L, Li Y, Yu Z, Li L, Asogbon MG, Zhan Y, Yan N, Guo X, Li G. Identifying Oscillations under Multi-site Sensory Stimulation for High-level Peripheral Nerve Injured Patients:A Pilot Study. J Neural Eng 2022; 19. [PMID: 35580572 DOI: 10.1088/1741-2552/ac7079] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2021] [Accepted: 05/17/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVE For high-level peripheral nerve injured (PNI) patients with severe sensory dysfunction of upper extremities, identifying the multi-site tactile stimulation is of great importance to provide neurorehabilitation with sensory feedback. In this pilot study, we showed the feasibility of identifying multi-site and multi-intensity tactile stimulation in terms of electroencephalography (EEG). APPROACH Three high-level PNI patients and eight non-PNI participants were recruited in this study. Four different sites over the upper arm, forearm, thumb finger and little finger were randomly stimulated at two intensities (both sensory-level) based on the transcutaneous electrical nerve stimulation (TENS). Meanwhile, 64-channel EEG signals were recorded during the passive tactile sense stimulation on each side. MAIN RESULTS The spatial-spectral distribution of brain oscillations underlying multi-site sensory stimulation showed dominant power attenuation over the somatosensory and prefrontal cortices in both alpha-band (8-12 Hz) and beta-band (13-30 Hz). But there was no significant difference among different stimulation sites in terms of the averaged power spectral density over the region of interest (ROI). By further identifying different stimulation sites using temporal-spectral features, we found the classification accuracies were all above 89% for the affected arm of PNI patients, comparable to that from their intact side and that from the non-PNI group. When the stimulation site-intensity combinations were treated as eight separate classes, the classification accuracies were ranging from 88.89% to 99.30% for the affected side of PNI subjects, similar to that from their non-affected side and that from the non-PNI group. Other performance metrics, including Specificity, Precision, and F1-Score, also showed a sound identification performance for both PNI patients and non-PNI subjects. SIGNIFICANCE These results suggest that reliable brain oscillations could be evoked and identified well, even though induced tactile sense could not be discerned by the PNI patients. This study have implication for facilitating bidirectional neurorehabilitation systems with sensory feedback.
Collapse
Affiliation(s)
- Yanjuan Geng
- Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences, 1068 Xueyuan Boulevard, University Town of Shenzhen, Xili Nanshan, Shenzhen 518055, China, Shenzhen, Guangdong, 518055, CHINA
| | - Liuni Qin
- Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences, 1068 Xueyuan Boulevard, University Town of Shenzhen, Xili Nanshan, Shenzhen 518055, China, Shenzhen, Guangdong, 518055, CHINA
| | - Yongcheng Li
- Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences, 1068 Xueyuan Boulevard, University Town of Shenzhen, Xili Nanshan, Shenzhen 518055, China, Shenzhen, Guangdong, 518055, CHINA
| | - Zhebin Yu
- Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences, 1068 Xueyuan Boulevard, University Town of Shenzhen, Xili Nanshan, Shenzhen 518055, China, Shenzhen, Guangdong, 518055, CHINA
| | - Linling Li
- Shenzhen University, 1066 Xueyuan Boulevard, University Town of Shenzhen, Xili Nanshan, Shenzhen 518055, China, Shenzhen, 518060, CHINA
| | - Mojisola Grace Asogbon
- Shenzhen Institutes of Advanced Technology, 1068 Xueyuan Boulevard, University Town of Shenzhen, Xili Nanshan, Shenzhen 518055, China, Shenzhen, Guangdong, 518055, CHINA
| | - Yang Zhan
- Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences, 1068 Xueyuan Boulevard, University Town of Shenzhen, Xili Nanshan, Shenzhen 518055, China, Shenzhen, Guangdong, 518055, CHINA
| | - Nan Yan
- Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences, 1068 Xueyuan Boulevard, University Town of Shenzhen, Xili Nanshan, Shenzhen 518055, China, Shenzhen, Guangdong, 518055, CHINA
| | - Xin Guo
- Hebei University of Technology, Hebei University of Technology, Tianjin 300130, China, Tianjin, Tianjin, 300401, CHINA
| | - Guanglin Li
- Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences, 1068 Xueyuan Boulevard, University Town of Shenzhen, Xili Nanshan, Shenzhen 518055, China, Shenzhen, Guangdong, 518055, CHINA
| |
Collapse
|
10
|
Asghar A, Jawaid Khan S, Azim F, Shakeel CS, Hussain A, Niazi IK. Review on electromyography based intention for upper limb control using pattern recognition for human-machine interaction. Proc Inst Mech Eng H 2022; 236:628-645. [DOI: 10.1177/09544119221074770] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Upper limb myoelectric prosthetic control is an essential topic in the field of rehabilitation. The technique controls prostheses using surface electromyogram (sEMG) and intramuscular EMG (iEMG) signals. EMG signals are extensively used in controlling prosthetic upper and lower limbs, virtual reality entertainment, and human-machine interface (HMI). EMG signals are vital parameters for machine learning and deep learning algorithms and help to give an insight into the human brain’s function and mechanisms. Pattern recognition techniques pertaining to support vector machine (SVM), k-nearest neighbor (KNN) and Bayesian classifiers have been utilized to classify EMG signals. This paper presents a review on current EMG signal techniques, including electrode array utilization, signal acquisition, signal preprocessing and post-processing, feature selection and extraction, data dimensionality reduction, classification, and ultimate application to the community. The paper also discusses using alternatives to EMG signals, such as force sensors, to measure muscle activity with reliable results. Future implications for EMG classification include employing deep learning techniques such as artificial neural networks (ANN) and recurrent neural networks (RNN) for achieving robust results.
Collapse
Affiliation(s)
- Ali Asghar
- Department of Biomedical Engineering, Faculty of Engineering, Science, Technology and Management, Ziauddin University, Karachi, Pakistan
- Department of Electrical Engineering, Faculty of Engineering, Science, Technology and Management, Ziauddin University, Karachi, Pakistan
| | - Saad Jawaid Khan
- Department of Biomedical Engineering, Faculty of Engineering, Science, Technology and Management, Ziauddin University, Karachi, Pakistan
| | - Fahad Azim
- Department of Electrical Engineering, Faculty of Engineering, Science, Technology and Management, Ziauddin University, Karachi, Pakistan
| | - Choudhary Sobhan Shakeel
- Department of Biomedical Engineering, Faculty of Engineering, Science, Technology and Management, Ziauddin University, Karachi, Pakistan
| | - Amatullah Hussain
- College of Rehabilitation Sciences, Ziauddin University, Karachi, Pakistan
| | - Imran Khan Niazi
- Centre for Chiropractic Research, New Zealand College of Chiropractic, New Zealand
- Faculty of Health & Environmental Sciences, Health & Rehabilitation Research Institute, AUT University, New Zealand
- Centre for Sensory-Motor Interactions, Department of Health Science and Technology, Aalborg University, Denmark
| |
Collapse
|
11
|
Myoelectric Pattern Recognition Performance Enhancement Using Nonlinear Features. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:6414664. [PMID: 35528339 PMCID: PMC9076314 DOI: 10.1155/2022/6414664] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/08/2022] [Revised: 04/07/2022] [Accepted: 04/11/2022] [Indexed: 11/17/2022]
Abstract
The multichannel electrode array used for electromyogram (EMG) pattern recognition provides good performance, but it has a high cost, is computationally expensive, and is inconvenient to wear. Therefore, researchers try to use as few channels as possible while maintaining improved pattern recognition performance. However, minimizing the number of channels affects the performance due to the least separable margin among the movements possessing weak signal strengths. To meet these challenges, two time-domain features based on nonlinear scaling, the log of the mean absolute value (LMAV) and the nonlinear scaled value (NSV), are proposed. In this study, we validate the proposed features on two datasets, the existing four feature extraction methods, variable window size, and various signal-to-noise ratios (SNR). In addition, we also propose a feature extraction method where the LMAV and NSV are grouped with the existing 11 time-domain features. The proposed feature extraction method enhances accuracy, sensitivity, specificity, precision, and F1 score by 1.00%, 5.01%, 0.55%, 4.71%, and 5.06% for dataset 1, and 1.18%, 5.90%, 0.66%, 5.63%, and 6.04% for dataset 2, respectively. Therefore, the experimental results strongly suggest the proposed feature extraction method, for taking a step forward with regard to improved myoelectric pattern recognition performance.
Collapse
|
12
|
Jarrah YA, Asogbon MG, Samuel OW, Wang X, Zhu M, Nsugbe E, Chen S, Li G. High-density surface EMG signal quality enhancement via optimized filtering technique for amputees’ motion intent characterization towards intuitive prostheses control. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103497] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
13
|
Mablekos-Alexiou A, Kontogiannopoulos S, Bertos GA, Papadopoulos E. A biomechatronics-based EPP topology for upper-limb prosthesis control: Modeling & benchtop prototype. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103454] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
14
|
Ma C, Guo W, Zhang H, Samuel OW, Ji X, Xu L, Li G. A Novel and Efficient Feature Extraction Method for Deep Learning Based Continuous Estimation. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3097257] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
15
|
Force-Invariant Improved Feature Extraction Method for Upper-Limb Prostheses of Transradial Amputees. Diagnostics (Basel) 2021; 11:diagnostics11050843. [PMID: 34067203 PMCID: PMC8151019 DOI: 10.3390/diagnostics11050843] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2021] [Revised: 04/28/2021] [Accepted: 05/05/2021] [Indexed: 11/16/2022] Open
Abstract
A force-invariant feature extraction method derives identical information for all force levels. However, the physiology of muscles makes it hard to extract this unique information. In this context, we propose an improved force-invariant feature extraction method based on nonlinear transformation of the power spectral moments, changes in amplitude, and the signal amplitude along with spatial correlation coefficients between channels. Nonlinear transformation balances the forces and increases the margin among the gestures. Additionally, the correlation coefficient between channels evaluates the amount of spatial correlation; however, it does not evaluate the strength of the electromyogram signal. To evaluate the robustness of the proposed method, we use the electromyogram dataset containing nine transradial amputees. In this study, the performance is evaluated using three classifiers with six existing feature extraction methods. The proposed feature extraction method yields a higher pattern recognition performance, and significant improvements in accuracy, sensitivity, specificity, precision, and F1 score are found. In addition, the proposed method requires comparatively less computational time and memory, which makes it more robust than other well-known feature extraction methods.
Collapse
|
16
|
Pan L, Huang H(H. A robust model-based neural-machine interface across different loading weights applied at distal forearm. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102509] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
17
|
Samuel OW, Asogbon MG, Geng Y, Jiang N, Mzurikwao D, Zheng Y, Wong KKL, Vollero L, Li G. Decoding movement intent patterns based on spatiotemporal and adaptive filtering method towards active motor training in stroke rehabilitation systems. Neural Comput Appl 2021. [DOI: 10.1007/s00521-020-05536-9] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
18
|
Feature Extraction of Surface Electromyography Using Wavelet Weighted Permutation Entropy for Hand Movement Recognition. JOURNAL OF HEALTHCARE ENGINEERING 2020; 2020:8824194. [PMID: 33299536 PMCID: PMC7707938 DOI: 10.1155/2020/8824194] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/19/2020] [Revised: 11/10/2020] [Accepted: 11/12/2020] [Indexed: 11/17/2022]
Abstract
The feature extraction of surface electromyography (sEMG) signals has been an important aspect of myoelectric prosthesis control. To improve the practicability of myoelectric prosthetic hands, we proposed a feature extraction method for sEMG signals that uses wavelet weighted permutation entropy (WWPE). First, wavelet transform was used to decompose and preprocess sEMG signals collected from the relevant muscles of the upper limbs to obtain the wavelet sub-bands in each frequency segment. Then, the weighted permutation entropies (WPEs) of the wavelet sub-bands were extracted to construct WWPE feature set. Lastly, the WWPE feature set was used as input to a support vector machine (SVM) classifier and a backpropagation neural network (BPNN) classifier to recognize seven hand movements. Experimental results show that the proposed method exhibits remarkable recognition accuracy that is superior to those of single sub-band feature set and commonly used time-domain feature set. The maximum recognition accuracy rate is 100% for hand movements, and the average recognition accuracy rates of SVM and BPNN are 100% and 98%, respectively.
Collapse
|
19
|
Samuel OW, Grace Asogbon M, Geng Y, Rusydi MI, Mzurikwao ZB, Chen S, Fang P, Li G. Characterizing Multiple Patterns of Motor Intent Using Spatial-Temporal Information for Intuitively Active Motor Training in Stroke Survivors. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:3831-3834. [PMID: 33018836 DOI: 10.1109/embc44109.2020.9176308] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Upper extremity motor function loss severely affects stroke survivors during daily life activities. Different rehabilitation robotic systems have been developed to allow stroke survivors regain their motor function. Meanwhile, most of the robots only operate in a passive mode and restrict the users to navigate predefined trajectories which may not align with their motion intent, thus limiting motor recovery. One way to resolve this issue would be to utilize a decoded movement intent to trigger intuitively active motor training for patients. In this direction, this study proposed and investigated the use of spatial-temporal neuromuscular descriptor (STD) for optimal decoding of multiple patterns of movement intents in patient to provide inputs for active motor training in the rehabilitation robotic systems. The STD performance was validated using High-Density surface electromyogram recordings from five stroke survivors who performed 21 limb movements. Experimental results show that the STD achieved a significant reduction in limb movement classification error (13.36%) even in the presence of the inevitable White Gaussian Noise compared to other methods (p<0.05). The STD also showed obvious class separability for individual movement. Findings from this study suggest that the STD may provide potential inputs for intuitively active motor training in stroke rehabilitation robotic systems.Clinical Relevance- This study showed that spatial-temporal neuromuscular information could aid adequate decoding of movement intents upon which intuitively active motor training could be achieved in stroke rehabilitation robotic systems.
Collapse
|
20
|
Appropriate Feature Set and Window Parameters Selection for Efficient Motion Intent Characterization towards Intelligently Smart EMG-PR System. Symmetry (Basel) 2020. [DOI: 10.3390/sym12101710] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
The constantly rising number of limb stroke survivors and amputees has motivated the development of intelligent prosthetic/rehabilitation devices for their arm function restoration. The device often integrates a pattern recognition (PR) algorithm that decodes amputees’ limb movement intent from electromyogram (EMG) signals, characterized by neural information and symmetric distribution. However, the control performance of the prostheses mostly rely on the interrelations among multiple dynamic factors of feature set, windowing parameters, and signal conditioning that have rarely been jointly investigated to date. This study systematically investigated the interaction effects of these dynamic factors on the performance of EMG-PR system towards constructing optimal parameters for accurately robust movement intent decoding in the context of prosthetic control. In this regard, the interaction effects of various features across window lengths (50 ms~300 ms), increments (50 ms~125 ms), robustness to external interferences and sensor channels (2 ch~6 ch), were examined using EMG signals obtained from twelve subjects through a symmetrical movement elicitation protocol. Compared to single features, multiple features consistently achieved minimum decoding error below 10% across optimal windowing parameters of 250 ms/100 ms. Also, the multiple features showed high robustness to additive noise with obvious trade-offs between accuracy and computation time. Consequently, our findings may provide proper insight for appropriate parameter selection in the context of robust PR-based control strategy for intelligent rehabilitation device.
Collapse
|
21
|
Capsi-Morales P, Piazza C, Catalano MG, Bicchi A, Grioli G. Exploring Stiffness Modulation in Prosthetic Hands and Its Perceived Function in Manipulation and Social Interaction. Front Neurorobot 2020; 14:33. [PMID: 32670044 PMCID: PMC7331496 DOI: 10.3389/fnbot.2020.00033] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2019] [Accepted: 05/11/2020] [Indexed: 12/05/2022] Open
Abstract
To physically interact with a rich variety of environments and to match situation-dependent requirements, humans adapt both the force and stiffness of their limbs. Reflecting this behavior in prostheses may promote a more natural and intuitive control and, consequently, improve prostheses acceptance in everyday life. This pilot study proposes a method to control a prosthetic robot hand and its impedance, and explores the utility of variable stiffness when performing activities of daily living and physical social interactions. The proposed method is capable of a simultaneous and proportional decoding of position and stiffness intentions from two surface electro-myographic sensors placed over a pair of antagonistic muscles. The feasibility of our approach is validated and compared to existing control modalities in a preliminary study involving one prosthesis user. The algorithm is implemented in a soft under-actuated prosthetic hand (SoftHand Pro). Then, we evaluate the usability of the proposed approach while executing a variety of tasks. Among these tasks, the user interacts with other 12 able-bodied subjects, whose experiences were also assessed. Several statistically significant aspects from the System Usability Scale indicate user's preference of variable stiffness control over low or high constant stiffness due to its reactivity and adaptability. Feedback reported by able-bodied subjects reveal a general tendency to favor soft interaction, i.e., low stiffness, which is perceived more human-like and comfortable. These combined results suggest the use of variable stiffness as a viable compromise between firm control and safe interaction which is worth investigating further.
Collapse
Affiliation(s)
- Patricia Capsi-Morales
- Centro "E. Piaggio" and Dipartimento di Ingegneria dell'Informazione, University of Pisa, Pisa, Italy.,Istituto Italiano di Tecnologia, Genova, Italy
| | - Cristina Piazza
- Centro "E. Piaggio" and Dipartimento di Ingegneria dell'Informazione, University of Pisa, Pisa, Italy
| | | | - Antonio Bicchi
- Centro "E. Piaggio" and Dipartimento di Ingegneria dell'Informazione, University of Pisa, Pisa, Italy.,Istituto Italiano di Tecnologia, Genova, Italy
| | | |
Collapse
|