1
|
Skoraczynski DJ, Chen C. Novel near E-Field Topography Sensor for Human-Machine Interfacing in Robotic Applications. SENSORS (BASEL, SWITZERLAND) 2024; 24:1379. [PMID: 38474915 DOI: 10.3390/s24051379] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/19/2024] [Revised: 02/10/2024] [Accepted: 02/14/2024] [Indexed: 03/14/2024]
Abstract
This work investigates a new sensing technology for use in robotic human-machine interface (HMI) applications. The proposed method uses near E-field sensing to measure small changes in the limb surface topography due to muscle actuation over time. The sensors introduced in this work provide a non-contact, low-computational-cost, and low-noise method for sensing muscle activity. By evaluating the key sensor characteristics, such as accuracy, hysteresis, and resolution, the performance of this sensor is validated. Then, to understand the potential performance in intention detection, the unmodified digital output of the sensor is analysed against movements of the hand and fingers. This is done to demonstrate the worst-case scenario and to show that the sensor provides highly targeted and relevant data on muscle activation before any further processing. Finally, a convolutional neural network is used to perform joint angle prediction over nine degrees of freedom, achieving high-level regression performance with an RMSE value of less than six degrees for thumb and wrist movements and 11 degrees for finger movements. This work demonstrates the promising performance of this novel approach to sensing for use in human-machine interfaces.
Collapse
Affiliation(s)
- Dariusz J Skoraczynski
- Laboratory of Motion Generation and Analysis (LMGA), Monash University, Clayton, VIC 3800, Australia
| | - Chao Chen
- Laboratory of Motion Generation and Analysis (LMGA), Monash University, Clayton, VIC 3800, Australia
| |
Collapse
|
2
|
Study on the methods of feature extraction based on electromyographic signal classification. Med Biol Eng Comput 2023:10.1007/s11517-023-02812-3. [PMID: 36894795 DOI: 10.1007/s11517-023-02812-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2022] [Accepted: 02/23/2023] [Indexed: 03/11/2023]
Abstract
Electromyography (EMG) is a form of biological information, which is used in many fields to help people study human muscle movement, especially in the study of bionic hands. EMG signals can be used to explain the activity at a certain moment through the signal changes of human muscles, and it is a very complex signal, so processing it is very important. The process of EMG signals can be divided into acquisition, pre-processing, feature extraction, and classification. Not all signal channels are useful in EMG acquisition, and it is important to select useful signals among them. Therefore, this study proposes a feature extraction method to extract the most representative two-channel signals from the eight-channel signals. In this paper, the traditional principal component analysis method and support vector machine feature elimination are used to extract signal channels. At the same time, a new method, correlation heat map, is proposed to implement feature extraction method by using three methods, and three classification algorithms of K-nearest neighbor, random forest, and support vector machine are used to verify. The results show that the classification accuracy of the proposed method is better than that of the other two traditional methods.
Collapse
|
3
|
Li J, Zhong J, Wang N. A multimodal human-robot sign language interaction framework applied in social robots. Front Neurosci 2023; 17:1168888. [PMID: 37113147 PMCID: PMC10126358 DOI: 10.3389/fnins.2023.1168888] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2023] [Accepted: 03/20/2023] [Indexed: 04/29/2023] Open
Abstract
Deaf-mutes face many difficulties in daily interactions with hearing people through spoken language. Sign language is an important way of expression and communication for deaf-mutes. Therefore, breaking the communication barrier between the deaf-mute and hearing communities is significant for facilitating their integration into society. To help them integrate into social life better, we propose a multimodal Chinese sign language (CSL) gesture interaction framework based on social robots. The CSL gesture information including both static and dynamic gestures is captured from two different modal sensors. A wearable Myo armband and a Leap Motion sensor are used to collect human arm surface electromyography (sEMG) signals and hand 3D vectors, respectively. Two modalities of gesture datasets are preprocessed and fused to improve the recognition accuracy and to reduce the processing time cost of the network before sending it to the classifier. Since the input datasets of the proposed framework are temporal sequence gestures, the long-short term memory recurrent neural network is used to classify these input sequences. Comparative experiments are performed on an NAO robot to test our method. Moreover, our method can effectively improve CSL gesture recognition accuracy, which has potential applications in a variety of gesture interaction scenarios not only in social robots.
Collapse
Affiliation(s)
- Jie Li
- School of Artificial Intelligence, Chongqing Technology and Business University, Chongqing, China
| | - Junpei Zhong
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
| | - Ning Wang
- Bristol Robotics Laboratory, University of the West of England, Bristol, United Kingdom
- *Correspondence: Ning Wang,
| |
Collapse
|
4
|
|
5
|
Bao T, Xie SQ, Yang P, Zhou P, Zhang ZQ. Towards Robust, Adaptive and Reliable Upper-limb Motion Estimation Using Machine Learning and Deep Learning--A Survey in Myoelectric Control. IEEE J Biomed Health Inform 2022; 26:3822-3835. [PMID: 35294368 DOI: 10.1109/jbhi.2022.3159792] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
To develop multi-functional human-machine interfaces that can help disabled people reconstruct lost functions of upper-limbs, machine learning (ML) and deep learning (DL) techniques have been widely implemented to decode human movement intentions from surface electromyography (sEMG) signals. However, due to the high complexity of upper-limb movements and the inherent non-stable characteristics of sEMG, the usability of ML/DL based control schemes is still greatly limited in practical scenarios. To this end, tremendous efforts have been made to improve model robustness, adaptation, and reliability. In this article, we provide a systematic review on recent achievements, mainly from three categories: multi-modal sensing fusion to gain additional information of the user, transfer learning (TL) methods to eliminate domain shift impacts on estimation models, and post-processing approaches to obtain more reliable outcomes. Special attention is given to fusion strategies, deep TL frameworks, and confidence estimation. \textcolor{red}{Research challenges and emerging opportunities, with respect to hardware development, public resources, and decoding strategies, are also analysed to provide perspectives for future developments.
Collapse
|
6
|
Khan SM, Khan AA, Farooq O. Pattern recognition of EMG signals for low level grip force classification. Biomed Phys Eng Express 2021; 7. [PMID: 34474400 DOI: 10.1088/2057-1976/ac2354] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2021] [Accepted: 09/02/2021] [Indexed: 12/29/2022]
Abstract
Grasping of the objects is the most frequent activity performed by the human upper limb. The amputations of the upper limb results in the need for prosthetic devices. The myoelectric prosthetic devices use muscle signals and apply control techniques for identification of different levels of hand gesture and force levels. In this study; a different level force contraction experiment was performed in which Electromyography (EMGs) signals and fingertip force signals were acquired. Using this experimental data; a two-step feature selection process is applied for the designing of a pattern recognition algorithm for the classification of different force levels. The two step feature selection process consist of generalized feature ranking using ReliefF, followed by personalized feature selection using Neighborhood Component Analysis (NCA) from the shortlisted features by earlier technique. The classification algorithms applied in this study were Support Vector Machines (SVM) and Random Forest (RF). Besides feature selection; optimization of the number of muscles during classification of force levels was also performed using designed algorithm. Based on this algorithm; the maximum classification accuracy using SVM classifier and two muscle set was achieved as high as 99%. The optimal feature set consisted features such as Auto Regressive coefficients, Willison Amplitude and Slope Sign Change. The mean classification accuracy for different subjects, achieved using SVM and RF was 94.5% and 91.7% respectively.
Collapse
Affiliation(s)
| | - Abid Ali Khan
- Department of Mechanical Engineering, AMU, Aligarh, UP, India
| | - Omar Farooq
- Department of Electronics Engineering, AMU, Aligarh, UP, India
| |
Collapse
|
7
|
Li Q, Liu Y, Zhu J, Chen Z, Liu L, Yang S, Zhu G, Zhu B, Li J, Jin R, Tao J, Chen L. Upper-Limb Motion Recognition Based on Hybrid Feature Selection: Algorithm Development and Validation. JMIR Mhealth Uhealth 2021; 9:e24402. [PMID: 34473067 PMCID: PMC8446846 DOI: 10.2196/24402] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2020] [Revised: 04/30/2021] [Accepted: 07/15/2021] [Indexed: 02/05/2023] Open
Abstract
Background For rehabilitation training systems, it is essential to automatically record and recognize exercises, especially when more than one type of exercise is performed without a predefined sequence. Most motion recognition methods are based on feature engineering and machine learning algorithms. Time-domain and frequency-domain features are extracted from original time series data collected by sensor nodes. For high-dimensional data, feature selection plays an important role in improving the performance of motion recognition. Existing feature selection methods can be categorized into filter and wrapper methods. Wrapper methods usually achieve better performance than filter methods; however, in most cases, they are computationally intensive, and the feature subset obtained is usually optimized only for the specific learning algorithm. Objective This study aimed to provide a feature selection method for motion recognition of upper-limb exercises and improve the recognition performance. Methods Motion data from 5 types of upper-limb exercises performed by 21 participants were collected by a customized inertial measurement unit (IMU) node. A total of 60 time-domain and frequency-domain features were extracted from the original sensor data. A hybrid feature selection method by combining filter and wrapper methods (FESCOM) was proposed to eliminate irrelevant features for motion recognition of upper-limb exercises. In the filter stage, candidate features were first selected from the original feature set according to the significance for motion recognition. In the wrapper stage, k-nearest neighbors (kNN), Naïve Bayes (NB), and random forest (RF) were evaluated as the wrapping components to further refine the features from the candidate feature set. The performance of the proposed FESCOM method was verified using experiments on motion recognition of upper-limb exercises and compared with the traditional wrapper method. Results Using kNN, NB, and RF as the wrapping components, the classification error rates of the proposed FESCOM method were 1.7%, 8.9%, and 7.4%, respectively, and the feature selection time in each iteration was 13 seconds, 71 seconds, and 541 seconds, respectively. Conclusions The experimental results demonstrated that, in the case of 5 motion types performed by 21 healthy participants, the proposed FESCOM method using kNN and NB as the wrapping components achieved better recognition performance than the traditional wrapper method. The FESCOM method dramatically reduces the search time in the feature selection process. The results also demonstrated that the optimal number of features depends on the classifier. This approach serves to improve feature selection and classification algorithm selection for upper-limb motion recognition based on wearable sensor data, which can be extended to motion recognition of more motion types and participants.
Collapse
Affiliation(s)
- Qiaoqin Li
- Knowledge and Data Engineering Laboratory of Chinese Medicine, School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Yongguo Liu
- Knowledge and Data Engineering Laboratory of Chinese Medicine, School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Jiajing Zhu
- Knowledge and Data Engineering Laboratory of Chinese Medicine, School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Zhi Chen
- Knowledge and Data Engineering Laboratory of Chinese Medicine, School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Lang Liu
- Knowledge and Data Engineering Laboratory of Chinese Medicine, School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Shangming Yang
- Knowledge and Data Engineering Laboratory of Chinese Medicine, School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Guanyi Zhu
- College of Electrical and Information Engineering, Hunan University, Changsha, China
| | - Bin Zhu
- Chengdu Chronic Diseases Hospital, Chengdu, China
| | - Juan Li
- College of Health Preservation and Rehabilitation, Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - Rongjiang Jin
- College of Health Preservation and Rehabilitation, Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - Jing Tao
- College of Rehabilitation Medicine, Fujian University of Traditional Chinese Medicine, Fuzhou, China
| | - Lidian Chen
- College of Rehabilitation Medicine, Fujian University of Traditional Chinese Medicine, Fuzhou, China
| |
Collapse
|
8
|
Walsh KA, Sanford SP, Collins BD, Harel NY, Nataraj R. Performance potential of classical machine learning and deep learning classifiers for isometric upper-body myoelectric control of direction in virtual reality with reduced muscle inputs. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102487] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
9
|
Shahzaib M, Shakil S, Ghuffar S, Maqsood M, Bhatti FA. Classification of forearm EMG signals for 10 motions using optimum feature-channel combinations. Comput Methods Biomech Biomed Engin 2020; 24:945-955. [PMID: 33356542 DOI: 10.1080/10255842.2020.1861256] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Abstract
Electromyography (EMG) is the study of electrical activity in the muscles. We classify EMG signals from surface electrodes (channels) using Artificial Neural Network (ANN). We evaluate classification performance of 10 different hand motions using several feature-channel combinations with wrapper method. Highest classification accuracy of 98.7% is achieved with each feature-channel combination. Compared to previous studies, we achieve the highest accuracy for 10 classes with lower number of feature-channel combination. We reduce ANN complexity without compromising the classification accuracy for deployment in low-end hardware with limited computational power along with improving the design of a low-cost hardware for EMG signal acquisition.
Collapse
Affiliation(s)
- Muhammad Shahzaib
- iVision Lab, Electrical Engineering Department, Institute of Space Technology, Islamabad, Pakistan
| | - Sadia Shakil
- iVision Lab, Electrical Engineering Department, Institute of Space Technology, Islamabad, Pakistan.,Turner Institute for Brain and Mental Health, Monash University, Melbourne, Australia
| | - Sajid Ghuffar
- Space Science Department, Institute of Space Technology, Islamabad, Pakistan
| | - Moazam Maqsood
- iVision Lab, Electrical Engineering Department, Institute of Space Technology, Islamabad, Pakistan
| | - Farrukh A Bhatti
- iVision Lab, Electrical Engineering Department, Institute of Space Technology, Islamabad, Pakistan
| |
Collapse
|
10
|
Xiao F, Chen Y, Zhu Y. GADF/GASF-HOG:feature extraction methods for hand movement classification from surface electromyography. J Neural Eng 2020; 17:046016. [DOI: 10.1088/1741-2552/ab9db9] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
|
11
|
Affiliation(s)
- Jing Luo
- School of Automation Science and Engineering, South China University of Technology, Guangzhou, People's Republic of China
- Department of Bioengineering, Imperial College of Science Technology and Medicine, London, UK
| | - Chao Liu
- Department of Robotics, LIRMM, UMR5506, University of Montpellier-CNRS, Montpellier, France
| | - Ying Feng
- School of Automation Science and Engineering, South China University of Technology, Guangzhou, People's Republic of China
| | - Chenguang Yang
- Bristol Robotics Laboratory, University of the West of England, Bristol, UK
| |
Collapse
|