1
|
Chen X, Cai S, Yu L, Li X, Fan B, Du M, Liu T, Bao G. A Novel CNN-BiLSTM Ensemble Model With Attention Mechanism for Sit-to-Stand Phase Identification Using Wearable Inertial Sensors. IEEE Trans Neural Syst Rehabil Eng 2024; 32:1068-1077. [PMID: 38373135 DOI: 10.1109/tnsre.2024.3366907] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/21/2024]
Abstract
Sit-to-stand transition phase identification is vital in the control of a wearable exoskeleton robot for assisting patients to stand stably. In this study, we aim to propose a method for segmenting and identifying the sit-to-stand phase using two inertial sensors. First, we defined the sit-to-stand transition into five phases, namely, the initial sitting phase, the flexion momentum phase, the momentum transfer phase, the extension phase, and the stable standing phase based on the preprocessed acceleration and angular velocity data. We then employed a threshold method to recognize the initial sitting and the stable standing phases. Finally, we designed a novel CNN-BiLSTM-Attention algorithm to identify the three transition phases, namely, the flexion momentum phase, the momentum transfer phase, and the extension phase. Fifteen subjects were recruited to perform sit-to-stand transition experiments under a specific paradigm. A combination of the acceleration and angular velocity data features for the sit-to-stand transition phase identification were validated for the model performance improvements. The integration of the CNN, Bi-LSTM, and Attention modules demonstrated the reasonableness of the proposed algorithms. The experimental results showed that the proposed CNN-BiLSTM-Attention algorithm achieved the highest average classification accuracy of 99.5% for all five phases when compared to both traditional machine learning algorithms and deep learning algorithms on our customized dataset (STS-PD). The proposed sit-to-stand phase recognition algorithm could serve as a foundation for the control of wearable exoskeletons and is important for the further development of intelligent wearable exoskeleton rehabilitation robots.
Collapse
|
2
|
Zhang Y, Cao G, Sun M, Zhao B, Wu Q, Xia C. Mechanomyography signals pattern recognition in hand movements using swarm intelligence algorithm optimized support vector machine based on acceleration sensors. Med Eng Phys 2024; 124:104060. [PMID: 38418032 DOI: 10.1016/j.medengphy.2023.104060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Revised: 09/19/2023] [Accepted: 10/02/2023] [Indexed: 03/01/2024]
Abstract
On the basis of extracting mechanomyography (MMG) signal features, the classification of hand movements has certain application values in human-machine interaction systems and wearable devices. In this paper, pattern recognition of hand movements based on MMG signal is studied with swarm intelligence algorithms introduced to optimize support vector machine (SVM). Time domain (TD) features, wavelet packet node energy (WPNE) features, frequency domain (FD) features, convolution neural network (CNN) features were extracted from each channel to constitute different feature sets. Three novel swarm intelligence algorithms (i.e., bald eagle search (BES), sparrow search algorithm (SSA), grey wolf optimization (GWO)) optimized SVM is proposed to train the models and recognition of hand movements are tested for each MMG feature extraction method. Using GWO as the optimization algorithm, time consumption is less than using the other two swarm algorithms. Using GWO with TD+FD features can obtain the classification accuracy of 93.55 %, which is higher than other methods while using CNN to extract features can be independent of domain knowledge. The results confirm GWO-SVM with TD + FD features is superior to some other methods in the classification problem for tiny samples based on MMG.
Collapse
Affiliation(s)
- Yue Zhang
- School of Mechanical Engineering, Nantong University, Nantong 226019 China
| | - Gangsheng Cao
- School of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai 200237 China
| | - Maoxun Sun
- School of Mechanical Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China
| | - Baigan Zhao
- School of Mechanical Engineering, Nantong University, Nantong 226019 China
| | - Qing Wu
- School of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai 200237 China
| | - Chunming Xia
- School of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai 200237 China; School of Mechanical and Automotive Engineering, Shanghai University of Engineering Science, Shanghai 201620 China.
| |
Collapse
|
3
|
Dai Q, Wong Y, Kankanhali M, Li X, Geng W. Improved Network and Training Scheme for Cross-Trial Surface Electromyography (sEMG)-Based Gesture Recognition. Bioengineering (Basel) 2023; 10:1101. [PMID: 37760203 PMCID: PMC10525369 DOI: 10.3390/bioengineering10091101] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Revised: 09/14/2023] [Accepted: 09/18/2023] [Indexed: 09/29/2023] Open
Abstract
To enhance the performance of surface electromyography (sEMG)-based gesture recognition, we propose a novel network-agnostic two-stage training scheme, called sEMGPoseMIM, that produces trial-invariant representations to be aligned with corresponding hand movements via cross-modal knowledge distillation. In the first stage, an sEMG encoder is trained via cross-trial mutual information maximization using the sEMG sequences sampled from the same time step but different trials in a contrastive learning manner. In the second stage, the learned sEMG encoder is fine-tuned with the supervision of gesture and hand movements in a knowledge-distillation manner. In addition, we propose a novel network called sEMGXCM as the sEMG encoder. Comprehensive experiments on seven sparse multichannel sEMG databases are conducted to demonstrate the effectiveness of the training scheme sEMGPoseMIM and the network sEMGXCM, which achieves an average improvement of +1.3% on the sparse multichannel sEMG databases compared to the existing methods. Furthermore, the comparison between training sEMGXCM and other existing networks from scratch shows that sEMGXCM outperforms the others by an average of +1.5%.
Collapse
Affiliation(s)
- Qingfeng Dai
- College of Computer Science and Technology, Faculty of Computer, Zhejiang University, Hangzhou 310058, China; (Q.D.); (X.L.)
| | - Yongkang Wong
- School of Computing, National University of Singapore, 21 Lower Kent Ridge Rd, Singapore 119077, Singapore; (Y.W.); (M.K.)
| | - Mohan Kankanhali
- School of Computing, National University of Singapore, 21 Lower Kent Ridge Rd, Singapore 119077, Singapore; (Y.W.); (M.K.)
| | - Xiangdong Li
- College of Computer Science and Technology, Faculty of Computer, Zhejiang University, Hangzhou 310058, China; (Q.D.); (X.L.)
| | | |
Collapse
|
4
|
Yu S, Zhan H, Lian X, Low SS, Xu Y, Li J, Zhang Y, Sun X, Liu J. A Smartphone-Based sEMG Signal Analysis System for Human Action Recognition. BIOSENSORS 2023; 13:805. [PMID: 37622891 PMCID: PMC10452551 DOI: 10.3390/bios13080805] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Revised: 07/26/2023] [Accepted: 08/04/2023] [Indexed: 08/26/2023]
Abstract
In lower-limb rehabilitation, human action recognition (HAR) technology can be introduced to analyze the surface electromyography (sEMG) signal generated by movements, which can provide an objective and accurate evaluation of the patient's action. To balance the long cycle required for rehabilitation and the inconvenient factors brought by wearing sEMG devices, a portable sEMG signal acquisition device was developed that can be used under daily scenarios. Additionally, a mobile application was developed to meet the demand for real-time monitoring and analysis of sEMG signals. This application can monitor data in real time and has functions such as plotting, filtering, storage, and action capture and recognition. To build the dataset required for the recognition model, six lower-limb motions were developed for rehabilitation (kick, toe off, heel off, toe off and heel up, step back and kick, and full gait). The sEMG segment and action label were combined for training a convolutional neural network (CNN) to achieve high-precision recognition performance for human lower-limb actions (with a maximum accuracy of 97.96% and recognition accuracy for all actions reaching over 97%). The results show that the smartphone-based sEMG analysis system proposed in this paper can provide reliable information for the clinical evaluation of lower-limb rehabilitation.
Collapse
Affiliation(s)
- Shixin Yu
- College of Automation Engineering, Northeast Electric Power University, Jilin 132012, China; (S.Y.); (H.Z.); (X.L.); (Y.X.); (J.L.); (Y.Z.); (X.S.)
| | - Hang Zhan
- College of Automation Engineering, Northeast Electric Power University, Jilin 132012, China; (S.Y.); (H.Z.); (X.L.); (Y.X.); (J.L.); (Y.Z.); (X.S.)
| | - Xingwang Lian
- College of Automation Engineering, Northeast Electric Power University, Jilin 132012, China; (S.Y.); (H.Z.); (X.L.); (Y.X.); (J.L.); (Y.Z.); (X.S.)
| | - Sze Shin Low
- Research Centre of Life Science and HealthCare, China Beacons Institute, University of Nottingham Ningbo China, 199 Taikang East Road, Ningbo 315100, China;
| | - Yifei Xu
- College of Automation Engineering, Northeast Electric Power University, Jilin 132012, China; (S.Y.); (H.Z.); (X.L.); (Y.X.); (J.L.); (Y.Z.); (X.S.)
| | - Jiangyong Li
- College of Automation Engineering, Northeast Electric Power University, Jilin 132012, China; (S.Y.); (H.Z.); (X.L.); (Y.X.); (J.L.); (Y.Z.); (X.S.)
| | - Yan Zhang
- College of Automation Engineering, Northeast Electric Power University, Jilin 132012, China; (S.Y.); (H.Z.); (X.L.); (Y.X.); (J.L.); (Y.Z.); (X.S.)
| | - Xiaojun Sun
- College of Automation Engineering, Northeast Electric Power University, Jilin 132012, China; (S.Y.); (H.Z.); (X.L.); (Y.X.); (J.L.); (Y.Z.); (X.S.)
| | - Jingjing Liu
- College of Automation Engineering, Northeast Electric Power University, Jilin 132012, China; (S.Y.); (H.Z.); (X.L.); (Y.X.); (J.L.); (Y.Z.); (X.S.)
| |
Collapse
|
5
|
Liu X, Zhang M, Wang J, Wang X, Liang T, Li J, Xiong P, Liu X. Gesture recognition of continuous wavelet transform and deep convolution attention network. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2023; 20:11139-11154. [PMID: 37322975 DOI: 10.3934/mbe.2023493] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/17/2023]
Abstract
To solve the problem of missing data features using a deep convolutional neural network (DCNN), this paper proposes an improved gesture recognition method. The method first extracts the time-frequency spectrogram of surface electromyography (sEMG) using the continuous wavelet transform. Then, the Spatial Attention Module (SAM) is introduced to construct the DCNN-SAM model. The residual module is embedded to improve the feature representation of relevant regions, and reduces the problem of missing features. Finally, experiments with 10 different gestures are done for verification. The results validate that the recognition accuracy of the improved method is 96.1%. Compared with the DCNN, the accuracy is improved by about 6 percentage points.
Collapse
Affiliation(s)
- Xiaoguang Liu
- College of Electronic and Information Engineering, Hebei University, Baoding, China
- Key Laboratory of Digital Medical Engineering of Hebei Province, Hebei University, Baoding, China
| | - Mingjin Zhang
- College of Electronic and Information Engineering, Hebei University, Baoding, China
- Key Laboratory of Digital Medical Engineering of Hebei Province, Hebei University, Baoding, China
| | - Jiawei Wang
- College of Electronic and Information Engineering, Hebei University, Baoding, China
- Key Laboratory of Digital Medical Engineering of Hebei Province, Hebei University, Baoding, China
| | - Xiaodong Wang
- Affiliated Hospital of Hebei University, Baoding, China
| | - Tie Liang
- College of Electronic and Information Engineering, Hebei University, Baoding, China
- Key Laboratory of Digital Medical Engineering of Hebei Province, Hebei University, Baoding, China
| | - Jun Li
- College of Electronic and Information Engineering, Hebei University, Baoding, China
- Key Laboratory of Digital Medical Engineering of Hebei Province, Hebei University, Baoding, China
| | - Peng Xiong
- College of Electronic and Information Engineering, Hebei University, Baoding, China
- Key Laboratory of Digital Medical Engineering of Hebei Province, Hebei University, Baoding, China
| | - Xiuling Liu
- College of Electronic and Information Engineering, Hebei University, Baoding, China
- Key Laboratory of Digital Medical Engineering of Hebei Province, Hebei University, Baoding, China
| |
Collapse
|
6
|
ViT-LLMR: Vision Transformer-based lower limb motion recognition from fusion signals of MMG and IMU. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104508] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/07/2023]
|
7
|
Bilius LB, Pentiuc ŞG, Vatavu RD. TIGER: A Tucker-based Instrument for Gesture Recognition with Inertial Sensors. Pattern Recognit Lett 2022. [DOI: 10.1016/j.patrec.2022.11.028] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
8
|
He Y, Li W. A Novel Lightweight Anonymous Proxy Traffic Detection Method Based on Spatio-Temporal Features. SENSORS (BASEL, SWITZERLAND) 2022; 22:4216. [PMID: 35684837 PMCID: PMC9185453 DOI: 10.3390/s22114216] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/04/2022] [Revised: 05/17/2022] [Accepted: 05/24/2022] [Indexed: 06/15/2023]
Abstract
Anonymous proxies are used by criminals for illegal network activities due to their anonymity, such as data theft and cyber attacks. Therefore, anonymous proxy traffic detection is very essential for network security. In recent years, detection based on deep learning has become a hot research topic, since deep learning can automatically extract and select traffic features. To make (heterogeneous) network traffic adapt to the homogeneous input of typical deep learning algorithms, a major branch of existing studies convert network traffic into images for detection. However, such studies are commonly subject to the limitation of large-sized image representation of network traffic, resulting in very large storage and computational resource overhead. To address this limitation, a novel method for anonymous proxy traffic detection is proposed. The method is one of the solutions to reduce storage and computational resource overhead. Specifically, it converts the sequences of the size and inter-arrival time of the first N packets of a flow into images, and then categorizes the converted images using the one-dimensional convolutional neural network. Both proprietary and public datasets are used to validate the proposed method. The experimental results show that the converted images of the method are at least 90% smaller than that of existing image-based deep learning methods. With substantially smaller image sizes, the method can still achieve F1 scores up to 98.51% in Shadowsocks traffic detection and 99.8% in VPN traffic detection.
Collapse
|