1
|
Lü C, Wang T, Xi X, Wang M, Wang J, Zhilenko A, Li L. A novel temporal-frequency combination pattern optimization approach based on information fusion for motor imagery BCIs. Comput Methods Biomech Biomed Engin 2024:1-13. [PMID: 38946233 DOI: 10.1080/10255842.2024.2371036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Accepted: 06/16/2024] [Indexed: 07/02/2024]
Abstract
Motor imagery (MI) stands as a powerful paradigm within Brain-Computer Interface (BCI) research due to its ability to induce changes in brain rhythms detectable through common spatial patterns (CSP). However, the raw feature sets captured often contain redundant and invalid information, potentially hindering CSP performance. Methodology-wise, we propose the Information Fusion for Optimizing Temporal-Frequency Combination Pattern (IFTFCP) algorithm to enhance raw feature optimization. Initially, preprocessed data undergoes simultaneous processing in both time and frequency domains via sliding overlapping time windows and filter banks. Subsequently, we introduce the Pearson-Fisher combinational method along with Discriminant Correlation Analysis (DCA) for joint feature selection and fusion. These steps aim to refine raw electroencephalogram (EEG) features. For precise classification of binary MI problems, an Radial Basis Function (RBF)-kernel Support Vector Machine classifier is trained. To validate the efficacy of IFTFCP and evaluate it against other techniques, we conducted experimental investigations using two EEG datasets. Results indicate a notably superior classification performance, boasting an average accuracy of 78.14% and 85.98% on dataset 1 and dataset 2, which is better than other methods outlined in this article. The study's findings suggest potential benefits for the advancement of MI-based BCI strategies, particularly in the domain of feature fusion.
Collapse
Affiliation(s)
- Chenyang Lü
- School of Automation, Hangzhou Dianzi University, Hangzhou, China
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou, China
| | - Ting Wang
- School of Automation, Hangzhou Dianzi University, Hangzhou, China
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou, China
| | - Xugang Xi
- School of Automation, Hangzhou Dianzi University, Hangzhou, China
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou, China
| | - Maofeng Wang
- Affiliated Dongyang Hospital of Wenzhou Medical University, Dongyang, China
| | - Jian Wang
- School of Automation, Hangzhou Dianzi University, Hangzhou, China
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou, China
| | - Anton Zhilenko
- Department of Cyber-Physical Systems, St. Petersburg State Marine Technical University, Saint-Petersburg, Russia
| | - Lihua Li
- School of Automation, Hangzhou Dianzi University, Hangzhou, China
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou, China
| |
Collapse
|
2
|
Deng H, Li M, Li J, Guo M, Xu G. A robust multi-branch multi-attention-mechanism EEGNet for motor imagery BCI decoding. J Neurosci Methods 2024; 405:110108. [PMID: 38458260 DOI: 10.1016/j.jneumeth.2024.110108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2023] [Revised: 02/28/2024] [Accepted: 03/05/2024] [Indexed: 03/10/2024]
Abstract
BACKGROUND Motor-Imagery-based Brain-Computer Interface (MI-BCI) is a promising technology to assist communication, movement, and neurological rehabilitation for motor-impaired individuals. Electroencephalography (EEG) decoding techniques using deep learning (DL) possess noteworthy advantages due to automatic feature extraction and end-to-end learning. However, the DL-based EEG decoding models tend to show large variations due to intersubject variability of EEG, which results from inconsistencies of different subjects' optimal hyperparameters. NEW METHODS This study proposes a multi-branch multi-attention mechanism EEGNet model (MBMANet) for robust decoding. It applies the multi-branch EEGNet structure to achieve various feature extractions. Further, the different attention mechanisms introduced in each branch attain diverse adaptive weight adjustments. This combination of multi-branch and multi-attention mechanisms allows for multi-level feature fusion to provide robust decoding for different subjects. RESULTS The MBMANet model has a four-classification accuracy of 83.18% and kappa of 0.776 on the BCI Competition IV-2a dataset, which outperforms other eight CNN-based decoding models. This consistently satisfactory performance across all nine subjects indicates that the proposed model is robust. CONCLUSIONS The combine of multi-branch and multi-attention mechanisms empowers the DL-based models to adaptively learn different EEG features, which provides a feasible solution for dealing with data variability. It also gives the MBMANet model more accurate decoding of motion intentions and lower training costs, thus improving the MI-BCI's utility and robustness.
Collapse
Affiliation(s)
- Haodong Deng
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China; Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, Tianjin 300132, China; Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Hebei University of Technology, Tianjin 300132, China; School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China
| | - Mengfan Li
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China; Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, Tianjin 300132, China; Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Hebei University of Technology, Tianjin 300132, China; School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China.
| | - Jundi Li
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China; Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, Tianjin 300132, China; Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Hebei University of Technology, Tianjin 300132, China; School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China
| | - Miaomiao Guo
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China; Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, Tianjin 300132, China; Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Hebei University of Technology, Tianjin 300132, China; School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China
| | - Guizhi Xu
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China; Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, Tianjin 300132, China; Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Hebei University of Technology, Tianjin 300132, China; School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300132, China
| |
Collapse
|
3
|
Qin Y, Yang B, Ke S, Liu P, Rong F, Xia X. M-FANet: Multi-Feature Attention Convolutional Neural Network for Motor Imagery Decoding. IEEE Trans Neural Syst Rehabil Eng 2024; 32:401-411. [PMID: 38194394 DOI: 10.1109/tnsre.2024.3351863] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2024]
Abstract
Motor imagery (MI) decoding methods are pivotal in advancing rehabilitation and motor control research. Effective extraction of spectral-spatial-temporal features is crucial for MI decoding from limited and low signal-to-noise ratio electroencephalogram (EEG) signal samples based on brain-computer interface (BCI). In this paper, we propose a lightweight Multi-Feature Attention Neural Network (M-FANet) for feature extraction and selection of multi-feature data. M-FANet employs several unique attention modules to eliminate redundant information in the frequency domain, enhance local spatial feature extraction and calibrate feature maps. We introduce a training method called Regularized Dropout (R-Drop) to address training-inference inconsistency caused by dropout and improve the model's generalization capability. We conduct extensive experiments on the BCI Competition IV 2a (BCIC-IV-2a) dataset and the 2019 World robot conference contest-BCI Robot Contest MI (WBCIC-MI) dataset. M-FANet achieves superior performance compared to state-of-the-art MI decoding methods, with 79.28% 4-class classification accuracy (kappa: 0.7259) on the BCIC-IV-2a dataset and 77.86% 3-class classification accuracy (kappa: 0.6650) on the WBCIC-MI dataset. The application of multi-feature attention modules and R-Drop in our lightweight model significantly enhances its performance, validated through comprehensive ablation experiments and visualizations.
Collapse
|
4
|
Wang X, Liesaputra V, Liu Z, Wang Y, Huang Z. An in-depth survey on Deep Learning-based Motor Imagery Electroencephalogram (EEG) classification. Artif Intell Med 2024; 147:102738. [PMID: 38184362 DOI: 10.1016/j.artmed.2023.102738] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2022] [Revised: 10/16/2023] [Accepted: 11/28/2023] [Indexed: 01/08/2024]
Abstract
Electroencephalogram (EEG)-based Brain-Computer Interfaces (BCIs) build a communication path between human brain and external devices. Among EEG-based BCI paradigms, the most commonly used one is motor imagery (MI). As a hot research topic, MI EEG-based BCI has largely contributed to medical fields and smart home industry. However, because of the low signal-to-noise ratio (SNR) and the non-stationary characteristic of EEG data, it is difficult to correctly classify different types of MI-EEG signals. Recently, the advances in Deep Learning (DL) significantly facilitate the development of MI EEG-based BCIs. In this paper, we provide a systematic survey of DL-based MI-EEG classification methods. Specifically, we first comprehensively discuss several important aspects of DL-based MI-EEG classification, covering input formulations, network architectures, public datasets, etc. Then, we summarize problems in model performance comparison and give guidelines to future studies for fair performance comparison. Next, we fairly evaluate the representative DL-based models using source code released by the authors and meticulously analyse the evaluation results. By performing ablation study on the network architecture, we found that (1) effective feature fusion is indispensable for multi-stream CNN-based models. (2) LSTM should be combined with spatial feature extraction techniques to obtain good classification performance. (3) the use of dropout contributes little to improving the model performance, and that (4) adding fully connected layers to the models significantly increases their parameters but it might not improve their performance. Finally, we raise several open issues in MI-EEG classification and provide possible future research directions.
Collapse
Affiliation(s)
- Xianheng Wang
- Department of Computer Science, University of Otago, Dunedin, New Zealand.
| | | | - Zhaobin Liu
- College of Information Science and Technology, Dalian Maritime University, Liaoning, PR China
| | - Yi Wang
- Laboratory for Circuit and Behavioral Physiology, RIKEN Center for Brain Science, Wako-shi, Saitama, Japan
| | - Zhiyi Huang
- Department of Computer Science, University of Otago, Dunedin, New Zealand.
| |
Collapse
|
5
|
Zhang D, Li H, Xie J, Li D. MI-DAGSC: A domain adaptation approach incorporating comprehensive information from MI-EEG signals. Neural Netw 2023; 167:183-198. [PMID: 37659115 DOI: 10.1016/j.neunet.2023.08.008] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Revised: 07/24/2023] [Accepted: 08/06/2023] [Indexed: 09/04/2023]
Abstract
Non-stationarity of EEG signals leads to high variability between subjects, making it challenging to directly use data from other subjects (source domain) for the classifier in the current subject (target domain). In this study, we propose MI-DAGSC to address domain adaptation challenges in EEG-based motor imagery (MI) decoding. By combining domain-level information, class-level information, and inter-sample structure information, our model effectively aligns the feature distributions of source and target domains. This work is an extension of our previous domain adaptation work MI-DABAN (Li et al., 2023). Based on MI-DABAN, MI-DAGSC designs Sample-Feature Blocks (SFBs) and Graph Convolution Blocks (GCBs) to focus on intra-sample and inter-sample information. The synergistic integration of SFBs and GCBs enable the model to capture comprehensive information and understand the relationship between samples, thus improving representation learning. Furthermore, we introduce a triplet loss to enhance the alignment and compactness of feature representations. Extensive experiments on real EEG datasets demonstrate the effectiveness of MI-DAGSC, confirming that our method makes a valuable contribution to the MI-EEG decoding. Moreover, it holds great potential for various applications in brain-computer interface systems and neuroscience research. And the code of the proposed architecture in this study is available under https://github.com/zhangdx21/MI-DAGSC.
Collapse
Affiliation(s)
- Dongxue Zhang
- Jilin University, College of Computer Science and Technology, Changchun, Jilin Province, China; Key Laboratory of Symbol Computation and Knowledge Engineering, Jilin University, Changchun 130012, China.
| | - Huiying Li
- Jilin University, College of Computer Science and Technology, Changchun, Jilin Province, China; Key Laboratory of Symbol Computation and Knowledge Engineering, Jilin University, Changchun 130012, China.
| | - Jingmeng Xie
- Xi'an Jiaotong University, College of Electronic information, Xi'an, Shanxi Province, China.
| | - Dajun Li
- Jilin Provincial People's Hospital, Changchun, Jilin Province, China
| |
Collapse
|