1
|
Cai M, Hong J. Joint multi-feature extraction and transfer learning in motor imagery brain computer interface. Comput Methods Biomech Biomed Engin 2024:1-12. [PMID: 39286921 DOI: 10.1080/10255842.2024.2404541] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2024] [Revised: 08/14/2024] [Accepted: 09/10/2024] [Indexed: 09/19/2024]
Abstract
Motor imagery brain computer interface (BCI) systems are considered one of the most crucial paradigms and have received extensive attention from researchers worldwide. However, the non-stationary from subject-to-subject transfer is a substantial challenge for robust BCI operations. To address this issue, this paper proposes a novel approach that integrates joint multi-feature extraction, specifically combining common spatial patterns (CSP) and wavelet packet transforms (WPT), along with transfer learning (TL) in motor imagery BCI systems. This approach leverages the time-frequency characteristics of WPT and the spatial characteristics of CSP while utilizing transfer learning to facilitate EEG identification for target subjects based on knowledge acquired from non-target subjects. Using dataset IVa from BCI Competition III, our proposed approach achieves an impressive average classification accuracy of 93.4%, outperforming five kinds of state-of-the-art approaches. Furthermore, it offers the advantage of enabling the design of various auxiliary problems to learn different aspects of the target problem from unlabeled data through transfer learning, thereby facilitating the implementation of innovative ideas within our proposed approach. Simultaneously, it demonstrates that integrating CSP and WPT while transferring knowledge from other subjects is highly effective in enhancing the average classification accuracy of EEG signals and it provides a novel solution to address subject-to-subject transfer challenges in motor imagery BCI systems.
Collapse
Affiliation(s)
- Miao Cai
- Department of Integrated Traditional Chinese and Western Medicine, Xi'an Children's Hospital, Xi'an, China
| | - Jie Hong
- School of Mechanical Engineering, Northwestern Polytechnical University, Xi'an, China
| |
Collapse
|
2
|
Huang Z, Wei Q. Tensor decomposition-based channel selection for motor imagery-based brain-computer interfaces. Cogn Neurodyn 2024; 18:877-892. [PMID: 39534365 PMCID: PMC11551095 DOI: 10.1007/s11571-023-09940-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 01/02/2023] [Accepted: 01/29/2023] [Indexed: 11/16/2024] Open
Abstract
The number of electrode channels in a brain-computer interface (BCI) affects not only its classification performance, but also its convenience in practical applications. Despite many studies on channel selection in motor imagery (MI)-based BCI systems, they consist in matrix analysis of EEG signals, which inevitably loses the interactive information among multiple domains such as space, time and frequency. In this paper, a tensor decomposition-based channel selection (TCS) method is employed for MI BCIs. A three-way tensor is yielded by wavelet transform of a single-trial EEG signal and decomposed into three factor matrices by a regularized canonical polyadic decomposition (CPD). The channel factor matrix is used for channel selection and the important channels are selected by calculating the correlation between channels. Regularized common spatial pattern (RCSP) is employed for feature extraction and support vector machine (SVM) for classification. The proposed TCS-RCSP algorithm was evaluated on three BCI data sets and compared with the RCSP with all channels (AC-RCSP) and the RCSP with selected channels by correlation-based channel selection method (CCS-RCSP). The results indicate that TCS-RCSP achieved significantly better overall accuracy than AC-RCSP (94.4% vs. 86.3%) with ρ < 0.01 and CCS-RCSP (94.4% vs. 90.2%) with ρ < 0.05, proving the efficacy of the proposed algorithm for classifying MI tasks.
Collapse
Affiliation(s)
- Ziwei Huang
- Department of Electronic Information Engineering, School of Information Engineering, Nanchang University, Nanchang, 330031 Jiangxi China
| | - Qingguo Wei
- Department of Electronic Information Engineering, School of Information Engineering, Nanchang University, Nanchang, 330031 Jiangxi China
| |
Collapse
|
3
|
Xie X, Chen L, Qin S, Zha F, Fan X. Bidirectional feature pyramid attention-based temporal convolutional network model for motor imagery electroencephalogram classification. Front Neurorobot 2024; 18:1343249. [PMID: 38352723 PMCID: PMC10861766 DOI: 10.3389/fnbot.2024.1343249] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2023] [Accepted: 01/15/2024] [Indexed: 02/16/2024] Open
Abstract
Introduction As an interactive method gaining popularity, brain-computer interfaces (BCIs) aim to facilitate communication between the brain and external devices. Among the various research topics in BCIs, the classification of motor imagery using electroencephalography (EEG) signals has the potential to greatly improve the quality of life for people with disabilities. Methods This technology assists them in controlling computers or other devices like prosthetic limbs, wheelchairs, and drones. However, the current performance of EEG signal decoding is not sufficient for real-world applications based on Motor Imagery EEG (MI-EEG). To address this issue, this study proposes an attention-based bidirectional feature pyramid temporal convolutional network model for the classification task of MI-EEG. The model incorporates a multi-head self-attention mechanism to weigh significant features in the MI-EEG signals. It also utilizes a temporal convolution network (TCN) to separate high-level temporal features. The signals are enhanced using the sliding-window technique, and channel and time-domain information of the MI-EEG signals is extracted through convolution. Results Additionally, a bidirectional feature pyramid structure is employed to implement attention mechanisms across different scales and multiple frequency bands of the MI-EEG signals. The performance of our model is evaluated on the BCI Competition IV-2a dataset and the BCI Competition IV-2b dataset, and the results showed that our model outperformed the state-of-the-art baseline model, with an accuracy of 87.5 and 86.3% for the subject-dependent, respectively. Discussion In conclusion, the BFATCNet model offers a novel approach for EEG-based motor imagery classification in BCIs, effectively capturing relevant features through attention mechanisms and temporal convolutional networks. Its superior performance on the BCI Competition IV-2a and IV-2b datasets highlights its potential for real-world applications. However, its performance on other datasets may vary, necessitating further research on data augmentation techniques and integration with multiple modalities to enhance interpretability and generalization. Additionally, reducing computational complexity for real-time applications is an important area for future work.
Collapse
Affiliation(s)
- Xinghe Xie
- Shenzhen Academy of Robotics, Shenzhen, Guangdong Province, China
- Faculty of Applied Science, Macao Polytechnic University, Macau, Macao SAR, China
| | - Liyan Chen
- Shenzhen Academy of Robotics, Shenzhen, Guangdong Province, China
| | - Shujia Qin
- Shenzhen Academy of Robotics, Shenzhen, Guangdong Province, China
| | - Fusheng Zha
- Harbin Institute of Technology, Harbin, Heilongjiang Province, China
| | - Xinggang Fan
- Information Engineering College, Zhijiang College of Zhejiang University of Technology, Shaoxing, China
| |
Collapse
|
4
|
Wang S, Luo Z, Zhao S, Zhang Q, Liu G, Wu D, Yin E, Chen C. Classification of EEG Signals Based on Sparrow Search Algorithm-Deep Belief Network for Brain-Computer Interface. Bioengineering (Basel) 2023; 11:30. [PMID: 38247907 PMCID: PMC10813095 DOI: 10.3390/bioengineering11010030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2023] [Revised: 12/15/2023] [Accepted: 12/22/2023] [Indexed: 01/23/2024] Open
Abstract
In brain-computer interface (BCI) systems, challenges are presented by the recognition of motor imagery (MI) brain signals. Established recognition approaches have achieved favorable performance from patterns like SSVEP, AEP, and P300, whereas the classification methods for MI need to be improved. Hence, seeking a classification method that exhibits high accuracy and robustness for application in MI-BCI systems is essential. In this study, the Sparrow search algorithm (SSA)-optimized Deep Belief Network (DBN), called SSA-DBN, is designed to recognize the EEG features extracted by the Empirical Mode Decomposition (EMD). The performance of the DBN is enhanced by the optimized hyper-parameters obtained through the SSA. Our method's efficacy was tested on three datasets: two public and one private. Results indicate a relatively high accuracy rate, outperforming three baseline methods. Specifically, on the private dataset, our approach achieved an accuracy of 87.83%, marking a significant 10.38% improvement over the standard DBN algorithm. For the BCI IV 2a dataset, we recorded an accuracy of 86.14%, surpassing the DBN algorithm by 9.33%. In the SMR-BCI dataset, our method attained a classification accuracy of 87.21%, which is 5.57% higher than that of the conventional DBN algorithm. This study demonstrates enhanced classification capabilities in MI-BCI, potentially contributing to advancements in the field of BCI.
Collapse
Affiliation(s)
- Shuai Wang
- School of Electrical Engineering and Automation, Tianjin University of Technology, Tianjin 300380, China; (S.W.); (Q.Z.); (G.L.); (D.W.)
| | - Zhiguo Luo
- Defense Innovation Institute, Academy of Military Sciences (AMS), Beijing 100071, China; (Z.L.); (S.Z.)
| | - Shaokai Zhao
- Defense Innovation Institute, Academy of Military Sciences (AMS), Beijing 100071, China; (Z.L.); (S.Z.)
| | - Qilong Zhang
- School of Electrical Engineering and Automation, Tianjin University of Technology, Tianjin 300380, China; (S.W.); (Q.Z.); (G.L.); (D.W.)
| | - Guangrong Liu
- School of Electrical Engineering and Automation, Tianjin University of Technology, Tianjin 300380, China; (S.W.); (Q.Z.); (G.L.); (D.W.)
| | - Dongyue Wu
- School of Electrical Engineering and Automation, Tianjin University of Technology, Tianjin 300380, China; (S.W.); (Q.Z.); (G.L.); (D.W.)
| | - Erwei Yin
- Defense Innovation Institute, Academy of Military Sciences (AMS), Beijing 100071, China; (Z.L.); (S.Z.)
| | - Chao Chen
- School of Electrical Engineering and Automation, Tianjin University of Technology, Tianjin 300380, China; (S.W.); (Q.Z.); (G.L.); (D.W.)
| |
Collapse
|
5
|
Maghsoudi A, Shalbaf A. Hand Motor Imagery Classification Using Effective Connectivity and Hierarchical Machine Learning in EEG Signals. J Biomed Phys Eng 2022; 12:161-170. [PMID: 35433527 PMCID: PMC8995751 DOI: 10.31661/jbpe.v0i0.1264] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2019] [Accepted: 11/14/2019] [Indexed: 11/16/2022]
Abstract
Background Motor Imagery (MI) Brain Computer Interface (BCI) directly links central nervous system to a computer or a device. Most MI-BCI structures rely on features of a single channel of Electroencephalogram (EEG). However, to provide more valuable features, the relationships among EEG channels in the form of effective brain connectivity analysis must be considered. Objective This study aims to identify a set of robust and discriminative effective connectivity features from EEG signals and to develop a hierarchical machine learning structure for discrimination of left and right hand MI task effectively. Material and Methods In this analytical study, we estimated effective connectivity using Granger Causality (GC) methods namely, Generalized Partial Directed Coherence (GPDC), Directed Transfer Function (DTF) and direct Directed Transfer Function (dDTF). These measures determine the transient causal relation between different brain areas. Then a feature subset selection method based on Kruskal-Wallis test was performed to choose most significant directed causal connection between channels. Moreover, the minimal-redundancy-maximal-relevance feature selection method is applied to discard non-significance features. Finally, support vector machine method is used for classification. Results The maximum value of the classification accuracies using GC methods over different frequency bands in 29 subjects in 60 trial is approximately 84% in Mu (8-12 Hz) - Beta1 (12 - 15 Hz) frequency band using GPDC method. Conclusion This new hierarchical automated BCI system could be applied for discrimination of left and right hand MI tasks from EEG signal, effectively.
Collapse
Affiliation(s)
- Arash Maghsoudi
- PhD, Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
| | - Ahmad Shalbaf
- PhD, Department of Biomedical Engineering and Medical Physics, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| |
Collapse
|
6
|
Mattioli F, Porcaro C, Baldassarre G. A 1D CNN for high accuracy classification and transfer learning in motor imagery EEG-based brain-computer interface. J Neural Eng 2021; 18. [PMID: 34920443 DOI: 10.1088/1741-2552/ac4430] [Citation(s) in RCA: 34] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2021] [Accepted: 12/17/2021] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Brain-computer interface (BCI) aims to establish communication paths between the brain processes and external devices. Different methods have been used to extract human intentions from electroencephalography (EEG) recordings. Those based on motor imagery (MI) seem to have a great potential for future applications. These approaches rely on the extraction of EEG distinctive patterns during imagined movements. Techniques able to extract patterns from raw signals represent an important target for BCI as they do not need labor-intensive data pre-processing. APPROACH We propose a new approach based on a 10-layer one-dimensional convolution neural network (1D-CNN) to classify five brain states (four MI classes plus a 'baseline' class) using a data augmentation algorithm and a limited number of EEG channels. In addition, we present a transfer learning method used to extract critical features from the EEG group dataset and then to customize the model to the single individual by training its outer layers with only 12-minute individual-related data. MAIN RESULTS The model tested with the 'EEG Motor Movement/Imagery Dataset' outperforms the current state-of-the-art models by achieving a 99.38% accuracy at the group level. In addition, the transfer learning approach we present achieves an average accuracy of 99.46%. SIGNIFICANCE The proposed methods could foster future BCI applications relying on few-channel portable recording devices and individual-based training.
Collapse
Affiliation(s)
- Francesco Mattioli
- Institute of Cognitive Sciences and Technologies (ISTC), CNR, Via San Martino della Battaglia, Roma, Lazio, 00185, ITALY
| | - Camillo Porcaro
- Istituto di Scienze e Tecnologie della Cognizione Consiglio Nazionale delle Ricerche, Via S. Martino della Battaglia, 44, Roma, 00185, ITALY
| | - Gianluca Baldassarre
- Istituto di Scienze e Tecnologie della Cognizione Consiglio Nazionale delle Ricerche, Via S. Martino della Battaglia, 44, Roma, 00185, ITALY
| |
Collapse
|
7
|
Spatio-time-frequency joint sparse optimization with transfer learning in motor imagery-based brain-computer interface system. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102702] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
8
|
Çınar S. Design of an automatic hybrid system for removal of eye-blink artifacts from EEG recordings. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102543] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
9
|
Singh A, Hussain AA, Lal S, Guesgen HW. A Comprehensive Review on Critical Issues and Possible Solutions of Motor Imagery Based Electroencephalography Brain-Computer Interface. SENSORS 2021; 21:s21062173. [PMID: 33804611 PMCID: PMC8003721 DOI: 10.3390/s21062173] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/28/2020] [Revised: 03/15/2021] [Accepted: 03/16/2021] [Indexed: 01/16/2023]
Abstract
Motor imagery (MI) based brain–computer interface (BCI) aims to provide a means of communication through the utilization of neural activity generated due to kinesthetic imagination of limbs. Every year, a significant number of publications that are related to new improvements, challenges, and breakthrough in MI-BCI are made. This paper provides a comprehensive review of the electroencephalogram (EEG) based MI-BCI system. It describes the current state of the art in different stages of the MI-BCI (data acquisition, MI training, preprocessing, feature extraction, channel and feature selection, and classification) pipeline. Although MI-BCI research has been going for many years, this technology is mostly confined to controlled lab environments. We discuss recent developments and critical algorithmic issues in MI-based BCI for commercial deployment.
Collapse
|
10
|
A Novel Simplified Convolutional Neural Network Classification Algorithm of Motor Imagery EEG Signals Based on Deep Learning. APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10051605] [Citation(s) in RCA: 35] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Left and right hand motor imagery electroencephalogram (MI-EEG) signals are widely used in brain-computer interface (BCI) systems to identify a participant intent in controlling external devices. However, due to a series of reasons, including low signal-to-noise ratios, there are great challenges for efficient motor imagery classification. The recognition of left and right hand MI-EEG signals is vital for the application of BCI systems. Recently, the method of deep learning has been successfully applied in pattern recognition and other fields. However, there are few effective deep learning algorithms applied to BCI systems, particularly for MI based BCI. In this paper, we propose an algorithm that combines continuous wavelet transform (CWT) and a simplified convolutional neural network (SCNN) to improve the recognition rate of MI-EEG signals. Using the CWT, the MI-EEG signals are mapped to time-frequency image signals. Then the image signals are input into the SCNN to extract the features and classify them. Tested by the BCI Competition IV Dataset 2b, the experimental results show that the average classification accuracy of the nine subjects is 83.2%, and the mean kappa value is 0.651, which is 11.9% higher than that of the champion in the BCI Competition IV. Compared with other algorithms, the proposed CWT-SCNN algorithm has a better classification performance and a shorter training time. Therefore, this algorithm could enhance the classification performance of MI based BCI and be applied in real-time BCI systems for use by disabled people.
Collapse
|
11
|
Wu X, Zhou B, Lv Z, Zhang C. To Explore the Potentials of Independent Component Analysis in Brain-Computer Interface of Motor Imagery. IEEE J Biomed Health Inform 2019; 24:775-787. [PMID: 31217132 DOI: 10.1109/jbhi.2019.2922976] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
This paper is focused on the experimental approach to explore the potential of independent component analysis (ICA) in the context of motor imagery (MI)-based brain-computer interface (BCI). We presented a simple and efficient algorithmic framework of ICA-based MI BCI (ICA-MIBCI) for the evaluation of four classical ICA algorithms (Infomax, FastICA, Jade, and Sobi) as well as a simplified Infomax (sInfomax). Two novel performance indexes, self-test accuracy and the number of invalid ICA filters, were employed to assess the performance of MIBCI based on different ICA variants. As a reference method, common spatial pattern (CSP), a commonly-used spatial filtering method, was employed for the comparative study between ICA-MIBCI and CSP-MIBCI. The experimental results showed that sInfomax-based spatial filters exhibited significantly better transferability in session to session and subject to subject transfer as compared to CSP-based spatial filters. The online experiment was also introduced to demonstrate the practicability and feasibility of sInfomax-based MIBCI. However, four classical ICA variants, especially FastICA, Jade, and Sobi, performed much worse as compared to sInfomax and CSP in terms of classification accuracy and stability. We consider that conventional ICA-based spatial filtering methods tend to be overfitting while applied to real-life electroencephalogram data. Nevertheless, the sInfomax-based experimental results indicate that ICA methods have a great space for improvement in the application of MIBCI. We believe that this paper could bring forth new ideas for the practical implementation of ICA-MIBCI.
Collapse
|