1
|
Chaudhary P, Dhankhar N, Singhal A, Rana KPS. A two-stage transformer based network for motor imagery classification. Med Eng Phys 2024; 128:104154. [PMID: 38697881 DOI: 10.1016/j.medengphy.2024.104154] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 02/18/2024] [Accepted: 03/16/2024] [Indexed: 05/05/2024]
Abstract
Brain-computer interfaces (BCIs) are used to understand brain functioning and develop therapies for neurological and neurodegenerative disorders. Therefore, BCIs are crucial in rehabilitating motor dysfunction and advancing motor imagery applications. For motor imagery, electroencephalogram (EEG) signals are used to classify the subject's intention of moving a body part without actually moving it. This paper presents a two-stage transformer-based architecture that employs handcrafted features and deep learning techniques to enhance the classification performance on benchmarked EEG signals. Stage-1 is built on parallel convolution based EEGNet, multi-head attention, and separable temporal convolution networks for spatiotemporal feature extraction. Further, for enhanced classification, in stage-2, additional features and embeddings extracted from stage-1 are used to train TabNet. In addition, a novel channel cluster swapping data augmentation technique is also developed to handle the issue of limited samples for training deep learning architectures. The developed two-stage architecture offered an average classification accuracy of 88.5 % and 88.3 % on the BCI Competition IV-2a and IV-2b datasets, respectively, which is approximately 3.0 % superior over similar recent reported works.
Collapse
Affiliation(s)
- Priyanshu Chaudhary
- Department of Electronics and Communication Engineering, Netaji Subhas University of Technology, Delhi, India
| | - Nischay Dhankhar
- Department of Electronics and Communication Engineering, Netaji Subhas University of Technology, Delhi, India.
| | - Amit Singhal
- Department of Electronics and Communication Engineering, Netaji Subhas University of Technology, Delhi, India
| | - K P S Rana
- Department of Instrumentation and Control Engineering, Netaji Subhas University of Technology, Delhi, India
| |
Collapse
|
2
|
Ma W, Wang C, Sun X, Lin X, Niu L, Wang Y. MBGA-Net: A multi-branch graph adaptive network for individualized motor imagery EEG classification. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 240:107641. [PMID: 37327754 DOI: 10.1016/j.cmpb.2023.107641] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Revised: 05/21/2023] [Accepted: 05/31/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND AND OBJECTIVE The development of deep learning has led to significant improvements in the decoding accuracy of Motor Imagery (MI) EEG signal classification. However, current models are inadequate in ensuring high levels of classification accuracy for an individual. Since MI EEG data is primarily used in medical rehabilitation and intelligent control, it is crucial to ensure that each individual's EEG signal is recognized with precision. METHODS We propose a multi-branch graph adaptive network (MBGA-Net), which matches each individual EEG signal with a suitable time-frequency domain processing method based on spatio-temporal domain features. We then feed the signal into the relevant model branch using an adaptive technique. Through an enhanced attention mechanism and deep convolutional method with residual connectivity, each model branch more effectively harvests the features of the related format data. RESULTS We validate the proposed model using the BCI Competition IV dataset 2a and dataset 2b. On dataset 2a, the average accuracy and kappa values are 87.49% and 0.83, respectively. The standard deviation of individual kappa values is only 0.08. For dataset 2b, the average classification accuracies obtained by feeding the data into the three branches of MBGA-Net are 85.71%, 85.83%, and 86.99%, respectively. CONCLUSIONS The experimental results demonstrate that MBGA-Net could effectively perform the classification task of motor imagery EEG signals, and it exhibits strong generalization performance. The proposed adaptive matching technique enhances the classification accuracy of each individual, which is beneficial for the practical application of EEG classification.
Collapse
Affiliation(s)
- Weifeng Ma
- School of Information and Electronic Engineering, Zhejiang University of Science and Technology, Hangzhou 310023, PR China
| | - Chuanlai Wang
- School of Information and Electronic Engineering, Zhejiang University of Science and Technology, Hangzhou 310023, PR China
| | - Xiaoyong Sun
- School of Information and Electronic Engineering, Zhejiang University of Science and Technology, Hangzhou 310023, PR China
| | - Xuefen Lin
- School of Information and Electronic Engineering, Zhejiang University of Science and Technology, Hangzhou 310023, PR China
| | - Lei Niu
- Faculty of Artificial Intelligence Education, Central China Normal University Wollongong Joint Institude, Wuhan 430079, PR China
| | - Yuchen Wang
- School of Information and Electronic Engineering, Zhejiang University of Science and Technology, Hangzhou 310023, PR China.
| |
Collapse
|
3
|
Ma W, Wang C, Sun X, Lin X, Wang Y. A double-branch graph convolutional network based on individual differences weakening for motor imagery EEG classification. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104684] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/24/2023]
|
4
|
Li MA, Ruan ZW. Decoding motor imagery with a simplified distributed dipoles model at source level. Cogn Neurodyn 2023; 17:445-457. [PMID: 37007206 PMCID: PMC10050617 DOI: 10.1007/s11571-022-09826-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2021] [Revised: 04/21/2022] [Accepted: 05/20/2022] [Indexed: 11/29/2022] Open
Abstract
Motor imagery (MI) based brain computer interface significantly oriented the development of neuro-rehabilitation, and the crucial issue is how to accurately detect the changes of cerebral cortex for MI decoding. The brain activity can be calculated based on the head model and observed scalp EEG, providing insights regarding cortical dynamics by using equivalent current dipoles with high spatial and temporal resolution. Now, all the dipoles within entire cortex or partial regions of interest are directly applied to data representation, this may make the key information weakened or lost, and it is worth studying how to choose the most important from numerous dipoles. In this paper, we devote to building a simplified distributed dipoles model (SDDM), which is combined with convolutional neural network (CNN), generating a MI decoding method at source level (called SDDM-CNN). First, all channels of raw MI-EEG signals are subdivided by a series of bandpass filters with width of 1 Hz, the average energies associated with any sub-band signals are calculated and ranked in a descending order to screen the top n sub-bands; then, the MI-EEG signals over each selected sub-band are mapped into source space by using EEG source imaging technology, and for each scout of neuroanatomical Desikan-Killiany partition, a centered dipole is selected as the most relevant dipole and put together to build a SDDM to reflect the neuroelectric activity of entire cerebral cortex; finally, the 4 dimensional (4D) magnitude matrix is constructed for each SDDM and fused into a novel data representation, which is further input to a well-designed 3DCNN with n parallel branches (nB3DCNN) to extract and classify the comprehensive features from time-frequency-space dimensions. Experiments are carried out on three public datasets, and the average ten-fold CV decoding accuracies achieve 95.09%, 97.98% and 94.53% respectively, and the statistical analysis is fulfilled by standard deviation, kappa value and confusion matrix. Experiment results suggest that it is beneficial to pick out the most sensitive sub-bands in sensor domain, and SDDM can sufficiently describe the dynamic changing of entire cortex, improving decoding performance while greatly reducing number of source signals. Also, nB3DCNN is capable of exploring spatial-temporal features from multi sub-bands.
Collapse
Affiliation(s)
- Ming-ai Li
- Faculty of Information Technology, Beijing University of Technology, Beijing, 100124 China
- Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing, 100124 China
- Engineering Research Center of Digital Community, Ministry of Education, Beijing, 100124 China
| | - Zi-wei Ruan
- Faculty of Information Technology, Beijing University of Technology, Beijing, 100124 China
| |
Collapse
|
5
|
A compact multi-branch 1D convolutional neural network for EEG-based motor imagery classification. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104456] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
|
6
|
Qiu Y, Wu Z, Wang J, Zhang C, Zhang H. Introduction of Materials Genome Technology and Its Applications in the Field of Biomedical Materials. MATERIALS (BASEL, SWITZERLAND) 2023; 16:1906. [PMID: 36903027 PMCID: PMC10004319 DOI: 10.3390/ma16051906] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/09/2022] [Revised: 02/13/2023] [Accepted: 02/17/2023] [Indexed: 06/18/2023]
Abstract
Traditional research and development (R&D) on biomedical materials depends heavily on the trial and error process, thereby leading to huge economic and time burden. Most recently, materials genome technology (MGT) has been recognized as an effective approach to addressing this problem. In this paper, the basic concepts involved in the MGT are introduced, and the applications of MGT in the R&D of metallic, inorganic non-metallic, polymeric, and composite biomedical materials are summarized; in view of the existing limitations of MGT for R&D of biomedical materials, potential strategies are proposed on the establishment and management of material databases, the upgrading of high-throughput experimental technology, the construction of data mining prediction platforms, and the training of relevant materials talents. In the end, future trend of MGT for R&D of biomedical materials is proposed.
Collapse
Affiliation(s)
| | | | | | - Chao Zhang
- Correspondence: (C.Z.); (H.Z.); Tel.: +86-20-39332145 (C.Z. & H.Z.)
| | - Heye Zhang
- Correspondence: (C.Z.); (H.Z.); Tel.: +86-20-39332145 (C.Z. & H.Z.)
| |
Collapse
|
7
|
A novel multi-branch hybrid neural network for motor imagery EEG signal classification. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103718] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
8
|
Arpaia P, Esposito A, Natalizio A, Parvis M. How to successfully classify EEG in motor imagery BCI: a metrological analysis of the state of the art. J Neural Eng 2022; 19. [PMID: 35640554 DOI: 10.1088/1741-2552/ac74e0] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2021] [Accepted: 05/31/2022] [Indexed: 11/11/2022]
Abstract
Objective. Processing strategies are analysed with respect to the classification of electroencephalographic signals related to brain-computer interfaces based on motor imagery. A review of literature is carried out to understand the achievements in motor imagery classification, the most promising trends, and the challenges in replicating these results. Main focus is placed on performance by means of a rigorous metrological analysis carried out in compliance with the international vocabulary of metrology. Hence, classification accuracy and its uncertainty are considered, as well as repeatability and reproducibility.Approach. The paper works included in the review concern the classification of electroencephalographic signals in motor-imagery- based brain-computer interfaces. Article search was carried out in accordance with the PRISMA standard and 89 studies were included.Main results. Statistically-based analyses show that brain-inspired approaches are increasingly proposed, and that these are particularly successful in discriminating against multiple classes. Notably, many proposals involve convolutional neural networks. Instead, classical machine learning approaches are still effective for binary classifications. Many proposals combine common spatial pattern, least absolute shrinkage and selection operator, and support vector machines. Regarding reported classification accuracies, performance above the upper quartile is in the 85 % to 100 % range for the binary case and in the 83 % to 93 % range for multi-class one. Associated uncertainties are up to 6 % while repeatability for a predetermined dataset is up to 8 %. Reproducibility assessment was instead prevented by lack of standardization in experiments.Significance. By relying on the analysed studies, the reader is guided towards the development of a successful processing strategy as a crucial part of a brain-computer interface. Moreover, it is suggested that future studies should extend these approaches on data from more subjects and with custom experiments, even by investigating online operation. This would also enable the quantification of results reproducibility.
Collapse
Affiliation(s)
- Pasquale Arpaia
- Centro Interdipartimentale di Ricerca in Management Sanitario e Innovazione in Sanità, Università degli Studi di Napoli Federico II, Via Claudio, 21, Napoli, Campania, 80125, ITALY
| | - Antonio Esposito
- Department of Electronics and Telecommunications (DET), Politecnico di Torino, Corso Castelfidardo, 39, Torino, 10129, ITALY
| | - Angela Natalizio
- Department of Electronics and Telecommunications (DET), Politecnico di Torino, Corso Castelfidardo, 39, Torino, Piemonte, 10129, ITALY
| | - Marco Parvis
- Department of Electronics and Telecommunications (DET), Politecnico di Torino, Corso Castelfidardo, 39, Torino, Piemonte, 10129, ITALY
| |
Collapse
|
9
|
Ma W, Gong Y, Xue H, Liu Y, Lin X, Zhou G, Li Y. A lightweight and accurate double-branch neural network for four-class motor imagery classification. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103582] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
10
|
Liu H, Liu Y, Zhang R, Wu X. Student Behavior Recognition From Heterogeneous View Perception in Class Based on 3-D Multiscale Residual Dense Network for the Analysis of Case Teaching. Front Neurorobot 2021; 15:675827. [PMID: 34393749 PMCID: PMC8358832 DOI: 10.3389/fnbot.2021.675827] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Accepted: 04/27/2021] [Indexed: 11/24/2022] Open
Abstract
The study of student behavior analysis in class plays a key role in teaching and educational reforms that can help the university to find an effective way to improve students' learning efficiency and innovation ability. It is also one of the effective ways to cultivate innovative talents. The traditional behavior recognition methods have many disadvantages, such as poor robustness and low efficiency. From a heterogeneous view perception point of view, it introduces the students' behavior recognition. Therefore, we propose a 3-D multiscale residual dense network from heterogeneous view perception for analysis of student behavior recognition in class. First, the proposed method adopts 3-D multiscale residual dense blocks as the basic module of the network, and the module extracts the hierarchical features of students' behavior through the densely connected convolutional layer. Second, the local dense feature of student behavior is to learn adaptively. Third, the residual connection module is used to improve the training efficiency. Finally, experimental results show that the proposed algorithm has good robustness and transfer learning ability compared with the state-of-the-art behavior recognition algorithms, and it can effectively handle multiple video behavior recognition tasks. The design of an intelligent human behavior recognition algorithm has great practical significance to analyze the learning and teaching of students in the class.
Collapse
Affiliation(s)
- Hui Liu
- School of Software Technology, Kaifaqu Campus of Dalian University of Technology, Dalian, China.,Faculty of Business and Management, University Teknologi Majlis Amanah Rakyat (MARA) Sarawak Branch, Kota Samarahan, Malaysia
| | - Yang Liu
- International School of Shenyang Jianzhu University, Shenyang, China
| | - Ran Zhang
- School of Software Technology, Kaifaqu Campus of Dalian University of Technology, Dalian, China
| | - Xia Wu
- Huawei Nanjing Research and Development Center, Nanjing, China
| |
Collapse
|