1
|
Lian S, Li Z. An end-to-end multi-task motor imagery EEG classification neural network based on dynamic fusion of spectral-temporal features. Comput Biol Med 2024; 178:108727. [PMID: 38897146 DOI: 10.1016/j.compbiomed.2024.108727] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/25/2023] [Revised: 05/18/2024] [Accepted: 06/07/2024] [Indexed: 06/21/2024]
Abstract
Electroencephalograph (EEG) brain-computer interfaces (BCI) have potential to provide new paradigms for controlling computers and devices. The accuracy of brain pattern classification in EEG BCI is directly affected by the quality of features extracted from EEG signals. Currently, feature extraction heavily relies on prior knowledge to engineer features (for example from specific frequency bands); therefore, better extraction of EEG features is an important research direction. In this work, we propose an end-to-end deep neural network that automatically finds and combines features for motor imagery (MI) based EEG BCI with 4 or more imagery classes (multi-task). First, spectral domain features of EEG signals are learned by compact convolutional neural network (CCNN) layers. Then, gated recurrent unit (GRU) neural network layers automatically learn temporal patterns. Lastly, an attention mechanism dynamically combines (across EEG channels) the extracted spectral-temporal features, reducing redundancy. We test our method using BCI Competition IV-2a and a data set we collected. The average classification accuracy on 4-class BCI Competition IV-2a was 85.1 % ± 6.19 %, comparable to recent work in the field and showing low variability among participants; average classification accuracy on our 6-class data was 64.4 % ± 8.35 %. Our dynamic fusion of spectral-temporal features is end-to-end and has relatively few network parameters, and the experimental results show its effectiveness and potential.
Collapse
Affiliation(s)
- Shidong Lian
- School of Systems Science, Beijing Normal University, Beijing, China; International Academic Center of Complex Systems, Beijing Normal University, Zhuhai, China
| | - Zheng Li
- Center for Cognition and Neuroergonomics, State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Zhuhai, China; Department of Psychology, Faculty of Arts and Sciences, Beijing Normal University, Zhuhai, China.
| |
Collapse
|
2
|
Nie F, Zhang C, Wang Z, Wang R, Li X. Local Embedding Learning via Landmark-Based Dynamic Connections. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2023; 34:9481-9492. [PMID: 36107894 DOI: 10.1109/tnnls.2022.3203014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Linear discriminant analysis (LDA) is one of the most effective and popular methods to reduce the dimensionality of data with Gaussian assumption. However, LDA cannot handle non-Gaussian data because the center point is incompetent to represent the distribution of data. Some existing methods based on graph embedding focus on exploring local structures via pairwise relationships of data for addressing the non-Gaussian issue. Due to massive pairwise relationships, the computational complexity is high as well as the locally optimal solution is hard to find. To address these issues, we propose a novel and efficient local embedding learning via landmark-based dynamic connections (LDC) in which we leverage several landmarks to represent different subclusters in the same class and establish the connections between each point and landmark. Furthermore, in order to explore the relationship of landmarks pairwise more precisely, the relationship between each point and their corresponding neighbor landmarks are found in the optimal subspace, rather than the original space, which can avoid the negative influence of the noises. We also propose an efficient iterative algorithm to deal with the proposed ratio minimization problem. Extensive experiments conducted on several real-world datasets have demonstrated the advantages of the proposed method.
Collapse
|
3
|
Santamaría-Vázquez E, Martínez-Cagigal V, Marcos-Martínez D, Rodríguez-González V, Pérez-Velasco S, Moreno-Calderón S, Hornero R. MEDUSA©: A novel Python-based software ecosystem to accelerate brain-computer interface and cognitive neuroscience research. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 230:107357. [PMID: 36693292 DOI: 10.1016/j.cmpb.2023.107357] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/22/2022] [Revised: 12/14/2022] [Accepted: 01/15/2023] [Indexed: 06/17/2023]
Abstract
BACKGROUND AND OBJECTIVE Neurotechnologies have great potential to transform our society in ways that are yet to be uncovered. The rate of development in this field has increased significantly in recent years, but there are still barriers that need to be overcome before bringing neurotechnologies to the general public. One of these barriers is the difficulty of performing experiments that require complex software, such as brain-computer interfaces (BCI) or cognitive neuroscience experiments. Current platforms have limitations in terms of functionality and flexibility to meet the needs of researchers, who often need to implement new experimentation settings. This work was aimed to propose a novel software ecosystem, called MEDUSA©, to overcome these limitations. METHODS We followed strict development practices to optimize MEDUSA© for research in BCI and cognitive neuroscience, making special emphasis in the modularity, flexibility and scalability of our solution. Moreover, it was implemented in Python, an open-source programming language that reduces the development cost by taking advantage from its high-level syntax and large number of community packages. RESULTS MEDUSA© provides a complete suite of signal processing functions, including several deep learning architectures or connectivity analysis, and ready-to-use BCI and neuroscience experiments, making it one of the most complete solutions nowadays. We also put special effort in providing tools to facilitate the development of custom experiments, which can be easily shared with the community through an app market available in our website to promote reproducibility. CONCLUSIONS MEDUSA© is a novel software ecosystem for modern BCI and neurotechnology experimentation that provides state-of-the-art tools and encourages the participation of the community to make a difference for the progress of these fields. Visit the official website at https://www.medusabci.com/ to know more about this project.
Collapse
Affiliation(s)
- Eduardo Santamaría-Vázquez
- Biomedical Engineering Group (GIB), E.T.S Ingenieros de Telecomunicación, University of Valladolid, Paseo de Belén 15, Valladolid, 47011, Spain; Centro de Investigación Biomédica en Red en Bioingeniería, Biomateriales y Nanomedicina, (CIBER-BBN), Spain.
| | - Víctor Martínez-Cagigal
- Biomedical Engineering Group (GIB), E.T.S Ingenieros de Telecomunicación, University of Valladolid, Paseo de Belén 15, Valladolid, 47011, Spain; Centro de Investigación Biomédica en Red en Bioingeniería, Biomateriales y Nanomedicina, (CIBER-BBN), Spain.
| | - Diego Marcos-Martínez
- Biomedical Engineering Group (GIB), E.T.S Ingenieros de Telecomunicación, University of Valladolid, Paseo de Belén 15, Valladolid, 47011, Spain.
| | - Víctor Rodríguez-González
- Biomedical Engineering Group (GIB), E.T.S Ingenieros de Telecomunicación, University of Valladolid, Paseo de Belén 15, Valladolid, 47011, Spain; Centro de Investigación Biomédica en Red en Bioingeniería, Biomateriales y Nanomedicina, (CIBER-BBN), Spain.
| | - Sergio Pérez-Velasco
- Biomedical Engineering Group (GIB), E.T.S Ingenieros de Telecomunicación, University of Valladolid, Paseo de Belén 15, Valladolid, 47011, Spain.
| | - Selene Moreno-Calderón
- Biomedical Engineering Group (GIB), E.T.S Ingenieros de Telecomunicación, University of Valladolid, Paseo de Belén 15, Valladolid, 47011, Spain.
| | - Roberto Hornero
- Biomedical Engineering Group (GIB), E.T.S Ingenieros de Telecomunicación, University of Valladolid, Paseo de Belén 15, Valladolid, 47011, Spain; Centro de Investigación Biomédica en Red en Bioingeniería, Biomateriales y Nanomedicina, (CIBER-BBN), Spain.
| |
Collapse
|
4
|
Triana-Guzman N, Orjuela-Cañon AD, Jutinico AL, Mendoza-Montoya O, Antelis JM. Decoding EEG rhythms offline and online during motor imagery for standing and sitting based on a brain-computer interface. Front Neuroinform 2022; 16:961089. [PMID: 36120085 PMCID: PMC9481272 DOI: 10.3389/fninf.2022.961089] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2022] [Accepted: 08/03/2022] [Indexed: 12/02/2022] Open
Abstract
Motor imagery (MI)-based brain-computer interface (BCI) systems have shown promising advances for lower limb motor rehabilitation. The purpose of this study was to develop an MI-based BCI for the actions of standing and sitting. Thirty-two healthy subjects participated in the study using 17 active EEG electrodes. We used a combination of the filter bank common spatial pattern (FBCSP) method and the regularized linear discriminant analysis (RLDA) technique for decoding EEG rhythms offline and online during motor imagery for standing and sitting. The offline analysis indicated the classification of motor imagery and idle state provided a mean accuracy of 88.51 ± 1.43% and 85.29 ± 1.83% for the sit-to-stand and stand-to-sit transitions, respectively. The mean accuracies of the sit-to-stand and stand-to-sit online experiments were 94.69 ± 1.29% and 96.56 ± 0.83%, respectively. From these results, we believe that the MI-based BCI may be useful to future brain-controlled standing systems.
Collapse
Affiliation(s)
| | | | - Andres L. Jutinico
- Facultad de Ingeniería Mecánica, Electrónica y Biomédica, Universidad Antonio Nariño, Bogota, Colombia
| | - Omar Mendoza-Montoya
- Tecnologico de Monterrey, Escuela de Ingeniería y Ciencias, Monterrey, Mexico
- *Correspondence: Omar Mendoza-Montoya
| | - Javier M. Antelis
- Tecnologico de Monterrey, Escuela de Ingeniería y Ciencias, Monterrey, Mexico
| |
Collapse
|
5
|
Harishvijey A, Benadict Raja J. Automated technique for EEG signal processing to detect seizure with optimized Variable Gaussian Filter and Fuzzy RBFELM classifier. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103450] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
6
|
Golden subject is everyone: A subject transfer neural network for motor imagery-based brain computer interfaces. Neural Netw 2022; 151:111-120. [DOI: 10.1016/j.neunet.2022.03.025] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2021] [Revised: 02/09/2022] [Accepted: 03/22/2022] [Indexed: 12/24/2022]
|
7
|
Improvement decoding performance based on GQDA during a high engagement demanding paradigm. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103205] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
8
|
Real-Time Process Monitoring Based on Multivariate Control Chart for Anomalies Driven by Frequency Signal via Sound and Electrocardiography Cases. Processes (Basel) 2021. [DOI: 10.3390/pr9091510] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Recent developments in network technologies have led to the application of cloud computing and big data analysis to industrial automation. However, the automation of process monitoring still has numerous issues that need to be addressed. Traditionally, offline statistical processes are generally used for process monitoring; thus, problems are often detected too late. This study focused on the construction of an automated process monitoring system based on sound and vibration frequency signals. First, empirical mode decomposition was combined with intrinsic mode functions to construct different sound frequency combinations and differentiate sound frequencies according to anomalies. Then, linear discriminant analysis (LDA) was adopted to classify abnormal and normal sound frequency signals, and a control line was constructed to monitor the sound frequency. In a case study, the proposed method was applied to detect abnormal sounds at high and low frequencies, and a detection accuracy of over 90% was realized. In another case study, the proposed method was applied to analyze electrocardiography signals and was similarly able to identify abnormal situations. Thus, the proposed method can be applied to real-time process monitoring and the detection of abnormalities with high accuracy in various situations.
Collapse
|
9
|
Abstract
Cyber-Physical System (CPS) applications including human-robot interaction call for automated reasoning for rational decision-making. In the latter context, typically, audio-visual signals are employed. Τhis work considers brain signals for emotion recognition towards an effective human-robot interaction. An ElectroEncephaloGraphy (EEG) signal here is represented by an Intervals’ Number (IN). An IN-based, optimizable parametric k Nearest Neighbor (kNN) classifier scheme for decision-making by fuzzy lattice reasoning (FLR) is proposed, where the conventional distance between two points is replaced by a fuzzy order function (σ) for reasoning-by-analogy. A main advantage of the employment of INs is that no ad hoc feature extraction is required since an IN may represent all-order data statistics, the latter are the features considered implicitly. Four different fuzzy order functions are employed in this work. Experimental results demonstrate comparably the good performance of the proposed techniques.
Collapse
|
10
|
A Fuzzy Shell for Developing an Interpretable BCI Based on the Spatiotemporal Dynamics of the Evoked Oscillations. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2021; 2021:6685672. [PMID: 33936191 PMCID: PMC8055434 DOI: 10.1155/2021/6685672] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/17/2020] [Revised: 03/05/2021] [Accepted: 03/17/2021] [Indexed: 12/26/2022]
Abstract
Researchers in neuroscience computing experience difficulties when they try to carry out neuroanalysis in practice or when they need to design an explainable brain-computer interface (BCI) with quick setup and minimal training phase. There is a need of interpretable computational intelligence techniques and new brain states decoding for more understandable interpretation of the sensory, cognitive, and motor brain processing. We propose a general-purpose fuzzy software system shell for developing a custom EEG BCI system. It relies on the bursts of the ongoing EEG frequency power synchronization/desynchronization at scalp level and supports quick BCI setup by linguistic features, ad hoc fuzzy membership construction, explainable IF-THEN rules, and the concept of the Internet of Things (IoT), which makes the BCI system device and service independent. It has a potential for designing both passive and event-related BCIs with options for visual representation at scalp-source level in response to time. The feasibility of the proposed system has been proven by real experiments and bursts for β and γ frequency power have been detected in real time in response to evoked visuospatial selective attention. The presence of the proposed new brain state decoding can be used as a feasible metric for interpretation of the spatiotemporal dynamics of the passive or evoked neural oscillations.
Collapse
|
11
|
Motor Imagery Classification Based on a Recurrent-Convolutional Architecture to Control a Hexapod Robot. MATHEMATICS 2021. [DOI: 10.3390/math9060606] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/07/2023]
Abstract
Advances in the field of Brain-Computer Interfaces (BCIs) aim, among other applications, to improve the movement capacities of people suffering from the loss of motor skills. The main challenge in this area is to achieve real-time and accurate bio-signal processing for pattern recognition, especially in Motor Imagery (MI). The significant interaction between brain signals and controllable machines requires instantaneous brain data decoding. In this study, an embedded BCI system based on fist MI signals is developed. It uses an Emotiv EPOC+ Brainwear®, an Altera SoCKit® development board, and a hexapod robot for testing locomotion imagery commands. The system is tested to detect the imagined movements of closing and opening the left and right hand to control the robot locomotion. Electroencephalogram (EEG) signals associated with the motion tasks are sensed on the human sensorimotor cortex. Next, the SoCKit processes the data to identify the commands allowing the controlled robot locomotion. The classification of MI-EEG signals from the F3, F4, FC5, and FC6 sensors is performed using a hybrid architecture of Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks. This method takes advantage of the deep learning recognition model to develop a real-time embedded BCI system, where signal processing must be seamless and precise. The proposed method is evaluated using k-fold cross-validation on both created and public Scientific-Data datasets. Our dataset is comprised of 2400 trials obtained from four test subjects, lasting three seconds of closing and opening fist movement imagination. The recognition tasks reach 84.69% and 79.2% accuracy using our data and a state-of-the-art dataset, respectively. Numerical results support that the motor imagery EEG signals can be successfully applied in BCI systems to control mobile robots and related applications such as intelligent vehicles.
Collapse
|
12
|
Alchalabi B, Faubert J, Labbé D. A multi-modal modified feedback self-paced BCI to control the gait of an avatar. J Neural Eng 2021; 18. [PMID: 33711832 DOI: 10.1088/1741-2552/abee51] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2020] [Accepted: 03/12/2021] [Indexed: 11/12/2022]
Abstract
Brain-computer interfaces (BCI) have been used to control the gait of a virtual self-avatar with a proposed application in the field of gait rehabilitation. OBJECTIVE to develop a high performance multi-modal BCI to control single steps and forward walking of an immersive virtual reality avatar. This system will overcome the limitation of existing systems. APPROACH This system used MI of these actions, in cue-paced and self-paced modes. Twenty healthy participants participated in this 4 sessions study across 4 different days. They were cued to imagine a single step forward with their right or left foot, or to imagine walking forward. They were instructed to reach a target by using the MI of multiple steps (self-paced switch-control mode) or by maintaining MI of forward walking (continuous-control mode). The movement of the avatar was controlled by two calibrated RLDA classifiers that used the µ power spectral density (PSD) over the foot area of the motor cortex as a feature. The classifiers were retrained after every session. For a subset of the trials, positive modified feedback was presented to half of the participants. MAIN RESULTS All participants were able to operate the BCI. Their average offline performance, after retraining the classifiers was 86.0 ± 6.1%, showing that the recalibration of the classifiers enhanced the offline performance of the BCI (p < 0.01). The average online performance was 85.9 ± 8.4% showing that modified feedback enhanced BCI performance (p =0.001). The average performance was 83% at self-paced switch control and 92% at continuous control mode. SIGNIFICANCE This study reports on the first novel integration of different design approaches, different control modes and different performance enhancement techniques, all in parallel in one single high performance and multi-modal BCI system, to control single steps and forward walking of an immersive virtual reality avatar.
Collapse
Affiliation(s)
- Bilal Alchalabi
- biomedical engineering, University of Montreal, 2900 Boulevard Edouard mon Petit, Montreal, Quebec, H3C 3J7, CANADA
| | - Jocelyn Faubert
- Université de Montréal, 3744 Rue Jean Brillant, Montreal, Quebec, H3T 1P1, CANADA
| | - David Labbé
- École de technologie supérieure, 1100 Rue Notre-Dame ouest, Montreal, Quebec, H3C 1K3, CANADA
| |
Collapse
|
13
|
Yang L, Song Y, Ma K, Xie L. Motor Imagery EEG Decoding Method Based on a Discriminative Feature Learning Strategy. IEEE Trans Neural Syst Rehabil Eng 2021; 29:368-379. [PMID: 33460382 DOI: 10.1109/tnsre.2021.3051958] [Citation(s) in RCA: 28] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
With the rapid development of deep learning, more and more deep learning-based motor imagery electroencephalograph (EEG) decoding methods have emerged in recent years. However, the existing deep learning-based methods usually only adopt the constraint of classification loss, which hardly obtains the features with high discrimination and limits the improvement of EEG decoding accuracy. In this paper, a discriminative feature learning strategy is proposed to improve the discrimination of features, which includes the central distance loss (CD-loss), the central vector shift strategy, and the central vector update process. First, the CD-loss is proposed to make the same class of samples converge to the corresponding central vector. Then, the central vector shift strategy extends the distance between different classes of samples in the feature space. Finally, the central vector update process is adopted to avoid the non-convergence of CD-loss and weaken the influence of the initial value of central vectors on the final results. In addition, overfitting is another severe challenge for deep learning-based EEG decoding methods. To deal with this problem, a data augmentation method based on circular translation strategy is proposed to expand the experimental datasets without introducing any extra noise or losing any information of the original data. To validate the effectiveness of the proposed method, we conduct some experiments on two public motor imagery EEG datasets (BCI competition IV 2a and 2b dataset), respectively. The comparison with current state-of-the-art methods indicates that our method achieves the highest average accuracy and good stability on the two experimental datasets.
Collapse
|
14
|
Fu R, Han M, Bao T, Wang F, Shi P. Discrimination Improvement Through Undesirable Feedback in Coupling Object Manipulation Tasks. Int J Neural Syst 2021; 31:2150012. [PMID: 33573533 DOI: 10.1142/s012906572150012x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Subjective effort can significantly affect the ability of humans to act optimally in dynamic manipulation tasks. In a previous study, we designed a complex object coupling manipulation task that required tight performance and induced high cognitive workload. We hypothesize that strong-effort-related physiological reactivity during the dynamic manipulation task improves the user performance in an undesired task feedback situation. To test this hypothesis, using the motor intentions' discrimination from electroencephalogram (EEG) measurements, we evaluate the effort expended by 20 participants in a controlling task with constraints involving complex coupling objects. Specifically, the finer motor decisions are obtained from the controlling information in EEG by using two fingers from the same hand rather than two hands. The motor intention is decoded from a task-dependent EEG through a regularized discriminant analysis, and the area under the curve is [Formula: see text]. Furthermore, we compare the undesired and desired task feedback conditions along with the individual's effort dynamic adjustment, and investigate whether the undesired task feedback improved the discrimination of the motor activities. A stronger effort to attain the desired feedback state corresponds to improved motor activity discrimination from the EEG in the undesired task feedback scenario. The differences in the brain activities under the undesired and desired task feedback conditions are analyzed using brain-network-based topographical scalp maps. Our experiment provides preliminary evidence that inducing strong effort can improve discrimination performance during highly demanding tasks. This finding can advance our understanding of human attention, potentially improve the accuracy of intention recognition, and may inspire better EEG acquisition contexts.
Collapse
Affiliation(s)
- Rongrong Fu
- Measurement Technology and Instrumentation Key Lab of Hebei Province, Department of Electrical Engineering, Yanshan University, Qinhuangdao, P. R. China
| | - Mengmeng Han
- Measurement Technology and Instrumentation Key Lab of Hebei Province, Department of Electrical Engineering, Yanshan University, Qinhuangdao, P. R. China
| | - Tiantian Bao
- Measurement Technology and Instrumentation Key Lab of Hebei Province, Department of Electrical Engineering, Yanshan University, Qinhuangdao, P. R. China
| | - Fuwang Wang
- School of Mechanical Engineering, Northeastern Electric Power University, P. R. China
| | - Peiming Shi
- Measurement Technology and Instrumentation Key Lab of Hebei Province, Department of Electrical Engineering, Yanshan University, Qinhuangdao, P. R. China
| |
Collapse
|
15
|
Zuo C, Jin J, Xu R, Wu L, Liu C, Miao Y, Wang X. Cluster decomposing and multi-objective optimization based-ensemble learning framework for motor imagery-based brain-computer interfaces. J Neural Eng 2021; 18. [PMID: 33524961 DOI: 10.1088/1741-2552/abe20f] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2020] [Accepted: 02/01/2021] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Motor imagery (MI) is a mental representation of motor behavior and a widely used pattern in electroencephalogram (EEG) based brain-computer interface (BCI) systems. EEG is known for its non-stationary, non-linear features and sensitivity to artifacts from various sources. This study aimed to design a powerful classifier with a strong generalization capability for MI based BCIs. APPROACH In this study, we proposed a cluster decomposing based ensemble learning framework (CDECL) for EEG classification of MI based BCIs. The EEG data was decomposed into sub-data sets with different distributions by clustering decomposition. Then a set of heterogeneous classifiers was trained on each sub-data set for generating a diversified classifier search space. To obtain the optimal classifier combination, the ensemble learning was formulated as a multi-objective optimization problem and a stochastic fractal based binary multi-objective fruit fly optimization algorithm was proposed for solving the ensemble learning problem. MAIN RESULTS The proposed method was validated on two public EEG datasets (BCI Competition IV datasets IIb and BCI Competition IV dataset IIa) and compared with several other competing classification methods. Experimental results showed that the proposed CDECL based methods can effectively construct a diversity ensemble classifier and exhibits superior classification performance in comparison with several competing methods. SIGNIFICANCE The proposed method is promising for improving the performance of MI-based BCIs.
Collapse
Affiliation(s)
- Cili Zuo
- East China University of Science and Technology, 130 Meilong road, Shanghai, Shanghai, 200237, CHINA
| | - Jing Jin
- Key Laboratory of Advanced Control and Optimization for Chemical Processes Ministry of Education, East China University of Science and Technology, 130 Meilong Road, Xuhui District, Shanghai 200237, SHANGHAI, 200237, CHINA
| | - Ren Xu
- Guger Technologies OG, Sierningstrasse 14, Graz, 8020, AUSTRIA
| | - Lianghong Wu
- Hunan University of Science and Technology, Tiaoyuan Road, Xiangtan, 411201, CHINA
| | - Chang Liu
- East China University of Science and Technology, 130 Meilong Road, Shanghai, Shanghai, 200237, CHINA
| | - Yangyang Miao
- East China University of Science and Technology, 130 Meilong raod, Shanghai, Shanghai, 200237, CHINA
| | - Xingyu Wang
- East China University of Science and Technology, 130 Meilong Road, Shanghai, Shanghai, 200237, CHINA
| |
Collapse
|
16
|
Fu R, Li W, Chen J, Han M. Recognizing single-trial motor imagery EEG based on interpretable clustering method. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2020.102171] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
17
|
Shamsi F, Haddad A, Najafizadeh L. Early classification of motor tasks using dynamic functional connectivity graphs from EEG. J Neural Eng 2020; 18. [PMID: 33246319 DOI: 10.1088/1741-2552/abce70] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2020] [Accepted: 11/27/2020] [Indexed: 12/13/2022]
Abstract
OBJECTIVE Classification of electroencephalography (EEG) signals with high accuracy using short recording intervals has been a challenging problem in developing brain computer interfaces (BCIs). This paper presents a novel feature extraction method for EEG recordings to tackle this problem. APPROACH The proposed approach is based on the concept that the brain functions in a dynamic manner, and utilizes dynamic functional connectivity graphs. The EEG data is first segmented into intervals during which functional networks sustain their connectivity. Functional connectivity networks for each identified segment are then localized, and graphs are constructed, which will be used as features. To take advantage of the dynamic nature of the generated graphs, a Long Short Term Memory (LSTM) classifier is employed for classification. MAIN RESULTS Features extracted from various durations of post-stimulus EEG data associated with motor execution and imagery tasks are used to test the performance of the classifier. Results show an average accuracy of 85.32% using features extracted from only 500 ms of the post-stimulus data. SIGNIFICANCE Our results demonstrate, for the first time, that using the proposed feature extraction method, it is possible to classify motor tasks from EEG recordings using a short interval of the data in the order of hundreds of milliseconds (e.g. 500 ms). This duration is considerably shorter than what has been reported before. These results will have significant implications for improving the effectiveness and the speed of BCIs, particularly for those used in assistive technologies.
Collapse
Affiliation(s)
- Foroogh Shamsi
- Electrical and Computer Engineering, Rutgers University, 94 Brett Rd, New Brunswick, New Jersey, NJ 08854, UNITED STATES
| | - Ali Haddad
- Electrical and Computer Engineering, Rutgers University, 94 Brett Rd, New Brunswick, New Jersey, NJ 08854, UNITED STATES
| | - Laleh Najafizadeh
- Electrical and Computer Engineering, Rutgers University, 94 Brett Rd, New Brunswick, New Jersey, 08901-8554, UNITED STATES
| |
Collapse
|
18
|
Motor-Imagery Classification Using Riemannian Geometry with Median Absolute Deviation. ELECTRONICS 2020. [DOI: 10.3390/electronics9101584] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Motor imagery (MI) from human brain signals can diagnose or aid specific physical activities for rehabilitation, recreation, device control, and technology assistance. It is a dynamic state in learning and practicing movement tracking when a person mentally imitates physical activity. Recently, it has been determined that a brain–computer interface (BCI) can support this kind of neurological rehabilitation or mental practice of action. In this context, MI data have been captured via non-invasive electroencephalogram (EEGs), and EEG-based BCIs are expected to become clinically and recreationally ground-breaking technology. However, determining a set of efficient and relevant features for the classification step was a challenge. In this paper, we specifically focus on feature extraction, feature selection, and classification strategies based on MI-EEG data. In an MI-based BCI domain, covariance metrics can play important roles in extracting discriminatory features from EEG datasets. To explore efficient and discriminatory features for the enhancement of MI classification, we introduced a median absolute deviation (MAD) strategy that calculates the average sample covariance matrices (SCMs) to select optimal accurate reference metrics in a tangent space mapping (TSM)-based MI-EEG. Furthermore, all data from SCM were projected using TSM according to the reference matrix that represents the featured vector. To increase performance, we reduced the dimensions and selected an optimum number of features using principal component analysis (PCA) along with an analysis of variance (ANOVA) that could classify MI tasks. Then, the selected features were used to develop linear discriminant analysis (LDA) training for classification. The benchmark datasets were considered for the evaluation and the results show that it provides better accuracy than more sophisticated methods.
Collapse
|
19
|
Intentions Recognition of EEG Signals with High Arousal Degree for Complex Task. J Med Syst 2020; 44:110. [DOI: 10.1007/s10916-020-01571-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2019] [Accepted: 03/31/2020] [Indexed: 10/24/2022]
|
20
|
Fu R, Wang H, Han M, Han D, Sun J. Scaling Analysis of Phase Fluctuations of Brain Networks in Dynamic Constrained Object Manipulation. Int J Neural Syst 2020; 30:2050002. [DOI: 10.1142/s0129065720500021] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
In this study, we investigated the dynamic properties of oscillatory activities in the scalp electro-encephalographs (EEGs) of 20 participants involved in a novel dynamic manipulating task using a physical interface and a virtual feedback. The complexity of such a task a rises from the unexpected relationship between the magnitude of the motion and the feedback. The characterization of complex patterns arising from EEG is an important problem in identifying different mental intentions. We proposed a scaling analysis of phase fluctuation in the scalp EEG to discriminate the network states related to different EEG patterns, which correspond to manipulating the task with right or left movement intention. These intentions are generated while the participant is engaged in such a complex task. The phase characterization method was used to calculate the instantaneous phase from the operational EEG. Then, functional brain networks (FBNs) of 20 subjects based on the task-related EEG were constructed by phase synchronization. The degree features representing the structures and scaling components of brain networks are sensitive to the EEG patterns with left or right motor intention. The correlation between features and mental intentions was investigated by discriminant analysis. For 20 subjects, the average accuracy of state detection is [Formula: see text], and the average mean-squared error (MSE) is [Formula: see text]. The brain state depicted by the results is related to high awareness, the phase characterization is of the effectiveness in EEG processing and FBN construction and the difference of control intentions can be explored by the phase characterization method. This finding may be relevant to understanding some neuronal mechanisms underlying the attention and some applications of closed-loop control for the safety operation of tools.
Collapse
Affiliation(s)
- Rongrong Fu
- Measurement Technology and Instrumentation Key Lab of Hebei Province, Department of Electrical Engineering, Yanshan University, Qinhuangdao, Hebei, P. R. China
| | - Han Wang
- Measurement Technology and Instrumentation Key Lab of Hebei Province, Department of Electrical Engineering, Yanshan University, Qinhuangdao, Hebei, P. R. China
| | - Mengmeng Han
- Measurement Technology and Instrumentation Key Lab of Hebei Province, Department of Electrical Engineering, Yanshan University, Qinhuangdao, Hebei, P. R. China
| | - Dongying Han
- School of Vehicles and Energy, Yanshan University, Qinhuangdao, Hebei, P. R. China
| | - Jiedi Sun
- School of Information Science and Engineering, Yanshan University, Qinhuangdao, Hebei, P. R. China
| |
Collapse
|
21
|
Towards Improved Detection of Cognitive Performance Using Bidirectional Multilayer Long-Short Term Memory Neural Network. Brain Inform 2020. [DOI: 10.1007/978-3-030-59277-6_27] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022] Open
|
22
|
Joadder MAM, Myszewski JJ, Rahman MH, Wang I. A performance based feature selection technique for subject independent MI based BCI. Health Inf Sci Syst 2019; 7:15. [PMID: 31428313 DOI: 10.1007/s13755-019-0076-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2019] [Accepted: 07/17/2019] [Indexed: 10/26/2022] Open
Abstract
Purpose Significant research has been conducted in the field of brain computer interface (BCI) algorithm development, however, many of the resulting algorithms are both complex, and specific to a particular user as the most successful methodology can vary between individuals and sessions. The objective of this study was to develop a simple yet effective method of feature selection to improve the accuracy of a subject independent BCI algorithm and streamline the process of BCI algorithm development. Over the past several years, several high precision features have been suggested by researchers to classify different motor imagery tasks. This research applies fourteen of these features as a feature pool that can be used as a reference for future researchers. Additionally, we look for the most efficient feature or feature set with four different classifiers that best differentiates several motor imagery tasks. In this work we have successfully employed a feature fusion method to obtain the best sub-set of features. We have proposed a novel computer aided feature selection method to determine the best set of features for distinguishing between motor imagery tasks in lieu of the manual feature selection that has been performed in past studies. The features selected by this method were then fed into a Linear Discriminant Analysis, K-nearest neighbor, decision tree, or support vector machine classifier for classification to determine the overall performance. Methods The methods used were a novel performance based additive feature fusion algorithm working in conjunction with machine learning in order to classify the motor imagery signals into particular states. The data used for this study was collected from BCI competition III dataset IVa. Result The result of this algorithm was a classification accuracy of 99% for a subject independent algorithm with less computation cost compared to traditional methods, in addition to multiple feature/classifier combinations that outperform current subject independent methods. Conclusion The conclusion of this study and its significance is that it developed a viable methodology for simple, efficient feature selection and BCI algorithm development, which leads to an overall increase in algorithm classification accuracy.
Collapse
Affiliation(s)
- Md A Mannan Joadder
- 1Department of Electrical, & Electronic Engineering, United International University, Dhaka, Bangladesh
| | - Joshua J Myszewski
- 2Department of Biomedical Engineering, University of Wisconsin-Milwaukee, Milwaukee, WI 53211 USA
| | - Mohammad H Rahman
- 2Department of Biomedical Engineering, University of Wisconsin-Milwaukee, Milwaukee, WI 53211 USA
| | - Inga Wang
- 3Department of Occupational Science & Technology, University of Wisconsin-Milwaukee, Milwaukee, WI 53211 USA
| |
Collapse
|