Motor Imagery Classification via Temporal Attention Cues of Graph Embedded EEG Signals

EEG-VTTCNet: A loss joint training model based on the vision transformer and the temporal convolution network for EEG-based motor imagery classification

Brain-computer interface (BCI) is a technology that directly connects signals between the human brain and a computer or other external device. Motor imagery electroencephalographic (MI-EEG) signals are considered a promising paradigm for BCI systems, with a wide range of potential applications in medical rehabilitation, human-computer interaction, and virtual reality. Accurate decoding of MI-EEG signals poses a significant challenge due to issues related to the quality of the collected EEG data and subject variability. Therefore, developing an efficient MI-EEG decoding network is crucial and warrants research. This paper proposes a loss joint training model based on the vision transformer (VIT) and the temporal convolutional network (EEG-VTTCNet) to classify MI-EEG signals. To take advantage of multiple modules together, the EEG-VTTCNet adopts a shared convolution strategy and a dual-branching strategy. The dual-branching modules perform complementary learning and jointly train shared convolutional modules with better performance. We conducted experiments on the BCI Competition IV-2a and IV-2b datasets, and the proposed network outperformed the current state-of-the-art techniques with an accuracy of 84.58% and 90.94%, respectively, for the subject-dependent mode. In addition, we used t-SNE to visualize the features extracted by the proposed network, further demonstrating the effectiveness of the feature extraction framework. We also conducted extensive ablation and hyperparameter tuning experiments to construct a robust network architecture that can be well generalized.

A two-stage transformer based network for motor imagery classification

Brain-computer interfaces (BCIs) are used to understand brain functioning and develop therapies for neurological and neurodegenerative disorders. Therefore, BCIs are crucial in rehabilitating motor dysfunction and advancing motor imagery applications. For motor imagery, electroencephalogram (EEG) signals are used to classify the subject's intention of moving a body part without actually moving it. This paper presents a two-stage transformer-based architecture that employs handcrafted features and deep learning techniques to enhance the classification performance on benchmarked EEG signals. Stage-1 is built on parallel convolution based EEGNet, multi-head attention, and separable temporal convolution networks for spatiotemporal feature extraction. Further, for enhanced classification, in stage-2, additional features and embeddings extracted from stage-1 are used to train TabNet. In addition, a novel channel cluster swapping data augmentation technique is also developed to handle the issue of limited samples for training deep learning architectures. The developed two-stage architecture offered an average classification accuracy of 88.5 % and 88.3 % on the BCI Competition IV-2a and IV-2b datasets, respectively, which is approximately 3.0 % superior over similar recent reported works.

Attention-guided graph structure learning network for EEG-enabled auditory attention detection

Objective: Decoding auditory attention from brain signals is essential for the development of neuro-steered hearing aids. This study aims to overcome the challenges of extracting discriminative feature representations from electroencephalography (EEG) signals for auditory attention detection (AAD) tasks, particularly focusing on the intrinsic relationships between different EEG channels.Approach: We propose a novel attention-guided graph structure learning network, AGSLnet, which leverages potential relationships between EEG channels to improve AAD performance. Specifically, AGSLnet is designed to dynamically capture latent relationships between channels and construct a graph structure of EEG signals.Main result: We evaluated AGSLnet on two publicly available AAD datasets and demonstrated its superiority and robustness over state-of-the-art models. Visualization of the graph structure trained by AGSLnet supports previous neuroscience findings, enhancing our understanding of the underlying neural mechanisms.Significance: This study presents a novel approach for examining brain functional connections, improving AAD performance in low-latency settings, and supporting the development of neuro-steered hearing aids.

EEG motor imagery decoding: a framework for comparative analysis with channel attention mechanisms

Objective.The objective of this study is to investigate the application of various channel attention mechanisms within the domain of brain-computer interface (BCI) for motor imagery decoding. Channel attention mechanisms can be seen as a powerful evolution of spatial filters traditionally used for motor imagery decoding. This study systematically compares such mechanisms by integrating them into a lightweight architecture framework to evaluate their impact.Approach.We carefully construct a straightforward and lightweight baseline architecture designed to seamlessly integrate different channel attention mechanisms. This approach is contrary to previous works which only investigate one attention mechanism and usually build a very complex, sometimes nested architecture. Our framework allows us to evaluate and compare the impact of different attention mechanisms under the same circumstances. The easy integration of different channel attention mechanisms as well as the low computational complexity enables us to conduct a wide range of experiments on four datasets to thoroughly assess the effectiveness of the baseline model and the attention mechanisms.Results.Our experiments demonstrate the strength and generalizability of our architecture framework as well as how channel attention mechanisms can improve the performance while maintaining the small memory footprint and low computational complexity of our baseline architecture.Significance.Our architecture emphasizes simplicity, offering easy integration of channel attention mechanisms, while maintaining a high degree of generalizability across datasets, making it a versatile and efficient solution for electroencephalogram motor imagery decoding within BCIs.

Manifold Learning-Based Common Spatial Pattern for EEG Signal Classification

EEG signal classification using Riemannian manifolds has shown great potential. However, the huge computational cost associated with Riemannian metrics poses challenges for applying Riemannian methods, particularly in high-dimensional feature data. To address these, we propose an efficient ensemble method called MLCSP-TSE-MLP, which aims to reduce the computational cost while achieving superior performance. MLCSP of the ensemble utilizes a Riemannian graph embedding strategy to learn intrinsic low-dimensional sub-manifolds, enhancing discrimination. TSE uses the Euclidean mean as the reference point for tangent space mapping and reducing computational cost. Finally, the ensemble incorporates the MLP classifier to offer improved classification performance. Classification results conducted on three datasets demonstrate that MLCSP-TSE-MLP achieves significant superior performance compared to various competing methods. Notably, the MLCSP-TSE module achieves a remarkable increase in training speed and exhibits much lower test time compared to traditional Riemannian methods. Based on these results, we believe that the proposed MLCSP-TSE-MLP is a powerful tool for handling high-dimensional data and holds great potential for practical applications.

Individual-finger motor imagery classification: a data-driven approach with Shapley-informed augmentation

Objective. Classifying motor imagery (MI) tasks that involve fine motor control of the individual five fingers presents unique challenges when utilizing electroencephalography (EEG) data. In this paper, we systematically assess the classification of MI functions for the individual five fingers using single-trial time-domain EEG signals. This assessment encompasses both within-subject and cross-subject scenarios, supported by data-driven analysis that provides statistical validation of the neural correlate that could potentially discriminate between the five fingers.Approach. We present Shapley-informed augmentation, an informed approach to enhance within-subject classification accuracy. This method is rooted in insights gained from our data-driven analysis, which revealed inconsistent temporal features encoding the five fingers MI across sessions of the same subject. To evaluate its impact, we compare within-subject classification performance both before and after implementing this augmentation technique.Main results. Both the data-driven approach and the model explainability analysis revealed that the parietal cortex contains neural information that helps discriminate the individual five fingers' MI apart. Shapley-informed augmentation successfully improved classification accuracy in sessions severely affected by inconsistent temporal features. The accuracy for sessions impacted by inconsistency in their temporal features increased by an average of26.3%±6.70, thereby enabling a broader range of subjects to benefit from brain-computer interaction (BCI) applications involving five-fingers MI classification. Conversely, non-impacted sessions experienced only a negligible average accuracy decrease of2.01±5.44%. The average classification accuracy achieved is around 60.0% (within-session), 50.0% (within-subject) and 40.0% (leave-one-subject-out).Significance. This research offers data-driven evidence of neural correlates that could discriminate between the individual five fingers MI and introduces a novel Shapley-informed augmentation method to address temporal variability of features, ultimately contributing to the development of personalized systems.

Noise-Factorized Disentangled Representation Learning for Generalizable Motor Imagery EEG Classification

Motor Imagery (MI) Electroencephalography (EEG) is one of the most common Brain-Computer Interface (BCI) paradigms that has been widely used in neural rehabilitation and gaming. Although considerable research efforts have been dedicated to developing MI EEG classification algorithms, they are mostly limited in handling scenarios where the training and testing data are not from the same subject or session. Such poor generalization capability significantly limits the realization of BCI in real-world applications. In this paper, we proposed a novel framework to disentangle the representation of raw EEG data into three components, subject/session-specific, MI-task-specific, and random noises, so that the subject/session-specific feature extends the generalization capability of the system. This is realized by a joint discriminative and generative framework, supported by a series of fundamental training losses and training strategies. We evaluated our framework on three public MI EEG datasets, and detailed experimental results show that our method can achieve superior performance by a large margin compared to current state-of-the-art benchmark algorithms.

Bidirectional feature pyramid attention-based temporal convolutional network model for motor imagery electroencephalogram classification

Introduction

As an interactive method gaining popularity, brain-computer interfaces (BCIs) aim to facilitate communication between the brain and external devices. Among the various research topics in BCIs, the classification of motor imagery using electroencephalography (EEG) signals has the potential to greatly improve the quality of life for people with disabilities.

Methods

This technology assists them in controlling computers or other devices like prosthetic limbs, wheelchairs, and drones. However, the current performance of EEG signal decoding is not sufficient for real-world applications based on Motor Imagery EEG (MI-EEG). To address this issue, this study proposes an attention-based bidirectional feature pyramid temporal convolutional network model for the classification task of MI-EEG. The model incorporates a multi-head self-attention mechanism to weigh significant features in the MI-EEG signals. It also utilizes a temporal convolution network (TCN) to separate high-level temporal features. The signals are enhanced using the sliding-window technique, and channel and time-domain information of the MI-EEG signals is extracted through convolution.

Results

Additionally, a bidirectional feature pyramid structure is employed to implement attention mechanisms across different scales and multiple frequency bands of the MI-EEG signals. The performance of our model is evaluated on the BCI Competition IV-2a dataset and the BCI Competition IV-2b dataset, and the results showed that our model outperformed the state-of-the-art baseline model, with an accuracy of 87.5 and 86.3% for the subject-dependent, respectively.

Discussion

In conclusion, the BFATCNet model offers a novel approach for EEG-based motor imagery classification in BCIs, effectively capturing relevant features through attention mechanisms and temporal convolutional networks. Its superior performance on the BCI Competition IV-2a and IV-2b datasets highlights its potential for real-world applications. However, its performance on other datasets may vary, necessitating further research on data augmentation techniques and integration with multiple modalities to enhance interpretability and generalization. Additionally, reducing computational complexity for real-time applications is an important area for future work.

ETCNet: An EEG-based motor imagery classification model combining efficient channel attention and temporal convolutional network

Brain-computer interface (BCI) enables the control of external devices using signals from the brain, offering immense potential in assisting individuals with neuromuscular disabilities. Among the different paradigms of BCI systems, the motor imagery (MI) based electroencephalogram (EEG) signal is widely recognized as exceptionally promising. Deep learning (DL) has found extensive applications in the processing of MI signals, wherein convolutional neural networks (CNN) have demonstrated superior performance compared to conventional machine learning (ML) approaches. Nevertheless, challenges related to subject independence and subject dependence persist, while the inherent low signal-to-noise ratio of EEG signals remains a critical aspect that demands attention. Accurately deciphering intentions from EEG signals continues to present a formidable challenge. This paper introduces an advanced end-to-end network that effectively combines the efficient channel attention (ECA) and temporal convolutional network (TCN) components for the classification of motor imagination signals. We incorporated an ECA module prior to feature extraction in order to enhance the extraction of channel-specific features. A compact convolutional network model uses for feature extraction in the middle part. Finally, the time characteristic information is obtained by using TCN. The results show that our network is a lightweight network that is characterized by few parameters and fast speed. Our network achieves an average accuracy of 80.71% on the BCI Competition IV-2a dataset.

Revealing brain connectivity: graph embeddings for EEG representation learning and comparative analysis of structural and functional connectivity

This study employs deep learning techniques to present a compelling approach for modeling brain connectivity in EEG motor imagery classification through graph embedding. The compelling aspect of this study lies in its combination of graph embedding, deep learning, and different brain connectivity types, which not only enhances classification accuracy but also enriches the understanding of brain function. The approach yields high accuracy, providing valuable insights into brain connections and has potential applications in understanding neurological conditions. The proposed models consist of two distinct graph-based convolutional neural networks, each leveraging different types of brain connectivities to enhance classification performance and gain a deeper understanding of brain connections. The first model, Adjacency-based Convolutional Neural Network Model (Adj-CNNM), utilizes a graph representation based on structural brain connectivity to embed spatial information, distinguishing it from prior spatial filtering approaches dependent on subjects and tasks. Extensive tests on a benchmark dataset-IV-2a demonstrate that an accuracy of 72.77% is achieved by the Adj-CNNM, surpassing baseline and state-of-the-art methods. The second model, Phase Locking Value Convolutional Neural Network Model (PLV-CNNM), incorporates functional connectivity to overcome structural connectivity limitations and identifies connections between distinct brain regions. The PLV-CNNM achieves an overall accuracy of 75.10% across the 1-51 Hz frequency range. In the preferred 8-30 Hz frequency band, known for motor imagery data classification (including α, μ, and β waves), individual accuracies of 91.9%, 90.2%, and 85.8% are attained for α, μ, and β, respectively. Moreover, the model performs admirably with 84.3% accuracy when considering the entire 8-30 Hz band. Notably, the PLV-CNNM reveals robust connections between different brain regions during motor imagery tasks, including the frontal and central cortex and the central and parietal cortex. These findings provide valuable insights into brain connectivity patterns, enriching the comprehension of brain function. Additionally, the study offers a comprehensive comparative analysis of diverse brain connectivity modeling methods.

An in-depth survey on Deep Learning-based Motor Imagery Electroencephalogram (EEG) classification

Electroencephalogram (EEG)-based Brain-Computer Interfaces (BCIs) build a communication path between human brain and external devices. Among EEG-based BCI paradigms, the most commonly used one is motor imagery (MI). As a hot research topic, MI EEG-based BCI has largely contributed to medical fields and smart home industry. However, because of the low signal-to-noise ratio (SNR) and the non-stationary characteristic of EEG data, it is difficult to correctly classify different types of MI-EEG signals. Recently, the advances in Deep Learning (DL) significantly facilitate the development of MI EEG-based BCIs. In this paper, we provide a systematic survey of DL-based MI-EEG classification methods. Specifically, we first comprehensively discuss several important aspects of DL-based MI-EEG classification, covering input formulations, network architectures, public datasets, etc. Then, we summarize problems in model performance comparison and give guidelines to future studies for fair performance comparison. Next, we fairly evaluate the representative DL-based models using source code released by the authors and meticulously analyse the evaluation results. By performing ablation study on the network architecture, we found that (1) effective feature fusion is indispensable for multi-stream CNN-based models. (2) LSTM should be combined with spatial feature extraction techniques to obtain good classification performance. (3) the use of dropout contributes little to improving the model performance, and that (4) adding fully connected layers to the models significantly increases their parameters but it might not improve their performance. Finally, we raise several open issues in MI-EEG classification and provide possible future research directions.

EEG-FMCNN: A fusion multi-branch 1D convolutional neural network for EEG-based motor imagery classification

Motor imagery (MI) electroencephalogram (EEG) signal is recognized as a promising paradigm for brain-computer interface (BCI) systems and has been extensively employed in various BCI applications, including assisting disabled individuals, controlling devices and environments, and enhancing human capabilities. The high-performance decoding capability of MI-EEG signals is a key issue that impacts the development of the industry. However, decoding MI-EEG signals is challenging due to the low signal-to-noise ratio and inter-subject variability. In response to the aforementioned core problems, this paper proposes a novel end-to-end network, a fusion multi-branch 1D convolutional neural network (EEG-FMCNN), to decode MI-EEG signals without pre-processing. The utilization of multi-branch 1D convolution not only exhibits a certain level of noise tolerance but also addresses the issue of inter-subject variability to some extent. This is attributed to the ability of multi-branch architectures to capture information from different frequency bands, enabling the establishment of optimal convolutional scales and depths. Furthermore, we incorporate 1D squeeze-and-excitation (SE) blocks and shortcut connections at appropriate locations to further enhance the generalization and robustness of the network. In the BCI Competition IV-2a dataset, our proposed model has obtained good experimental results, achieving accuracies of 78.82% and 68.41% for subject-dependent and subject-independent modes, respectively. In addition, extensive ablative experiments and fine-tuning experiments were conducted, resulting in a notable 7% improvement in the average performance of the network, which holds significant implications for the generalization and application of the network.

EEG decoding for datasets with heterogenous electrode configurations using transfer learning graph neural networks

Objective. Brain-machine interfacing (BMI) has greatly benefited from adopting machine learning methods for feature learning that require extensive data for training, which are often unavailable from a single dataset. Yet, it is difficult to combine data across labs or even data within the same lab collected over the years due to the variation in recording equipment and electrode layouts resulting in shifts in data distribution, changes in data dimensionality, and altered identity of data dimensions. Our objective is to overcome this limitation and learn from many different and diverse datasets across labs with different experimental protocols.Approach. To tackle the domain adaptation problem, we developed a novel machine learning framework combining graph neural networks (GNNs) and transfer learning methodologies for non-invasive motor imagery (MI) EEG decoding, as an example of BMI. Empirically, we focus on the challenges of learning from EEG data with different electrode layouts and varying numbers of electrodes. We utilize three MI EEG databases collected using very different numbers of EEG sensors (from 22 channels to 64) and layouts (from custom layouts to 10-20).Main results. Our model achieved the highest accuracy with lower standard deviations on the testing datasets. This indicates that the GNN-based transfer learning framework can effectively aggregate knowledge from multiple datasets with different electrode layouts, leading to improved generalization in subject-independent MI EEG classification.Significance. The findings of this study have important implications for brain-computer-interface research, as they highlight a promising method for overcoming the limitations posed by non-unified experimental setups. By enabling the integration of diverse datasets with varying electrode layouts, our proposed approach can help advance the development and application of BMI technologies.

A multi-band centroid contrastive reconstruction fusion network for motor imagery electroencephalogram signal decoding

Motor imagery (MI) brain-computer interface (BCI) assist users in establishing direct communication between their brain and external devices by decoding the movement intention of human electroencephalogram (EEG) signals. However, cerebral cortical potentials are highly rhythmic and sub-band features, different experimental situations and subjects have different categories of semantic information in specific sample target spaces. Feature fusion can lead to more discriminative features, but simple fusion of features from different embedding spaces leading to the model global loss is not easily convergent and ignores the complementarity of features. Considering the similarity and category contribution of different sub-band features, we propose a multi-band centroid contrastive reconstruction fusion network (MB-CCRF). We obtain multi-band spatio-temporal features by frequency division, preserving the task-related rhythmic features of different EEG signals; use a multi-stream cross-layer connected convolutional network to perform a deep feature representation for each sub-band separately; propose a centroid contrastive reconstruction fusion module, which maps different sub-band and category features into the same shared embedding space by comparing with category prototypes, reconstructing the feature semantic structure to ensure that the global loss of the fused features converges more easily. Finally, we use a learning mechanism to model the similarity between channel features and use it as the weight of fused sub-band features, thus enhancing the more discriminative features, suppressing the useless features. The experimental accuracy is 79.96% in the BCI competition Ⅳ-Ⅱa dataset. Moreover, the classification effect of sub-band features of different subjects is verified by comparison tests, the category propensity of different sub-band features is verified by confusion matrix tests and the distribution in different classes of each sub-band feature and fused feature are showed by visual analysis, revealing the importance of different sub-band features for the EEG-based MI classification task.

Motor Imagery EEG Classification Based on a Weighted Multi-Branch Structure Suitable for Multisubject Data

OBJECTIVE

Electroencephalogram (EEG) signal recognition based on deep learning technology requires the support of sufficient data. However, training data scarcity usually occurs in subject-specific motor imagery tasks unless multisubject data can be used to enlarge training data. Unfortunately, because of the large discrepancies between data distributions from different subjects, model performance could only be improved marginally or even worsened by simply training on multisubject data.

METHOD

This article proposes a novel weighted multi-branch (WMB) structure for handling multisubject data to solve the problem, in which each branch is responsible for fitting a pair of source-target subject data and adaptive weights are used to integrate all branches or select branches with the largest weights to make the final decision. The proposed WMB structure was applied to six well-known deep learning models (EEGNet, Shallow ConvNet, Deep ConvNet, ResNet, MSFBCNN, and EEG_TCNet) and comprehensive experiments were conducted on EEG datasets BCICIV-2a, BCICIV-2b, high gamma dataset (HGD) and two supplementary datasets.

RESULT

Superior results against the state-of-the-art models have demonstrated the efficacy of the proposed method in subject-specific motor imagery EEG classification. For example, the proposed WMB_EEGNet achieved classification accuracies of 84.14%, 90.23%, and 97.81% on BCICIV-2a, BCICIV-2b and HGD, respectively.

CONCLUSION

It is clear that the proposed WMB structure is capable to make good use of multisubject data with large distribution discrepancies for subject-specific EEG classification.

A Real-Time Non-Implantation Bi-Directional Brain-Computer Interface Solution Without Stimulation Artifacts

The non-implantation bi-directional brain-computer interface (BCI) is a neural interface technology that enables direct two-way communication between the brain and the external world by both "reading" neural signals and "writing" stimulation patterns to the brain. This technology has vast potential applications, such as improving the quality of life for individuals with neurological and mental illnesses and even expanding the boundaries of human capabilities. Nonetheless, non-implantation bi-directional BCIs face challenges in generating real-time feedback and achieving compatibility between stimulation and recording. These issues arise due to the considerable overlap between electrical stimulation frequencies and electrophysiological recording frequencies, as well as the impediment caused by the skull to the interaction of external and internal currents. To address those challenges, this work proposes a novel solution that combines the temporal interference stimulation paradigm and minimally invasive skull modification. A longitudinal animal experiment has preliminarily validated the feasibility of the proposed method. In signal recording experiments, the average impedance of our scheme decreased by 4.59 kΩ , about 67%, compared to the conventional technique at 18 points. The peak-to-peak value of the Somatosensory Evoked Potential increased by 8%. Meanwhile, the signal-to-noise ratio of Steady-State Visual Evoked Potential increased by 5.13 dB, and its classification accuracy increased by 44%. The maximum bandwidth of the resting state rose by 63%. In electrical stimulation experiments, the signal-to-noise ratio of the low-frequency response evoked by our scheme rose by 8.04 dB, and no stimulation artifacts were generated. The experimental results show that signal quality in acquisition has significantly improved, and frequency-band isolation eliminates stimulation artifacts at the source. The acquisition and stimulation pathways are real-time compatible in this non-implantation bi-directional BCI solution, which can provide technical support and theoretical guidance for creating closed-loop adaptive systems coupled with particular application scenarios in the future.

Functional Connectivity and Feature Fusion Enhance Multiclass Motor-Imagery Brain-Computer Interface Performance

(1) Background: in the field of motor-imagery brain-computer interfaces (MI-BCIs), obtaining discriminative features among multiple MI tasks poses a significant challenge. Typically, features are extracted from single electroencephalography (EEG) channels, neglecting their interconnections, which leads to limited results. To address this limitation, there has been growing interest in leveraging functional brain connectivity (FC) as a feature in MI-BCIs. However, the high inter- and intra-subject variability has so far limited its effectiveness in this domain. (2) Methods: we propose a novel signal processing framework that addresses this challenge. We extracted translation-invariant features (TIFs) obtained from a scattering convolution network (SCN) and brain connectivity features (BCFs). Through a feature fusion approach, we combined features extracted from selected channels and functional connectivity features, capitalizing on the strength of each component. Moreover, we employed a multiclass support vector machine (SVM) model to classify the extracted features. (3) Results: using a public dataset (IIa of the BCI Competition IV), we demonstrated that the feature fusion approach outperformed existing state-of-the-art methods. Notably, we found that the best results were achieved by merging TIFs with BCFs, rather than considering TIFs alone. (4) Conclusions: our proposed framework could be the key for improving the performance of a multiclass MI-BCI system.

Temporal-frequency-phase feature classification using 3D-convolutional neural networks for motor imagery and movement

Recently, convolutional neural networks (CNNs) have been widely applied in brain-computer interface (BCI) based on electroencephalogram (EEG) signals. Due to the subject-specific nature of EEG signal patterns and the multi-dimensionality of EEG features, it is necessary to employ appropriate feature representation methods to enhance the decoding accuracy of EEG. In this study, we proposed a method for representing EEG temporal, frequency, and phase features, aiming to preserve the multi-domain information of EEG signals. Specifically, we generated EEG temporal segments using a sliding window strategy. Then, temporal, frequency, and phase features were extracted from different temporal segments and stacked into 3D feature maps, namely temporal-frequency-phase features (TFPF). Furthermore, we designed a compact 3D-CNN model to extract these multi-domain features efficiently. Considering the inter-individual variability in EEG data, we conducted individual testing for each subject. The proposed model achieved an average accuracy of 89.86, 78.85, and 63.55% for 2-class, 3-class, and 4-class motor imagery (MI) classification tasks, respectively, on the PhysioNet dataset. On the GigaDB dataset, the average accuracy for 2-class MI classification was 91.91%. For the comparison between MI and real movement (ME) tasks, the average accuracy for the 2-class were 87.66 and 80.13% on the PhysioNet and GigaDB datasets, respectively. Overall, the method presented in this paper have obtained good results in MI/ME tasks and have a good application prospect in the development of BCI systems based on MI/ME.

An improved model using convolutional sliding window-attention network for motor imagery EEG classification

Introduction

The classification model of motor imagery-based electroencephalogram (MI-EEG) is a new human-computer interface pattern and a new neural rehabilitation assessment method for diseases such as Parkinson's and stroke. However, existing MI-EEG models often suffer from insufficient richness of spatiotemporal feature extraction, learning ability, and dynamic selection ability.

Methods

To solve these problems, this work proposed a convolutional sliding window-attention network (CSANet) model composed of novel spatiotemporal convolution, sliding window, and two-stage attention blocks.

Results

The model outperformed existing state-of-the-art (SOTA) models in within- and between-individual classification tasks on commonly used MI-EEG datasets BCI-2a and Physionet MI-EEG, with classification accuracies improved by 4.22 and 2.02%, respectively.

Discussion

The experimental results also demonstrated that the proposed type token, sliding window, and local and global multi-head self-attention mechanisms can significantly improve the model's ability to construct, learn, and adaptively select multi-scale spatiotemporal features in MI-EEG signals, and accurately identify electroencephalogram signals in the unilateral motor area. This work provided a novel and accurate classification model for MI-EEG brain-computer interface tasks and proposed a feasible neural rehabilitation assessment scheme based on the model, which could promote the further development and application of MI-EEG methods in neural rehabilitation.

Decoding motor imagery with a simplified distributed dipoles model at source level

Motor imagery (MI) based brain computer interface significantly oriented the development of neuro-rehabilitation, and the crucial issue is how to accurately detect the changes of cerebral cortex for MI decoding. The brain activity can be calculated based on the head model and observed scalp EEG, providing insights regarding cortical dynamics by using equivalent current dipoles with high spatial and temporal resolution. Now, all the dipoles within entire cortex or partial regions of interest are directly applied to data representation, this may make the key information weakened or lost, and it is worth studying how to choose the most important from numerous dipoles. In this paper, we devote to building a simplified distributed dipoles model (SDDM), which is combined with convolutional neural network (CNN), generating a MI decoding method at source level (called SDDM-CNN). First, all channels of raw MI-EEG signals are subdivided by a series of bandpass filters with width of 1 Hz, the average energies associated with any sub-band signals are calculated and ranked in a descending order to screen the top n sub-bands; then, the MI-EEG signals over each selected sub-band are mapped into source space by using EEG source imaging technology, and for each scout of neuroanatomical Desikan-Killiany partition, a centered dipole is selected as the most relevant dipole and put together to build a SDDM to reflect the neuroelectric activity of entire cerebral cortex; finally, the 4 dimensional (4D) magnitude matrix is constructed for each SDDM and fused into a novel data representation, which is further input to a well-designed 3DCNN with n parallel branches (nB3DCNN) to extract and classify the comprehensive features from time-frequency-space dimensions. Experiments are carried out on three public datasets, and the average ten-fold CV decoding accuracies achieve 95.09%, 97.98% and 94.53% respectively, and the statistical analysis is fulfilled by standard deviation, kappa value and confusion matrix. Experiment results suggest that it is beneficial to pick out the most sensitive sub-bands in sensor domain, and SDDM can sufficiently describe the dynamic changing of entire cortex, improving decoding performance while greatly reducing number of source signals. Also, nB3DCNN is capable of exploring spatial-temporal features from multi sub-bands.

An attention-based deep learning approach for the classification of subjective cognitive decline and mild cognitive impairment using resting-state EEG

Objective. This study aims to design and implement the first deep learning (DL) model to classify subjects in the prodromic states of Alzheimer's disease (AD) based on resting-state electroencephalographic (EEG) signals.Approach. EEG recordings of 17 healthy controls (HCs), 56 subjective cognitive decline (SCD) and 45 mild cognitive impairment (MCI) subjects were acquired at resting state. After preprocessing, we selected sections corresponding to eyes-closed condition. Five different datasets were created by extracting delta, theta, alpha, beta and delta-to-theta frequency bands using bandpass filters. To classify SCDvsMCI and HCvsSCDvsMCI, we propose a framework based on the transformer architecture, which uses multi-head attention to focus on the most relevant parts of the input signals. We trained and validated the model on each dataset with a leave-one-subject-out cross-validation approach, splitting the signals into 10 s epochs. Subjects were assigned to the same class as the majority of their epochs. Classification performances of the transformer were assessed for both epochs and subjects and compared with other DL models.Main results. Results showed that the delta dataset allowed our model to achieve the best performances for the discrimination of SCD and MCI, reaching an Area Under the ROC Curve (AUC) of 0.807, while the highest results for the HCvsSCDvsMCI classification were obtained on alpha and theta with a micro-AUC higher than 0.74.Significance. We demonstrated that DL approaches can support the adoption of non-invasive and economic techniques as EEG to stratify patients in the clinical population at risk for AD. This result was achieved since the attention mechanism was able to learn temporal dependencies of the signal, focusing on the most discriminative patterns, achieving state-of-the-art results by using a deep model of reduced complexity. Our results were consistent with clinical evidence that changes in brain activity are progressive when considering early stages of AD.

Continuous versus discrete robotic feedback for brain-computer interfaces aimed for neurorehabilitation

Introduction

Brain-Computer Interfaces (BCI) can allow control of external devices using motor imagery (MI) decoded from electroencephalography (EEG). Although BCI have a wide range of applications including neurorehabilitation, the low spatial resolution of EEG, coupled to the variability of cortical activations during MI, make control of BCI based on EEG a challenging task.

Methods

An assessment of BCI control with different feedback timing strategies was performed. Two different feedback timing strategies were compared, comprised by passive hand movement provided by a robotic hand orthosis. One of the timing strategies, the continuous, involved the partial movement of the robot immediately after the recognition of each time segment in which hand MI was performed. The other feedback, the discrete, was comprised by the entire movement of the robot after the processing of the complete MI period. Eighteen healthy participants performed two sessions of BCI training and testing, one with each feedback.

Results

Significantly higher BCI performance (65.4 ± 17.9% with the continuous and 62.1 ± 18.6% with the discrete feedback) and pronounced bilateral alpha and ipsilateral beta cortical activations were observed with the continuous feedback.

Discussion

It was hypothesized that these effects, although heterogenous across participants, were caused by the enhancement of attentional and closed-loop somatosensory processes. This is important, since a continuous feedback timing could increase the number of BCI users that can control a MI-based system or enhance cortical activations associated with neuroplasticity, important for neurorehabilitation applications.

Multibranch convolutional neural network with contrastive representation learning for decoding same limb motor imagery tasks

Introduction

Emerging deep learning approaches to decode motor imagery (MI) tasks have significantly boosted the performance of brain-computer interfaces. Although recent studies have produced satisfactory results in decoding MI tasks of different body parts, the classification of such tasks within the same limb remains challenging due to the activation of overlapping brain regions. A single deep learning model may be insufficient to effectively learn discriminative features among tasks.

Methods

The present study proposes a framework to enhance the decoding of multiple hand-MI tasks from the same limb using a multi-branch convolutional neural network. The CNN framework utilizes feature extractors from established deep learning models, as well as contrastive representation learning, to derive meaningful feature representations for classification.

Results

The experimental results suggest that the proposed method outperforms several state-of-the-art methods by obtaining a classification accuracy of 62.98% with six MI classes and 76.15 % with four MI classes on the Tohoku University MI-BCI and BCI Competition IV datasets IIa, respectively.

Discussion

Despite requiring heavy data augmentation and multiple optimization steps, resulting in a relatively long training time, this scheme is still suitable for online use. However, the trade-of between the number of base learners, training time, prediction time, and system performance should be carefully considered.

EEG-Based Mental Tasks Recognition via a Deep Learning-Driven Anomaly Detector

This paper introduces an unsupervised deep learning-driven scheme for mental tasks' recognition using EEG signals. To this end, the Multichannel Wiener filter was first applied to EEG signals as an artifact removal algorithm to achieve robust recognition. Then, a quadratic time-frequency distribution (QTFD) was applied to extract effective time-frequency signal representation of the EEG signals and catch the EEG signals' spectral variations over time to improve the recognition of mental tasks. The QTFD time-frequency features are employed as input for the proposed deep belief network (DBN)-driven Isolation Forest (iF) scheme to classify the EEG signals. Indeed, a single DBN-based iF detector is constructed based on each class's training data, with the class's samples as inliers and all other samples as anomalies (i.e., one-vs.-rest). The DBN is considered to learn pertinent information without assumptions on the data distribution, and the iF scheme is used for data discrimination. This approach is assessed using experimental data comprising five mental tasks from a publicly available database from the Graz University of Technology. Compared to the DBN-based Elliptical Envelope, Local Outlier Factor, and state-of-the-art EEG-based classification methods, the proposed DBN-based iF detector offers superior discrimination performance of mental tasks.

Dual attentive fusion for EEG-based brain-computer interfaces

The classification based on Electroencephalogram (EEG) is a challenging task in the brain-computer interface (BCI) field due to data with a low signal-to-noise ratio. Most current deep learning based studies in this challenge focus on designing a desired convolutional neural network (CNN) to learn and classify the raw EEG signals. However, only CNN itself may not capture the highly discriminative patterns of EEG due to a lack of exploration of attentive spatial and temporal dynamics. To improve information utilization, this study proposes a Dual Attentive Fusion Model (DAFM) for the EEG-based BCI. DAFM is employed to capture the spatial and temporal information by modeling the interdependencies between the features from the EEG signals. To our best knowledge, our method is the first to fuse spatial and temporal dimensions in an interactive attention module. This module improves the expression ability of the extracted features. Extensive experiments implemented on four publicly available datasets demonstrate that our method outperforms state-of-the-art methods. Meanwhile, this work also indicates the effectiveness of Dual Attentive Fusion Module.

Brain-Computer Interface using neural network and temporal-spectral features

Brain-Computer Interfaces (BCIs) are increasingly useful for control. Such BCIs can be used to assist individuals who lost mobility or control over their limbs, for recreational purposes such as gaming or semi-autonomous driving, or as an interface toward man-machine integration. Thus far, the performance of algorithms used for thought decoding has been limited. We show that by extracting temporal and spectral features from electroencephalography (EEG) signals and, following, using deep learning neural network to classify those features, one can significantly improve the performance of BCIs in predicting which motor action was imagined by a subject. Our movement prediction algorithm uses Sequential Backward Selection technique to jointly choose temporal and spectral features and a radial basis function neural network for the classification. The method shows an average performance increase of 3.50% compared to state-of-the-art benchmark algorithms. Using two popular public datasets our algorithm reaches 90.08% accuracy (compared to an average benchmark of 79.99%) on the first dataset and 88.74% (average benchmark: 82.01%) on the second dataset. Given the high variability within- and across-subjects in EEG-based action decoding, we suggest that using features from multiple modalities along with neural network classification protocol is likely to increase the performance of BCIs across various tasks.

A Transformer-Based Approach Combining Deep Learning Network and Spatial-Temporal Information for Raw EEG Classification

The attention mechanism of the Transformer has the advantage of extracting feature correlation in the long-sequence data and visualizing the model. As time-series data, the spatial and temporal dependencies of the EEG signals between the time points and the different channels contain important information for accurate classification. So far, Transformer-based approaches have not been widely explored in motor-imagery EEG classification and visualization, especially lacking general models based on cross-individual validation. Taking advantage of the Transformer model and the spatial-temporal characteristics of the EEG signals, we designed Transformer-based models for classifications of motor imagery EEG based on the PhysioNet dataset. With 3s EEG data, our models obtained the best classification accuracy of 83.31%, 74.44%, and 64.22% on two-, three-, and four-class motor-imagery tasks in cross-individual validation, which outperformed other state-of-the-art models by 0.88%, 2.11%, and 1.06%. The inclusion of the positional embedding modules in the Transformer could improve the EEG classification performance. Furthermore, the visualization results of attention weights provided insights into the working mechanism of the Transformer-based networks during motor imagery tasks. The topography of the attention weights revealed a pattern of event-related desynchronization (ERD) which was consistent with the results from the spectral analysis of Mu and beta rhythm over the sensorimotor areas. Together, our deep learning methods not only provide novel and powerful tools for classifying and understanding EEG data but also have broad applications for brain-computer interface (BCI) systems.

Overview of the EEG-Based Classification of Motor Imagery Activities Using Machine Learning Methods and Inference Acceleration with FPGA-Based Cards

In this article, we provide a brief overview of the EEG-based classification of motor imagery activities using machine learning methods. We examined the effect of data segmentation and different neural network structures. By applying proper window size and using a purely convolutional neural network, we achieved 97.7% recognition accuracy on data from twenty subjects in three classes. The proposed architecture outperforms several networks used in previous research and makes the motor imagery-based BCI more efficient in some applications. In addition, we examined the performance of the neural network on a FPGA-based card and compared it with the inference speed and accuracy provided by a general-purpose processor.

Exploiting pretrained CNN models for the development of an EEG-based robust BCI framework

Identifying motor and mental imagery electroencephalography (EEG) signals is imperative to realizing automated, robust brain-computer interface (BCI) systems. In the present study, we proposed a pretrained convolutional neural network (CNN)-based new automated framework feasible for robust BCI systems with small and ample samples of motor and mental imagery EEG training data. The framework is explored by investigating the implications of different limiting factors, such as learning rates and optimizers, processed versus unprocessed scalograms, and features derived from untuned pretrained models in small, medium, and large pretrained CNN models. The experiments were performed on three public datasets obtained from BCI Competition III. The datasets were denoised with multiscale principal component analysis, and time-frequency scalograms were obtained by employing a continuous wavelet transform. The scalograms were fed into several variants of ten pretrained models for feature extraction and identification of different EEG tasks. The experimental results showed that ShuffleNet yielded the maximum average classification accuracy of 99.52% using an RMSProp optimizer with a learning rate of 0.000 1. It was observed that low learning rates converge to more optimal performances compared to high learning rates. Moreover, noisy scalograms and features extracted from untuned networks resulted in slightly lower performance than denoised scalograms and tuned networks, respectively. The overall results suggest that pretrained models are robust when identifying EEG signals because of their ability to preserve the time-frequency structure of EEG signals and promising classification outcomes.

Feature Extraction Method Based on Filter Banks and Riemannian Tangent Space in Motor-Imagery BCI

Optimal feature extraction for multi-category motor imagery brain-computer interfaces (MI-BCIs) is a research hotspot. The common spatial pattern (CSP) algorithm is one of the most widely used methods in MI-BCIs. However, its performance is adversely affected by variance in the operational frequency band and noise interference. Furthermore, the performance of CSP is not satisfactory when addressing multi-category classification problems. In this work, we propose a fusion method combining Filter Banks and Riemannian Tangent Space (FBRTS) in multiple time windows. FBRTS uses multiple filter banks to overcome the problem of variance in the operational frequency band. It also applies the Riemannian method to the covariance matrix extracted by the spatial filter to obtain more robust features in order to overcome the problem of noise interference. In addition, we use a One-Versus-Rest support vector machine (OVR-SVM) model to classify multi-category features. We evaluate our FBRTS method using BCI competition IV dataset 2a and 2b. The experimental results show that the average classification accuracy of our FBRTS method is 77.7% and 86.9% in datasets 2a and 2b respectively. By analyzing the influence of the different numbers of filter banks and time windows on the performance of our FBRTS method, we can identify the optimal number of filter banks and time windows. Additionally, our FBRTS method can obtain more distinctive features than the filter banks common spatial pattern (FBCSP) method in two-dimensional embedding space. These results show that our proposed method can improve the performance of MI-BCIs.

Gated Transformer for Decoding Human Brain EEG Signals

In this work, we propose to use a deep learning framework for decoding the electroencephalogram (EEG) signals of human brain activities. More specifically, we learn an end-to-end model that recognizes natural images or motor imagery by the EEG data that is collected from the corresponding human neural activities. In order to capture the temporal information encoded in the long EEG sequences, we first employ an enhanced version of Transformer, i.e., gated Transformer, on EEG signals to learn the feature representation along a sequence of embeddings. Then a fully-connected Softmax layer is used to predict the classification results of the decoded representations. To demonstrate the effectiveness of the gated Transformer approach, we conduct experiments on the image classification task for a human brain-visual dataset and the classification task for a motor imagery dataset. The experimental results show that our method achieves new state-of-the-art performance compared to multiple existing methods that are widely used for EEG classification.

Motor Intention Decoding from the Upper Limb by Graph Convolutional Network Based on Functional Connectivity

Decoding brain intention from noninvasively measured neural signals has recently been a hot topic in brain-computer interface (BCI). The motor commands about the movements of fine parts can increase the degrees of freedom under control and be applied to external equipment without stimulus. In the decoding process, the classifier is one of the key factors, and the graph information of the EEG was ignored by most researchers. In this paper, a graph convolutional network (GCN) based on functional connectivity was proposed to decode the motor intention of four fine parts movements (shoulder, elbow, wrist, hand). First, event-related desynchronization was analyzed to reveal the differences between the four classes. Second, functional connectivity was constructed by using synchronization likelihood (SL), phase-locking value (PLV), H index (H), mutual information (MI), and weighted phase-lag index (WPLI) to acquire the electrode pairs with a difference. Subsequently, a GCN and convolutional neural networks (CNN) were performed based on functional topological structures and time points, respectively. The results demonstrated that the proposed method achieved a decoding accuracy of up to 92.81% in the four-class task. Besides, the combination of GCN and functional connectivity can promote the development of BCI.

Review on Emotion Recognition Based on Electroencephalography

Emotions are closely related to human behavior, family, and society. Changes in emotions can cause differences in electroencephalography (EEG) signals, which show different emotional states and are not easy to disguise. EEG-based emotion recognition has been widely used in human-computer interaction, medical diagnosis, military, and other fields. In this paper, we describe the common steps of an emotion recognition algorithm based on EEG from data acquisition, preprocessing, feature extraction, feature selection to classifier. Then, we review the existing EEG-based emotional recognition methods, as well as assess their classification effect. This paper will help researchers quickly understand the basic theory of emotion recognition and provide references for the future development of EEG. Moreover, emotion is an important representation of safety psychology.

Multi-class motor imagery EEG classification method with high accuracy and low individual differences based on hybrid neural network

Objective.Most current methods of classifying different patterns for motor imagery EEG signals require complex pre-processing and feature extraction steps, which consume time and lack adaptability, ignoring individual differences in EEG signals. It is essential to improve algorithm performance with the increased classes and diversity of subjects.Approach.This study introduces deep learning method for end-to-end learning to complete the classification of four-class MI tasks, aiming to improve the recognition rate and balance the classification accuracy among different subjects. A new one-dimensional input data representation method is proposed. This representation method can increase the number of samples and ignore the influence of channel correlation. In addition, a cascade network of convolutional neural network and gated recurrent unit is designed to learn time-frequency information from EEG data without extracting features manually, this model can capture the hidden representations related to different MI mode of each people.Main results. Experiments on BCI Competition 2a dataset and actual collected dataset achieve high accuracy near 99.40% and 92.56%, and the standard deviation is 0.34 and 1.35 respectively. Results demonstrate that the proposed method outperforms the advanced methods and baseline models.Significance.Experimental results show that the proposed method improves the accuracy of multi-classification and overcomes the impact of individual differences on classification by training neural network subject-dependent, which promotes the development of actual brain-computer interface systems.

Attention-Based Parallel Multiscale Convolutional Neural Network for Visual Evoked Potentials EEG Classification

Electroencephalography (EEG) decoding is an important part of Visual Evoked Potentials-based Brain-Computer Interfaces (BCIs), which directly determines the performance of BCIs. However, long-time attention to repetitive visual stimuli could cause physical and psychological fatigue, resulting in weaker reliable response and stronger noise interference, which exacerbates the difficulty of Visual Evoked Potentials EEG decoding. In this state, subjects' attention could not be concentrated enough and the frequency response of their brains becomes less reliable. To solve these problems, we propose an attention-based parallel multiscale convolutional neural network (AMS-CNN). Specifically, the AMS-CNN first extract robust temporal representations via two parallel convolutional layers with small and large temporal filters respectively. Then, we employ two sequential convolution blocks for spatial fusion and temporal fusion to extract advanced feature representations. Further, we use attention mechanism to weight the features at different moments according to the output-related interest. Finally, we employ a full connected layer with softmax activation function for classification. Two fatigue datasets collected from our lab are implemented to validate the superior classification performance of the proposed method compared to the state-of-the-art methods. Analysis reveals the competitiveness of multiscale convolution and attention mechanism. These results suggest that the proposed framework is a promising solution to improving the decoding performance of Visual Evoked Potential BCIs.

BENDR: Using Transformers and a Contrastive Self-Supervised Learning Task to Learn From Massive Amounts of EEG Data

Deep neural networks (DNNs) used for brain-computer interface (BCI) classification are commonly expected to learn general features when trained across a variety of contexts, such that these features could be fine-tuned to specific contexts. While some success is found in such an approach, we suggest that this interpretation is limited and an alternative would better leverage the newly (publicly) available massive electroencephalography (EEG) datasets. We consider how to adapt techniques and architectures used for language modeling (LM) that appear capable of ingesting awesome amounts of data toward the development of encephalography modeling with DNNs in the same vein. We specifically adapt an approach effectively used for automatic speech recognition, which similarly (to LMs) uses a self-supervised training objective to learn compressed representations of raw data signals. After adaptation to EEG, we find that a single pre-trained model is capable of modeling completely novel raw EEG sequences recorded with differing hardware, and different subjects performing different tasks. Furthermore, both the internal representations of this model and the entire architecture can be fine-tuned to a variety of downstream BCI and EEG classification tasks, outperforming prior work in more task-specific (sleep stage classification) self-supervision.

Cross-Subject EEG Emotion Recognition With Self-Organized Graph Neural Network

As a physiological process and high-level cognitive behavior, emotion is an important subarea in neuroscience research. Emotion recognition across subjects based on brain signals has attracted much attention. Due to individual differences across subjects and the low signal-to-noise ratio of EEG signals, the performance of conventional emotion recognition methods is relatively poor. In this paper, we propose a self-organized graph neural network (SOGNN) for cross-subject EEG emotion recognition. Unlike the previous studies based on pre-constructed and fixed graph structure, the graph structure of SOGNN are dynamically constructed by self-organized module for each signal. To evaluate the cross-subject EEG emotion recognition performance of our model, leave-one-subject-out experiments are conducted on two public emotion recognition datasets, SEED and SEED-IV. The SOGNN is able to achieve state-of-the-art emotion recognition performance. Moreover, we investigated the performance variances of the models with different graph construction techniques or features in different frequency bands. Furthermore, we visualized the graph structure learned by the proposed model and found that part of the structure coincided with previous neuroscience research. The experiments demonstrated the effectiveness of the proposed model for cross-subject EEG emotion recognition.

A Survey on Deep Learning-Based Short/Zero-Calibration Approaches for EEG-Based Brain-Computer Interfaces

Brain-computer interfaces (BCIs) utilizing machine learning techniques are an emerging technology that enables a communication pathway between a user and an external system, such as a computer. Owing to its practicality, electroencephalography (EEG) is one of the most widely used measurements for BCI. However, EEG has complex patterns and EEG-based BCIs mostly involve a cost/time-consuming calibration phase; thus, acquiring sufficient EEG data is rarely possible. Recently, deep learning (DL) has had a theoretical/practical impact on BCI research because of its use in learning representations of complex patterns inherent in EEG. Moreover, algorithmic advances in DL facilitate short/zero-calibration in BCI, thereby suppressing the data acquisition phase. Those advancements include data augmentation (DA), increasing the number of training samples without acquiring additional data, and transfer learning (TL), taking advantage of representative knowledge obtained from one dataset to address the so-called data insufficiency problem in other datasets. In this study, we review DL-based short/zero-calibration methods for BCI. Further, we elaborate methodological/algorithmic trends, highlight intriguing approaches in the literature, and discuss directions for further research. In particular, we search for generative model-based and geometric manipulation-based DA methods. Additionally, we categorize TL techniques in DL-based BCIs into explicit and implicit methods. Our systematization reveals advances in the DA and TL methods. Among the studies reviewed herein, ~45% of DA studies used generative model-based techniques, whereas ~45% of TL studies used explicit knowledge transferring strategy. Moreover, based on our literature review, we recommend an appropriate DA strategy for DL-based BCIs and discuss trends of TLs used in DL-based BCIs.

A novel decoding method for motor imagery tasks with 4D data representation and 3D convolutional neural networks

Objective. Motor imagery electroencephalography (MI-EEG) produces one of the most commonly used biosignals in intelligent rehabilitation systems. The newly developed 3D convolutional neural network (3DCNN) is gaining increasing attention for its ability to recognize MI tasks. The key to successful identification of movement intention is dependent on whether the data representation can faithfully reflect the cortical activity induced by MI. However, the present data representation, which is often generated from partial source signals with time-frequency analysis, contains incomplete information. Therefore, it would be beneficial to explore a new type of data representation using raw spatiotemporal dipole information as well as the possible development of a matching 3DCNN.Approach.Based on EEG source imaging and 3DCNN, a novel decoding method for identifying MI tasks is proposed, called ESICNND. MI-EEG is mapped to the cerebral cortex by the standardized low resolution electromagnetic tomography algorithm, and the optimal sampling points of the dipoles are selected as the time of interest to best reveal the difference between any two MI tasks. Then, the initial subject coordinate system is converted to a magnetic resonance imaging coordinate system, followed by dipole interpolation and volume down-sampling; the resulting 3D dipole amplitude matrices are merged at the selected sampling points to obtain 4D dipole feature matrices (4DDFMs). These matrices are augmented by sliding window technology and input into a 3DCNN with a cascading architecture of three modules (3M3DCNN) to perform the extraction and classification of comprehensive features.Main results.Experiments are carried out on two public datasets; the average ten-fold CV classification accuracies reach 88.73% and 96.25%, respectively, and the statistical analysis demonstrates outstanding consistency and stability.Significance.The 4DDFMs reveals the variation of cortical activation in a 3D spatial cube with a temporal dimension and matches the 3M3DCNN well, making full use of the high-resolution spatiotemporal information from all dipoles.

Recognition of Consumer Preference by Analysis and Classification EEG Signals

Neuromarketing has gained attention to bridge the gap between conventional marketing studies and electroencephalography (EEG)-based brain-computer interface (BCI) research. It determines what customers actually want through preference prediction. The performance of EEG-based preference detection systems depends on a suitable selection of feature extraction techniques and machine learning algorithms. In this study, We examined preference detection of neuromarketing dataset using different feature combinations of EEG indices and different algorithms for feature extraction and classification. For EEG feature extraction, we employed discrete wavelet transform (DWT) and power spectral density (PSD), which were utilized to measure the EEG-based preference indices that enhance the accuracy of preference detection. Moreover, we compared deep learning with other traditional classifiers, such as k-nearest neighbor (KNN), support vector machine (SVM), and random forest (RF). We also studied the effect of preference indicators on the performance of classification algorithms. Through rigorous offline analysis, we investigated the computational intelligence for preference detection and classification. The performance of the proposed deep neural network (DNN) outperforms KNN and SVM in accuracy, precision, and recall; however, RF achieved results similar to those of the DNN for the same dataset.

Uncovering the structure of clinical EEG signals with self-supervised learning

OBJECTIVE

Supervised learning paradigms are often limited by the amount of labeled data that is available. This phenomenon is particularly problematic in clinically-relevant data, such as electroencephalography (EEG), where labeling can be costly in terms of specialized expertise and human processing time. Consequently, deep learning architectures designed to learn on EEG data have yielded relatively shallow models and performances at best similar to those of traditional feature-based approaches. However, in most situations, unlabeled data is available in abundance. By extracting information from this unlabeled data, it might be possible to reach competitive performance with deep neural networks despite limited access to labels.

APPROACH

We investigated self-supervised learning (SSL), a promising technique for discovering structure in unlabeled data, to learn representations of EEG signals. Specifically, we explored two tasks based on temporal context prediction as well as contrastive predictive coding on two clinically-relevant problems: EEG-based sleep staging and pathology detection. We conducted experiments on two large public datasets with thousands of recordings and performed baseline comparisons with purely supervised and hand-engineered approaches.

MAIN RESULTS

Linear classifiers trained on SSL-learned features consistently outperformed purely supervised deep neural networks in low-labeled data regimes while reaching competitive performance when all labels were available. Additionally, the embeddings learned with each method revealed clear latent structures related to physiological and clinical phenomena, such as age effects.

SIGNIFICANCE

We demonstrate the benefit of SSL approaches on EEG data. Our results suggest that self-supervision may pave the way to a wider use of deep learning models on EEG data.