Harnessing Few-Shot Learning for EEG signal classification: a survey of state-of-the-art techniques and future directions

This paper presents a systematic literature review, providing a comprehensive taxonomy of Data Augmentation (DA), Transfer Learning (TL), and Self-Supervised Learning (SSL) techniques within the context of Few-Shot Learning (FSL) for EEG signal classification. EEG signals have shown significant potential in various paradigms, including Motor Imagery, Emotion Recognition, Visual Evoked Potentials, Steady-State Visually Evoked Potentials, Rapid Serial Visual Presentation, Event-Related Potentials, and Mental Workload. However, challenges such as limited labeled data, noise, and inter/intra-subject variability have impeded the effectiveness of traditional machine learning (ML) and deep learning (DL) models. This review methodically explores how FSL approaches, incorporating DA, TL, and SSL, can address these challenges and enhance classification performance in specific EEG paradigms. It also delves into the open research challenges related to these techniques in EEG signal classification. Specifically, the review examines the identification of DA strategies tailored to various EEG paradigms, the creation of TL architectures for efficient knowledge transfer, and the formulation of SSL methods for unsupervised representation learning from EEG data. Addressing these challenges is crucial for enhancing the efficacy and robustness of FSL-based EEG signal classification. By presenting a structured taxonomy of FSL techniques and discussing the associated research challenges, this systematic review offers valuable insights for future investigations in EEG signal classification. The findings aim to guide and inspire researchers, promoting advancements in applying FSL methodologies for improved EEG signal analysis and classification in real-world settings.

Suppression of negative transfer in motor imagery brain-computer interface based on mutual information and Pearson correlation coefficient

The focus of this paper is on the main challenges in brain-computer interface transfer learning: how to address data characteristic length and the source domain sample selection problems caused by individual differences. To overcome the negative migration that results from feature length, we propose a migration algorithm based on mutual information transfer (MIT), which selects effective features by calculating the entropy value of the probability distribution and conditional distribution, thereby reducing negative migration and improving learning efficiency. Source domain participants who differ too much from the target domain distribution can affect the overall classification performance. On the basis of MIT, we propose the Pearson correlation coefficient source domain automatic selection algorithm (PDAS algorithm). The PDAS algorithm can automatically select the appropriate source domain participants according to the target domain distribution, which reduces the negative migration of participant data among the source domain participants, improves experimental accuracy, and greatly reduces training time. The two proposed algorithms were tested offline and online on two public datasets, and the results were compared with those from existing advanced algorithms. The experimental results showed that the MIT algorithm and the MIT + PDAS algorithm had obvious advantages.

A systematic evaluation of Euclidean alignment with deep learning for EEG decoding

Objective:Electroencephalography signals are frequently used for various Brain-Computer interface (BCI) tasks. While deep learning (DL) techniques have shown promising results, they are hindered by the substantial data requirements. By leveraging data from multiple subjects, transfer learning enables more effective training of DL models. A technique that is gaining popularity is Euclidean alignment (EA) due to its ease of use, low computational complexity, and compatibility with DL models. However, few studies evaluate its impact on the training performance of shared and individual DL models. In this work, we systematically evaluate the effect of EA combined with DL for decoding BCI signals.Approach:We used EA as a pre-processing step to train shared DL models with data from multiple subjects and evaluated their transferability to new subjects.Main results:Our experimental results show that it improves decoding in the target subject by 4.33% and decreases convergence time by more than 70%. We also trained individual models for each subject to use as a majority-voting ensemble classifier. In this scenario, using EA improved the 3-model ensemble accuracy by 3.71%. However, when compared to the shared model with EA, the ensemble accuracy was 3.62% lower.Significance:EA succeeds in the task of improving transfer learning performance with DL models and, could be used as a standard pre-processing technique.

Three-stage transfer learning for motor imagery EEG recognition

Motor imagery (MI) paradigms have been widely used in neural rehabilitation and drowsiness state assessment. The progress in brain-computer interface (BCI) technology has emphasized the importance of accurately and efficiently detecting motor imagery intentions from electroencephalogram (EEG). Despite the recent breakthroughs made in developing EEG-based algorithms for decoding MI, the accuracy and efficiency of these models remain limited by technical challenges posed by cross-subject heterogeneity in EEG data processing and the scarcity of EEG data for training. Inspired by the optimal transport theory, this study aims to develop a novel three-stage transfer learning (TSTL) method, which uses the existing labeled data from a source domain to improve classification performance on an unlabeled target domain. Notably, the proposed method comprises three components, namely, the Riemannian tangent space mapping (RTSM), source domain transformer (SDT), and optimal subspace mapping (OSM). The RTSM maps a symmetric positive definite matrix from the Riemannian space to the tangent space to minimize the marginal probability distribution drift. The SDT transforms the source domain to a target domain by finding the optimal transport mapping matrix to reduce the joint probability distribution differences. The OSM finally maps the transformed source domain and original target domain to the same subspace to further mitigate the distribution discrepancy. The performance of the proposed method was validated on two public BCI datasets, and the average accuracy of the algorithm on two datasets was 72.24% and 69.29%. Our results demonstrated the improved performance of EEG-based MI detection in comparison with state-of-the-art algorithms.

Riemannian transfer learning based on log-Euclidean metric for EEG classification

Introduction

Brain computer interfaces (BCI), which establish a direct interaction between the brain and the external device bypassing peripheral nerves, is one of the hot research areas. How to effectively convert brain intentions into instructions for controlling external devices in real-time remains a key issue that needs to be addressed in brain computer interfaces. The Riemannian geometry-based methods have achieved competitive results in decoding EEG signals. However, current Riemannian classifiers tend to overlook changes in data distribution, resulting in degenerated classification performance in cross-session and/or cross subject scenarios.

Methods

This paper proposes a brain signal decoding method based on Riemannian transfer learning, fully considering the drift of the data distribution. Two Riemannian transfer learning methods based log-Euclidean metric are developed, such that historical data (source domain) can be used to aid the training of the Riemannian decoder for the current task, or data from other subjects can be used to boost the training of the decoder for the target subject.

Results

The proposed methods were verified on BCI competition III, IIIa, and IV 2a datasets. Compared with the baseline that without transfer learning, the proposed algorithm demonstrates superior classification performance. In contrast to the Riemann transfer learning method based on the affine invariant Riemannian metric, the proposed method obtained comparable classification performance, but is much more computationally efficient.

Discussion

With the help of proposed transfer learning method, the Riemannian classifier obtained competitive performance to existing methods in the literature. More importantly, the transfer learning process is unsupervised and time-efficient, possessing potential for online learning scenarios.

SSVEP-DAN: Cross-Domain Data Alignment for SSVEP-Based Brain-Computer Interfaces

Steady-state visual-evoked potential (SSVEP)-based brain-computer interfaces (BCIs) offer a non-invasive means of communication through high-speed speller systems. However, their efficiency is highly dependent on individual training data acquired during time-consuming calibration sessions. To address the challenge of data insufficiency in SSVEP-based BCIs, we introduce SSVEP-DAN, the first dedicated neural network model designed to align SSVEP data across different domains, encompassing various sessions, subjects, or devices. Our experimental results demonstrate the ability of SSVEP-DAN to transform existing source SSVEP data into supplementary calibration data. This results in a significant improvement in SSVEP decoding accuracy while reducing the calibration time. We envision SSVEP-DAN playing a crucial role in future applications of high-performance SSVEP-based BCIs. The source code for this work is available at: https://github.com/CECNL/SSVEP-DAN.

Cross-Subject Motor Imagery Decoding by Transfer Learning of Tactile ERD

For Brain-Computer Interface (BCI) based on motor imagery (MI), the MI task is abstract and spontaneous, presenting challenges in measurement and control and resulting in a lower signal-to-noise ratio. The quality of the collected MI data significantly impacts the cross-subject calibration results. To address this challenge, we introduce a novel cross-subject calibration method based on passive tactile afferent stimulation, in which data induced by tactile stimulation is utilized to calibrate transfer learning models for cross-subject decoding. During the experiments, tactile stimulation was applied to either the left or right hand, with subjects only required to sense tactile stimulation. Data from these tactile tasks were used to train or fine-tune models and subsequently applied to decode pure MI data. We evaluated BCI performance using both the classical Common Spatial Pattern (CSP) combined with the Linear Discriminant Analysis (LDA) algorithm and a state-of-the-art deep transfer learning model. The results demonstrate that the proposed calibration method achieved decoding performance at an equivalent level to traditional MI calibration, with the added benefit of outperforming traditional MI calibration with fewer trials. The simplicity and effectiveness of the proposed cross-subject tactile calibration method make it valuable for practical applications of BCI, especially in clinical settings.

A comprehensive dataset for home appliance control using ERP-based BCIs with the application of inter-subject transfer learning

Brain-computer interfaces (BCIs) have a potential to revolutionize human-computer interaction by enabling direct links between the brain and computer systems. Recent studies are increasingly focusing on practical applications of BCIs-e.g., home appliance control just by thoughts. One of the non-invasive BCIs using electroencephalography (EEG) capitalizes on event-related potentials (ERPs) in response to target stimuli and have shown promise in controlling home appliance. In this paper, we present a comprehensive dataset of online ERP-based BCIs for controlling various home appliances in diverse stimulus presentation environments. We collected online BCI data from a total of 84 subjects among whom 60 subjects controlled three types of appliances (TV: 30, door lock: 15, and electric light: 15) with 4 functions per appliance, 14 subjects controlled a Bluetooth speaker with 6 functions via an LCD monitor, and 10 subjects controlled air conditioner with 4 functions via augmented reality (AR). Using the dataset, we aimed to address the issue of inter-subject variability in ERPs by employing the transfer learning in two different approaches. The first approach, "within-paradigm transfer learning," aimed to generalize the model within the same paradigm of stimulus presentation. The second approach, "cross-paradigm transfer learning," involved extending the model from a 4-class LCD environment to different paradigms. The results demonstrated that transfer learning can effectively enhance the generalizability of BCIs based on ERP across different subjects and environments.

Transfer learning promotes acquisition of individual BCI skills

Subject training is crucial for acquiring brain-computer interface (BCI) control. Typically, this requires collecting user-specific calibration data due to high inter-subject neural variability that limits the usability of generic decoders. However, calibration is cumbersome and may produce inadequate data for building decoders, especially with naïve subjects. Here, we show that a decoder trained on the data of a single expert is readily transferrable to inexperienced users via domain adaptation techniques allowing calibration-free BCI training. We introduce two real-time frameworks, (i) Generic Recentering (GR) through unsupervised adaptation and (ii) Personally Assisted Recentering (PAR) that extends GR by employing supervised recalibration of the decoder parameters. We evaluated our frameworks on 18 healthy naïve subjects over five online sessions, who operated a customary synchronous bar task with continuous feedback and a more challenging car racing game with asynchronous control and discrete feedback. We show that along with improved task-oriented BCI performance in both tasks, our frameworks promoted subjects' ability to acquire individual BCI skills, as the initial neurophysiological control features of an expert subject evolved and became subject specific. Furthermore, those features were task-specific and were learned in parallel as participants practiced the two tasks in every session. Contrary to previous findings implying that supervised methods lead to improved online BCI control, we observed that longitudinal training coupled with unsupervised domain matching (GR) achieved similar performance to supervised recalibration (PAR). Therefore, our presented frameworks facilitate calibration-free BCIs and have immediate implications for broader populations-such as patients with neurological pathologies-who might struggle to provide suitable initial calibration data.

Beta bursts question the ruling power for brain-computer interfaces

Objective: Current efforts to build reliable brain-computer interfaces (BCI) span multiple axes from hardware, to software, to more sophisticated experimental protocols, and personalized approaches. However, despite these abundant efforts, there is still room for significant improvement. We argue that a rather overlooked direction lies in linking BCI protocols with recent advances in fundamental neuroscience.Approach: In light of these advances, and particularly the characterization of the burst-like nature of beta frequency band activity and the diversity of beta bursts, we revisit the role of beta activity in 'left vs. right hand' motor imagery (MI) tasks. Current decoding approaches for such tasks take advantage of the fact that MI generates time-locked changes in induced power in the sensorimotor cortex and rely on band-passed power changes in single or multiple channels. Although little is known about the dynamics of beta burst activity during MI, we hypothesized that beta bursts should be modulated in a way analogous to their activity during performance of real upper limb movements.Main results and Significance: We show that classification features based on patterns of beta burst modulations yield decoding results that are equivalent to or better than typically used beta power across multiple open electroencephalography datasets, thus providing insights into the specificity of these bio-markers.

MetaBCI: An open-source platform for brain-computer interfaces

BACKGROUND

Recently, brain-computer interfaces (BCIs) have attracted worldwide attention for their great potential in clinical and real-life applications. To implement a complete BCI system, one must set up several links to translate the brain intent into computer commands. However, there is not an open-source software platform that can cover all links of the BCI chain.

METHOD

This study developed a one-stop open-source BCI software, namely MetaBCI, to facilitate the construction of a BCI system. MetaBCI is written in Python, and has the functions of stimulus presentation (Brainstim), data loading and processing (Brainda), and online information flow (Brainflow). This paper introduces the detailed information of MetaBCI and presents four typical application cases.

RESULTS

The results showed that MetaBCI was an extensible and feature-rich software platform for BCI research and application, which could effectively encode, decode, and feedback brain activities.

CONCLUSIONS

MetaBCI can greatly lower the BCI's technical threshold for BCI beginners and can save time and cost to build up a practical BCI system. The source code is available at https://github.com/TBC-TJU/MetaBCI, expecting new contributions from the BCI community.

Recognition of motor intentions from EEGs of the same upper limb by signal traceability and Riemannian geometry features

Introduction

The electroencephalographic (EEG) based on the motor imagery task is derived from the physiological electrical signal caused by the autonomous activity of the brain. Its weak potential difference changes make it easy to be overwhelmed by noise, and the EEG acquisition method has a natural limitation of low spatial resolution. These have brought significant obstacles to high-precision recognition, especially the recognition of the motion intention of the same upper limb.

Methods

This research proposes a method that combines signal traceability and Riemannian geometric features to identify six motor intentions of the same upper limb, including grasping/holding of the palm, flexion/extension of the elbow, and abduction/adduction of the shoulder. First, the EEG data of electrodes irrelevant to the task were screened out by low-resolution brain electromagnetic tomography. Subsequently, tangential spatial features are extracted by the Riemannian geometry framework in the covariance matrix estimated from the reconstructed EEG signals. The learned Riemannian geometric features are used for pattern recognition by a support vector machine with a linear kernel function.

Results

The average accuracy of the six classifications on the data set of 15 participants is 22.47%, the accuracy is 19.34% without signal traceability, the accuracy is 18.07% when the features are the filter bank common spatial pattern (FBCSP), and the accuracy is 16.7% without signal traceability and characterized by FBCSP.

Discussion

The results show that the proposed method can significantly improve the accuracy of intent recognition. In addressing the issue of temporal variability in EEG data for active Brain-Machine Interfaces, our method achieved an average standard deviation of 2.98 through model transfer on different days' data.

Transfer Learning for P300 Brain-Computer Interfaces by Joint Alignment of Feature Vectors

This article presents a new transfer learning method named group learning, that jointly aligns multiple domains (many-to-many) and an extension named fast alignment that aligns any further domain to previously aligned group of domains (many-to-one). The proposed group alignment algorithm (GALIA) is evaluated on brain-computer interface (BCI) data and optimal hyper-parameter values of the algorithm are studied for classification performance and computational cost. Six publicly available P300 databases comprising 333 sessions from 177 subjects are used. As compared to the conventional subject-specific train/test pipeline, both group learning and fast alignment significantly improve the classification accuracy except for the database with clinical subjects (average improvement: 2.12±1.88%). GALIA utilizes cyclic approximate joint diagonalization (AJD) to find a set of linear transformations, one for each domain, jointly aligning the feature vectors of all domains. Group learning achieves a many-to-many transfer learning without compromising the classification performance on non-clinical BCI data. Fast alignment further extends the group learning for any unseen domains, allowing a many-to-one transfer learning with the same properties. The former method creates a single machine learning model using data from previous subjects and/or sessions, whereas the latter exploits the trained model for an unseen domain requiring no further training of the classifier.

Decoding Multi-Class Motor Imagery and Motor Execution Tasks Using Riemannian Geometry Algorithms on Large EEG Datasets

The use of Riemannian geometry decoding algorithms in classifying electroencephalography-based motor-imagery brain-computer interfaces (BCIs) trials is relatively new and promises to outperform the current state-of-the-art methods by overcoming the noise and nonstationarity of electroencephalography signals. However, the related literature shows high classification accuracy on only relatively small BCI datasets. The aim of this paper is to provide a study of the performance of a novel implementation of the Riemannian geometry decoding algorithm using large BCI datasets. In this study, we apply several Riemannian geometry decoding algorithms on a large offline dataset using four adaptation strategies: baseline, rebias, supervised, and unsupervised. Each of these adaptation strategies is applied in motor execution and motor imagery for both scenarios 64 electrodes and 29 electrodes. The dataset is composed of four-class bilateral and unilateral motor imagery and motor execution of 109 subjects. We run several classification experiments and the results show that the best classification accuracy is obtained for the scenario where the baseline minimum distance to Riemannian mean has been used. The mean accuracy values up to 81.5% for motor execution, and up to 76.4% for motor imagery. The accurate classification of EEG trials helps to realize successful BCI applications that allow effective control of devices.

Multimodal motor imagery decoding method based on temporal spatial feature alignment and fusion

Objective. A motor imagery-based brain-computer interface (MI-BCI) translates spontaneous movement intention from the brain to outside devices. Multimodal MI-BCI that uses multiple neural signals contains rich common and complementary information and is promising for enhancing the decoding accuracy of MI-BCI. However, the heterogeneity of different modalities makes the multimodal decoding task difficult. How to effectively utilize multimodal information remains to be further studied.Approach. In this study, a multimodal MI decoding neural network was proposed. Spatial feature alignment losses were designed to enhance the feature representations extracted from the heterogeneous data and guide the fusion of features from different modalities. An attention-based modality fusion module was built to align and fuse the features in the temporal dimension. To evaluate the proposed decoding method, a five-class MI electroencephalography (EEG) and functional near infrared spectroscopy (fNIRS) dataset were constructed.Main results and significance. The comparison experimental results showed that the proposed decoding method achieved higher decoding accuracy than the compared methods on both the self-collected dataset and a public dataset. The ablation results verified the effectiveness of each part of the proposed method. Feature distribution visualization results showed that the proposed losses enhance the feature representation of EEG and fNIRS modalities. The proposed method based on EEG and fNIRS modalities has significant potential for improving decoding performance of MI tasks.

Case-Based and Quantum Classification for ERP-Based Brain-Computer Interfaces

Low transfer rates are a major bottleneck for brain-computer interfaces based on electroencephalography (EEG). This problem has led to the development of more robust and accurate classifiers. In this study, we investigated the performance of variational quantum, quantum-enhanced support vector, and hypergraph case-based reasoning classifiers in the binary classification of EEG data from a P300 experiment. On the one hand, quantum classification is a promising technology to reduce computational time and improve learning outcomes. On the other hand, case-based reasoning has an excellent potential to simplify the preprocessing steps of EEG analysis. We found that the balanced training (prediction) accuracy of each of these three classifiers was 56.95 (51.83), 83.17 (50.25), and 71.10% (52.04%), respectively. In addition, case-based reasoning performed significantly lower with a simplified (49.78%) preprocessing pipeline. These results demonstrated that all classifiers were able to learn from the data and that quantum classification of EEG data was implementable; however, more research is required to enable a greater prediction accuracy because none of the classifiers were able to generalize from the data. This could be achieved by improving the configuration of the quantum classifiers (e.g., increasing the number of shots) and increasing the number of trials for hypergraph case-based reasoning classifiers through transfer learning.

A transfer learning-based feedback training motivates the performance of SMR-BCI

Objective. Feedback training is a practical approach to brain-computer interface (BCI) end-users learning to modulate their sensorimotor rhythms (SMRs). BCI self-regulation learning has been shown to be influenced by subjective psychological factors, such as motivation. However, few studies have taken into account the users' self-motivation as additional guidance for the cognitive process involved in BCI learning. In this study we tested a transfer learning (TL) feedback method designed to increase self-motivation by providing information about past performance.Approach. Electroencephalography (EEG) signals from the previous runs were affine transformed and displayed as points on the screen, along with the newly recorded EEG signals in the current run, giving the subjects a context for self-motivation. Subjects were asked to separate the feedback points for the current run under the display of the separability of prior training. We conducted a between-subject feedback training experiment, in which 24 healthy SMR-BCI naive subjects were trained to imagine left- and right-hand movements. The participants were provided with either TL feedback or typical cursor-bar (CB) feedback (control condition), for three sessions on separate days.Main results. The behavioral results showed an increased challenge and stable mastery confidence, suggesting that subjects' motivation grew as the feedback training went on. The EEG results showed favorable overall training effects with TL feedback in terms of the class distinctiveness and EEG discriminancy. Performance was 28.5% higher in the third session than in the first. About 41.7% of the subjects were 'learners' including not only low-performance subjects, but also good-performance subjects who might be affected by the ceiling effect. Subjects were able to control BCI with TL feedback with a higher performance of 60.5% during the last session compared to CB feedback.Significance. The present study demonstrated that the proposed TL feedback method boosted psychological engagement through the self-motivated context, and further allowed subjects to modulate SMR effectively. The proposed TL feedback method also provided an alternative to typical CB feedback.

Transfer learning of an ensemble of DNNs for SSVEP BCI spellers without user-specific training

Objective.Steady-state visually evoked potentials (SSVEPs), measured with electroencephalogram (EEG), yield decent information transfer rates (ITRs) in brain-computer interface (BCI) spellers. However, the current high performing SSVEP BCI spellers in the literature require an initial lengthy and tiring user-specific training for each new user for system adaptation, including data collection with EEG experiments, algorithm training and calibration (all are before the actual use of the system). This impedes the widespread use of BCIs. To ensure practicality, we propose a novel target identification method based on an ensemble of deep neural networks (DNNs), which does not require any sort of user-specific training.Approach.We exploit already-existing literature datasets from participants of previously conducted EEG experiments to train a global target identifier DNN first, which is then fine-tuned to each participant. We transfer this ensemble of fine-tuned DNNs to the new user instance, determine thekmost representative DNNs according to the participants' statistical similarities to the new user, and predict the target character through a weighted combination of the ensemble predictions.Main results.The proposed method significantly outperforms all the state-of-the-art alternatives for all stimulation durations in [0.2-1.0] s on two large-scale benchmark and BETA datasets, and achieves impressive 155.51 bits/min and 114.64 bits/min ITRs. Code is available for reproducibility:https://github.com/osmanberke/Ensemble-of-DNNs.Significance.Our Ensemble-DNN method has the potential to promote the practical widespread deployment of BCI spellers in daily lives as we provide the highest performance while enabling the immediate system use without any user-specific training.

Tangent space alignment: Transfer learning for Brain-Computer Interface

Statistical variability of electroencephalography (EEG) between subjects and between sessions is a common problem faced in the field of Brain-Computer Interface (BCI). Such variability prevents the usage of pre-trained machine learning models and requires the use of a calibration for every new session. This paper presents a new transfer learning (TL) method that deals with this variability. This method aims to reduce calibration time and even improve accuracy of BCI systems by aligning EEG data from one subject to the other in the tangent space of the positive definite matrices Riemannian manifold. We tested the method on 18 BCI databases comprising a total of 349 subjects pertaining to three BCI paradigms, namely, event related potentials (ERP), motor imagery (MI), and steady state visually evoked potentials (SSVEP). We employ a support vector classifier for feature classification. The results demonstrate a significant improvement of classification accuracy, as compared to a classical training-test pipeline, in the case of the ERP paradigm, whereas for both the MI and SSVEP paradigm no deterioration of performance is observed. A global 2.7% accuracy improvement is obtained compared to a previously published Riemannian method, Riemannian Procrustes Analysis (RPA). Interestingly, tangent space alignment has an intrinsic ability to deal with transfer learning for sets of data that have different number of channels, naturally applying to inter-dataset transfer learning.

M3CV: A multi-subject, multi-session, and multi-task database for EEG-based biometrics challenge

EEG signals exhibit commonality and variability across subjects, sessions, and tasks. But most existing EEG studies focus on mean group effects (commonality) by averaging signals over trials and subjects. The substantial intra- and inter-subject variability of EEG have often been overlooked. The recently significant technological advances in machine learning, especially deep learning, have brought technological innovations to EEG signal application in many aspects, but there are still great challenges in cross-session, cross-task, and cross-subject EEG decoding. In this work, an EEG-based biometric competition based on a large-scale M³CV (A Multi-subject, Multi-session, and Multi-task Database for investigation of EEG Commonality and Variability) database was launched to better characterize and harness the intra- and inter-subject variability and promote the development of machine learning algorithm in this field. In the M³CV database, EEG signals were recorded from 106 subjects, of which 95 subjects repeated two sessions of the experiments on different days. The whole experiment consisted of 6 paradigms, including resting-state, transient-state sensory, steady-state sensory, cognitive oddball, motor execution, and steady-state sensory with selective attention with 14 types of EEG signals, 120000 epochs. Two learning tasks (identification and verification), performance metrics, and baseline methods were introduced in the competition. In general, the proposed M³CV dataset and the EEG-based biometric competition aim to provide the opportunity to develop advanced machine learning algorithms for achieving an in-depth understanding of the commonality and variability of EEG signals across subjects, sessions, and tasks.

Data augmentation for learning predictive models on EEG: a systematic comparison

Objective.The use of deep learning for electroencephalography (EEG) classification tasks has been rapidly growing in the last years, yet its application has been limited by the relatively small size of EEG datasets. Data augmentation, which consists in artificially increasing the size of the dataset during training, can be employed to alleviate this problem. While a few augmentation transformations for EEG data have been proposed in the literature, their positive impact on performance is often evaluated on a single dataset and compared to one or two competing augmentation methods. This work proposes to better validate the existing data augmentation approaches through a unified and exhaustive analysis.Approach.We compare quantitatively 13 different augmentations with two different predictive tasks, datasets and models, using three different types of experiments.Main results.We demonstrate that employing the adequate data augmentations can bring up to 45% accuracy improvements in low data regimes compared to the same model trained without any augmentation. Our experiments also show that there is no single best augmentation strategy, as the good augmentations differ on each task.Significance.Our results highlight the best data augmentations to consider for sleep stage classification and motor imagery brain-computer interfaces. More broadly, it demonstrates that EEG classification tasks benefit from adequate data augmentation.

Multi-Task Learning-Based Deep Neural Network for Steady-State Visual Evoked Potential-Based Brain-Computer Interfaces

Amyotrophic lateral sclerosis (ALS) causes people to have difficulty communicating with others or devices. In this paper, multi-task learning with denoising and classification tasks is used to develop a robust steady-state visual evoked potential-based brain-computer interface (SSVEP-based BCI), which can help people communicate with others. To ease the operation of the input interface, a single channel-based SSVEP-based BCI is selected. To increase the practicality of SSVEP-based BCI, multi-task learning is adopted to develop the neural network-based intelligent system, which can suppress the noise components and obtain a high level of accuracy of classification. Thus, denoising and classification tasks are selected in multi-task learning. The experimental results show that the proposed multi-task learning can effectively integrate the advantages of denoising and discriminative characteristics and outperform other approaches. Therefore, multi-task learning with denoising and classification tasks is very suitable for developing an SSVEP-based BCI for practical applications. In the future, an augmentative and alternative communication interface can be implemented and examined for helping people with ALS communicate with others in their daily lives.

Supervised and Semisupervised Manifold Embedded Knowledge Transfer in Motor Imagery-Based BCI

A long calibration procedure limits the use in practice for a motor imagery (MI)-based brain-computer interface (BCI) system. To tackle this problem, we consider supervised and semisupervised transfer learning. However, it is a challenge for them to cope with high intersession/subject variability in the MI electroencephalographic (EEG) signals. Based on the framework of unsupervised manifold embedded knowledge transfer (MEKT), we propose a supervised MEKT algorithm (sMEKT) and a semisupervised MEKT algorithm (ssMEKT), respectively. sMEKT only has limited labelled samples from a target subject and abundant labelled samples from multiple source subjects. Compared to sMEKT, ssMEKT adds comparably more unlabelled samples from the target subject. After performing Riemannian alignment (RA) and tangent space mapping (TSM), both sMEKT and ssMEKT execute domain adaptation to shorten the differences among subjects. During domain adaptation, to make use of the available samples, two algorithms preserve the source domain discriminability, and ssMEKT preserves the geometric structure embedded in the labelled and unlabelled target domains. Moreover, to obtain a subject-specific classifier, sMEKT minimizes the joint probability distribution shift between the labelled target and source domains, whereas ssMEKT performs the joint probability distribution shift minimization between the unlabelled target domain and all labelled domains. Experimental results on two publicly available MI datasets demonstrate that our algorithms outperform the six competing algorithms, where the sizes of labelled and unlabelled target domains are variable. Especially for the target subjects with 10 labelled samples and 270/190 unlabelled samples, ssMEKT shows 5.27% and 2.69% increase in average accuracy on the two abovementioned datasets compared to the previous best semisupervised transfer learning algorithm (RA-regularized common spatial patterns-weighted adaptation regularization, RA-RCSP-wAR), respectively. Therefore, our algorithms can effectively reduce the need of labelled samples for the target subject, which is of importance for the MI-based BCI application.

SPD-CNN: A plain CNN-based model using the symmetric positive definite matrices for cross-subject EEG classification with meta-transfer-learning

The electroencephalography (EEG) signals are easily contaminated by various artifacts and noise, which induces a domain shift in each subject and significant pattern variability among different subjects. Therefore, it hinders the improvement of EEG classification accuracy in the cross-subject learning scenario. Convolutional neural networks (CNNs) have been extensively applied to EEG-based Brain-Computer Interfaces (BCIs) by virtue of the capability of performing automatic feature extraction and classification. However, they have been mainly applied to the within-subject classification which would consume lots of time for training and calibration. Thus, it limits the further applications of CNNs in BCIs. In order to build a robust classification algorithm for a calibration-less BCI system, we propose an end-to-end model that transforms the EEG signals into symmetric positive definite (SPD) matrices and captures the features of SPD matrices by using a CNN. To avoid the time-consuming calibration and ensure the application of the proposed model, we use the meta-transfer-learning (MTL) method to learn the essential features from different subjects. We validate our model by making extensive experiments on three public motor-imagery datasets. The experimental results demonstrate the effectiveness of our proposed method in the cross-subject learning scenario.

A reusable benchmark of brain-age prediction from M/EEG resting-state signals

Population-level modeling can define quantitative measures of individual aging by applying machine learning to large volumes of brain images. These measures of brain age, obtained from the general population, helped characterize disease severity in neurological populations, improving estimates of diagnosis or prognosis. Magnetoencephalography (MEG) and Electroencephalography (EEG) have the potential to further generalize this approach towards prevention and public health by enabling assessments of brain health at large scales in socioeconomically diverse environments. However, more research is needed to define methods that can handle the complexity and diversity of M/EEG signals across diverse real-world contexts. To catalyse this effort, here we propose reusable benchmarks of competing machine learning approaches for brain age modeling. We benchmarked popular classical machine learning pipelines and deep learning architectures previously used for pathology decoding or brain age estimation in 4 international M/EEG cohorts from diverse countries and cultural contexts, including recordings from more than 2500 participants. Our benchmarks were built on top of the M/EEG adaptations of the BIDS standard, providing tools that can be applied with minimal modification on any M/EEG dataset provided in the BIDS format. Our results suggest that, regardless of whether classical machine learning or deep learning was used, the highest performance was reached by pipelines and architectures involving spatially aware representations of the M/EEG signals, leading to R^2 scores between 0.60-0.71. Hand-crafted features paired with random forest regression provided robust benchmarks even in situations in which other approaches failed. Taken together, this set of benchmarks, accompanied by open-source software and high-level Python scripts, can serve as a starting point and quantitative reference for future efforts at developing M/EEG-based measures of brain aging. The generality of the approach renders this benchmark reusable for other related objectives such as modeling specific cognitive variables or clinical endpoints.

A meta-learning BCI for estimating decision confidence

OBJECTIVE

We investigated whether a recently introduced transfer-learning technique based on meta-learning could improve the performance of Brain-Computer Interfaces (BCIs) for decision-confidence prediction with respect to more traditional machine learning methods.

APPROACH

We adapted the meta-learning by biased regularisation algorithm to the problem of predicting decision confidence from EEG and EOG data on a decision-by-decision basis in a difficult target discrimination task based on video feeds. The method exploits previous participants' data to produce a prediction algorithm that is then quickly tuned to new participants. We compared it with with the traditional single-subject training almost universally adopted in BCIs, a state-of-the-art transfer learning technique called Domain Adversarial Neural Networks (DANN), a transfer-learning adaptation of a zero-training method we used recently for a similar task, and with a simple baseline algorithm.

MAIN RESULTS

The meta-learning approach was significantly better than other approaches in most conditions, and much better in situations where limited data from a new participant are available for training/tuning. Meta-learning by biased regularisation allowed our BCI to seamlessly integrate information from past participants with data from a specific user to produce high-performance predictors. Its robustness in the presence of small training sets is a real-plus in BCI applications, as new users need to train the BCI for a much shorter period.

SIGNIFICANCE

Due to the variability and noise of EEG/EOG data, BCIs need to be normally trained with data from a specific participant. This work shows that even better performance can be obtained using our version of meta-learning by biased regularisation.

Transfer learning for motor imagery based brain-computer interfaces: A tutorial

A brain-computer interface (BCI) enables a user to communicate directly with an external device, e.g., a computer, using brain signals. It can be used to research, map, assist, augment, or repair human cognitive or sensory-motor functions. A closed-loop BCI system performs signal acquisition, temporal filtering, spatial filtering, feature engineering and classification, before sending out the control signal to an external device. Transfer learning (TL) has been widely used in motor imagery (MI) based BCIs to reduce the calibration effort for a new subject, greatly increasing their utility. This tutorial describes how TL can be considered in as many components of a BCI system as possible, and introduces a complete TL pipeline for MI-based BCIs. Examples on two MI datasets demonstrated the advantages of considering TL in multiple components of MI-based BCIs. Especially, integrating data alignment and sophisticated TL approaches can significantly improve the classification performance, and hence greatly reduces the calibration effort.

Riemannian geometry-based transfer learning for reducing training time in c-VEP BCIs

One of the main problems that a brain-computer interface (BCI) face is that a training stage is required for acquiring training data to calibrate its classification model just before every use. Transfer learning is a promising method for addressing the problem. In this paper, we propose a Riemannian geometry-based transfer learning algorithm for code modulated visual evoked potential (c-VEP)-based BCIs, which can effectively reduce the calibration time without sacrificing the classification accuracy. The algorithm includes the main procedures of log-Euclidean data alignment (LEDA), super-trial construction, covariance matrix estimation, training accuracy-based subject selection (TSS) and minimum distance to mean classification. Among them, the LEDA reduces the difference in data distribution between subjects, whereas the TSS promotes the similarity between a target subject and the source subjects. The resulting performance of transfer learning is improved significantly. Sixteen subjects participated in a c-VEP BCI experiment and the recorded data were used in offline analysis. Leave-one subject-out (LOSO) cross-validation was used to evaluate the proposed algorithm on the data set. The results showed that the algorithm achieved much higher classification accuracy than the subject-specific (baseline) algorithm with the same number of training trials. Equivalently, the algorithm reduces the training time of the BCI at the same performance level and thus facilitates its application in real world.

Not All Electrode Channels Are Needed: Knowledge Transfer From Only Stimulated Brain Regions for EEG Emotion Recognition

Emotion recognition from affective brain-computer interfaces (aBCI) has garnered a lot of attention in human-computer interactions. Electroencephalographic (EEG) signals collected and stored in one database have been mostly used due to their ability to detect brain activities in real time and their reliability. Nevertheless, large EEG individual differences occur amongst subjects making it impossible for models to share information across. New labeled data is collected and trained separately for new subjects which costs a lot of time. Also, during EEG data collection across databases, different stimulation is introduced to subjects. Audio-visual stimulation (AVS) is commonly used in studying the emotional responses of subjects. In this article, we propose a brain region aware domain adaptation (BRADA) algorithm to treat features from auditory and visual brain regions differently, which effectively tackle subject-to-subject variations and mitigate distribution mismatch across databases. BRADA is a new framework that works with the existing transfer learning method. We apply BRADA to both cross-subject and cross-database settings. The experimental results indicate that our proposed transfer learning method can improve valence-arousal emotion recognition tasks.

Motor Imagery Classification via Kernel-Based Domain Adaptation on an SPD Manifold

Background: Recording the calibration data of a brain–computer interface is a laborious process and is an unpleasant experience for the subjects. Domain adaptation is an effective technology to remedy the shortage of target data by leveraging rich labeled data from the sources. However, most prior methods have needed to extract the features of the EEG signal first, which triggers another challenge in BCI classification, due to small sample sets or a lack of labels for the target. Methods: In this paper, we propose a novel domain adaptation framework, referred to as kernel-based Riemannian manifold domain adaptation (KMDA). KMDA circumvents the tedious feature extraction process by analyzing the covariance matrices of electroencephalogram (EEG) signals. Covariance matrices define a symmetric positive definite space (SPD) that can be described by Riemannian metrics. In KMDA, the covariance matrices are aligned in the Riemannian manifold, and then are mapped to a high dimensional space by a log-Euclidean metric Gaussian kernel, where subspace learning is performed by minimizing the conditional distribution distance between the sources and the target while preserving the target discriminative information. We also present an approach to convert the EEG trials into 2D frames (E-frames) to further lower the dimension of covariance descriptors. Results: Experiments on three EEG datasets demonstrated that KMDA outperforms several state-of-the-art domain adaptation methods in classification accuracy, with an average Kappa of 0.56 for BCI competition IV dataset IIa, 0.75 for BCI competition IV dataset IIIa, and an average accuracy of 81.56% for BCI competition III dataset IVa. Additionally, the overall accuracy was further improved by 5.28% with the E-frames. KMDA showed potential in addressing subject dependence and shortening the calibration time of motor imagery-based brain–computer interfaces.

Increasing Robustness of Brain-Computer Interfaces Through Automatic Detection and Removal of Corrupted Input Signals

For brain–computer interfaces (BCIs) to be viable for long-term daily usage, they must be able to quickly identify and adapt to signal disruptions. Furthermore, the detection and mitigation steps need to occur automatically and without the need for user intervention while also being computationally tractable for the low-power hardware that will be used in a deployed BCI system. Here, we focus on disruptions that are likely to occur during chronic use that cause some recording channels to fail but leave the remaining channels unaffected. In these cases, the algorithm that translates recorded neural activity into actions, the neural decoder, should seamlessly identify and adjust to the altered neural signals with minimal inconvenience to the user. First, we introduce an adapted statistical process control (SPC) method that automatically identifies disrupted channels so that both decoding algorithms can be adjusted, and technicians can be alerted. Next, after identifying corrupted channels, we demonstrate the automated and rapid removal of channels from a neural network decoder using a masking approach that does not change the decoding architecture, making it amenable for transfer learning. Finally, using transfer and unsupervised learning techniques, we update the model weights to adjust for the corrupted channels without requiring the user to collect additional calibration data. We demonstrate with both real and simulated neural data that our approach can maintain high-performance while simultaneously minimizing computation time and data storage requirements. This framework is invisible to the user but can dramatically increase BCI robustness and usability.

Riemannian channel selection for BCI with between-session non-stationarity reduction capabilities

OBJECTIVE

Between-session non-stationarity is a major challenge of current Brain-Computer Interfaces (BCIs) that affects system performance. In this paper, we investigate the use of channel selection for reducing between-session non-stationarity with Riemannian BCI classifiers. We use the Riemannian geometry framework of covariance matrices due to its robustness and promising performances. Current Riemannian channel selection methods do not consider between-session non-stationarity and are usually tested on a single session. Here, we propose a new channel selection approach that specifically considers non-stationarity effects and is assessed on multi-session BCI data sets.

METHODS

We remove the least significant channels using a sequential floating backward selection search strategy. Our contributions include: 1) quantifying the non-stationarity effects on brain activity in multi-class problems by different criteria in a Riemannian framework and 2) a method to predict whether BCI performance can improve using channel selection.

RESULTS

We evaluate the proposed approaches on three multi-session and multi-class mental tasks (MT)-based BCI datasets. They could lead to significant improvements in performance as compared to using all channels for datasets affected by between-session non-stationarity and to significant superiority to the state-of-the-art Riemannian channel selection methods over all datasets, notably when selecting small channel set sizes.

CONCLUSION

Reducing non-stationarity by channel selection could significantly improve Riemannian BCI classification accuracy.

SIGNIFICANCE

Our proposed channel selection approach contributes to make Riemannian BCI classifiers more robust to between-session non-stationarities.

Workshops of the Eighth International Brain-Computer Interface Meeting: BCIs: The Next Frontier

The Eighth International Brain-Computer Interface (BCI) Meeting was held June 7-9th, 2021 in a virtual format. The conference continued the BCI Meeting series' interactive nature with 21 workshops covering topics in BCI (also called brain-machine interface) research. As in the past, workshops covered the breadth of topics in BCI. Some workshops provided detailed examinations of specific methods, hardware, or processes. Others focused on specific BCI applications or user groups. Several workshops continued consensus building efforts designed to create BCI standards and increase the ease of comparisons between studies and the potential for meta-analysis and large multi-site clinical trials. Ethical and translational considerations were both the primary topic for some workshops or an important secondary consideration for others. The range of BCI applications continues to expand, with more workshops focusing on approaches that can extend beyond the needs of those with physical impairments. This paper summarizes each workshop, provides background information and references for further study, presents an overview of the discussion topics, and describes the conclusion, challenges, or initiatives that resulted from the interactions and discussion at the workshop.

Selective Cross-Subject Transfer Learning Based on Riemannian Tangent Space for Motor Imagery Brain-Computer Interface

A motor imagery (MI) brain-computer interface (BCI) plays an important role in the neurological rehabilitation training for stroke patients. Electroencephalogram (EEG)-based MI BCI has high temporal resolution, which is convenient for real-time BCI control. Therefore, we focus on EEG-based MI BCI in this paper. The identification of MI EEG signals is always quite challenging. Due to high inter-session/subject variability, each subject should spend long and tedious calibration time in collecting amounts of labeled samples for a subject-specific model. To cope with this problem, we present a supervised selective cross-subject transfer learning (sSCSTL) approach which simultaneously makes use of the labeled samples from target and source subjects based on Riemannian tangent space. Since the covariance matrices representing the multi-channel EEG signals belong to the smooth Riemannian manifold, we perform the Riemannian alignment to make the covariance matrices from different subjects close to each other. Then, all aligned covariance matrices are converted into the Riemannian tangent space features to train a classifier in the Euclidean space. To investigate the role of unlabeled samples, we further propose semi-supervised and unsupervised versions which utilize the total samples and unlabeled samples from target subject, respectively. Sequential forward floating search (SFFS) method is executed for source selection. All our proposed algorithms transfer the labeled samples from most suitable source subjects into the feature space of target subject. Experimental results on two publicly available MI datasets demonstrated that our algorithms outperformed several state-of-the-art algorithms using small number of the labeled samples from target subject, especially for good target subjects.

A Tensor-Based Frequency Features Combination Method for Brain-Computer Interfaces

With the development of the brain-computer interface (BCI) community, motor imagery-based BCI system using electroencephalogram (EEG) has attracted increasing attention because of its portability and low cost. Concerning the multi-channel EEG, the frequency component is one of the most critical features. However, insufficient extraction hinders the development and application of MI-BCIs. To deeply mine the frequency information, we proposed a method called tensor-based frequency feature combination (TFFC). It combined tensor-to-vector projection (TVP), fast fourier transform (FFT), common spatial pattern (CSP) and feature fusion to construct a new feature set. With two datasets, we used different classifiers to compare TFFC with the state-of-the-art feature extraction methods. The experimental results showed that our proposed TFFC could robustly improve the classification accuracy of about 5% (p < 0.01). Moreover, visualization analysis implied that the TFFC was a generalization of CSP and Filter Bank CSP (FBCSP). Also, a complementarity between weighted narrowband features (wNBFs) and broadband features (BBFs) was observed from the averaged fusion ratio. This article certificates the importance of frequency information in the MI-BCI system and provides a new direction for designing a feature set of MI-EEG.

Improving Transfer Performance of Deep Learning with Adaptive Batch Normalization for Brain-computer Interfaces. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021;2021:5800-5803. [PMID: 34892438 DOI: 10.1109/embc46164.2021.9629529] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Recently, transfer learning and deep learning have been introduced to solve intra- and inter-subject variability problems in Brain-Computer Interfaces. However, the generalization ability of these BCIs is still to be further verified in a cross-dataset scenario. This study compared the transfer performance of manifold embedded knowledge transfer and pre-trained EEGNet with three preprocessing strategies. This study also introduced AdaBN for target domain adaptation. The results showed that EEGNet with Riemannian alignment and AdaBN could achieve the best transfer accuracy about 65.6% on the target dataset. This study may provide new insights into the design of transfer neural networks for BCIs by separating source and target batch normalization layers in the domain adaptation process.

A Review on Signal Processing Approaches to Reduce Calibration Time in EEG-Based Brain-Computer Interface

In an electroencephalogram- (EEG-) based brain–computer interface (BCI), a subject can directly communicate with an electronic device using his EEG signals in a safe and convenient way. However, the sensitivity to noise/artifact and the non-stationarity of EEG signals result in high inter-subject/session variability. Therefore, each subject usually spends long and tedious calibration time in building a subject-specific classifier. To solve this problem, we review existing signal processing approaches, including transfer learning (TL), semi-supervised learning (SSL), and a combination of TL and SSL. Cross-subject TL can transfer amounts of labeled samples from different source subjects for the target subject. Moreover, Cross-session/task/device TL can reduce the calibration time of the subject for the target session, task, or device by importing the labeled samples from the source sessions, tasks, or devices. SSL simultaneously utilizes the labeled and unlabeled samples from the target subject. The combination of TL and SSL can take advantage of each other. For each kind of signal processing approaches, we introduce their concepts and representative methods. The experimental results show that TL, SSL, and their combination can obtain good classification performance by effectively utilizing the samples available. In the end, we draw a conclusion and point to research directions in the future.

Classification of motor imagery using multisource joint transfer learning

As an important way for human-computer interaction, the motor imagery brain-computer interface (MI-BCI) can decode personal motor intention directly by analyzing electroencephalogram (EEG) signals. However, a large amount of labeled data has to be collected for each new subject since EEG patterns vary between individuals. The long calibration phase severely limits the further development of MI-BCI. To tackle this problem, multi-source joint domain adaption (MJDA) and multi-source joint Riemannian adaption (MJRA) algorithms are proposed in this paper. Both methods aim to transfer knowledge from other subjects to the current subject who has only a small amount of labeled data. First, the common spatial pattern with Euclidean alignment is used to select source subjects who have similar spatial patterns to the target subject. Second, the covariance matrices of EEG trials are aligned in Riemannian space by removing subject-specific baselines. These two steps are shared by MJDA and MJRA. In the last step, MJDA attempts to minimize the feature distribution mismatch in the Riemannian tangent space, while MJRA attempts to find an adaptive Riemannian classifier. Finally, the proposed methods are validated on two datasets: BCI Competition IV 2a and online event-related desynchronization (ERD)-BCI. The experimental results demonstrate that both MJDA and MJRA outperform the state-of-the-art approaches. The MJDA provides a new idea for the offline analysis of MI-BCI, while MJRA could make a big difference to the online calibration of MI-BCI.

Enhancing transfer performance across datasets for brain-computer interfaces using a combination of alignment strategies and adaptive batch normalization

Objective. Recently, transfer learning (TL) and deep learning (DL) have been introduced to solve intra- and inter-subject variability problems in brain-computer interfaces (BCIs). However, current TL and DL algorithms are usually validated within a single dataset, assuming that data of the test subjects are acquired under the same condition as that of training (source) subjects. This assumption is generally violated in practice because of different acquisition systems and experimental settings across studies and datasets. Thus, the generalization ability of these algorithms needs further validations in a cross-dataset scenario, which is closer to the actual situation. This study compared the transfer performance of pre-trained deep-learning models with different preprocessing strategies in a cross-dataset scenario.Approach. This study used four publicly available motor imagery datasets, each was successively selected as a source dataset, and the others were used as target datasets. EEGNet and ShallowConvNet with four preprocessing strategies, namely channel normalization, trial normalization, Euclidean alignment, and Riemannian alignment, were trained with the source dataset. The transfer performance of pre-trained models was validated on the target datasets. This study also used adaptive batch normalization (AdaBN) for reducing interval covariate shift across datasets. This study compared the transfer performance of using the four preprocessing strategies and that of a baseline approach based on manifold embedded knowledge transfer (MEKT). This study also explored the possibility and performance of fusing MEKT and EEGNet.Main results. The results show that DL models with alignment strategies had significantly better transfer performance than the other two preprocessing strategies. As an unsupervised domain adaptation method, AdaBN could also significantly improve the transfer performance of DL models. The transfer performance of DL models that combined AdaBN and alignment strategies significantly outperformed MEKT. Moreover, the generalizability of EEGNet models that combined AdaBN and alignment strategies could be further improved via the domain adaptation step in MEKT, achieving the best generalization ability among multiple datasets (BNCI2014001: 0.788, PhysionetMI: 0.679, Weibo2014: 0.753, Cho2017: 0.650).Significance. The combination of alignment strategies and AdaBN could easily improve the generalizability of DL models without fine-tuning. This study may provide new insights into the design of transfer neural networks for BCIs by separating source and target batch normalization layers in the domain adaptation process.

Transfer Learning based on Optimal Transport for Motor Imagery Brain-Computer Interfaces

OBJECTIVE

This paper tackles the cross-sessions variability of electroencephalography-based brain-computer interfaces (BCIs) in order to avoid the lengthy recalibration step of the decoding method before every use.

METHODS

We develop a new approach of domain adaptation based on optimal transport to tackle brain signal variability between sessions of motor imagery BCIs. We propose a backward method where, unlike the original formulation, the data from a new session are transported to a calibration session, and thereby avoiding model retraining. Several domain adaptation approaches are evaluated and compared. We simulated two possible online scenarios: i) block-wise adaptation and ii) sample-wise adaptation. In this study, we collect a dataset of 10 subjects performing a hand motor imagery task in 2 sessions. A publicly available dataset is also used.

RESULTS

For the first scenario, results indicate that classifier retraining can be avoided by means of our backward formulation yielding to equivalent classification performance as compared to retraining solutions. In the second scenario, classification performance rises up to 90.23\% overall accuracy when the label of the indicated mental task is used to learn the transport. Adaptive time is between 10 and 80 times faster than the other methods.

CONCLUSIONS

The proposed method is able to mitigate the cross-session variability in motor imagery BCIs.

SIGNIFICANCE

The backward formulation is an efficient retraining-free approach built to avoid lengthy calibration times. Thus, the BCI can be actively used after just a few minutes of setup. This is important for practical applications such as BCI-based motor rehabilitation.

CNN based classification of motor imaginary using variational mode decomposed EEG-spectrum image

A novel approach of preprocessing EEG signals by generating spectrum image for effective Convolutional Neural Network (CNN) based classification for Motor Imaginary (MI) recognition is proposed. The approach involves extracting the Variational Mode Decomposition (VMD) modes of EEG signals, from which the Short Time Fourier Transform (STFT) of all the modes are arranged to form EEG spectrum images. The EEG spectrum images generated are provided as input image to CNN. The two generic CNN architectures for MI classification (EEGNet and DeepConvNet) and the architectures for pattern recognition (AlexNet and LeNet) are used in this study. Among the four architectures, EEGNet provides average accuracies of 91.37%, 94.41%, 85.67% and 90.21% for the four datasets used to validate the proposed approach. Consistently better results in comparison with results in recent literature demonstrate that the EEG spectrum image generation using VMD-STFT is a promising method for the time frequency analysis of EEG signals.

Review of brain encoding and decoding mechanisms for EEG-based brain-computer interface

A brain-computer interface (BCI) can connect humans and machines directly and has achieved successful applications in the past few decades. Many new BCI paradigms and algorithms have been developed in recent years. Therefore, it is necessary to review new progress in BCIs. This paper summarizes progress for EEG-based BCIs from the perspective of encoding paradigms and decoding algorithms, which are two key elements of BCI systems. Encoding paradigms are grouped by their underlying neural meachanisms, namely sensory- and motor-related, vision-related, cognition-related and hybrid paradigms. Decoding algorithms are reviewed in four categories, namely decomposition algorithms, Riemannian geometry, deep learning and transfer learning. This review will provide a comprehensive overview of both modern primary paradigms and algorithms, making it helpful for those who are developing BCI systems.

Sub-band target alignment common spatial pattern in brain-computer interface

BACKGROUND AND OBJECTIVE

In the brain computer interface (BCI) field, using sub-band common spatial pattern (SBCSP) and filter bank common spatial pattern (FBCSP) can improve the accuracy of classification by selection a specific frequency band. However, in the cross-subject classification, due to the individual differences between different subjects, the performance is limited.

METHODS

This paper introduces the idea of transfer learning and presents the sub-band target alignment common spatial pattern (SBTACSP) method and applies it to the cross-subject classification of motor imagery (MI) EEG signals. First, the EEG signals are bandpass-filtered into multiple frequency bands (sub-band filtering). Subsequently, the source domain trails are aligned into the target domain space in each frequency band. The CSP algorithm is then employed to extract features among which more representative features are selected by the minimum redundancy maximum relevance (mRMR) approach from each sub-band. Then the features of all sub-bands are fused. Finally, conventional linear discriminant analysis (LDA) algorithm is used for MI classification.

RESULTS

Our method is evaluated on Datasets Ⅱa and Ⅱb of the BCI Competition Ⅳ. Compared with six state-of-the-art algorithms, the proposed SBTACSP method performed relatively the best and achieved a mean classification accuracy of 75.15% and 66.85% in cross-subject classification of Datasets Ⅱa and Ⅱb respectively.

CONCLUSION

Therefore, the combination of sub-band filtering and transfer learning achieves superior classification performance compared to either one. The proposed algorithms will greatly promote the practical application of MI based BCIs.

Low-Dimensional Subject Representation-Based Transfer Learning in EEG Decoding

Recently, the advances in passive brain-computer interfaces (BCIs) based on electroencephalogram (EEG) have shed light on real-world neuromonitoring technologies. However, human variability in the EEG activities hinders the development of practical applications of EEG-based BCI. To tackle this problem, many transfer-learning techniques perform supervised calibration. This kind of calibration approach requires task-relevant data, which is impractical in real-life scenarios such as drowsiness during driving. This study presents a transfer-learning framework for EEG decoding based on the low-dimensional representations of subjects learned from the pre-trial EEG. Tensor decomposition was applied to the pre-trial EEG of subjects to extract the underlying characteristics in subject, spatial, and spectral domains. Then, the proposed framework assessed the characteristics to obtain the low-dimensional subject representations such that the subjects with similar brain dynamics can be identified. This method can leverage the existing data from other users, and a small number of data from a rapid, non-task, unsupervised calibration from a new user to build an accurate BCI. Our results demonstrated that, in terms of prediction accuracy, the proposed low-dimensional subject representation-based transfer learning (LDSR-TL) framework outperformed the random selection, and the Riemannian manifold approach in cognitive-state tracking, while requiring fewer training data. The results can greatly improve the practicability, and usability of EEG-based BCI in the real world.

Learning Invariant Patterns Based on a Convolutional Neural Network and Big Electroencephalography Data for Subject-Independent P300 Brain-Computer Interfaces

A brain-computer interface (BCI) measures and analyzes brain activity and converts this activity into computer commands to control external devices. In contrast to traditional BCIs that require a subject-specific calibration process before being operated, a subject-independent BCI learns a subject-independent model and eliminates subject-specific calibration for new users. However, building subject-independent BCIs remains difficult because electroencephalography (EEG) is highly noisy and varies by subject. In this study, we propose an invariant pattern learning method based on a convolutional neural network (CNN) and big EEG data for subject-independent P300 BCIs. The CNN was trained using EEG data from a large number of subjects, allowing it to extract subject-independent features and make predictions for new users. We collected EEG data from 200 subjects in a P300-based spelling task using two different types of amplifiers. The offline analysis showed that almost all subjects obtained significant cross-subject and cross-amplifier effects, with an average accuracy of more than 80%. Furthermore, more than half of the subjects achieved accuracies above 85%. These results indicated that our method was effective for building a subject-independent P300 BCI, with which more than 50% of users could achieve high accuracies without subject-specific calibration.