Transfer Learning based on Optimal Transport for Motor Imagery Brain-Computer Interfaces

Avoidance of specific calibration sessions in motor intention recognition for exoskeleton-supported rehabilitation through transfer learning on EEG data

Exoskeleton-based support for patients requires the learning of individual machine-learning models to recognize movement intentions of patients based on the electroencephalogram (EEG). A major issue in EEG-based movement intention recognition is the long calibration time required to train a model. In this paper, we propose a transfer learning approach that eliminates the need for a calibration session. This approach is validated on healthy subjects in this study. We will use the proposed approach in our future rehabilitation application, where the movement intention of the affected arm of a patient can be inferred from the EEG data recorded during bilateral arm movements enabled by the exoskeleton mirroring arm movements from the unaffected to the affected arm. For the initial evaluation, we compared two trained models for predicting unilateral and bilateral movement intentions without applying a classifier transfer. For the main evaluation, we predicted unilateral movement intentions without a calibration session by transferring the classifier trained on data from bilateral movement intentions. Our results showed that the classification performance for the transfer case was comparable to that in the non-transfer case, even with only 4 or 8 EEG channels. Our results contribute to robotic rehabilitation by eliminating the need for a calibration session, since EEG data for training is recorded during the rehabilitation session, and only a small number of EEG channels are required for model training.

Three-stage transfer learning for motor imagery EEG recognition

Motor imagery (MI) paradigms have been widely used in neural rehabilitation and drowsiness state assessment. The progress in brain-computer interface (BCI) technology has emphasized the importance of accurately and efficiently detecting motor imagery intentions from electroencephalogram (EEG). Despite the recent breakthroughs made in developing EEG-based algorithms for decoding MI, the accuracy and efficiency of these models remain limited by technical challenges posed by cross-subject heterogeneity in EEG data processing and the scarcity of EEG data for training. Inspired by the optimal transport theory, this study aims to develop a novel three-stage transfer learning (TSTL) method, which uses the existing labeled data from a source domain to improve classification performance on an unlabeled target domain. Notably, the proposed method comprises three components, namely, the Riemannian tangent space mapping (RTSM), source domain transformer (SDT), and optimal subspace mapping (OSM). The RTSM maps a symmetric positive definite matrix from the Riemannian space to the tangent space to minimize the marginal probability distribution drift. The SDT transforms the source domain to a target domain by finding the optimal transport mapping matrix to reduce the joint probability distribution differences. The OSM finally maps the transformed source domain and original target domain to the same subspace to further mitigate the distribution discrepancy. The performance of the proposed method was validated on two public BCI datasets, and the average accuracy of the algorithm on two datasets was 72.24% and 69.29%. Our results demonstrated the improved performance of EEG-based MI detection in comparison with state-of-the-art algorithms.

Frequency Domain Channel-Wise Attack to CNN Classifiers in Motor Imagery Brain-Computer Interfaces

OBJECTIVE

Convolutional neural network (CNN), a classical structure in deep learning, has been commonly deployed in the motor imagery brain-computer interface (MIBCI). Many methods have been proposed to evaluate the vulnerability of such CNN models, primarily by attacking them using direct temporal perturbations. In this work, we propose a novel attacking approach based on perturbations in the frequency domain instead.

METHODS

For a given natural MI trial in the frequency domain, the proposed approach, called frequency domain channel-wise attack (FDCA), generates perturbations at each channel one after another to fool the CNN classifiers. The advances of this strategy are two-fold. First, instead of focusing on the temporal domain, perturbations are generated in the frequency domain where discriminative patterns can be extracted for motor imagery (MI) classification tasks. Second, the perturbing optimization is performed based on differential evolution algorithm in a black-box scenario where detailed model knowledge is not required.

RESULTS

Experimental results demonstrate the effectiveness of the proposed FDCA which achieves a significantly higher success rate than the baselines and existing methods in attacking three major CNN classifiers on four public MI benchmarks.

CONCLUSION

Perturbations generated in the frequency domain yield highly competitive results in attacking MIBCI deployed by CNN models even in a black-box setting, where the model information is well-protected.

SIGNIFICANCE

To our best knowledge, existing MIBCI attack approaches are all gradient-based methods and require details about the victim model, e.g., the parameters and objective function. We provide a more flexible strategy that does not require model details but still produces an effective attack outcome.

Transfer learning with data alignment and optimal transport for EEG based motor imagery classification

Objective. The non-stationarity of electroencephalogram (EEG) signals and the variability among different subjects present significant challenges in current Brain-Computer Interfaces (BCI) research, which requires a time-consuming specific calibration procedure to address. Transfer Learning (TL) offers a potential solution by leveraging data or models from one or more source domains to facilitate learning in the target domain, so as to address these challenges.Approach. In this paper, a novel Multi-source domain Transfer Learning Fusion (MTLF) framework is proposed to address the calibration problem. Firstly, the method transforms the source domain data with the resting state segment data, in order to decrease the differences between the source domain and the target domain. Subsequently, feature extraction is performed using common spatial pattern. Finally, an improved TL classifier is employed to classify the target samples. Notably, this method does not require the label information of target domain samples, while concurrently reducing the calibration workload.Main results. The proposed MTLF is assessed on Datasets 2a and 2b from the BCI Competition IV. Compared with other algorithms, our method performed relatively the best and achieved mean classification accuracy of 73.69% and 70.83% on Datasets 2a and 2b respectively.Significance.Experimental results demonstrate that the MTLF framework effectively reduces the discrepancy between the source and target domains and acquires better classification performance on two motor imagery datasets.

Weighted Domain Adaptation Using the Graph-Structured Dataset Representation for Machinery Fault Diagnosis under Varying Operating Conditions

Data-driven fault diagnosis has received significant attention in the era of big data. Most data-driven methods have been developed under the assumption that both training and test data come from identical data distributions. However, in real-world industrial scenarios, data distribution often changes due to varying operating conditions, leading to a degradation of diagnostic performance. Although several domain adaptation methods have shown their feasibility, existing methods have overlooked metadata from the manufacturing process and treated all domains uniformly. To address these limitations, this article proposes a weighted domain adaptation method using a graph-structured dataset representation. Our framework involves encoding a collection of datasets into the proposed graph structure, which captures relations between datasets based on metadata and raw data simultaneously. Then, transferability scores of candidate source datasets for a target are estimated using the constructed graph and a graph embedding model. Finally, the fault diagnosis model is established with a voting ensemble of the base classifiers trained on candidate source datasets and their estimated transferability scores. For validation, two case studies on rotor machinery, specifically tool wear and bearing fault detection, were conducted. The experimental results demonstrate the effectiveness and superiority of the proposed method over other existing domain adaptation methods.

A novel heterogeneous transfer learning method based on data stitching for the sequential coding brain computer interface

OBJECTIVE

For the brain computer interface (BCI), it is necessary to collect enough electroencephalography (EEG) signals to train the classification model. When the operation dimension of BCI is large, it will bring great burden to data acquisition. Fortunately, this problem can be solved by our proposed transfer learning method.

METHOD

For the sequential coding experimental paradigm, the multi-band data stitching with label alignment and tangent space mapping (MDSLATSM) algorithm is proposed as a novel heterogeneous transfer learning method. After filtering by multi-band filtering, the artificial signals can be obtained by data stitching from the source domain, which build a bridge between the source domain and target domain. To make the distribution of two domains closer, their covariance matrices are aligned by label alignment. After mapping to the tangent space, the features are extracted from the Riemannian manifold. Finally, the classification results are obtained with feature selection and classification.

RESULTS

Our data set includes the EEG signals from 16 subjects. For the heterogeneous transfer learning of cross-label, the average classification accuracy is 78.28%. MDSLATSM is also tested for cross-subject, and the average classification accuracy is 64.01%, which is better than existing methods.

SIGNIFICANCE

Combining multi-band filtering, data stitching, label alignment and tangent space mapping, a novel heterogeneous transfer learning method can be achieved with superior performance, which promotes the practical application of the BCI systems.

Dual attentive fusion for EEG-based brain-computer interfaces

The classification based on Electroencephalogram (EEG) is a challenging task in the brain-computer interface (BCI) field due to data with a low signal-to-noise ratio. Most current deep learning based studies in this challenge focus on designing a desired convolutional neural network (CNN) to learn and classify the raw EEG signals. However, only CNN itself may not capture the highly discriminative patterns of EEG due to a lack of exploration of attentive spatial and temporal dynamics. To improve information utilization, this study proposes a Dual Attentive Fusion Model (DAFM) for the EEG-based BCI. DAFM is employed to capture the spatial and temporal information by modeling the interdependencies between the features from the EEG signals. To our best knowledge, our method is the first to fuse spatial and temporal dimensions in an interactive attention module. This module improves the expression ability of the extracted features. Extensive experiments implemented on four publicly available datasets demonstrate that our method outperforms state-of-the-art methods. Meanwhile, this work also indicates the effectiveness of Dual Attentive Fusion Module.

A large EEG dataset for studying cross-session variability in motor imagery brain-computer interface

In building a practical and robust brain-computer interface (BCI), the classification of motor imagery (MI) from electroencephalography (EEG) across multiple days is a long-standing challenge due to the large variability of the EEG signals. We collected a large dataset of MI from 5 different days with 25 subjects, the first open-access dataset to address BCI issues across 5 different days with a large number of subjects. The dataset includes 5 session data from 5 different days (2–3 days apart) for each subject. Each session contains 100 trials of left-hand and right-hand MI. In this report, we provide the benchmarking classification accuracy for three conditions, namely, within-session classification (WS), cross-session classification (CS), and cross-session adaptation (CSA), with subject-specific models. WS achieves an average classification accuracy of up to 68.8%, while CS degrades the accuracy to 53.7% due to the cross-session variability. However, by adaptation, CSA improves the accuracy to 78.9%. We anticipate this new dataset will significantly push further progress in MI BCI research in addressing the cross-session and cross-subject challenge.

Measurement(s)	Electroencephalography
Technology Type(s)	motor imagery

Characterizing reproducibility of cerebral hemodynamic responses when applying short-channel regression in functional near-infrared spectroscopy

Significance: Functional near-infrared spectroscopy (fNIRS) enables the measurement of brain activity noninvasively. Optical neuroimaging with fNIRS has been shown to be reproducible on the group level and hence is an excellent research tool, but the reproducibility on the single-subject level is still insufficient, challenging the use for clinical applications. Aim: We investigated the effect of short-channel regression (SCR) as an approach to obtain fNIRS measurements with higher reproducibility on a single-subject level. SCR simultaneously considers contributions from long- and short-separation channels and removes confounding physiological changes through the regression of the short-separation channel information. Approach: We performed a test-retest study with a hand grasping task in 15 healthy subjects using a wearable fNIRS device, optoHIVE. Relevant brain regions were localized with transcranial magnetic stimulation to ensure correct placement of the optodes. Reproducibility was assessed by intraclass correlation, correlation analysis, mixed effects modeling, and classification accuracy of the hand grasping task. Further, we characterized the influence of SCR on reproducibility. Results: We found a high reproducibility of fNIRS measurements on a single-subject level ( ICC single = 0.81 and correlation r = 0.81 ). SCR increased the reproducibility from 0.64 to 0.81 ( ICC single ) but did not affect classification (85% overall accuracy). Significant intersubject variability in the reproducibility was observed and was explained by Mayer wave oscillations and low raw signal strength. The raw signal-to-noise ratio (threshold at 40 dB) allowed for distinguishing between persons with weak and strong activations. Conclusions: We report, for the first time, that fNIRS measurements are reproducible on a single-subject level using our optoHIVE fNIRS system and that SCR improves reproducibility. In addition, we give a benchmark to easily assess the ability of a subject to elicit sufficiently strong hemodynamic responses. With these insights, we pave the way for the reliable use of fNIRS neuroimaging in single subjects for neuroscientific research and clinical applications.