A large EEG dataset for studying cross-session variability in motor imagery brain-computer interface

Characterization and classification of kinesthetic motor imagery levels

Objective.Kinesthetic Motor Imagery (KMI) represents a robust brain paradigm intended for electroencephalography (EEG)-based commands in brain-computer interfaces (BCIs). However, ensuring high accuracy in multi-command execution remains challenging, with data from C3 and C4 electrodes reaching up to 92% accuracy. This paper aims to characterize and classify EEG-based KMI of multilevel muscle contraction without relying on primary motor cortex signals.Approach.A new method based on Hurst exponents is introduced to characterize EEG signals of multilevel KMI of muscle contraction from electrodes placed on the premotor, dorsolateral prefrontal, and inferior parietal cortices. EEG signals were recorded during a hand-grip task at four levels of muscle contraction (0%, 10%, 40%, and 70% of the maximal isometric voluntary contraction). The task was executed under two conditions: first, physically, to train subjects in achieving muscle contraction at each level, followed by mental imagery under the KMI paradigm for each contraction level. EMG signals were recorded in both conditions to correlate muscle contraction execution, whether correct or null accurately. Independent component analysis (ICA) maps EEG signals from the sensor to the source space for preprocessing. For characterization, three algorithms based on Hurst exponents were used: the original (HO), using partitions (HRS), and applying semivariogram (HV). Finally, seven classifiers were used: Bayes network (BN), naive Bayes (NB), support vector machine (SVM), random forest (RF), random tree (RT), multilayer perceptron (MP), and k-nearest neighbors (kNN).Main results.A combination of the three Hurst characterization algorithms produced the highest average accuracy of 96.42% from kNN, followed by MP (92.85%), SVM (92.85%), NB (91.07%), RF (91.07%), BN (91.07%), and RT (80.35%). of 96.42% for kNN.Significance.Results show the feasibility of KMI multilevel muscle contraction detection and, thus, the viability of non-binary EEG-based BCI applications without using signals from the motor cortex.

An EEG & eye-tracking dataset of ALS patients & healthy people during eye-tracking-based spelling system usage

This research presents a dataset consisting of electroencephalogram and eye tracking recordings obtained from six patients with amyotrophic lateral sclerosis (ALS) in a locked-in state and one hundred seventy healthy individuals. The ALS patients exhibited varying degrees of disease progression, ranging from partial mobility and weakened speech to complete paralysis and loss of speech. Despite these physical impairments, the ALS patients retained good eye function, which allowed them to use a virtual keyboard for communication. Data from ALS patients was recorded multiple times at their homes, while data from healthy individuals was recorded once in a laboratory setting. For each data recording, the experimental design involved nine recording sessions per participant, each corresponding to a common human action or demand. This dataset can serve as a valuable benchmark for several applications, such as improving spelling systems with brain-computer interfaces, investigating motor imagination, exploring motor cortex function, monitoring motor impairment progress in patients undergoing rehabilitation, and studying the effects of ALS on cognitive and motor processes.

Aggregating intrinsic information to enhance BCI performance through federated learning

Insufficient data is a long-standing challenge for Brain-Computer Interface (BCI) to build a high-performance deep learning model. Though numerous research groups and institutes collect a multitude of EEG datasets for the same BCI task, sharing EEG data from multiple sites is still challenging due to the heterogeneity of devices. The significance of this challenge cannot be overstated, given the critical role of data diversity in fostering model robustness. However, existing works rarely discuss this issue, predominantly centering their attention on model training within a single dataset, often in the context of inter-subject or inter-session settings. In this work, we propose a hierarchical personalized Federated Learning EEG decoding (FLEEG) framework to surmount this challenge. This innovative framework heralds a new learning paradigm for BCI, enabling datasets with disparate data formats to collaborate in the model training process. Each client is assigned a specific dataset and trains a hierarchical personalized model to manage diverse data formats and facilitate information exchange. Meanwhile, the server coordinates the training procedure to harness knowledge gleaned from all datasets, thus elevating overall performance. The framework has been evaluated in Motor Imagery (MI) classification with nine EEG datasets collected by different devices but implementing the same MI task. Results demonstrate that the proposed framework can boost classification performance up to 8.4% by enabling knowledge sharing between multiple datasets, especially for smaller datasets. Visualization results also indicate that the proposed framework can empower the local models to put a stable focus on task-related areas, yielding better performance. To the best of our knowledge, this is the first end-to-end solution to address this important challenge.

Single-Trial EEG Classification Using Spatio-Temporal Weighting and Correlation Analysis for RSVP-Based Collaborative Brain Computer Interface

OBJECTIVE

Since single brain computer interface (BCI) is limited in performance, it is necessary to develop collaborative BCI (cBCI) systems which integrate multi-user electroencephalogram (EEG) information to improve system performance. However, there are still some challenges in cBCI systems, including effective discriminant feature extraction of multi-user EEG data, fusion algorithms, time reduction of system calibration, etc. Methods: This study proposed an event-related potential (ERP) feature extraction and classification algorithm of spatio-temporal weighting and correlation analysis (STC) to improve the performance of cBCI systems. The proposed STC algorithm consisted of three modules. First, source extraction and interval modeling were used to overcome the problem of inter-trial variability. Second, spatio-temporal weighting and temporal projection were utilized to extract effective discriminant features for multi-user information fusion and cross-session transfer. Third, correlation analysis was conducted to match target/non-target templates for classification of multi-user and cross-session datasets.

RESULTS

The collaborative cross-session datasets of rapid serial visual presentation (RSVP) from 14 subjects were used to evaluate the performance of the EEG classification algorithm. For single-user/collaborative EEG classification of within-session and cross-session datasets, STC had significantly higher performance than the existing state-of-the-art machine learning algorithms.

CONCLUSION

It was demonstrated that STC was effective to improve the classification performance of multi-user collaboration and cross-session transfer for RSVP-based BCI systems, and was helpful to reduce the system calibration time.

Deep Unsupervised Representation Learning for Feature-Informed EEG Domain Extraction

In electroencephalography (EEG) classification paradigms, data from a target subject is often difficult to obtain, leading to difficulties in training a robust deep learning network. Transfer learning and their variations are effective tools in improving such models suffering from lack of data. However, many of the proposed variations and deep models often rely on a single assumed distribution to represent the latent features which may not scale well due to inter- and intra-subject variations in signals. This leads to significant instability in individual subject decoding performances. The presence of non-trivial domain differences between different sets of training or transfer learning data causes poorer model generalization towards the target subject. However, the detection of these domain differences is often difficult to perform due to the ill-defined nature of the EEG domain features. This study proposes a novel inference model, the Joint Embedding Variational Autoencoder, that offers conditionally tighter approximation of the estimated spatiotemporal feature distribution through the use of jointly optimised variational autoencoders to achieve optimizable data dependent inputs as an additional variable for improved overall model optimisation and scaling without sacrificing model tightness. To learn the variational bound, we show that maximising the marginal log-likelihood of only the second embedding section is required to achieve conditionally tighter lower bounds. Furthermore, we show that this model provides state-of-the-art EEG data reconstruction and deep feature extraction. The extracted domains of the EEG signals across each subject displays the rationale as to why there exists disparity between subjects' adaptation efficacy.

Spatial Variation Generation Algorithm for Motor Imagery Data Augmentation: Increasing the Density of Sample Vicinity

The imbalanced development between deep learning-based model design and motor imagery (MI) data acquisition raises concerns about the potential overfitting issue-models can identify training data well but fail to generalize test data. In this study, a Spatial Variation Generation (SVG) algorithm for MI data augmentation is proposed to alleviate the overfitting issue. In essence, SVG generates MI data using variations of electrode placement and brain spatial pattern, ultimately elevating the density of the raw sample vicinity. The proposed SVG prevents models from memorizing the training data by replacing the raw samples with the proper vicinal distribution. Moreover, SVG generates a uniform distribution and stabilizes the training process of models. In comparison studies involving five deep learning-based models across eight datasets, the proposed SVG algorithm exhibited a notable improvement of 0.021 in the area under the receiver operating characteristic curve (AUC). The improvement achieved by SVG outperforms other data augmentation algorithms. Further results from the ablation study verify the effectiveness of each component of SVG. Finally, the studies in the control group with varying numbers of samples show that the SVG algorithm consistently improves the AUC, with improvements ranging from approximately 0.02 to 0.15.

The challenge of measuring physiological parameters during motor imagery engagement in patients after a stroke

Introduction

It is suggested that eye movement recordings could be used as an objective evaluation method of motor imagery (MI) engagement. Our investigation aimed to evaluate MI engagement in patients after stroke (PaS) compared with physical execution (PE) of a clinically relevant unilateral upper limb movement task of the patients' affected body side.

Methods

In total, 21 PaS fulfilled the MI ability evaluation [Kinaesthetic and Visual Imagery Questionnaire (KVIQ-10), body rotation task (BRT), and mental chronometry task (MC)]. During the experiment, PaS moved a cup to distinct fields while wearing smart eyeglasses (SE) with electrooculography electrodes integrated into the nose pads and electrodes for conventional electrooculography (EOG). To verify MI engagement, heart rate (HR) and oxygen saturation (SpO2) were recorded, simultaneously with electroencephalography (EEG). Eye movements were recorded during MI, PE, and rest in two measurement sessions to compare the SE performance between conditions and SE's psychometric properties.

Results

MI and PE correlation of SE signals varied between r = 0.12 and r = 0.76. Validity (cross-correlation with EOG signals) was calculated for MI (r = 0.53) and PE (r = 0.57). The SE showed moderate test-retest reliability (intraclass correlation coefficient) with r = 0.51 (95% CI 0.26-0.80) for MI and with r = 0.53 (95% CI 0.29 - 0.76) for PE. Event-related desynchronization and event-related synchronization changes of EEG showed a large variability. HR and SpO2 recordings showed similar values during MI and PE. The linear mixed model to examine HR and SpO2 between conditions (MI, PE, rest) revealed a significant difference in HR between rest and MI, and between rest and PE but not for SpO2. A Pearson correlation between MI ability assessments (KVIQ, BRT, MC) and physiological parameters showed no association between MI ability and HR and SpO2.

Conclusion

The objective assessment of MI engagement in PaS remains challenging in clinical settings. However, HR was confirmed as a reliable parameter to assess MI engagement in PaS. Eye movements measured with the SE during MI did not resemble those during PE, which is presumably due to the demanding task. A re-evaluation with task adaptation is suggested.

EEG motor imagery classification using deep learning approaches in naïve BCI users

Motor Imagery (MI)-Brain Computer-Interfaces (BCI) illiteracy defines that not all subjects can achieve a good performance in MI-BCI systems due to different factors related to the fatigue, substance consumption, concentration, and experience in the use. To reduce the effects of lack of experience in the use of BCI systems (naïve users), this paper presents the implementation of three Deep Learning (DL) methods with the hypothesis that the performance of BCI systems could be improved compared with baseline methods in the evaluation of naïve BCI users. The methods proposed here are based on Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM)/Bidirectional Long Short-Term Memory (BiLSTM), and a combination of CNN and LSTM used for upper limb MI signal discrimination on a dataset of 25 naïve BCI users. The results were compared with three widely used baseline methods based on the Common Spatial Pattern (CSP), Filter Bank Common Spatial Pattern (FBCSP), and Filter Bank Common Spatial-Spectral Pattern (FBCSSP), in different temporal window configurations. As results, the LSTM-BiLSTM-based approach presented the best performance, according to the evaluation metrics of Accuracy, F-score, Recall, Specificity, Precision, and ITR, with a mean performance of 80% (maximum 95%) and ITR of 10 bits/min using a temporal window of 1.5 s. The DL Methods represent a significant increase of 32% compared with the baseline methods (p< 0.05). Thus, with the outcomes of this study, it is expected to increase the controllability, usability, and reliability of the use of robotic devices in naïve BCI users.

Discrepancy between inter- and intra-subject variability in EEG-based motor imagery brain-computer interface: Evidence from multiple perspectives

Introduction

Inter- and intra-subject variability are caused by the variability of the psychological and neurophysiological factors over time and across subjects. In the application of in Brain-Computer Interfaces (BCI), the existence of inter- and intra-subject variability reduced the generalization ability of machine learning models seriously, which further limited the use of BCI in real life. Although many transfer learning methods can compensate for the inter- and intra-subject variability to some extent, there is still a lack of clear understanding about the change of feature distribution between the cross-subject and cross-session electroencephalography (EEG) signal.

Methods

To investigate this issue, an online platform for motor-imagery BCI decoding has been built in this work. The EEG signal from both the multi-subject (Exp1) and multi-session (Exp2) experiments has been analyzed from multiple perspectives.

Results

Firstly we found that with the similar variability of classification results, the time-frequency response of the EEG signal within-subject in Exp2 is more consistent than cross-subject results in Exp1. Secondly, the standard deviation of the common spatial pattern (CSP) feature has a significant difference between Exp1 and Exp2. Thirdly, for model training, different strategies for the training sample selection should be applied for the cross-subject and cross-session tasks.

Discussion

All these findings have deepened the understanding of inter- and intra-subject variability. They can also guide practice for the new transfer learning methods development in EEG-based BCI. In addition, these results also proved that BCI inefficiency was not caused by the subject's unable to generate the event-related desynchronization/synchronization (ERD/ERS) signal during the motor imagery.