Co-adaptive Training Improves Efficacy of a Multi-Day EEG-Based Motor Imagery BCI Training

Improving inter-session performance via relevant session-transfer for multi-session motor imagery classification

Motor imagery (MI)-based brain-computer interfaces (BCIs) using electroencephalography (EEG) have found practical applications in external device control. However, the non-stationary nature of EEG signals remains to obstruct BCI performance across multiple sessions, even for the same user. In this study, we aim to address the impact of non-stationarity, also known as inter-session variability, on multi-session MI classification performance by introducing a novel approach, the relevant session-transfer (RST) method. Leveraging the cosine similarity as a benchmark, the RST method transfers relevant EEG data from the previous session to the current one. The effectiveness of the proposed RST method was investigated through performance comparisons with the self-calibrating method, which uses only the data from the current session, and the whole-session transfer method, which utilizes data from all prior sessions. We validated the effectiveness of these methods using two datasets: a large MI public dataset (Shu Dataset) and our own dataset of gait-related MI, which includes both healthy participants and individuals with spinal cord injuries. Our experimental results revealed that the proposed RST method leads to a 2.29 % improvement (p < 0.001) in the Shu Dataset and up to a 6.37 % improvement in our dataset when compared to the self-calibrating method. Moreover, our method surpassed the performance of the recent highest-performing method that utilized the Shu Dataset, providing further support for the efficacy of the RST method in improving multi-session MI classification performance. Consequently, our findings confirm that the proposed RST method can improve classification performance across multiple sessions in practical MI-BCIs.

Improved motor imagery training for subject's self-modulation in EEG-based brain-computer interface

For the electroencephalogram- (EEG-) based motor imagery (MI) brain-computer interface (BCI) system, more attention has been paid to the advanced machine learning algorithms rather than the effective MI training protocols over past two decades. However, it is crucial to assist the subjects in modulating their active brains to fulfill the endogenous MI tasks during the calibration process, which will facilitate signal processing using various machine learning algorithms. Therefore, we propose a trial-feedback paradigm to improve MI training and introduce a non-feedback paradigm for comparison. Each paradigm corresponds to one session. Two paradigms are applied to the calibration runs of corresponding sessions. And their effectiveness is verified in the subsequent testing runs of respective sessions. Different from the non-feedback paradigm, the trial-feedback paradigm presents a topographic map and its qualitative evaluation in real time after each MI training trial, so the subjects can timely realize whether the current trial successfully induces the event-related desynchronization/event-related synchronization (ERD/ERS) phenomenon, and then they can adjust their brain rhythm in the next MI trial. Moreover, after each calibration run of the trial-feedback session, a feature distribution is visualized and quantified to show the subjects' abilities to distinguish different MI tasks and promote their self-modulation in the next calibration run. Additionally, if the subjects feel distracted during the training processes of the non-feedback and trial-feedback sessions, they can execute the blinking movement which will be captured by the electrooculogram (EOG) signals, and the corresponding MI training trial will be abandoned. Ten healthy participants sequentially performed the non-feedback and trial-feedback sessions on the different days. The experiment results showed that the trial-feedback session had better spatial filter visualization, more beneficiaries, higher average off-line and on-line classification accuracies than the non-feedback session, suggesting the trial-feedback paradigm's usefulness in subject's self-modulation and good ability to perform MI tasks.

Identifying optimal channels and features for multi-participant motor imagery experiments across a participant's multi-day multi-class EEG data

The concept of the brain-computer interface (BCI) has become one of the popular research topics of recent times because it allows people to express their thoughts and control different applications and devices without actual movement. The communication between the brain and the computer or a machine is generally provided through Electroencephalogram (EEG) signals because they are cost-effective and easy to implement in normal life, not just in healthcare facilities. On the other hand, they are hard to process efficiently due to their nonlinearity and noisy nature. Thus, the field of BCI and EEG needs constant work and improvement. This paper focuses on generalizing the most efficient EEG channels and the most significant features of motor imagery (MI) signals by analyzing the recordings of one participant obtained over 20 different days. Because the classification performance usually decreases with an increasing number of class labels, we have realized the study by analyzing the signals through a new paradigm consisting of multi-class directional labels: right, left, forward, and backward. Afterward, the results are tested on EEG data obtained from 5 participants to see if the results are consistent with each other. The average accuracy of binary and multi-class classification using the Ensemble Subspace Discriminant classifier was found as 87.39 and 61.44%, respectively, with the most efficient 3-channel combination for daily BCI evaluation of one participant. On the other hand, the average accuracy of binary and multi-class classification was found as 71.84 and 50.42%, respectively, for 5 participants, with the most efficient channel combination of 4, where the first three are the same as the daily performance of one participant. During signal processing, the outliers of the signals were discarded by considering the channels separately. An algorithm was developed to dismiss the inconsistent samples within the classes. A novel adaptive filtering approach, correlation-based adaptive variational mode decomposition (CBAVMD), was proposed. The feature selection was realized based on the standard deviation values of the features between the classes. The paradigm based on the direction movements was found to be most effective, especially for binary classification of right and left directions. The generalization of effective channels and features was found to be generally successful.

Deep Unsupervised Representation Learning for Feature-Informed EEG Domain Extraction

In electroencephalography (EEG) classification paradigms, data from a target subject is often difficult to obtain, leading to difficulties in training a robust deep learning network. Transfer learning and their variations are effective tools in improving such models suffering from lack of data. However, many of the proposed variations and deep models often rely on a single assumed distribution to represent the latent features which may not scale well due to inter- and intra-subject variations in signals. This leads to significant instability in individual subject decoding performances. The presence of non-trivial domain differences between different sets of training or transfer learning data causes poorer model generalization towards the target subject. However, the detection of these domain differences is often difficult to perform due to the ill-defined nature of the EEG domain features. This study proposes a novel inference model, the Joint Embedding Variational Autoencoder, that offers conditionally tighter approximation of the estimated spatiotemporal feature distribution through the use of jointly optimised variational autoencoders to achieve optimizable data dependent inputs as an additional variable for improved overall model optimisation and scaling without sacrificing model tightness. To learn the variational bound, we show that maximising the marginal log-likelihood of only the second embedding section is required to achieve conditionally tighter lower bounds. Furthermore, we show that this model provides state-of-the-art EEG data reconstruction and deep feature extraction. The extracted domains of the EEG signals across each subject displays the rationale as to why there exists disparity between subjects' adaptation efficacy.

2020 International brain-computer interface competition: A review

The brain-computer interface (BCI) has been investigated as a form of communication tool between the brain and external devices. BCIs have been extended beyond communication and control over the years. The 2020 international BCI competition aimed to provide high-quality neuroscientific data for open access that could be used to evaluate the current degree of technical advances in BCI. Although there are a variety of remaining challenges for future BCI advances, we discuss some of more recent application directions: (i) few-shot EEG learning, (ii) micro-sleep detection (iii) imagined speech decoding, (iv) cross-session classification, and (v) EEG(+ear-EEG) detection in an ambulatory environment. Not only did scientists from the BCI field compete, but scholars with a broad variety of backgrounds and nationalities participated in the competition to address these challenges. Each dataset was prepared and separated into three data that were released to the competitors in the form of training and validation sets followed by a test set. Remarkable BCI advances were identified through the 2020 competition and indicated some trends of interest to BCI researchers.

Riemannian channel selection for BCI with between-session non-stationarity reduction capabilities

OBJECTIVE

Between-session non-stationarity is a major challenge of current Brain-Computer Interfaces (BCIs) that affects system performance. In this paper, we investigate the use of channel selection for reducing between-session non-stationarity with Riemannian BCI classifiers. We use the Riemannian geometry framework of covariance matrices due to its robustness and promising performances. Current Riemannian channel selection methods do not consider between-session non-stationarity and are usually tested on a single session. Here, we propose a new channel selection approach that specifically considers non-stationarity effects and is assessed on multi-session BCI data sets.

METHODS

We remove the least significant channels using a sequential floating backward selection search strategy. Our contributions include: 1) quantifying the non-stationarity effects on brain activity in multi-class problems by different criteria in a Riemannian framework and 2) a method to predict whether BCI performance can improve using channel selection.

RESULTS

We evaluate the proposed approaches on three multi-session and multi-class mental tasks (MT)-based BCI datasets. They could lead to significant improvements in performance as compared to using all channels for datasets affected by between-session non-stationarity and to significant superiority to the state-of-the-art Riemannian channel selection methods over all datasets, notably when selecting small channel set sizes.

CONCLUSION

Reducing non-stationarity by channel selection could significantly improve Riemannian BCI classification accuracy.

SIGNIFICANCE

Our proposed channel selection approach contributes to make Riemannian BCI classifiers more robust to between-session non-stationarities.

Combining Statistical Analysis and Machine Learning for EEG Scalp Topograms Classification

Incorporating brain-computer interfaces (BCIs) into daily life requires reducing the reliance of decoding algorithms on the calibration or enabling calibration with the minimal burden on the user. A potential solution could be a pre-trained decoder demonstrating a reasonable accuracy on the naive operators. Addressing this issue, we considered ambiguous stimuli classification tasks and trained an artificial neural network to classify brain responses to the stimuli of low and high ambiguity. We built a pre-trained classifier utilizing time-frequency features corresponding to the fundamental neurophysiological processes shared between subjects. To extract these features, we statistically contrasted electroencephalographic (EEG) spectral power between the classes in the representative group of subjects. As a result, the pre-trained classifier achieved 74% accuracy on the data of newly recruited subjects. Analysis of the literature suggested that a pre-trained classifier could help naive users to start using BCI bypassing training and further increased accuracy during the feedback session. Thus, our results contribute to using BCI during paralysis or limb amputation when there is no explicit user-generated kinematic output to properly train a decoder. In machine learning, our approach may facilitate the development of transfer learning (TL) methods for addressing the cross-subject problem. It allows extracting the interpretable feature subspace from the source data (the representative group of subjects) related to the target data (a naive user), preventing the negative transfer in the cross-subject tasks.

Feasibility of using discrete Brain Computer Interface for people with Multiple Sclerosis

AIM

Brain-Computer Interfaces (BCIs) hold promise to provide people with partial or complete paralysis, the ability to control assistive technology. This study reports offline classification of imagined and executed movements of the upper and lower limb in one participant with multiple sclerosis and people with no limb function deficits.

METHODS

We collected neural signals using electroencephalography (EEG) while participants performed executed and imagined motor tasks as directed by prompts shown on a screen.

RESULTS

Participants with no limb function attained >70% decoding accuracy on their best-imagined task compared to rest and on at-least one task comparison. The participant with multiple sclerosis also achieved accuracies within the range of participants with no limb function loss.Clinical Relevance - While only one case study is provided it was promising that the participant with MS was able to achieve comparable classification to that of the seven healthy controls. Further studies are needed to assess whether people suffering from MS may be able to use a BCI to improve their quality of life.

Seeming confines: Electrophysiological evidence of peripersonal space remapping following tool-use in humans

The peripersonal space (PPS) is a special portion of space immediately surrounding the body, where the integration between tactile stimuli delivered on the body and auditory or visual events emanating from the environment occurs. Interestingly, PPS can widen if a tool is employed to interact with objects in the far space. However, electrophysiological evidence of such tool-use dependent plasticity in the human brain is scarce. Here, in a series of three experiments, participants were asked to respond to tactile stimuli, delivered to their right hand, either in isolation (unimodal condition) or combined with auditory stimulation, which could occur near (bimodal-near) or far from the stimulated hand (bimodal-far). According to multisensory integration spatial rule, when bimodal stimuli are presented at the same location, we expected a response enhancement (response time - RT - facilitation and event-related potential - ERP - super-additivity). In Experiment 1, we verified that RT facilitation was driven by bimodal input spatial congruency, independently from auditory stimulus intensity. In Experiment 2, we showed that our bimodal task was effective in eliciting the magnification of ERPs in bimodal conditions, with significantly larger responses in the near as compared to far condition. In Experiment 3 (main experiment), we explored tool-use driven PPS plasticity. Our audio-tactile task was performed either following tool-use (a 20-min reaching task, performed using a 145 cm-long rake) or after a control cognitive training (a 20-min visual discrimination task) performed in the far space. Following the control training, faster RTs and greater super-additive ERPs were found in bimodal-near as compared to bimodal-far condition (replicating Experiment 2 results). Crucially, this far-near differential response was significantly reduced after tool-use. Altogether our results indicate a selective effect of tool-use remapping in extending the boundaries of PPS. The present finding might be considered as an electrophysiological evidence of tool-use dependent plasticity in the human brain.

Relative Power Correlates With the Decoding Performance of Motor Imagery Both Across Time and Subjects

One of the most significant challenges in the application of brain-computer interfaces (BCI) is the large performance variation, which often occurs over time or across users. Recent evidence suggests that the physiological states may explain this performance variation in BCI, however, the underlying neurophysiological mechanism is unclear. In this study, we conducted a seven-session motor-imagery (MI) experiment on 20 healthy subjects to investigate the neurophysiological mechanism on the performance variation. The classification accuracy was calculated offline by common spatial pattern (CSP) and support vector machine (SVM) algorithms to measure the MI performance of each subject and session. Relative Power (RP) values from different rhythms and task stages were used to reflect the physiological states and their correlation with the BCI performance was investigated. Results showed that the alpha band RP from the supplementary motor area (SMA) within a few seconds before MI was positively correlated with performance. Besides, the changes of RP between task and pre-task stage from theta, alpha, and gamma band were also found to be correlated with performance both across time and subjects. These findings reveal a neurophysiological manifestation of the performance variations, and would further provide a way to improve the BCI performance.

A Framework for Optimizing Co-adaptation in Body-Machine Interfaces

The operation of a human-machine interface is increasingly often referred to as a two-learners problem, where both the human and the interface independently adapt their behavior based on shared information to improve joint performance over a specific task. Drawing inspiration from the field of body-machine interfaces, we take a different perspective and propose a framework for studying co-adaptation in scenarios where the evolution of the interface is dependent on the users' behavior and that do not require task goals to be explicitly defined. Our mathematical description of co-adaptation is built upon the assumption that the interface and the user agents co-adapt toward maximizing the interaction efficiency rather than optimizing task performance. This work describes a mathematical framework for body-machine interfaces where a naïve user interacts with an adaptive interface. The interface, modeled as a linear map from a space with high dimension (the user input) to a lower dimensional feedback, acts as an adaptive “tool” whose goal is to minimize transmission loss following an unsupervised learning procedure and has no knowledge of the task being performed by the user. The user is modeled as a non-stationary multivariate Gaussian generative process that produces a sequence of actions that is either statistically independent or correlated. Dependent data is used to model the output of an action selection module concerned with achieving some unknown goal dictated by the task. The framework assumes that in parallel to this explicit objective, the user is implicitly learning a suitable but not necessarily optimal way to interact with the interface. Implicit learning is modeled as use-dependent learning modulated by a reward-based mechanism acting on the generative distribution. Through simulation, the work quantifies how the system evolves as a function of the learning time scales when a user learns to operate a static vs. an adaptive interface. We show that this novel framework can be directly exploited to readily simulate a variety of interaction scenarios, to facilitate the exploration of the parameters that lead to optimal learning dynamics of the joint system, and to provide an empirical proof for the superiority of human-machine co-adaptation over user adaptation.

Multimodal signal dataset for 11 intuitive movement tasks from single upper extremity during multiple recording sessions

BACKGROUND

Non-invasive brain-computer interfaces (BCIs) have been developed for realizing natural bi-directional interaction between users and external robotic systems. However, the communication between users and BCI systems through artificial matching is a critical issue. Recently, BCIs have been developed to adopt intuitive decoding, which is the key to solving several problems such as a small number of classes and manually matching BCI commands with device control. Unfortunately, the advances in this area have been slow owing to the lack of large and uniform datasets. This study provides a large intuitive dataset for 11 different upper extremity movement tasks obtained during multiple recording sessions. The dataset includes 60-channel electroencephalography, 7-channel electromyography, and 4-channel electro-oculography of 25 healthy participants collected over 3-day sessions for a total of 82,500 trials across all the participants.

FINDINGS

We validated our dataset via neurophysiological analysis. We observed clear sensorimotor de-/activation and spatial distribution related to real-movement and motor imagery, respectively. Furthermore, we demonstrated the consistency of the dataset by evaluating the classification performance of each session using a baseline machine learning method.

CONCLUSIONS

The dataset includes the data of multiple recording sessions, various classes within the single upper extremity, and multimodal signals. This work can be used to (i) compare the brain activities associated with real movement and imagination, (ii) improve the decoding performance, and (iii) analyze the differences among recording sessions. Hence, this study, as a Data Note, has focused on collecting data required for further advances in the BCI technology.

Regression Networks for Neurophysiological Indicator Evaluation in Practicing Motor Imagery Tasks

Motor Imagery (MI) promotes motor learning in activities, like developing professional motor skills, sports gestures, and patient rehabilitation. However, up to 30% of users may not develop enough coordination skills after training sessions because of inter and intra-subject variability. Here, we develop a data-driven estimator, termed Deep Regression Network (DRN), which jointly extracts and performs the regression analysis in order to assess the efficiency of the individual brain networks in practicing MI tasks. The proposed double-stage estimator initially learns a pool of deep patterns, extracted from the input data, in order to feed a neural regression model, allowing for infering the distinctiveness between subject assemblies having similar variability. The results, which were obtained on real-world MI data, prove that the DRN estimator fosters pre-training neural desynchronization and initial training synchronization to predict the bi-class accuracy response, thus providing a better understanding of the Brain-Computer Interface inefficiency of subjects.