True zero-training brain-computer interfacing--an online study

Electroencephalogram-based adaptive closed-loop brain-computer interface in neurorehabilitation: a review

Brain-computer interfaces (BCIs) represent a groundbreaking approach to enabling direct communication for individuals with severe motor impairments, circumventing traditional neural and muscular pathways. Among the diverse array of BCI technologies, electroencephalogram (EEG)-based systems are particularly favored due to their non-invasive nature, user-friendly operation, and cost-effectiveness. Recent advancements have facilitated the development of adaptive bidirectional closed-loop BCIs, which dynamically adjust to users' brain activity, thereby enhancing responsiveness and efficacy in neurorehabilitation. These systems support real-time modulation and continuous feedback, fostering personalized therapeutic interventions that align with users' neural and behavioral responses. By incorporating machine learning algorithms, these BCIs optimize user interaction and promote recovery outcomes through mechanisms of activity-dependent neuroplasticity. This paper reviews the current landscape of EEG-based adaptive bidirectional closed-loop BCIs, examining their applications in the recovery of motor and sensory functions, as well as the challenges encountered in practical implementation. The findings underscore the potential of these technologies to significantly enhance patients' quality of life and social interaction, while also identifying critical areas for future research aimed at improving system adaptability and performance. As advancements in artificial intelligence continue, the evolution of sophisticated BCI systems holds promise for transforming neurorehabilitation and expanding applications across various domains.

On decoding of rapid motor imagery in a diverse population using a high-density NIRS device

Introduction

Functional near-infrared spectroscopy (fNIRS) aims to infer cognitive states such as the type of movement imagined by a study participant in a given trial using an optical method that can differentiate between oxygenation states of blood in the brain and thereby indirectly between neuronal activity levels. We present findings from an fNIRS study that aimed to test the applicability of a high-density (>3000 channels) NIRS device for use in short-duration (2 s) left/right hand motor imagery decoding in a diverse, but not explicitly balanced, subject population. A side aim was to assess relationships between data quality, self-reported demographic characteristics, and brain-computer interface (BCI) performance, with no subjects rejected from recruitment or analysis.

Methods

BCI performance was quantified using several published methods, including subject-specific and subject-independent approaches, along with a high-density fNIRS decoder previously validated in a separate study.

Results

We found that decoding of motor imagery on this population proved extremely challenging across all tested methods. Overall accuracy of the best-performing method (the high-density decoder) was 59.1 +/- 6.7% after excluding subjects where almost no optode-scalp contact was made over motor cortex and 54.7 +/- 7.6% when all recorded sessions were included. Deeper investigation revealed that signal quality, hemodynamic responses, and BCI performance were all strongly impacted by the hair phenotypical and demographic factors under investigation, with over half of variance in signal quality explained by demographic factors alone.

Discussion

Our results contribute to the literature reporting on challenges in using current-generation NIRS devices on subjects with long, dense, dark, and less pliable hair types along with the resulting potential for bias. Our findings confirm the need for increased focus on these populations, accurate reporting of data rejection choices across subject intake, curation, and final analysis in general, and signal a need for NIRS optode designs better optimized for the general population to facilitate more robust and inclusive research outcomes.

A comprehensive dataset for home appliance control using ERP-based BCIs with the application of inter-subject transfer learning

Brain-computer interfaces (BCIs) have a potential to revolutionize human-computer interaction by enabling direct links between the brain and computer systems. Recent studies are increasingly focusing on practical applications of BCIs-e.g., home appliance control just by thoughts. One of the non-invasive BCIs using electroencephalography (EEG) capitalizes on event-related potentials (ERPs) in response to target stimuli and have shown promise in controlling home appliance. In this paper, we present a comprehensive dataset of online ERP-based BCIs for controlling various home appliances in diverse stimulus presentation environments. We collected online BCI data from a total of 84 subjects among whom 60 subjects controlled three types of appliances (TV: 30, door lock: 15, and electric light: 15) with 4 functions per appliance, 14 subjects controlled a Bluetooth speaker with 6 functions via an LCD monitor, and 10 subjects controlled air conditioner with 4 functions via augmented reality (AR). Using the dataset, we aimed to address the issue of inter-subject variability in ERPs by employing the transfer learning in two different approaches. The first approach, "within-paradigm transfer learning," aimed to generalize the model within the same paradigm of stimulus presentation. The second approach, "cross-paradigm transfer learning," involved extending the model from a 4-class LCD environment to different paradigms. The results demonstrated that transfer learning can effectively enhance the generalizability of BCIs based on ERP across different subjects and environments.

SSVEP unsupervised adaptive feature recognition method based on self-similarity of same-frequency signals

Introduction

As an important human-computer interaction technology, steady-state visual evoked potential (SSVEP) plays a key role in the application of brain computer interface (BCI) systems by accurately decoding SSVEP signals. Currently, the majority SSVEP feature recognition methods use a static classifier. However, electroencephalogram (EEG) signals are non-stationary and time-varying. Hence, an adaptive classification method would be an alternative option to a static classifier for tracking the changes in EEG feature distribution, as its parameters can be re-estimated and updated with the input of new EEG data.

Methods

In this study, an unsupervised adaptive classification algorithm is designed based on the self-similarity of same-frequency signals. The proposed classification algorithm saves the EEG data that has undergone feature recognition as a template signal in accordance with its estimated label, and the new testing signal is superimposed with the template signals at each stimulus frequency as the new test signals to be analyzed. With the continuous input of EEG data, the template signals are continuously updated.

Results

By comparing the classification accuracy of the original testing signal and the testing signal superimposed with the template signals, this study demonstrates the effectiveness of using the self-similarity of same-frequency signals in the adaptive classification algorithm. The experimental results also show that the longer the SSVEP-BCI system is used, the better the responses of users on SSVEP are, and the more significantly the adaptive classification algorithm performs in terms of feature recognition. The testing results of two public datasets show that the adaptive classification algorithm outperforms the static classification method in terms of feature recognition.

Discussion

The proposed adaptive classification algorithm can update the parameters with the input of new EEG data, which is of favorable impact for the accurate analysis of EEG data with time-varying characteristics.

2020 International brain-computer interface competition: A review

The brain-computer interface (BCI) has been investigated as a form of communication tool between the brain and external devices. BCIs have been extended beyond communication and control over the years. The 2020 international BCI competition aimed to provide high-quality neuroscientific data for open access that could be used to evaluate the current degree of technical advances in BCI. Although there are a variety of remaining challenges for future BCI advances, we discuss some of more recent application directions: (i) few-shot EEG learning, (ii) micro-sleep detection (iii) imagined speech decoding, (iv) cross-session classification, and (v) EEG(+ear-EEG) detection in an ambulatory environment. Not only did scientists from the BCI field compete, but scholars with a broad variety of backgrounds and nationalities participated in the competition to address these challenges. Each dataset was prepared and separated into three data that were released to the competitors in the form of training and validation sets followed by a test set. Remarkable BCI advances were identified through the 2020 competition and indicated some trends of interest to BCI researchers.

Neural correlates of user learning during long-term BCI training for the Cybathlon competition

Background

Brain-computer interfaces (BCIs) are systems capable of translating human brain patterns, measured through electroencephalography (EEG), into commands for an external device. Despite the great advances in machine learning solutions to enhance the performance of BCI decoders, the translational impact of this technology remains elusive. The reliability of BCIs is often unsatisfactory for end-users, limiting their application outside a laboratory environment.

Methods

We present the analysis on the data acquired from an end-user during the preparation for two Cybathlon competitions, where our pilot won the gold medal twice in a row. These data are of particular interest given the mutual learning approach adopted during the longitudinal training phase (8 months), the long training break in between the two events (1 year) and the demanding evaluation scenario. A multifaceted perspective on long-term user learning is proposed: we enriched the information gathered through conventional metrics (e.g., accuracy, application performances) by investigating novel neural correlates of learning in different neural domains.

Results

First, we showed that by focusing the training on user learning, the pilot was capable of significantly improving his performance over time even with infrequent decoder re-calibrations. Second, we revealed that the analysis of the within-class modifications of the pilot’s neural patterns in the Riemannian domain is more effective in tracking the acquisition and the stabilization of BCI skills, especially after the 1-year break. These results further confirmed the key role of mutual learning in the acquisition of BCI skills, and particularly highlighted the importance of user learning as a key to enhance BCI reliability.

Conclusion

We firmly believe that our work may open new perspectives and fuel discussions in the BCI field to shift the focus of future research: not only to the machine learning of the decoder, but also in investigating novel training procedures to boost the user learning and the stability of the BCI skills in the long-term. To this end, the analyses and the metrics proposed could be used to monitor the user learning during training and provide a marker guiding the decoder re-calibration to maximize the mutual adaptation of the user to the BCI system.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12984-022-01047-x.

Selective Subject Pooling Strategy to Improve Model Generalization for a Motor Imagery BCI

Brain–computer interfaces (BCIs) facilitate communication for people who cannot move their own body. A BCI system requires a lengthy calibration phase to produce a reasonable classifier. To reduce the duration of the calibration phase, it is natural to attempt to create a subject-independent classifier with all subject datasets that are available; however, electroencephalogram (EEG) data have notable inter-subject variability. Thus, it is very challenging to achieve subject-independent BCI performance comparable to subject-specific BCI performance. In this study, we investigate the potential for achieving better subject-independent motor imagery BCI performance by conducting comparative performance tests with several selective subject pooling strategies (i.e., choosing subjects who yield reasonable performance selectively and using them for training) rather than using all subjects available. We observed that the selective subject pooling strategy worked reasonably well with public MI BCI datasets. Finally, based upon the findings, criteria to select subjects for subject-independent BCIs are proposed here.

An Open Source-Based BCI Application for Virtual World Tour and Its Usability Evaluation

Brain-computer interfaces can provide a new communication channel and control functions to people with restricted movements. Recent studies have indicated the effectiveness of brain-computer interface (BCI) applications. Various types of applications have been introduced so far in this field, but the number of those available to the public is still insufficient. Thus, there is a need to expand the usability and accessibility of BCI applications. In this study, we introduce a BCI application for users to experience a virtual world tour. This software was built on three open-source environments and is publicly available through the GitHub repository. For a usability test, 10 healthy subjects participated in an electroencephalography (EEG) experiment and evaluated the system through a questionnaire. As a result, all the participants successfully played the BCI application with 96.6% accuracy with 20 blinks from two sessions and gave opinions on its usability (e.g., controllability, completeness, comfort, and enjoyment) through the questionnaire. We believe that this open-source BCI world tour system can be used in both research and entertainment settings and hopefully contribute to open science in the BCI field.

Learning Invariant Patterns Based on a Convolutional Neural Network and Big Electroencephalography Data for Subject-Independent P300 Brain-Computer Interfaces

A brain-computer interface (BCI) measures and analyzes brain activity and converts this activity into computer commands to control external devices. In contrast to traditional BCIs that require a subject-specific calibration process before being operated, a subject-independent BCI learns a subject-independent model and eliminates subject-specific calibration for new users. However, building subject-independent BCIs remains difficult because electroencephalography (EEG) is highly noisy and varies by subject. In this study, we propose an invariant pattern learning method based on a convolutional neural network (CNN) and big EEG data for subject-independent P300 BCIs. The CNN was trained using EEG data from a large number of subjects, allowing it to extract subject-independent features and make predictions for new users. We collected EEG data from 200 subjects in a P300-based spelling task using two different types of amplifiers. The offline analysis showed that almost all subjects obtained significant cross-subject and cross-amplifier effects, with an average accuracy of more than 80%. Furthermore, more than half of the subjects achieved accuracies above 85%. These results indicated that our method was effective for building a subject-independent P300 BCI, with which more than 50% of users could achieve high accuracies without subject-specific calibration.

A Usability Study of Low-cost Wireless Brain-Computer Interface for Cursor Control Using Online Linear Model

Computer cursor control using electroencephalogram (EEG) signals is a common and well-studied brain-computer interface (BCI). The emphasis of the literature has been primarily on evaluation of the objective measures of assistive BCIs such as accuracy of the neural decoder whereas the subjective measures such as user's satisfaction play an essential role for the overall success of a BCI. As far as we know, the BCI literature lacks a comprehensive evaluation of the usability of the mind-controlled computer cursor in terms of decoder efficiency (accuracy), user experience, and relevant confounding variables concerning the platform for the public use. To fill this gap, we conducted a two-dimensional EEG-based cursor control experiment among 28 healthy participants. The computer cursor velocity was controlled by the imagery of hand movement using a paradigm presented in the literature named imagined body kinematics (IBK) with a low-cost wireless EEG headset. We evaluated the usability of the platform for different objective and subjective measures while we investigated the extent to which the training phase may influence the ultimate BCI outcome. We conducted pre- and post- BCI experiment interview questionnaires to evaluate the usability. Analyzing the questionnaires and the testing phase outcome shows a positive correlation between the individuals' ability of visualization and their level of mental controllability of the cursor. Despite individual differences, analyzing training data shows the significance of electrooculogram (EOG) on the predictability of the linear model. The results of this work may provide useful insights towards designing a personalized user-centered assistive BCI.

A Generic Error-related Potential Classifier Offers a Comparable Performance to a Personalized Classifier

Brain-computer interfaces (BCIs) provide more independence to people with severe motor disabilities but current BCIs' performance is still not optimal and often the user's intentions are misinterpreted. Error-related potentials (ErrPs) are the neurophysiological signature of error processing and their detection can help improving a BCI's performance.A major inconvenience of BCIs is that they commonly require a long calibration period, before the user can receive feedback of their own brain signals. Here, we use the data of 15 participants and compare the performance of a personalized ErrP classifier with a generic ErrP classifier. We concluded that there was no significant difference in classification performance between the generic and the personalized classifiers (Wilcoxon signed rank tests, two-sided and one-sided left and right). This results indicate that the use of a generic ErrP classifier is a good strategy to remove the calibration period of a ErrP classifier, allowing participants to receive immediate feedback of the ErrP detections.

Comparison of Classifiers for the Transfer Learning of Affective Auditory P300-Based BCIs

The auditory P300-based BCI was improved by changing stimuli. However, the current method needed time for recording training data. The time can be saved by the subject-to-subject transfer learning. However, the suitable classifier for the learning remains unknown. As a first step, this study compared the classifiers for the transfer learning of the BCI. They were evaluated on the dataset of a five-class affective auditory P300-based BCI. EEG data from sixteen subjects were assigned for the training, then data from the other six subjects were used for the testing. Classifiers such as the linear support-vector machine (SVM lin.), the kernel SVM (SVM RBF), the quadratic discriminant analysis were applied and compared. As a result, the SVM lin. and the SVM RBF were suitable for this problem. The best mean classification accuracy was achieved by the SVM lin. (68.7%), and a subject showed 86% accuracy at best. These results suggest that some subjects can operate the BCI without recording his/her training data.

A TrAdaBoost Method for Detecting Multiple Subjects' N200 and P300 Potentials Based on Cross-Validation and an Adaptive Threshold

Traditional training methods need to collect a large amount of data for every subject to train a subject-specific classifier, which causes subjects fatigue and training burden. This study proposes a novel training method, TrAdaBoost based on cross-validation and an adaptive threshold (CV-T-TAB), to reduce the amount of data required for training by selecting and combining multiple subjects' classifiers that perform well on a new subject to train a classifier. This method adopts cross-validation to extend the amount of the new subject's training data and sets an adaptive threshold to select the optimal combination of the classifiers. Twenty-five subjects participated in the N200- and P300-based brain-computer interface. The study compares CV-T-TAB to five traditional training methods by testing them on the training of a support vector machine. The accuracy, information transfer rate, area under the curve, recall and precision are used to evaluate the performances under nine conditions with different amounts of data. CV-T-TAB outperforms the other methods and retains a high accuracy even when the amount of data is reduced to one-third of the original amount. The results imply that CV-T-TAB is effective in improving the performance of a subject-specific classifier with a small amount of data by adopting multiple subjects' classifiers, which reduces the training cost.

Learning Invariant Representations from EEG via Adversarial Inference

Discovering and exploiting shared, invariant neural activity in electroencephalogram (EEG) based classification tasks is of significant interest for generalizability of decoding models across subjects or EEG recording sessions. While deep neural networks are recently emerging as generic EEG feature extractors, this transfer learning aspect usually relies on the prior assumption that deep networks naturally behave as subject- (or session-) invariant EEG feature extractors. We propose a further step towards invariance of EEG deep learning frameworks in a systemic way during model training. We introduce an adversarial inference approach to learn representations that are invariant to inter-subject variabilities within a discriminative setting. We perform experimental studies using a publicly available motor imagery EEG dataset, and state-of-the-art convolutional neural network based EEG decoding models within the proposed adversarial learning framework. We present our results in cross-subject model transfer scenarios, demonstrate neurophysiological interpretations of the learned networks, and discuss potential insights offered by adversarial inference to the growing field of deep learning for EEG.

Workshops of the Seventh International Brain-Computer Interface Meeting: Not Getting Lost in Translation

The Seventh International Brain-Computer Interface (BCI) Meeting was held May 21-25th, 2018 at the Asilomar Conference Grounds, Pacific Grove, California, United States. The interactive nature of this conference was embodied by 25 workshops covering topics in BCI (also called brain-machine interface) research. Workshops covered foundational topics such as hardware development and signal analysis algorithms, new and imaginative topics such as BCI for virtual reality and multi-brain BCIs, and translational topics such as clinical applications and ethical assumptions of BCI development. BCI research is expanding in the diversity of applications and populations for whom those applications are being developed. BCI applications are moving toward clinical readiness as researchers struggle with the practical considerations to make sure that BCI translational efforts will be successful. This paper summarizes each workshop, providing an overview of the topic of discussion, references for additional information, and identifying future issues for research and development that resulted from the interactions and discussion at the workshop.

Novel hybrid brain-computer interface system based on motor imagery and P300

Motor imagery (MI) is a mental representation of motor behavior and has been widely used in electroencephalogram based brain-computer interfaces (BCIs). Several studies have demonstrated the efficacy of MI-based BCI-feedback training in post-stroke rehabilitation. However, in the earliest stage of the training, calibration data typically contain insufficient discriminability, resulting in unreliable feedback, which may decrease subjects' motivation and even hinder their training. To improve the performance in the early stages of MI training, a novel hybrid BCI paradigm based on MI and P300 is proposed in this study. In this paradigm, subjects are instructed to imagine writing the Chinese character following the flash order of the desired Chinese character displayed on the screen. The event-related desynchronization/synchronization (ERD/ERS) phenomenon is produced with writing based on one's imagination. Simultaneously, the P300 potential is evoked by the flash of each stroke. Moreover, a fusion method of P300 and MI classification is proposed, in which unreliable P300 classifications are corrected by reliable MI classifications. Twelve healthy naïve MI subjects participated in this study. Results demonstrated that the proposed hybrid BCI paradigm yielded significantly better performance than the single-modality BCI paradigm. The recognition accuracy of the fusion method is significantly higher than that of P300 (p < 0.05) and MI (p < 0.01). Moreover, the training data size can be reduced through fusion of these two modalities.

Application of identity vectors for EEG classification

BACKGROUND

Finding an optimal EEG subject verification algorithm is a long standing goal within the EEG community. For every advancement made, another feature set, classifier, or dataset is often introduced; tracking improvements in classification without a consistent benchmark, such as a classifier-feature pairing tested on a publicly available dataset, makes it difficult to understand how and why these improvements occur.

NEW METHOD

Following on previous biometric experiments, I-Vectors and Gaussian Mixture Model-Universal Background Models are compared to an established Mahalanobis classifier. A second experiment then addresses the impact of epoch duration as a function of classification performance across all three classifiers.

RESULTS

The experimental classification results indicate that I-Vectors are more robust than the other classifiers displaying less sensitivity to epoch duration, data composition, and feature selection.

COMPARISON WITH EXISTING METHODS

This I-Vector based approach is compared against commonly used EEG classifiers, such as Mahalanobis and Gaussian mixture models. These classifiers are benchmarked using the publicly available PhysioNet database converted into three feature sets, spectral coherence, power spectral density, and cepstral coefficients.

CONCLUSIONS

The experimental results suggests I-Vectors provide reliable baseline performance by leveling the field between feature set and datasets making them well suited for EEG signal processing tasks.

Training set extension for SVM ensemble in P300-speller with familiar face paradigm

BACKGROUND

P300-spellers are brain-computer interface (BCI)-based character input systems. Support vector machine (SVM) ensembles are trained with large-scale training sets and used as classifiers in these systems. However, the required large-scale training data necessitate a prolonged collection time for each subject, which results in data collected toward the end of the period being contaminated by the subject's fatigue.

OBJECTIVE

This study aimed to develop a method for acquiring more training data based on a collected small training set.

METHODS

A new method was developed in which two corresponding training datasets in two sequences are superposed and averaged to extend the training set. The proposed method was tested offline on a P300-speller with the familiar face paradigm.

RESULTS

The SVM ensemble with extended training set achieved 85% classification accuracy for the averaged results of four sequences, and 100% for 11 sequences in the P300-speller. In contrast, the conventional SVM ensemble with non-extended training set achieved only 65% accuracy for four sequences, and 92% for 11 sequences.

CONCLUSION

The SVM ensemble with extended training set achieves higher classification accuracies than the conventional SVM ensemble, which verifies that the proposed method effectively improves the classification performance of BCI P300-spellers, thus enhancing their practicality.

Eyes-Closed Increases the Usability of Brain-Computer Interfaces Based on Auditory Event-Related Potentials

Recent research has demonstrated how brain-computer interfaces (BCI) based on auditory stimuli can be used for communication and rehabilitation. In these applications, users are commonly instructed to avoid eye movements while keeping their eyes open. This secondary task can lead to exhaustion and subjects may not succeed in suppressing eye movements. In this work, we investigate the option to use a BCI with eyes-closed. Twelve healthy subjects participated in a single electroencephalography (EEG) session where they were listening to a rapid stream of bisyllabic words while alternatively having their eyes open or closed. In addition, we assessed usability aspects for the two conditions with a questionnaire. Our analysis shows that eyes-closed does not reduce the number of eye artifacts and that event-related potential (ERP) responses and classification accuracies are comparable between both conditions. Importantly, we found that subjects expressed a significant general preference toward the eyes-closed condition and were also less tensed in that condition. Furthermore, switching between eyes-closed and eyes-open and vice versa is possible without a severe drop in classification accuracy. These findings suggest that eyes-closed should be considered as a viable alternative in auditory BCIs that might be especially useful for subjects with limited control over their eye movements.

To train or not to train? A survey on training of feature extraction methods for SSVEP-based BCIs

OBJECTIVE

Despite the vast research aimed at improving the performance of steady-state visually evoked potential (SSVEP)-based brain-computer interfaces (BCIs), several limitations exist that restrict the use of such applications for long-term users in the real-world. One of the main challenges has been to reduce training time while maintaining good BCI performance. In view of this challenge, this survey identifies and compares the different training requirements of feature extraction methods for SSVEP-based BCIs.

APPROACH

This paper reviews the various state-of-the-art SSVEP feature extraction methods that have been developed and are most widely used in the literature.

MAIN RESULTS

The main contributions compared to existing reviews are the following: (i) a detailed summary, including a brief mathematical description of each feature extraction algorithm, providing a guide to the basic concepts of the state-of-the-art techniques for SSVEP-based BCIs found in literature; (ii) a categorisation of the training requirements of SSVEP-based methods into three categories, defined as training-free methods, subject-specific and subject-independent training methods; (iii) a comparative review of the training requirements of SSVEP feature extraction methods, providing a reference for future work on SSVEP-based BCIs.

SIGNIFICANCE

This review highlights the strengths and weaknesses of the three categories of SSVEP training methods. Training-free systems are more practical but their performance is limited due to inter-subject variability resulting from the complex EEG activity. Feature extraction methods that incorporate some training data address this issue and in fact have outperformed training-free methods: subject-specific BCIs are tuned to the individual yielding the best performance at the cost of long, tiring training sessions making these methods unsuitable for everyday use; subject-independent BCIs that make use of training data from various subjects offer a good trade-off between training effort and performance, making these BCIs better suited for practical use.

Enhancing P300-BCI performance using latency estimation

Brain Computer Interfaces (BCIs) offer restoration of communication to those with the most severe movement impairments, but performance is not yet ideal. Previous work has demonstrated that latency jitter, the variation in timing of the brain responses, plays a critical role in determining BCI performance. In this study, we used Classifier-Based Latency Estimation (CBLE) and a wavelet transform to provide information about latency jitter to a second-level classifier. Three second-level classifiers were tested: least squares (LS), step-wise linear discriminant analysis (SWLDA), and support vector machine (SVM). Of these three, LS and SWLDA performed better than the original online classifier. The resulting combination demonstrated improved detection of brain responses for many participants, resulting in better BCI performance. Interestingly, the performance gain was greatest for those individuals for whom the BCI did not work well online, indicating that this method may be most suitable for improving performance of otherwise marginal participants.

Learning from label proportions in brain-computer interfaces: Online unsupervised learning with guarantees

Objective

Using traditional approaches, a brain-computer interface (BCI) requires the collection of calibration data for new subjects prior to online use. Calibration time can be reduced or eliminated e.g., by subject-to-subject transfer of a pre-trained classifier or unsupervised adaptive classification methods which learn from scratch and adapt over time. While such heuristics work well in practice, none of them can provide theoretical guarantees. Our objective is to modify an event-related potential (ERP) paradigm to work in unison with the machine learning decoder, and thus to achieve a reliable unsupervised calibrationless decoding with a guarantee to recover the true class means.

Method

We introduce learning from label proportions (LLP) to the BCI community as a new unsupervised, and easy-to-implement classification approach for ERP-based BCIs. The LLP estimates the mean target and non-target responses based on known proportions of these two classes in different groups of the data. We present a visual ERP speller to meet the requirements of LLP. For evaluation, we ran simulations on artificially created data sets and conducted an online BCI study with 13 subjects performing a copy-spelling task.

Results

Theoretical considerations show that LLP is guaranteed to minimize the loss function similar to a corresponding supervised classifier. LLP performed well in simulations and in the online application, where 84.5% of characters were spelled correctly on average without prior calibration.

Significance

The continuously adapting LLP classifier is the first unsupervised decoder for ERP BCIs guaranteed to find the optimal decoder. This makes it an ideal solution to avoid tedious calibration sessions. Additionally, LLP works on complementary principles compared to existing unsupervised methods, opening the door for their further enhancement when combined with LLP.

Improving zero-training brain-computer interfaces by mixing model estimators

OBJECTIVE

Brain-computer interfaces (BCI) based on event-related potentials (ERP) incorporate a decoder to classify recorded brain signals and subsequently select a control signal that drives a computer application. Standard supervised BCI decoders require a tedious calibration procedure prior to every session. Several unsupervised classification methods have been proposed that tune the decoder during actual use and as such omit this calibration. Each of these methods has its own strengths and weaknesses. Our aim is to improve overall accuracy of ERP-based BCIs without calibration.

APPROACH

We consider two approaches for unsupervised classification of ERP signals. Learning from label proportions (LLP) was recently shown to be guaranteed to converge to a supervised decoder when enough data is available. In contrast, the formerly proposed expectation maximization (EM) based decoding for ERP-BCI does not have this guarantee. However, while this decoder has high variance due to random initialization of its parameters, it obtains a higher accuracy faster than LLP when the initialization is good. We introduce a method to optimally combine these two unsupervised decoding methods, letting one method's strengths compensate for the weaknesses of the other and vice versa. The new method is compared to the aforementioned methods in a resimulation of an experiment with a visual speller.

MAIN RESULTS

Analysis of the experimental results shows that the new method exceeds the performance of the previous unsupervised classification approaches in terms of ERP classification accuracy and symbol selection accuracy during the spelling experiment. Furthermore, the method shows less dependency on random initialization of model parameters and is consequently more reliable.

SIGNIFICANCE

Improving the accuracy and subsequent reliability of calibrationless BCIs makes these systems more appealing for frequent use.

Heading for new shores! Overcoming pitfalls in BCI design

Research in brain-computer interfaces has achieved impressive progress towards implementing assistive technologies for restoration or substitution of lost motor capabilities, as well as supporting technologies for able-bodied subjects. Notwithstanding this progress, effective translation of these interfaces from proof-of concept prototypes into reliable applications remains elusive. As a matter of fact, most of the current BCI systems cannot be used independently for long periods of time by their intended end-users. Multiple factors that impair achieving this goal have already been identified. However, it is not clear how do they affect the overall BCI performance or how they should be tackled. This is worsened by the publication bias where only positive results are disseminated, preventing the research community from learning from its errors. This paper is the result of a workshop held at the 6th International BCI meeting in Asilomar. We summarize here the discussion on concrete research avenues and guidelines that may help overcoming common pitfalls and make BCIs become a useful alternative communication device.

A Fully Automated Trial Selection Method for Optimization of Motor Imagery Based Brain-Computer Interface

Independent component analysis (ICA) as a promising spatial filtering method can separate motor-related independent components (MRICs) from the multichannel electroencephalogram (EEG) signals. However, the unpredictable burst interferences may significantly degrade the performance of ICA-based brain-computer interface (BCI) system. In this study, we proposed a new algorithm frame to address this issue by combining the single-trial-based ICA filter with zero-training classifier. We developed a two-round data selection method to identify automatically the badly corrupted EEG trials in the training set. The "high quality" training trials were utilized to optimize the ICA filter. In addition, we proposed an accuracy-matrix method to locate the artifact data segments within a single trial and investigated which types of artifacts can influence the performance of the ICA-based MIBCIs. Twenty-six EEG datasets of three-class motor imagery were used to validate the proposed methods, and the classification accuracies were compared with that obtained by frequently used common spatial pattern (CSP) spatial filtering algorithm. The experimental results demonstrated that the proposed optimizing strategy could effectively improve the stability, practicality and classification performance of ICA-based MIBCI. The study revealed that rational use of ICA method may be crucial in building a practical ICA-based MIBCI system.

Integrating language models into classifiers for BCI communication: a review

OBJECTIVE

The present review systematically examines the integration of language models to improve classifier performance in brain-computer interface (BCI) communication systems.

APPROACH

The domain of natural language has been studied extensively in linguistics and has been used in the natural language processing field in applications including information extraction, machine translation, and speech recognition. While these methods have been used for years in traditional augmentative and assistive communication devices, information about the output domain has largely been ignored in BCI communication systems. Over the last few years, BCI communication systems have started to leverage this information through the inclusion of language models.

MAIN RESULTS

Although this movement began only recently, studies have already shown the potential of language integration in BCI communication and it has become a growing field in BCI research. BCI communication systems using language models in their classifiers have progressed down several parallel paths, including: word completion; signal classification; integration of process models; dynamic stopping; unsupervised learning; error correction; and evaluation.

SIGNIFICANCE

Each of these methods have shown significant progress, but have largely been addressed separately. Combining these methods could use the full potential of language model, yielding further performance improvements. This integration should be a priority as the field works to create a BCI system that meets the needs of the amyotrophic lateral sclerosis population.

Decoding motor responses from the EEG during altered states of consciousness induced by propofol

OBJECTIVE

Patients undergoing general anesthesia may awaken and become aware of the surgical procedure. Due to neuromuscular blocking agents, patients could be conscious yet unable to move. Using brain-computer interface (BCI) technology, it may be possible to detect movement attempts from the EEG. However, it is unknown how an anesthetic influences the brain response to motor tasks.

APPROACH

We tested the offline classification performance of a movement-based BCI in 12 healthy subjects at two effect-site concentrations of propofol. For each subject a second classifier was trained on the subject's data obtained before sedation, then tested on the data obtained during sedation ('transfer classification').

MAIN RESULTS

At concentration 0.5 μg ml(-1), despite an overall propofol EEG effect, the mean single trial classification accuracy was 85% (95% CI 81%-89%), and 83% (79%-88%) for the transfer classification. At 1.0 μg ml(-1), the accuracies were 81% (76%-86%), and 72% (66%-79%), respectively. At the highest propofol concentration for four subjects, unlike the remaining subjects, the movement-related brain response had been largely diminished, and the transfer classification accuracy was not significantly above chance. These subjects showed a slower and more erratic task response, indicating an altered state of consciousness distinct from that of the other subjects.

SIGNIFICANCE

The results show the potential of using a BCI to detect intra-operative awareness and justify further development of this paradigm. At the same time, the relationship between motor responses and consciousness and its clinical relevance for intraoperative awareness requires further investigation.

Autonomous Parameter Adjustment for SSVEP-Based BCIs with a Novel BCI Wizard

Brain-Computer Interfaces (BCIs) transfer human brain activities into computer commands and enable a communication channel without requiring movement. Among other BCI approaches, steady-state visual evoked potential (SSVEP)-based BCIs have the potential to become accurate, assistive technologies for persons with severe disabilities. Those systems require customization of different kinds of parameters (e.g., stimulation frequencies). Calibration usually requires selecting predefined parameters by experienced/trained personnel, though in real-life scenarios an interface allowing people with no experience in programming to set up the BCI would be desirable. Another occurring problem regarding BCI performance is BCI illiteracy (also called BCI deficiency). Many articles reported that BCI control could not be achieved by a non-negligible number of users. In order to bypass those problems we developed a SSVEP-BCI wizard, a system that automatically determines user-dependent key-parameters to customize SSVEP-based BCI systems. This wizard was tested and evaluated with 61 healthy subjects. All subjects were asked to spell the phrase "RHINE WAAL UNIVERSITY" with a spelling application after key parameters were determined by the wizard. Results show that all subjects were able to control the spelling application. A mean (SD) accuracy of 97.14 (3.73)% was reached (all subjects reached an accuracy above 85% and 25 subjects even reached 100% accuracy).

Classifying the auditory P300 using mobile EEG recordings without calibration phase

One of the major drawbacks in mobile EEG Brain Computer Interfaces (BCI) is the need for subject specific training data to train a classifier. By removing the need for supervised classification and calibration phase, new users could start immediate interaction with a BCI. We propose a solution to exploit the structural difference by means of canonical polyadic decomposition (CPD) for three-class auditory oddball data without the need for subject-specific information. We achieve this by adding average event-related-potential (ERP) templates to the CPD model. This constitutes a novel similarity measure between single-trial pairs and known-templates, which results in a fast and interpretable classifier. These results have similar accuracy to those of the supervised and cross-validated stepwise LDA approach but without the need for having subject-dependent data. Therefore the described CPD method has a significant practical advantage over the traditional and widely used approach.

Exploiting Task Constraints for Self-Calibrated Brain-Machine Interface Control Using Error-Related Potentials

This paper presents a new approach for self-calibration BCI for reaching tasks using error-related potentials. The proposed method exploits task constraints to simultaneously calibrate the decoder and control the device, by using a robust likelihood function and an ad-hoc planner to cope with the large uncertainty resulting from the unknown task and decoder. The method has been evaluated in closed-loop online experiments with 8 users using a previously proposed BCI protocol for reaching tasks over a grid. The results show that it is possible to have a usable BCI control from the beginning of the experiment without any prior calibration. Furthermore, comparisons with simulations and previous results obtained using standard calibration hint that both the quality of recorded signals and the performance of the system were comparable to those obtained with a standard calibration approach.

Decoding the attended speech stream with multi-channel EEG: implications for online, daily-life applications

OBJECTIVE

Recent studies have provided evidence that temporal envelope driven speech decoding from high-density electroencephalography (EEG) and magnetoencephalography recordings can identify the attended speech stream in a multi-speaker scenario. The present work replicated the previous high density EEG study and investigated the necessary technical requirements for practical attended speech decoding with EEG.

APPROACH

Twelve normal hearing participants attended to one out of two simultaneously presented audiobook stories, while high density EEG was recorded. An offline iterative procedure eliminating those channels contributing the least to decoding provided insight into the necessary channel number and optimal cross-subject channel configuration. Aiming towards the future goal of near real-time classification with an individually trained decoder, the minimum duration of training data necessary for successful classification was determined by using a chronological cross-validation approach.

MAIN RESULTS

Close replication of the previously reported results confirmed the method robustness. Decoder performance remained stable from 96 channels down to 25. Furthermore, for less than 15 min of training data, the subject-independent (pre-trained) decoder performed better than an individually trained decoder did.

SIGNIFICANCE

Our study complements previous research and provides information suggesting that efficient low-density EEG online decoding is within reach.