Learning from label proportions in brain-computer interfaces: Online unsupervised learning with guarantees

Introducing block-Toeplitz covariance matrices to remaster linear discriminant analysis for event-related potential brain-computer interfaces

Objective.Covariance matrices of noisy multichannel electroencephalogram (EEG) time series data provide essential information for the decoding of brain signals using machine learning methods. However, small datasets and high dimensionality make it hard to estimate these matrices. In brain-computer interfaces (BCI) based on event-related potentials (ERP) and a linear discriminant analysis (LDA) classifier, the state of the art covariance estimation uses shrinkage regularization. As this is a general covariance regularization approach, we aim at improving LDA further by better exploiting the domain-specific characteristics of the EEG to regularize the covariance estimates.Approach.We propose to enforce a block-Toeplitz structure for the covariance matrix of the LDA, which implements an assumption of signal stationarity in short time windows for each channel.Main results.An offline re-analysis of data collected from 213 subjects under 13 different event-related potential BCI protocols showed a significantly increased binary classification performance of this 'ToeplitzLDA' compared to shrinkage regularized LDA (up to 6 AUC points,p < 0.001) and Riemannian classification approaches (up to 2 AUC points,p < 0.001). In an unsupervised visual speller application, this improvement would translate to a relative reduction of spelling errors by 81% on average for 25 subjects. Additionally, aside from lower memory and reduced time complexity for LDA training, ToeplitzLDA proves to be robust against drastic increases of the number of temporal features.Significance.The proposed covariance estimation allows BCI researchers to improve classification rates and reduce calibration times of BCI protocols using event-related potentials and thus support the usability of corresponding applications. Its lower computational and memory needs could make it a valuable algorithm especially for mobile BCIs.

Language Model-Guided Classifier Adaptation for Brain-Computer Interfaces for Communication

Brain-computer interfaces (BCIs), such as the P300 speller, can provide a means of communication for individuals with severe neuromuscular limitations. BCIs interpret electroencephalography (EEG) signals in order to translate embedded information about a user's intent into executable commands to control external devices. However, EEG signals are inherently noisy and nonstationary, posing a challenge to extended BCI use. Conventionally, a BCI classifier is trained via supervised learning in an offline calibration session; once trained, the classifier is deployed for online use and is not updated. As the statistics of a user's EEG data change over time, the performance of a static classifier may decline with extended use. It is therefore desirable to automatically adapt the classifier to current data statistics without requiring offline recalibration. In an existing semi-supervised learning approach, the classifier is trained on labeled EEG data and is then updated using incoming unlabeled EEG data and classifier-predicted labels. To reduce the risk of learning from incorrect predictions, a threshold is imposed to exclude unlabeled data with low-confidence label predictions from the expanded training set when retraining the adaptive classifier. In this work, we propose the use of a language model for spelling error correction and disambiguation to provide information about label correctness during semi-supervised learning. Results from simulations with multi-session P300 speller user EEG data demonstrate that our language-guided semi-supervised approach significantly improves spelling accuracy relative to conventional BCI calibration and threshold-based semi-supervised learning.

Improving Covariance Matrices Derived from Tiny Training Datasets for the Classification of Event-Related Potentials with Linear Discriminant Analysis

Electroencephalogram data used in the domain of brain–computer interfaces typically has subpar signal-to-noise ratio and data acquisition is expensive. An effective and commonly used classifier to discriminate event-related potentials is the linear discriminant analysis which, however, requires an estimate of the feature distribution. While this information is provided by the feature covariance matrix its large number of free parameters calls for regularization approaches like Ledoit–Wolf shrinkage. Assuming that the noise of event-related potential recordings is not time-locked, we propose to decouple the time component from the covariance matrix of event-related potential data in order to further improve the estimates of the covariance matrix for linear discriminant analysis. We compare three regularized variants thereof and a feature representation based on Riemannian geometry against our proposed novel linear discriminant analysis with time-decoupled covariance estimates. Extensive evaluations on 14 electroencephalogram datasets reveal, that the novel approach increases the classification performance by up to four percentage points for small training datasets, and gracefully converges to the performance of standard shrinkage-regularized LDA for large training datasets. Given these results, practitioners in this field should consider using our proposed time-decoupled covariance estimation when they apply linear discriminant analysis to classify event-related potentials, especially when few training data points are available.

Error Correction Regression Framework for Enhancing the Decoding Accuracies of Ear-EEG Brain-Computer Interfaces

Ear-electroencephalography (EEG) is a promising tool for practical brain-computer interface (BCI) applications because it is more unobtrusive, comfortable, and mobile than a typical scalp-EEG system. However, an ear-EEG has a natural constraint of electrode location (e.g., limited in or around the ear) for acquiring informative brain signals sufficiently. Achieving reliable performance of ear-EEG in specific BCI paradigms that do not utilize brain signals on the temporal lobe around the ear is difficult. For example, steady-state visual evoked potentials (SSVEPs), which are mainly generated in the occipital area, have a significantly attenuated and distorted amplitude in ear-EEG. Therefore, preserving the high level of decoding accuracy is challenging and essential for SSVEP BCI based on ear-EEG. In this paper, we first investigate linear and nonlinear regression methods to increase the decoding accuracy of ear-EEG regarding SSVEP paradigm by utilizing the estimated target EEG signals on the occipital area. Then, we investigate an ensemble method to consider the prediction variability of the regression methods. Finally, we propose an error correction regression (ECR) framework to reduce the prediction errors by adding an additional nonlinear regression process (i.e., kernel ridge regression). We evaluate the ECR framework in terms of single session, session-to-session transfer, and subject-transfer decoding. We also validate the online decoding ability of the proposed framework with a short-time window size. The average accuracies are observed to be 91.11±9.14%, 90.52±8.67%, 86.96±12.13%, and 78.79±12.59%. This paper demonstrates that SSVEP BCI based on ear-EEG can achieve reliable performance with the proposed ECR framework.

The combination of brain-computer interfaces and artificial intelligence: applications and challenges

Brain-computer interfaces (BCIs) have shown great prospects as real-time bidirectional links between living brains and actuators. Artificial intelligence (AI), which can advance the analysis and decoding of neural activity, has turbocharged the field of BCIs. Over the past decade, a wide range of BCI applications with AI assistance have emerged. These "smart" BCIs including motor and sensory BCIs have shown notable clinical success, improved the quality of paralyzed patients' lives, expanded the athletic ability of common people and accelerated the evolution of robots and neurophysiological discoveries. However, despite technological improvements, challenges remain with regard to the long training periods, real-time feedback, and monitoring of BCIs. In this article, the authors review the current state of AI as applied to BCIs and describe advances in BCI applications, their challenges and where they could be headed in the future.

A Robust Screen-Free Brain-Computer Interface for Robotic Object Selection

Brain signals represent a communication modality that can allow users of assistive robots to specify high-level goals, such as the object to fetch and deliver. In this paper, we consider a screen-free Brain-Computer Interface (BCI), where the robot highlights candidate objects in the environment using a laser pointer, and the user goal is decoded from the evoked responses in the electroencephalogram (EEG). Having the robot present stimuli in the environment allows for more direct commands than traditional BCIs that require the use of graphical user interfaces. Yet bypassing a screen entails less control over stimulus appearances. In realistic environments, this leads to heterogeneous brain responses for dissimilar objects-posing a challenge for reliable EEG classification. We model object instances as subclasses to train specialized classifiers in the Riemannian tangent space, each of which is regularized by incorporating data from other objects. In multiple experiments with a total of 19 healthy participants, we show that our approach not only increases classification performance but is also robust to both heterogeneous and homogeneous objects. While especially useful in the case of a screen-free BCI, our approach can naturally be applied to other experimental paradigms with potential subclass structure.

General principles of machine learning for brain-computer interfacing

Brain-computer interfaces (BCIs) are systems that translate brain activity patterns into commands that can be executed by an artificial device. This enables the possibility of controlling devices such as a prosthetic arm or exoskeleton, a wheelchair, typewriting applications, or games directly by modulating our brain activity. For this purpose, BCI systems rely on signal processing and machine learning algorithms to decode the brain activity. This chapter provides an overview of the main steps required to do such a process, including signal preprocessing, feature extraction and selection, and decoding. Given the large amount of possible methods that can be used for these processes, a comprehensive review of them is beyond the scope of this chapter, and it is focused instead on the general principles that should be taken into account, as well as discussing good practices on how these methods should be applied and evaluated for proper design of reliable BCI systems.

Two Player Online Brain-Controlled Chess

Brain-computer interfaces (BCIs) allow for translating brain signals into control commands, e.g. to control games. One of the biggest quests of the BCI community is to realize new exciting applications. In this paper, we present a two player online chess application where both players control their pieces solely with their brain activity. Control is realized in a two-step process where players first select the desired chess piece and then the field they want to move it to. A selection is accomplished with visual event-related potentials that are elicited by highlighting each possible piece or field one by one. Six healthy volunteers participated in our study by playing a game against each other in pairs over a free chess server. They successfully completed at least one match per pair. Our results show that even BCI-naive players obtain almost perfect control over the application. On average, 96% of the moves were correct. Most of the errors came from technical problems that can be corrected in future versions of the application. Given the high popularity of chess, this intuitive BCI game is an appealing new application for patients and for healthy users that want to explore BCIs.

Workshops of the Seventh International Brain-Computer Interface Meeting: Not Getting Lost in Translation

The Seventh International Brain-Computer Interface (BCI) Meeting was held May 21-25th, 2018 at the Asilomar Conference Grounds, Pacific Grove, California, United States. The interactive nature of this conference was embodied by 25 workshops covering topics in BCI (also called brain-machine interface) research. Workshops covered foundational topics such as hardware development and signal analysis algorithms, new and imaginative topics such as BCI for virtual reality and multi-brain BCIs, and translational topics such as clinical applications and ethical assumptions of BCI development. BCI research is expanding in the diversity of applications and populations for whom those applications are being developed. BCI applications are moving toward clinical readiness as researchers struggle with the practical considerations to make sure that BCI translational efforts will be successful. This paper summarizes each workshop, providing an overview of the topic of discussion, references for additional information, and identifying future issues for research and development that resulted from the interactions and discussion at the workshop.

FBCSP-based Multi-class Motor Imagery Classification using BP and TDP features

Use of Motor Imagery in EEG signals is gaining importance to develop Brain Computer Interface (BCI) applications in various fields ranging from bio-medical to entertainment. Filter Bank Common Spatial Pattern (FBCSP) algorithm is a promising feature extraction technique to deal with subject-specific behavior in Motor Imagery classification. Using FBCSP on EEG we have developed an accurate but less computationally expensive approach by making use of Time Domain Parameters (TDP) and Band Power (BP) features to form a combined feature set. The novelty of our approach is also the use of optimal time segmentation to overcome non-stationary state behavior of Event-Related Desynchronization (ERD) and Event-Related Synchronization (ERS) over time. We analyzed the impact of parameter variations on classification accuracy and achieved 0.59 mean kappa value for Dataset 2a BCI competition IV, the highest reported for FBCSP approaches, along with the lowest inter-subject variation.