Estimating person-specific neural correlates of mental rotation: A machine learning approach

Using neurophysiological measures to model how the brain performs complex cognitive tasks such as mental rotation is a promising way towards precise predictions of behavioural responses. The mental rotation task requires objects to be mentally rotated in space. It has been used to monitor progressive neurological disorders. Up until now, research on neural correlates of mental rotation have largely focused on group analyses yielding models with features common across individuals. Here, we propose an individually tailored machine learning approach to identify person-specific patterns of neural activity during mental rotation. We trained ridge regressions to predict the reaction time of correct responses in a mental rotation task using task-related, electroencephalographic (EEG) activity of the same person. When tested on independent data of the same person, the regression model predicted the reaction times significantly more accurately than when only the average reaction time was used for prediction (bootstrap mean difference of 0.02, 95% CI: 0.01-0.03, p < .001). When tested on another person's data, the predictions were significantly less accurate compared to within-person predictions. Further analyses revealed that considering person-specific reaction times and topographical activity patterns substantially improved a model's generalizability. Our results indicate that a more individualized approach towards neural correlates can improve their predictive performance of behavioural responses, particularly when combined with machine learning.

Perspectives of Implementation of Closed-Loop Deep Brain Stimulation: From Neurological to Psychiatric Disorders

BACKGROUND

Deep brain stimulation (DBS) is a highly efficient, evidence-based therapy to alleviate symptoms and improve quality of life in movement disorders such as Parkinson's disease, essential tremor, and dystonia, which is also being applied in several psychiatric disorders, such as obsessive-compulsive disorder and depression, when they are otherwise resistant to therapy.

SUMMARY

At present, DBS is clinically applied in the so-called open-loop approach, with fixed stimulation parameters, irrespective of the patients' clinical state(s). This approach ignores the brain states or feedback from the central nervous system or peripheral recordings, thus potentially limiting its efficacy and inducing side effects by stimulation of the targeted networks below or above the therapeutic level.

KEY MESSAGES

The currently emerging closed-loop (CL) approaches are designed to adapt stimulation parameters to the electrophysiological surrogates of disease symptoms and states. CL-DBS paves the way for adaptive personalized DBS protocols. This review elaborates on the perspectives of the CL technology and discusses its opportunities as well as its potential pitfalls for both clinical and research use in neuropsychiatric disorders.

Introducing block-Toeplitz covariance matrices to remaster linear discriminant analysis for event-related potential brain-computer interfaces

Objective.Covariance matrices of noisy multichannel electroencephalogram (EEG) time series data provide essential information for the decoding of brain signals using machine learning methods. However, small datasets and high dimensionality make it hard to estimate these matrices. In brain-computer interfaces (BCI) based on event-related potentials (ERP) and a linear discriminant analysis (LDA) classifier, the state of the art covariance estimation uses shrinkage regularization. As this is a general covariance regularization approach, we aim at improving LDA further by better exploiting the domain-specific characteristics of the EEG to regularize the covariance estimates.Approach.We propose to enforce a block-Toeplitz structure for the covariance matrix of the LDA, which implements an assumption of signal stationarity in short time windows for each channel.Main results.An offline re-analysis of data collected from 213 subjects under 13 different event-related potential BCI protocols showed a significantly increased binary classification performance of this 'ToeplitzLDA' compared to shrinkage regularized LDA (up to 6 AUC points,p < 0.001) and Riemannian classification approaches (up to 2 AUC points,p < 0.001). In an unsupervised visual speller application, this improvement would translate to a relative reduction of spelling errors by 81% on average for 25 subjects. Additionally, aside from lower memory and reduced time complexity for LDA training, ToeplitzLDA proves to be robust against drastic increases of the number of temporal features.Significance.The proposed covariance estimation allows BCI researchers to improve classification rates and reduce calibration times of BCI protocols using event-related potentials and thus support the usability of corresponding applications. Its lower computational and memory needs could make it a valuable algorithm especially for mobile BCIs.

Manipulating Single-Trial Motor Performance in Chronic Stroke Patients by Closed-Loop Brain State Interaction

Motor impaired patients performing repetitive motor tasks often reveal large single-trial performance variations. Based on a data-driven framework, we extracted robust oscillatory brain states from pre-trial intervals, which are predictive for the upcoming motor performance on the level of single trials. Based on the brain state estimate, i.e. whether the brain state predicts a good or bad upcoming performance, we implemented a novel gating strategy for the start of trials by selecting specifically suitable or unsuitable trial starting time points. In a pilot study with four chronic stroke patients with hand motor impairments, we conducted a total of 41 sessions. After few initial calibration sessions, patients completed approximately 15 hours of effective hand motor training during eight online sessions using the gating strategy. Patients' reaction times were significantly reduced for suitable trials compared to unsuitable trials and shorter overall trial durations under suitable states were found in two patients. Overall, this successful proof-of-concept pilot study motivates to transfer this closed-loop training framework to a clinical study and to other application fields, such as cognitive rehabilitation, sport sciences or systems neuroscience.

Improving Covariance Matrices Derived from Tiny Training Datasets for the Classification of Event-Related Potentials with Linear Discriminant Analysis

Electroencephalogram data used in the domain of brain–computer interfaces typically has subpar signal-to-noise ratio and data acquisition is expensive. An effective and commonly used classifier to discriminate event-related potentials is the linear discriminant analysis which, however, requires an estimate of the feature distribution. While this information is provided by the feature covariance matrix its large number of free parameters calls for regularization approaches like Ledoit–Wolf shrinkage. Assuming that the noise of event-related potential recordings is not time-locked, we propose to decouple the time component from the covariance matrix of event-related potential data in order to further improve the estimates of the covariance matrix for linear discriminant analysis. We compare three regularized variants thereof and a feature representation based on Riemannian geometry against our proposed novel linear discriminant analysis with time-decoupled covariance estimates. Extensive evaluations on 14 electroencephalogram datasets reveal, that the novel approach increases the classification performance by up to four percentage points for small training datasets, and gracefully converges to the performance of standard shrinkage-regularized LDA for large training datasets. Given these results, practitioners in this field should consider using our proposed time-decoupled covariance estimation when they apply linear discriminant analysis to classify event-related potentials, especially when few training data points are available.

A Pilot Study on Data-Driven Adaptive Deep Brain Stimulation in Chronically Implanted Essential Tremor Patients

Deep brain stimulation (DBS) is an established therapy for Parkinson's disease (PD) and essential-tremor (ET). In adaptive DBS (aDBS) systems, online tuning of stimulation parameters as a function of neural signals may improve treatment efficacy and reduce side-effects. State-of-the-art aDBS systems use symptom surrogates derived from neural signals-so-called neural markers (NMs)-defined on the patient-group level, and control strategies assuming stationarity of symptoms and NMs. We aim at improving these aDBS systems with (1) a data-driven approach for identifying patient- and session-specific NMs and (2) a control strategy coping with short-term non-stationary dynamics. The two building blocks are implemented as follows: (1) The data-driven NMs are based on a machine learning model estimating tremor intensity from electrocorticographic signals. (2) The control strategy accounts for local variability of tremor statistics. Our study with three chronically implanted ET patients amounted to five online sessions. Tremor quantified from accelerometer data shows that symptom suppression is at least equivalent to that of a continuous DBS strategy in 3 out-of 4 online tests, while considerably reducing net stimulation (at least 24%). In the remaining online test, symptom suppression was not significantly different from either the continuous strategy or the no treatment condition. We introduce a novel aDBS system for ET. It is the first aDBS system based on (1) a machine learning model to identify session-specific NMs, and (2) a control strategy coping with short-term non-stationary dynamics. We show the suitability of our aDBS approach for ET, which opens the door to its further study in a larger patient population.

Identifying controllable cortical neural markers with machine learning for adaptive deep brain stimulation in Parkinson's disease

The identification of oscillatory neural markers of Parkinson's disease (PD) can contribute not only to the understanding of functional mechanisms of the disorder, but may also serve in adaptive deep brain stimulation (DBS) systems. These systems seek online adaptation of stimulation parameters in closed-loop as a function of neural markers, aiming at improving treatment's efficacy and reducing side effects. Typically, the identification of PD neural markers is based on group-level studies. Due to the heterogeneity of symptoms across patients, however, such group-level neural markers, like the beta band power of the subthalamic nucleus, are not present in every patient or not informative about every patient's motor state. Instead, individual neural markers may be preferable for providing a personalized solution for the adaptation of stimulation parameters. Fortunately, data-driven bottom-up approaches based on machine learning may be utilized. These approaches have been developed and applied successfully in the field of brain-computer interfaces with the goal of providing individuals with means of communication and control. In our contribution, we present results obtained with a novel supervised data-driven identification of neural markers of hand motor performance based on a supervised machine learning model. Data of 16 experimental sessions obtained from seven PD patients undergoing DBS therapy show that the supervised patient-specific neural markers provide improved decoding accuracy of hand motor performance, compared to group-level neural markers reported in the literature. We observed that the individual markers are sensitive to DBS therapy and thus, may represent controllable variables in an adaptive DBS system.

A Robust Screen-Free Brain-Computer Interface for Robotic Object Selection

Brain signals represent a communication modality that can allow users of assistive robots to specify high-level goals, such as the object to fetch and deliver. In this paper, we consider a screen-free Brain-Computer Interface (BCI), where the robot highlights candidate objects in the environment using a laser pointer, and the user goal is decoded from the evoked responses in the electroencephalogram (EEG). Having the robot present stimuli in the environment allows for more direct commands than traditional BCIs that require the use of graphical user interfaces. Yet bypassing a screen entails less control over stimulus appearances. In realistic environments, this leads to heterogeneous brain responses for dissimilar objects-posing a challenge for reliable EEG classification. We model object instances as subclasses to train specialized classifiers in the Riemannian tangent space, each of which is regularized by incorporating data from other objects. In multiple experiments with a total of 19 healthy participants, we show that our approach not only increases classification performance but is also robust to both heterogeneous and homogeneous objects. While especially useful in the case of a screen-free BCI, our approach can naturally be applied to other experimental paradigms with potential subclass structure.

Real-time detection of neural oscillation bursts allows behaviourally relevant neurofeedback

Neural oscillations as important information carrier in the brain, are increasingly interpreted as transient bursts rather than as sustained oscillations. Short (<150 ms) bursts of beta-waves (15-30 Hz) have been documented in humans, monkeys and mice. These events were correlated with memory, movement and perception, and were even suggested as the primary ingredient of all beta-band activity. However, a method to measure these short-lived events in real-time and to investigate their impact on behaviour is missing. Here we present a real-time data analysis system, capable to detect short narrowband bursts, and demonstrate its usefulness to increase the beta-band burst-rate in rats. This neurofeedback training induced changes in overall oscillatory power, and bursts could be decoded from the movement of the rats, thus enabling future investigation of the role of oscillatory bursts.

Influence of User Tasks on EEG-based Classification Performance in a Hazard Detection Paradigm

Attention-based brain-computer interface (BCI) paradigms offer a way to exert control, but also to provide insight into a user's perception and judgment of the environment. For a sufficient classification performance, user engagement and motivation are critical aspects. Consequently, many paradigms require the user to perform an auxiliary task, such as mentally counting subsets of stimuli or pressing a button when encountering them. In this work, we compare two user tasks, mental counting and button-presses, in a hazard detection paradigm in driving videos. We find that binary classification performance of events based on the electroencephalogram as well as user preference are higher for button presses. Amplitudes of evoked responses are higher for the counting task-an observation which holds even after projecting out motor-related potentials during the data preprocessing. Our results indicate that the choice of button-presses can be a preferable choice in such BCIs based on prediction performance as well as user preference.

Two Player Online Brain-Controlled Chess

Brain-computer interfaces (BCIs) allow for translating brain signals into control commands, e.g. to control games. One of the biggest quests of the BCI community is to realize new exciting applications. In this paper, we present a two player online chess application where both players control their pieces solely with their brain activity. Control is realized in a two-step process where players first select the desired chess piece and then the field they want to move it to. A selection is accomplished with visual event-related potentials that are elicited by highlighting each possible piece or field one by one. Six healthy volunteers participated in our study by playing a game against each other in pairs over a free chess server. They successfully completed at least one match per pair. Our results show that even BCI-naive players obtain almost perfect control over the application. On average, 96% of the moves were correct. Most of the errors came from technical problems that can be corrected in future versions of the application. Given the high popularity of chess, this intuitive BCI game is an appealing new application for patients and for healthy users that want to explore BCIs.

A simulated environment for early development stages of reinforcement learning algorithms for closed-loop deep brain stimulation

In recent years, closed-loop adaptive deep brain stimulation (aDBS) for Parkinson's disease (PD) has gained focus in the research community, due to promising proof-of-concept studies showing its suitability for improving DBS therapy and ameliorating related side effects.The main challenges faced in the aDBS control problem is the presence of non-stationary/non-linear dynamics and the heterogeneity of PD's phenotype, making the exploration of data-driven dynamics-aware control algorithms a promising research direction. However, due to the severe safety constraints related to working with patients, aDBS is a sensitive research field that requires surrogate development platforms with growing complexity, as novel control algorithms are validated.With our current contribution, we propose the characterization and categorization of non-stationary dynamics found in the aDBS problem. We show how knowledge about these dynamics can be embedded in a surrogate simulation environment, which has been designed to support early development stages of aDBS control strategies, specifically those based on reinforcement learning (RL) algorithms. Finally, we present a comparison of representative RL methods designed to cope with the type of non-stationary dynamics found in aDBS.To allow reproducibility and encourage adoption of our approach, the source code of the developed methods and simulation environment are made available online.

Extremely Reduced Data Sets Indicate Optimal Stimulation Parameters for Classification in Brain-Computer Interfaces

The time between the onset of subsequent auditory or visual stimuli - also known as stimulus onset asynchrony (SOA) - determines many of the event-related potential characteristics of the resulting evoked brain signals. Specifically, the SOA value influences the performance of an individual subject in brain-computer interface (BCI) applications like spellers. In the past, subject-specific optimization of the SOA was rarely considered in BCI studies. Our research strives to reduce the time requirements of individual BCI stimulus parameter optimization. This work contributes to this goal in two ways. First, we show that even the classification performance on extremely reduced training data subsets reveals the influence of SOA. Second, we show, that these noisy estimates are sufficient to make decisions for individual choices of the SOA that transfer to good classification performance on large training data sets. Thus our work contributes to individually tailored SOA selection procedures for BCI users.

Workshops of the Seventh International Brain-Computer Interface Meeting: Not Getting Lost in Translation

The Seventh International Brain-Computer Interface (BCI) Meeting was held May 21-25th, 2018 at the Asilomar Conference Grounds, Pacific Grove, California, United States. The interactive nature of this conference was embodied by 25 workshops covering topics in BCI (also called brain-machine interface) research. Workshops covered foundational topics such as hardware development and signal analysis algorithms, new and imaginative topics such as BCI for virtual reality and multi-brain BCIs, and translational topics such as clinical applications and ethical assumptions of BCI development. BCI research is expanding in the diversity of applications and populations for whom those applications are being developed. BCI applications are moving toward clinical readiness as researchers struggle with the practical considerations to make sure that BCI translational efforts will be successful. This paper summarizes each workshop, providing an overview of the topic of discussion, references for additional information, and identifying future issues for research and development that resulted from the interactions and discussion at the workshop.

An Easy-to-Use and Fast Assessment of Patient-Specific DBS-Induced Changes in Hand Motor Control in Parkinson's Disease

For Parkinson's disease (PD), efficient and fast monitoring of fine motor function is fundamental for capturing transient phenomena induced by deep brain stimulation (DBS), thus, enabling a fast and accurate tuning of stimulation parameters. Tuning of DBS parameters is important for obtaining a patient-specific optimal clinical effect and to regularly compensate for disease progress. We propose a fine motor function assessment framework for capturing transient DBS-induced changes. The main goals are to obtain a fast, repeatable, objective, robust, and DBS-sensitive motor-score, in addition to a high-dimensional characterization of motor components by means of an interpretable data-driven model. To achieve this, we combine a hand motor-task, termed the copy-draw test, with a linear model for analyzing features extracted from the proposed task. The approach was tested with four patients totaling eight sessions analyzed. Our approach delivers a motor-score that is sensitive to DBS-induced changes in motor function. It can be applied repeatedly within seconds. The interpretability of the underlying machine learning model provides a direct overview of the feature relevance. This analysis allows to detect and characterize single movement components that are sensitive to DBS. The proposed assessment framework is an useful tool to push forward the data-driven identification of PD-relevant neural markers. Consequent to this end, the source code of the paradigm is made publicly available.

Characterizing Regularization Techniques for Spatial Filter Optimization in Oscillatory EEG Regression Problems : Guidelines Derived from Simulation and Real-World Data

We report on novel supervised algorithms for single-trial brain state decoding. Their reliability and robustness are essential to efficiently perform neurotechnological applications in closed-loop. When brain activity is assessed by multichannel recordings, spatial filters computed by the source power comodulation (SPoC) algorithm allow identifying oscillatory subspaces. They regress to a known continuous trial-wise variable reflecting, e.g. stimulus characteristics, cognitive processing or behavior. In small dataset scenarios, this supervised method tends to overfit to its training data as the involved recordings via electroencephalogram (EEG), magnetoencephalogram or local field potentials generally provide a low signal-to-noise ratio. To improve upon this, we propose and characterize three types of regularization techniques for SPoC: approaches using Tikhonov regularization (which requires model selection via cross-validation), combinations of Tikhonov regularization and covariance matrix normalization as well as strategies exploiting analytical covariance matrix shrinkage. All proposed techniques were evaluated both in a novel simulation framework and on real-world data. Based on the simulation findings, we saw our expectations fulfilled, that SPoC regularization generally reveals the largest benefit for small training sets and under severe label noise conditions. Relevant for practitioners, we derived operating ranges of regularization hyperparameters for cross-validation based approaches and offer open source code. Evaluating all methods additionally on real-world data, we observed an improved regression performance mainly for datasets from subjects with initially poor performance. With this proof-of-concept paper, we provided a generalizable regularization framework for SPoC which may serve as a starting point for implementing advanced techniques in the future.

Post-hoc Labeling of Arbitrary M/EEG Recordings for Data-Efficient Evaluation of Neural Decoding Methods

Many cognitive, sensory and motor processes have correlates in oscillatory neural source activity, which is embedded as a subspace in the recorded brain signals. Decoding such processes from noisy magnetoencephalogram/electroencephalogram (M/EEG) signals usually requires data-driven analysis methods. The objective evaluation of such decoding algorithms on experimental raw signals, however, is a challenge: the amount of available M/EEG data typically is limited, labels can be unreliable, and raw signals often are contaminated with artifacts. To overcome some of these problems, simulation frameworks have been introduced which support the development of data-driven decoding algorithms and their benchmarking. For generating artificial brain signals, however, most of the existing frameworks make strong and partially unrealistic assumptions about brain activity. This limits the generalization of results observed in the simulation to real-world scenarios. In the present contribution, we show how to overcome several shortcomings of existing simulation frameworks. We propose a versatile alternative, which allows for an objective evaluation and benchmarking of novel decoding algorithms using real neural signals. It allows to generate comparatively large datasets with labels being deterministically recoverable from the arbitrary M/EEG recordings. A novel idea to generate these labels is central to this framework: we determine a subspace of the true M/EEG recordings and utilize it to derive novel labels. These labels contain realistic information about the oscillatory activity of some underlying neural sources. For two categories of subspace-defining methods, we showcase how such labels can be obtained-either by an exclusively data-driven approach (independent component analysis-ICA), or by a method exploiting additional anatomical constraints (minimum norm estimates-MNE). We term our framework post-hoc labeling of M/EEG recordings. To support the adoption of the framework by practitioners, we have exemplified its use by benchmarking three standard decoding methods-i.e., common spatial patterns (CSP), source power-comodulation (SPoC), and convolutional neural networks (ConvNets)-wrt. Varied dataset sizes, label noise, and label variability. Source code and data are made available to the reader for facilitating the application of our post-hoc labeling framework.

Mining Within-Trial Oscillatory Brain Dynamics to Address the Variability of Optimized Spatial Filters

Data-driven spatial filtering algorithms optimize scores, such as the contrast between two conditions to extract oscillatory brain signal components. Most machine learning approaches for the filter estimation, however, disregard within-trial temporal dynamics and are extremely sensitive to changes in training data and involved hyperparameters. This leads to highly variable solutions and impedes the selection of a suitable candidate for, e.g., neurotechnological applications. Fostering component introspection, we propose to embrace this variability by condensing the functional signatures of a large set of oscillatory components into homogeneous clusters, each representing specific within-trial envelope dynamics. The proposed method is exemplified by and evaluated on a complex hand force task with a rich within-trial structure. Based on electroencephalography data of 18 healthy subjects, we found that the components' distinct temporal envelope dynamics are highly subject-specific. On average, we obtained seven clusters per subject, which were strictly confined regarding their underlying frequency bands. As the analysis method is not limited to a specific spatial filtering algorithm, it could be utilized for a wide range of neurotechnological applications, e.g., to select and monitor functionally relevant features for brain-computer interface protocols in stroke rehabilitation.

Eyes-Closed Increases the Usability of Brain-Computer Interfaces Based on Auditory Event-Related Potentials

Recent research has demonstrated how brain-computer interfaces (BCI) based on auditory stimuli can be used for communication and rehabilitation. In these applications, users are commonly instructed to avoid eye movements while keeping their eyes open. This secondary task can lead to exhaustion and subjects may not succeed in suppressing eye movements. In this work, we investigate the option to use a BCI with eyes-closed. Twelve healthy subjects participated in a single electroencephalography (EEG) session where they were listening to a rapid stream of bisyllabic words while alternatively having their eyes open or closed. In addition, we assessed usability aspects for the two conditions with a questionnaire. Our analysis shows that eyes-closed does not reduce the number of eye artifacts and that event-related potential (ERP) responses and classification accuracies are comparable between both conditions. Importantly, we found that subjects expressed a significant general preference toward the eyes-closed condition and were also less tensed in that condition. Furthermore, switching between eyes-closed and eyes-open and vice versa is possible without a severe drop in classification accuracy. These findings suggest that eyes-closed should be considered as a viable alternative in auditory BCIs that might be especially useful for subjects with limited control over their eye movements.

Deep learning with convolutional neural networks for EEG decoding and visualization

Deep learning with convolutional neural networks (deep ConvNets) has revolutionized computer vision through end-to-end learning, that is, learning from the raw data. There is increasing interest in using deep ConvNets for end-to-end EEG analysis, but a better understanding of how to design and train ConvNets for end-to-end EEG decoding and how to visualize the informative EEG features the ConvNets learn is still needed. Here, we studied deep ConvNets with a range of different architectures, designed for decoding imagined or executed tasks from raw EEG. Our results show that recent advances from the machine learning field, including batch normalization and exponential linear units, together with a cropped training strategy, boosted the deep ConvNets decoding performance, reaching at least as good performance as the widely used filter bank common spatial patterns (FBCSP) algorithm (mean decoding accuracies 82.1% FBCSP, 84.0% deep ConvNets). While FBCSP is designed to use spectral power modulations, the features used by ConvNets are not fixed a priori. Our novel methods for visualizing the learned features demonstrated that ConvNets indeed learned to use spectral power modulations in the alpha, beta, and high gamma frequencies, and proved useful for spatially mapping the learned features by revealing the topography of the causal contributions of features in different frequency bands to the decoding decision. Our study thus shows how to design and train ConvNets to decode task-related information from the raw EEG without handcrafted features and highlights the potential of deep ConvNets combined with advanced visualization techniques for EEG-based brain mapping. Hum Brain Mapp 38:5391-5420, 2017. © 2017 Wiley Periodicals, Inc.

Learning from label proportions in brain-computer interfaces: Online unsupervised learning with guarantees

Objective

Using traditional approaches, a brain-computer interface (BCI) requires the collection of calibration data for new subjects prior to online use. Calibration time can be reduced or eliminated e.g., by subject-to-subject transfer of a pre-trained classifier or unsupervised adaptive classification methods which learn from scratch and adapt over time. While such heuristics work well in practice, none of them can provide theoretical guarantees. Our objective is to modify an event-related potential (ERP) paradigm to work in unison with the machine learning decoder, and thus to achieve a reliable unsupervised calibrationless decoding with a guarantee to recover the true class means.

Method

We introduce learning from label proportions (LLP) to the BCI community as a new unsupervised, and easy-to-implement classification approach for ERP-based BCIs. The LLP estimates the mean target and non-target responses based on known proportions of these two classes in different groups of the data. We present a visual ERP speller to meet the requirements of LLP. For evaluation, we ran simulations on artificially created data sets and conducted an online BCI study with 13 subjects performing a copy-spelling task.

Results

Theoretical considerations show that LLP is guaranteed to minimize the loss function similar to a corresponding supervised classifier. LLP performed well in simulations and in the online application, where 84.5% of characters were spelled correctly on average without prior calibration.

Significance

The continuously adapting LLP classifier is the first unsupervised decoder for ERP BCIs guaranteed to find the optimal decoder. This makes it an ideal solution to avoid tedious calibration sessions. Additionally, LLP works on complementary principles compared to existing unsupervised methods, opening the door for their further enhancement when combined with LLP.

Improving zero-training brain-computer interfaces by mixing model estimators

OBJECTIVE

Brain-computer interfaces (BCI) based on event-related potentials (ERP) incorporate a decoder to classify recorded brain signals and subsequently select a control signal that drives a computer application. Standard supervised BCI decoders require a tedious calibration procedure prior to every session. Several unsupervised classification methods have been proposed that tune the decoder during actual use and as such omit this calibration. Each of these methods has its own strengths and weaknesses. Our aim is to improve overall accuracy of ERP-based BCIs without calibration.

APPROACH

We consider two approaches for unsupervised classification of ERP signals. Learning from label proportions (LLP) was recently shown to be guaranteed to converge to a supervised decoder when enough data is available. In contrast, the formerly proposed expectation maximization (EM) based decoding for ERP-BCI does not have this guarantee. However, while this decoder has high variance due to random initialization of its parameters, it obtains a higher accuracy faster than LLP when the initialization is good. We introduce a method to optimally combine these two unsupervised decoding methods, letting one method's strengths compensate for the weaknesses of the other and vice versa. The new method is compared to the aforementioned methods in a resimulation of an experiment with a visual speller.

MAIN RESULTS

Analysis of the experimental results shows that the new method exceeds the performance of the previous unsupervised classification approaches in terms of ERP classification accuracy and symbol selection accuracy during the spelling experiment. Furthermore, the method shows less dependency on random initialization of model parameters and is consequently more reliable.

SIGNIFICANCE

Improving the accuracy and subsequent reliability of calibrationless BCIs makes these systems more appealing for frequent use.

Workshops of the Sixth International Brain-Computer Interface Meeting: brain-computer interfaces past, present, and future

The Sixth International Brain-Computer Interface (BCI) Meeting was held 30 May-3 June 2016 at the Asilomar Conference Grounds, Pacific Grove, California, USA. The conference included 28 workshops covering topics in BCI and brain-machine interface research. Topics included BCI for specific populations or applications, advancing BCI research through use of specific signals or technological advances, and translational and commercial issues to bring both implanted and non-invasive BCIs to market. BCI research is growing and expanding in the breadth of its applications, the depth of knowledge it can produce, and the practical benefit it can provide both for those with physical impairments and the general public. Here we provide summaries of each workshop, illustrating the breadth and depth of BCI research and highlighting important issues and calls for action to support future research and development.

On the influence of high-pass filtering on ICA-based artifact reduction in EEG-ERP

Standard artifact removal methods for electroencephalographic (EEG) signals are either based on Independent Component Analysis (ICA) or they regress out ocular activity measured at electrooculogram (EOG) channels. Successful ICA-based artifact reduction relies on suitable pre-processing. Here we systematically evaluate the effects of high-pass filtering at different frequencies. Offline analyses were based on event-related potential data from 21 participants performing a standard auditory oddball task and an automatic artifactual component classifier method (MARA). As a pre-processing step for ICA, high-pass filtering between 1-2 Hz consistently produced good results in terms of signal-to-noise ratio (SNR), single-trial classification accuracy and the percentage of `near-dipolar' ICA components. Relative to no artifact reduction, ICA-based artifact removal significantly improved SNR and classification accuracy. This was not the case for a regression-based approach to remove EOG artifacts.

Pre-Trial EEG-Based Single-Trial Motor Performance Prediction to Enhance Neuroergonomics for a Hand Force Task

We propose a framework for building electrophysiological predictors of single-trial motor performance variations, exemplified for SVIPT, a sequential isometric force control task suitable for hand motor rehabilitation after stroke. Electroencephalogram (EEG) data of 20 subjects with mean age of 53 years was recorded prior to and during 400 trials of SVIPT. They were executed within a single session with the non-dominant left hand, while receiving continuous visual feedback of the produced force trajectories. The behavioral data showed strong trial-by-trial performance variations for five clinically relevant metrics, which accounted for reaction time as well as for the smoothness and precision of the produced force trajectory. 18 out of 20 tested subjects remained after preprocessing and entered offline analysis. Source Power Comodulation (SPoC) was applied on EEG data of a short time interval prior to the start of each SVIPT trial. For 11 subjects, SPoC revealed robust oscillatory EEG subspace components, whose bandpower activity are predictive for the performance of the upcoming trial. Since SPoC may overfit to non-informative subspaces, we propose to apply three selection criteria accounting for the meaningfulness of the features. Across all subjects, the obtained components were spread along the frequency spectrum and showed a variety of spatial activity patterns. Those containing the highest level of predictive information resided in and close to the alpha band. Their spatial patterns resemble topologies reported for visual attention processes as well as those of imagined or executed hand motor tasks. In summary, we identified subject-specific single predictors that explain up to 36% of the performance fluctuations and may serve for enhancing neuroergonomics of motor rehabilitation scenarios.

Probing meaningfulness of oscillatory EEG components with bootstrapping, label noise and reduced training sets

As oscillatory components of the Electroencephalogram (EEG) and other electrophysiological signals may co-modulate in power with a target variable of interest (e.g. reaction time), data-driven supervised methods have been developed to automatically identify such components based on labeled example trials. Under conditions of challenging signal-to-noise ratio, high-dimensional data and small training sets, however, these methods may overfit to meaningless solutions. Examples are spatial filtering methods like Common Spatial Patterns (CSP) and Source Power Comodulation (SPoC). It is difficult for the practitioner to tell apart meaningful from arbitrary, random components. We propose three approaches to probe the robustness of extracted oscillatory components and show their application to both, simulated and EEG data recorded during a visually cued hand motor reaction time task.

Motor imagery for severely motor-impaired patients: evidence for brain-computer interfacing as superior control solution

Brain-Computer Interfaces (BCIs) strive to decode brain signals into control commands for severely handicapped people with no means of muscular control. These potential users of noninvasive BCIs display a large range of physical and mental conditions. Prior studies have shown the general applicability of BCI with patients, with the conflict of either using many training sessions or studying only moderately restricted patients. We present a BCI system designed to establish external control for severely motor-impaired patients within a very short time. Within only six experimental sessions, three out of four patients were able to gain significant control over the BCI, which was based on motor imagery or attempted execution. For the most affected patient, we found evidence that the BCI could outperform the best assistive technology (AT) of the patient in terms of control accuracy, reaction time and information transfer rate. We credit this success to the applied user-centered design approach and to a highly flexible technical setup. State-of-the art machine learning methods allowed the exploitation and combination of multiple relevant features contained in the EEG, which rapidly enabled the patients to gain substantial BCI control. Thus, we could show the feasibility of a flexible and tailorable BCI application in severely disabled users. This can be considered a significant success for two reasons: Firstly, the results were obtained within a short period of time, matching the tight clinical requirements. Secondly, the participating patients showed, compared to most other studies, very severe communication deficits. They were dependent on everyday use of AT and two patients were in a locked-in state. For the most affected patient a reliable communication was rarely possible with existing AT.

True zero-training brain-computer interfacing--an online study

Despite several approaches to realize subject-to-subject transfer of pre-trained classifiers, the full performance of a Brain-Computer Interface (BCI) for a novel user can only be reached by presenting the BCI system with data from the novel user. In typical state-of-the-art BCI systems with a supervised classifier, the labeled data is collected during a calibration recording, in which the user is asked to perform a specific task. Based on the known labels of this recording, the BCI's classifier can learn to decode the individual's brain signals. Unfortunately, this calibration recording consumes valuable time. Furthermore, it is unproductive with respect to the final BCI application, e.g. text entry. Therefore, the calibration period must be reduced to a minimum, which is especially important for patients with a limited concentration ability. The main contribution of this manuscript is an online study on unsupervised learning in an auditory event-related potential (ERP) paradigm. Our results demonstrate that the calibration recording can be bypassed by utilizing an unsupervised trained classifier, that is initialized randomly and updated during usage. Initially, the unsupervised classifier tends to make decoding mistakes, as the classifier might not have seen enough data to build a reliable model. Using a constant re-analysis of the previously spelled symbols, these initially misspelled symbols can be rectified posthoc when the classifier has learned to decode the signals. We compare the spelling performance of our unsupervised approach and of the unsupervised posthoc approach to the standard supervised calibration-based dogma for n = 10 healthy users. To assess the learning behavior of our approach, it is unsupervised trained from scratch three times per user. Even with the relatively low SNR of an auditory ERP paradigm, the results show that after a limited number of trials (30 trials), the unsupervised approach performs comparably to a classic supervised model.

Towards user-friendly spelling with an auditory brain-computer interface: the CharStreamer paradigm

Realizing the decoding of brain signals into control commands, brain-computer interfaces (BCI) aim to establish an alternative communication pathway for locked-in patients. In contrast to most visual BCI approaches which use event-related potentials (ERP) of the electroencephalogram, auditory BCI systems are challenged with ERP responses, which are less class-discriminant between attended and unattended stimuli. Furthermore, these auditory approaches have more complex interfaces which imposes a substantial workload on their users. Aiming for a maximally user-friendly spelling interface, this study introduces a novel auditory paradigm: "CharStreamer". The speller can be used with an instruction as simple as "please attend to what you want to spell". The stimuli of CharStreamer comprise 30 spoken sounds of letters and actions. As each of them is represented by the sound of itself and not by an artificial substitute, it can be selected in a one-step procedure. The mental mapping effort (sound stimuli to actions) is thus minimized. Usability is further accounted for by an alphabetical stimulus presentation: contrary to random presentation orders, the user can foresee the presentation time of the target letter sound. Healthy, normal hearing users (n = 10) of the CharStreamer paradigm displayed ERP responses that systematically differed between target and non-target sounds. Class-discriminant features, however, varied individually from the typical N1-P2 complex and P3 ERP components found in control conditions with random sequences. To fully exploit the sequential presentation structure of CharStreamer, novel data analysis approaches and classification methods were introduced. The results of online spelling tests showed that a competitive spelling speed can be achieved with CharStreamer. With respect to user rating, it clearly outperforms a control setup with random presentation sequences.

Integrating dynamic stopping, transfer learning and language models in an adaptive zero-training ERP speller

OBJECTIVE

Most BCIs have to undergo a calibration session in which data is recorded to train decoders with machine learning. Only recently zero-training methods have become a subject of study. This work proposes a probabilistic framework for BCI applications which exploit event-related potentials (ERPs). For the example of a visual P300 speller we show how the framework harvests the structure suitable to solve the decoding task by (a) transfer learning, (b) unsupervised adaptation, (c) language model and (d) dynamic stopping.

APPROACH

A simulation study compares the proposed probabilistic zero framework (using transfer learning and task structure) to a state-of-the-art supervised model on n = 22 subjects. The individual influence of the involved components (a)-(d) are investigated.

MAIN RESULTS

Without any need for a calibration session, the probabilistic zero-training framework with inter-subject transfer learning shows excellent performance--competitive to a state-of-the-art supervised method using calibration. Its decoding quality is carried mainly by the effect of transfer learning in combination with continuous unsupervised adaptation.

SIGNIFICANCE

A high-performing zero-training BCI is within reach for one of the most popular BCI paradigms: ERP spelling. Recording calibration data for a supervised BCI would require valuable time which is lost for spelling. The time spent on calibration would allow a novel user to spell 29 symbols with our unsupervised approach. It could be of use for various clinical and non-clinical ERP-applications of BCI.

Workshops of the Fifth International Brain-Computer Interface Meeting: Defining the Future

The Fifth International Brain-Computer Interface (BCI) Meeting met June 3-7th, 2013 at the Asilomar Conference Grounds, Pacific Grove, California. The conference included 19 workshops covering topics in brain-computer interface and brain-machine interface research. Topics included translation of BCIs into clinical use, standardization and certification, types of brain activity to use for BCI, recording methods, the effects of plasticity, special interest topics in BCIs applications, and future BCI directions. BCI research is well established and transitioning to practical use to benefit people with physical impairments. At the same time, new applications are being explored, both for people with physical impairments and beyond. Here we provide summaries of each workshop, illustrating the breadth and depth of BCI research and high-lighting important issues for future research and development.

User-centered design in brain-computer interfaces-a case study

OBJECTIVE

The array of available brain-computer interface (BCI) paradigms has continued to grow, and so has the corresponding set of machine learning methods which are at the core of BCI systems. The latter have evolved to provide more robust data analysis solutions, and as a consequence the proportion of healthy BCI users who can use a BCI successfully is growing. With this development the chances have increased that the needs and abilities of specific patients, the end-users, can be covered by an existing BCI approach. However, most end-users who have experienced the use of a BCI system at all have encountered a single paradigm only. This paradigm is typically the one that is being tested in the study that the end-user happens to be enrolled in, along with other end-users. Though this corresponds to the preferred study arrangement for basic research, it does not ensure that the end-user experiences a working BCI. In this study, a different approach was taken; that of a user-centered design. It is the prevailing process in traditional assistive technology. Given an individual user with a particular clinical profile, several available BCI approaches are tested and - if necessary - adapted to him/her until a suitable BCI system is found.

METHODS

Described is the case of a 48-year-old woman who suffered from an ischemic brain stem stroke, leading to a severe motor- and communication deficit. She was enrolled in studies with two different BCI systems before a suitable system was found. The first was an auditory event-related potential (ERP) paradigm and the second a visual ERP paradigm, both of which are established in literature.

RESULTS

The auditory paradigm did not work successfully, despite favorable preconditions. The visual paradigm worked flawlessly, as found over several sessions. This discrepancy in performance can possibly be explained by the user's clinical deficit in several key neuropsychological indicators, such as attention and working memory. While the auditory paradigm relies on both categories, the visual paradigm could be used with lower cognitive workload. Besides attention and working memory, several other neurophysiological and -psychological indicators - and the role they play in the BCIs at hand - are discussed.

CONCLUSION

The user's performance on the first BCI paradigm would typically have excluded her from further ERP-based BCI studies. However, this study clearly shows that, with the numerous paradigms now at our disposal, the pursuit for a functioning BCI system should not be stopped after an initial failed attempt.

How stimulation speed affects Event-Related Potentials and BCI performance

In most paradigms for Brain-Computer Interfaces (BCIs) that are based on Event-Related Potentials (ERPs), stimuli are presented with a pre-defined and constant speed. In order to boost BCI performance by optimizing the parameters of stimulation, this offline study investigates the impact of the stimulus onset asynchrony (SOA) on ERPs and the resulting classification accuracy. The SOA is defined as the time between the onsets of two consecutive stimuli, which represents a measure for stimulation speed. A simple auditory oddball paradigm was tested in 14 SOA conditions with a SOA between 50 ms and 1000 ms. Based on an offline ERP analysis, the BCI performance (quantified by the Information Transfer Rate, ITR in bits/min) was simulated. A great variability in the simulated BCI performance was observed within subjects (N=11). This indicates a potential increase in BCI performance (≥ 1.6 bits/min) for ERP-based paradigms, if the stimulation speed is specified for each user individually.

Optimizing event-related potential based brain-computer interfaces: a systematic evaluation of dynamic stopping methods

OBJECTIVE

In brain-computer interface (BCI) research, systems based on event-related potentials (ERP) are considered particularly successful and robust. This stems in part from the repeated stimulation which counteracts the low signal-to-noise ratio in electroencephalograms. Repeated stimulation leads to an optimization problem, as more repetitions also cost more time. The optimal number of repetitions thus represents a data-dependent trade-off between the stimulation time and the obtained accuracy. Several methods for dealing with this have been proposed as 'early stopping', 'dynamic stopping' or 'adaptive stimulation'. Despite their high potential for BCI systems at the patient's bedside, those methods are typically ignored in current BCI literature. The goal of the current study is to assess the benefit of these methods.

APPROACH

This study assesses for the first time the existing methods on a common benchmark of both artificially generated data and real BCI data of 83 BCI sessions, allowing for a direct comparison between these methods in the context of text entry.

MAIN RESULTS

The results clearly show the beneficial effect on the online performance of a BCI system, if the trade-off between the number of stimulus repetitions and accuracy is optimized. All assessed methods work very well for data of good subjects, and worse for data of low-performing subjects. Most methods, however, are robust in the sense that they do not reduce the performance below the baseline of a simple no stopping strategy.

SIGNIFICANCE

Since all methods can be realized as a module between the BCI and an application, minimal changes are needed to include these methods into existing BCI software architectures. Furthermore, the hyperparameters of most methods depend to a large extend on only a single variable-the discriminability of the training data. For the convenience of BCI practitioners, the present study proposes linear regression coefficients for directly estimating the hyperparameters from the data based on this discriminability. The data that were used in this publication are made publicly available to benchmark future methods.

Exploring the use of tactile feedback in an ERP-based auditory BCI

Giving direct, continuous feedback on a brain state is common practice in motor imagery based brain-computer interfaces (BCI), but has not been reported for BCIs based on event-related potentials (ERP), where feedback is only given once after a sequence of stimuli. Potentially, direct feedback could allow the user to adjust his strategy during a running trial to obtain the required response. In order to test the usefulness of such feedback, directionally congruent vibrotactile feedback was given during an online auditory BCI experiment. Users received either no feedback, short feedback pulses or continuous feedback. The feedback conditions showed reduced performance both on a behavioral task and in terms of classification accuracy. Several explanations are discussed that give interesting starting points for further research on this topic.