Latency correction of error potentials between different experiments reduces calibration time for single-trial classification

A generic error-related potential classifier based on simulated subjects

Error-related potentials (ErrPs) are brain signals known to be generated as a reaction to erroneous events. Several works have shown that not only self-made errors but also mistakes generated by external agents can elicit such event-related potentials. The possibility of reliably measuring ErrPs through non-invasive techniques has increased the interest in the brain-computer interface (BCI) community in using such signals to improve performance, for example, by performing error correction. Extensive calibration sessions are typically necessary to gather sufficient trials for training subject-specific ErrP classifiers. This procedure is not only time-consuming but also boresome for participants. In this paper, we explore the effectiveness of ErrPs in closed-loop systems, emphasizing their dependency on precise single-trial classification. To guarantee the presence of an ErrPs signal in the data we employ and to ensure that the parameters defining ErrPs are systematically varied, we utilize the open-source toolbox SEREEGA for data simulation. We generated training instances and evaluated the performance of the generic classifier on both simulated and real-world datasets, proposing a promising alternative to conventional calibration techniques. Results show that a generic support vector machine classifier reaches balanced accuracies of 72.9%, 62.7%, 71.0%, and 70.8% on each validation dataset. While performing similarly to a leave-one-subject-out approach for error class detection, the proposed classifier shows promising generalization across different datasets and subjects without further adaptation. Moreover, by utilizing SEREEGA, we can systematically adjust parameters to accommodate the variability in the ErrP, facilitating the systematic validation of closed-loop setups. Furthermore, our objective is to develop a universal ErrP classifier that captures the signal's variability, enabling it to determine the presence or absence of an ErrP in real EEG data.

Single trial detection of error-related potentials in brain-machine interfaces: a survey and comparison of methods

Objective.Error-related potential (ErrP) is a potential elicited in the brain when humans perceive an error. ErrPs have been researched in a variety of contexts, such as to increase the reliability of brain-computer interfaces (BCIs), increase the naturalness of human-machine interaction systems, teach systems, as well as study clinical conditions. Still, there is a significant challenge in detecting ErrP from a single trial, which may hamper its effective use. The literature presents ErrP detection accuracies quite variable across studies, which raises the question of whether this variability depends more on classification pipelines or on the quality of elicited ErrPs (mostly directly related to the underlying paradigms).Approach.With this purpose, 11 datasets have been used to compare several classification pipelines which were selected according to the studies that reported online performance above 75%. We also analyze the effects of different steps of the pipelines, such as resampling, window selection, augmentation, feature extraction, and classification.Main results.From our analysis, we have found that shrinkage-regularized linear discriminant analysis is the most robust method for classification, and for feature extraction, using Fisher criterion beamformer spatial features and overlapped window averages result in better classification performance. The overall experimental results suggest that classification accuracy is highly dependent on user tasks in BCI experiments and on signal quality (in terms of ErrP morphology, signal-to-noise ratio (SNR), and discrimination).Significance.This study contributes to the BCI research field by responding to the need for a guideline that can direct researchers in designing ErrP-based BCI tasks by accelerating the design steps.

Online detection of class-imbalanced error-related potentials evoked by motor imagery

Objective.Error-related potentials (ErrPs) are spontaneous electroencephalogram signals related to the awareness of erroneous responses within brain domain. ErrPs-based correction mechanisms can be applied to motor imagery-brain-computer interface (MI-BCI) to prevent incorrect actions and ultimately improve the performance of the hybrid BCI. Many studies on ErrPs detection are mostly conducted under offline conditions with poor classification accuracy and the error rates of ErrPs are preset in advance, which is too ideal to apply in realistic applications. In order to solve these problems, a novel method based on adaptive autoregressive (AAR) model and common spatial pattern (CSP) is proposed for ErrPs feature extraction. In addition, an adaptive threshold classification method based spectral regression discriminant analysis (SRDA) is suggested for class-unbalanced ErrPs data to reduce the false positives and false negatives.Approach.As for ErrPs feature extraction, the AAR coefficients in the temporal domain and CSP in the spatial domain are fused. Given that the performance of different subjects' MI tasks is different but stable, and the samples of ErrPs are class-imbalanced, an adaptive threshold based SRDA is suggested for classification. Two datasets are used in this paper. The open public clinical neuroprosthetics and brain interaction (CNBI) dataset is used to validate the performance of the proposed feature extraction algorithm and the real-time data recorded in our self-designed system is used to validate the performance of the proposed classification algorithm under class-imbalanced situations. Different from the pseudo-random paradigm, the ErrPs signals collected in our experiments are all elicited by four-class of online MI-BCI tasks, and the sample distribution is more natural and suitable for practical tests.Main results.The experimental results on the CNBI dataset show that the average accuracy and false positive rate for ErrPs detection are 94.1% and 8.1%, which outperforms methods using features extracted from a single domain. What's more, although the ErrPs induction rate is affected by the performance of subjects' MI-BCI tasks, experimental results on data recorded in the self-designed system prove that the ErrPs classification algorithm based on an adaptive threshold is robust under different ErrPs data distributions. Compared with two other methods, the proposed algorithm has advantages in all three measures which are accuracy, F1-score and false positive rate. Finally, ErrPs detection results were used to prevent wrong actions in a MI-BCI experiment, and it leads to a reduction of the hybrid BCI error rate from 48.9% to 24.3% in online tests.Significance.Both the AAR-CSP fused feature extraction and the adaptive threshold based SRDA classification methods suggested in our work are efficient in improving the ErrPs detection accuracy and reducing the false positives. In addition, by introducing ErrPs to multi-class MI-BCIs, the MI decoding results can be corrected after ErrPs are detected to avoid executing wrong instructions, thereby improving the BCI accuracy and lays the foundation for using MI-BCIs in practical applications.

Online asynchronous detection of error-related potentials in participants with a spinal cord injury using a generic classifier

For brain-computer interface (BCI) users, the awareness of an error is associated with a cortical signature known as an error-related potential (ErrP). The incorporation of ErrP detection into BCIs can improve their performance.

OBJECTIVE

This work has three main aims. First, we investigate whether an ErrP classifier is transferable from able-bodied participants to participants with a spinal cord injury (SCI). Second, we test this generic ErrP classifier with SCI and control participants, in an online experiment without offline calibration. Third, we investigate the morphology of ErrPs in both groups of participants.

APPROACH

We used previously recorded electroencephalographic data from able-bodied participants to train an ErrP classifier. We tested the classifier asynchronously, in an online experiment with 16 new participants: 8 participants with SCI and 8 able-bodied control participants. The experiment had no offline calibration and participants received feedback regarding the ErrP detections from the start. To increase the fluidity of the experiment, feedback regarding false positive ErrP detections was not presented to the participants, but these detections were taken into account in the evaluation of the classifier. The generic classifier was not trained with the user's brain signals. However, its performance was optimized during the online experiment by the use of personalized decision thresholds. The classifier's performance was evaluated using trial-based metrics, which considered the asynchronous detection of ErrPs during the entire trial's duration.

MAIN RESULTS

Participants with SCI presented a non-homogenous ErrP morphology, and four of them did not present clear ErrP signals. The generic classifier performed better than chance in participants with clear ErrP signals, independently of the SCI (11 out of 16 participants). Three out of the five participants that obtained chance level results with the generic classifier would have not benefitted from the use of a personalized classifier.

SIGNIFICANCE

This work shows the feasibility of transferring an ErrP classifier from able-bodied participants to participants with SCI, for asynchronous detection of ErrPs in an online experiment without offline calibration, which provided immediate feedback to the users.

Classification of error-related potentials from single-trial EEG in association with executed and imagined movements: a feature and classifier investigation

Error-related potentials (ErrPs) have been proposed for designing adaptive brain-computer interfaces (BCIs). Therefore, ErrPs must be decoded. The aim of this study was to evaluate ErrP decoding using combinations of different feature types and classifiers in BCI paradigms involving motor execution (ME) and imagination (MI). Fifteen healthy subjects performed 510 (ME) and 390 (MI) trials of right/left wrist extensions and foot dorsiflexions. Sham BCI feedback was delivered with an accuracy of 80% (ME) and 70% (MI). Continuous EEG was recorded and divided into ErrP and NonErrP epochs. Temporal, spectral, and discrete wavelet transform (DWT) marginals and template matching features were extracted, and all combinations of feature types were classified using linear discriminant analysis, support vector machine, and random forest classifiers. ErrPs were elicited for both ME and MI paradigms, and the average classification accuracies were significantly higher than the chance level. The highest average classification accuracy was obtained using temporal features and a combination of temporal + DWT features classified with random forest; 89 ± 9% and 83 ± 9% for ME and MI, respectively. These results generally indicate that temporal features should be used when detecting ErrPs, but there is great inter-subject variability, which means that user-specific features should be derived to maximize the performance. Graphical abstract.

P300 Latency Estimation Using Least Mean Squares Filter

Event-related potentials (ERPs) are the brain response directly related to specific events or stimuli. The P300 ERP is a positive deflection nominally 300ms post-stimulus that is related to mental decision making processes and also used in P300-based speller systems. Single-trial estimation of P300 responses will help to understand the underlying cognitive process more precisely and also to improve the speed of speller brain-computer interfaces (BCIs). This paper aims to develop a single-trial estimation of the P300 amplitudes and latencies by using the least mean squares (LMS) adaptive filtering method. Results for real data from people with amyotrophic lateral sclerosis (ALS) have shown that the LMS filter can be effectively used to estimate P300 latencies.

Masked and unmasked error-related potentials during continuous control and feedback

The detection of error-related potentials (ErrPs) in tasks with discrete feedback is well established in the brain-computer interface (BCI) field. However, the decoding of ErrPs in tasks with continuous feedback is still in its early stages.

OBJECTIVE

We developed a task in which subjects have continuous control of a cursor's position by means of a joystick. The cursor's position was shown to the participants in two different modalities of continuous feedback: normal and jittered. The jittered feedback was created to mimic the instability that could exist if participants controlled the trajectory directly with brain signals.

APPROACH

This paper studies the electroencephalographic (EEG)-measurable signatures caused by a loss of control over the cursor's trajectory, causing a target miss.

MAIN RESULTS

In both feedback modalities, time-locked potentials revealed the typical frontal-central components of error-related potentials. Errors occurring during the jittered feedback (masked errors) were delayed in comparison to errors occurring during normal feedback (unmasked errors). Masked errors displayed lower peak amplitudes than unmasked errors. Time-locked classification analysis allowed a good distinction between correct and error classes (average Cohen-[Formula: see text], average TPR  =  81.8% and average TNR  =  96.4%). Time-locked classification analysis between masked error and unmasked error classes revealed results at chance level (average Cohen-[Formula: see text], average TPR  =  60.9% and average TNR  =  58.3%). Afterwards, we performed asynchronous detection of ErrPs, combining both masked and unmasked trials. The asynchronous detection of ErrPs in a simulated online scenario resulted in an average TNR of 84.0% and in an average TPR of 64.9%.

SIGNIFICANCE

The time-locked classification results suggest that the masked and unmasked errors were indistinguishable in terms of classification. The asynchronous classification results suggest that the feedback modality did not hinder the asynchronous detection of ErrPs.

Workshops of the Sixth International Brain-Computer Interface Meeting: brain-computer interfaces past, present, and future

The Sixth International Brain-Computer Interface (BCI) Meeting was held 30 May-3 June 2016 at the Asilomar Conference Grounds, Pacific Grove, California, USA. The conference included 28 workshops covering topics in BCI and brain-machine interface research. Topics included BCI for specific populations or applications, advancing BCI research through use of specific signals or technological advances, and translational and commercial issues to bring both implanted and non-invasive BCIs to market. BCI research is growing and expanding in the breadth of its applications, the depth of knowledge it can produce, and the practical benefit it can provide both for those with physical impairments and the general public. Here we provide summaries of each workshop, illustrating the breadth and depth of BCI research and highlighting important issues and calls for action to support future research and development.

Shared-control brain-computer interface for a two dimensional reaching task using EEG error-related potentials

One of the main problems of EEG-based brain computer interfaces (BCIs) is their low information rate, thus for complex tasks the user needs large amounts of time to solve the task. In an attempt to reduce this time and improve the application robustness, recent works have explored shared-control strategies where the device does not only execute the decoded commands, but it is also involved in executing the task. This work proposes a shared-control BCI using error potentials for a 2D reaching task with discrete actions and states. The proposed system has several interesting properties: the system is scalable without increasing the complexity of the user's mental task; the interaction is natural for the user, as the mental task is to monitor the device performance to promote its task learning (in this context the reaching task); and the system has the potential to be combined with additional brain signals to recover or learn from interaction errors. Online control experiments were performed with four subjects, showing that it was possible to reach a goal location from any starting point within a 5×5 grid in around 23 actions (about 19 seconds of EEG signal), both with fixed goals and goals freely chosen by the users.

Intersession adaptation of the EEG-based detector of self-paced walking intention in stroke patients

Brain-computer interfaces (BCIs) have been used in patients with motor impairments as a rehabilitation tool, allowing the control of prosthetic devices with their brain signals. Typically, before each rehabilitation session a calibration phase is recorded to account for session-specific signal changes. Calibration is often an inconvenient process due to its length and patients' fatigue-proneness. This paper focuses on improving the performance of an EEG-based detector of walking intention for intersession transfer. Nine stroke subjects executed a self-paced walking task during three sessions, with one week between sessions. We performed an intersession adaptation by using 80% of one session's data and an additional 20% of a next session for training, and then we tested the detection model on the remaining part of the next session. In practice, this would constitute a longer initial calibration (40 minutes) and a shorter recalibration in subsequent sessions (10 minutes). After training set adaption we attain an average increase in performance of 13.5% over non-adaptive training. Furthermore, we used an approximation of Kullback-Leibler (KL) divergence to quantify the difference between training and testing sets for the non-adaptive and adaptive transfer. As a potential explanation for the improvement of intersession performance, we found a significant decrease in KL-divergence in the case of adaptive transfer.

Errare machinale est: the use of error-related potentials in brain-machine interfaces

The ability to recognize errors is crucial for efficient behavior. Numerous studies have identified electrophysiological correlates of error recognition in the human brain (error-related potentials, ErrPs). Consequently, it has been proposed to use these signals to improve human-computer interaction (HCI) or brain-machine interfacing (BMI). Here, we present a review of over a decade of developments toward this goal. This body of work provides consistent evidence that ErrPs can be successfully detected on a single-trial basis, and that they can be effectively used in both HCI and BMI applications. We first describe the ErrP phenomenon and follow up with an analysis of different strategies to increase the robustness of a system by incorporating single-trial ErrP recognition, either by correcting the machine's actions or by providing means for its error-based adaptation. These approaches can be applied both when the user employs traditional HCI input devices or in combination with another BMI channel. Finally, we discuss the current challenges that have to be overcome in order to fully integrate ErrPs into practical applications. This includes, in particular, the characterization of such signals during real(istic) applications, as well as the possibility of extracting richer information from them, going beyond the time-locked decoding that dominates current approaches.

Workshops of the Fifth International Brain-Computer Interface Meeting: Defining the Future

The Fifth International Brain-Computer Interface (BCI) Meeting met June 3-7th, 2013 at the Asilomar Conference Grounds, Pacific Grove, California. The conference included 19 workshops covering topics in brain-computer interface and brain-machine interface research. Topics included translation of BCIs into clinical use, standardization and certification, types of brain activity to use for BCI, recording methods, the effects of plasticity, special interest topics in BCIs applications, and future BCI directions. BCI research is well established and transitioning to practical use to benefit people with physical impairments. At the same time, new applications are being explored, both for people with physical impairments and beyond. Here we provide summaries of each workshop, illustrating the breadth and depth of BCI research and high-lighting important issues for future research and development.

Using frequency-domain features for the generalization of EEG error-related potentials among different tasks

EEG brain-computer interfaces (BCI) require a calibration phase prior to the on-line control of the device, which is a difficulty for the practical development of this technology as it is user-, session- and task-specific. The large body of research in BCIs based on event-related potentials (ERP) use temporal features, which have demonstrated to be stable for each user along time, but do not generalize well among tasks different from the calibration task. This paper explores the use of low frequency features to improve the generalization capabilities of the BCIs using error-potentials. The results show that there exists a stable pattern in the frequency domain that allows a classifier to generalize among the tasks. Furthermore, the study also shows that it is possible to combine temporal and frequency features to obtain the best of both domains.