Classifier-based latency estimation: a novel way to estimate and predict BCI accuracy

Correcting for ERP latency jitter improves gaze-independent BCI decoding

Objective.Patients suffering from heavy paralysis or Locked-in-Syndrome can regain communication using a Brain-Computer Interface (BCI). Visual event-related potential (ERP) based BCI paradigms exploit visuospatial attention (VSA) to targets laid out on a screen. However, performance drops if the user does not direct their eye gaze at the intended target, harming the utility of this class of BCIs for patients suffering from eye motor deficits. We aim to create an ERP decoder that is less dependent on eye gaze.Approach.ERP component latency jitter plays a role in covert visuospatial attention (VSA) decoding. We introduce a novel decoder which compensates for these latency effects, termed Woody Classifier-based Latency Estimation (WCBLE). We carried out a BCI experiment recording ERP data in overt and covert visuospatial attention (VSA), and introduce a novel special case of covert VSA termed split VSA, simulating the experience of patients with severely impaired eye motor control. We evaluate WCBLE on this dataset and the BNCI2014-009 dataset, within and across VSA conditions to study the dependency on eye gaze and the variation thereof during the experiment.Main results.WCBLE outperforms state-of-the-art methods in the VSA conditions of interest in gaze-independent decoding, without reducing overt VSA performance. Results from across-condition evaluation show that WCBLE is more robust to varying VSA conditions throughout a BCI operation session.Significance. Together, these results point towards a pathway to achieving gaze independence through suited ERP decoding. Our proposed gaze-independent solution enhances decoding performance in those cases where performing overt VSA is not possible.

Multimodal pre-screening can predict BCI performance variability: A novel subject-specific experimental scheme

BACKGROUND

Brain-computer interface (BCI) systems currently lack the required robustness for long-term daily use due to inter- and intra-subject performance variability. In this study, we propose a novel personalized scheme for a multimodal BCI system, primarily using functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG), to identify, predict, and compensate for factors affecting competence-related and interfering factors associated with performance.

METHOD

11 (out of 13 recruited) participants, including five participants with motor deficits, completed four sessions on average. During the training sessions, the subjects performed a short pre-screening phase, followed by three variations of a novel visou-mental (VM) protocol. Features extracted from the pre-screening phase were used to construct predictive platforms using stepwise multivariate linear regression (MLR) models. In the test sessions, we employed a task-correction phase where our predictive models were used to predict the ideal task variation to maximize performance, followed by an interference-correction phase. We then investigated the associations between predicted and actual performances and evaluated the outcome of correction strategies.

RESULT

The predictive models resulted in respective adjusted R-squared values of 0.942, 0.724, and 0.939 for the first, second, and third variation of the task, respectively. The statistical analyses showed significant associations between the performances predicted by predictive models and the actual performances for the first two task variations, with rhos of 0.7289 (p-value = 0.011) and 0.6970 (p-value = 0.017), respectively. For 81.82 % of the subjects, the task/workload correction stage correctly determined which task variation provided the highest accuracy, with an average performance gain of 5.18 % when applying the correction strategies.

CONCLUSION

Our proposed method can lead to an integrated multimodal predictive framework to compensate for BCI performance variability, particularly, for people with severe motor deficits.

Preliminary psychometric properties of a standard vocabulary test administered using a non-invasive brain-computer interface

Objective

To examine measurement agreement between a vocabulary test that is administered in the standardized manner and a version that is administered with a brain-computer interface (BCI).

Method

The sample was comprised of 21 participants, ages 9–27, mean age 16.7 (5.4) years, 61.9% male, including 10 with congenital spastic cerebral palsy (CP), and 11 comparison peers. Participants completed both standard and BCI-facilitated alternate versions of the Peabody Picture Vocabulary Test - 4 (PPVT™-4). The BCI-facilitated PPVT-4 uses items identical to the unmodified PPVT-4, but each quadrant forced-choice item is presented on a computer screen for use with the BCI.

Results

Measurement agreement between instruments was excellent, including an intra-class correlation coefficient of 0.98, and Bland-Altman plots and tests indicating adequate limits of agreement and no systematic test version bias. The mean standard score difference between test versions was 2.0 points (SD 6.3).

Conclusion

These results demonstrate that BCI-facilitated quadrant forced-choice vocabulary testing has the potential to measure aspects of language without requiring any overt physical or communicative response. Thus, it may be possible to identify the language capabilities and needs of many individuals who have not had access to standardized clinical and research instruments.

BCI Accuracy Using Classifier-Based Latency Estimation and the Optimal Interstimulus Interval

PURPOSE

Detection of event-related potentials (ERPs) in brain-computer interfaces (BCIs) allow for communication by individuals with neuromuscular disorders. Enhancing BCI accuracy may be possible through the exploration of the optimal interstimulus interval (ISI). Our objective is to investigate the relationship between BCI accuracy and the optimal ISI value for an individual.

APPROACH

Using the previously developed classifier-based latency estimation (CBLE) [1], we investigated the relationship between the interstimulus interval (ISI) and P3 Speller BCI accuracy. Participants underwent two consecutive sessions in one day. The first session had a default ISI value of 120ms. An optimal ISI value calculated from the first session was used in the second.

RESULTS

Ten subjects participated in the study. Of the ten, half received an optimal ISI value of 120ms and half 160ms. Accuracy differences after implementing the adjusted ISI ranged from -26.1 percent to 4.35 percent. Suggestions for additional experimental design adjustments are highlighted under the discussion portion of this manuscript.

Study of the human brain potentials variability effects in P300 based brain–computer interface

The P300-based brain–computer interfaces (P300 BCI) allow the user to select commands by focusing on them. The technology involves electroencephalographic (EEG) representation of the event-related potentials (ERP) that arise in response to repetitive external stimulation. Conventional procedures for ERP extraction and analysis imply that identical stimuli produce identical responses. However, the floating onset of EEG reactions is a known neurophysiological phenomenon. A failure to account for this source of variability may considerably skew the output and undermine the overall accuracy of the interface. This study aimed to analyze the effects of ERP variability in EEG reactions in order to minimize their influence on P300 BCI command classification accuracy. Healthy subjects aged 21–22 years (n = 12) were presented with a modified P300 BCI matrix moving with specified parameters within the working area. The results strongly support the inherent significance of ERP variability in P300 BCI environments. The correction of peak latencies in single EEG reactions provided a 1.5–2 fold increase in ERP amplitude with a concomitant enhancement of classification accuracy (from 71–78% to 92–95%, p < 0.0005). These effects were particularly pronounced in attention-demanding tasks with the highest matrix velocities. The findings underscore the importance of accounting for ERP variability in advanced BCI systems.

A Comparison of Classification Techniques to Predict Brain-Computer Interfaces Accuracy Using Classifier-Based Latency Estimation

P300-based Brain-Computer Interface (BCI) performance is vulnerable to latency jitter. To investigate the role of latency jitter on BCI system performance, we proposed the classifier-based latency estimation (CBLE) method. In our previous study, CBLE was based on least-squares (LS) and stepwise linear discriminant analysis (SWLDA) classifiers. Here, we aim to extend the CBLE method using sparse autoencoders (SAE) to compare the SAE-based CBLE method with LS- and SWLDA-based CBLE. The newly-developed SAE-based CBLE and previously used methods are also applied to a newly-collected dataset to reduce the possibility of spurious correlations. Our results showed a significant (p<0.001) negative correlation between BCI accuracy and estimated latency jitter. Furthermore, we also examined the effect of the number of electrodes on each classification technique. Our results showed that on the whole, CBLE worked regardless of the classification method and electrode count; by contrast the effect of the number of electrodes on BCI performance was classifier dependent.

Cross-correlation task-related component analysis (xTRCA) for enhancing evoked and induced responses of event-related potentials

We propose an analysis method that extracts trial-reproducible (i.e., recurring) event-related spatiotemporal EEG patterns by optimizing a spatial filter as well as trial timings of task-related components in the time domain simultaneously in a unified manner. Event-related responses are broadly categorized into evoked and induced responses, but those are analyzed commonly in the time and the time-frequency domain, respectively. To facilitate a comparison of evoked and induced responses, a unified method for analyzing both evoked and induced responses is desired. Here we propose a method of cross-correlation task-related component analysis (xTRCA) as an extension of our previous method. xTRCA constructs a linear spatial filter and then optimizes trial timings of single trials based on trial reproducibility as an objective function. The spatial filter enhances event-related responses, and the temporal optimization compensates trial-by-trial latencies that are inherent to ERPs. We first applied xTRCA to synthetic data of induced responses whose phases varied from trial to trial, and found that xTRCA could realign the induced responses by compensating the phase differences. We then demonstrated with mismatch negativity data that xTRCA enhanced the event-related-potential waveform observed at a single channel. Finally, a classification accuracy was improved when trial timings were optimized by xTRCA, suggesting a practical application of the method for a brain computer interface. We conclude that xTRCA provides a unified framework to analyze and enhance event-related evoked and induced responses in the time domain by objectively maximizing trial reproducibility.

Learning Discriminative Spatiospectral Features of ERPs for Accurate Brain-Computer Interfaces

Constructing accurate predictive models is at the heart of brain-computer interfaces (BCIs) because these models can ultimately translate brain activities into communication and control commands. The majority of the previous work in BCI use spatial, temporal, or spatiotemporal features of event-related potentials (ERPs). In this study, we examined the discriminatory effect of their spatiospectral features to capture the most relevant set of neural activities from electroencephalographic recordings that represent users' mental intent. In this regard, we model ERP waveforms using a sum of sinusoids with unknown amplitudes, frequencies, and phases. The effect of this signal modeling step is to represent high-dimensional ERP waveforms in a substantially lower dimensionality space, which includes their dominant power spectral contents. We found that the most discriminative frequencies for accurate decoding of visual attention modulated ERPs lie in a spectral range less than 6.4 Hz. This was empirically verified by treating dominant frequency contents of ERP waveforms as feature vectors in the state-of-the-art machine learning techniques used herein. The constructed predictive models achieved remarkable performance, which for some subjects was as high as 94% as measured by the area under curve. Using these spectral contents, we further studied the discriminatory effect of each channel and proposed an efficient strategy to choose subject-specific subsets of channels that generally led to classifiers with comparable performance.

P300 Latency Estimation Using Least Mean Squares Filter

Event-related potentials (ERPs) are the brain response directly related to specific events or stimuli. The P300 ERP is a positive deflection nominally 300ms post-stimulus that is related to mental decision making processes and also used in P300-based speller systems. Single-trial estimation of P300 responses will help to understand the underlying cognitive process more precisely and also to improve the speed of speller brain-computer interfaces (BCIs). This paper aims to develop a single-trial estimation of the P300 amplitudes and latencies by using the least mean squares (LMS) adaptive filtering method. Results for real data from people with amyotrophic lateral sclerosis (ALS) have shown that the LMS filter can be effectively used to estimate P300 latencies.

On the Relationship Between Attention Processing and P300-Based Brain Computer Interface Control in Amyotrophic Lateral Sclerosis

Our objective was to investigate the capacity to control a P3-based brain-computer interface (BCI) device for communication and its related (temporal) attention processing in a sample of amyotrophic lateral sclerosis (ALS) patients with respect to healthy subjects. The ultimate goal was to corroborate the role of cognitive mechanisms in event-related potential (ERP)-based BCI control in ALS patients. Furthermore, the possible differences in such attentional mechanisms between the two groups were investigated in order to unveil possible alterations associated with the ALS condition. Thirteen ALS patients and 13 healthy volunteers matched for age and years of education underwent a P3-speller BCI task and a rapid serial visual presentation (RSVP) task. The RSVP task was performed by participants in order to screen their temporal pattern of attentional resource allocation, namely: (i) the temporal attentional filtering capacity (scored as T1%); and (ii) the capability to adequately update the attentive filter in the temporal dynamics of the attentional selection (scored as T2%). For the P3-speller BCI task, the online accuracy and information transfer rate (ITR) were obtained. Centroid Latency and Mean Amplitude of N200 and P300 were also obtained. No significant differences emerged between ALS patients and Controls with regards to online accuracy (p = 0.13). Differently, the performance in controlling the P3-speller expressed as ITR values (calculated offline) were compromised in ALS patients (p < 0.05), with a delay in the latency of P3 when processing BCI stimuli as compared with Control group (p < 0.01). Furthermore, the temporal aspect of attentional filtering which was related to BCI control (r = 0.51; p < 0.05) and to the P3 wave amplitude (r = 0.63; p < 0.05) was also altered in ALS patients (p = 0.01). These findings ground the knowledge required to develop sensible classes of BCI specifically designed by taking into account the influence of the cognitive characteristics of the possible candidates in need of a BCI system for communication.

Enhancing P300-BCI performance using latency estimation

Brain Computer Interfaces (BCIs) offer restoration of communication to those with the most severe movement impairments, but performance is not yet ideal. Previous work has demonstrated that latency jitter, the variation in timing of the brain responses, plays a critical role in determining BCI performance. In this study, we used Classifier-Based Latency Estimation (CBLE) and a wavelet transform to provide information about latency jitter to a second-level classifier. Three second-level classifiers were tested: least squares (LS), step-wise linear discriminant analysis (SWLDA), and support vector machine (SVM). Of these three, LS and SWLDA performed better than the original online classifier. The resulting combination demonstrated improved detection of brain responses for many participants, resulting in better BCI performance. Interestingly, the performance gain was greatest for those individuals for whom the BCI did not work well online, indicating that this method may be most suitable for improving performance of otherwise marginal participants.

Using the detectability index to predict P300 speller performance

OBJECTIVE

The P300 speller is a popular brain-computer interface (BCI) system that has been investigated as a potential communication alternative for individuals with severe neuromuscular limitations. To achieve acceptable accuracy levels for communication, the system requires repeated data measurements in a given signal condition to enhance the signal-to-noise ratio of elicited brain responses. These elicited brain responses, which are used as control signals, are embedded in noisy electroencephalography (EEG) data. The discriminability between target and non-target EEG responses defines a user's performance with the system. A previous P300 speller model has been proposed to estimate system accuracy given a certain amount of data collection. However, the approach was limited to a static stopping algorithm, i.e. averaging over a fixed number of measurements, and the row-column paradigm. A generalized method that is also applicable to dynamic stopping (DS) algorithms and other stimulus paradigms is desirable.

APPROACH

We developed a new probabilistic model-based approach to predicting BCI performance, where performance functions can be derived analytically or via Monte Carlo methods. Within this framework, we introduce a new model for the P300 speller with the Bayesian DS algorithm, by simplifying a multi-hypothesis to a binary hypothesis problem using the likelihood ratio test. Under a normality assumption, the performance functions for the Bayesian algorithm can be parameterized with the detectability index, a measure which quantifies the discriminability between target and non-target EEG responses.

MAIN RESULTS

Simulations with synthetic and empirical data provided initial verification of the proposed method of estimating performance with Bayesian DS using the detectability index. Analysis of results from previous online studies validated the proposed method.

SIGNIFICANCE

The proposed method could serve as a useful tool to initially assess BCI performance without extensive online testing, in order to estimate the amount of data required to achieve a desired accuracy level.

P300 latency Jitter occurrence in patients with disorders of consciousness: Toward a better design for Brain Computer Interface applications

In this study the P300 latency jitter has been explored in an EEG data set collected from a group of patients with disorders of consciousness (DOC; n=13) that was administered with an auditory Oddball paradigm under passive and active conditions. A method based on wavelet transform was applied to estimate single trial P300 waveforms. Preliminary results showed that 5 Vegetative State (VS) and 8 Minimally Conscious Staten (MCS) patients exhibited significantly higher values of P300 latency jitter as compared to those obtained from a control group of 12 healthy subjects. In addition, the magnitude of the P300 latency jitter negatively correlated with patients' clinical status. The existence of such phenomenon might substantially limit an effective use of Brain Computer Interface systems for communication.

The LDA beamformer: Optimal estimation of ERP source time series using linear discriminant analysis

We introduce a novel beamforming approach for estimating event-related potential (ERP) source time series based on regularized linear discriminant analysis (LDA). The optimization problems in LDA and linearly-constrained minimum-variance (LCMV) beamformers are formally equivalent. The approaches differ in that, in LCMV beamformers, the spatial patterns are derived from a source model, whereas in an LDA beamformer the spatial patterns are derived directly from the data (i.e., the ERP peak). Using a formal proof and MEG simulations, we show that the LDA beamformer is robust to correlated sources and offers a higher signal-to-noise ratio than the LCMV beamformer and PCA. As an application, we use EEG data from an oddball experiment to show how the LDA beamformer can be harnessed to detect single-trial ERP latencies and estimate connectivity between ERP sources. Concluding, the LDA beamformer optimally reconstructs ERP sources by maximizing the ERP signal-to-noise ratio. Hence, it is a highly suited tool for analyzing ERP source time series, particularly in EEG/MEG studies wherein a source model is not available.

Effects of text generation on P300 brain-computer interface performance

Brain-computer interfaces (BCIs) are intended to provide independent communication for those with the most severe physical impairments. However, development and testing of BCIs is typically conducted with copy-spelling of provided text, which models only a small portion of a functional communication task. This study was designed to determine how BCI performance is affected by novel text generation. We used a within-subject single-session study design in which subjects used a BCI to perform copy-spelling of provided text and to generate self-composed text to describe a picture. Additional off-line analysis was performed to identify changes in the event-related potentials that the BCI detects and to examine the effects of training the BCI classifier on task-specific data. Accuracy was reduced during the picture description task; (t(8)=2.59 p=0.0321). Creating the classifier using self-generated text data significantly improved accuracy on these data; (t(7)=-2.68, p=0.0317), but did not bring performance up to the level achieved during copy-spelling. Thus, this study shows that the task for which the BCI is used makes a difference in BCI accuracy. Task-specific BCI classifiers are a first step to counteract this effect, but additional study is needed.