BCI Competition 2003—Data Set IIb: Support Vector Machines for the P300 Speller Paradigm

Training set extension for SVM ensemble in P300-speller with familiar face paradigm

BACKGROUND

P300-spellers are brain-computer interface (BCI)-based character input systems. Support vector machine (SVM) ensembles are trained with large-scale training sets and used as classifiers in these systems. However, the required large-scale training data necessitate a prolonged collection time for each subject, which results in data collected toward the end of the period being contaminated by the subject's fatigue.

OBJECTIVE

This study aimed to develop a method for acquiring more training data based on a collected small training set.

METHODS

A new method was developed in which two corresponding training datasets in two sequences are superposed and averaged to extend the training set. The proposed method was tested offline on a P300-speller with the familiar face paradigm.

RESULTS

The SVM ensemble with extended training set achieved 85% classification accuracy for the averaged results of four sequences, and 100% for 11 sequences in the P300-speller. In contrast, the conventional SVM ensemble with non-extended training set achieved only 65% accuracy for four sequences, and 92% for 11 sequences.

CONCLUSION

The SVM ensemble with extended training set achieves higher classification accuracies than the conventional SVM ensemble, which verifies that the proposed method effectively improves the classification performance of BCI P300-spellers, thus enhancing their practicality.

EEG Waveform Analysis of P300 ERP with Applications to Brain Computer Interfaces

The Electroencephalography (EEG) is not just a mere clinical tool anymore. It has become the de-facto mobile, portable, non-invasive brain imaging sensor to harness brain information in real time. It is now being used to translate or decode brain signals, to diagnose diseases or to implement Brain Computer Interface (BCI) devices. The automatic decoding is mainly implemented by using quantitative algorithms to detect the cloaked information buried in the signal. However, clinical EEG is based intensively on waveforms and the structure of signal plots. Hence, the purpose of this work is to establish a bridge to fill this gap by reviewing and describing the procedures that have been used to detect patterns in the electroencephalographic waveforms, benchmarking them on a controlled pseudo-real dataset of a P300-Based BCI Speller and verifying their performance on a public dataset of a BCI Competition.

A comparison of feature extraction strategies using wavelet dictionaries and feature selection methods for single trial P300-based BCI

The P300 component of event-related potentials (ERPs) is widely used in the implementation of brain computer interfaces (BCI). In this context, one of the main issues to solve is the binary classification problem that entails differentiating between electroencephalographic (EEG) signals with and without P300. Given the particularly unfavorable signal-to-noise ratio (SNR) in the single-trial detection scenario, this is a challenging problem in the pattern recognition field. To the best of our knowledge, there are no previous experimental studies comparing feature extraction and selection methods for single trial P300-based BCIs using unified criteria and data. In order to improve the performance and robustness of single-trial classifiers, we analyzed and compared different alternatives for the feature generation and feature selection blocks. We evaluated different orthogonal decompositions based on the wavelet transform for feature extraction, as well as different filter, wrapper, and embedded alternatives for feature selection. Accuracies over 75% were obtained for most of the analyzed strategies with a relatively low computational cost, making them attractive for a practical BCI implementation using inexpensive hardware. Graphical Abstract Experiments performed for P300 detection.

Comparison of logistic regression, support vector machines, and deep learning classifiers for predicting memory encoding success using human intracranial EEG recordings

OBJECTIVE

We sought to test the performance of three strategies for binary classification (logistic regression, support vector machines, and deep learning) for the problem of predicting successful episodic memory encoding using direct brain recordings obtained from human stereo EEG subjects. We also sought to test the impact of applying t-distributed stochastic neighbor embedding (tSNE) for unsupervised dimensionality reduction, as well as testing the effect of reducing input features to a core set of memory relevant brain areas. This work builds upon published efforts to develop a closed-loop stimulation device to improve memory performance.

APPROACH

We used a unique data set consisting of 30 stereo EEG patients with electrodes implanted into a core set of five common brain regions (along with other areas) who performed the free recall episodic memory task as brain activity was recorded. Using three different machine learning strategies, we trained classifiers to predict successful versus unsuccessful memory encoding and compared the difference in classifier performance (as measured by the AUC) at the subject level and in aggregate across modalities. We report the impact of feature reduction on the classifiers, including reducing the number of input brain regions, frequency bands, and the impact of tSNE.

RESULTS

Deep learning classifiers outperformed both support vector machines (SVM) and logistic regression (LR). A priori selection of core brain regions also improved classifier performance for LR and SVM models, especially when combined with tSNE.

SIGNIFICANCE

We report for the first time a direct comparison among traditional and deep learning methods of binary classification to the problem of predicting successful memory encoding using human brain electrophysiological data. Our findings will inform the design of brain machine interface devices to affect memory processing.

Enhancing Performance and Bit Rates in a Brain-Computer Interface System With Phase-to-Amplitude Cross-Frequency Coupling: Evidences From Traditional c-VEP, Fast c-VEP, and SSVEP Designs

A brain–computer interface (BCI) is a channel of communication that transforms brain activity into specific commands for manipulating a personal computer or other home or electrical devices. In other words, a BCI is an alternative way of interacting with the environment by using brain activity instead of muscles and nerves. For that reason, BCI systems are of high clinical value for targeted populations suffering from neurological disorders. In this paper, we present a new processing approach in three publicly available BCI data sets: (a) a well-known multi-class (N = 6) coded-modulated Visual Evoked potential (c-VEP)-based BCI system for able-bodied and disabled subjects; (b) a multi-class (N = 32) c-VEP with slow and fast stimulus representation; and (c) a steady-state Visual Evoked potential (SSVEP) multi-class (N = 5) flickering BCI system. Estimating cross-frequency coupling (CFC) and namely δ-θ [δ: (0.5–4 Hz), θ: (4–8 Hz)] phase-to-amplitude coupling (PAC) within sensor and across experimental time, we succeeded in achieving high classification accuracy and Information Transfer Rates (ITR) in the three data sets. Our approach outperformed the originally presented ITR on the three data sets. The bit rates obtained for both the disabled and able-bodied subjects reached the fastest reported level of 324 bits/min with the PAC estimator. Additionally, our approach outperformed alternative signal features such as the relative power (29.73 bits/min) and raw time series analysis (24.93 bits/min) and also the original reported bit rates of 10–25 bits/min. In the second data set, we succeeded in achieving an average ITR of 124.40 ± 11.68 for the slow 60 Hz and an average ITR of 233.99 ± 15.75 for the fast 120 Hz. In the third data set, we succeeded in achieving an average ITR of 106.44 ± 8.94. Current methodology outperforms any previous methodologies applied to each of the three free available BCI datasets.

Improving P300 Spelling Rate using Language Models and Predictive Spelling

The P300 Speller Brain-Computer Interface (BCI) provides a means of communication for those suffering from advanced neuromuscular diseases such as amyotrophic lateral sclerosis (ALS). Recent literature has incorporated language-based modelling, which uses previously chosen characters and the structure of natural language to modify the interface and classifier. Two complementary methods of incorporating language models have previously been independently studied: predictive spelling uses language models to generate suggestions of complete words to allow for the selection of multiple characters simultaneously, and language model-based classifiers have used prior characters to create a prior probability distribution over the characters based on how likely they are to follow. In this study, we propose a combined method which extends a language-based classifier to generate prior probabilities for both individual characters and complete words. In order to gauge the efficiency of this new model, results across 12 healthy subjects were measured. Incorporating predictive spelling increased typing speed using the P300 speller, with an average increase of 15.5% in typing rate across subjects, demonstrating that language models can be effectively utilized to create full word suggestions for predictive spelling. When combining predictive spelling with language model classification, typing speed is significantly improved, resulting in better typing performance.

Enhancing performance of P300-Speller under mental workload by incorporating dual-task data during classifier training

As one of the most important brain-computer interface (BCI) paradigms, P300-Speller was shown to be significantly impaired once applied in practical situations due to effects of mental workload. This study aims to provide a new method of building training models to enhance performance of P300-Speller under mental workload. Three experiment conditions based on row-column P300-Speller paradigm were performed including speller-only, 3-back-speller and mental-arithmetic-speller. Data under dual-task conditions were introduced to speller-only data respectively to build new training models. Then performance of classifiers with different models was compared under the same testing condition. The results showed that when tasks of imported training data and testing data were the same, character recognition accuracies and round accuracies of P300-Speller with mixed-data training models significantly improved (FDR, p < 0.005). When they were different, performance significantly improved when tested on mental-arithmetic-speller (FDR, p < 0.05) while the improvement was modest when tested on n-back-speller (FDR, p < 0.1). The analysis of ERPs revealed that ERP difference between training data and testing data was significantly diminished when the dual-task data was introduced to training data (FDR, p < 0.05). The new method of training classifier on mixed data proved to be effective in enhancing performance of P300-Speller under mental workload, confirmed the feasibility to build a universal training model and overcome the effects of mental workload in its practical applications.

Spatial-Temporal Feature Analysis on Single-Trial Event Related Potential for Rapid Face Identification

The event-related potential (ERP) is the brain response measured in electroencephalography (EEG), which reflects the process of human cognitive activity. ERP has been introduced into brain computer interfaces (BCIs) to communicate the computer with the subject's intention. Due to the low signal-to-noise ratio of EEG, most ERP studies are based on grand-averaging over many trials. Recently single-trial ERP detection attracts more attention, which enables real time processing tasks as rapid face identification. All the targets needed to be retrieved may appear only once, and there is no knowledge of target label for averaging. More interestingly, how the features contribute temporally and spatially to single-trial ERP detection has not been fully investigated. In this paper, we propose to implement a local-learning-based (LLB) feature extraction method to investigate the importance of spatial-temporal components of ERP in a task of rapid face identification using single-trial detection. Comparing to previous methods, LLB method preserves the nonlinear structure of EEG signal distribution, and analyze the importance of original spatial-temporal components via optimization in feature space. As a data-driven methods, the weighting of the spatial-temporal component does not depend on the ERP detection method. The importance weights are optimized by making the targets more different from non-targets in feature space, and regularization penalty is introduced in optimization for sparse weights. This spatial-temporal feature extraction method is evaluated on the EEG data of 15 participants in performing a face identification task using rapid serial visual presentation paradigm. Comparing with other methods, the proposed spatial-temporal analysis method uses sparser (only 10% of the total) features, and could achieve comparable performance (98%) of single-trial ERP detection as the whole features across different detection methods. The interesting finding is that the N250 is the earliest temporal component that contributes to single-trial ERP detection in face identification. And the importance of N250 components is more laterally distributed toward the left hemisphere. We show that using only the left N250 component over-performs the right N250 in the face identification task using single-trial ERP detection. The finding is also important in building a fast and efficient (fewer electrodes) BCI system for rapid face identification.

Open Access Dataset for EEG+NIRS Single-Trial Classification

We provide an open access dataset for hybrid brain-computer interfaces (BCIs) using electroencephalography (EEG) and near-infrared spectroscopy (NIRS). For this, we con-ducted two BCI experiments (left vs. right hand motor imagery; mental arithmetic vs. resting state). The dataset was validated using baseline signal analysis methods, with which classification performance was evaluated for each modality and a combination of both modalities. As already shown in previous literature, the capability of discriminating different mental states can be en-hanced by using a hybrid approach, when comparing to single modality analyses. This makes the provided data highly suitable for hybrid BCI investigations. Since our open access dataset also comprises motion artifacts and physiological data, we expect that it can be used in a wide range of future validation approaches in multimodal BCI research.

Evaluating brain-computer interface performance using color in the P300 checkerboard speller

OBJECTIVE

Current Brain-Computer Interface (BCI) systems typically flash an array of items from grey to white (GW). The objective of this study was to evaluate BCI performance using uniquely colored stimuli.

METHODS

In addition to the GW stimuli, the current study tested two types of color stimuli (grey to color [GC] and color intensification [CI]). The main hypotheses were that in a checkboard paradigm, unique color stimuli will: (1) increase BCI performance over the standard GW paradigm; (2) elicit larger event-related potentials (ERPs); and, (3) improve offline performance with an electrode selection algorithm (i.e., Jumpwise).

RESULTS

Online results (n=36) showed that GC provides higher accuracy and information transfer rate than the CI and GW conditions. Waveform analysis showed that GC produced higher amplitude ERPs than CI and GW. Information transfer rate was improved by the Jumpwise-selected channel locations in all conditions.

CONCLUSIONS

Unique color stimuli (GC) improved BCI performance and enhanced ERPs. Jumpwise-selected electrode locations improved offline performance.

SIGNIFICANCE

These results show that in a checkerboard paradigm, unique color stimuli increase BCI performance, are preferred by participants, and are important to the design of end-user applications; thus, could lead to an increase in end-user performance and acceptance of BCI technology.

Enhancing P300-BCI performance using latency estimation

Brain Computer Interfaces (BCIs) offer restoration of communication to those with the most severe movement impairments, but performance is not yet ideal. Previous work has demonstrated that latency jitter, the variation in timing of the brain responses, plays a critical role in determining BCI performance. In this study, we used Classifier-Based Latency Estimation (CBLE) and a wavelet transform to provide information about latency jitter to a second-level classifier. Three second-level classifiers were tested: least squares (LS), step-wise linear discriminant analysis (SWLDA), and support vector machine (SVM). Of these three, LS and SWLDA performed better than the original online classifier. The resulting combination demonstrated improved detection of brain responses for many participants, resulting in better BCI performance. Interestingly, the performance gain was greatest for those individuals for whom the BCI did not work well online, indicating that this method may be most suitable for improving performance of otherwise marginal participants.

A comparison of stimulus types in online classification of the P300 speller using language models

The P300 Speller is a common brain-computer interface communication system. There are many parallel lines of research underway to overcome the system's low signal to noise ratio and thereby improve performance, including using famous face stimuli and integrating language information into the classifier. While both have been shown separately to provide significant improvements, the two methods have not yet been implemented together to demonstrate that the improvements are complimentary. The goal of this study is therefore twofold. First, we aim to compare the famous faces stimulus paradigm with an existing alternative stimulus paradigm currently used in commercial systems (i.e., character inversion). Second, we test these methods with language model integration to assess whether different optimization approaches can be combined to further improve BCI communication. In offline analysis using a previously published particle filter method, famous faces stimuli yielded superior results to both standard and inverting stimuli. In online trials using the particle filter method, all 10 subjects achieved a higher selection rate when using the famous faces flashing paradigm than when using inverting flashes. The improvements achieved by these methods are therefore complementary and a combination yields superior results to either method implemented individually when tested in healthy subjects.

An improved P300 pattern in BCI to catch user's attention

OBJECTIVE

Brain-computer interfaces (BCIs) can help patients who have lost control over most muscles but are still conscious and able to communicate or interact with the environment. One of the most popular types of BCI is the P300-based BCI. With this BCI, users are asked to count the number of appearances of target stimuli in an experiment. To date, the majority of visual P300-based BCI systems developed have used the same character or picture as the target for every stimulus presentation, which can bore users. Consequently, users attention may decrease or be negatively affected by adjacent stimuli.

APPROACH

In this study, a new stimulus is presented to increase user concentration. Honeycomb-shaped figures with 1-3 red dots were used as stimuli. The number and the positions of the red dots in the honeycomb-shaped figure were randomly changed during BCI control. The user was asked to count the number of the dots presented in each flash instead of the number of times they flashed. To assess the performance of this new stimulus, another honeycomb-shaped stimulus, without red dots, was used as a control condition.

MAIN RESULTS

The results showed that the honeycomb-shaped stimuli with red dots obtained significantly higher classification accuracies and information transfer rates (p  <  0.05) compared to the honeycomb-shaped stimulus without red dots.

SIGNIFICANCE

The results indicate that this proposed method can be a promising approach to improve the performance of the BCI system and can be an efficient method in daily application.

Multimodal Neural Network for Rapid Serial Visual Presentation Brain Computer Interface

Brain computer interfaces allow users to preform various tasks using only the electrical activity of the brain. BCI applications often present the user a set of stimuli and record the corresponding electrical response. The BCI algorithm will then have to decode the acquired brain response and perform the desired task. In rapid serial visual presentation (RSVP) tasks, the subject is presented with a continuous stream of images containing rare target images among standard images, while the algorithm has to detect brain activity associated with target images. In this work, we suggest a multimodal neural network for RSVP tasks. The network operates on the brain response and on the initiating stimulus simultaneously, providing more information for the BCI application. We present two variants of the multimodal network, a supervised model, for the case when the targets are known in advanced, and a semi-supervised model for when the targets are unknown. We test the neural networks with a RSVP experiment on satellite imagery carried out with two subjects. The multimodal networks achieve a significant performance improvement in classification metrics. We visualize what the networks has learned and discuss the advantages of using neural network models for BCI applications.

MindEdit: A P300-based text editor for mobile devices

Practical application of Brain-Computer Interfaces (BCIs) requires that the whole BCI system be portable. The mobility of BCI systems involves two aspects: making the electroencephalography (EEG) recording devices portable, and developing software applications with low computational complexity to be able to run on low computational-power devices such as tablets and smartphones. This paper addresses the development of MindEdit; a P300-based text editor for Android-based devices. Given the limited resources of mobile devices and their limited computational power, a novel ensemble classifier is utilized that uses Principal Component Analysis (PCA) features to identify P300 evoked potentials from EEG recordings. PCA computations in the proposed method are channel-based as opposed to concatenating all channels as in traditional feature extraction methods; thus, this method has less computational complexity compared to traditional P300 detection methods. The performance of the method is demonstrated on data recorded from MindEdit on an Android tablet using the Emotiv wireless neuroheadset. Results demonstrate the capability of the introduced PCA ensemble classifier to classify P300 data with maximum average accuracy of 78.37±16.09% for cross-validation data and 77.5±19.69% for online test data using only 10 trials per symbol and a 33-character training dataset. Our analysis indicates that the introduced method outperforms traditional feature extraction methods. For a faster operation of MindEdit, a variable number of trials scheme is introduced that resulted in an online average accuracy of 64.17±19.6% and a maximum bitrate of 6.25bit/min. These results demonstrate the efficacy of using the developed BCI application with mobile devices.

Online BCI Typing using Language Model Classifiers by ALS Patients in their Homes

The P300 speller is a common brain-computer interface system that can provide a means of communication for patients with amyotrophic lateral sclerosis (ALS). Recent studies have shown that incorporating language information in signal classification can improve system performance, but they have largely been tested on healthy volunteers in a laboratory setting. The goal of this study was to demonstrate the functionality of the P300 speller system with language models when used by ALS patients in their homes. Six ALS patients with functional ratings ranging from two to 28 participated in this study. All subjects had improved offline performance when using a language model and five subjects were able to type at least six characters per minute with over 84% accuracy in online sessions. The results of this study indicate that the improvements in performance using language models in the P300 speller translate into the ALS population, which could help to make it a viable assistive device.

Sparse Bayesian Classification of EEG for Brain-Computer Interface

Regularization has been one of the most popular approaches to prevent overfitting in electroencephalogram (EEG) classification of brain-computer interfaces (BCIs). The effectiveness of regularization is often highly dependent on the selection of regularization parameters that are typically determined by cross-validation (CV). However, the CV imposes two main limitations on BCIs: 1) a large amount of training data is required from the user and 2) it takes a relatively long time to calibrate the classifier. These limitations substantially deteriorate the system's practicability and may cause a user to be reluctant to use BCIs. In this paper, we introduce a sparse Bayesian method by exploiting Laplace priors, namely, SBLaplace, for EEG classification. A sparse discriminant vector is learned with a Laplace prior in a hierarchical fashion under a Bayesian evidence framework. All required model parameters are automatically estimated from training data without the need of CV. Extensive comparisons are carried out between the SBLaplace algorithm and several other competing methods based on two EEG data sets. The experimental results demonstrate that the SBLaplace algorithm achieves better overall performance than the competing algorithms for EEG classification.

Fuzzy clustering-based feature extraction method for mental task classification

A brain computer interface (BCI) is a communication system by which a person can send messages or requests for basic necessities without using peripheral nerves and muscles. Response to mental task-based BCI is one of the privileged areas of investigation. Electroencephalography (EEG) signals are used to represent the brain activities in the BCI domain. For any mental task classification model, the performance of the learning model depends on the extraction of features from EEG signal. In literature, wavelet transform and empirical mode decomposition are two popular feature extraction methods used to analyze a signal having non-linear and non-stationary property. By adopting the virtue of both techniques, a theoretical adaptive filter-based method to decompose non-linear and non-stationary signal has been proposed known as empirical wavelet transform (EWT) in recent past. EWT does not work well for the signals having overlapped in frequency and time domain and failed to provide good features for further classification. In this work, Fuzzy c-means algorithm is utilized along with EWT to handle this problem. It has been observed from the experimental results that EWT along with fuzzy clustering outperforms in comparison to EWT for the EEG-based response to mental task problem. Further, in case of mental task classification, the ratio of samples to features is very small. To handle the problem of small ratio of samples to features, in this paper, we have also utilized three well-known multivariate feature selection methods viz. Bhattacharyya distance (BD), ratio of scatter matrices (SR), and linear regression (LR). The results of experiment demonstrate that the performance of mental task classification has improved considerably by aforesaid methods. Ranking method and Friedman’s statistical test are also performed to rank and compare different combinations of feature extraction methods and feature selection methods which endorse the efficacy of the proposed approach.

Decoding the Locus of Covert Visuospatial Attention from EEG Signals

Visuospatial attention can be deployed to different locations in space independently of ocular fixation, and studies have shown that event-related potential (ERP) components can effectively index whether such covert visuospatial attention is deployed to the left or right visual field. However, it is not clear whether we may obtain a more precise spatial localization of the focus of attention based on the EEG signals during central fixation. In this study, we used a modified Posner cueing task with an endogenous cue to determine the degree to which information in the EEG signal can be used to track visual spatial attention in presentation sequences lasting 200 ms. We used a machine learning classification method to evaluate how well EEG signals discriminate between four different locations of the focus of attention. We then used a multi-class support vector machine (SVM) and a leave-one-out cross-validation framework to evaluate the decoding accuracy (DA). We found that ERP-based features from occipital and parietal regions showed a statistically significant valid prediction of the location of the focus of visuospatial attention (DA = 57%, p < .001, chance-level 25%). The mean distance between the predicted and the true focus of attention was 0.62 letter positions, which represented a mean error of 0.55 degrees of visual angle. In addition, ERP responses also successfully predicted whether spatial attention was allocated or not to a given location with an accuracy of 79% (p < .001). These findings are discussed in terms of their implications for visuospatial attention decoding and future paths for research are proposed.

Time-Shift Correlation Algorithm for P300 Event Related Potential Brain-Computer Interface Implementation

A high efficient time-shift correlation algorithm was proposed to deal with the peak time uncertainty of P300 evoked potential for a P300-based brain-computer interface (BCI). The time-shift correlation series data were collected as the input nodes of an artificial neural network (ANN), and the classification of four LED visual stimuli was selected as the output node. Two operating modes, including fast-recognition mode (FM) and accuracy-recognition mode (AM), were realized. The proposed BCI system was implemented on an embedded system for commanding an adult-size humanoid robot to evaluate the performance from investigating the ground truth trajectories of the humanoid robot. When the humanoid robot walked in a spacious area, the FM was used to control the robot with a higher information transfer rate (ITR). When the robot walked in a crowded area, the AM was used for high accuracy of recognition to reduce the risk of collision. The experimental results showed that, in 100 trials, the accuracy rate of FM was 87.8% and the average ITR was 52.73 bits/min. In addition, the accuracy rate was improved to 92% for the AM, and the average ITR decreased to 31.27 bits/min. due to strict recognition constraints.

A P300-Based Threshold-Free Brain Switch and Its Application in Wheelchair Control

The key issue of electroencephalography (EEG)-based brain switches is to detect the control and idle states in an asynchronous manner. Most existing methods rely on a threshold. However, it is often time consuming to select a satisfactory threshold, and the chosen threshold might be inappropriate over a long period of time due to the variability of the EEG signals. This paper presents a new P300-based threshold-free brain switch. Specifically, one target button and three pseudo buttons, which are intensified in a random order to produce P300 potential, are set in the graphical user interface. The user can issue a switch command by focusing on the target button. Two support vector machine (SVM) classifiers, namely, SVM1 and SVM2, are used in the detection algorithm. During detection, we first obtained four SVM scores, corresponding to the four flashing buttons, by applying SVM1 to the ongoing EEG. If the SVM score corresponding to the target button was negative or not at the maximum, then an idle state was determined. Moreover, if the target button had a maximum and positive score, then we fed the four SVM scores as features into SVM2 to further discriminate the control and idle states. As an application, this brain switch was used to produce a start/stop command for an intelligent wheelchair, of which the left, right, forward, backward functions were carried out by an autonomous navigation system. Several experiments were conducted with eight healthy subjects and five patients with spinal cord injuries (SCIs). The experimental results not only demonstrated the effectiveness of our approach but also illustrated the potential application for patients with SCIs.

Integrating language models into classifiers for BCI communication: a review

OBJECTIVE

The present review systematically examines the integration of language models to improve classifier performance in brain-computer interface (BCI) communication systems.

APPROACH

The domain of natural language has been studied extensively in linguistics and has been used in the natural language processing field in applications including information extraction, machine translation, and speech recognition. While these methods have been used for years in traditional augmentative and assistive communication devices, information about the output domain has largely been ignored in BCI communication systems. Over the last few years, BCI communication systems have started to leverage this information through the inclusion of language models.

MAIN RESULTS

Although this movement began only recently, studies have already shown the potential of language integration in BCI communication and it has become a growing field in BCI research. BCI communication systems using language models in their classifiers have progressed down several parallel paths, including: word completion; signal classification; integration of process models; dynamic stopping; unsupervised learning; error correction; and evaluation.

SIGNIFICANCE

Each of these methods have shown significant progress, but have largely been addressed separately. Combining these methods could use the full potential of language model, yielding further performance improvements. This integration should be a priority as the field works to create a BCI system that meets the needs of the amyotrophic lateral sclerosis population.

Incorporation of Inter-Subject Information to Improve the Accuracy of Subject-Specific P300 Classifiers

Although the inter-subject information has been demonstrated to be effective for a rapid calibration of the P300-based brain–computer interface (BCI), it has never been comprehensively tested to find if the incorporation of heterogeneous data could enhance the accuracy. This study aims to improve the subject-specific P300 classifier by adding other subject’s data. A classifier calibration strategy, weighted ensemble learning generic information (WELGI), was developed, in which elementary classifiers were constructed by using both the intra- and inter-subject information and then integrated into a strong classifier with a weight assessment. 55 subjects were recruited to spell 20 characters offline using the conventional P300-based BCI, i.e. the P300-speller. Four different metrics, the P300 accuracy and precision, the round accuracy, and the character accuracy, were performed for a comprehensive investigation. The results revealed that the classifier constructed on the training dataset in combination with adding other subject’s data was significantly superior to that without the inter-subject information. Therefore, the WELGI is an effective classifier calibration strategy which uses the inter-subject information to improve the accuracy of subject-specific P300 classifiers, and could also be applied to other BCI paradigms.

Functional Brain Connectivity as a New Feature for P300 Speller

The brain is a large-scale complex network often referred to as the "connectome". Cognitive functions and information processing are mainly based on the interactions between distant brain regions. However, most of the 'feature extraction' methods used in the context of Brain Computer Interface (BCI) ignored the possible functional relationships between different signals recorded from distinct brain areas. In this paper, the functional connectivity quantified by the phase locking value (PLV) was introduced to characterize the evoked responses (ERPs) obtained in the case of target and non-targets visual stimuli. We also tested the possibility of using the functional connectivity in the context of 'P300 speller'. The proposed approach was compared to the well-known methods proposed in the state of the art of "P300 Speller", mainly the peak picking, the area, time/frequency based features, the xDAWN spatial filtering and the stepwise linear discriminant analysis (SWLDA). The electroencephalographic (EEG) signals recorded from ten subjects were analyzed offline. The results indicated that phase synchrony offers relevant information for the classification in a P300 speller. High synchronization between the brain regions was clearly observed during target trials, although no significant synchronization was detected for a non-target trial. The results showed also that phase synchrony provides higher performance than some existing methods for letter classification in a P300 speller principally when large number of trials is available. Finally, we tested the possible combination of both approaches (classical features and phase synchrony). Our findings showed an overall improvement of the performance of the P300-speller when using Peak picking, the area and frequency based features. Similar performances were obtained compared to xDAWN and SWLDA when using large number of trials.

P300 Detection Based on EEG Shape Features

We present a novel approach to describe a P300 by a shape-feature vector, which offers several advantages over the feature vector used by the BCI2000 system. Additionally, we present a calibration algorithm that reduces the dimensionality of the shape-feature vector, the number of trials, and the electrodes needed by a Brain Computer Interface to accurately detect P300s; we also define a method to find a template that best represents, for a given electrode, the subject's P300 based on his/her own acquired signals. Our experiments with 21 subjects showed that the SWLDA's performance using our shape-feature vector was 93%, that is, 10% higher than the one obtained with BCI2000-feature's vector. The shape-feature vector is 34-dimensional for every electrode; however, it is possible to significantly reduce its dimensionality while keeping a high sensitivity. The validation of the calibration algorithm showed an averaged area under the ROC (AUROC) curve of 0.88. Also, most of the subjects needed less than 15 trials to have an AUROC superior to 0.8. Finally, we found that the electrode C4 also leads to better classification.

Convolutional Neural Network for Multi-Category Rapid Serial Visual Presentation BCI

Brain computer interfaces rely on machine learning (ML) algorithms to decode the brain's electrical activity into decisions. For example, in rapid serial visual presentation (RSVP) tasks, the subject is presented with a continuous stream of images containing rare target images among standard images, while the algorithm has to detect brain activity associated with target images. Here, we continue our previous work, presenting a deep neural network model for the use of single trial EEG classification in RSVP tasks. Deep neural networks have shown state of the art performance in computer vision and speech recognition and thus have great promise for other learning tasks, like classification of EEG samples. In our model, we introduce a novel spatio-temporal regularization for EEG data to reduce overfitting. We show improved classification performance compared to our earlier work on a five categories RSVP experiment. In addition, we compare performance on data from different sessions and validate the model on a public benchmark data set of a P300 speller task. Finally, we discuss the advantages of using neural network models compared to manually designing feature extraction algorithms.

Multivariate assessment of event-related potentials with the t-CWT method

BACKGROUND

Event-related brain potentials (ERPs) are usually assessed with univariate statistical tests although they are essentially multivariate objects. Brain-computer interface applications are a notable exception to this practice, because they are based on multivariate classification of single-trial ERPs. Multivariate ERP assessment can be facilitated by feature extraction methods. One such method is t-CWT, a mathematical-statistical algorithm based on the continuous wavelet transform (CWT) and Student's t-test.

RESULTS

This article begins with a geometric primer on some basic concepts of multivariate statistics as applied to ERP assessment in general and to the t-CWT method in particular. Further, it presents for the first time a detailed, step-by-step, formal mathematical description of the t-CWT algorithm. A new multivariate outlier rejection procedure based on principal component analysis in the frequency domain is presented as an important pre-processing step. The MATLAB and GNU Octave implementation of t-CWT is also made publicly available for the first time as free and open source code. The method is demonstrated on some example ERP data obtained in a passive oddball paradigm. Finally, some conceptually novel applications of the multivariate approach in general and of the t-CWT method in particular are suggested and discussed.

CONCLUSIONS

Hopefully, the publication of both the t-CWT source code and its underlying mathematical algorithm along with a didactic geometric introduction to some basic concepts of multivariate statistics would make t-CWT more accessible to both users and developers in the field of neuroscience research.

Incorporating advanced language models into the P300 speller using particle filtering

OBJECTIVE

The P300 speller is a common brain-computer interface (BCI) application designed to communicate language by detecting event related potentials in a subject's electroencephalogram signal. Information about the structure of natural language can be valuable for BCI communication, but attempts to use this information have thus far been limited to rudimentary n-gram models. While more sophisticated language models are prevalent in natural language processing literature, current BCI analysis methods based on dynamic programming cannot handle their complexity.

APPROACH

Sampling methods can overcome this complexity by estimating the posterior distribution without searching the entire state space of the model. In this study, we implement sequential importance resampling, a commonly used particle filtering (PF) algorithm, to integrate a probabilistic automaton language model.

MAIN RESULT

This method was first evaluated offline on a dataset of 15 healthy subjects, which showed significant increases in speed and accuracy when compared to standard classification methods as well as a recently published approach using a hidden Markov model (HMM). An online pilot study verified these results as the average speed and accuracy achieved using the PF method was significantly higher than that using the HMM method.

SIGNIFICANCE

These findings strongly support the integration of domain-specific knowledge into BCI classification to improve system performance.

Grouped Automatic Relevance Determination and Its Application in Channel Selection for P300 BCIs

During the development of a brain-computer interface, it is beneficial to exploit information in multiple electrode signals. However, a small channel subset is favored for not only machine learning feasibility, but also practicality in commercial and clinical BCI applications. An embedded channel selection approach based on grouped automatic relevance determination is proposed. The proposed Gaussian conjugate group-sparse prior and the embedded nature of the concerned Bayesian linear model enable simultaneous channel selection and feature classification. Moreover, with the marginal likelihood (evidence) maximization technique, hyper-parameters that determine the sparsity of the model are directly estimated from the training set, avoiding time-consuming cross-validation. Experiments have been conducted on P300 speller BCIs. The results for both public and in-house datasets show that the channels selected by our techniques yield competitive classification performance with the state-of-the-art and are biologically relevant to P300.

Electrode replacement does not affect classification accuracy in dual-session use of a passive brain-computer interface for assessing cognitive workload

The passive brain-computer interface (pBCI) framework has been shown to be a very promising construct for assessing cognitive and affective state in both individuals and teams. There is a growing body of work that focuses on solving the challenges of transitioning pBCI systems from the research laboratory environment to practical, everyday use. An interesting issue is what impact methodological variability may have on the ability to reliably identify (neuro)physiological patterns that are useful for state assessment. This work aimed at quantifying the effects of methodological variability in a pBCI design for detecting changes in cognitive workload. Specific focus was directed toward the effects of replacing electrodes over dual sessions (thus inducing changes in placement, electromechanical properties, and/or impedance between the electrode and skin surface) on the accuracy of several machine learning approaches in a binary classification problem. In investigating these methodological variables, it was determined that the removal and replacement of the electrode suite between sessions does not impact the accuracy of a number of learning approaches when trained on one session and tested on a second. This finding was confirmed by comparing to a control group for which the electrode suite was not replaced between sessions. This result suggests that sensors (both neurological and peripheral) may be removed and replaced over the course of many interactions with a pBCI system without affecting its performance. Future work on multi-session and multi-day pBCI system use should seek to replicate this (lack of) effect between sessions in other tasks, temporal time courses, and data analytic approaches while also focusing on non-stationarity and variable classification performance due to intrinsic factors.

Non-target adjacent stimuli classification improves performance of classical ERP-based brain computer interface

OBJECTIVE

The classical ERP-based speller, or P300 Speller, is one of the most commonly used paradigms in the field of Brain Computer Interfaces (BCI). Several alterations to the visual stimuli presentation system have been developed to avoid unfavorable effects elicited by adjacent stimuli. However, there has been little, if any, regard to useful information contained in responses to adjacent stimuli about spatial location of target symbols. This paper aims to demonstrate that combining the classification of non-target adjacent stimuli with standard classification (target versus non-target) significantly improves classical ERP-based speller efficiency.

APPROACH

Four SWLDA classifiers were trained and combined with the standard classifier: the lower row, upper row, right column and left column classifiers. This new feature extraction procedure and the classification method were carried out on three open databases: the UAM P300 database (Universidad Autonoma Metropolitana, Mexico), BCI competition II (dataset IIb) and BCI competition III (dataset II).

MAIN RESULTS

The inclusion of the classification of non-target adjacent stimuli improves target classification in the classical row/column paradigm. A gain in mean single trial classification of 9.6% and an overall improvement of 25% in simulated spelling speed was achieved.

SIGNIFICANCE

We have provided further evidence that the ERPs produced by adjacent stimuli present discriminable features, which could provide additional information about the spatial location of intended symbols. This work promotes the searching of information on the peripheral stimulation responses to improve the performance of emerging visual ERP-based spellers.

A novel algorithm to enhance P300 in single trials: application to lie detection using F-score and SVM

The investigation of lie detection methods based on P300 potentials has drawn much interest in recent years. We presented a novel algorithm to enhance signal-to-noise ratio (SNR) of P300 and applied it in lie detection to increase the classification accuracy. Thirty-four subjects were divided randomly into guilty and innocent groups, and the EEG signals on 14 electrodes were recorded. A novel spatial denoising algorithm (SDA) was proposed to reconstruct the P300 with a high SNR based on independent component analysis. The differences between the proposed method and our/other early published methods mainly lie in the extraction and feature selection method of P300. Three groups of features were extracted from the denoised waves; then, the optimal features were selected by the F-score method. Selected feature samples were finally fed into three classical classifiers to make a performance comparison. The optimal parameter values in the SDA and the classifiers were tuned using a grid-searching training procedure with cross-validation. The support vector machine (SVM) approach was adopted to combine with an F-score because this approach had the best performance. The presented model F-score_SVM reaches a significantly higher classification accuracy for P300 (specificity of 96.05%) and non-P300 (sensitivity of 96.11%) compared with the results obtained without using SDA and compared with the results obtained by other classification models. Moreover, a higher individual diagnosis rate can be obtained compared with previous methods, and the presented method requires only a small number of stimuli in the real testing application.

A method for optimizing EEG electrode number and configuration for signal acquisition in P300 speller systems

OBJECTIVE

The P300 speller is intended to restore communication to patients with advanced neuromuscular disorders, but clinical implementation may be hindered by several factors, including system setup, burden, and cost. Our goal was to develop a method that can overcome these barriers by optimizing EEG electrode number and placement for P300 studies within a population of subjects.

METHODS

A Gibbs sampling method was developed to find the optimal electrode configuration given a set of P300 speller data. The method was tested on a set of data from 15 healthy subjects using an established 32-electrode pattern. Resulting electrode configurations were then validated using online prospective testing with a naïve Bayes classifier in 15 additional healthy subjects.

RESULTS

The method yielded a set of four posterior electrodes (PO₈, PO₇, POZ, CPZ), which produced results that are likely sufficient to be clinically effective. In online prospective validation testing, no significant difference was found between subjects' performances using the reduced and the full electrode configurations.

CONCLUSIONS

The proposed method can find reduced sets of electrodes within a subject population without reducing performance.

SIGNIFICANCE

Reducing the number of channels may reduce costs, set-up time, signal bandwidth, and computation requirements for practical online P300 speller implementation.