Bilinear common spatial pattern for single-trial ERP-based rapid serial visual presentation triage

A spatial-temporal linear feature learning algorithm for P300-based brain-computer interfaces

Speller brain-computer interface (BCI) systems can help neuromuscular disorders patients write their thoughts by using the electroencephalogram (EEG) signals by just focusing on the speller tasks. For practical speller-based BCI systems, the P300 event-related brain potential is measured by using the EEG signal. In this paper, we design a robust machine-learning algorithm for P300 target detection. The novel spatial-temporal linear feature learning (STLFL) algorithm is proposed to extract high-level P300 features. The STLFL method is a modified linear discriminant analysis technique focusing on the spatial-temporal aspects of information extraction. A new P300 detection structure is then proposed based on the combination of the novel STLFL feature extraction and discriminative restricted Boltzmann machine (DRBM) for the classification approach (STLFL + DRBM). The effectiveness of the proposed technique is evaluated using two state-of-the-art P300 BCI datasets. Across the two available databases, we show that in terms of average target recognition accuracy and standard deviation values, the proposed STLFL + DRBM method outperforms traditional methods by 33.5, 78.5, 93.5, and 98.5% for 1, 5, 10, and 15 repetitions, respectively, in BCI competition III datasets II and by 71.3, 100, 100, and 100% for 1, 5, 10, and 15 repetitions, respectively, in BCI competition II datasets II and by 67.5 ± 4, 84.2 ± 2.5, 93.5 ± 1, 96.3 ± 1, and 98.4 ± 0.5% for rapid serial visual presentation (RSVP) based dataset in repetitions 1-5. The method has some advantages over the existing variants including its efficiency, robustness with a small number of training samples, and a high ability to create discriminative features between classes.

Optimization of data pre-processing methods for time-series classification of electroencephalography data

The performance of time-series classification of electroencephalographic data varies strongly across experimental paradigms and study participants. Reasons are task-dependent differences in neuronal processing and seemingly random variations between subjects, amongst others. The effect of data pre-processing techniques to ameliorate these challenges is relatively little studied. Here, the influence of spatial filter optimization methods and non-linear data transformation on time-series classification performance is analyzed by the example of high-frequency somatosensory evoked responses. This is a model paradigm for the analysis of high-frequency electroencephalography data at a very low signal-to-noise ratio, which emphasizes the differences of the explored methods. For the utilized data, it was found that the individual signal-to-noise ratio explained up to 74% of the performance differences between subjects. While data pre-processing was shown to increase average time-series classification performance, it could not fully compensate the signal-to-noise ratio differences between the subjects. This study proposes an algorithm to prototype and benchmark pre-processing pipelines for a paradigm and data set at hand. Extreme learning machines, Random Forest, and Logistic Regression can be used quickly to compare a set of potentially suitable pipelines. For subsequent classification, however, machine learning models were shown to provide better accuracy.

An improved EEGNet for single-trial EEG classification in rapid serial visual presentation task

As a new type of brain–computer interface (BCI), the rapid serial visual presentation (RSVP) paradigm has attracted significant attention. The mechanism of RSVP is detecting the P300 component corresponding to the target image to realize fast and correct recognition. This paper proposed an improved EEGNet model to achieve good performance in offline and online data. Specifically, the data were filtered by xDAWN to enhance the signal-to-noise ratio of the electroencephalogram (EEG) signals. The focal loss function was used instead of the cross-entropy loss function to solve the classification problems of unbalanced samples. Additionally, the subject-specific data were fed to the improved EEGNet model to obtain a subject-specific model. We applied the proposed model at the BCI Controlled Robot Contest in World Robot Contest 2021 and won the second place. The average recall rate of the four participants reached 51.56% in triple classification. In the offline data benchmark dataset (64 subjects-RSVP tasks), the average recall rates of groups A and B reached 76.07% and 78.11%, respectively. We provided an alternative method to identify targets based on the RSVP paradigm.

Toward passive BCI: asynchronous decoding of neural responses to direction- and angle-specific perturbations during a simulated cockpit scenario

Neuroimaging studies have provided proof that loss of balance evokes specific neural transient wave complexes in electroencephalography (EEG), called perturbation evoked potentials (PEPs). Online decoding of balance perturbations from ongoing EEG signals can establish the possibility of implementing passive brain-computer interfaces (pBCIs) as a part of aviation/driving assistant systems. In this study, we investigated the feasibility of identifying the existence and expression of perturbations in four different conditions by using EEG signals. Fifteen healthy participants experienced four various postural changes while they sat in a glider cockpit. Sudden perturbations were exposed by a robot connected to a glider and moved to the right and left directions with tilting angles of 5 and 10 degrees. Perturbations occurred in an oddball paradigm in which participants were not aware of the time and expression of the perturbations. We employed a hierarchical approach to separate the perturbation and rest, and then discriminate the expression of perturbations. The performance of the BCI system was evaluated by using classification accuracy and F1 score. Asynchronously, we achieved average accuracies of 89.83 and 73.64% and average F1 scores of 0.93 and 0.60 for binary and multiclass classification, respectively. These results manifest the practicality of pBCI for the detection of balance disturbances in a realistic situation.

Asynchronous Video Target Detection Based on Single-Trial EEG Signals

Event-related potentials (ERPs) are widely used in brain-computer interface (BCI) systems to detect sensitive targets. However, asynchronous BCI systems based on video-target-evoked ERPs can pose a challenge in real-world applications due to the absence of an explicit target onset time and the time jitter of the detection latency. To address this challenge, we developed an asynchronous detection framework for video target detection. In this framework, an ERP alignment method based on the principle of iterative minimum distance square error (MDSE) was proposed for constructing an ERP template and aligning signals on the same base to compensate for possible time jitter. Using this method, ERP response characteristics induced by video targets were estimated. Online video target detection results indicated that alignment methods reduced the false alarm more effectively than non-alignment methods. The false alarm of the proposed Aligned-MDSE method was one-third lower than that of existing alignment methods under the same right hit level using limited individual samples. Furthermore, cross-subject results indicated that untrained subjects could directly perform online detection tasks and achieve excellent performance by a general model trained from more than 10 subjects. The proposed asynchronous video target detection framework can thus have a significant impact on real-world BCI applications.

Spatial Filtering in SSVEP-Based BCIs: Unified Framework and New Improvements

OBJECTIVE

In the steady-state visual evoked potential (SSVEP)-based brain computer interfaces (BCIs), spatial filtering, which combines the multi-channel electroencephalography (EEG) signals in order to reduce the non-SSVEP-related component and thus enhance the signal-to-noise ratio (SNR), plays an important role in target recognition. Recently, various spatial filtering algorithms have been developed employing different prior knowledge and characteristics of SSVEPs, however how these algorithms interconnect and differ is not yet fully explored, leading to difficulties in further understanding, utilizing and improving them.

METHODS

We propose a unified framework under which the spatial filtering algorithms can be formulated as generalized eigenvalue problems (GEPs) with four different elements: data, temporal filter, orthogonal projection and spatial filter. Based on the framework, we design new spatial filtering algorithms for improvements through the choice of different elements.

RESULTS

The similarities, differences and relationships among nineteen mainstream spatial filtering algorithms are revealed under the proposed framework. Particularly, it is found that they originate from the canonical correlation analysis (CCA), principal component analysis (PCA), and multi-set CCA, respectively. Furthermore, three new spatial filtering algorithms are developed with enhanced performance validated on two public SSVEP datasets with 45 subjects.

CONCLUSION

The proposed framework provides insights into the underlying relationships among different spatial filtering algorithms and helps the design of new spatial filtering algorithms.

SIGNIFICANCE

This is a systematic study to explore, compare and improve the existing spatial filtering algorithms, which would be significant for further understanding and future development of high performance SSVEP-based BCIs.

A novel dual and triple shifted RSVP paradigm for P300 speller

BACKGROUND

A speller system enables disabled people, specifically those with spinal cord injuries, to visually select and spell characters. A problem of primary speller systems is that they are gaze shift dependent. To overcome this problem, a single Rapid Serial Visual Presentation (RSVP) paradigm was initially introduced in which characters are displayed one-by-one at the center of a screen.

NEW METHOD

Two new protocols, Dual and Triple shifted RSVP paradigms, are introduced and compared against the single paradigm. In the Dual and Triple paradigms, two and three characters are displayed at the center of the screen simultaneously, holding the advantage of displaying the target character twice and three times respectively, compared to the one-time appearance in the single paradigm. To compare the named paradigms, three subjects participated in experiments using all three paradigms.

RESULTS

Offline results demonstrate an average character detection accuracy of 97% for the single and double protocols, and 80% for the Triple paradigm. In addition, average ITR is calculated to be 5.45, 7.62 and 7.90 bit/min for the single, Dual and Triple paradigms respectively. Results identify the Dual RSVP paradigm as the most suitable approach that provides the best balance between ITR and character detection accuracy.

COMPARISON WITH EXISTING METHODS

The novel speller system (the Dual paradigm) suggested in this paper demonstrates improved performance compared to existing methods, and overcomes the gaze dependency issue.

CONCLUSIONS

Overall, our novel method is a reliable alternative that both removes limitations for users suffering from impaired oculomotor control and improves performance.

Enhancement of Group Perception via a Collaborative Brain-Computer Interface

OBJECTIVE

We aimed at improving group performance in a challenging visual search task via a hybrid collaborative brain-computer interface (cBCI).

METHODS

Ten participants individually undertook a visual search task where a display was presented for 250 ms, and they had to decide whether a target was present or not. Local temporal correlation common spatial pattern (LTCCSP) was used to extract neural features from response- and stimulus-locked EEG epochs. The resulting feature vectors were extended by including response times and features extracted from eye movements. A classifier was trained to estimate the confidence of each group member. cBCI-assisted group decisions were then obtained using a confidence-weighted majority vote.

RESULTS

Participants were combined in groups of different sizes to assess the performance of the cBCI. Results show that LTCCSP neural features, response times, and eye movement features significantly improve the accuracy of the cBCI over what we achieved with previous systems. For most group sizes, our hybrid cBCI yields group decisions that are significantly better than majority-based group decisions.

CONCLUSION

The visual task considered here was much harder than a task we used in previous research. However, thanks to a range of technological enhancements, our cBCI has delivered a significant improvement over group decisions made by a standard majority vote.

SIGNIFICANCE

With previous cBCIs, groups may perform better than single non-BCI users. Here, cBCI-assisted groups are more accurate than identically sized non-BCI groups. This paves the way to a variety of real-world applications of cBCIs where reducing decision errors is vital.

Spatio-temporal common pattern: A companion method for ERP analysis in the time domain

BACKGROUND

Already used at the incept of research on event-related potentials (ERP) over half a century ago, the arithmetic mean is still the benchmark for ERP estimation. Such estimation, however, requires a large number of sweeps and/or a careful rejection of artifacts affecting the electroencephalographic recording.

NEW METHOD

In this article we propose a method for estimating ERPs as they are naturally contaminated by biological and instrumental artifacts. The proposed estimator makes use of multivariate spatio-temporal filtering to increase the signal-to-noise ratio. This approach integrates a number of relevant advances in ERP data analysis, such as single-sweep adaptive estimation of amplitude and latency and the use of multivariate regression to account for ERP overlapping in time.

RESULTS

We illustrate the effectiveness of the proposed estimator analyzing a dataset comprising 24 subjects involving a visual odd-ball paradigm, without performing any artifact rejection.

COMPARISON WITH EXISTING METHOD(S)

As compared to the arithmetic average, a lower number of sweeps is needed. Furthermore, artifact rejection can be performed roughly using permissive automatic procedures.

CONCLUSION

The proposed ensemble average estimator yields a reference companion to the arithmetic ensemble average estimation, suitable both in clinical and research settings. The method can be applied equally to event related fields (ERF) recorded by means of magnetoencephalography. In this article we describe all necessary methodological details to promote testing and comparison of this proposed method by peers. Furthermore, we release a MATLAB toolbox, a plug-in for the EEGLAB software suite and a stand-alone executable application.

The effect of target and non-target similarity on neural classification performance: a boost from confidence

Brain computer interaction (BCI) technologies have proven effective in utilizing single-trial classification algorithms to detect target images in rapid serial visualization presentation tasks. While many factors contribute to the accuracy of these algorithms, a critical aspect that is often overlooked concerns the feature similarity between target and non-target images. In most real-world environments there are likely to be many shared features between targets and non-targets resulting in similar neural activity between the two classes. It is unknown how current neural-based target classification algorithms perform when qualitatively similar target and non-target images are presented. This study address this question by comparing behavioral and neural classification performance across two conditions: first, when targets were the only infrequent stimulus presented amongst frequent background distracters; and second when targets were presented together with infrequent non-targets containing similar visual features to the targets. The resulting findings show that behavior is slower and less accurate when targets are presented together with similar non-targets; moreover, single-trial classification yielded high levels of misclassification when infrequent non-targets are included. Furthermore, we present an approach to mitigate the image misclassification. We use confidence measures to assess the quality of single-trial classification, and demonstrate that a system in which low confidence trials are reclassified through a secondary process can result in improved performance.

Non-invasive single-trial EEG detection of evoked human neocortical population spikes

QUESTION

Human high-frequency (>400 Hz) components of somatosensory evoked potentials (hf-SEPs), which can be recorded non-invasively at the scalp, are generated by cortical population spikes, as inferred from microelectrode recordings in non-human primates. It is a critical limitation to broader neurophysiological study of hf-SEPs in that hundreds of responses have to be averaged to detect hf-SEPs reliably. Here, we establish a framework for detecting human hf-SEPs non-invasively in single trials.

METHODS

Spatio-temporal features were extracted from band-pass filtered (400-900 Hz) hf-SEPs by bilinear Common Spatio-Temporal Patterns (bCSTP) and then classified by a weighted Extreme Learning Machine (w-ELM). The effect of varying signal-to-noise ratio (SNR), number of trials, and degree of w-ELM re-weighting was characterized using surrogate data. For practical demonstration of the algorithm, median nerve hf-SEPs were recorded inside a shielded room in four subjects, spanning the hf-SEP signal-to-noise ratio characteristic for a larger population, utilizing a custom-built 29-channel low-noise EEG amplifier.

RESULTS

Using surrogate data, the SNR proved to be pivotal to detect hf-SEPs in single trials efficiently, with the trade-off between sensitivity and specificity of the algorithm being obtained by the w-ELM re-weighting parameter. In practice, human hf-SEPs were detected non-invasively in single trials with a sensitivity of up to 99% and a specificity of up to 97% in two subjects, even without any recourse to knowledge of stimulus timing. Matching with the results of the surrogate data analysis, these rates dropped to 62-79% sensitivity and 18-31% specificity in two subjects with lower SNR.

CONCLUSIONS

Otherwise buried in background noise, human high-frequency EEG components can be extracted from low-noise recordings. Specifically, refined supervised filter optimization and classification enables the reliable detection of single-trial hf-SEPs, representing non-invasive correlates of cortical population spikes.

SIGNIFICANCE

While low-frequency EEG reflects summed postsynaptic potentials, and thereby neuronal input, we suggest that high-frequency EEG (>400 Hz) can provide non-invasive access to the unaveraged output of neuronal computation, i.e., single-trial population spike activity evoked in the responsive neuronal ensemble.

Recording human cortical population spikes non-invasively--An EEG tutorial

BACKGROUND

Non-invasively recorded somatosensory high-frequency oscillations (sHFOs) evoked by electric nerve stimulation are markers of human cortical population spikes. Previously, their analysis was based on massive averaging of EEG responses. Advanced neurotechnology and optimized off-line analysis can enhance the signal-to-noise ratio of sHFOs, eventually enabling single-trial analysis.

METHODS

The rationale for developing dedicated low-noise EEG technology for sHFOs is unfolded. Detailed recording procedures and tailored analysis principles are explained step-by-step. Source codes in Matlab and Python are provided as supplementary material online.

RESULTS

Combining synergistic hardware and analysis improvements, evoked sHFOs at around 600 Hz ('σ-bursts') can be studied in single-trials. Additionally, optimized spatial filters increase the signal-to-noise ratio of components at about 1 kHz ('κ-bursts') enabling their detection in non-invasive surface EEG.

CONCLUSIONS

sHFOs offer a unique possibility to record evoked human cortical population spikes non-invasively. The experimental approaches and algorithms presented here enable also non-specialized EEG laboratories to combine measurements of conventional low-frequency EEG with the analysis of concomitant cortical population spike responses.

The analytic bilinear discrimination of single-trial EEG signals in rapid image triage

The linear discriminant analysis (LDA) method is a classical and commonly utilized technique for dimensionality reduction and classification in brain-computer interface (BCI) systems. Being a first-order discriminator, LDA is usually preceded by the feature extraction of electroencephalogram (EEG) signals, as multi-density EEG data are of second order. In this study, an analytic bilinear classification method which inherits and extends LDA is proposed. This method considers 2-dimentional EEG signals as the feature input and performs classification using the optimized complex-valued bilinear projections. Without being transformed into frequency domain, the complex-valued bilinear projections essentially spatially and temporally modulate the phases and magnitudes of slow event-related potentials (ERPs) elicited by distinct brain states in the sense that they become more separable. The results show that the proposed method has demonstrated its discriminating capability in the development of a rapid image triage (RIT) system, which is a challenging variant of BCIs due to the fast presentation speed and consequently overlapping of ERPs.

Estimating endogenous changes in task performance from EEG

Brain wave activity is known to correlate with decrements in behavioral performance as individuals enter states of fatigue, boredom, or low alertness.Many BCI technologies are adversely affected by these changes in user state, limiting their application and constraining their use to relatively short temporal epochs where behavioral performance is likely to be stable. Incorporating a passive BCI that detects when the user is performing poorly at a primary task, and adapts accordingly may prove to increase overall user performance. Here, we explore the potential for extending an established method to generate continuous estimates of behavioral performance from ongoing neural activity; evaluating the extended method by applying it to the original task domain, simulated driving; and generalizing the method by applying it to a BCI-relevant perceptual discrimination task. Specifically, we used EEG log power spectra and sequential forward floating selection (SFFS) to estimate endogenous changes in behavior in both a simulated driving task and a perceptual discrimination task. For the driving task the average correlation coefficient between the actual and estimated lane deviation was 0.37 ± 0.22 (μ ± σ). For the perceptual discrimination task we generated estimates of accuracy, reaction time, and button press duration for each participant. The correlation coefficients between the actual and estimated behavior were similar for these three metrics (accuracy = 0.25 ± 0.37, reaction time = 0.33 ± 0.23, button press duration = 0.36 ± 0.30). These findings illustrate the potential for modeling time-on-task decrements in performance from concurrent measures of neural activity.

The synergy between complex channel-specific FIR filter and spatial filter for single-trial EEG classification

The common spatial pattern analysis (CSP), a frequently utilized feature extraction method in brain-computer-interface applications, is believed to be time-invariant and sensitive to noises, mainly due to an inherent shortcoming of purely relying on spatial filtering. Therefore, temporal/spectral filtering which can be very effective to counteract the unfavorable influence of noises is usually used as a supplement. This work integrates the CSP spatial filters with complex channel-specific finite impulse response (FIR) filters in a natural and intuitive manner. Each hybrid spatial-FIR filter is of high-order, data-driven and is unique to its corresponding channel. They are derived by introducing multiple time delays and regularization into conventional CSP. The general framework of the method follows that of CSP but performs better, as proven in single-trial classification tasks like event-related potential detection and motor imagery.