Enhancement for P300-speller classification using multi-window discriminative canonical pattern matching

A comparative study of stereo-dependent SSVEP targets and their impact on VR-BCI performance

Steady-state visual evoked potential brain-computer interfaces (SSVEP-BCI) have attracted significant attention due to their ease of deployment and high performance in terms of information transfer rate (ITR) and accuracy, making them a promising candidate for integration with consumer electronics devices. However, as SSVEP characteristics are directly associated with visual stimulus attributes, the influence of stereoscopic vision on SSVEP as a critical visual attribute has yet to be fully explored. Meanwhile, the promising combination of virtual reality (VR) devices and BCI applications is hampered by the significant disparity between VR environments and traditional 2D displays. This is not only due to the fact that screen-based SSVEP generally operates under static, stable conditions with simple and unvaried visual stimuli but also because conventional luminance-modulated stimuli can quickly induce visual fatigue. This study attempts to address these research gaps by designing SSVEP paradigms with stereo-related attributes and conducting a comparative analysis with the traditional 2D planar paradigm under the same VR environment. This study proposed two new paradigms: the 3D paradigm and the 3D-Blink paradigm. The 3D paradigm induces SSVEP by modulating the luminance of spherical targets, while the 3D-Blink paradigm employs modulation of the spheres' opacity instead. The results of offline 4-object selection experiments showed that the accuracy of 3D and 2D paradigm was 85.67 and 86.17% with canonical correlation analysis (CCA) and 86.17 and 91.73% with filter bank canonical correlation analysis (FBCCA), which is consistent with the reduction in the signal-to-noise ratio (SNR) of SSVEP harmonics for the 3D paradigm observed in the frequency-domain analysis. The 3D-Blink paradigm achieved 75.00% of detection accuracy and 27.02 bits/min of ITR with 0.8 seconds of stimulus time and task-related component analysis (TRCA) algorithm, demonstrating its effectiveness. These findings demonstrate that the 3D and 3D-Blink paradigms supported by VR can achieve improved user comfort and satisfactory performance, while further algorithmic optimization and feature analysis are required for the stereo-related paradigms. In conclusion, this study contributes to a deeper understanding of the impact of binocular stereoscopic vision mechanisms on SSVEP paradigms and promotes the application of SSVEP-BCI in diverse VR environments.

A Methodology for Enhancing SSVEP Features Using Adaptive Filtering Based on the Spatial Distribution of EEG Signals

In this paper, we propose a classification algorithm of EEG signal based on canonical correlation analysis (CCA) and integrated with adaptive filtering. It can enhance the detection of steady-state visual evoked potentials (SSVEPs) in a brain-computer interface (BCI) speller. An adaptive filter is employed in front of the CCA algorithm to improve the signal-to-noise ratio (SNR) of SSVEP signals by removing background electroencephalographic (EEG) activities. The ensemble method is developed to integrate recursive least squares (RLS) adaptive filter corresponding to multiple stimulation frequencies. The method is tested by the SSVEP signal recorded from six targets by actual experiment and the EEG in a public SSVEP dataset of 40 targets from Tsinghua University. The accuracy rates of the CCA method and the CCA-based integrated RLS filter algorithm (RLS-CCA method) are compared. Experiment results show that the proposed RLS-CCA-based method significantly improves the classification accuracy compared with the pure CCA method. Especially when the number of EEG leads is low (three occipital electrodes and five non occipital electrodes), its advantage is more significant, and accuracy reaches 91.23%, which is more suitable for wearable environments where high-density EEG is not easy to collect.

Single-trial P300 classification algorithm based on centralized multi-person data fusion CNN

Introduction

Currently, it is still a challenge to detect single-trial P300 from electroencephalography (EEG) signals. In this paper, to address the typical problems faced by existing single-trial P300 classification, such as complex, time-consuming and low accuracy processes, a single-trial P300 classification algorithm based on multiplayer data fusion convolutional neural network (CNN) is proposed to construct a centralized collaborative brain-computer interfaces (cBCI) for fast and highly accurate classification of P300 EEG signals.

Methods

In this paper, two multi-person data fusion methods (parallel data fusion and serial data fusion) are used in the data pre-processing stage to fuse multi-person EEG information stimulated by the same task instructions, and then the fused data is fed as input to the CNN for classification. In building the CNN network for single-trial P300 classification, the Conv layer was first used to extract the features of single-trial P300, and then the Maxpooling layer was used to connect the Flatten layer for secondary feature extraction and dimensionality reduction, thereby simplifying the computation. Finally batch normalisation is used to train small batches of data in order to better generalize the network and speed up single-trial P300 signal classification.

Results

In this paper, the above new algorithms were tested on the Kaggle dataset and the Brain-Computer Interface (BCI) Competition III dataset, and by analyzing the P300 waveform features and EEG topography and the four standard evaluation metrics, namely Accuracy, Precision, Recall and F1-score,it was demonstrated that the single-trial P300 classification algorithm after two multi-person data fusion CNNs significantly outperformed other classification algorithms.

Discussion

The results show that the single-trial P300 classification algorithm after two multi-person data fusion CNNs significantly outperformed the single-person model, and that the single-trial P300 classification algorithm with two multi-person data fusion CNNs involves smaller models, fewer training parameters, higher classification accuracy and improves the overall P300-cBCI classification rate and actual performance more effectively with a small amount of sample information compared to other algorithms.

Dynamic probability integration for electroencephalography-based rapid serial visual presentation performance enhancement: Application in nighttime vehicle detection

Background

Rapid serial visual presentation (RSVP) has become a popular target detection method by decoding electroencephalography (EEG) signals, owing to its sensitivity and effectiveness. Most current research on EEG-based RSVP tasks focused on feature extraction algorithms developed to deal with the non-stationarity and low signal-to-noise ratio (SNR) of EEG signals. However, these algorithms cannot handle the problem of no event-related potentials (ERP) component or miniature ERP components caused by the attention lapses of human vision in abnormal conditions. The fusion of human-computer vision can obtain complementary information, making it a promising way to become an efficient and general way to detect objects, especially in attention lapses.

Methods

Dynamic probability integration (DPI) was proposed in this study to fuse human vision and computer vision. A novel basic probability assignment (BPA) method was included, which can fully consider the classification capabilities of different heterogeneous information sources for targets and non-targets and constructs the detection performance model for the weight generation based on classification capabilities. Furthermore, a spatial-temporal hybrid common spatial pattern-principal component analysis (STHCP) algorithm was designed to decode EEG signals in the RSVP task. It is a simple and effective method of distinguishing target and non-target using spatial-temporal features.

Results

A nighttime vehicle detection based on the RSVP task was performed to evaluate the performance of DPI and STHCP, which is one of the conditions of attention lapses because of its decrease in visual information. The average AUC of DPI was 0.912 ± 0.041 and increased by 11.5, 5.2, 3.4, and 1.7% compared with human vision, computer vision, naive Bayesian fusion, and dynamic belief fusion (DBF), respectively. A higher average balanced accuracy of 0.845 ± 0.052 was also achieved using DPI, representing that DPI has the balanced detection capacity of target and non-target. Moreover, STHCP obtained the highest AUC of 0.818 ± 0.06 compared with the other two baseline methods and increased by 15.4 and 23.4%.

Conclusion

Experimental results indicated that the average AUC and balanced accuracy of the proposed fusion method were higher than individual detection methods used for fusion, as well as two excellent fusion methods. It is a promising way to improve detection performance in RSVP tasks, even in abnormal conditions.

Toward P300-brain-computer interface access to contextual scene displays for AAC: An initial exploration of context and asymmetry processing in healthy adults

Brain-computer interfaces for augmentative and alternative communication (BCI-AAC) may help overcome physical barriers to AAC access. Traditionally, visually based P300-BCI-AAC displays utilize a symmetrical grid layout. Contextual scene displays are composed of context-rich images (e.g., photographs) and may support AAC success. However, contextual scene displays contrast starkly with the standard P300-grid approach. Understanding the neurological processes from which BCI-AAC devices function is crucial to human-centered computing for BCI-AAC. Therefore, the aim of this multidisciplinary investigation is to provide an initial exploration of contextual scene use for BCI-AAC.

METHODS

Participants completed three experimental conditions to evaluate the effects of item arrangement asymmetry and context on P300-based BCI-AAC signals and offline BCI-AAC accuracy, including 1) the full contextual scene condition, 2) asymmetrical item arraignment without context condition and 3) the grid condition. Following each condition, participants completed task-evaluation ratings (e.g., engagement). Offline BCI-AAC accuracy for each condition was evaluated using cross-validation.

RESULTS

Display asymmetry significantly decreased P300 latency in the centro-parietal cluster. P300 amplitudes in the frontal cluster were decreased, though nonsignificantly. Display context significantly increased N170 amplitudes in the occipital cluster, and N400 amplitudes in the centro-parietal and occipital clusters. Scenes were rated as more visually appealing and engaging, and offline BCI-AAC performance for the scene condition was not statistically different from the grid standard.

CONCLUSION

Findings support the feasibility of incorporating scene-based displays for P300-BCI-AAC development to help provide communication for individuals with minimal or emerging language and literacy skills.

Latency Aligning Task-related Component Analysis Using Wave Propagation for Enhancing SSVEP-based BCIs

Due to the high robustness to artifacts, steady-state visual evoked potential (SSVEP) has been widely applied to construct high-speed brain-computer interfaces (BCIs). Thus far, many spatial filtering methods have been proposed to enhance the target identification performance for SSVEP-based BCIs, and task-related component analysis (TRCA) is among the most effective ones. In this paper, we further extend TRCA and propose a new method called Latency Aligning TRCA (LA-TRCA), which aligns visual latencies on channels to obtain accurate phase information from task-related signals. Based on the SSVEP wave propagation theory, SSVEP spreads from posterior occipital areas over the cortex with a fixed phase velocity. Via estimation of the phase velocity using phase shifts of channels, the visual latencies on different channels can be determined for inter-channel alignment. TRCA is then applied to aligned data epochs for target recognition. For the validation purpose, the classification performance comparison between the proposed LA-TRCA and TRCA-based expansions were performed on two different SSVEP datasets. The experimental results illustrated that the proposed LA-TRCA method outperformed the other TRCA-based expansions, which thus demonstrated the effectiveness of the proposed approach for enhancing the SSVEP detection performance.

Effects of inter-stimulus intervals on concurrent P300 and SSVEP features for hybrid Brain-computer interfaces

BACKGROUND

Recently, we have implemented a high-speed brain-computer interface (BCI) system with a large instruction set using the concurrent P300 and steady-state visual evoked potential (SSVEP) features (also known as hybrid features). However, it remains unclear how to select inter-stimulus interval (ISI) for the proposed BCI system to balance the encoding efficiency and decoding performance.

NEW METHOD

This study developed a 6⁎9 hybrid P300-SSVEP BCI system and investigated a series of ISIs ranged from -175ms to 0ms with a step of 25ms. The influence of ISI on the hybrid features was analyzed from several aspects, including the amplitude of the induced features, classification accuracy, information transfer rate (ITR). Twelve naive subjects were recruited for the experiment.

RESULTS

The results showed the ISI factor had a significant impact on the hybrid features. Specifically, as the values of ISI decreased, the amplitudes of the induced features and accuracies decreased gradually, while the ITRs increased rapidly. It's achieved the highest ITR of 158.50 bits/min when ISI equal to -175ms.

COMPARISON WITH EXISTING METHOD

The optimal ISI in this study achieved superior performance in comparison with the one we used in the previous study.

CONCLUSIONS

The ISI can exert an important influence on the P300-SSVEP BCI system and its optimal value is -175ms in this study, which is significant for developing the high-speed BCI system with larger instruction sets in the future.

Feature selection method based on Menger curvature and LDA theory for a P300 brain-computer interface

Brain-computer interface (BCI) systems decode electroencephalogram signals to establish a channel for direct interaction between the human brain and the external world without the need for muscle or nerve control. The P300 speller, one of the most widely used BCI applications, presents a selection of characters to the user and performs character recognition by identifying P300 event-related potentials from the EEG. Such P300-based BCI systems can reach good levels of accuracy but are difficult to use in day-to-day life due to redundancy and noisy signal. A room for improvement should be considered. We propose a novel hybrid feature selection method for the P300-based BCI system to address the problem of feature redundancy, which combines the Menger curvature and linear discriminant analysis. First, selected strategies are applied separately to a given dataset to estimate the gain for application to each feature. Then, each generated value set is ranked in descending order and judged by a predefined criterion to be suitable in classification models. The intersection of the two approaches is then evaluated to identify an optimal feature subset. The proposed method is evaluated using three public datasets, i.e., BCI Competition III dataset II, BNCI Horizon dataset, and EPFL dataset. Experimental results indicate that compared with other typical feature selection and classification methods, our proposed method has better or comparable performance. Additionally, our proposed method can achieve the best classification accuracy after all epochs in three datasets. In summary, our proposed method provides a new way to enhance the performance of the P300-based BCI speller.

Reduction of the ERP Measurement Time by a Weighted Averaging Using Deep Learning

In clinical examination, event-related potentials (ERPs) are estimated by averaging across multiple responses, which suppresses background EEG. However, acquiring the number of responses needed for this process is time consuming. We therefore propose a method for shortening the measurement time using weighted-average processing based on the output of deep learning. Using P300 as a representative component, here we focused on the shape of the ERP and evaluated whether our method emphasizes the P300 peak amplitude more than conventional averaging, while still maintaining the waveform shape and the P300 peak latency. Thus, using either CNN or EEGNet, the correlation coefficient reflecting the waveform shape, the peak P300 amplitude, and the peak latency were evaluated and compared with the same factors obtained from conventional waveform averaging. Additionally, the degree of background EEG suppression provided by our method was evaluated using the root mean square of the pre-stimulation waveform, and the number of fewer responses required for averaging (i.e., the reduction in measurement time) was calculated.The results showed that compared with P300 values obtained through conventional averaging, our method allowed for the same shape and response latency, but with a higher amplitude, while requiring a smaller number of responses. Our method showed that by using EEGNet, measurement time could be reduced by 13.7%. This corresponds to approximately a 40-second reduction for every 5 minutes of measurement time.