A comparative study of stereo-dependent SSVEP targets and their impact on VR-BCI performance

Steady-state visual evoked potential brain-computer interfaces (SSVEP-BCI) have attracted significant attention due to their ease of deployment and high performance in terms of information transfer rate (ITR) and accuracy, making them a promising candidate for integration with consumer electronics devices. However, as SSVEP characteristics are directly associated with visual stimulus attributes, the influence of stereoscopic vision on SSVEP as a critical visual attribute has yet to be fully explored. Meanwhile, the promising combination of virtual reality (VR) devices and BCI applications is hampered by the significant disparity between VR environments and traditional 2D displays. This is not only due to the fact that screen-based SSVEP generally operates under static, stable conditions with simple and unvaried visual stimuli but also because conventional luminance-modulated stimuli can quickly induce visual fatigue. This study attempts to address these research gaps by designing SSVEP paradigms with stereo-related attributes and conducting a comparative analysis with the traditional 2D planar paradigm under the same VR environment. This study proposed two new paradigms: the 3D paradigm and the 3D-Blink paradigm. The 3D paradigm induces SSVEP by modulating the luminance of spherical targets, while the 3D-Blink paradigm employs modulation of the spheres' opacity instead. The results of offline 4-object selection experiments showed that the accuracy of 3D and 2D paradigm was 85.67 and 86.17% with canonical correlation analysis (CCA) and 86.17 and 91.73% with filter bank canonical correlation analysis (FBCCA), which is consistent with the reduction in the signal-to-noise ratio (SNR) of SSVEP harmonics for the 3D paradigm observed in the frequency-domain analysis. The 3D-Blink paradigm achieved 75.00% of detection accuracy and 27.02 bits/min of ITR with 0.8 seconds of stimulus time and task-related component analysis (TRCA) algorithm, demonstrating its effectiveness. These findings demonstrate that the 3D and 3D-Blink paradigms supported by VR can achieve improved user comfort and satisfactory performance, while further algorithmic optimization and feature analysis are required for the stereo-related paradigms. In conclusion, this study contributes to a deeper understanding of the impact of binocular stereoscopic vision mechanisms on SSVEP paradigms and promotes the application of SSVEP-BCI in diverse VR environments.

Signal processing algorithms for SSVEP-based brain computer interface: State-of-the-art and recent developments

Over past two decades, steady-state evoked potentials (SSVEP)-based brain computer interface (BCI) systems have been extensively developed. As we all know, signal processing algorithms play an important role in this BCI. However, there is no comprehensive review of the latest development of signal processing algorithms for SSVEP-based BCI. By analyzing the papers published in authoritative journals in nearly five years, signal processing algorithms of preprocessing, feature extraction and classification modules are discussed in detail. In addition, other aspects existed in this BCI are mentioned. The following key problems are solved. (1) In recent years, which signal processing algorithms are frequently used in each module? (2) Which signal processing algorithms attract more attention in recent years? (3) Which modules are the key to signal processing in BCI field? This information is very important for choosing the appropriate algorithms, and can also be considered as a reference for further research. Simultaneously, we hope that this work can provide relevant BCI researchers with valuable information about the latest trends of signal processing algorithms for SSVEP-based BCI systems.

A Maximum Likelihood Perspective of Spatial Filter Design in SSVEP-Based BCIs

In steady-state visual-evoked potential (SSVEP) based brain-computer interfaces (BCIs), existing detection algorithms utilizing spatial filters like task-related component analysis (TRCA) derive the spatial filters mainly through maximizing the inter-trial similarity between the combined signals over the training set. Although they achieve by far the best classification performance in SSVEP-based BCIs, some important problems are still unresolved. Especially, the mechanism of how spatial filters cancel the background noise in brain signals and optimize the signal-to-noise ratio (SNR) of SSVEPs is still not figured out. Therefore, to solve these problems, in this paper a new perspective of spatial filter design is proposed. Specifically, a linear generative signal model of SSVEP is adopted and the spatial filters are obtained automatically through maximum likelihood estimation of source signals and channel vectors. In the same time, the relation between maximum likelihood estimation and signal-to-noise ratio (SNR) maximization is discussed. Through a step-by-step formulation, this paper provides a theoretical justification for those conventional algorithms utilizing spatial filters. As for the classification performance, the proposed scheme is tested on a benchmark dataset of 35 subjects. Experiment results show that the classification performance of the proposed scheme is competitive against three benchmark algorithms, which include TRCA. Especially, the proposed scheme achieves a fair performance improvement over the benchmark methods in the cases where a shorter time window, or a larger number of electrodes, or a smaller number of training blocks are adopted.

Single stimulus location for two inputs: A combined brain-computer interface based on Steady-State Visual Evoked Potential (SSVEP)

Brain-computer interfaces (BCI) help severely paralyzed people communicate with the outside world. One type of BCI depends on eye movements and has high information transfer (ITR) but is tiring for users and not applicable to people with eye dyskinesia. Conversely, independent BCIs enable attention shifts across visual stimuli without eye movement, but at the cost of a lower ITR. Steady-state visual evoked potential (SSVEP) is an oscillatory brain response and typically used as BCI signal sources because of high signal-to-noise ratio (SNR). Considering the effect of attentional modulation on the SSVEP, we proposed the novel concept of one-to-two BCI to optimize existing problems, wherein the target and other stimuli shared the same location. Specifically, two spatially overlapping stimuli were displayed in the center-of-view field, as in the independent BCI, and participants were required to divide their attention between the right and left visual fields, as in the dependent BCI. Using three different design schemes in two experiments, we aimed to provide a new framework for BCI design by exploring the feasibility of a combined BCI that can realize a single stimulus location for two inputs. The results strongly demonstrated that, even when the targets and distractors overlapped spatially, the former evoked stronger SSVEP responses. Notably, the BCI scheme based on the object-based attention could achieve a recognition rate as high as 83.2% and an ITR of 12.5 bits per minute. The feasibility of a one-to-two BCI design, which simplified the keyboard layout, reduced the attention shift, and relieved user fatigue, was established.

Decoding overt shifts of attention in depth through pupillary and cortical frequency tagging

OBJECTIVE

We have recently developed a prototype of a novel human-computer interface for assistive communication based on voluntary shifts of attention (gaze) from a far target to a near target associated with a decrease of pupil size (Pupillary Accommodative Response, PAR), an automatic vegetative response that can be easily recorded. We report here an extension of that approach based on pupillary and cortical frequency tagging.

APPROACH

In 18 healthy volunteers, we investigated the possibility of decoding attention shifts in depth by exploiting the evoked oscillatory responses of the pupil (Pupillary Oscillatory Response, POR, recorded through a low-cost device) and visual cortex (Steady-State Visual Evoked Potentials, SSVEP, recorded from 4 scalp electrodes). With a simple binary communication protocol (focusing on a far target meaning "No", focusing on the near target meaning "Yes"), we aimed at discriminating when observer's overt attention (gaze) shifted from the far to the near target, which were flickering at different frequencies.

MAIN RESULTS

By applying a binary linear classifier (Support Vector Machine, SVM, with leave-one-out cross validation) to POR and SSVEP signals, we found that, with only twenty trials and no subjects' behavioural training, the offline median decoding accuracy was 75% and 80% with POR and SSVEP signals, respectively. When the two signals were combined together, accuracy reached 83%. The number of observers for whom accuracy was higher than 70% was 11/18, 12/18 and 14/18 with POR, SVVEP and combined features, respectively. A signal detection analysis confirmed these results.

SIGNIFICANCE

The present findings suggest that exploiting frequency tagging with pupillary or cortical responses during an attention shift in the depth plane, either separately or combined together, is a promising approach to realize a device for communicating with Complete Locked-In Syndrome (CLIS) patients when oculomotor control is unreliable and traditional assistive communication, even based on PAR, is unsuccessful.

Assessment of high-frequency steady-state visual evoked potentials from below-the-hairline areas for a brain-computer interface based on Depth-of-Field

BACKGROUND AND OBJECTIVE

Recently, a promising Brain-Computer Interface based on Steady-State Visual Evoked Potential (SSVEP-BCI) was proposed, which composed of two stimuli presented together in the center of the subject's field of view, but at different depth planes (Depth-of-Field setup). Thus, users were easily able to select one of them by shifting their eye focus. However, in that work, EEG signals were collected through electrodes placed on occipital and parietal regions (hair-covered areas), which demanded a long preparation time. Also, that work used low-frequency stimuli, which can produce visual fatigue and increase the risk of photosensitive epileptic seizures. In order to improve the practicality and visual comfort, this work proposes a BCI based on Depth-of-Field using the high-frequency SSVEP response measured from below-the-hairline areas (behind-the-ears).

METHODS

Two high-frequency stimuli (31 Hz and 32 Hz) were used in a Depth-of-Field setup to study the SSVEP response from behind-the-ears (TP9 and TP10). Multivariate Spectral F-test (MSFT) method was used to verify the elicited response. Afterwards, a BCI was proposed to command a mobile robot in a virtual reality environment. The commands were recognized through Temporally Local Multivariate Synchronization Index (TMSI) method.

RESULTS

The data analysis reveal that the focused stimuli elicit distinguishable SSVEP response when measured from hairless areas, in spite of the fact that the non-focused stimulus is also present in the field of view. Also, our BCI shows a satisfactory result, reaching average accuracy of 91.6% and Information Transfer Rate (ITR) of 5.3 bits/min.

CONCLUSION

These findings contribute to the development of more safe and practical BCI.

Adaptive Spatial Filter Based on Similarity Indices to Preserve the Neural Information on EEG Signals during On-Line Processing

This work presents a new on-line adaptive filter, which is based on a similarity analysis between standard electrode locations, in order to reduce artifacts and common interferences throughout electroencephalography (EEG) signals, but preserving the useful information. Standard deviation and Concordance Correlation Coefficient (CCC) between target electrodes and its correspondent neighbor electrodes are analyzed on sliding windows to select those neighbors that are highly correlated. Afterwards, a model based on CCC is applied to provide higher values of weight to those correlated electrodes with lower similarity to the target electrode. The approach was applied to brain computer-interfaces (BCIs) based on Canonical Correlation Analysis (CCA) to recognize 40 targets of steady-state visual evoked potential (SSVEP), providing an accuracy (ACC) of 86.44 ± 2.81%. In addition, also using this approach, features of low frequency were selected in the pre-processing stage of another BCI to recognize gait planning. In this case, the recognition was significantly (p<0.01) improved for most of the subjects (ACC≥74.79%), when compared with other BCIs based on Common Spatial Pattern, Filter Bank-Common Spatial Pattern, and Riemannian Geometry.