Optimizing spatial properties of a new checkerboard-like visual stimulus for user-friendly SSVEP-based BCIs

Optimizing Visual Stimulation Paradigms for User-Friendly SSVEP-Based BCIs

In steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) systems, traditional flickering stimulation patterns face challenges in achieving a trade-off in both BCI performance and visual comfort across various frequency bands. To investigate the optimal stimulation paradigms with high performance and high comfort for each frequency band, this study systematically compared the characteristics of SSVEP and user experience of different stimulation paradigms with a wide stimulation frequency range of 1-60 Hz. The findings suggest that, for a better balance between system performance and user experience, ON and OFF grid stimuli with a Weber contrast of 50% can be utilized as alternatives to traditional flickering stimulation paradigms in the frequency band of 1-25 Hz. In the 25-35 Hz range, uniform flicker stimuli with the same 50% contrast are more suitable. In the higher frequency band, traditional uniform flicker stimuli with a high 300% contrast are preferred. These results are significant for developing high performance and user-friendly SSVEP-based BCI systems.

A data expansion technique based on training and testing sample to boost the detection of SSVEPs for brain-computer interfaces

Objective.Currently, steady-state visual evoked potentials (SSVEPs)-based brain-computer interfaces (BCIs) have achieved the highest interaction accuracy and speed among all BCI paradigms. However, its decoding efficacy depends deeply on the number of training samples, and the system performance would have a dramatic drop when the training dataset decreased to a small size. To date, no study has been reported to incorporate the unsupervised learning information from testing trails into the construction of supervised classification model, which is a potential way to mitigate the overfitting effect of limited samples.Approach.This study proposed a novel method for SSVEPs detection, i.e. cyclic shift trials (CSTs), which could combine unsupervised learning information from test trials and supervised learning information from train trials. Furthermore, since SSVEPs are time-locked and phase-locked to the onset of specific flashes, CST could also expand training samples on the basis of its regularity and periodicity. In order to verify the effectiveness of CST, we designed an online SSVEP-BCI system, and tested this system combined CST with two common classification algorithms, i.e. extended canonical correlation analysis and ensemble task-related component analysis.Main results.CST could significantly enhance the signal to noise ratios of SSVEPs and improve the performance of systems especially for the condition of few training samples and short stimulus time. The online information transfer rate could reach up to 236.19 bits min-1using 36 s calibration time of only one training sample for each category.Significance.The proposed CST method can take full advantages of supervised learning information from training samples and unsupervised learning information of testing samples. Furthermore, it is a data expansion technique, which can enhance the SSVEP characteristics and reduce dependence on sample size. Above all, CST is a promising method to improve the performance of SSVEP-based BCI without any additional experimental burden.

Influence of spatial frequency in visual stimuli for cVEP-based BCIs: evaluation of performance and user experience

Code-modulated visual evoked potentials (c-VEPs) are an innovative control signal utilized in brain-computer interfaces (BCIs) with promising performance. Prior studies on steady-state visual evoked potentials (SSVEPs) have indicated that the spatial frequency of checkerboard-like stimuli influences both performance and user experience. Spatial frequency refers to the dimensions of the individual squares comprising the visual stimulus, quantified in cycles (i.e., number of black-white squares pairs) per degree of visual angle. However, the specific effects of this parameter on c-VEP-based BCIs remain unexplored. Therefore, the objective of this study is to investigate the role of spatial frequency of checkerboard-like visual stimuli in a c-VEP-based BCI. Sixteen participants evaluated selection matrices with eight spatial frequencies: C001 (0 c/°, 1×1 squares), C002 (0.15 c/°, 2×2 squares), C004 (0.3 c/°, 4×4 squares), C008 (0.6 c/°, 8×8 squares), C016 (1.2 c/°, 16×16 squares), C032 (2.4 c/°, 32×32 squares), C064 (4.79 c/°, 64×64 squares), and C128 (9.58 c/°, 128×128 squares). These conditions were tested in an online spelling task, which consisted of 18 trials each conducted on a 3×3 command interface. In addition to accuracy and information transfer rate (ITR), subjective measures regarding comfort, ocular irritation, and satisfaction were collected. Significant differences in performance and comfort were observed based on different stimulus spatial frequencies. Although all conditions achieved mean accuracy over 95% after 2.1 s of trial duration, C016 stood out in terms user experience. The proposed condition not only achieved a mean accuracy of 96.53% and 164.54 bits/min with a trial duration of 1.05s, but also was reported to be significantly more comfortable than the traditional C001 stimulus. Since both features are key for BCI development, higher spatial frequencies than the classical black-to-white stimulus might be more adequate for c-VEP systems. Hence, we assert that the spatial frequency should be carefully considered in the development of future applications for c-VEP-based BCIs.

A Calibration-Free Hybrid BCI Speller System Based on High-Frequency SSVEP and sEMG

Hybrid brain-computer interface (hBCI) systems that combine steady-state visual evoked potential (SSVEP) and surface electromyography (sEMG) signals have attracted attention of researchers due to the advantage of exhibiting significantly improved system performance. However, almost all existing studies adopt low-frequency SSVEP to build hBCI. It produces much more visual fatigue than high-frequency SSVEP. Therefore, the current study attempts to build a hBCI based on high-frequency SSVEP and sEMG. With these two signals, this study designed and realized a 32-target hBCI speller system. Thirty-two targets were separated from the middle into two groups. Each side contained 16 sets of targets with different high-frequency visual stimuli (i.e., 31-34.75 Hz with an interval of 0.25 Hz). sEMG was utilized to choose the group and SSVEP was adopted to identify intra-group targets. The filter bank canonical correlation analysis (FBCCA) and the root mean square value (RMS) methods were used to identify signals. Therefore, the proposed system allowed users to operate it without system calibration. A total of 12 healthy subjects participated in online experiment, with an average accuracy of 93.52 ± 1.66% and the average information transfer rate (ITR) reached 93.50 ± 3.10 bits/min. Furthermore, 12 participants perfectly completed the free-spelling tasks. These results of the experiments indicated feasibility and practicality of the proposed hybrid BCI speller system.

A high-frequency SSVEP-BCI system based on a 360 Hz refresh rate

Objective. Steady-state visual evoked potential (SSVEP) based brain-computer interfaces (BCIs) often struggle to balance user experience and system performance. To address this challenge, this study employed stimuli in the 55-62.8 Hz frequency range to implement a 40-target BCI speller that offered both high-performance and user-friendliness.Approach. This study proposed a method that presents stable multi-target stimuli on a monitor with a 360 Hz refresh rate. Real-time generation of stimulus matrix and stimulus rendering was used to ensure stable presentation while reducing the computational load. The 40 targets were encoded using the joint frequency and phase modulation method, offline and online BCI experiments were conducted on 16 subjects using the task discriminant component analysis algorithm for feature extraction and classification.Main results. The online BCI system achieved an average accuracy of 88.87% ± 3.05% and an information transfer rate of 51.83 ± 2.77 bits min-1under the low flickering perception condition.Significance. These findings suggest the feasibility and significant practical value of the proposed high-frequency SSVEP BCI system in advancing the visual BCI technology.

A new grid stimulus with subtle flicker perception for user-friendly SSVEP-based BCIs

Objective.The traditional uniform flickering stimulation pattern shows strong steady-state visual evoked potential (SSVEP) responses and poor user experience with intense flicker perception. To achieve a balance between performance and comfort in SSVEP-based brain-computer interface (BCI) systems, this study proposed a new grid stimulation pattern with reduced stimulation area and low spatial contrast.Approach.A spatial contrast scanning experiment was conducted first to clarify the relationship between the SSVEP characteristics and the signs and values of spatial contrast. Four stimulation patterns were involved in the experiment: the ON and OFF grid stimulation patterns that separately activated the positive or negative contrast information processing pathways, the ON-OFF grid stimulation pattern that simultaneously activated both pathways, and the uniform flickering stimulation pattern that served as a control group. The contrast-intensity and contrast-user experience curves were obtained for each stimulation pattern. Accordingly, the optimized stimulation schemes with low spatial contrast (the ON-50% grid stimulus, the OFF-50% grid stimulus, and the Flicker-30% stimulus) were applied in a 12-target and a 40-target BCI speller and compared with the traditional uniform flickering stimulus (the Flicker-500% stimulus) in the evaluation of BCI performance and subjective experience.Main results.The OFF-50% grid stimulus showed comparable online performance (12-target, 2 s: 69.87 ± 0.74 vs. 69.76 ± 0.58 bits min^-1, 40-target, 4 s: 57.02 ± 2.53 vs. 60.79 ± 1.08 bits min^-1) and improved user experience (better comfortable level, weaker flicker perception and higher preference level) compared to the traditional Flicker-500% stimulus in both multi-targets BCI spellers.Significance.Selective activation of the negative contrast information processing pathway using the new OFF-50% grid stimulus evoked robust SSVEP responses. On this basis, high-performance and user-friendly SSVEP-based BCIs have been developed and implemented, which has important theoretical significance and application value in promoting the development of the visual BCI technology.

A high-performance SSVEP-based BCI using imperceptible flickers

Objective.Existing steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) struggle to balance user experience and system performance. This study proposed an individualized space and phase modulation method to code imperceptible flickers at 60 Hz towards a user-friendly SSVEP-based BCI with high performance.Approach.The individualized customization of visual stimulation took the subject-to-subject variability in cortex geometry into account. An annulus global-stimulation was divided into local-stimulations of eight annular sectors and presented to subjects separately. The local-stimulation SSVEPs were superimposed to simulate global-stimulation SSVEPs with 4⁷space and phase coding combinations. A four-class phase-coded BCI diagram was used to evaluate the simulated classification performance. The performance ranking of all simulated global-stimulation SSVEPs were obtained and three performance levels (optimal, medium, worst) of individualized modulation groups were searched for each subject. The standard-modulation group conforming to the V1 'cruciform' geometry and the non-modulation group were involved as controls. A four-target phase-coded BCI system with SSVEPs at 60 Hz was implemented with the five modulation groups and questionnaires were used to evaluate user experience.Main results.The proposed individualized space and phase modulation method effectively modulated the SSVEP intensity without affecting the user experience. The online BCI system using the 60 Hz stimuli achieved mean information transfer rates of 52.8 ± 1.9 bits min^-1, 16.8 ± 2.4 bits min^-1, and 42.4 ± 3.0 bits min^-1with individualized optimal-modulation, individualized worst-modulation, and non-modulation groups, respectively.Significance.Structural and functional characteristics of the human visual cortex were exploited to enhance the response intensity of SSVEPs at 60 Hz, resulting in a high-performance BCI system with good user experience. This study has important theoretical significance and application value for promoting the development of the visual BCI technology.

Does Real-Time Feedback Improve User Performance in SSVEP-based Brain-Computer Interfaces?

Offline and online experiments are both widely used in SSVEP-based BCI research and development for different purposes. One of the major differences between offline and online experiments is the existence of real-time feedback to the user while they are using the interface. However, the role of feedback in SSVEP-based BCIs has not yet been well studied. This work focuses on understanding the effect of feedback in SSVEP-based BCIs and if there exists any relationship between offline and online BCI performance. An experiment was designed to compare directly the accuracies of the BCI with and without feedback for participants. Results showed that feedback can improve performance in a complex task, but no clear improvement was observed in a simple task.

A Hybrid Brain-Computer Interface Based on Visual Evoked Potential and Pupillary Response

Brain-computer interface (BCI) based on steady-state visual evoked potential (SSVEP) has been widely studied due to the high information transfer rate (ITR), little user training, and wide subject applicability. However, there are also disadvantages such as visual discomfort and “BCI illiteracy.” To address these problems, this study proposes to use low-frequency stimulations (12 classes, 0.8–2.12 Hz with an interval of 0.12 Hz), which can simultaneously elicit visual evoked potential (VEP) and pupillary response (PR) to construct a hybrid BCI (h-BCI) system. Classification accuracy was calculated using supervised and unsupervised methods, respectively, and the hybrid accuracy was obtained using a decision fusion method to combine the information of VEP and PR. Online experimental results from 10 subjects showed that the averaged accuracy was 94.90 ± 2.34% (data length 1.5 s) for the supervised method and 91.88 ± 3.68% (data length 4 s) for the unsupervised method, which correspond to the ITR of 64.35 ± 3.07 bits/min (bpm) and 33.19 ± 2.38 bpm, respectively. Notably, the hybrid method achieved higher accuracy and ITR than that of VEP and PR for most subjects, especially for the short data length. Together with the subjects’ feedback on user experience, these results indicate that the proposed h-BCI with the low-frequency stimulation paradigm is more comfortable and favorable than the traditional SSVEP-BCI paradigm using the alpha frequency range.

A high-performance brain switch based on code-modulated visual evoked potentials

OBJECTIVE

Asynchronous brain-computer interfaces (BCIs) are more practical and natural compared to synchronous BCIs. A brain switch is a standard asynchronous BCI, which can automatically detect the specified change of the brain and discriminate between the control state and the idle state. The current brain switches still face challenges on relatively long reaction time (RT) and high false positive rate (FPR).

APPROACH

In this paper, an online electroencephalography-based brain switch is designed to realize a fast reaction and keep long idle time (IDLE) without false positives (FPs) using code-modulated visual evoked potentials (c-VEPs). Two stimulation paradigms were designed and compared in the experiments: multi-code concatenate modulation (concatenation mode) and single-code periodic modulation (periodic mode). Using a task-related component analysis-based detection algorithm, EEG data can be decoded into a series of code indices. Brain states can be detected by a template matching approach with a sliding window on the output series.

MAIN RESULTS

The online experiments achieved an average RT of 1.49 seconds when the average IDLE for each FP was 68.57 minutes (1.46e-2 FP/min) or an average RT of 1.67 seconds without FPs.

SIGNIFICANCE

This study provides a practical c-VEP based brain switch system with both fast reaction and low FPR during idle state, which can be used in various BCI applications.