Tradeoff between User Experience and BCI Classification Accuracy with Frequency Modulated Steady-State Visual Evoked Potentials

An open dataset for human SSVEPs in the frequency range of 1-60 Hz

A steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) system relies on the photic driving response to effectively elicit characteristic electroencephalogram (EEG) signals. However, traditional visual stimuli mainly adopt high-contrast black-and-white flickering stimulations, which are easy to cause visual fatigue. This paper presents an SSVEP dataset acquired at a wide frequency range from 1 to 60 Hz with an interval of 1 Hz using flickering stimuli under two different modulation depths. This dataset contains 64-channel EEG data from 30 healthy subjects when they fixated on a single flickering stimulus. The stimulus was rendered on an LCD display with a refresh rate of 240 Hz. Initially, the dataset was rigorously validated through comprehensive data analysis to investigate SSVEP responses and user experiences. Subsequently, BCI performance was evaluated through offline simulations of frequency-coded and phase-coded BCI paradigms. This dataset provides comprehensive and high-quality data for studying and developing SSVEP-based BCI systems.

A High-Resolution LED Stimulator for Steady-State Visual Stimulation: Customizable, Affordable, and Open Source

Visually evoked steady-state potentials (SSVEPs) are neural responses elicited by visual stimuli oscillating at specific frequencies. In this study, we introduce a novel LED stimulator system explicitly designed for steady-state visual stimulation, offering precise control over visual stimulus parameters, including frequency resolution, luminance, and the ability to control the phase at the end of the stimulation. The LED stimulator provides a personalized, modular, and affordable option for experimental setups. Based on the Teensy 3.2 board, the stimulator utilizes direct digital synthesis and pulse width modulation techniques to control the LEDs. We validated its performance through four experiments: the first two measured LED light intensities directly, while the last two assessed the stimulator's impact on EEG recordings. The results demonstrate that the stimulator can deliver a stimulus suitable for generating SSVEPs with the desired frequency and phase resolution. As an open source resource, we provide comprehensive documentation, including all necessary codes and electrical diagrams, which facilitates the system's replication and adaptation for specific experimental requirements, enhancing its potential for widespread use in the field of neuroscience setups.

Modulation of rhythmic visual stimulation on left-right attentional asymmetry

The rhythmic visual stimulation (RVS)-induced oscillatory brain responses, namely steady-state visual evoked potentials (SSVEPs), have been widely used as a biomarker in studies of neural processing based on the assumption that they would not affect cognition. However, recent studies have suggested that the generation of SSVEPs might be attributed to neural entrainment and thus could impact brain functions. But their neural and behavioral effects are yet to be explored. No study has reported the SSVEP influence on functional cerebral asymmetry (FCA). We propose a novel lateralized visual discrimination paradigm to test the SSVEP effects on visuospatial selective attention by FCA analyses. Thirty-eight participants covertly shifted their attention to a target triangle appearing in either the lower-left or -right visual field (LVF or RVF), and judged its orientation. Meanwhile, participants were exposed to a series of task-independent RVSs at different frequencies, including 0 (no RVS), 10, 15, and 40-Hz. As a result, it showed that target discrimination accuracy and reaction time (RT) varied significantly across RVS frequency. Furthermore, attentional asymmetries differed for the 40-Hz condition relative to the 10-Hz condition as indexed by enhanced RT bias to the right visual field, and larger Pd EEG component for attentional suppression. Our results demonstrated that RVSs had frequency-specific effects on left-right attentional asymmetries in both behavior and neural activities. These findings provided new insights into the functional role of SSVEP on FCAs.

Inter-individual single-trial classification of MEG data using M-CCA

Neuroscientific studies often involve some form of group analysis over multiple participants. This requires alignment of recordings across participants. A naive solution is to assume that participants' recordings can be aligned anatomically in sensor space. However, this assumption is likely violated due to anatomical and functional differences between individual brains. In magnetoencephalography (MEG) recordings the problem of inter-subject alignment is exacerbated by the susceptibility of MEG to individual cortical folding patterns as well as the inter-subject variability of sensor locations over the brain due to the use of a fixed helmet. Hence, an approach to combine MEG data over individual brains should relax the assumptions that a) brain anatomy and function are tightly linked and b) that the same sensors capture functionally comparable brain activation across individuals. Here we use multiset canonical correlation analysis (M-CCA) to find a common representation of MEG activations recorded from 15 participants performing a grasping task. The M-CCA algorithm was applied to transform the data of a set of multiple participants into a common space with maximum correlation between participants. Importantly, we derive a method to transform data from a new, previously unseen participant into this common representation. This makes it useful for applications that require transfer of models derived from a group of individuals to new individuals. We demonstrate the usefulness and superiority of the approach with respect to previously used approaches. Finally, we show that our approach requires only a small number of labeled data from the new participant. The proposed method demonstrates that functionally motivated common spaces have potential applications in reducing training time of online brain-computer interfaces, where models can be pre-trained on previous participants/sessions. Moreover, inter-subject alignment via M-CCA has the potential for combining data of different participants and could become helpful in future endeavors on large open datasets.

The Influence of the Modulation Index on Frequency-Modulated Steady-State Visual Evoked Potentials

Based on increased user experience during stimulation, frequency-modulated steady-state visual evoked potentials (FM-SSVEPs) have been suggested as an improved stimulation method for brain-computer interfaces. Adapting such a novel stimulation paradigm requires in-depth analyses of all different stimulation parameters and their influence on brain responses as well as the user experience during the stimulation. In the current manuscript, we assess the influence of different values for the modulation index, which determine the spectral distribution in the stimulation signal on FM-SSVEPs. We visually stimulated 14 participants at different target frequencies with four different values for the modulation index. Our results reveal that changing the modulation index in a way that elevates the stimulation power in the targeted sideband leads to increased FM-SSVEP responses. There is, however, a tradeoff with user experience as increased modulation indices also lead to increased perceived flicker intensity as well as decreased stimulation comfort in our participants. Our results can guide the choice of parameters in future FM-SSVEP implementations.

A Hybrid Brain-Computer Interface Based on Visual Evoked Potential and Pupillary Response

Brain-computer interface (BCI) based on steady-state visual evoked potential (SSVEP) has been widely studied due to the high information transfer rate (ITR), little user training, and wide subject applicability. However, there are also disadvantages such as visual discomfort and “BCI illiteracy.” To address these problems, this study proposes to use low-frequency stimulations (12 classes, 0.8–2.12 Hz with an interval of 0.12 Hz), which can simultaneously elicit visual evoked potential (VEP) and pupillary response (PR) to construct a hybrid BCI (h-BCI) system. Classification accuracy was calculated using supervised and unsupervised methods, respectively, and the hybrid accuracy was obtained using a decision fusion method to combine the information of VEP and PR. Online experimental results from 10 subjects showed that the averaged accuracy was 94.90 ± 2.34% (data length 1.5 s) for the supervised method and 91.88 ± 3.68% (data length 4 s) for the unsupervised method, which correspond to the ITR of 64.35 ± 3.07 bits/min (bpm) and 33.19 ± 2.38 bpm, respectively. Notably, the hybrid method achieved higher accuracy and ITR than that of VEP and PR for most subjects, especially for the short data length. Together with the subjects’ feedback on user experience, these results indicate that the proposed h-BCI with the low-frequency stimulation paradigm is more comfortable and favorable than the traditional SSVEP-BCI paradigm using the alpha frequency range.

Artificial Intelligence Algorithms in Visual Evoked Potential-Based Brain-Computer Interfaces for Motor Rehabilitation Applications: Systematic Review and Future Directions

Brain-Computer Interface (BCI) is a technology that uses electroencephalographic (EEG) signals to control external devices, such as Functional Electrical Stimulation (FES). Visual BCI paradigms based on P300 and Steady State Visually Evoked potentials (SSVEP) have shown high potential for clinical purposes. Numerous studies have been published on P300- and SSVEP-based non-invasive BCIs, but many of them present two shortcomings: (1) they are not aimed for motor rehabilitation applications, and (2) they do not report in detail the artificial intelligence (AI) methods used for classification, or their performance metrics. To address this gap, in this paper the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology was applied to prepare a systematic literature review (SLR). Papers older than 10 years, repeated or not related to a motor rehabilitation application, were excluded. Of all the studies, 51.02% referred to theoretical analysis of classification algorithms. Of the remaining, 28.48% were for spelling, 12.73% for diverse applications (control of wheelchair or home appliances), and only 7.77% were focused on motor rehabilitation. After the inclusion and exclusion criteria were applied and quality screening was performed, 34 articles were selected. Of them, 26.47% used the P300 and 55.8% the SSVEP signal. Five applications categories were established: Rehabilitation Systems (17.64%), Virtual Reality environments (23.52%), FES (17.64%), Orthosis (29.41%), and Prosthesis (11.76%). Of all the works, only four performed tests with patients. The most reported machine learning (ML) algorithms used for classification were linear discriminant analysis (LDA) (48.64%) and support vector machine (16.21%), while only one study used a deep learning algorithm: a Convolutional Neural Network (CNN). The reported accuracy ranged from 38.02 to 100%, and the Information Transfer Rate from 1.55 to 49.25 bits per minute. While LDA is still the most used AI algorithm, CNN has shown promising results, but due to their high technical implementation requirements, many researchers do not justify its implementation as worthwile. To achieve quick and accurate online BCIs for motor rehabilitation applications, future works on SSVEP-, P300-based and hybrid BCIs should focus on optimizing the visual stimulation module and the training stage of ML and DL algorithms.

Brain Oscillation Entrainment by Perceptible and Non-perceptible Rhythmic Light Stimulation

Objective and Background: Decades of research in the field of steady-state visual evoked potentials (SSVEPs) have revealed great potential of rhythmic light stimulation for brain-computer interfaces. Additionally, rhythmic light stimulation provides a non-invasive method for entrainment of oscillatory activity in the brain. Especially effective protocols enabling non-perceptible rhythmic stimulation and, thereby, reducing eye fatigue and user discomfort are favorable. Here, we investigate effects of (1) perceptible and (2) non-perceptible rhythmic light stimulation as well as attention-based effects of the stimulation by asking participants to focus (a) on the stimulation source directly in an overt attention condition or (b) on a cross-hair below the stimulation source in a covert attention condition. Method: SSVEPs at 10 Hz were evoked with a light-emitting diode (LED) driven by frequency-modulated signals and amplitudes of the current intensity either below or above a previously estimated individual threshold. Furthermore, we explored the effect of attention by asking participants to fixate on the LED directly in the overt attention condition and indirectly attend it in the covert attention condition. By measuring electroencephalography, we analyzed differences between conditions regarding the detection of reliable SSVEPs via the signal-to-noise ratio (SNR) and functional connectivity in occipito-frontal(-central) regions. Results: We could observe SSVEPs at 10 Hz for the perceptible and non-perceptible rhythmic light stimulation not only in the overt but also in the covert attention condition. The SNR and SSVEP amplitudes did not differ between the conditions and SNR values were in all except one participant above significance thresholds suggested by previous literature indicating reliable SSVEP responses. No difference between the conditions could be observed in the functional connectivity in occipito-frontal(-central) regions. Conclusion: The finding of robust SSVEPs even for non-intrusive rhythmic stimulation protocols below an individual perceptibility threshold and without direct fixation on the stimulation source reveals strong potential as a safe stimulation method for oscillatory entrainment in naturalistic applications.

The Validity of Steady-State Visual Evoked Potentials as Attention Tags and Input Signals: A Critical Perspective of Frequency Allocation and Number of Stimuli

Steady-state visual evoked potential (SSVEP) is a periodic response to a repetitive visual stimulus at a specific frequency. Currently, SSVEP is widely treated as an attention tag in cognitive activities and is used as an input signal for brain-computer interfaces (BCIs). However, whether SSVEP can be used as a reliable indicator has been a controversial issue. We focused on the independence of SSVEP from frequency allocation and number of stimuli. First, a cue-target paradigm was adopted to examine the interaction between SSVEPs evoked by two stimuli with different frequency allocations under different attention conditions. Second, we explored whether signal strength and the performance of SSVEP-based BCIs were affected by the number of stimuli. The results revealed that no significant interaction of SSVEP responses appeared between attended and unattended stimuli under various frequency allocations, regardless of their appearance in the fundamental or second-order harmonic. The amplitude of SSVEP suffered no significant gain or loss under different numbers of stimuli, but the performance of SSVEP-based BCIs varied along with duration of stimuli; that is, the recognition rate was not affected by the number of stimuli when the duration of stimuli was long enough, while the information transfer rate (ITR) presented the opposite trend. It can be concluded that SSVEP is a reliable tool for marking and monitoring multiple stimuli simultaneously in cognitive studies, but much caution should be taken when choosing a suitable duration and the number of stimuli, in order to achieve optimal utility of BCIs in the future.

Exploration of User's Mental State Changes during Performing Brain-Computer Interface

Substantial developments have been established in the past few years for enhancing the performance of brain–computer interface (BCI) based on steady-state visual evoked potential (SSVEP). The past SSVEP-BCI studies utilized different target frequencies with flashing stimuli in many different applications. However, it is not easy to recognize user’s mental state changes when performing the SSVEP-BCI task. What we could observe was the increasing EEG power of the target frequency from the user’s visual area. BCI user’s cognitive state changes, especially in mental focus state or lost-in-thought state, will affect the BCI performance in sustained usage of SSVEP. Therefore, how to differentiate BCI users’ physiological state through exploring their neural activities changes while performing SSVEP is a key technology for enhancing the BCI performance. In this study, we designed a new BCI experiment which combined working memory task into the flashing targets of SSVEP task using 12 Hz or 30 Hz frequencies. Through exploring the EEG activity changes corresponding to the working memory and SSVEP task performance, we can recognize if the user’s cognitive state is in mental focus or lost-in-thought. Experiment results show that the delta (1–4 Hz), theta (4–7 Hz), and beta (13–30 Hz) EEG activities increased more in mental focus than in lost-in-thought state at the frontal lobe. In addition, the powers of the delta (1–4 Hz), alpha (8–12 Hz), and beta (13–30 Hz) bands increased more in mental focus in comparison with the lost-in-thought state at the occipital lobe. In addition, the average classification performance across subjects for the KNN and the Bayesian network classifiers were observed as 77% to 80%. These results show how mental state changes affect the performance of BCI users. In this work, we developed a new scenario to recognize the user’s cognitive state during performing BCI tasks. These findings can be used as the novel neural markers in future BCI developments.

Comparison of Visual Stimuli for Steady-State Visual Evoked Potential-Based Brain-Computer Interfaces in Virtual Reality Environment in terms of Classification Accuracy and Visual Comfort

Recent studies on brain-computer interfaces (BCIs) based on the steady-state visual evoked potential (SSVEP) have demonstrated their use to control objects or generate commands in virtual reality (VR) environments. However, most SSVEP-based BCI studies performed in VR environments have adopted visual stimuli that are typically used in conventional LCD environments without considering the differences in the rendering devices (head-mounted displays (HMDs) used in the VR environments). The proximity between the visual stimuli and the eyes in HMDs can readily cause eyestrain, degrading the overall performance of SSVEP-based BCIs. Therefore, in the present study, we have tested two different types of visual stimuli—pattern-reversal checkerboard stimulus (PRCS) and grow/shrink stimulus (GSS)—on young healthy participants wearing HMDs. Preliminary experiments were conducted to investigate the visual comfort of each participant during the presentation of the visual stimuli. In subsequent online avatar control experiments, we observed considerable differences in the classification accuracy of individual participants based on the type of visual stimuli used to elicit SSVEP. Interestingly, there was a close relationship between the subjective visual comfort score and the online performance of the SSVEP-based BCI: most participants showed better classification accuracy under visual stimulus they were more comfortable with. Our experimental results suggest the importance of an appropriate visual stimulus to enhance the overall performance of the SSVEP-based BCIs in VR environments. In addition, it is expected that the appropriate visual stimulus for a certain user might be readily selected by surveying the user's visual comfort for different visual stimuli, without the need for the actual BCI experiments.

WaveCSP: a robust motor imagery classifier for consumer EEG devices

There is an increasing demand for reliable motor imagery (MI) classification algorithms for applications in consumer level brain-computer interfacing (BCI). For the practical use, such algorithms must be robust to both device limitations and subject variability, which make MI classification a challenging task. This study proposes methods to study the effect of limitations including a limited number of electrodes, limited spatial distribution of electrodes, lower signal quality, subject variabilities and BCI literacy, on the performance of MI classification. To mitigate these limitations, we propose a machine learning approach, WaveCSP that uses 24 features extracted from EEG signals using wavelet transform and common spatial pattern (CSP) filtering techniques. The algorithm shows better performance in terms of subject variability compared to existing work. The application of WaveCSP to Physionet MI database shows more than 50% of the 109 subjects achieving accuracy higher than 64%. The data obtained from a commercial EEG headset using the same experimental protocol result in up to four out of five subjects who had prior BCI experience (out of a total of 25 subjects) performing with accuracy higher than 64%.

Use of Sine Shaped High-Frequency Rhythmic Visual Stimuli Patterns for SSVEP Response Analysis and Fatigue Rate Evaluation in Normal Subjects

Background: Recent EEG-SSVEP signal based BCI studies have used high frequency square pulse visual stimuli to reduce subjective fatigue. However, the effect of total harmonic distortion (THD) has not been considered. Compared to CRT and LCD monitors, LED screen displays high-frequency wave with better refresh rate. In this study, we present high frequency sine wave simple and rhythmic patterns with low THD rate by LED to analyze SSVEP responses and evaluate subjective fatigue in normal subjects. Materials and Methods: We used patterns of 3-sequence high-frequency sine waves (25, 30, and 35 Hz) to design our visual stimuli. Nine stimuli patterns, 3 simple (repetition of each of above 3 frequencies e.g., P25-25-25) and 6 rhythmic (all of the frequencies in 6 different sequences e.g., P25-30-35) were chosen. A hardware setup with low THD rate (<0.1%) was designed to present these patterns on LED. Twenty two normal subjects (aged 23-30 (25 ± 2.1) yrs) were enrolled. Visual analog scale (VAS) was used for subjective fatigue evaluation after presentation of each stimulus pattern. PSD, CCA, and LASSO methods were employed to analyze SSVEP responses. The data including SSVEP features and fatigue rate for different visual stimuli patterns were statistically evaluated. Results: All 9 visual stimuli patterns elicited SSVEP responses. Overall, obtained accuracy rates were 88.35% for PSD and > 90% for CCA and LASSO (for TWs > 1 s). High frequency rhythmic patterns group with low THD rate showed higher accuracy rate (99.24%) than simple patterns group (98.48%). Repeated measure ANOVA showed significant difference between rhythmic pattern features (P < 0.0005). Overall, there was no significant difference between the VAS of rhythmic [3.85 ± 2.13] compared to the simple patterns group [3.96 ± 2.21], (P = 0.63). Rhythmic group had lower within group VAS variation (min = P25-30-35 [2.90 ± 2.45], max = P35-25-30 [4.81 ± 2.65]) as well as least individual pattern VAS (P25-30-35). Discussion and Conclusion: Overall, rhythmic and simple pattern groups had higher and similar accuracy rates. Rhythmic stimuli patterns showed insignificantly lower fatigue rate than simple patterns. We conclude that both rhythmic and simple visual high frequency sine wave stimuli require further research for human subject SSVEP-BCI studies.