Effects of high stimulus presentation rate on EEG template characteristics and performance of c-VEP based BCIs

High-Density Electroencephalogram Facilitates the Detection of Small Stimuli in Code-Modulated Visual Evoked Potential Brain-Computer Interfaces

In recent years, there has been a considerable amount of research on visual evoked potential (VEP)-based brain-computer interfaces (BCIs). However, it remains a big challenge to detect VEPs elicited by small visual stimuli. To address this challenge, this study employed a 256-electrode high-density electroencephalogram (EEG) cap with 66 electrodes in the parietal and occipital lobes to record EEG signals. An online BCI system based on code-modulated VEP (C-VEP) was designed and implemented with thirty targets modulated by a time-shifted binary pseudo-random sequence. A task-discriminant component analysis (TDCA) algorithm was employed for feature extraction and classification. The offline and online experiments were designed to assess EEG responses and classification performance for comparison across four different stimulus sizes at visual angles of 0.5°, 1°, 2°, and 3°. By optimizing the data length for each subject in the online experiment, information transfer rates (ITRs) of 126.48 ± 14.14 bits/min, 221.73 ± 15.69 bits/min, 258.39 ± 9.28 bits/min, and 266.40 ± 6.52 bits/min were achieved for 0.5°, 1°, 2°, and 3°, respectively. This study further compared the EEG features and classification performance of the 66-electrode layout from the 256-electrode EEG cap, the 32-electrode layout from the 128-electrode EEG cap, and the 21-electrode layout from the 64-electrode EEG cap, elucidating the pivotal importance of a higher electrode density in enhancing the performance of C-VEP BCI systems using small stimuli.

Burst c-VEP Based BCI: Optimizing stimulus design for enhanced classification with minimal calibration data and improved user experience

The utilization of aperiodic flickering visual stimuli under the form of code-modulated Visual Evoked Potentials (c-VEP) represents a pivotal advancement in the field of reactive Brain-Computer Interface (rBCI). A major advantage of the c-VEP approach is that the training of the model is independent of the number and complexity of targets, which helps reduce calibration time. Nevertheless, the existing designs of c-VEP stimuli can be further improved in terms of visual user experience but also to achieve a higher signal-to-noise ratio, while shortening the selection time and calibration process. In this study, we introduce an innovative variant of code-VEP, referred to as "Burst c-VEP". This original approach involves the presentation of short bursts of aperiodic visual flashes at a deliberately slow rate, typically ranging from two to four flashes per second. The rationale behind this design is to leverage the sensitivity of the primary visual cortex to transient changes in low-level stimuli features to reliably elicit distinctive series of visual evoked potentials. In comparison to other types of faster-paced code sequences, burst c-VEP exhibit favorable properties to achieve high bitwise decoding performance using convolutional neural networks (CNN), which yields potential to attain faster selection time with the need for less calibration data. Furthermore, our investigation focuses on reducing the perceptual saliency of c-VEP through the attenuation of visual stimuli contrast and intensity to significantly improve users' visual comfort. The proposed solutions were tested through an offline 4-classes c-VEP protocol involving 12 participants. Following a factorial design, participants were instructed to focus on c-VEP targets whose pattern (burst and maximum-length sequences) and amplitude (100% or 40% amplitude depth modulations) were manipulated across experimental conditions. Firstly, the full amplitude burst c-VEP sequences exhibited higher accuracy, ranging from 90.5% (with 17.6s of calibration data) to 95.6% (with 52.8s of calibration data), compared to its m-sequence counterpart (71.4% to 85.0%). The mean selection time for both types of codes (1.5 s) compared favorably to reports from previous studies. Secondly, our findings revealed that lowering the intensity of the stimuli only slightly decreased the accuracy of the burst code sequences to 94.2% while leading to substantial improvements in terms of user experience. Taken together, these results demonstrate the high potential of the proposed burst codes to advance reactive BCI both in terms of performance and usability. The collected dataset, along with the proposed CNN architecture implementation, are shared through open-access repositories.

Combining brain-computer interfaces and multiplayer video games: an application based on c-VEPs

Introduction and objective

Video games are crucial to the entertainment industry, nonetheless they can be challenging to access for those with severe motor disabilities. Brain-computer interfaces (BCI) systems have the potential to help these individuals by allowing them to control video games using their brain signals. Furthermore, multiplayer BCI-based video games may provide valuable insights into how competitiveness or motivation affects the control of these interfaces. Despite the recent advancement in the development of code-modulated visual evoked potentials (c-VEPs) as control signals for high-performance BCIs, to the best of our knowledge, no studies have been conducted to develop a BCI-driven video game utilizing c-VEPs. However, c-VEPs could enhance user experience as an alternative method. Thus, the main goal of this work was to design, develop, and evaluate a version of the well-known 'Connect 4' video game using a c-VEP-based BCI, allowing 2 users to compete by aligning 4 same-colored coins vertically, horizontally or diagonally.

Methods

The proposed application consists of a multiplayer video game controlled by a real-time BCI system processing 2 electroencephalograms (EEGs) sequentially. To detect user intention, columns in which the coin can be placed was encoded with shifted versions of a pseudorandom binary code, following a traditional circular shifting c-VEP paradigm. To analyze the usability of our application, the experimental protocol comprised an evaluation session by 22 healthy users. Firstly, each user had to perform individual tasks. Afterward, users were matched and the application was used in competitive mode. This was done to assess the accuracy and speed of selection. On the other hand, qualitative data on satisfaction and usability were collected through questionnaires.

Results

The average accuracy achieved was 93.74% ± 1.71%, using 5.25 seconds per selection. The questionnaires showed that users felt a minimal workload. Likewise, high satisfaction values were obtained, highlighting that the application was intuitive and responds quickly and smoothly.

Conclusions

This c-VEP based multiplayer video game has reached suitable performance on 22 users, supported by high motivation and minimal workload. Consequently, compared to other versions of "Connect 4" that utilized different control signals, this version has exhibited superior performance.

Intensity dependence of sub-harmonics in cortical response to photic stimulation

Objective. Periodic photic stimulation of human volunteers at 10 Hz is known to entrain their Electroencephalography (EEG) signals. This entrainment manifests as an increment in power at 10, 20, 30 Hz. We observed that this entrainment is accompanied by the emergence of sub-harmonics, but only at specific frequencies and higher intensities of the stimulating signal. Thereafter, we describe our results and explain them using the physiologically inspired Jansen and Rit Neural Mass Model (NMM).Approach. Four human volunteers were separately exposed to both high and low intensity 10 Hz and 6 Hz stimulation. A total of 4 experiments per subject were therefore performed. Simulations and bifurcation analysis of the NMM were carried out and compared with the experimental findings. <i> Main results. High intensity 10 Hz stimulation led to an increment in power at 5 Hz across all the 4 subjects. No increment of power was observed with low intensity stimulation. However, when the same protocol was repeated with a 6 Hz photic stimulation, neither high nor low intensity stimulation were found to cause a discernible change in power at 3 Hz. We found that the NMM was able to recapitulate these results. A further numerical analysis indicated that this arises from the underlying bifurcation structure of the NMM. <i> Significance. The excellent match between theory and experiment suggest that the bifurcation properties of the NMM are mirroring similar features possessed by the actual neural masses producing the EEG dynamics. Neural Mass Models could thus be valuable for understanding properties and pathologies of EEG dynamics, and may contribute to the engineering of brain-computer interface technologies.

Classification of code-modulated visual evoked potentials using adaptive modified covariance beamformer and EEG signals

OBJECTIVE

In general, brain computer interface (BCI) studies based on code-modulated Visual Evoked Potentials (c-VEP) use m-sequences to decode EEG responses to visual stimuli. BCI systems based on the c-VEP paradigm can simultaneously present a large number of commands, which results in a significantly high information transfer rate (ITR). Spatiotemporal beamforming (STB) is one of the commonly used approaches in c-VEP-based BCI systems.

APPROACH

In the current work, a novel STB-based technique is proposed to detect the gazed targets. The proposed method improves the performance of conventional STB-based techniques by providing a robust estimation of the covariance matrix in short stimulation times. Different user parameter-free methods, including the convex combination (CC), the general linear combination (GLC), and the modified versions of these techniques, are used to estimate a reliable and robust covariance matrix when a small number of repetitions are available.

MAIN RESULTS

The stimulus presentation rate of 120 Hz is used to assess the performance of the proposed structures. Our proposed methods improved the classification accuracy by an average of 20% compared to the conventional STB method at the shortest stimulation time. The proposed method achieves an average ITR of 157.07 bits/min by using only two repetitions of the m-sequences.

SIGNIFICANCE

The results show that our proposed methods perform significantly better than the conventional STB technique in all stimulation times.

Assessing the Effect of the Refresh Rate of a Device on Various Motion Stimulation Frequencies Based on Steady-State Motion Visual Evoked Potentials

The refresh rate is one of the important parameters of visual presentation devices, and assessing the effect of the refresh rate of a device on motion perception has always been an important direction in the field of visual research. This study examined the effect of the refresh rate of a device on the motion perception response at different stimulation frequencies and provided an objective visual electrophysiological assessment method for the correct selection of display parameters in a visual perception experiment. In this study, a flicker-free steady-state motion visual stimulation with continuous scanning frequency and different forms (sinusoidal or triangular) was presented on a low-latency LCD monitor at different refresh rates. Seventeen participants were asked to observe the visual stimulation without head movement or eye movement, and the effect of the refresh rate was assessed by analyzing the changes in the intensity of their visual evoked potentials. The results demonstrated that an increased refresh rate significantly improved the intensity of motion visual evoked potentials at stimulation frequency ranges of 7-28 Hz, and there was a significant interaction between the refresh rate and motion frequency. Furthermore, the increased refresh rate also had the potential to enhance the ability to perceive similar motion. Therefore, we recommended using a refresh rate of at least 120 Hz in motion visual perception experiments to ensure a better stimulation effect. If the motion frequency or velocity is high, a refresh rate of≥240 Hz is also recommended.

Brain-computer interfaces based on code-modulated visual evoked potentials (c-VEP): a literature review

Objective.Code-modulated visual evoked potentials (c-VEP) have been consolidated in recent years as robust control signals capable of providing non-invasive brain-computer interfaces (BCIs) for reliable, high-speed communication. Their usefulness for communication and control purposes has been reflected in an exponential increase of related articles in the last decade. The aim of this review is to provide a comprehensive overview of the literature to gain understanding of the existing research on c-VEP-based BCIs, since its inception (1984) until today (2021), as well as to identify promising future research lines.Approach.The literature review was conducted according to the Preferred Reporting Items for Systematic reviews and Meta-Analysis guidelines. After assessing the eligibility of journal manuscripts, conferences, book chapters and non-indexed documents, a total of 70 studies were included. A comprehensive analysis of the main characteristics and design choices of c-VEP-based BCIs was discussed, including stimulation paradigms, signal processing, modeling responses, applications, etc.Main results.The literature review showed that state-of-the-art c-VEP-based BCIs are able to provide an accurate control of the system with a large number of commands, high selection speeds and even without calibration. In general, a lack of validation in real setups was observed, especially regarding the validation with disabled populations. Future work should be focused toward developing self-paced c-VEP-based portable BCIs applied in real-world environments that could exploit the unique benefits of c-VEP paradigms. Some aspects such as asynchrony, unsupervised training, or code optimization still require further research and development.Significance.Despite the growing popularity of c-VEP-based BCIs, to the best of our knowledge, this is the first literature review on the topic. In addition to providing a joint discussion of the advances in the field, some future lines of research are suggested to contribute to the development of reliable plug-and-play c-VEP-based BCIs.

A visual brain-computer interface as communication aid for patients with amyotrophic lateral sclerosis

OBJECTIVE

Brain-Computer Interface (BCI) spellers that make use of code-modulated Visual Evoked Potentials (cVEP) may provide a fast and more accurate alternative to existing visual BCI spellers for patients with Amyotrophic Lateral Sclerosis (ALS). However, so far the cVEP speller has only been tested on healthy participants.

METHODS

We assess the brain responses, BCI performance and user experience of the cVEP speller in 20 healthy participants and 10 ALS patients. All participants performed a cued and free spelling task, and a free selection of Yes/No answers.

RESULTS

27 out of 30 participants could perform the cued spelling task with an average accuracy of 79% for ALS patients, 88% for healthy older participants and 94% for healthy young participants. All 30 participants could answer Yes/No questions freely, with an average accuracy of around 90%.

CONCLUSIONS

With ALS patients typing on average 10 characters per minute, the cVEP speller presented in this paper outperforms other visual BCI spellers.

SIGNIFICANCE

These results support a general usability of cVEP signals for ALS patients, which may extend far beyond the tested speller to control e.g. an alarm, automatic door, or TV within a smart home.

Asynchronous c-VEP communication tools-efficiency comparison of low-target, multi-target and dictionary-assisted BCI spellers

Keyboards and smartphones allow users to express their thoughts freely via manual control. Hands-free communication can be realized with brain-computer interfaces (BCIs) based on code-modulated visual evoked potentials (c-VEPs). Various variations of such spellers have been developed: Low-target systems, multi-target systems and systems with dictionary support. In general, it is not clear which kinds of systems are optimal in terms of reliability, speed, cognitive load, and visual load. The presented study investigates the feasibility of different speller variations. 58 users tested a 4-target speller and a 32-target speller with and without dictionary functionality. For classification, multiple individualized spatial filters were generated via canonical correlation analysis (CCA). We used an asynchronous implementation allowing non-control state, thus aiming for high accuracy rather than speed. All users were able to control the tested spellers. Interestingly, no significant differences in accuracy were found: 94.4%, 95.5% and 94.0% for 4-target spelling, 32-target spelling, and dictionary-assisted 32-target spelling. The mean ITRs were highest for the 32-target interface: 45.2, 96.9 and 88.9 bit/min. The output speed in characters per minute, was highest in dictionary-assisted spelling: 8.2, 19.5 and 31.6 characters/min. According to questionnaire results, 86% of the participants preferred the 32-target speller over the 4-target speller.

New approach for designing cVEP BCI stimuli based on superposition of edge responses

The purpose of this study is to develop a new methodology for designing stimulus sequences for Brain Computer Interfaces that utilize code modulated Visually Evoked Potentials (cVEP BCIs), based on experimental results regarding the behavior and the properties of the actual EEG responses of the visual system to binary-coded visual stimuli, such that training time is reduced and possible number of targets is increased. EEG from 8 occipital sites is recorded with 2000 sps, in response to visual stimuli presented on a computer monitor with 60 Hz refresh rate. EEG responses of the visual system to black-to-white and white-to-black transitions of a target area on the monitor are recorded for 500 ms, for 160 trials, and signal-averaged to obtain the onset (positive edge) and offset (negative edge) responses, respectively. It is found that both edge responses are delayed by 50 ms and wane completely within 350 ms. These edge responses are then used to generate (predict) the EEG responses to arbitrary binary stimulus sequences using the superposition principle. It is found that the generated and the measured EEG responses to certain (16) simple short sequences (16.67-350 ms) are highly correlated. These 'optimal short patterns' are then randomly combined to design the long (120 bit, 2 sec) 'Superposition Optimized Pulse (SOP)' sequences, and their EEG response templates are obtained by superposition of the edge responses. A SOP sequence-based Visual Speller BCI application yielded higher accuracy (95.9%) and Information Transfer Rate (ITR) (57.2 bpm), compared to when superposition principle is applied to conventional m-sequences and randomly generated sequences. Training for the BCI application involves only the acquisition of the edge responses and takes less than 4 min. This is the first study in which the EEG templates for cVEP BCI sequences are obtained by the superposition of edge responses.