Improve the Classification Efficiency of High-Frequency Phase-Tagged SSVEP by a Recursive Bayesian-Based Approach

A user-friendly visual brain-computer interface based on high-frequency steady-state visual evoked fields recorded by OPM-MEG

Objective. Magnetoencephalography (MEG) shares a comparable time resolution with electroencephalography. However, MEG excels in spatial resolution, enabling it to capture even the subtlest and weakest brain signals for brain-computer interfaces (BCIs). Leveraging MEG's capabilities, specifically with optically pumped magnetometers (OPM-MEG), proves to be a promising avenue for advancing MEG-BCIs, owing to its exceptional sensitivity and portability. This study harnesses the power of high-frequency steady-state visual evoked fields (SSVEFs) to build an MEG-BCI system that is flickering-imperceptible, user-friendly, and highly accurate.Approach.We have constructed a nine-command BCI that operates on high-frequency SSVEF (58-62 Hz with a 0.5 Hz interval) stimulation. We achieved this by placing the light source inside and outside the magnetic shielding room, ensuring compliance with non-magnetic and visual stimulus presentation requirements. Five participants took part in offline experiments, during which we collected six-channel multi-dimensional MEG signals along both the vertical (Z-axis) and tangential (Y-axis) components. Our approach leveraged the ensemble task-related component analysis algorithm for SSVEF identification and system performance evaluation.Main Results.The offline average accuracy of our proposed system reached an impressive 92.98% when considering multi-dimensional conjoint analysis using data from both theZandYaxes. Our method achieved a theoretical average information transfer rate (ITR) of 58.36 bits min-1with a data length of 0.7 s, and the highest individual ITR reached an impressive 63.75 bits min-1.Significance.This study marks the first exploration of high-frequency SSVEF-BCI based on OPM-MEG. These results underscore the potential and feasibility of MEG in detecting subtle brain signals, offering both theoretical insights and practical value in advancing the development and application of MEG in BCI systems.

A fused multi-subfrequency bands and CBAM SSVEP-BCI classification method based on convolutional neural network

For the brain-computer interface (BCI) system based on steady-state visual evoked potential (SSVEP), it is difficult to obtain satisfactory classification performance for short-time window SSVEP signals by traditional methods. In this paper, a fused multi-subfrequency bands and convolutional block attention module (CBAM) classification method based on convolutional neural network (CBAM-CNN) is proposed for discerning SSVEP-BCI tasks. This method extracts multi-subfrequency bands SSVEP signals as the initial input of the network model, and then carries out feature fusion on all feature inputs. In addition, CBAM is embedded in both parts of the initial input and feature fusion for adaptive feature refinement. To verify the effectiveness of the proposed method, this study uses the datasets of Inner Mongolia University of Technology (IMUT) and Tsinghua University (THU) to evaluate the performance of the proposed method. The experimental results show that the highest accuracy of CBAM-CNN reaches 0.9813 percentage point (pp). Within 0.1-2 s time window, the accuracy of CBAM-CNN is 0.0201-0.5388 (pp) higher than that of CNN, CCA-CWT-SVM, CCA-SVM, CCA-GNB, FBCCA, and CCA. Especially in the short-time window range of 0.1-1 s, the performance advantage of CBAM-CNN is more significant. The maximum information transmission rate (ITR) of CBAM-CNN is 503.87 bit/min, which is 227.53 bit/min-503.41 bit/min higher than the above six EEG decoding methods. The study further results show that CBAM-CNN has potential application value in SSVEP decoding.

A high-performance SSVEP-based BCI using imperceptible flickers

Objective.Existing steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) struggle to balance user experience and system performance. This study proposed an individualized space and phase modulation method to code imperceptible flickers at 60 Hz towards a user-friendly SSVEP-based BCI with high performance.Approach.The individualized customization of visual stimulation took the subject-to-subject variability in cortex geometry into account. An annulus global-stimulation was divided into local-stimulations of eight annular sectors and presented to subjects separately. The local-stimulation SSVEPs were superimposed to simulate global-stimulation SSVEPs with 4⁷space and phase coding combinations. A four-class phase-coded BCI diagram was used to evaluate the simulated classification performance. The performance ranking of all simulated global-stimulation SSVEPs were obtained and three performance levels (optimal, medium, worst) of individualized modulation groups were searched for each subject. The standard-modulation group conforming to the V1 'cruciform' geometry and the non-modulation group were involved as controls. A four-target phase-coded BCI system with SSVEPs at 60 Hz was implemented with the five modulation groups and questionnaires were used to evaluate user experience.Main results.The proposed individualized space and phase modulation method effectively modulated the SSVEP intensity without affecting the user experience. The online BCI system using the 60 Hz stimuli achieved mean information transfer rates of 52.8 ± 1.9 bits min^-1, 16.8 ± 2.4 bits min^-1, and 42.4 ± 3.0 bits min^-1with individualized optimal-modulation, individualized worst-modulation, and non-modulation groups, respectively.Significance.Structural and functional characteristics of the human visual cortex were exploited to enhance the response intensity of SSVEPs at 60 Hz, resulting in a high-performance BCI system with good user experience. This study has important theoretical significance and application value for promoting the development of the visual BCI technology.

Multisymbol Time Division Coding for High-Frequency Steady-State Visual Evoked Potential-Based Brain-Computer Interface

The optimization of coding stimulus is a crucial factor in the study of steady-state visual evoked potential (SSVEP)-based brain-computer interface(BCI).This study proposed an encoding approach named Multi-Symbol Time Division Coding (MSTDC). This approach is based on a protocol of maximizing the distance between neural responses, which aims to encode stimulation systems implementing any number of targets with finite stimulations of different frequencies and phases. Firstly, this study designed an SSVEP-based BCI system containing forty targets with this approach. The stimulation encoding of this system was achieved with four temporal-divided stimuli that adopt the same frequency of 30 Hz and different phases. During the online experiments of twelve subjects, this system achieved an average accuracy of 96.77 ±2.47 % and an average information transfer rate (ITR) of 119.05 ± 6.11 bits/min. This study also devised an SSVEP-based BCI system containing 72 targets and proposed a Template Splicing task-related component analysis (TRCA) algorithm that utilized the dataset of the previous system containing forty targets as the training dataset. The subjects acquired an average accuracy of 86.23 ± 7.75% and an average ITR of 95.68 ± 14.19 bits/min. It can be inferred that MSTDC can encode multiple targets with limited frequencies and phases of stimuli. Meanwhile, this protocol can be effortlessly expanded into other systems and sufficiently reduce the cost of collecting training data. This study provides a feasible technique for obtaining a comfortable SSVEP-based BCI with multiple targets while maintaining high information transfer rate.

A Hybrid Brain-Computer Interface Based on Visual Evoked Potential and Pupillary Response

Brain-computer interface (BCI) based on steady-state visual evoked potential (SSVEP) has been widely studied due to the high information transfer rate (ITR), little user training, and wide subject applicability. However, there are also disadvantages such as visual discomfort and “BCI illiteracy.” To address these problems, this study proposes to use low-frequency stimulations (12 classes, 0.8–2.12 Hz with an interval of 0.12 Hz), which can simultaneously elicit visual evoked potential (VEP) and pupillary response (PR) to construct a hybrid BCI (h-BCI) system. Classification accuracy was calculated using supervised and unsupervised methods, respectively, and the hybrid accuracy was obtained using a decision fusion method to combine the information of VEP and PR. Online experimental results from 10 subjects showed that the averaged accuracy was 94.90 ± 2.34% (data length 1.5 s) for the supervised method and 91.88 ± 3.68% (data length 4 s) for the unsupervised method, which correspond to the ITR of 64.35 ± 3.07 bits/min (bpm) and 33.19 ± 2.38 bpm, respectively. Notably, the hybrid method achieved higher accuracy and ITR than that of VEP and PR for most subjects, especially for the short data length. Together with the subjects’ feedback on user experience, these results indicate that the proposed h-BCI with the low-frequency stimulation paradigm is more comfortable and favorable than the traditional SSVEP-BCI paradigm using the alpha frequency range.

Filter Bank Convolutional Neural Network for Short Time-Window Steady-State Visual Evoked Potential Classification

Convolutional neural network (CNN) has been gradually applied to steady-state visual evoked potential (SSVEP) of the brain-computer interface (BCI). Frequency-domain features extracted by fast Fourier Transform (FFT) or time-domain signals are used as network input. In the frequency-domain diagram, the features at the short time-window are not obvious and the phase information of each electrode channel may be ignored as well. Hence we propose a time-domain-based CNN method (tCNN), using the time-domain signal as network input. And the filter bank tCNN (FB-tCNN) is further proposed to improve its performance in the short time-window. We compare FB-tCNN with the canonical correlation analysis (CCA) methods and other CNN methods in our dataset and public dataset. And FB-tCNN shows superior performance at the short time-window in the intra-individual test. At the 0.2 s time-window, the accuracy of our method reaches 88.36 ± 4.89 % in our dataset, 77.78 ± 2.16 % and 79.21 ± 1.80 % respectively in the two sessions of the public dataset, which is higher than other methods. The impacts of training-subject number and data length in inter-individual or cross-individual are studied. FB-tCNN shows the potential in implementing inter-individual BCI. Further analysis shows that the deep learning method is easier in terms of the implementation of the asynchronous BCI system than the training data-driven CCA. The code is available for reproducibility at https://github.com/DingWenl/FB-tCNN.

A Novel Approach to Decode Covert Spatial Attention Using SSVEP and Single-Frequency Phase-Coded Stimuli

This paper investigates for the first time the use of single-frequency phase-coded stimuli to detect covert visuo-spatial attention (CVSA) with steady-state visual evoked potentials (SSVEP). Two 15Hz pattern-onset stimulations were encoded with opposite phases and simultaneously presented on a LCD monitor. The effects of attending each stimulus on the amplitudes and phases of the evoked SSVEPs across the visual cortex are explored. A real-time CVSA classification experiment was simulated offline with 9 BCI-naive subjects, achieving an average classification accuracy of 88.4 ± 8% SE. Our results are, to our knowledge, the first report that CVSA can be decoded with SSVEP using single-frequency phase-coded stimuli. This opens opportunities for attention-tracking applications with largely increased number of targets.

Signal processing algorithms for SSVEP-based brain computer interface: State-of-the-art and recent developments

Over past two decades, steady-state evoked potentials (SSVEP)-based brain computer interface (BCI) systems have been extensively developed. As we all know, signal processing algorithms play an important role in this BCI. However, there is no comprehensive review of the latest development of signal processing algorithms for SSVEP-based BCI. By analyzing the papers published in authoritative journals in nearly five years, signal processing algorithms of preprocessing, feature extraction and classification modules are discussed in detail. In addition, other aspects existed in this BCI are mentioned. The following key problems are solved. (1) In recent years, which signal processing algorithms are frequently used in each module? (2) Which signal processing algorithms attract more attention in recent years? (3) Which modules are the key to signal processing in BCI field? This information is very important for choosing the appropriate algorithms, and can also be considered as a reference for further research. Simultaneously, we hope that this work can provide relevant BCI researchers with valuable information about the latest trends of signal processing algorithms for SSVEP-based BCI systems.