An improved SSVEP-based brain-computer interface with low-contrast visual stimulation and its application in UAV control

Efficient communication and regulation are crucial for advancing brain-computer interfaces (BCIs), with the steady-state visual-evoked potential (SSVEP) paradigm demonstrating high accuracy and information transfer rates. However, the conventional SSVEP paradigm encounters challenges related to visual occlusion and fatigue. In this study, we propose an improved SSVEP paradigm that addresses these issues by lowering the contrast of visual stimulation. The improved paradigms outperform the traditional paradigm in the experiments, significantly reducing the visual stimulation of the SSVEP paradigm. Furthermore, we apply this enhanced paradigm to a BCI navigation system, enabling two-dimensional navigation of unmanned aerial vehicles (UAVs) through a first-person perspective. Experimental results indicate the enhanced SSVEP-based BCI system's accuracy in performing navigation and search tasks. Our findings highlight the feasibility of the enhanced SSVEP paradigm in mitigating visual occlusion and fatigue issues, presenting a more intuitive and natural approach for BCIs to control external equipment.NEW & NOTEWORTHY In this article, we proposed an improved steady-state visual-evoked potential (SSVEP) paradigm and constructed an SSVEP-based brain-computer interface (BCI) system to navigate the unmanned aerial vehicle (UAV) in two-dimensional (2-D) physical space. We proposed a modified method for evaluating visual fatigue including subjective score and objective indices. The results indicated that the improved SSVEP paradigm could effectively reduce visual fatigue while maintaining high accuracy.

The SSHVEP Paradigm-Based Brain Controlled Method for Grasping Robot Using MVMD Combined CNN Model

In recent years, the steady-state visual evoked potentials (SSVEP) based brain control method has been employed to help people with disabilities because of its advantages of high information transmission rate and low training time. However, the existing SSVEP brain control methods cannot adapt to dynamic or unstructured environments. Moreover, the recognition accuracy from the conventional decoding algorithm still needs to improve. To address the above problems, this study proposed a steady-state hybrid visual evoked potentials (SSHVEP) paradigm using the grasping targets in their environment to improve the connection between the subjects' and their dynamic environments. Moreover, a novel EEG decoding method, using the multivariate variational mode decomposition (MVMD) algorithm for adaptive sub-band division and convolutional neural network (CNN) for target recognition, was applied to improve the decoding accuracy of the SSHVEPs. 18 subjects participated in the offline and online experiments. The offline accuracy across 18 subjects by the 9-target SSHVEP paradigm was up to 95.41 ± 2.70 %, which is a 5.80% improvement compared to the conventional algorithm. To further validate the performance of the proposed method, the brain-controlled grasping robot system using the SSHVEP paradigm was built. The average accuracy reached 93.21 ± 10.18 % for the online experiment. All the experimental results demonstrated the effectiveness of the brain-computer interaction method based on the SSHVEP paradigm and the MVMD combined CNN algorithm studied in this paper.

SSVEP unsupervised adaptive feature recognition method based on self-similarity of same-frequency signals

Introduction

As an important human-computer interaction technology, steady-state visual evoked potential (SSVEP) plays a key role in the application of brain computer interface (BCI) systems by accurately decoding SSVEP signals. Currently, the majority SSVEP feature recognition methods use a static classifier. However, electroencephalogram (EEG) signals are non-stationary and time-varying. Hence, an adaptive classification method would be an alternative option to a static classifier for tracking the changes in EEG feature distribution, as its parameters can be re-estimated and updated with the input of new EEG data.

Methods

In this study, an unsupervised adaptive classification algorithm is designed based on the self-similarity of same-frequency signals. The proposed classification algorithm saves the EEG data that has undergone feature recognition as a template signal in accordance with its estimated label, and the new testing signal is superimposed with the template signals at each stimulus frequency as the new test signals to be analyzed. With the continuous input of EEG data, the template signals are continuously updated.

Results

By comparing the classification accuracy of the original testing signal and the testing signal superimposed with the template signals, this study demonstrates the effectiveness of using the self-similarity of same-frequency signals in the adaptive classification algorithm. The experimental results also show that the longer the SSVEP-BCI system is used, the better the responses of users on SSVEP are, and the more significantly the adaptive classification algorithm performs in terms of feature recognition. The testing results of two public datasets show that the adaptive classification algorithm outperforms the static classification method in terms of feature recognition.

Discussion

The proposed adaptive classification algorithm can update the parameters with the input of new EEG data, which is of favorable impact for the accurate analysis of EEG data with time-varying characteristics.

Age-related differences in the transient and steady state responses to different visual stimuli

Objective

Brain-computer interface (BCI) has great potential in geriatric applications. However, most BCI studies in the literature used data from young population, and dedicated studies investigating the feasibility of BCIs among senior population are scarce. The current study, we analyzed the age-related differences in the transient electroencephalogram (EEG) response used in visual BCIs, i.e., visual evoked potential (VEP)/motion onset VEP (mVEP), and steady state-response, SSVEP/SSMVEP, between the younger group (age ranges from 22 to 30) and senior group (age ranges from 60 to 75).

Methods

The visual stimulations, including flicker, checkerboard, and action observation (AO), were designed with a periodic frequency. Videos of several hand movement, including grasping, dorsiflexion, the thumb opposition, and pinch were utilized to generate the AO stimuli. Eighteen senior and eighteen younger participants were enrolled in the experiments. Spectral-temporal characteristics of induced EEG were compared. Three EEG algorithms, canonical correlation analysis (CCA), task-related component analysis (TRCA), and extended CCA, were utilized to test the performance of the respective BCI systems.

Results

In the transient response analysis, the motion checkerboard and AO stimuli were able to elicit prominent mVEP with a specific P1 peak and N2 valley, and the amplitudes of P1 elicited in the senior group were significantly higher than those in the younger group. In the steady-state analysis, SSVEP/SSMVEP could be clearly elicited in both groups. The CCA accuracies of SSVEPs/SSMVEPs in the senior group were slightly lower than those in the younger group in most cases. With extended CCA, the performance of both groups improved significantly. However, for AO targets, the improvement of the senior group (from 63.1 to 71.9%) was lower than that of the younger group (from 63.6 to 83.6%).

Conclusion

Compared with younger subjects, the amplitudes of P1 elicited by motion onset is significantly higher in the senior group, which might be a potential advantage for seniors if mVEP-based BCIs is used. This study also shows for the first time that AO-based BCI is feasible for the senior population. However, new algorithms for senior subjects, especially in identifying AO targets, are needed.

An improved cross-subject spatial filter transfer method for SSVEP-based BCI

Steady-state visual evoked potential (SSVEP) training feature recognition algorithms utilize user training data to reduce the interference of spontaneous electroencephalogram (EEG) activities on SSVEP response for improved recognition accuracy. The data collection process can be tedious, increasing the mental fatigue of users and also seriously affecting the practicality of SSVEP-based brain-computer interface (BCI) systems. As an alternative, a cross-subject spatial filter transfer (CSSFT) method to transfer an existing user data model with good SSVEP response to new user test data has been proposed. The CSSFT method uses superposition averages of data for multiple blocks of data as transfer data. However, the amplitude and pattern of brain signals are often significantly different across trials. The goal of this study was to improve superposition averaging for the CSSFT method and propose an Ensemble scheme based on ensemble learning, and an Expansion scheme based on matrix expansion. The feature recognition performance was compared for CSSFT and the proposed improved CSSFT method using two public datasets. The results demonstrated that the improved CSSFT method can significantly improve the recognition accuracy and information transmission rate of existing methods. This strategy avoids a tedious data collection process, and promotes the potential practical application of BCI systems.

An extended binary subband canonical correlation analysis detection algorithm oriented to the radial contraction-expansion motion steady-state visual evoked paradigm

The radial contraction-expansion motion paradigm is a novel steady-state visual evoked experimental paradigm, and the electroencephalography (EEG) evoked potential is different from the traditional luminance modulation paradigm. The signal energy is concentrated chiefly in the fundamental frequency, while the higher harmonic power is lower. Therefore, the conventional steady-state visual evoked potential recognition algorithms optimizing multiple harmonic response components, such as the extended canonical correlation analysis (eCCA) and task-related component analysis (TRCA) algorithm, have poor recognition performance under the radial contraction-expansion motion paradigm. This paper proposes an extended binary subband canonical correlation analysis (eBSCCA) algorithm for the radial contraction-expansion motion paradigm. For the radial contraction-expansion motion paradigm, binary subband filtering was used to optimize the weighting coefficients of different frequency response signals, thereby improving the recognition performance of EEG signals. The results of offline experiments involving 13 subjects showed that the eBSCCA algorithm exhibits a better performance than the eCCA and TRCA algorithms under the stimulation of the radial contraction-expansion motion paradigm. In the online experiment, the average recognition accuracy of 13 subjects was 88.68% ± 6.33%, and the average information transmission rate (ITR) was 158.77 ± 43.67 bits/min, which proved that the algorithm had good recognition effect signals evoked by the radial contraction-expansion motion paradigm.

Cross-subject spatial filter transfer method for SSVEP-EEG feature recognition

Objective.Steady-state visual evoked potential (SSVEP) is an important control method of the brain-computer interface (BCI) system. The development of an efficient SSVEP feature decoding algorithm is the core issue in SSVEP-BCI. It has been proposed to use user training data to reduce the spontaneous electroencephalogram activity interference on SSVEP response, thereby improving the feature recognition accuracy of the SSVEP signal. Nevertheless, the tedious data collection process increases the mental fatigue of the user and severely affects the applicability of the BCI system.Approach.A cross-subject spatial filter transfer (CSSFT) method that transfer the existing user model with good SSVEP response to the new user test data without collecting any training data from the new user is proposed.Main results.Experimental results demonstrate that the transfer model increases the distinction of the feature discriminant coefficient between the gaze following target and the non-gaze following target and accurately identifies the wrong target in the fundamental algorithm model. The public datasets show that the CSSFT method significantly increases the recognition performance of canonical correlation analysis (CCA) and filter bank CCA. Additionally, when the data used to calculate the transfer model contains one data block only, the CSSFT method retains its effective feature recognition capabilities.Significance.The proposed method requires no tedious data calibration process for new users, provides an effective technical solution for the transfer of the cross-subject model, and has potential application value for promoting the application of the BCI system.

DF-SSmVEP: Dual Frequency Aggregated Steady-State Motion Visual Evoked Potential Design with Bifold Canonical Correlation Analysis

Recent advancements in Electroencephalographic (EEG) sensor technologies and signal processing algorithms have paved the way for further evolution of Brain Computer Interfaces (BCI) in several practical applications, ranging from rehabilitation systems to smart consumer technologies. When it comes to Signal Processing (SP) for BCI, there has been a surge of interest on Steady-State motion Visual Evoked Potentials (SSmVEP), where motion stimulation is used to address key issues associated with conventional light flashing/flickering. Such benefits, however, come with the price of being less accurate and having a lower Information Transfer Rate (ITR). From this perspective, this paper focuses on the design of a novel SSmVEP paradigm without using resources such as trial time, phase, and/or number of targets to enhance the ITR. The proposed design is based on the intuitively pleasing idea of integrating more than one motion within a single SSmVEP target stimuli, simultaneously. To elicit SSmVEP, we designed a novel and innovative dual frequency aggregated modulation paradigm, called the Dual Frequency Aggregated Steady-State motion Visual Evoked Potential (DF-SSmVEP), by concurrently integrating "Radial Zoom" and "Rotation" motions in a single target without increasing the trial length. Compared to conventional SSmVEPs, the proposed DF-SSmVEP framework consists of two motion modes integrated and shown simultaneously each modulated by a specific target frequency. The paper also develops a specific unsupervised classification model, referred to as the Bifold Canonical Correlation Analysis (BCCA), based on two motion frequencies per target. The corresponding covariance coefficients are used as extra features improving the classification accuracy. The proposed DF-SSmVEP is evaluated based on a real EEG dataset and the results corroborate its superiority. The proposed DF-SSmVEP outperforms its counterparts and achieved an average ITR of 30.7 ± 1.97 and an average accuracy of 92.5 ± 2.04, while the Radial Zoom and Rotation result in average ITRs of 18.35 ± 1 and 20.52 ± 2.5, and average accuracies of 68.12 ± 3.5 and 77.5 ± 3.5, respectively.

SSVEP-EEG Feature Enhancement Method Using an Image Sharpening Filter

Steady-state visual evoked potential (SSVEP) is widely used in brain computer interface (BCI), medical detection, and neuroscience, so there is significant interest in enhancing SSVEP features via signal processing for better performance. In this study, an image processing method was combined with brain signal analysis and a sharpening filter was used to extract image details and features for the enhancement of SSVEP features. The results demonstrated that sharpening filter could eliminate the SSVEP signal trend term and suppress its low-frequency component. Meanwhile, sharpening filter effectively enhanced the signal-to-noise ratios (SNRs) of the single-channel and multi-channel fused signals. Image sharpening filter also significantly improved the recognition accuracy of canonical correlation analysis (CCA), filter bank canonical correlation analysis (FBCCA), and task-related component analysis (TRCA). The tools developed here effectively enhanced the SSVEP signal features, suggesting that image processing methods can be considered for improved brain signal analysis.

Enhanced System Robustness of Asynchronous BCI in Augmented Reality using Steady-state Motion Visual Evoked Potential

This study evaluated the effect of change in background on steady state visually evoked potentials (SSVEP) and steady state motion visually evoked potentials (SSMVEP) based brain computer interfaces (BCI) in a small-profile augmented reality (AR) headset. A four target SSVEP and SSMVEP BCI was implemented using the Cognixion AR headset prototype. An active (AB) and a non-active background (NB) were evaluated. The signal characteristics and classification performance of the two BCI paradigms were studied. Offline analysis was performed using canonical correlation analysis (CCA) and complex-spectrum based convolutional neural network (C-CNN). Finally, the asynchronous pseudo-online performance of the SSMVEP BCI was evaluated. Signal analysis revealed that the SSMVEP stimulus was more robust to change in background compared to SSVEP stimulus in AR. The decoding performance revealed that the C-CNN method outperformed CCA for both stimulus types and NB background, in agreement with results in the literature. The average offline accuracies for W=1s of C-CNN were (NB vs. AB): SSVEP: 82% ±15% vs. 60% ±21% and SSMVEP: 71.4% ± 22% vs. 63.5% ± 18%. Additionally, for W=2s, the AR-SSMVEP BCI with the C-CNN method was 83.3% ± 27% (NB) and 74.1% ±22% (AB). The results suggest that with the C-CNN method, the AR-SSMVEP BCI is both robust to change in background conditions and provides high decoding accuracy compared to the AR-SSVEP BCI. This study presents novel results that highlight the robustness and practical application of SSMVEP BCIs developed with a low-cost AR headset.

Enhancing Performance of SSVEP-Based Visual Acuity via Spatial Filtering

The purpose of this study was to enhance the performance of steady-state visual evoked potential (SSVEP)-based visual acuity assessment with spatial filtering methods. Using the vertical sinusoidal gratings at six spatial frequency steps as the visual stimuli for 11 subjects, SSVEPs were recorded from six occipital electrodes (O1, Oz, O2, PO3, POz, and PO4). Ten commonly used training-free spatial filtering methods, i.e., native combination (single-electrode), bipolar combination, Laplacian combination, average combination, common average reference (CAR), minimum energy combination (MEC), maximum contrast combination (MCC), canonical correlation analysis (CCA), multivariate synchronization index (MSI), and partial least squares (PLS), were compared for multielectrode signals combination in SSVEP visual acuity assessment by statistical analyses, e.g., Bland–Altman analysis and repeated-measures ANOVA. The SSVEP signal characteristics corresponding to each spatial filtering method were compared, determining the chosen spatial filtering methods of CCA and MSI with a higher performance than the native combination for further signal processing. After the visual acuity threshold estimation criterion, the agreement between the subjective Freiburg Visual Acuity and Contrast Test (FrACT) and SSVEP visual acuity for the native combination (0.253 logMAR), CCA (0.202 logMAR), and MSI (0.208 logMAR) was all good, and the difference between FrACT and SSVEP visual acuity was also all acceptable for the native combination (−0.095 logMAR), CCA (0.039 logMAR), and MSI (−0.080 logMAR), where CCA-based SSVEP visual acuity had the best performance and the native combination had the worst. The study proved that the performance of SSVEP-based visual acuity can be enhanced by spatial filtering methods of CCA and MSI and also recommended CCA as the spatial filtering method for multielectrode signals combination in SSVEP visual acuity assessment.

SSVEP-EEG Denoising via Image Filtering Methods

Steady-state visual evoked potential (SSVEP) is widely used in electroencephalogram (EEG) control, medical detection, cognitive neuroscience, and other fields. However, successful application requires improving the detection performance of SSVEP signal frequency characteristics. Most strategies to enhance the signal-to-noise ratio of SSVEP utilize application of a spatial filter. Here, we propose a method for image filtering denoising (IFD) of the SSVEP signal from the perspective of image analysis, as a preprocessing step for signal analysis. Arithmetic mean, geometric mean, Gaussian, and non-local means filtering methods were tested, and the experimental results show that image filtering of SSVEP cannot effectively remove the noise. Thus, we proposed a reverse solution in which the SSVEP noise signal was obtained by image filtering, and then the noise was subtracted from the original signal. Comparison of the recognition accuracy of the SSVEP signal before and after denoising was used to evaluate the denoising performance for stimuli of different duration. After IFD processing, SSVEP exhibited higher recognition accuracy, indicating the effectiveness of this proposed denoising method. Application of this method improves the detection performance of SSVEP signal frequency characteristics, combines image processing and brain signal analysis, and expands the research scope of brain signal analysis for widespread application.

Attention-Based Parallel Multiscale Convolutional Neural Network for Visual Evoked Potentials EEG Classification

Electroencephalography (EEG) decoding is an important part of Visual Evoked Potentials-based Brain-Computer Interfaces (BCIs), which directly determines the performance of BCIs. However, long-time attention to repetitive visual stimuli could cause physical and psychological fatigue, resulting in weaker reliable response and stronger noise interference, which exacerbates the difficulty of Visual Evoked Potentials EEG decoding. In this state, subjects' attention could not be concentrated enough and the frequency response of their brains becomes less reliable. To solve these problems, we propose an attention-based parallel multiscale convolutional neural network (AMS-CNN). Specifically, the AMS-CNN first extract robust temporal representations via two parallel convolutional layers with small and large temporal filters respectively. Then, we employ two sequential convolution blocks for spatial fusion and temporal fusion to extract advanced feature representations. Further, we use attention mechanism to weight the features at different moments according to the output-related interest. Finally, we employ a full connected layer with softmax activation function for classification. Two fatigue datasets collected from our lab are implemented to validate the superior classification performance of the proposed method compared to the state-of-the-art methods. Analysis reveals the competitiveness of multiscale convolution and attention mechanism. These results suggest that the proposed framework is a promising solution to improving the decoding performance of Visual Evoked Potential BCIs.

Enhancing Detection of SSMVEP Induced by Action Observation Stimuli Based on Task-Related Component Analysis

Action observation (AO)-based brain-computer interface (BCI) is an important technology in stroke rehabilitation training. It has the advantage of simultaneously inducing steady-state motion visual evoked potential (SSMVEP) and activating sensorimotor rhythm. Moreover, SSMVEP could be utilized to perform classification. However, SSMVEP is composed of complex modulation frequencies. Traditional canonical correlation analysis (CCA) suffers from poor recognition performance in identifying those modulation frequencies at short stimulus duration. To address this issue, task-related component analysis (TRCA) was utilized to deal with SSMVEP for the first time. An interesting phenomenon was found: different modulated frequencies in SSMVEP distributed in different task-related components. On this basis, a multi-component TRCA method was proposed. All the significant task-related components were utilized to construct multiple spatial filters to enhance the detection of SSMVEP. Further, a combination of TRCA and CCA was proposed to utilize both advantages. Results showed that the accuracies using the proposed methods were significant higher than that using CCA at all window lengths and significantly higher than that using ensemble-TRCA at short window lengths (≤2 s). Therefore, the proposed methods further validate the induced modulation frequencies and will speed up the application of the AO-based BCI in rehabilitation.

Similarity-constrained task-related component analysis for enhancing SSVEP detection

Objective. Task-related component analysis (TRCA) is a representative subject-specific training algorithm in steady-state visual evoked potential (SSVEP)-based brain-computer interfaces. Task-related components (TRCs), extracted by the TRCA-based spatial filtering from electroencephalogram (EEG) signals through maximizing the reproducibility across trials, may contain some task-related inherent noise that is still trial-reproducible.Approach. To address this problem, this study proposed a similarity-constrained TRCA (scTRCA) algorithm to remove the task-related noise and extract TRCs maximally correlated with SSVEPs for enhancing SSVEP detection. Similarity constraints, which were created by introducing covariance matrices between EEG training data and an artificial SSVEP template, were added to the objective function of TRCA. Therefore, a better spatial filter was obtained by maximizing not only the reproducibility across trials but also the similarity between TRCs and SSVEPs. The proposed scTRCA was compared with TRCA, multi-stimulus TRCA, and sine-cosine reference signal based on two public datasets.Main results. The performance of TRCA in target identification of SSVEPs is improved by introducing similarity constraints. The proposed scTRCA significantly outperformed the other three methods, and the improvement was more significant especially with insufficient training data.Significance. The proposed scTRCA algorithm is promising for enhancing SSVEP detection considering its better performance and robustness against insufficient calibration.

Weak Feature Extraction and Strong Noise Suppression for SSVEP-EEG Based on Chaotic Detection Technology

Brain computer interface (BCI) is a novel communication method that does not rely on the normal neural pathway between the brain and muscle of human. It can transform mental activities into relevant commands to control external equipment and establish direct communication pathway. Among different paradigms, steady-state visual evoked potential (SSVEP) is widely used due to its certain periodicity and stability of control. However, electroencephalogram (EEG) of SSVEP is extremely weak and companied with multi-scale and strong noise. Existing algorithms for classification are based on the principle of template matching and spatial filtering, which cannot obtain satisfied performance of feature extraction under the multi-scale noise. Especially for the subjects produce weak response for external stimuli in EEG representation, i.e., BCI-Illiteracy subject, traditional algorithms are difficult to recognize the internal patterns of brain. To address this issue, a novel method based on Chaos theory is proposed to extract feature of SSVEP. The rule of this method is applying the peculiarity of nonlinear dynamics system to detect feature of SSVEP by judging the state changes of chaotic systems after adding weak EEG. To evaluate the validity of proposed method, this research recruit 32 subjects to participate the experiment. All subjects are divided into two groups according to the preliminary classification accuracy (mean acc >70% or < 70%) by canonical correlation analysis and we define the accuracy above 70% as group A (normal subjects), below 70% as group B (BCI-Illiteracy). Then, the classification accuracy and information transmission rate of two groups are verified using Chaotic theory. Experimental results show that all classification methods using in our study achieve good performance for normal subjects while chaos obtain excellent performance and significant improvements than traditional methods for BCI-Illiteracy.

Real-time, precise, rapid and objective visual acuity assessment by self-adaptive step SSVEPs

Objective. This study aimed to explore an online, real-time, and precise method to assess steady-state visual evoked potential (SSVEP)-based visual acuity more rapidly and objectively with self-adaptive spatial frequency steps.Approach. Taking the vertical sinusoidal reversal gratings with different spatial frequencies and temporal frequencies as the visual stimuli, according to the psychometric function for visual acuity assessment, a self-adaptive procedure, the best parameter estimation by sequential testing algorithm, was used to calculate the spatial frequency sequence based on all the previous spatial frequencies and their significance of the SSVEP response. Simultaneously, the canonical correlation analysis (CCA) method with a signal-to-noise ratio (SNR) significance detection criterion was used to judge the significance of the SSVEP response.Main results.After 18 iterative trails, the spatial frequency to be presented converged to a value, which was exactly defined as the SSVEP visual acuity threshold. Our results indicated that this SSVEP acuity had a good agreement and correlation with subjective Freiburg Visual Acuity and Contrast Test acuity, and the test-retest repeatability was also good.Significance. The self-adaptive step SSVEP procedure combined with the CCA method and SNR significance detection criterion appears to be an alternative method in the real-time SSVEP acuity test to obtain objective visual acuity more rapidly and precisely.

A comfortable steady state visual evoked potential stimulation paradigm using peripheral vision

Objective. Steady-state visual evoked potential (SSVEP)-brain-computer interfaces (BCIs) can cause much visual discomfort if the users use the SSVEP-BCIs for a long time. As an alternative scheme to reduce users' visual fatigue, this study proposes a new stimulation paradigm (termed as steady state peripheral visual evoked potential, abbreviated as SSPVEP) which makes full use of peripheral vision. The electroencephalography (EEG) signals are classifiable which means this proposed stimulation paradigm can be used in BCI system with the aid of the latest hybrid signal processing approach.Approach. Under the SSPVEP stimulation paradigm, 20 targets are mounted on 20 frequencies and other targets are set between two targets with flicker stimuli coding. In order to ensure the classification accuracy of SSPVEP signal detection under the proposed stimulation paradigm, two optimization schemes are proposed for the detection stage of the conventional ensemble task-related component analysis (ETRCA) algorithm. The first optimization scheme uses nonlinear correlation coefficient at the detection part for the first time to improve the classification accuracy of the system. The second optimization scheme usesγcorrection to enhance the time domain features of the SSPVEP signals, and uses Manhattan distance for the final detection.Main results. According to the response waveforms of the EEG signals generated under the SSPVEP stimulation paradigm and the results of the questionnaire on user's comfort level to the two stimulation paradigms (SSPVEP paradigm and conventional SSVEP paradigm), the proposed stimulation paradigm brings less visual fatigue. The comparison results indicate that the proposed detection methods (ETRCA +γcorrection + Manhattan distance, ETRCA + Spearman correlation) can greatly improve the classification accuracy compared with the individual template canonical correlation analysis method and conventional ETRCA method based on Pearson correlation.Significance. The SSPVEP stimulation paradigm reduces users' visual fatigue via using peripheral vision, which provides a new design idea for SSVEP stimulation paradigm aimed at visual comfort.

Enhancing detection of steady-state visual evoked potentials using channel ensemble method

OBJECTIVE

This study proposed and evaluated a channel ensemble approach to enhance detection of steady-state visual evoked potentials (SSVEPs).

APPROACH

Collected multi-channel electroencephalogram (EEG) signals were classified into multiple groups of new analysis signals based on correlation analysis, and each group of analysis signals contained signals from a different number of electrode channels. These groups of analysis signals were used as the input of a training-free feature extraction model, and the obtained feature coefficients were converted into feature probability values using the softmax function. The ensemble value of multiple sets of feature probability values was determined and used as the final discrimination coefficient.

MAIN RESULTS

Compared with canonical correlation analysis (CCA), likelihood ratio test (LRT), and multivariate synchronization index (MSI) analysis methods using a standard approach, the recognition accuracies of the methods using a channel ensemble approach were improved by 5.05%, 3.87%, and 3.42%, and the information transfer rates (ITRs) were improved by 6.00%, 4.61%, and 3.71%, respectively. The channel ensemble method also obtained better recognition results than the standard algorithm on the public dataset. This study validated the efficiency of the proposed method to enhance the detection of SSVEPs, demonstrating its potential use in practical brain-computer interface (BCI) systems.

SIGNIFICANCE

A SSVEP-based BCI system using a channel ensemble method could achieve high ITR, indicating great potential of this design for various applications with improved control and interaction.

Effects of fatigue on steady state motion visual evoked potentials: Optimised stimulus parameters for a zoom motion-based brain-computer interface

BACKGROUND AND OBJECTIVE

In flicker-based steady-state visual evoked potentials (SSVEP) brain-computer interface (BCI), the system performance decreases due to prolonged repeated visual stimulation. To reduce the performance decrease due to visual fatigue, the zoom motion based steady-state motion visual evoked potentials (SSMVEPs) paradigm had been proposed. In this study, the stimulation parameters of the paradigm are optimised to mitigate the decrease in detection accuracy for SSMVEP due to visual fatigue.

METHODS

Eight zoom motion-based SSMVEP paradigms with different stimulation parameters were compared. The graph size, luminance, colour, and shape, as well as the frequency range and interval of the stimulation and refresh rate of the screen was changed to determine the optimal paradigm with high recognition accuracy and reduced fatigue effects. The signal-to-noise ratio (SNR) of SSMVEP was also calculated for four fatigue levels. Moreover, the power spectral density of electroencephalograph (EEG) alpha and theta bands during ongoing activity was calculated for the stimulation experiment to evaluate fatigue at the start and end of the stimulation task.

RESULTS

All stimulation SSMVEP paradigms exhibited high accuracies. Changes in luminance, colour, and shape did not impact the recognition accuracy, nor did a higher stimulation frequency or lower frequency interval of each stimulation block. However, the paradigm with smaller stimulus achieved the highest average target selection accuracy of 97.2%, compared to 94.9% for the standard paradigm. Furthermore, it exhibited almost zero reduction in recognition accuracy due to fatigue. From fatigue level 1 to level 4, the smaller zoom motion-based SSMVEP exhibited a lower decrease in the SNR of SSMVEP and a lower alpha/theta ratio decrease during ongoing stimulation activity compared to the standard paradigm.

CONCLUSIONS

For a zoom motion-based SSMVEP paradigm, changing multiple stimulation parameters can lead to the same high performance as the standard paradigm. Moreover, using a smaller stimulus can reduce the accuracy decrease caused by fatigue because the SNR decrease in the evoked SSMVEP state was negligible and the alpha/theta index decrease during ongoing activity was lower than that for the standard paradigm.

Assessment of Human Visual Acuity Using Visual Evoked Potential: A Review

Visual evoked potential (VEP) has been used as an alternative method to assess visual acuity objectively, especially in non-verbal infants and adults with low intellectual abilities or malingering. By sweeping the spatial frequency of visual stimuli and recording the corresponding VEP, VEP acuity can be defined by analyzing electroencephalography (EEG) signals. This paper presents a review on the VEP-based visual acuity assessment technique, including a brief overview of the technique, the effects of the parameters of visual stimuli, and signal acquisition and analysis of the VEP acuity test, and a summary of the current clinical applications of the technique. Finally, we discuss the current problems in this research domain and potential future work, which may enable this technique to be used more widely and quickly, deepening the VEP and even electrophysiology research on the detection and diagnosis of visual function.

Comparison of Modern Highly Interactive Flicker-Free Steady State Motion Visual Evoked Potentials for Practical Brain-Computer Interfaces

Motion-based visual evoked potentials (mVEP) is a new emerging trend in the field of steady-state visual evoked potentials (SSVEP)-based brain-computer interfaces (BCI). In this paper, we introduce different movement-based stimulus patterns (steady-state motion visual evoked potentials-SSMVEP), without employing the typical flickering. The tested movement patterns for the visual stimuli included a pendulum-like movement, a flipping illusion, a checkerboard pulsation, checkerboard inverse arc pulsations, and reverse arc rotations, all with a spelling task consisting of 18 trials. In an online experiment with nine participants, the movement-based BCI systems were evaluated with an online four-target BCI-speller, in which each letter may be selected in three steps (three trials). For classification, the minimum energy combination and a filter bank approach were used. The following frequencies were utilized: 7.06 Hz, 7.50 Hz, 8.00 Hz, and 8.57 Hz, reaching an average accuracy between 97.22% and 100% and an average information transfer rate (ITR) between 15.42 bits/min and 33.92 bits/min. All participants successfully used the SSMVEP-based speller with all types of stimulation pattern. The most successful SSMVEP stimulus was the SSMVEP1 (pendulum-like movement), with the average results reaching 100% accuracy and 33.92 bits/min for the ITR.

Anti-fatigue Performance in SSVEP-Based Visual Acuity Assessment: A Comparison of Six Stimulus Paradigms

Purpose

The occurrence of mental fatigue when users stare at stimuli is a critical problem in the implementation of steady-state visual evoked potential (SSVEP)-based visual acuity assessment, which may weaken the SSVEP amplitude and signal-to-noise ratio (SNR) and subsequently affect the results of visual acuity assessment. This study aimed to explore the anti-fatigue performance of six stimulus paradigms (reverse vertical sinusoidal gratings, reverse horizontal sinusoidal gratings, reverse vertical square-wave gratings, brief-onset vertical sinusoidal gratings, reversal checkerboards, and oscillating expansion–contraction concentric rings) in SSVEP acuity assessment.

Methods

Based on four indices of α + θ index, pupil diameter, National Aeronautics and Space Administration Task Load Index (NASA-TLX), and amplitude and SNR of SSVEPs, this study quantitatively evaluated mental fatigue in six SSVEP visual attention runs corresponding to six paradigms with 12 subjects.

Results

These indices of mental fatigue showed a good agreement. The results showed that the paradigm of motion expansion–contraction concentric rings had a superior anti-fatigue efficacy than the other five paradigms of conventional onset mode or pattern reversal mode during prolonged SSVEP experiment. The paradigm of brief-onset mode showed the lowest anti-fatigue efficacy, and the other paradigms of pattern reversal SSVEP paradigms showed a similar anti-fatigue efficacy, which was between motion expansion–contraction mode and onset mode.

Conclusion

This study recommended the paradigm of oscillating expansion–contraction concentric rings as the stimulation paradigm in SSVEP visual acuity because of its superior anti-fatigue efficacy.

Brain-computer interface method based on light-flashing and motion hybrid coding

The human best response frequency band for steady-state visual evoked potential stimulus is limited. This results in a reduced number of encoded targets. To circumvent this, we proposed a brain-computer interface (BCI) method based on light-flashing and motion hybrid coding. The hybrid paradigm pattern consisted of a circular light-flashing pattern and a motion pattern located in the inner ring of light-flashing pattern. The motion and light-flashing patterns had different frequencies. This study used five frequencies to encode nine targets. The motion frequency and the light-flashing frequency of the hybrid paradigm consisted of two frequencies in five frequencies. The experimental results showed that the hybrid paradigm could induce stable motion frequency, light-flashing frequency and its harmonic components. Moreover, the modulation between motion and light-flashing was weak. The average accuracy was 92.96% and the information transfer rate was 26.10 bits/min. The experimental results showed that the proposed method could be considered for practical BCI systems.

A multi-source co-frequency stimulus method for electroencephalogram (EEG) enhancement

The electroencephalogram (EEG) induced by steady-state visual evoked potential (SSVEP) will contain background noise. Most existing research on this problem uses signal-processing methods to enhance the EEG. The purpose of this paper is to explore another method that can be used to enhance the EEG. We creatively combined motion stimuli with light-flashing stimuli and designed a paradigm in which motion and light-flashing simultaneously will stimulate with the same frequency; this is called multi-source co-frequency stimulus. To avoid the direct stimulus of light-flashing in the human eye and ensure that the composite paradigm provided adequate comfort, the light-flashing pattern was presented in a ring form and the motion stimulus was presented in the center of that ring. Our hypothesis is that when the motion and the light-flashing are simultaneously stimulated with the same frequency, the EEG they induce will be superimposed in some way, and this will enhance the EEG. The multi-source co-frequency stimulus was found to achieve a higher signal-to-noise ratio (SNR), better accuracy, and a higher information transmission rate (ITR) than single stimulus. The experimental results showed that it is feasible to use the method proposed in this study to enhance the EEG.

Comparison of the performance of six stimulus paradigms in visual acuity assessment based on steady-state visual evoked potentials

PURPOSE

There are several stimulus paradigms used in objective visual acuity assessment based on steady-state visual evoked potentials (SSVEPs). The aim of this study was to explore the difference and performance of common used six stimulus paradigms (reverse vertical sinusoidal gratings, reverse horizontal sinusoidal gratings, reverse vertical square-wave gratings, brief-onset vertical sinusoidal gratings, reversal checkerboards and oscillating expansion-contraction concentric-rings) of SSVEP acuity assessment.

METHODS

We tested subjective visual acuity both by tumbling E and Freiburg Visual Acuity and Contrast Test (FrACT) in 11 subjects. SSVEPs were induced by 11 spatial frequencies for each paradigm, and then a threshold determination criterion was used to define the objective SSVEP visual acuity.

RESULTS

After SSVEP signal analysis, we found there was difference in SSVEP response of harmonic components and no difference in sensitive electrode placement for the six paradigms. We selected six electrodes (PO3, POz, PO4, O1, Oz and O2) as the sensitive electrodes to use in data processing for each paradigm. The results showed that except for brief-onset vertical sinusoidal gratings, the correlation and agreement between objective SSVEP and subjective FrACT acuity were all quite good, demonstrating good performance in acuity detection for the rest five paradigms.

CONCLUSION

Except for brief-onset vertical sinusoidal gratings, all the five stimulus paradigms of reverse vertical sinusoidal gratings, reverse horizontal sinusoidal gratings, reverse vertical square-wave gratings, reversal checkerboards and oscillating expansion-contraction concentric-rings performed quite well in objective SSVEP visual acuity assessment.

Can a highly accurate multi-class SSMVEP BCI induce sensory-motor rhythm in sensorimotor area?

Different visual stimuli might have different effects on the brain, e.g. the change of brightness, non-biological movement and biological movement. In this study, flicker, checkerboard, and gaiting stimuli were chosen as visual stimuli to investigate whether steady-state motion visual evoked potential (SSMVEP) effected on the sensorimotor area for rehabilitation. The gaiting stimulus was designed as the gaiting sequence of a human. The hypothesis is that only observing the designed gaiting stimulus would simultaneously induce 1) SSMVEP in the occipital area, similarly to an SSVEP stimulus; and 2) sensorimotor rhythm (SMR) in the primary sensorimotor area, because such action observation could activate the mirror neuron system. Canonical correlation analysis was used to detect SSMVEP from occipital EEG, and event-related spectral perturbation was used to identify SMR in the EEG from the sensorimotor area. The results showed that the designed gaiting stimulus-induced SSMVEP, with classification accuracies of 88.9 ± 12.0% in a four-class scenario. More importantly, it induced clear and sustained event-related desynchronization/synchronization (ERD/ERS), while no ERD/ERS could be observed when the other two SSVEP stimuli were used. Further, for participants with a sufficiently clear SSMVEP pattern (classification accuracy > 85%), the ERD index values in the mu-beta band induced by the proposed gaiting stimulus were statistically different from that of the other two types of stimulus. Therefore, a novel BCI based on the designed stimulus has potential in neurorehabilitation applications because it simultaneously has the high accuracy of an SSMVEP (~90% accuracy in a four-class setup) and the ability to activate the sensorimotor area.

Characterization of SSMVEP-based EEG signals using multiplex limited penetrable horizontal visibility graph

The steady state motion visual evoked potential (SSMVEP)-based brain computer interface (BCI), which incorporates the motion perception capabilities of the human visual system to alleviate the negative effects caused by strong visual stimulation from steady-state VEP, has attracted a great deal of attention. In this paper, we design a SSMVEP-based experiment by Newton's ring paradigm. Then, we use the canonical correlation analysis and Support Vector Machines to classify SSMVEP signals for the SSMVEP-based electroencephalography (EEG) signal detection. We find that the classification accuracy of different subjects under fatigue state is much lower than that in the normal state. To probe into this, we develop a multiplex limited penetrable horizontal visibility graph method, which enables to infer a brain network from 62-channel EEG signals. Subsequently, we analyze the variation of the average weighted clustering coefficient and the weighted global efficiency corresponding to these two brain states and find that both network measures are lower under fatigue state. The results suggest that the associations and information transfer efficiency among different brain regions become weaker when the brain state changes from normal to fatigue, which provide new insights into the explanations for the reduced classification accuracy. The promising classification results and the findings render the proposed methods particularly useful for analyzing EEG recordings from SSMVEP-based BCI system.

Objective and quantitative assessment of visual acuity and contrast sensitivity based on steady-state motion visual evoked potentials using concentric-ring paradigm

PURPOSE

The traditional assessment of visual acuity and contrast sensitivity depends more on subjective judgments. Steady-state motion visual evoked potentials (SSMVEPs) can provide an objective and quantitative method to evaluate visual functions such as visual acuity and contrast sensitivity. Here, we explored the possibility of objective SSMVEP visual acuity and contrast sensitivity testing, and compared its performance with that of psychophysical methods.

METHODS

In this study, we designed a specific concentric ring with oscillating expansion and contraction SSMVEP paradigm to assess visual acuity and contrast sensitivity. By changing the parameters of the paradigm, the SSMVEP paradigm with different contrasts and spatial frequencies corresponding to different visual acuity and contrast sensitivity was designed. Moreover, we proposed a threshold determination criterion to define the corresponding objective SSMVEP visual acuity and contrast sensitivity.

RESULTS

We tested visual acuity and contrast sensitivity of sixteen healthy adults utilizing this paradigm with an electroencephalography system. Our data suggested that there was no significant difference between objective visual acuity and contrast sensitivity measurements based on the SSMVEPs and subjective psychophysical ones.

CONCLUSION

Our study proved that SSMVEPs can be an objective and quantitative method to measure visual acuity and contrast sensitivity.

Steady-State Motion Visual Evoked Potential (SSMVEP) Enhancement Method Based on Time-Frequency Image Fusion

The steady-state motion visual evoked potential (SSMVEP) collected from the scalp suffers from strong noise and is contaminated by artifacts such as the electrooculogram (EOG) and the electromyogram (EMG). Spatial filtering methods can fuse the information of different brain regions, which is beneficial for the enhancement of the active components of the SSMVEP. Traditional spatial filtering methods fuse electroencephalogram (EEG) in the time domain. Based on the idea of image fusion, this study proposed an SSMVEP enhancement method based on time-frequency (T-F) image fusion. The purpose is to fuse SSMVEP in the T-F domain and improve the enhancement effect of the traditional spatial filtering method on SSMVEP active components. Firstly, two electrode signals were transformed from the time domain to the T-F domain via short-time Fourier transform (STFT). The transformed T-F signals can be regarded as T-F images. Then, two T-F images were decomposed via two-dimensional multiscale wavelet decomposition, and both the high-frequency coefficients and low-frequency coefficients of the wavelet were fused by specific fusion rules. The two images were fused into one image via two-dimensional wavelet reconstruction. The fused image was subjected to mean filtering, and finally, the fused time-domain signal was obtained by inverse STFT (ISTFT). The experimental results show that the proposed method has better enhancement effect on SSMVEP active components than the traditional spatial filtering methods. This study indicates that it is feasible to fuse SSMVEP in the T-F domain, which provides a new idea for SSMVEP analysis.

A Convolutional Neural Network for the Detection of Asynchronous Steady State Motion Visual Evoked Potential

A key issue in brain-computer interface (BCI) is the detection of intentional control (IC) states and non-intentional control (NC) states in an asynchronous manner. Furthermore, for steady-state visual evoked potential (SSVEP) BCI systems, multiple states (sub-states) exist within the IC state. Existing recognition methods rely on a threshold technique, which is difficult to realize high accuracy, i.e., simultaneously high true positive rate and low false positive rate. To address this issue, we proposed a novel convolutional neural network (CNN) to detect IC and NC states in a SSVEP-BCI system for the first time. Specifically, the steady-state motion visual evoked potentials (SSMVEP) paradigm, which has been shown to induce less visual discomfort, was chosen as the experimental paradigm. Two processing pipelines were proposed for the detection of IC and NC states. The first one was using CNN as a multi-class classifier to discriminate between all the states in IC and NC state (FFT-CNN). The second one was using CNN to discriminate between IC and NC states, and using canonical correlation analysis (CCA) to perform classification tasks within the IC (FFT-CNN-CCA). We demonstrated that both pipelines achieved a significant increase in accuracy for low-performance healthy participants when traditional algorithms such as CCA threshold were used. Furthermore, the FFT-CNN-CCA pipeline achieved better performance than the FFT-CNN pipeline based on the stroke patients' data. In summary, we showed that CNN can be used for robust detection in an asynchronous SSMVEP-BCI with great potential for out-of-lab BCI applications.

A Radial Zoom Motion-Based Paradigm for Steady State Motion Visual Evoked Potentials

Background: In steady state visual evoked potential (SSVEP)-based brain-computer interfaces, prolonged repeated flicker stimulation would reduce the system performance. To reduce the visual discomfort and fatigue, while ensuring recognition accuracy, and information transmission rate (ITR), a novel motion paradigm based on the steady-state motion visual evoked potentials (SSMVEPs) is proposed.

Methods: The novel SSMVEP paradigm of the radial zoom motion was realized using the sinusoidal form to modulate the size of the stimuli. The radial zoom motion-based SSMVEP paradigm was compared with the flicker-based SSVEP paradigm and the SSMVEP paradigm based on Newton's ring motion. The canonical correlation analysis was used to identify the frequency of the eight targets, the recognition accuracy of different paradigms with different stimulation frequencies, and the ITR under different stimulation durations were calculated. The subjective comfort scores and fatigue scores, and decrease in the accuracy due to fatigue was evaluated.

Results: The average recognition accuracy of the novel radial zoom motion-based SSMVEP paradigm was 93.4%, and its ITR reached 42.5 bit/min, which was greater than the average recognition accuracy of the SSMVEP paradigm based on Newton's ring motion. The comfort score of the novel paradigm was greater than both the flicker-based SSVEP paradigm and SSMVEP paradigm based on Newton's ring motion. The decrease in the recognition accuracy due to fatigue was less than that of the SSSMVEP paradigm based on Newton's ring motion.

Conclusion: The SSMVEP paradigm based on radial zoom motion has high recognition accuracy and ITR with low visual discomfort and fatigue scores. The method has potential advantages in overcoming the performance decline caused by fatigue.

A Light Spot Humanoid Motion Paradigm Modulated by the Change of Brightness to Recognize the Stride Motion Frequency

The steady-state visual evoked potential (SSVEP)-based brain–computer interface (BCI) usually has the advantages of high information transfer rate (ITR) and no need for training. However, low frequencies, such as the human stride motion frequency, cannot easily induce SSVEP. To solve this problem, a light spot humanoid motion paradigm modulated by the change of brightness was designed in this study. The characteristics of the brain response to the motion paradigm modulated by the change of brightness were analyzed for the first time. The results showed that the designed paradigm could induce not only the high flicker frequency but also the modulation frequencies between the change of brightness and the motion in the primary visual cortex. Thus, the stride motion frequency can be recognized through the modulation frequencies by using the designed paradigm. Also, in an online experiment, this paradigm was employed to control a lower limb robot to achieve same frequency stimulation, which meant that the visual stimulation frequency was the same as the motion frequency of the robot. Also, canonical correlation analysis (CCA) was used to distinguish three different stride motion frequencies. The average accuracies of the classification in three walking speeds using the designed paradigm with the same and different high frequencies reached 87 and 95% respectively. Furthermore, the angles of the knee joint of the robot were obtained to demonstrate the feasibility of the electroencephalograph (EEG)-driven robot with same stimulation.