Enhancing Detection of SSVEPs for a High-Speed Brain Speller Using Task-Related Component Analysis

Decoding motor imagery loaded on steady-state somatosensory evoked potential based on complex task-related component analysis

BACKGROUND AND OBJECTIVE

Motor Imagery (MI) recognition is one of the most critical decoding problems in brain- computer interface field. Combined with the steady-state somatosensory evoked potential (MI-SSSEP), this new paradigm can achieve higher recognition accuracy than the traditional MI paradigm. Typical algorithms do not fully consider the characteristics of MI-SSSEP signals. Developing an algorithm that fully captures the paradigm's characteristics to reduce false triggering rate is the new step in improving performance.

METHODS

The idea to use complex signal task-related component analysis (cTRCA) algorithm for spatial filtering processing has been proposed in this paper according to the features of SSSEP signal. In this research, it's proved from the analysis of simulation signals that task-related component analysis (TRCA) as typical method is affected when the response between stimuli has reduced correlation and the proposed algorithm can effectively overcome this problem. The experimental data under the MI-SSSEP paradigm have been used to identify right-handed target tasks and three unique interference tasks are used to test the false triggering rate. cTRCA demonstrates superior performance as confirmed by the Wilcoxon signed-rank test.

RESULTS

The recognition algorithm of cTRCA combined with mutual information-based best individual feature (MIBIF) and minimum distance to mean (MDM) can obtain AUC value up to 0.89, which is much higher than traditional algorithm common spatial pattern (CSP) combined with support vector machine (SVM) (the average AUC value is 0.77, p < 0.05). Compared to CSP+SVM, this algorithm model reduced the false triggering rate from 38.69 % to 20.74 % (p < 0.001).

CONCLUSIONS

The research prove that TRCA is influenced by MI-SSSEP signals. The results further prove that the motor imagery task in the new paradigm MI-SSSEP causes the phase change in evoked potential. and the cTRCA algorithm based on such phase change is more suitable for this hybrid paradigm and more conducive to decoding the motor imagery task and reducing false triggering rate.

OS-SSVEP: One-shot SSVEP classification

It is extremely challenging to classify steady-state visual evoked potentials (SSVEPs) in scenarios characterized by a huge scarcity of calibration data where only one calibration trial is available for each stimulus target. To address this challenge, we introduce a novel approach named OS-SSVEP, which combines a dual domain cross-subject fusion network (CSDuDoFN) with the task-related and task-discriminant component analysis (TRCA and TDCA) based on data augmentation. The CSDuDoFN framework is designed to comprehensively transfer information from source subjects, while TRCA and TDCA are employed to exploit the information from the single available calibration trial of the target subject. Specifically, CSDuDoFN uses multi-reference least-squares transformation (MLST) to map data from both the source subjects and the target subject into the domain of sine-cosine templates, thereby reducing cross-subject domain gap and benefiting transfer learning. In addition, CSDuDoFN is fed with both transformed and original data, with an adequate fusion of their features occurring at different network layers. To capitalize on the calibration trial of the target subject, OS-SSVEP utilizes source aliasing matrix estimation (SAME)-based data augmentation to incorporate into the training process of the ensemble TRCA (eTRCA) and TDCA models. Ultimately, the outputs of CSDuDoFN, eTRCA, and TDCA are combined for the SSVEP classification. The effectiveness of our proposed approach is comprehensively evaluated on three publicly available SSVEP datasets, achieving the best performance on two datasets and competitive performance on the third. Further, it is worth noting that our method follows a different technical route from the current state-of-the-art (SOTA) method and the two are complementary. The performance is significantly improved when our method is combined with the SOTA method. This study underscores the potential to integrate the SSVEP-based brain-computer interface (BCI) into daily life. The corresponding source code is accessible at https://github.com/Sungden/One-shot-SSVEP-classification.

Task-oriented EEG denoising generative adversarial network for enhancing SSVEP-BCI performance

Objective.The quality of electroencephalogram (EEG) signals directly impacts the performance of brain-computer interface (BCI) tasks. Many methods have been proposed to eliminate noise from EEG signals, but most of these methods focus solely on signal denoising itself, disregarding the impact on subsequent tasks, which deviates from the original intention of EEG denoising. The main objective of this study is to optimize EEG denoising models with a purpose of improving the performance of BCI tasks.Approach.To this end, we proposed an innovative task-oriented EEG denoising generative adversarial network (TOED-GAN) method. This network utilizes the generator of GAN to decompose and reconstruct clean signals from the raw EEG signals, and the discriminator to learn to distinguish the generated signals from the true clean signals, resulting in a remarkable increase of the signal-to-noise ratio by simultaneously enhancing task-related components and removing task-irrelevant noise from the original contaminated signals.Main results.We evaluated the performance of the model on a public dataset and a self-collected dataset respectively, with canonical correlation analysis classification tasks of the steady-state visual evoked potential (SSVEP) based BCI. Experimental results demonstrate that TOED-GAN exhibits excellent performance in removing EEG noise and improving performance for SSVEP-BCI, with accuracy improvement rates reaching 18.47% and 21.33% in contrast to the baseline methods of convolutional neural networks, respectively.Significance.This work proves that the proposed TOED-GAN, as an EEG denoising method tailored for SSVEP tasks, contributes to enhancing the performance of BCIs in practical application scenarios.

Filter banks guided correlational convolutional neural network for SSVEPs based BCI classification

Objective.In the field of steady-state visual evoked potential brain computer interfaces (SSVEP-BCIs) research, convolutional neural networks (CNNs) have gradually been proved to be an effective method. Whereas, majority works apply the frequency domain characteristics in long time window to train the network, thus lead to insufficient performance of those networks in short time window. Furthermore, only the frequency domain information for classification lacks of other task-related information.Approach.To address these issues, we propose a time-frequency domain generalized filter-bank convolutional neural network (FBCNN-G) to improve the SSVEP-BCIs classification performance. The network integrates multiple frequency information of electroencephalogram (EEG) with template and predefined prior of sine-cosine signals to perform feature extraction, which contains correlation analyses in both template and signal aspects. Then the classification is performed at the end of the network. In addition, the method proposes the use of filter banks divided into specific frequency bands as pre-filters in the network to fully consider the fundamental and harmonic frequency characteristics of the signal.Main results.The proposed FBCNN-G model is compared with other methods on the public dataset Benchmark. The results manifest that this model has higher accuracy of character recognition accuracy and information transfer rates in several time windows. Particularly, in the 0.2 s time window, the mean accuracy of the proposed method reaches62.02%±5.12%, indicating its superior performance.Significance.The proposed FBCNN-G model is critical for the exploitation of SSVEP-BCIs character recognition models.

Using the Cocktail Party Effect to Add the Coding Dimension of Auditory Event Related Potential Brain-Computer Interface

OBJECTIVE

The auditory event-related potential based brain-computer interface (aERP-BCI) is a classical paradigm of brain-computer communication. To improve the coding efficiency of aERP-BCI, this study proposes a method using two parallel voice channels to add the coding dimension based on the cocktail party effect.

METHODS

The novel paradigm used male and female voices to establish two parallel oddball sound stimulus sequences. In comparison, the baseline paradigm only presented male or female stimulus sequences. Both the double voice condition (DVC) and the single voice condition (SVC) paradigms carried out offline experiments and the DVC also carried out online experiment. Subsequently, the EEG signal and BCI operation results were compared and analyzed.

CONCLUSION

The cocktail party effect caused a significant difference in the EEG responses of non-target stimulus between the focused vocal channel and the ignored vocal channel under the DVC paradigm, and the focused and ignored channels achieved a recognition accuracy of 97.2%. The target recognition rate of DVC was 82.3%, with no significant difference compared with 85% of SVC while the information transfer rate (ITR) of DVC reaching 15.3 bits/min was significantly higher than that of SVC.

SIGNIFICANCE

The cocktail party effect improves the coding efficiency by adding parallel channels without reducing the target/non-target stimulus recognition in the focused vocal channel. This provides a novel direction for the performance improvement of aERP-BCI.

Multiclass motor imagery classification with Riemannian geometry and temporal-spectral selection

Motor imagery (MI) based brain-computer interfaces (BCIs) decode the users' intentions from electroencephalography (EEG) to achieve information control and interaction between the brain and external devices. In this paper, firstly, we apply Riemannian geometry to the covariance matrix extracted by spatial filtering to obtain robust and distinct features. Then, a multiscale temporal-spectral segmentation scheme is developed to enrich the feature dimensionality. In order to determine the optimal feature configurations, we utilize a linear learning-based temporal window and spectral band (TWSB) selection method to evaluate the feature contributions, which efficiently reduces the redundant features and improves the decoding efficiency without excessive loss of accuracy. Finally, support vector machines are used to predict the classification labels based on the selected MI features. To evaluate the performance of our model, we test it on the publicly available BCI Competition IV dataset 2a and 2b. The results show that the method has an average accuracy of 79.1% and 83.1%, which outperforms other existing methods. Using TWSB feature selection instead of selecting all features improves the accuracy by up to about 6%. Moreover, the TWSB selection method can effectively reduce the computational burden. We believe that the framework reveals more interpretable feature information of motor imagery EEG signals, provides neural responses discriminative with high accuracy, and facilitates the performance of real-time MI-BCI.

Filter bank temporally delayed CCA for uncalibrated SSVEP-BCI

The uncalibrated brain-computer interface (BCI) system based on steady-state visual evoked potential (SSVEP) can omit the training process and is closer to the practical application. Filter bank canonical correlation analysis (FBCCA), as a classical approach of uncalibrated SSVEP-based BCI, extracts the fundamental and harmonic ingredients through filter bank decomposition. Nevertheless, this method fails to fully leverage the temporal feature of the signal. The paper suggested utilizing reconstructed data with temporal delay in the computation of the canonical correlation coefficient, and the different combinations of the time-delayed embedding and FBCCA were discussed. We selected the data from seven participants in the Benchmark dataset for parameter optimization and evaluated the method across all participants. The experimental results showed that only embedding the time-delayed version into the first subband (FBdCCA) was better than embedding it into all subbands (FBdCCA(all)), and the accuracy of FBdCCA surpassed that of FBCCA significantly. This suggests that the approach of time-delayed embedding can further enhance the performance of FBCCA.

Binocularly incongruent, multifrequency-coded SSVEP in VR: feasibility and characteristics

Objective.Steady-state visual evoked potentials (SSVEPs) in response to flickering stimuli are popular in brain-computer interfacing but their implementation in virtual reality (VR) offers new opportunities also for clinical applications. While traditional SSVEP target selection relies on single-frequency stimulation of both eyes simultaneously, further called congruent stimulation, recent studies attempted to improve the information transfer rate by using dual-frequency-coded SSVEP where each eye is presented with a stimulus flickering at a different frequency, further called incongruent stimulation. However, few studies have investigated incongruent multifrequency-coded SSVEP (MultiIncong-SSVEP).Approach.This paper reports on a systematical investigation of incongruent dual-, triple-, and quadruple-frequency-coded SSVEP for use in VR, several of which are entirely novel, and compares their performance with that of congruent dual-frequency-coded SSVEP.Main results.We were able to confirm the presence of a summation effect when comparing monocular- and binocular single-frequency congruent stimulation, and a suppression effect when comparing monocular- and binocular dual-frequency incongruent stimulation, as both tap into the binocular vision capabilities which, when hampered, could signal amblyopia.Significance.In sum, our findings not only evidence the potential of VR-based binocularly incongruent SSVEP but also underscore the importance of paradigm choice and decoder design to optimize system performance and user comfort.

An improved SSVEP-based brain-computer interface with low-contrast visual stimulation and its application in UAV control

Efficient communication and regulation are crucial for advancing brain-computer interfaces (BCIs), with the steady-state visual-evoked potential (SSVEP) paradigm demonstrating high accuracy and information transfer rates. However, the conventional SSVEP paradigm encounters challenges related to visual occlusion and fatigue. In this study, we propose an improved SSVEP paradigm that addresses these issues by lowering the contrast of visual stimulation. The improved paradigms outperform the traditional paradigm in the experiments, significantly reducing the visual stimulation of the SSVEP paradigm. Furthermore, we apply this enhanced paradigm to a BCI navigation system, enabling two-dimensional navigation of unmanned aerial vehicles (UAVs) through a first-person perspective. Experimental results indicate the enhanced SSVEP-based BCI system's accuracy in performing navigation and search tasks. Our findings highlight the feasibility of the enhanced SSVEP paradigm in mitigating visual occlusion and fatigue issues, presenting a more intuitive and natural approach for BCIs to control external equipment.NEW & NOTEWORTHY In this article, we proposed an improved steady-state visual-evoked potential (SSVEP) paradigm and constructed an SSVEP-based brain-computer interface (BCI) system to navigate the unmanned aerial vehicle (UAV) in two-dimensional (2-D) physical space. We proposed a modified method for evaluating visual fatigue including subjective score and objective indices. The results indicated that the improved SSVEP paradigm could effectively reduce visual fatigue while maintaining high accuracy.

Comparative Study of Frequency Recognition Techniques for Steady-State Visual Evoked Potentials According to the Frequency Harmonics and Stimulus Number

Background

A key challenge in steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) systems is to effectively recognize frequencies within a short time window. To address this challenge, the specific characteristics of the data are needed to select the frequency recognition method. These characteristics include factors, such as the number of stimulation targets and the presence of harmonic frequencies, resulting in optimizing the performance and accuracy of SSVEP-based BCI systems.

Objective

The current study aimed to examine the effect of data characteristics on frequency recognition accuracy.

Material and Methods

In this analytical study, five commonly used frequency recognition methods were examined, used to various datasets containing different numbers of frequencies, including sub-data with and without frequency harmonics.

Results

The increase in the number of frequencies in the Multivariate Linear Regression (MLR) method has led to a decrease in frequency recognition accuracy by 9%. Additionally, the presence of harmonic frequencies resulted in an 8% decrease in accuracy for the MLR method.

Conclusion

Frequency recognition using the MLR method reduces the effect of the number of different frequencies and harmonics of the stimulation frequencies on the frequency recognition accuracy.

Time-frequency feature extraction based on multivariable synchronization index for training-free SSVEP-based BCI

Multivariate synchronization index (MSI), as an effective recognition algorithm for steady-state visual evoked potential (SSVEP) brain-computer interface (BCI), can accurately decode target frequencies without training. To further consider temporal features or extract harmonic components, extended MSI (EMSI), temporally local MSI (TMSI), and filter bank MSI (FBMSI) have been proposed. However, the promotion effects of the above three strategies on MSI have not been compared in detail. In this paper, the performance of EMSI, TMSI, and FBMSI under different time windows was analyzed with the same dataset. The results indicated that the improvement effect of the temporally local method on MSI was better than that of the other two methods under the short time window, and the effect of the filter bank method was better when the time window was greater than 0.8 s. Based on the idea of simultaneously extracting time-frequency features, FBEMSI and FBTMSI were proposed by integrating time delay embedding and temporally local method into FBMSI respectively. The two improved methods, which has no significant difference, can improve the recognition effect of FBMSI. But the computing time of FBEMSI was shorter, which can be a potential method for SSVEP-BCI.

Research on a soft saturation nonlinear SSVEP signal feature extraction algorithm

Brain-computer interfaces (BCIs) based on steady-state visual evoked potentials (SSVEP) have received widespread attention due to their high information transmission rate, high accuracy, and rich instruction set. However, the performance of its identification methods strongly depends on the amount of calibration data for within-subject classification. Some studies use deep learning (DL) algorithms for inter-subject classification, which can reduce the calculation process, but there is still much room for improvement in performance compared with intra-subject classification. To solve these problems, an efficient SSVEP signal recognition deep learning network model e-SSVEPNet based on the soft saturation nonlinear module is proposed in this paper. The soft saturation nonlinear module uses a similar exponential calculation method for output when it is less than zero, improving robustness to noise. Under the conditions of the SSVEP data set, two sliding time window lengths (1 s and 0.5 s), and three training data sizes, this paper evaluates the proposed network model and compares it with other traditional and deep learning model baseline methods. The experimental results of the nonlinear module were classified and compared. A large number of experimental results show that the proposed network has the highest average accuracy of intra-subject classification on the SSVEP data set, improves the performance of SSVEP signal classification and recognition, and has higher decoding accuracy under short signals, so it has huge potential ability to realize high-speed SSVEP-based for BCI.

Multiband task related components enhance rapid cognition decoding for both small and similar objects

The cortically-coupled target recognition system based on rapid serial visual presentation (RSVP) has a wide range of applications in brain computer interface (BCI) fields such as medical and military. However, in the complex natural environment backgrounds, the identification of event-related potentials (ERP) of both small and similar objects that are quickly presented is a research challenge. Therefore, we designed corresponding experimental paradigms and proposed a multi-band task related components matching (MTRCM) method to improve the rapid cognitive decoding of both small and similar objects. We compared the areas under the receiver operating characteristic curve (AUC) between MTRCM and other 9 methods under different numbers of training sample using RSVP-ERP data from 50 subjects. The results showed that MTRCM maintained an overall superiority and achieved the highest average AUC (0.6562 ± 0.0091). We also optimized the frequency band and the time parameters of the method. The verification on public data sets further showed the necessity of designing MTRCM method. The MTRCM method provides a new approach for neural decoding of both small and similar RSVP objects, which is conducive to promote the further development of RSVP-BCI.

A high-performance general computer cursor control scheme based on a hybrid BCI combining motor imagery and eye-tracking

This study introduces a novel virtual cursor control system designed to empower individuals with neuromuscular disabilities in the digital world. By combining eye-tracking with motor imagery (MI) in a hybrid brain-computer interface (BCI), the system enhances cursor control accuracy and simplicity. Real-time classification accuracy reaches 87.92% (peak of 93.33%), with cursor stability in the gazing state at 96.1%. Integrated into common operating systems, it enables tasks like text entry, online chatting, email, web surfing, and picture dragging, with an average text input rate of 53.2 characters per minute (CPM). This technology facilitates fundamental computing tasks for patients, fostering their integration into the online community and paving the way for future developments in BCI systems.

Narrow band-pass filtered canonical correlation analysis for frequency identification in SSVEP signals

Steady-state visual evoked potentials (SSVEP) are generated in the parieto-occipital regions, accompanied by background noise and artifacts. A strong pre-processing method is required to reduce this background noise and artifacts. This study proposed a narrow band-pass filtered canonical correlation analysis (NBPFCCA) to recognize frequency components in SSVEP signals. The proposed method is tested on the publicly available 40 stimulus frequencies dataset recorded from 35 subjects and 4 class in-house dataset acquired from 10 subjects. The performance of the proposed NBPFCCA method is compared with the standard canonical correlation analysis (CCA) and the filter bank CCA (FBCCA). The mean frequency detection accuracy of the standard CCA is 86.21% for the benchmark dataset, and it is improved to 95.58% in the proposed method. Results indicate that the proposed method significantly outperforms the standard canonical correlation analysis with an increase of 9.37% and 17% in frequency recognition accuracy of the benchmark and in-house datasets, respectively.

A Dynamic Window Method Based on Reinforcement Learning for SSVEP Recognition

Steady-state visual evoked potential (SSVEP) is one of the most used brain-computer interface (BCI) paradigms. Conventional methods analyze SSVEPs at a fixed window length. Compared with these methods, dynamic window methods can achieve a higher information transfer rate (ITR) by selecting an appropriate window length. These methods dynamically evaluate the credibility of the result by linear discriminant analysis (LDA) or Bayesian estimation and extend the window length until credible results are obtained. However, the hypotheses introduced by LDA and Bayesian estimation may not align with the collected real-world SSVEPs, which leads to an inappropriate window length. To address the issue, we propose a novel dynamic window method based on reinforcement learning (RL). The proposed method optimizes the decision of whether to extend the window length based on the impact of decisions on the ITR, without additional hypotheses. The decision model can automatically learn a strategy that maximizes the ITR through trial and error. In addition, compared with traditional methods that manually extract features, the proposed method uses neural networks to automatically extract features for the dynamic selection of window length. Therefore, the proposed method can more accurately decide whether to extend the window length and select an appropriate window length. To verify the performance, we compared the novel method with other dynamic window methods on two public SSVEP datasets. The experimental results demonstrate that the novel method achieves the highest performance by using RL.

SSVEP-DAN: Cross-Domain Data Alignment for SSVEP-Based Brain-Computer Interfaces

Steady-state visual-evoked potential (SSVEP)-based brain-computer interfaces (BCIs) offer a non-invasive means of communication through high-speed speller systems. However, their efficiency is highly dependent on individual training data acquired during time-consuming calibration sessions. To address the challenge of data insufficiency in SSVEP-based BCIs, we introduce SSVEP-DAN, the first dedicated neural network model designed to align SSVEP data across different domains, encompassing various sessions, subjects, or devices. Our experimental results demonstrate the ability of SSVEP-DAN to transform existing source SSVEP data into supplementary calibration data. This results in a significant improvement in SSVEP decoding accuracy while reducing the calibration time. We envision SSVEP-DAN playing a crucial role in future applications of high-performance SSVEP-based BCIs. The source code for this work is available at: https://github.com/CECNL/SSVEP-DAN.

An SSVEP-based BCI with 112 targets using frequency spatial multiplexing

Objective.Brain-computer interface (BCI) systems with large directly accessible instruction sets are one of the difficulties in BCI research. Research to achieve high target resolution (⩾100) has not yet entered a rapid development stage, which contradicts the application requirements. Steady-state visual evoked potential (SSVEP) based BCIs have an advantage in terms of the number of targets, but the competitive mechanism between the target stimulus and its neighboring stimuli is a key challenge that prevents the target resolution from being improved significantly.Approach.In this paper, we reverse the competitive mechanism and propose a frequency spatial multiplexing method to produce more targets with limited frequencies. In the proposed paradigm, we replicated each flicker stimulus as a 2 × 2 matrix and arrange the matrices of all frequencies in a tiled fashion to form the interaction interface. With different arrangements, we designed and tested three example paradigms with different layouts. Further we designed a graph neural network that distinguishes between targets of the same frequency by recognizing the different electroencephalography (EEG) response distribution patterns evoked by each target and its neighboring targets.Main results.Extensive experiment studies employing eleven subjects have been performed to verify the validity of the proposed method. The average classification accuracies in the offline validation experiments for the three paradigms are 89.16%, 91.38%, and 87.90%, with information transfer rates (ITR) of 51.66, 53.96, and 50.55 bits/min, respectively.Significance.This study utilized the positional relationship between stimuli and did not circumvent the competing response problem. Therefore, other state-of-the-art methods focusing on enhancing the efficiency of SSVEP detection can be used as a basis for the present method to achieve very promising improvements.

High-Frequency Discrete-Interval Binary Sequence in Asynchronous C-VEP-Based BCI for Visual Fatigue Reduction

In code-modulated visual evoked potential (c-VEP) based BCI systems, flickering visual stimuli may result in visual fatigue. Thus, we introduced a discrete-interval binary sequence (DIBS) as visual stimulus modulation, with its power spectrum optimized to emphasize high-frequency components (40 Hz-60 Hz). 8 and 17 subjects participated, respectively, in offline and online experiments on a 4-target asynchronous c-VEP-based BCI system designed to realize a high positive predictive value (PPV), a low false positive rate (FPR) during idle states, and a high true positive rate (TPR) in control states, while minimizing visual fatigue level. Two visual stimuli modulations were introduced and compared: a maximum length sequence (m-sequence) and the high-frequency discrete-interval binary sequence (DIBS). The decoding algorithm was compared among the canonical correlation analysis (CCA), the task-related component analysis (TRCA), and two approaches of sub-band component weight calculation (the traditional method and the proportional method) for FBCCA and FBTRCA. In the online experiments, the average PPV, FPR and TPR achieved, respectively [Formula: see text], [Formula: see text], [Formula: see text] with m-sequence, while [Formula: see text], [Formula: see text] and [Formula: see text] with DIBS. Estimated by objective eye-related metrics and a subjective questionnaire, the visual fatigue in DIBS cases is significantly smaller than that in m-sequence cases. In this study, the feasibility of a novel modulation approach for visual fatigue reduction was proved in an asynchronous c-VEP system, while maintaining comparable performance to existing methods, which provides further insights towards enhancing this field's long-term viability and user-friendliness.

Improving SSVEP-BCI Performance Through Repetitive Anodal tDCS-Based Neuromodulation: Insights From Fractal EEG and Brain Functional Connectivity

This study embarks on a comprehensive investigation of the effectiveness of repetitive transcranial direct current stimulation (tDCS)-based neuromodulation in augmenting steady-state visual evoked potential (SSVEP) brain-computer interfaces (BCIs), alongside exploring pertinent electroencephalography (EEG) biomarkers for assessing brain states and evaluating tDCS efficacy. EEG data were garnered across three distinct task modes (eyes open, eyes closed, and SSVEP stimulation) and two neuromodulation patterns (sham-tDCS and anodal-tDCS). Brain arousal and brain functional connectivity were measured by extracting features of fractal EEG and information flow gain, respectively. Anodal-tDCS led to diminished offsets and enhanced information flow gains, indicating improvements in both brain arousal and brain information transmission capacity. Additionally, anodal-tDCS markedly enhanced SSVEP-BCIs performance as evidenced by increased amplitudes and accuracies, whereas sham-tDCS exhibited lesser efficacy. This study proffers invaluable insights into the application of neuromodulation methods for bolstering BCI performance, and concurrently authenticates two potent electrophysiological markers for multifaceted characterization of brain states.

Visual tracking brain-computer interface

Brain-computer interfaces (BCIs) offer a way to interact with computers without relying on physical movements. Non-invasive electroencephalography-based visual BCIs, known for efficient speed and calibration ease, face limitations in continuous tasks due to discrete stimulus design and decoding methods. To achieve continuous control, we implemented a novel spatial encoding stimulus paradigm and devised a corresponding projection method to enable continuous modulation of decoded velocity. Subsequently, we conducted experiments involving 17 participants and achieved Fitt's information transfer rate (ITR) of 0.55 bps for the fixed tracking task and 0.37 bps for the random tracking task. The proposed BCI with a high Fitt's ITR was then integrated into two applications, including painting and gaming. In conclusion, this study proposed a visual BCI based-control method to go beyond discrete commands, allowing natural continuous control based on neural activity.

A Time-Local Weighted Transformation Recognition Framework for Steady State Visual Evoked Potentials Based Brain-Computer Interfaces

Canonical correlation analysis (CCA), Multivariate synchronization index (MSI), and their extended methods have been widely used for target recognition in Brain-computer interfaces (BCIs) based on Steady State Visual Evoked Potentials (SSVEP), and covariance calculation is an important process for these algorithms. Some studies have proved that embedding time-local information into the covariance can optimize the recognition effect of the above algorithms. However, the optimization effect can only be observed from the recognition results and the improvement principle of time-local information cannot be explained. Therefore, we propose a time-local weighted transformation (TT) recognition framework that directly embeds the time-local information into the electroencephalography signal through weighted transformation. The influence mechanism of time-local information on the SSVEP signal can then be observed in the frequency domain. Low-frequency noise is suppressed on the premise of sacrificing part of the SSVEP fundamental frequency energy, the harmonic energy of SSVEP is enhanced at the cost of introducing a small amount of high-frequency noise. The experimental results show that the TT recognition framework can significantly improve the recognition ability of the algorithms and the separability of extracted features. Its enhancement effect is significantly better than the traditional time-local covariance extraction method, which has enormous application potential.

Multi-Stimulus Least-Squares Transformation With Online Adaptation Scheme to Reduce Calibration Effort for SSVEP-Based BCIs

Steady-state visual evoked potential (SSVEP), one of the most popular electroencephalography (EEG)-based brain-computer interface (BCI) paradigms, can achieve high performance using calibration-based recognition algorithms. As calibration-based recognition algorithms are time-consuming to collect calibration data, the least-squares transformation (LST) has been used to reduce the calibration effort for SSVEP-based BCI. However, the transformation matrices constructed by current LST methods are not precise enough, resulting in large differences between the transformed data and the real data of the target subject. This ultimately leads to the constructed spatial filters and reference templates not being effective enough. To address these issues, this paper proposes multi-stimulus LST with online adaptation scheme (ms-LST-OA).

METHODS

The proposed ms-LST-OA consists of two parts. Firstly, to improve the precision of the transformation matrices, we propose the multi-stimulus LST (ms-LST) using cross-stimulus learning scheme as the cross-subject data transformation method. The ms-LST uses the data from neighboring stimuli to construct a higher precision transformation matrix for each stimulus to reduce the differences between transformed data and real data. Secondly, to further optimize the constructed spatial filters and reference templates, we use an online adaptation scheme to learn more features of the EEG signals of the target subject through an iterative process trial-by-trial.

RESULTS

ms-LST-OA performance was measured for three datasets (Benchmark Dataset, BETA Dataset, and UCSD Dataset). Using few calibration data, the ITR of ms-LST-OA achieved 210.01±10.10 bits/min, 172.31±7.26 bits/min, and 139.04±14.90 bits/min for all three datasets, respectively.

CONCLUSION

Using ms-LST-OA can reduce calibration effort for SSVEP-based BCIs.

A fused multi-subfrequency bands and CBAM SSVEP-BCI classification method based on convolutional neural network

For the brain-computer interface (BCI) system based on steady-state visual evoked potential (SSVEP), it is difficult to obtain satisfactory classification performance for short-time window SSVEP signals by traditional methods. In this paper, a fused multi-subfrequency bands and convolutional block attention module (CBAM) classification method based on convolutional neural network (CBAM-CNN) is proposed for discerning SSVEP-BCI tasks. This method extracts multi-subfrequency bands SSVEP signals as the initial input of the network model, and then carries out feature fusion on all feature inputs. In addition, CBAM is embedded in both parts of the initial input and feature fusion for adaptive feature refinement. To verify the effectiveness of the proposed method, this study uses the datasets of Inner Mongolia University of Technology (IMUT) and Tsinghua University (THU) to evaluate the performance of the proposed method. The experimental results show that the highest accuracy of CBAM-CNN reaches 0.9813 percentage point (pp). Within 0.1-2 s time window, the accuracy of CBAM-CNN is 0.0201-0.5388 (pp) higher than that of CNN, CCA-CWT-SVM, CCA-SVM, CCA-GNB, FBCCA, and CCA. Especially in the short-time window range of 0.1-1 s, the performance advantage of CBAM-CNN is more significant. The maximum information transmission rate (ITR) of CBAM-CNN is 503.87 bit/min, which is 227.53 bit/min-503.41 bit/min higher than the above six EEG decoding methods. The study further results show that CBAM-CNN has potential application value in SSVEP decoding.

A comparative study of stereo-dependent SSVEP targets and their impact on VR-BCI performance

Steady-state visual evoked potential brain-computer interfaces (SSVEP-BCI) have attracted significant attention due to their ease of deployment and high performance in terms of information transfer rate (ITR) and accuracy, making them a promising candidate for integration with consumer electronics devices. However, as SSVEP characteristics are directly associated with visual stimulus attributes, the influence of stereoscopic vision on SSVEP as a critical visual attribute has yet to be fully explored. Meanwhile, the promising combination of virtual reality (VR) devices and BCI applications is hampered by the significant disparity between VR environments and traditional 2D displays. This is not only due to the fact that screen-based SSVEP generally operates under static, stable conditions with simple and unvaried visual stimuli but also because conventional luminance-modulated stimuli can quickly induce visual fatigue. This study attempts to address these research gaps by designing SSVEP paradigms with stereo-related attributes and conducting a comparative analysis with the traditional 2D planar paradigm under the same VR environment. This study proposed two new paradigms: the 3D paradigm and the 3D-Blink paradigm. The 3D paradigm induces SSVEP by modulating the luminance of spherical targets, while the 3D-Blink paradigm employs modulation of the spheres' opacity instead. The results of offline 4-object selection experiments showed that the accuracy of 3D and 2D paradigm was 85.67 and 86.17% with canonical correlation analysis (CCA) and 86.17 and 91.73% with filter bank canonical correlation analysis (FBCCA), which is consistent with the reduction in the signal-to-noise ratio (SNR) of SSVEP harmonics for the 3D paradigm observed in the frequency-domain analysis. The 3D-Blink paradigm achieved 75.00% of detection accuracy and 27.02 bits/min of ITR with 0.8 seconds of stimulus time and task-related component analysis (TRCA) algorithm, demonstrating its effectiveness. These findings demonstrate that the 3D and 3D-Blink paradigms supported by VR can achieve improved user comfort and satisfactory performance, while further algorithmic optimization and feature analysis are required for the stereo-related paradigms. In conclusion, this study contributes to a deeper understanding of the impact of binocular stereoscopic vision mechanisms on SSVEP paradigms and promotes the application of SSVEP-BCI in diverse VR environments.

A calibration-free c-VEP based BCI employing narrow-band random sequences

Objective.Code-modulated visual evoked potential (c-VEP) based brain-computer interfaces (BCIs) exhibit high encoding efficiency. Nevertheless, the majority of c-VEP based BCIs necessitate an initial training or calibration session, particularly when the number of targets expands, which impedes the practicality. To address this predicament, this study introduces a calibration-free c-VEP based BCI employing narrow-band random sequences.Approach.For the encoding method, a series of random sequences were generated within a specific frequency band. The c-VEP signals were subsequently elicited through the application of on-type grid flashes that were modulated by these sequences. For the calibration-free decoding algorithm, filter-bank canonical correlation analysis (FBCCA) was utilized with the reference templates generated from the original sequences. Thirty-five subjects participated into an online BCI experiment. The performances of c-VEP based BCIs utilizing narrow-band random sequences with frequency bands of 15-25 Hz (NBRS-15) and 8-16 Hz (NBRS-8) were compared with that of a steady-state visual evoked potential (SSVEP) based BCI within a frequency range of 8-15.8 Hz.Main results.The offline analysis results demonstrated a substantial correlation between the c-VEPs and the original narrow-band random sequences. After parameter optimization, the calibration-free system employing the NBRS-15 frequency band achieved an average information transfer rate (ITR) of 78.56 ± 37.03 bits/min, which exhibited no significant difference compared to the performance of the SSVEP based system when utilizing FBCCA. The proposed system achieved an average ITR of 102.1 ± 57.59 bits/min in a simulation of a 1000-target BCI system.Significance.This study introduces a novel calibration-free c-VEP based BCI system employing narrow-band random sequences and shows great potential of the proposed system in achieving a large number of targets and high ITR.

Dynamic decomposition graph convolutional neural network for SSVEP-based brain-computer interface

The SSVEP-based paradigm serves as a prevalent approach in the realm of brain-computer interface (BCI). However, the processing of multi-channel electroencephalogram (EEG) data introduces challenges due to its non-Euclidean characteristic, necessitating methodologies that account for inter-channel topological relations. In this paper, we introduce the Dynamic Decomposition Graph Convolutional Neural Network (DDGCNN) designed for the classification of SSVEP EEG signals. Our approach incorporates layerwise dynamic graphs to address the oversmoothing issue in Graph Convolutional Networks (GCNs), employing a dense connection mechanism to mitigate the gradient vanishing problem. Furthermore, we enhance the traditional linear transformation inherent in GCNs with graph dynamic fusion, thereby elevating feature extraction and adaptive aggregation capabilities. Our experimental results demonstrate the effectiveness of proposed approach in learning and extracting features from EEG topological structure. The results shown that DDGCNN outperforms other state-of-the-art (SOTA) algorithms reported on two datasets (Dataset 1: 54 subjects, 4 targets, 2 sessions; Dataset 2: 35 subjects, 40 targets). Additionally, we showcase the implementation of DDGCNN in the context of synchronized BCI robotic fish control. This work represents a significant advancement in the field of EEG signal processing for SSVEP-based BCIs. Our proposed method processes SSVEP time domain signals directly as an end-to-end system, making it easy to deploy. The code is available at https://github.com/zshubin/DDGCNN.

Estimating and approaching the maximum information rate of noninvasive visual brain-computer interface

An essential priority of visual brain-computer interfaces (BCIs) is to enhance the information transfer rate (ITR) to achieve high-speed communication. Despite notable progress, noninvasive visual BCIs have encountered a plateau in ITRs, leaving it uncertain whether higher ITRs are achievable. In this study, we used information theory to study the characteristics and capacity of the visual-evoked channel, which leads us to investigate whether and how we can decode higher information rates in a visual BCI system. Using information theory, we estimate the upper and lower bounds of the information rate with the white noise (WN) stimulus. Consequently, we found out that the information rate is determined by the signal-to-noise ratio (SNR) in the frequency domain, which reflects the spectrum resources of the channel. Based on this discovery, we propose a broadband WN BCI by implementing stimuli on a broader frequency band than the steady-state visual evoked potentials (SSVEPs)-based BCI. Through validation, the broadband BCI outperforms the SSVEP BCI by an impressive 7 bps, setting a record of 50 bps. The integration of information theory and the decoding analysis presented in this study offers valuable insights applicable to general sensory-evoked BCIs, providing a potential direction of next-generation human-machine interaction systems.

A novel brain-controlled prosthetic hand method integrating AR-SSVEP augmentation, asynchronous control, and machine vision assistance

Background and objective

The brain-computer interface (BCI) system based on steady-state visual evoked potentials (SSVEP) is expected to help disabled patients achieve alternative prosthetic hand assistance. However, the existing study still has some shortcomings in interaction aspects such as stimulus paradigm and control logic. The purpose of this study is to innovate the visual stimulus paradigm and asynchronous decoding/control strategy by integrating augmented reality technology, and propose an asynchronous pattern recognition algorithm, thereby improving the interaction logic and practical application capabilities of the prosthetic hand with the BCI system.

Methods

An asynchronous visual stimulus paradigm based on an augmented reality (AR) interface was proposed in this paper, in which there were 8 control modes, including Grasp, Put down, Pinch, Point, Fist, Palm push, Hold pen, and Initial. According to the attentional orienting characteristics of the paradigm, a novel asynchronous pattern recognition algorithm that combines center extended canonical correlation analysis and support vector machine (Center-ECCA-SVM) was proposed. Then, this study proposed an intelligent BCI system switch based on a deep learning object detection algorithm (YOLOv4) to improve the level of user interaction. Finally, two experiments were designed to test the performance of the brain-controlled prosthetic hand system and its practical performance in real scenarios.

Results

Under the AR paradigm of this study, compared with the liquid crystal display (LCD) paradigm, the average SSVEP spectrum amplitude of multiple subjects increased by 17.41%, and the signal-noise ratio (SNR) increased by 3.52%. The average stimulus pattern recognition accuracy was 96.71 ± 3.91%, which was 2.62% higher than the LCD paradigm. Under the data analysis time of 2s, the Center-ECCA-SVM classifier obtained 94.66 ± 3.87% and 97.40 ± 2.78% asynchronous pattern recognition accuracy under the Normal metric and the Tolerant metric, respectively. And the YOLOv4-tiny model achieves a speed of 25.29fps and a 96.4% confidence in the prosthetic hand in real-time detection. Finally, the brain-controlled prosthetic hand helped the subjects to complete 4 kinds of daily life tasks in the real scene, and the time-consuming were all within an acceptable range, which verified the effectiveness and practicability of the system.

Conclusion

This research is based on improving the user interaction level of the prosthetic hand with the BCI system, and has made improvements in the SSVEP paradigm, asynchronous pattern recognition, interaction, and control logic. Furthermore, it also provides support for BCI areas for alternative prosthetic control, and movement disorder rehabilitation programs.

Optimizing Visual Stimulation Paradigms for User-Friendly SSVEP-Based BCIs

In steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) systems, traditional flickering stimulation patterns face challenges in achieving a trade-off in both BCI performance and visual comfort across various frequency bands. To investigate the optimal stimulation paradigms with high performance and high comfort for each frequency band, this study systematically compared the characteristics of SSVEP and user experience of different stimulation paradigms with a wide stimulation frequency range of 1-60 Hz. The findings suggest that, for a better balance between system performance and user experience, ON and OFF grid stimuli with a Weber contrast of 50% can be utilized as alternatives to traditional flickering stimulation paradigms in the frequency band of 1-25 Hz. In the 25-35 Hz range, uniform flicker stimuli with the same 50% contrast are more suitable. In the higher frequency band, traditional uniform flicker stimuli with a high 300% contrast are preferred. These results are significant for developing high performance and user-friendly SSVEP-based BCI systems.

Realness of face images can be decoded from non-linear modulation of EEG responses

Artificially created human faces play an increasingly important role in our digital world. However, the so-called uncanny valley effect may cause people to perceive highly, yet not perfectly human-like faces as eerie, bringing challenges to the interaction with virtual agents. At the same time, the neurocognitive underpinnings of the uncanny valley effect remain elusive. Here, we utilized an electroencephalography (EEG) dataset of steady-state visual evoked potentials (SSVEP) in which participants were presented with human face images of different stylization levels ranging from simplistic cartoons to actual photographs. Assessing neuronal responses both in frequency and time domain, we found a non-linear relationship between SSVEP amplitudes and stylization level, that is, the most stylized cartoon images and the real photographs evoked stronger responses than images with medium stylization. Moreover, realness of even highly similar stylization levels could be decoded from the EEG data with task-related component analysis (TRCA). Importantly, we also account for confounding factors, such as the size of the stimulus face's eyes, which previously have not been adequately addressed. Together, this study provides a basis for future research and neuronal benchmarking of real-time detection of face realness regarding three aspects: SSVEP-based neural markers, efficient classification methods, and low-level stimulus confounders.

An open dataset for human SSVEPs in the frequency range of 1-60 Hz

A steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) system relies on the photic driving response to effectively elicit characteristic electroencephalogram (EEG) signals. However, traditional visual stimuli mainly adopt high-contrast black-and-white flickering stimulations, which are easy to cause visual fatigue. This paper presents an SSVEP dataset acquired at a wide frequency range from 1 to 60 Hz with an interval of 1 Hz using flickering stimuli under two different modulation depths. This dataset contains 64-channel EEG data from 30 healthy subjects when they fixated on a single flickering stimulus. The stimulus was rendered on an LCD display with a refresh rate of 240 Hz. Initially, the dataset was rigorously validated through comprehensive data analysis to investigate SSVEP responses and user experiences. Subsequently, BCI performance was evaluated through offline simulations of frequency-coded and phase-coded BCI paradigms. This dataset provides comprehensive and high-quality data for studying and developing SSVEP-based BCI systems.

Multi-information improves the performance of CCA-based SSVEP classification

The target recognition algorithm based on canonical correlation analysis (CCA) has been widely used in steady-state visual evoked potential (SSVEP)-based brain-computer interfaces. To reduce visual fatigue and improve the information transfer rate (ITR), how to improve the accuracy of algorithms within a short time window has become one of the main problems at present. There were filter bank CCA (FBCCA), individual template CCA (ITCCA), and temporally local CCA (TCCA), which improve the CCA algorithm from different aspects.This paper proposed to consider individual, frequency, and time information at the same time, so as to extract features more effectively. A comparison of the various methods was performed using benchmark dataset. Classification accuracy and ITR were used for performance evaluation. In the different extensions of CCA, the method incorporating the above three kinds of information simultaneously achieved the best performance within a short time window. This study explores the effect of using a variety of information to improve the CCA algorithm.

Multi-frequency steady-state visual evoked potential dataset

The Steady-State Visual Evoked Potential (SSVEP) is a widely used modality in Brain-Computer Interfaces (BCIs). Existing research has demonstrated the capabilities of SSVEP that use single frequencies for each target in various applications with relatively small numbers of commands required in the BCI. Multi-frequency SSVEP has been developed to extend the capability of single-frequency SSVEP to tasks that involve large numbers of commands. However, the development on multi-frequency SSVEP methodologies is falling behind compared to the number of studies with single-frequency SSVEP. This dataset was constructed to promote research in multi-frequency SSVEP by making SSVEP signals collected with different frequency stimulation settings publicly available. In this dataset, SSVEPs were collected from 35 participants using single-, dual-, and tri-frequency stimulation and with three different multi-frequency stimulation variants.

MetaBCI: An open-source platform for brain-computer interfaces

BACKGROUND

Recently, brain-computer interfaces (BCIs) have attracted worldwide attention for their great potential in clinical and real-life applications. To implement a complete BCI system, one must set up several links to translate the brain intent into computer commands. However, there is not an open-source software platform that can cover all links of the BCI chain.

METHOD

This study developed a one-stop open-source BCI software, namely MetaBCI, to facilitate the construction of a BCI system. MetaBCI is written in Python, and has the functions of stimulus presentation (Brainstim), data loading and processing (Brainda), and online information flow (Brainflow). This paper introduces the detailed information of MetaBCI and presents four typical application cases.

RESULTS

The results showed that MetaBCI was an extensible and feature-rich software platform for BCI research and application, which could effectively encode, decode, and feedback brain activities.

CONCLUSIONS

MetaBCI can greatly lower the BCI's technical threshold for BCI beginners and can save time and cost to build up a practical BCI system. The source code is available at https://github.com/TBC-TJU/MetaBCI, expecting new contributions from the BCI community.

Effect of motion state variability on error-related potentials during continuous feedback paradigms and their consequences for classification

BACKGROUND

An erroneous motion would elicit the error-related potential (ErrP) when humans monitor the behavior of the external devices. This EEG modality has been largely applied to brain-computer interface in an active or passive manner with discrete visual feedback. However, the effect of variable motion state on ErrP morphology and classification performance raises concerns when the interaction is conducted with continuous visual feedback.

NEW METHOD

In the present study, we designed a cursor control experiment. Participants monitored a continuously moving cursor to reach the target on one side of the screen. Motion state varied multiple times with two factors: (1) motion direction and (2) motion speed. The effects of these two factors on the morphological characteristics and classification performance of ErrP were analyzed. Furthermore, an offline simulation was performed to evaluate the effectiveness of the proposed extended ErrP-decoder in resolving the interference by motion direction changes.

RESULTS

The statistical analyses revealed that motion direction and motion speed significantly influenced the amplitude of feedback-ERN and frontal-Pe components, while only motion direction significantly affected the classification performance.

COMPARISON WITH EXISTING METHODS

Significant deviation was found in ErrP detection utilizing classical correct-versus-erroneous event training. However, this bias can be alleviated by 16% by the extended ErrP-decoder.

CONCLUSION

The morphology and classification performance of ErrP signal can be affected by motion state variability during continuous feedback paradigms. The results enhance the comprehension of ErrP morphological components and shed light on the detection of BCI's error behavior in practical continuous control.

Burst c-VEP Based BCI: Optimizing stimulus design for enhanced classification with minimal calibration data and improved user experience

The utilization of aperiodic flickering visual stimuli under the form of code-modulated Visual Evoked Potentials (c-VEP) represents a pivotal advancement in the field of reactive Brain-Computer Interface (rBCI). A major advantage of the c-VEP approach is that the training of the model is independent of the number and complexity of targets, which helps reduce calibration time. Nevertheless, the existing designs of c-VEP stimuli can be further improved in terms of visual user experience but also to achieve a higher signal-to-noise ratio, while shortening the selection time and calibration process. In this study, we introduce an innovative variant of code-VEP, referred to as "Burst c-VEP". This original approach involves the presentation of short bursts of aperiodic visual flashes at a deliberately slow rate, typically ranging from two to four flashes per second. The rationale behind this design is to leverage the sensitivity of the primary visual cortex to transient changes in low-level stimuli features to reliably elicit distinctive series of visual evoked potentials. In comparison to other types of faster-paced code sequences, burst c-VEP exhibit favorable properties to achieve high bitwise decoding performance using convolutional neural networks (CNN), which yields potential to attain faster selection time with the need for less calibration data. Furthermore, our investigation focuses on reducing the perceptual saliency of c-VEP through the attenuation of visual stimuli contrast and intensity to significantly improve users' visual comfort. The proposed solutions were tested through an offline 4-classes c-VEP protocol involving 12 participants. Following a factorial design, participants were instructed to focus on c-VEP targets whose pattern (burst and maximum-length sequences) and amplitude (100% or 40% amplitude depth modulations) were manipulated across experimental conditions. Firstly, the full amplitude burst c-VEP sequences exhibited higher accuracy, ranging from 90.5% (with 17.6s of calibration data) to 95.6% (with 52.8s of calibration data), compared to its m-sequence counterpart (71.4% to 85.0%). The mean selection time for both types of codes (1.5 s) compared favorably to reports from previous studies. Secondly, our findings revealed that lowering the intensity of the stimuli only slightly decreased the accuracy of the burst code sequences to 94.2% while leading to substantial improvements in terms of user experience. Taken together, these results demonstrate the high potential of the proposed burst codes to advance reactive BCI both in terms of performance and usability. The collected dataset, along with the proposed CNN architecture implementation, are shared through open-access repositories.

Using oscillatory and aperiodic neural activity features for identifying idle state in SSVEP-based BCIs reduces false triggers

Objective.In existing studies, rhythmic (oscillatory) components were used as main features to identify brain states, such as control and idle states, while non-rhythmic (aperiodic) components were ignored. Recent studies have shown that aperiodic (1/f) activity is functionally related to cognitive processes. It is not clear if aperiodic activity can distinguish brain states in asynchronous brain-computer interfaces (BCIs) to reduce false triggers. In this paper, we propose an asynchronous method based on the fusion of oscillatory and aperiodic features for steady-state visual evoked potential-based BCIs.Approach.The proposed method first evaluates the oscillatory and aperiodic components of control and idle states using irregular-resampling auto-spectral analysis. Oscillatory features are then extracted using the spectral power of fundamental, second-harmonic, and third-harmonic frequencies of the oscillatory component, and aperiodic features are extracted using the slope and intercept of the first-order polynomial of the spectral fit of the aperiodic component under a log-logarithmic axis. The process produces two types of feature pools (oscillatory, aperiodic features). Next, feature selection (dimensionality reduction) is applied to the feature pools by Bonferroni correctedp-values from two-way analysis of variance. Last, these spatial-specific statistically significant features are used as input for classification to identify the idle state.Mainresults.On a 7-target dataset from 15 subjects, the mix of oscillatory and aperiodic features achieved an average accuracy of 88.39% compared to 83.53% when using oscillatory features alone (4.86% improvement). The results demonstrated that the proposed idle state recognition method achieved enhanced performance by incorporating aperiodic features.Significance.Our results demonstrated that (1) aperiodic features were effective in recognizing idle states and (2) fusing features of oscillatory and aperiodic components enhanced classification performance by 4.86% compared to oscillatory features alone.

A data expansion technique based on training and testing sample to boost the detection of SSVEPs for brain-computer interfaces

Objective.Currently, steady-state visual evoked potentials (SSVEPs)-based brain-computer interfaces (BCIs) have achieved the highest interaction accuracy and speed among all BCI paradigms. However, its decoding efficacy depends deeply on the number of training samples, and the system performance would have a dramatic drop when the training dataset decreased to a small size. To date, no study has been reported to incorporate the unsupervised learning information from testing trails into the construction of supervised classification model, which is a potential way to mitigate the overfitting effect of limited samples.Approach.This study proposed a novel method for SSVEPs detection, i.e. cyclic shift trials (CSTs), which could combine unsupervised learning information from test trials and supervised learning information from train trials. Furthermore, since SSVEPs are time-locked and phase-locked to the onset of specific flashes, CST could also expand training samples on the basis of its regularity and periodicity. In order to verify the effectiveness of CST, we designed an online SSVEP-BCI system, and tested this system combined CST with two common classification algorithms, i.e. extended canonical correlation analysis and ensemble task-related component analysis.Main results.CST could significantly enhance the signal to noise ratios of SSVEPs and improve the performance of systems especially for the condition of few training samples and short stimulus time. The online information transfer rate could reach up to 236.19 bits min-1using 36 s calibration time of only one training sample for each category.Significance.The proposed CST method can take full advantages of supervised learning information from training samples and unsupervised learning information of testing samples. Furthermore, it is a data expansion technique, which can enhance the SSVEP characteristics and reduce dependence on sample size. Above all, CST is a promising method to improve the performance of SSVEP-based BCI without any additional experimental burden.

Almost free of calibration for SSVEP-based brain-computer interfaces

Objective. Steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) is a promising technology that can achieve high information transfer rate (ITR) with supervised algorithms such as ensemble task-related component analysis (eTRCA) and task-discriminant component analysis (TDCA). However, training individual models requires a tedious and time-consuming calibration process, which hinders the real-life use of SSVEP-BCIs. A recent data augmentation method, called source aliasing matrix estimation (SAME), can generate new EEG samples from a few calibration trials. But SAME does not exploit the information across stimuli as well as only reduces the number of calibration trials per command, so it still has some limitations.Approach. This study proposes an extended version of SAME, called multi-stimulus SAME (msSAME), which exploits the similarity of the aliasing matrix across frequencies to enhance the performance of SSVEP-BCI with insufficient calibration trials. We also propose a semi-supervised approach based on msSAME that can further reduce the number of SSVEP frequencies needed for calibration. We evaluate our method on two public datasets, Benchmark and BETA, and an online experiment.Main results. The results show that msSAME outperforms SAME for both eTRCA and TDCA on the public datasets. Moreover, the semi-supervised msSAME-based method achieves comparable performance to the fully calibrated methods and outperforms the conventional free-calibrated methods. Remarkably, our method only needs 24 s to calibrate 40 targets in the online experiment and achieves an average ITR of 213.8 bits min-1with a peak of 242.6 bits min-1.Significance. This study significantly reduces the calibration effort for individual SSVEP-BCIs, which is beneficial for developing practical plug-and-play SSVEP-BCIs.

VEPdgets: Towards Richer Interaction Elements Based on Visually Evoked Potentials

For brain-computer interfaces, a variety of technologies and applications already exist. However, current approaches use visual evoked potentials (VEP) only as action triggers or in combination with other input technologies. This paper shows that the losing visually evoked potentials after looking away from a stimulus is a reliable temporal parameter. The associated latency can be used to control time-varying variables using the VEP. In this context, we introduced VEP interaction elements (VEP widgets) for a value input of numbers, which can be applied in various ways and is purely based on VEP technology. We carried out a user study in a desktop as well as in a virtual reality setting. The results for both settings showed that the temporal control approach using latency correction could be applied to the input of values using the proposed VEP widgets. Even though value input is not very accurate under untrained conditions, users could input numerical values. Our concept of applying latency correction to VEP widgets is not limited to the input of numbers.

Automated atrial fibrillation and ventricular fibrillation recognition using a multi-angle dual-channel fusion network

Atrial fibrillation (AFIB) and ventricular fibrillation (VFIB) are two common cardiovascular diseases that cause numerous deaths worldwide. Medical staff usually adopt long-term ECGs as a tool to diagnose AFIB and VFIB. However, since ECG changes are occasionally subtle and similar, visual observation of ECG changes is challenging. To address this issue, we proposed a multi-angle dual-channel fusion network (MDF-Net) to automatically recognize AFIB and VFIB heartbeats in this work. MDF-Net can be seen as the fusion of a task-related component analysis (TRCA)-principal component analysis (PCA) network (TRPC-Net), a canonical correlation analysis (CCA)-PCA network (CPC-Net), and the linear support vector machine-weighted softmax with average (LS-WSA) method. TRPC-Net and CPC-Net are employed to extract deep task-related and correlation features, respectively, from two-lead ECGs, by which multi-angle feature-level information fusion is realized. Since the convolution kernels of the above methods can be directly extracted through TRCA, CCA and PCA technologies, their training time is faster than that of convolutional neural networks. Finally, LS-WSA is employed to fuse the above features at the decision level, by which the classification results are obtained. In distinguishing AFIB and VFIB heartbeats, the proposed method achieved accuracies of 99.39 % and 97.17 % in intra- and inter-patient experiments, respectively. In addition, this method performed well on noisy data and extremely imbalanced data, in which abnormal heatbeats are much less than normal heartbeats. Our proposed method has the potential to be used as a diagnostic tool in the clinic.

Spectrum-Enhanced TRCA (SE-TRCA): A novel approach for direction detection in SSVEP-based BCI

The Steady State Visual Evoked Potential (SSVEP) is a widely used component in BCIs due to its high noise resistance and low equipment requirements. Recently, a novel SSVEP-based paradigm has been introduced for direction detection, in which, unlike the common SSVEP paradigms that use several frequency stimuli, only one flickering stimulus is used and it makes direction detection very challenging. So far, only the CCA method has been used for direction detection using SSVEP component analysis. Since Canonical Correlation Analysis (CCA) has some limitations, a Task-Related Component Analysis (TRCA) based method has been introduced for feature extraction to improve the direction detection performance. Although these methods have been proven efficient, they do not utilize the latent frequency information in the EEG signal. Therefore, the performance of direction detection using SSVEP component analysis is still suboptimal. For further improvement, the TRCA-based algorithm is enhanced by incorporating frequency information and introducing Spectrum-Enhanced TRCA (SE-TRCA). SE-TRCA method can utilize frequency information in conjunction with spatial information by concatenating the EEG signal and its shifted version. Accordingly, the obtained spatio-spectral filters perform as a Finite Impulse Response (FIR) filter. To evaluate the proposed SE-TRCA method, two different sorts of datasets (1) a hybrid BCI dataset (including SSVEP component for direction detection) and (2) a pure benchmark SSVEP dataset (including SSVEP component for frequency detection) have been used. Our experiments showed that the accuracy of direction detection using the proposed SE-TRCA and TRCA approaches compared to CCA-based approach have been increased by 23.35% and 28.24%, respectively. Furthermore, the accuracy of character recognition obtained from integrating P300 and SSVEP components in CCA, TRCA, and SETRCA approaches are 54.01%, 56.02%, and 58.56%, on the hybrid dataset, respectively. The evaluation of the SE-TRCA method on the benchmark SSVEP dataset demonstrates that the SE-TRCA method outperforms both CCA and TRCA, particularly regarding frequency detection accuracy. In this specific dataset, the SE-TRCA method achieved an impressive frequency detection accuracy of 98.19% for a 3-s signal, surpassing the accuracies of TRCA and CCA, which were 97.91% and 90.47%, respectively. These results demonstrated that the TRCA-based approach is more efficient than the CCA approach to extracting spatial filters. Moreover, SE-TRCA, extracting both Spectral and spatial information from the EEG signal, can capture more discriminative features from the SSVEP component and increase the accuracy of classification. The results of this study emphasize the effectiveness of the proposed SE-TRCA approach across different SSVEP paradigms and tasks. These findings provide strong evidence for the method's ability to generalize well in SSVEP analysis.

Improving Generalized Zero-Shot Learning SSVEP Classification Performance From Data-Efficient Perspective

Generalized zero-shot learning (GZSL) has significantly reduced the training requirements for steady-state visual evoked potential (SSVEP) based brain-computer interfaces (BCIs). Traditional methods require complete class data sets for training, but GZSL allows for only partial class data sets, dividing them into 'seen' (those with training data) and 'unseen' classes (those without training data). However, inefficient utilization of SSVEP data limits the accuracy and information transfer rate (ITR) of existing GZSL methods. To this end, we proposed a framework for more effective utilization of SSVEP data at three systematically combined levels: data acquisition, feature extraction, and decision-making. First, prevalent SSVEP-based BCIs overlook the inter-subject variance in visual latency and employ fixed sampling starting time (SST). We introduced a dynamic sampling starting time (DSST) strategy at the data acquisition level. This strategy uses the classification results on the validation set to find the optimal sampling starting time (OSST) for each subject. In addition, we developed a Transformer structure to capture the global information of input data for compensating the small receptive field of existing networks. The global receptive fields of the Transformer can adequately process the information from longer input sequences. For the decision-making level, we designed a classifier selection strategy that can automatically select the optimal classifier for the seen and unseen classes, respectively. We also proposed a training procedure to make the above solutions in conjunction with each other. Our method was validated on three public datasets and outperformed the state-of-the-art (SOTA) methods. Crucially, we also outperformed the representative methods that require training data for all classes.

Phase-Locked Time-Shift Data Augmentation Method for SSVEP Brain-Computer Interfaces

Steady-state visual evoked potential (SSVEP) based brain-computer interfaces (BCIs) have achieved an information transfer rate (ITR) of over 300 bits/min, but abundant training data is required. The performance of SSVEP algorithms deteriorates greatly under limited data, and the existing time-shift data augmentation method fails to improve it because the phase-locked requirement between training samples is violated. To address this issue, this study proposes a novel augmentation method, namely phase-locked time-shift (PLTS), for SSVEP-BCI. The similarity between epochs at different time moments was evaluated, and a unique time-shift step was calculated for each class to augment additional data epochs in each trial. The results showed that the PLTS significantly improved the classification performance of SSVEP algorithms on the BETA SSVEP datasets. Moreover, under the condition of one calibration block, by slightly prolonging the calibration duration (from 48 s to 51.5 s), the ITR increased from 40.88±4.54 bits/min to 122.61±7.05 bits/min with the PLTS. This study provides a new perspective on augmenting data epochs for training-based SSVEP-BCI, promotes the classification accuracy and ITR under limited training data, and thus facilitates the real-life applications of SSVEP-based brain spellers.

A novel untrained SSVEP-EEG feature enhancement method using canonical correlation analysis and underdamped second-order stochastic resonance

Objective

Compared with the light-flashing paradigm, the ring-shaped motion checkerboard patterns avoid uncomfortable flicker or brightness modulation, improving the practical interactivity of brain-computer interface (BCI) applications. However, due to fewer harmonic responses and more concentrated frequency energy elicited by the ring-shaped checkerboard patterns, the mainstream untrained algorithms such as canonical correlation analysis (CCA) and filter bank canonical correlation analysis (FBCCA) methods have poor recognition performance and low information transmission rate (ITR).

Methods

To address this issue, a novel untrained SSVEP-EEG feature enhancement method using CCA and underdamped second-order stochastic resonance (USSR) is proposed to extract electroencephalogram (EEG) features.

Results

In contrast to typical unsupervised dimensionality reduction methods such as common average reference (CAR), principal component analysis (PCA), multidimensional scaling (MDS), and locally linear embedding (LLE), CCA exhibits higher adaptability for SSVEP rhythm components.

Conclusion

This study recruits 42 subjects to evaluate the proposed method and experimental results show that the untrained method can achieve higher detection accuracy and robustness.

Significance

This untrained method provides the possibility of applying a nonlinear model from one-dimensional signals to multi-dimensional signals.

Challenges of neural interfaces for stroke motor rehabilitation

More than 85% of stroke survivors suffer from different degrees of disability for the rest of their lives. They will require support that can vary from occasional to full time assistance. These conditions are also associated to an enormous economic impact for their families and health care systems. Current rehabilitation treatments have limited efficacy and their long-term effect is controversial. Here we review different challenges related to the design and development of neural interfaces for rehabilitative purposes. We analyze current bibliographic evidence of the effect of neuro-feedback in functional motor rehabilitation of stroke patients. We highlight the potential of these systems to reconnect brain and muscles. We also describe all aspects that should be taken into account to restore motor control. Our aim with this work is to help researchers designing interfaces that demonstrate and validate neuromodulation strategies to enforce a contingent and functional neural linkage between the central and the peripheral nervous system. We thus give clues to design systems that can improve or/and re-activate neuroplastic mechanisms and open a new recovery window for stroke patients.

Cross Domain Correlation Maximization for Enhancing the Target Recognition of SSVEP-Based Brain-Computer Interfaces

The target recognition performance of steady-state visual evoked potential (SSVEP)-based brain-computer interfaces can be significantly improved with a training-based approach. However, the training procedure is time consuming and often causes fatigue. Consequently, the number of training data should be limited, which may reduce the classification performance. Thus, how to improve classification accuracy without increasing the training time is crucial to SSVEP-based BCI system. This study proposes a transfer-related component analysis (TransRCA) method for addressing the above issue. In this method, the SSVEP-related components are extracted from a small number of training data of the current individual and combined with those extracted from a large number of existing training data of other individuals. The TransRCA method maximizes not only the inter-trial covariances between the source and target subjects, but also the correlation between the reference signals and SSVEP signals from the source and target subjects. The proposed method was validated on the SSVEP public Benchmark and BETA datasets, and the classification accuracy and information transmission rate of the ensemble version of the proposed TransRCA method were compared with those of the state-of-the-art eCCA, eTRCA, ttCCA, LSTeTRCA, and eIISMC methods on both datasets. The comparison results indicate that the proposed method provides a superior performance compared with these state-of-the-art methods, and thus has high potential for the development of a SSVEP-based brain-computer interface system with high classification performance that only uses a small number of training data.

A Calibration-Free Hybrid Approach Combining SSVEP and EOG for Continuous Control

While SSVEP-BCI has been widely developed to control external devices, most of them rely on the discrete control strategy. The continuous SSVEP-BCI enables users to continuously deliver commands and receive real-time feedback from the devices, but it suffers from the transition state problem, a period the erroneous recognition, when users shift their gazes between targets. To resolve this issue, we proposed a novel calibration-free Bayesian approach by hybridizing SSVEP and electrooculography (EOG). First, canonical correlation analysis (CCA) was applied to detect the evoked SSVEPs, and saccade during the gaze shift was detected by EOG data using an adaptive threshold method. Then, the new target after the gaze shift was recognized based on a Bayesian optimization approach, which combined the detection of SSVEP and saccade together and calculated the optimized probability distribution of the targets. Eighteen healthy subjects participated in the offline and online experiments. The offline experiments showed that the proposed hybrid BCI had significantly higher overall continuous accuracy and shorter gaze-shifting time compared to FBCCA, CCA, MEC, and PSDA. In online experiments, the proposed hybrid BCI significantly outperformed CCA-based SSVEP-BCI in terms of continuous accuracy (77.61 ± 1.36%vs. 68.86 ± 1.08% and gaze-shifting time (0.93 ± 0.06s vs. 1.94 ± 0.08s). Additionally, participants also perceived a significant improvement over the CCA-based SSVEP-BCI when the newly proposed decoding approach was used. These results validated the efficacy of the proposed hybrid Bayesian approach for the BCI continuous control without any calibration. This study provides an effective framework for combining SSVEP and EOG, and promotes the potential applications of plug-and-play BCIs in continuous control.

A Calibration-Free Hybrid BCI Speller System Based on High-Frequency SSVEP and sEMG

Hybrid brain-computer interface (hBCI) systems that combine steady-state visual evoked potential (SSVEP) and surface electromyography (sEMG) signals have attracted attention of researchers due to the advantage of exhibiting significantly improved system performance. However, almost all existing studies adopt low-frequency SSVEP to build hBCI. It produces much more visual fatigue than high-frequency SSVEP. Therefore, the current study attempts to build a hBCI based on high-frequency SSVEP and sEMG. With these two signals, this study designed and realized a 32-target hBCI speller system. Thirty-two targets were separated from the middle into two groups. Each side contained 16 sets of targets with different high-frequency visual stimuli (i.e., 31-34.75 Hz with an interval of 0.25 Hz). sEMG was utilized to choose the group and SSVEP was adopted to identify intra-group targets. The filter bank canonical correlation analysis (FBCCA) and the root mean square value (RMS) methods were used to identify signals. Therefore, the proposed system allowed users to operate it without system calibration. A total of 12 healthy subjects participated in online experiment, with an average accuracy of 93.52 ± 1.66% and the average information transfer rate (ITR) reached 93.50 ± 3.10 bits/min. Furthermore, 12 participants perfectly completed the free-spelling tasks. These results of the experiments indicated feasibility and practicality of the proposed hybrid BCI speller system.