Multivariate synchronization index for frequency recognition of SSVEP-based brain–computer interface

Filter bank temporally local multivariate synchronization index for SSVEP-based BCI

BACKGROUND

Multivariate synchronization index (MSI) has been successfully applied for frequency detection in steady state visual evoked potential (SSVEP) based brain-computer interface (BCI) systems. However, the standard MSI algorithm and its variants cannot simultaneously take full advantage of the time-local structure and the harmonic components in SSVEP signals, which are both crucial for frequency detection performance. To overcome the limitation, we propose a novel filter bank temporally local MSI (FBTMSI) algorithm to further improve SSVEP frequency detection accuracy. The method explicitly utilizes the temporal information of signal for covariance matrix estimation and employs filter bank decomposition to exploits SSVEP-related harmonic components.

RESULTS

We employed the cross-validation strategy on the public Benchmark dataset to optimize the parameters and evaluate the performance of the FBTMSI algorithm. Experimental results show that FBTMSI outperforms the standard MSI, temporally local MSI (TMSI) and filter bank driven MSI (FBMSI) algorithms across multiple experimental settings. In the case of data length of one second, the average accuracy of FBTMSI is 9.85% and 3.15% higher than that of the FBMSI and the TMSI, respectively.

CONCLUSIONS

The promising results demonstrate the effectiveness of the FBTMSI algorithm for frequency recognition and show its potential in SSVEP-based BCI applications.

A Time-Local Weighted Transformation Recognition Framework for Steady State Visual Evoked Potentials Based Brain-Computer Interfaces

Canonical correlation analysis (CCA), Multivariate synchronization index (MSI), and their extended methods have been widely used for target recognition in Brain-computer interfaces (BCIs) based on Steady State Visual Evoked Potentials (SSVEP), and covariance calculation is an important process for these algorithms. Some studies have proved that embedding time-local information into the covariance can optimize the recognition effect of the above algorithms. However, the optimization effect can only be observed from the recognition results and the improvement principle of time-local information cannot be explained. Therefore, we propose a time-local weighted transformation (TT) recognition framework that directly embeds the time-local information into the electroencephalography signal through weighted transformation. The influence mechanism of time-local information on the SSVEP signal can then be observed in the frequency domain. Low-frequency noise is suppressed on the premise of sacrificing part of the SSVEP fundamental frequency energy, the harmonic energy of SSVEP is enhanced at the cost of introducing a small amount of high-frequency noise. The experimental results show that the TT recognition framework can significantly improve the recognition ability of the algorithms and the separability of extracted features. Its enhancement effect is significantly better than the traditional time-local covariance extraction method, which has enormous application potential.

Brain-computer interface digital prescription for neurological disorders

Neurological and psychiatric diseases can lead to motor, language, emotional disorder, and cognitive, hearing or visual impairment By decoding the intention of the brain in real time, the Brain-computer interface (BCI) can first assist in the diagnosis of diseases, and can also compensate for its damaged function by directly interacting with the environment; In addition, provide output signals in various forms, such as actual motion, tactile or visual feedback, to assist in rehabilitation training; Further intervention in brain disorders is achieved by close-looped neural modulation. In this article, we envision the future BCI digital prescription system for patients with different functional disorders and discuss the key contents in the prescription the brain signals, coding and decoding protocols and interaction paradigms, and assistive technology. Then, we discuss the details that need to be specially included in the digital prescription for different intervention technologies. The third part summarizes previous examples of intervention, focusing on how to select appropriate interaction paradigms for patients with different functional impairments. For the last part, we discussed the indicators and influencing factors in evaluating the therapeutic effect of BCI as intervention.

Multi-information improves the performance of CCA-based SSVEP classification

The target recognition algorithm based on canonical correlation analysis (CCA) has been widely used in steady-state visual evoked potential (SSVEP)-based brain-computer interfaces. To reduce visual fatigue and improve the information transfer rate (ITR), how to improve the accuracy of algorithms within a short time window has become one of the main problems at present. There were filter bank CCA (FBCCA), individual template CCA (ITCCA), and temporally local CCA (TCCA), which improve the CCA algorithm from different aspects.This paper proposed to consider individual, frequency, and time information at the same time, so as to extract features more effectively. A comparison of the various methods was performed using benchmark dataset. Classification accuracy and ITR were used for performance evaluation. In the different extensions of CCA, the method incorporating the above three kinds of information simultaneously achieved the best performance within a short time window. This study explores the effect of using a variety of information to improve the CCA algorithm.

Filter bank second-order underdamped stochastic resonance analysis for implementing a short-term high-speed SSVEP detection

OBJECTIVE

The progression of brain-computer interfaces (BCIs) has been propelled by breakthroughs in neuroscience, signal processing, and machine learning, marking it as a dynamic field of study over the past few decades. Nevertheless, the nonlinear and non-stationary characteristics of steady-state visual evoked potentials (SSVEPs), coupled with the incongruity between frequently employed linear techniques and nonlinear signal attributes, resulted in the subpar performance of mainstream non-training algorithms like canonical correlation analysis (CCA), multivariate synchronization index (MSI), and filter bank CCA (FBCCA) in short-term SSVEP detection.

METHODS

To tackle this problem, the novel fusions of common filter bank analysis, CCA dimensionality reduction methods, USSR models, and MSI recognition models are used in SSVEP signal recognition.

RESULTS

Unlike conventional linear techniques such as CCA, MSI, and FBCCA, the filter bank second-order underdamped stochastic resonance (FBUSSR) analysis demonstrates superior efficacy in the detection of short-term high-speed SSVEPs.

CONCLUSION

This research enlists 32 subjects and uses a public dataset to assess the proposed approach, and the experimental outcomes indicate that the non-training method can attain greater recognition precision and stability. Furthermore, under the conditions of the newly proposed fusion method and light stimulation, the USSR model exhibits the most optimal enhancement effect.

SIGNIFICANCE

The findings of this study underscore the expansive potential for the application of BCI systems in the realm of neuroscience and signal processing.

Phase-Locked Time-Shift Data Augmentation Method for SSVEP Brain-Computer Interfaces

Steady-state visual evoked potential (SSVEP) based brain-computer interfaces (BCIs) have achieved an information transfer rate (ITR) of over 300 bits/min, but abundant training data is required. The performance of SSVEP algorithms deteriorates greatly under limited data, and the existing time-shift data augmentation method fails to improve it because the phase-locked requirement between training samples is violated. To address this issue, this study proposes a novel augmentation method, namely phase-locked time-shift (PLTS), for SSVEP-BCI. The similarity between epochs at different time moments was evaluated, and a unique time-shift step was calculated for each class to augment additional data epochs in each trial. The results showed that the PLTS significantly improved the classification performance of SSVEP algorithms on the BETA SSVEP datasets. Moreover, under the condition of one calibration block, by slightly prolonging the calibration duration (from 48 s to 51.5 s), the ITR increased from 40.88±4.54 bits/min to 122.61±7.05 bits/min with the PLTS. This study provides a new perspective on augmenting data epochs for training-based SSVEP-BCI, promotes the classification accuracy and ITR under limited training data, and thus facilitates the real-life applications of SSVEP-based brain spellers.

A novel untrained SSVEP-EEG feature enhancement method using canonical correlation analysis and underdamped second-order stochastic resonance

Objective

Compared with the light-flashing paradigm, the ring-shaped motion checkerboard patterns avoid uncomfortable flicker or brightness modulation, improving the practical interactivity of brain-computer interface (BCI) applications. However, due to fewer harmonic responses and more concentrated frequency energy elicited by the ring-shaped checkerboard patterns, the mainstream untrained algorithms such as canonical correlation analysis (CCA) and filter bank canonical correlation analysis (FBCCA) methods have poor recognition performance and low information transmission rate (ITR).

Methods

To address this issue, a novel untrained SSVEP-EEG feature enhancement method using CCA and underdamped second-order stochastic resonance (USSR) is proposed to extract electroencephalogram (EEG) features.

Results

In contrast to typical unsupervised dimensionality reduction methods such as common average reference (CAR), principal component analysis (PCA), multidimensional scaling (MDS), and locally linear embedding (LLE), CCA exhibits higher adaptability for SSVEP rhythm components.

Conclusion

This study recruits 42 subjects to evaluate the proposed method and experimental results show that the untrained method can achieve higher detection accuracy and robustness.

Significance

This untrained method provides the possibility of applying a nonlinear model from one-dimensional signals to multi-dimensional signals.

Cross Domain Correlation Maximization for Enhancing the Target Recognition of SSVEP-Based Brain-Computer Interfaces

The target recognition performance of steady-state visual evoked potential (SSVEP)-based brain-computer interfaces can be significantly improved with a training-based approach. However, the training procedure is time consuming and often causes fatigue. Consequently, the number of training data should be limited, which may reduce the classification performance. Thus, how to improve classification accuracy without increasing the training time is crucial to SSVEP-based BCI system. This study proposes a transfer-related component analysis (TransRCA) method for addressing the above issue. In this method, the SSVEP-related components are extracted from a small number of training data of the current individual and combined with those extracted from a large number of existing training data of other individuals. The TransRCA method maximizes not only the inter-trial covariances between the source and target subjects, but also the correlation between the reference signals and SSVEP signals from the source and target subjects. The proposed method was validated on the SSVEP public Benchmark and BETA datasets, and the classification accuracy and information transmission rate of the ensemble version of the proposed TransRCA method were compared with those of the state-of-the-art eCCA, eTRCA, ttCCA, LSTeTRCA, and eIISMC methods on both datasets. The comparison results indicate that the proposed method provides a superior performance compared with these state-of-the-art methods, and thus has high potential for the development of a SSVEP-based brain-computer interface system with high classification performance that only uses a small number of training data.

A Novel Method for Constructing EEG Large-Scale Cortical Dynamical Functional Network Connectivity (dFNC): WTCS

As a kind of biological network, the brain network conduces to understanding the mystery of high-efficiency information processing in the brain, which will provide instructions to develop efficient brain-like neural networks. Large-scale dynamical functional network connectivity (dFNC) provides a more context-sensitive, dynamical, and straightforward sight at a higher network level. Nevertheless, dFNC analysis needs good enough resolution in both temporal and spatial domains, and the construction of dFNC needs to capture the time-varying correlations between two multivariate time series with unmatched spatial dimensions. Effective methods still lack. With well-developed source imaging techniques, electroencephalogram (EEG) has the potential to possess both high temporal and spatial resolutions. Therefore, we proposed to construct the EEG large-scale cortical dFNC based on brain atlas to probe the subtle dynamic activities in the brain and developed a novel method, that is, wavelet coherence-S estimator (WTCS), to assess the dynamic couplings among functional subnetworks with different spatial dimensions. The simulation study demonstrated its robustness and availability of applying to dFNC. The application in real EEG data revealed the appealing "Primary peak" and "P3-like peak" in dFNC network properties and meaningful evolutions in dFNC network topology for P300. Our study brings new insights for probing brain activities at a more dynamical and higher hierarchical level and pushing forward the development of brain-inspired artificial neural networks. The proposed WTCS not only benefits the dFNC studies but also gives a new solution to capture the time-varying couplings between the multivariate time series that is often encountered in signal processing disciplines.

An Electric Wheelchair Manipulating System Using SSVEP-Based BCI System

Most people with motor disabilities use a joystick to control an electric wheelchair. However, those who suffer from multiple sclerosis or amyotrophic lateral sclerosis may require other methods to control an electric wheelchair. This study implements an electroencephalography (EEG)-based brain–computer interface (BCI) system and a steady-state visual evoked potential (SSVEP) to manipulate an electric wheelchair. While operating the human–machine interface, three types of SSVEP scenarios involving a real-time virtual stimulus are displayed on a monitor or mixed reality (MR) goggles to produce the EEG signals. Canonical correlation analysis (CCA) is used to classify the EEG signals into the corresponding class of command and the information transfer rate (ITR) is used to determine the effect. The experimental results show that the proposed SSVEP stimulus generates the EEG signals because of the high classification accuracy of CCA. This is used to control an electric wheelchair along a specific path. Simultaneous localization and mapping (SLAM) is the mapping method that is available in the robotic operating software (ROS) platform that is used for the wheelchair system for this study.

An efficient CNN-LSTM Network with spectral normalization and label smoothing technologies for SSVEP frequency recognition

OBJECTIVE

Steady-state visual evoked potentials(SSVEPs) based braincomputer interface(BCI) has received great interests owing to the high information transfer rate(ITR) and available large number of targets. However, the performance of frequency recognition methods heavily depends on the amount of the calibration data for intra-subject classification. Some research adopted the deep learning(DL) algorithm to conduct the inter-subject classification, which could reduce the calculation procedure, but the performance still has large room to improve compared with the intra-subject classification.

APPROACH

To address these issues, we proposed an efficient SSVEP DL NETwork (termed SSVEPNET) based on 1D convolution and long short-term memory (LSTM) module. To enhance the performance of SSVEPNT, we adopted the spectral normalization and label smoothing technologies during implementing the network architecture. We evaluated the SSVEPNET and compared it with other methods for the intra- and inter-subject classification under different conditions, i.e., two datasets, two time-window lengths (1 s and 0.5 s), three sizes of training data.

MAIN RESULTS

Under all the experimental settings, the proposed SSVEPNET achieved the highest average accuracy for the intra- and inter-subject classification on the two SSVEP datasets, when compared with other traditional and DL baseline methods. Signif icance. The extensive experimental results demonstrate that the proposed DL model holds promise to enhance frequency recognition performance in SSVEP-based BCIs. Besides, the mixed network structures with CNN and LSTM, and the spectral normalization and label smoothing could be useful optimization strategies to design efficient models for EEG data.

Boosting the Evoked Response of Brain to Enhance the Reference Signals of CCA Method

Brain-computer interface (BCI) systems can be used to communicate and express desires from people with severe nervous system damage. Among BCI systems based on evoked responses, steady state visual evoked potential (SSVEP) responses are the most widely used. Canonical correlation analysis (CCA)-based methods have been widely used in SSVEP-based online BCIs due to their low computation and high speed, and many methods have been introduced to improve the results. In this research, a method for constructing reference signals used in CCA based on the amplified evoked response of brain is introduced. In the proposed method, after removing the latency in the training signals, to construct reference signals, multilayer perceptron neural networks of the fitting type are used instead of the usual sine/cosine signals. The results show the success of this method in boosting the evoked responses of brain. The detection accuracy in 100-second time windows was 100%, and the information transfer rate in the same period was 240 bits per minute. Making reference signals similar to the recorded electroencephalogram allowed us to make more similarities in the CCA between the signals under consideration, and the reference signals, and to dramatically improve the results.

An improved cross-subject spatial filter transfer method for SSVEP-based BCI

Steady-state visual evoked potential (SSVEP) training feature recognition algorithms utilize user training data to reduce the interference of spontaneous electroencephalogram (EEG) activities on SSVEP response for improved recognition accuracy. The data collection process can be tedious, increasing the mental fatigue of users and also seriously affecting the practicality of SSVEP-based brain-computer interface (BCI) systems. As an alternative, a cross-subject spatial filter transfer (CSSFT) method to transfer an existing user data model with good SSVEP response to new user test data has been proposed. The CSSFT method uses superposition averages of data for multiple blocks of data as transfer data. However, the amplitude and pattern of brain signals are often significantly different across trials. The goal of this study was to improve superposition averaging for the CSSFT method and propose an Ensemble scheme based on ensemble learning, and an Expansion scheme based on matrix expansion. The feature recognition performance was compared for CSSFT and the proposed improved CSSFT method using two public datasets. The results demonstrated that the improved CSSFT method can significantly improve the recognition accuracy and information transmission rate of existing methods. This strategy avoids a tedious data collection process, and promotes the potential practical application of BCI systems.

Overview of the winning approaches in BCI Controlled Robot Contest in World Robot Contest 2021: Calibration-free SSVEP

Recently, steady-state visual evoked potential (SSVEP) has become one of the most popular electroencephalography paradigms due to its high information transfer rate. Several approaches have been proposed to improve the performance of SSVEP. The calibration-free scenario is significant in SSVEP-based brain–computer interface systems, where the subject is the first time to use the system. The participating teams proposed several effective calibration-free algorithm frameworks in the SSVEP competition (calibration-free) of the BCI Controlled Robot Contest in World Robot Contest 2021. This paper introduces the approaches used in the algorithms of the top five teams in the final. The results of the five subjects in the final proved the effectiveness of the approaches. This paper discusses the effectiveness of each approach in improving the system performance in the calibration-free scenario and gives suggestions on how to use these approaches in a real-world system.

eldBETA: A Large Eldercare-oriented Benchmark Database of SSVEP-BCI for the Aging Population

Global population aging poses an unprecedented challenge and calls for a rising effort in eldercare and healthcare. Steady-state visual evoked potential based brain-computer interface (SSVEP-BCI) boasts its high transfer rate and shows great promise in real-world applications to support aging. Public database is critically important for designing the SSVEP-BCI systems. However, the SSVEP-BCI database tailored for the elder is scarce in existing studies. Therefore, in this study, we present a large eldercare-oriented BEnchmark database of SSVEP-BCI for The Aging population (eldBETA). The eldBETA database consisted of the 64-channel electroencephalogram (EEG) from 100 elder participants, each of whom performed seven blocks of 9-target SSVEP-BCI task. The quality and characteristics of the eldBETA database were validated by a series of analyses followed by a classification analysis of thirteen frequency recognition methods. We expect that the eldBETA database would provide a substrate for the design and optimization of the BCI systems intended for the elders. The eldBETA database is open-access for research and can be downloaded from the website 10.6084/m9.figshare.18032669.

Measurement(s)	Steady-state visual evoked potential (SSVEP)
Technology Type(s)	Electroencephalography (EEG)
Factor Type(s)	Elder population
Sample Characteristic - Organism	Homo sapiens
Sample Characteristic - Environment	Electromagnetic shielding room

Cross-subject spatial filter transfer method for SSVEP-EEG feature recognition

Objective.Steady-state visual evoked potential (SSVEP) is an important control method of the brain-computer interface (BCI) system. The development of an efficient SSVEP feature decoding algorithm is the core issue in SSVEP-BCI. It has been proposed to use user training data to reduce the spontaneous electroencephalogram activity interference on SSVEP response, thereby improving the feature recognition accuracy of the SSVEP signal. Nevertheless, the tedious data collection process increases the mental fatigue of the user and severely affects the applicability of the BCI system.Approach.A cross-subject spatial filter transfer (CSSFT) method that transfer the existing user model with good SSVEP response to the new user test data without collecting any training data from the new user is proposed.Main results.Experimental results demonstrate that the transfer model increases the distinction of the feature discriminant coefficient between the gaze following target and the non-gaze following target and accurately identifies the wrong target in the fundamental algorithm model. The public datasets show that the CSSFT method significantly increases the recognition performance of canonical correlation analysis (CCA) and filter bank CCA. Additionally, when the data used to calculate the transfer model contains one data block only, the CSSFT method retains its effective feature recognition capabilities.Significance.The proposed method requires no tedious data calibration process for new users, provides an effective technical solution for the transfer of the cross-subject model, and has potential application value for promoting the application of the BCI system.

An MVMD-CCA Recognition Algorithm in SSVEP-Based BCI and Its Application in Robot Control

This article proposes a novel recognition algorithm for the steady-state visual evoked potentials (SSVEP)-based brain-computer interface (BCI) system. By combining the advantages of multivariate variational mode decomposition (MVMD) and canonical correlation analysis (CCA), an MVMD-CCA algorithm is investigated to improve the detection ability of SSVEP electroencephalogram (EEG) signals. In comparison with the classical filter bank canonical correlation analysis (FBCCA), the nonlinear and non-stationary EEG signals are decomposed into a fixed number of sub-bands by MVMD, which can enhance the effect of SSVEP-related sub-bands. The experimental results show that MVMD-CCA can effectively reduce the influence of noise and EEG artifacts and improve the performance of SSVEP-based BCI. The offline experiments show that the average accuracies of MVMD-CCA in the training dataset and testing dataset are improved by 3.08% and 1.67%, respectively. In the SSVEP-based online robotic manipulator grasping experiment, the recognition accuracies of the four subjects are 92.5%, 93.33%, 90.83%, and 91.67%, respectively.

SSVEP-EEG Feature Enhancement Method Using an Image Sharpening Filter

Steady-state visual evoked potential (SSVEP) is widely used in brain computer interface (BCI), medical detection, and neuroscience, so there is significant interest in enhancing SSVEP features via signal processing for better performance. In this study, an image processing method was combined with brain signal analysis and a sharpening filter was used to extract image details and features for the enhancement of SSVEP features. The results demonstrated that sharpening filter could eliminate the SSVEP signal trend term and suppress its low-frequency component. Meanwhile, sharpening filter effectively enhanced the signal-to-noise ratios (SNRs) of the single-channel and multi-channel fused signals. Image sharpening filter also significantly improved the recognition accuracy of canonical correlation analysis (CCA), filter bank canonical correlation analysis (FBCCA), and task-related component analysis (TRCA). The tools developed here effectively enhanced the SSVEP signal features, suggesting that image processing methods can be considered for improved brain signal analysis.

Multivariate synchronization curve: A measure of synchronization in different multivariate signals

As a method to measure the synchronization between two different sets of signals, the multivariate synchronization index (MSI) has played an irreplaceable role in the field of frequency recognition of brain-computer interface since it was proposed. On this basis, we make a generalization of MSI by using the escort distribution to replace the original distribution. In this way, MSI can be converted from a determined value to the multivariate synchronization curve, which will vary as the parameter q of the escort distribution changes. Numerical experiments are carried out on both simulated and real-world data to confirm the effectiveness of this new method. Compared with the case of MSI (i.e., q = 1), the extended form of MSI proposed in this article can obviously capture the relationship between signals more comprehensively, implying that it is a more perfect method to describe the synchronization between them. The results reveal that this method can not only effectively extract the important information contained in different signals, but also has the potential to become a practical synchronization measurement method of multivariate signals.

Influence of the Number of Channels and Classification Algorithm on the Performance Robustness to Electrode Shift in Steady-State Visual Evoked Potential-Based Brain-Computer Interfaces

There remains an active investigation on elevating the classification accuracy and information transfer rate of brain-computer interfaces based on steady-state visual evoked potential. However, it has often been ignored that the performance of steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) can be affected through the minor displacement of the electrodes from their optimal locations in practical applications because of the mislocation of electrodes and/or concurrent use of electroencephalography (EEG) devices with external devices, such as virtual reality headsets. In this study, we evaluated the performance robustness of SSVEP-based BCIs with respect to the changes in electrode locations for various channel configurations and classification algorithms. Our experiments involved 21 participants, where EEG signals were recorded from the scalp electrodes densely attached to the occipital area of the participants. The classification accuracies for all the possible cases of electrode location shifts for various channel configurations (1–3 channels) were calculated using five training-free SSVEP classification algorithms, i.e., the canonical correlation analysis (CCA), extended CCA, filter bank CCA, multivariate synchronization index (MSI), and extended MSI (EMSI). Then, the performances of the BCIs were evaluated using two measures, i.e., the average classification accuracy (ACA) across the electrode shifts and robustness to the electrode shift (RES). Our results showed that the ACA increased with an increase in the number of channels regardless of the algorithm. However, the RES was enhanced with an increase in the number of channels only when MSI and EMSI were employed. While both ACA and RES values for the five algorithms were similar under the single-channel condition, both ACA and RES values for MSI and EMSI were higher than those of the other algorithms under the multichannel (i.e., two or three electrodes) conditions. In addition, EMSI outperformed MSI when comparing the ACA and RES values under the multichannel conditions. In conclusion, our results suggested that the use of multichannel configuration and employment of EMSI could make the performance of SSVEP-based BCIs more robust to the electrode shift from the optimal locations.

Filter bank approach for enhancement of supervised Canonical Correlation Analysis methods for SSVEP-based BCI spellers

Canonical correlation analysis (CCA) is one of the most used algorithms in the steady-state visual evoked potentials (SSVEP)-based brain-computer interface (BCI) systems due to its simplicity, efficiency, and robustness. Researchers have proposed modifications to CCA to improve its speed, allowing high-speed spelling and thus a more natural communication. In this work, we combine two approaches, the filter-bank (FB) approach to extract more information from the harmonics, and a range of different supervised methods which optimize the reference signals to improve the SSVEP detection. The proposed models are tested on the publicly available benchmark dataset for SSVEP-based BCIs and the results show improved performance compared to the state-of-the-art methods and, in particular, the proposed FBMwayCCA approach achieves the best results with an information transfer rate (ITR) of 134.8±8.4 bits/minute. This study indeed suggests the feasibility of combining the fundamental and harmonic SSVEP components with supervised methods in target identification to develop high-speed BCI spellers.

Multi-scale noise transfer and feature frequency detection in SSVEP based on FitzHugh-Nagumo neuron system

Objective. The steady-state visual evoked potential (SSVEP) is one of the most commonly used control signals for brain-computer interfaces (BCIs) due to its excellent interactive potential, such as high tolerance to noises and robust performance across users. In addition, it has a stable cycle, obvious characteristics and minimal training requirements. However, the SSVEP is extremely weak and companied with strong and multi-scale noise, resulting in a poor signal-to-noise ratio in practice. Common algorithms for classification are based on the principle of template matching and spatial filtering, which cannot obtain satisfied performance of SSVEP detection under the multi-scale noise. Therefore, using linear methods to extract SSVEP with obvious nonlinear and non-stationary characteristics, the useful signal will be attenuated or lost.Approach.To address this issue, two novel frameworks based on a two-dimensional nonlinear FitzHugh-Nagumo (FHN) neuron system are proposed to extract feature frequency of SSVEP.Results.In order to evaluate the effectiveness of the proposed methods, this research recruit 22 subjects to participate the experiment. Experimental results show that nonlinear FHN neuron model can force the energy of noise to be transferred into SSVEP and hence amplifying the amplitude of the target frequency. Compared with the traditional methods, the FHN and FHNCCA methods can achieve higher classification accuracy and faster processing speed, which effectively improves the information transmission rate of SSVEP-based BCI.

Improving the Performance of Individually Calibrated SSVEP-BCI by Task- Discriminant Component Analysis

A brain-computer interface (BCI) provides a direct communication channel between a brain and an external device. Steady-state visual evoked potential based BCI (SSVEP-BCI) has received increasing attention due to its high information transfer rate, which is accomplished by individual calibration for frequency recognition. Task-related component analysis (TRCA) is a recent and state-of-the-art method for individually calibrated SSVEP-BCIs. However, in TRCA, the spatial filter learned from each stimulus may be redundant and temporal information is not fully utilized. To address this issue, this paper proposes a novel method, i.e., task-discriminant component analysis (TDCA), to further improve the performance of individually calibrated SSVEP-BCI. The performance of TDCA was evaluated by two publicly available benchmark datasets, and the results demonstrated that TDCA outperformed ensemble TRCA and other competing methods by a significant margin. An offline and online experiment testing 12 subjects further validated the effectiveness of TDCA. The present study provides a new perspective for designing decoding methods in individually calibrated SSVEP-BCI and presents insight for its implementation in high-speed brain speller applications.

Enhancing Performance of SSVEP-Based Visual Acuity via Spatial Filtering

The purpose of this study was to enhance the performance of steady-state visual evoked potential (SSVEP)-based visual acuity assessment with spatial filtering methods. Using the vertical sinusoidal gratings at six spatial frequency steps as the visual stimuli for 11 subjects, SSVEPs were recorded from six occipital electrodes (O1, Oz, O2, PO3, POz, and PO4). Ten commonly used training-free spatial filtering methods, i.e., native combination (single-electrode), bipolar combination, Laplacian combination, average combination, common average reference (CAR), minimum energy combination (MEC), maximum contrast combination (MCC), canonical correlation analysis (CCA), multivariate synchronization index (MSI), and partial least squares (PLS), were compared for multielectrode signals combination in SSVEP visual acuity assessment by statistical analyses, e.g., Bland–Altman analysis and repeated-measures ANOVA. The SSVEP signal characteristics corresponding to each spatial filtering method were compared, determining the chosen spatial filtering methods of CCA and MSI with a higher performance than the native combination for further signal processing. After the visual acuity threshold estimation criterion, the agreement between the subjective Freiburg Visual Acuity and Contrast Test (FrACT) and SSVEP visual acuity for the native combination (0.253 logMAR), CCA (0.202 logMAR), and MSI (0.208 logMAR) was all good, and the difference between FrACT and SSVEP visual acuity was also all acceptable for the native combination (−0.095 logMAR), CCA (0.039 logMAR), and MSI (−0.080 logMAR), where CCA-based SSVEP visual acuity had the best performance and the native combination had the worst. The study proved that the performance of SSVEP-based visual acuity can be enhanced by spatial filtering methods of CCA and MSI and also recommended CCA as the spatial filtering method for multielectrode signals combination in SSVEP visual acuity assessment.

A Voting-Enhanced Dynamic-Window-Length Classifier for SSVEP-Based BCIs

We present a dynamic window-length classifier for steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) that does not require the user to choose a feature extraction method or channel set. Instead, the classifier uses multiple feature extraction methods and channel selections to infer the SSVEP and relies on majority voting to pick the most likely target. The classifier extends the window length dynamically if no target obtains the majority of votes. Compared with existing solutions, our classifier: (i) does not assume that any single feature extraction method will consistently outperform the others; (ii) adapts the channel selection to individual users or tasks; (iii) uses dynamic window lengths; (iv) is unsupervised (i.e., does not need training). Collectively, these characteristics make the classifier easy-to-use, especially for caregivers and others with limited technical expertise. We evaluated the performance of our classifier on a publicly available benchmark dataset from 35 healthy participants. We compared the information transfer rate (ITR) of this new classifier to those of the minimum energy combination (MEC), maximum synchronization index (MSI), and filter bank canonical correlation analysis (FBCCA). The new classifier increases average ITR to 123.5 bits-per-minute (bpm), 47.5, 51.2, and 19.5 bpm greater than the MEC, MSI, and FBCCA classifiers, respectively.

SSVEP-EEG Denoising via Image Filtering Methods

Steady-state visual evoked potential (SSVEP) is widely used in electroencephalogram (EEG) control, medical detection, cognitive neuroscience, and other fields. However, successful application requires improving the detection performance of SSVEP signal frequency characteristics. Most strategies to enhance the signal-to-noise ratio of SSVEP utilize application of a spatial filter. Here, we propose a method for image filtering denoising (IFD) of the SSVEP signal from the perspective of image analysis, as a preprocessing step for signal analysis. Arithmetic mean, geometric mean, Gaussian, and non-local means filtering methods were tested, and the experimental results show that image filtering of SSVEP cannot effectively remove the noise. Thus, we proposed a reverse solution in which the SSVEP noise signal was obtained by image filtering, and then the noise was subtracted from the original signal. Comparison of the recognition accuracy of the SSVEP signal before and after denoising was used to evaluate the denoising performance for stimuli of different duration. After IFD processing, SSVEP exhibited higher recognition accuracy, indicating the effectiveness of this proposed denoising method. Application of this method improves the detection performance of SSVEP signal frequency characteristics, combines image processing and brain signal analysis, and expands the research scope of brain signal analysis for widespread application.

Enhancing P300 based character recognition performance using a combination of ensemble classifiers and a fuzzy fusion method

BACKGROUND

P300-based brain-computer interfaces provide communication pathways without the need for muscle activity by recognizing electrical signals from the brain. The P300 speller is one of the most commonly used BCI applications, as it is very simple and reliable, and it is capable of reaching satisfactory communication performance. However, as with other BCIs, it remains a challenge to improve the P300 speller's performance to increase its practical usability.

NEW METHODS

In this study, we propose a novel multi-feature subset fuzzy fusion (MSFF) framework for the P300 speller to recognize the users' spelling intention. This method includes two parts: 1) feature selection by the Lasso algorithm and feature division; 2) the construction of ensemble LDA classifiers and the fuzzy fusion of those classifiers to recognize user intention.

RESULTS

The proposed framework is evaluated in three public datasets and achieves an average accuracy of 100% after 4 epochs for BCI Competition II Dataset IIb, 96% for BCI Competition III dataset II and 98.3% for the BNCI Horizon Dataset. It indicates that the proposed MSFF method can make use of temporal information of signals and helps to enhance classification performance.

COMPARISON WITH EXISTING METHODS

The proposed MSFF method yields better or comparable performance than previously reported machine learning algorithms.

CONCLUSIONS

The proposed MSFF method is able to improve the performance of P300-based BCIs.

Similarity-constrained task-related component analysis for enhancing SSVEP detection

Objective. Task-related component analysis (TRCA) is a representative subject-specific training algorithm in steady-state visual evoked potential (SSVEP)-based brain-computer interfaces. Task-related components (TRCs), extracted by the TRCA-based spatial filtering from electroencephalogram (EEG) signals through maximizing the reproducibility across trials, may contain some task-related inherent noise that is still trial-reproducible.Approach. To address this problem, this study proposed a similarity-constrained TRCA (scTRCA) algorithm to remove the task-related noise and extract TRCs maximally correlated with SSVEPs for enhancing SSVEP detection. Similarity constraints, which were created by introducing covariance matrices between EEG training data and an artificial SSVEP template, were added to the objective function of TRCA. Therefore, a better spatial filter was obtained by maximizing not only the reproducibility across trials but also the similarity between TRCs and SSVEPs. The proposed scTRCA was compared with TRCA, multi-stimulus TRCA, and sine-cosine reference signal based on two public datasets.Main results. The performance of TRCA in target identification of SSVEPs is improved by introducing similarity constraints. The proposed scTRCA significantly outperformed the other three methods, and the improvement was more significant especially with insufficient training data.Significance. The proposed scTRCA algorithm is promising for enhancing SSVEP detection considering its better performance and robustness against insufficient calibration.

Weak Feature Extraction and Strong Noise Suppression for SSVEP-EEG Based on Chaotic Detection Technology

Brain computer interface (BCI) is a novel communication method that does not rely on the normal neural pathway between the brain and muscle of human. It can transform mental activities into relevant commands to control external equipment and establish direct communication pathway. Among different paradigms, steady-state visual evoked potential (SSVEP) is widely used due to its certain periodicity and stability of control. However, electroencephalogram (EEG) of SSVEP is extremely weak and companied with multi-scale and strong noise. Existing algorithms for classification are based on the principle of template matching and spatial filtering, which cannot obtain satisfied performance of feature extraction under the multi-scale noise. Especially for the subjects produce weak response for external stimuli in EEG representation, i.e., BCI-Illiteracy subject, traditional algorithms are difficult to recognize the internal patterns of brain. To address this issue, a novel method based on Chaos theory is proposed to extract feature of SSVEP. The rule of this method is applying the peculiarity of nonlinear dynamics system to detect feature of SSVEP by judging the state changes of chaotic systems after adding weak EEG. To evaluate the validity of proposed method, this research recruit 32 subjects to participate the experiment. All subjects are divided into two groups according to the preliminary classification accuracy (mean acc >70% or < 70%) by canonical correlation analysis and we define the accuracy above 70% as group A (normal subjects), below 70% as group B (BCI-Illiteracy). Then, the classification accuracy and information transmission rate of two groups are verified using Chaotic theory. Experimental results show that all classification methods using in our study achieve good performance for normal subjects while chaos obtain excellent performance and significant improvements than traditional methods for BCI-Illiteracy.

Filter Bank-Driven Multivariate Synchronization Index for Training-Free SSVEP BCI

In recent years, multivariate synchronization index (MSI) algorithm, as a novel frequency detection method, has attracted increasing attentions in the study of brain-computer interfaces (BCIs) based on steady state visual evoked potential (SSVEP). However, MSI algorithm is hard to fully exploit SSVEP-related harmonic components in the electroencephalogram (EEG), which limits the application of MSI algorithm in BCI systems. In this paper, we propose a novel filter bank-driven MSI algorithm (FBMSI) to overcome the limitation and further improve the accuracy of SSVEP recognition. We evaluate the efficacy of the FBMSI method by developing a 6-command SSVEP-NAO robot system with extensive experimental analyses. An offline experimental study is first performed with EEG collected from nine subjects to investigate the effects of varying parameters on the model performance. Offline results show that the proposed method has achieved a stable improvement effect. We further conduct an online experiment with six subjects to assess the efficacy of the developed FBMSI algorithm in a real-time BCI application. The online experimental results show that the FBMSI algorithm yields a promising average accuracy of 83.56% using a data length of even only one second, which was 12.26% higher than the standard MSI algorithm. These extensive experimental results confirmed the effectiveness of the FBMSI algorithm in SSVEP recognition and demonstrated its potential application in the development of improved BCI systems.

Brain-Computer Interface Speller Based on Steady-State Visual Evoked Potential: A Review Focusing on the Stimulus Paradigm and Performance

The steady-state visual evoked potential (SSVEP), measured by the electroencephalograph (EEG), has high rates of information transfer and signal-to-noise ratio, and has been used to construct brain-computer interface (BCI) spellers. In BCI spellers, the targets of alphanumeric characters are assigned different visual stimuli and the fixation of each target generates a unique SSVEP. Matching the SSVEP to the stimulus allows users to select target letters and numbers. Many BCI spellers that harness the SSVEP have been proposed over the past two decades. Various paradigms of visual stimuli, including the procedure of target selection, layout of targets, stimulus encoding, and the combination with other triggering methods are used and considered to influence on the BCI speller performance significantly. This paper reviews these stimulus paradigms and analyzes factors influencing their performance. The fundamentals of BCI spellers are first briefly described. SSVEP-based BCI spellers, where only the SSVEP is used, are classified by stimulus paradigms and described in chronological order. Furthermore, hybrid spellers that involve the use of the SSVEP are presented in parallel. Factors influencing the performance and visual fatigue of BCI spellers are provided. Finally, prevailing challenges and prospective research directions are discussed to promote the development of BCI spellers.

Cross-Subject Assistance: Inter- and Intra-Subject Maximal Correlation for Enhancing the Performance of SSVEP-Based BCIs

OBJECTIVE

The current state-of-the-art methods significantly improve the detection performance of the steady-state visual evoked potentials (SSVEPs) by using the individual calibration data. However, the time-consuming calibration sessions limit the number of training trials and may give rise to visual fatigue, which weakens the effectiveness of the individual training data. For addressing this issue, this study proposes a novel inter- and intra-subject maximal correlation (IISMC) method to enhance the robustness of SSVEP recognition via employing the inter- and intra-subject similarity and variability. Through efficient transfer learning, similar experience under the same task is shared across subjects.

METHODS

IISMC extracts subject-specific information and similar task-related information from oneself and other subjects performing the same task by maximizing the inter- and intra-subject correlation. Multiple weak classifiers are built from several existing subjects and then integrated to construct the strong classifiers by the average weighting. Finally, a powerful fusion predictor is obtained for target recognition.

RESULTS

The proposed framework is validated on a benchmark data set of 35 subjects, and the experimental results demonstrate that IISMC obtains better performance than the state of the art task-related component analysis (TRCA).

SIGNIFICANCE

The proposed method has great potential for developing high-speed BCIs.

An Introductory Tutorial on Brain–Computer Interfaces and Their Applications

The prospect and potentiality of interfacing minds with machines has long captured human imagination. Recent advances in biomedical engineering, computer science, and neuroscience are making brain–computer interfaces a reality, paving the way to restoring and potentially augmenting human physical and mental capabilities. Applications of brain–computer interfaces are being explored in applications as diverse as security, lie detection, alertness monitoring, gaming, education, art, and human cognition augmentation. The present tutorial aims to survey the principal features and challenges of brain–computer interfaces (such as reliable acquisition of brain signals, filtering and processing of the acquired brainwaves, ethical and legal issues related to brain–computer interface (BCI), data privacy, and performance assessment) with special emphasis to biomedical engineering and automation engineering applications. The content of this paper is aimed at students, researchers, and practitioners to glimpse the multifaceted world of brain–computer interfacing.

Enhancing detection of steady-state visual evoked potentials using channel ensemble method

OBJECTIVE

This study proposed and evaluated a channel ensemble approach to enhance detection of steady-state visual evoked potentials (SSVEPs).

APPROACH

Collected multi-channel electroencephalogram (EEG) signals were classified into multiple groups of new analysis signals based on correlation analysis, and each group of analysis signals contained signals from a different number of electrode channels. These groups of analysis signals were used as the input of a training-free feature extraction model, and the obtained feature coefficients were converted into feature probability values using the softmax function. The ensemble value of multiple sets of feature probability values was determined and used as the final discrimination coefficient.

MAIN RESULTS

Compared with canonical correlation analysis (CCA), likelihood ratio test (LRT), and multivariate synchronization index (MSI) analysis methods using a standard approach, the recognition accuracies of the methods using a channel ensemble approach were improved by 5.05%, 3.87%, and 3.42%, and the information transfer rates (ITRs) were improved by 6.00%, 4.61%, and 3.71%, respectively. The channel ensemble method also obtained better recognition results than the standard algorithm on the public dataset. This study validated the efficiency of the proposed method to enhance the detection of SSVEPs, demonstrating its potential use in practical brain-computer interface (BCI) systems.

SIGNIFICANCE

A SSVEP-based BCI system using a channel ensemble method could achieve high ITR, indicating great potential of this design for various applications with improved control and interaction.