Filter Bank Convolutional Neural Network for Short Time-Window Steady-State Visual Evoked Potential Classification

Filter banks guided correlational convolutional neural network for SSVEPs based BCI classification

Objective.In the field of steady-state visual evoked potential brain computer interfaces (SSVEP-BCIs) research, convolutional neural networks (CNNs) have gradually been proved to be an effective method. Whereas, majority works apply the frequency domain characteristics in long time window to train the network, thus lead to insufficient performance of those networks in short time window. Furthermore, only the frequency domain information for classification lacks of other task-related information.Approach.To address these issues, we propose a time-frequency domain generalized filter-bank convolutional neural network (FBCNN-G) to improve the SSVEP-BCIs classification performance. The network integrates multiple frequency information of electroencephalogram (EEG) with template and predefined prior of sine-cosine signals to perform feature extraction, which contains correlation analyses in both template and signal aspects. Then the classification is performed at the end of the network. In addition, the method proposes the use of filter banks divided into specific frequency bands as pre-filters in the network to fully consider the fundamental and harmonic frequency characteristics of the signal.Main results.The proposed FBCNN-G model is compared with other methods on the public dataset Benchmark. The results manifest that this model has higher accuracy of character recognition accuracy and information transfer rates in several time windows. Particularly, in the 0.2 s time window, the mean accuracy of the proposed method reaches62.02%±5.12%, indicating its superior performance.Significance.The proposed FBCNN-G model is critical for the exploitation of SSVEP-BCIs character recognition models.

A Convolutional Neural Network for SSVEP Identification by Using a Few-Channel EEG

The application of wearable electroencephalogram (EEG) devices is growing in brain-computer interfaces (BCI) owing to their good wearability and portability. Compared with conventional devices, wearable devices typically support fewer EEG channels. Devices with few-channel EEGs have been proven to be available for steady-state visual evoked potential (SSVEP)-based BCI. However, fewer-channel EEGs can cause the BCI performance to decrease. To address this issue, an attention-based complex spectrum-convolutional neural network (atten-CCNN) is proposed in this study, which combines a CNN with a squeeze-and-excitation block and uses the spectrum of the EEG signal as the input. The proposed model was assessed on a wearable 40-class dataset and a public 12-class dataset under subject-independent and subject-dependent conditions. The results show that whether using a three-channel EEG or single-channel EEG for SSVEP identification, atten-CCNN outperformed the baseline models, indicating that the new model can effectively enhance the performance of SSVEP-BCI with few-channel EEGs. Therefore, this SSVEP identification algorithm based on a few-channel EEG is particularly suitable for use with wearable EEG devices.

Dynamic decomposition graph convolutional neural network for SSVEP-based brain-computer interface

The SSVEP-based paradigm serves as a prevalent approach in the realm of brain-computer interface (BCI). However, the processing of multi-channel electroencephalogram (EEG) data introduces challenges due to its non-Euclidean characteristic, necessitating methodologies that account for inter-channel topological relations. In this paper, we introduce the Dynamic Decomposition Graph Convolutional Neural Network (DDGCNN) designed for the classification of SSVEP EEG signals. Our approach incorporates layerwise dynamic graphs to address the oversmoothing issue in Graph Convolutional Networks (GCNs), employing a dense connection mechanism to mitigate the gradient vanishing problem. Furthermore, we enhance the traditional linear transformation inherent in GCNs with graph dynamic fusion, thereby elevating feature extraction and adaptive aggregation capabilities. Our experimental results demonstrate the effectiveness of proposed approach in learning and extracting features from EEG topological structure. The results shown that DDGCNN outperforms other state-of-the-art (SOTA) algorithms reported on two datasets (Dataset 1: 54 subjects, 4 targets, 2 sessions; Dataset 2: 35 subjects, 40 targets). Additionally, we showcase the implementation of DDGCNN in the context of synchronized BCI robotic fish control. This work represents a significant advancement in the field of EEG signal processing for SSVEP-based BCIs. Our proposed method processes SSVEP time domain signals directly as an end-to-end system, making it easy to deploy. The code is available at https://github.com/zshubin/DDGCNN.

FB-CCNN: A Filter Bank Complex Spectrum Convolutional Neural Network with Artificial Gradient Descent Optimization

The brain-computer interface (BCI) provides direct communication between human brains and machines, including robots, drones and wheelchairs, without the involvement of peripheral systems. BCI based on electroencephalography (EEG) has been applied in many fields, including aiding people with physical disabilities, rehabilitation, education and entertainment. Among the different EEG-based BCI paradigms, steady-state visual evoked potential (SSVEP)-based BCIs are known for their lower training requirements, high classification accuracy and high information transfer rate (ITR). In this article, a filter bank complex spectrum convolutional neural network (FB-CCNN) was proposed, and it achieved leading classification accuracies of 94.85 ± 6.18% and 80.58 ± 14.43%, respectively, on two open SSVEP datasets. An optimization algorithm named artificial gradient descent (AGD) was also proposed to generate and optimize the hyperparameters of the FB-CCNN. AGD also revealed correlations between different hyperparameters and their corresponding performances. It was experimentally demonstrated that FB-CCNN performed better when the hyperparameters were fixed values rather than channel number-based. In conclusion, a deep learning model named FB-CCNN and a hyperparameter-optimizing algorithm named AGD were proposed and demonstrated to be effective in classifying SSVEP through experiments. The hyperparameter design process and analysis were carried out using AGD, and advice on choosing hyperparameters for deep learning models in classifying SSVEP was provided.

An Analysis of Deep Learning Models in SSVEP-Based BCI: A Survey

The brain-computer interface (BCI), which provides a new way for humans to directly communicate with robots without the involvement of the peripheral nervous system, has recently attracted much attention. Among all the BCI paradigms, BCIs based on steady-state visual evoked potentials (SSVEPs) have the highest information transfer rate (ITR) and the shortest training time. Meanwhile, deep learning has provided an effective and feasible solution for solving complex classification problems in many fields, and many researchers have started to apply deep learning to classify SSVEP signals. However, the designs of deep learning models vary drastically. There are many hyper-parameters that influence the performance of the model in an unpredictable way. This study surveyed 31 deep learning models (2011-2023) that were used to classify SSVEP signals and analyzed their design aspects including model input, model structure, performance measure, etc. Most of the studies that were surveyed in this paper were published in 2021 and 2022. This survey is an up-to-date design guide for researchers who are interested in using deep learning models to classify SSVEP signals.

An efficient CNN-LSTM Network with spectral normalization and label smoothing technologies for SSVEP frequency recognition

OBJECTIVE

Steady-state visual evoked potentials(SSVEPs) based braincomputer interface(BCI) has received great interests owing to the high information transfer rate(ITR) and available large number of targets. However, the performance of frequency recognition methods heavily depends on the amount of the calibration data for intra-subject classification. Some research adopted the deep learning(DL) algorithm to conduct the inter-subject classification, which could reduce the calculation procedure, but the performance still has large room to improve compared with the intra-subject classification.

APPROACH

To address these issues, we proposed an efficient SSVEP DL NETwork (termed SSVEPNET) based on 1D convolution and long short-term memory (LSTM) module. To enhance the performance of SSVEPNT, we adopted the spectral normalization and label smoothing technologies during implementing the network architecture. We evaluated the SSVEPNET and compared it with other methods for the intra- and inter-subject classification under different conditions, i.e., two datasets, two time-window lengths (1 s and 0.5 s), three sizes of training data.

MAIN RESULTS

Under all the experimental settings, the proposed SSVEPNET achieved the highest average accuracy for the intra- and inter-subject classification on the two SSVEP datasets, when compared with other traditional and DL baseline methods. Signif icance. The extensive experimental results demonstrate that the proposed DL model holds promise to enhance frequency recognition performance in SSVEP-based BCIs. Besides, the mixed network structures with CNN and LSTM, and the spectral normalization and label smoothing could be useful optimization strategies to design efficient models for EEG data.

Simultaneous sEMG Recognition of Gestures and Force Levels for Interaction With Prosthetic Hand

The natural interaction between the prosthetic hand and the upper limb amputation patient is important and directly affects the rehabilitation effect and operation ability. Most previous studies only focused on the interaction of gestures but ignored the force levels. This paper proposes a simultaneous recognition method of gestures and forces for interaction with a prosthetic hand. The multitask classification algorithm based on a convolutional neural network (CNN) is designed to improve recognition efficiency and ensure recognition accuracy. The offline experimental results show that the algorithm proposed in this study outperforms other methods in both training speed and accuracy. To prove the effectiveness of the proposed method, a myoelectric prosthetic hand integrated with tactile sensors is developed, and surface electromyography (sEMG) datasets of healthy persons and amputees are built. The online experimental results show that the amputee can control the prosthetic hand to continuously make gestures under different force levels, and the effect of hand coordination on the hand perception of amputees is explored. The results show that gesture classification operation tasks with different force levels based on sEMG signals can be accurately recognized and comfortably interact with prosthetic hands in real time. It improves the amputees' operation ability and relieves their muscle fatigue.

Bidirectional siamese correlation analysis method for enhancing the detection of SSVEPs

OBJECTIVE

Steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) have attracted increasing attention due to their high information transfer rate. To improve the performance of SSVEP detection, we propose a bidirectional Siamese correlation analysis (bi-SiamCA) model.

APPROACH

In this model, an LSTM-based Siamese architecture is designed to measure the similarity between the SSVEP signal and the template in each frequency and obtain the probability that the SSVEP signal belongs to each frequency. Additionally, a maximize agreement module with a designed contrastive loss is adopted in the Siamese architecture to increase the similarity between the SSVEP signal and the reference signal in the same frequency. Moreover, a two-way signal processing mechanism is built to effectively integrate complementary information from two temporal directions of the input signals. Our model uses raw SSVEPs as inputs and can be trained end-to-end.

MAIN RESULTS

Experimental results on a 40-class dataset and a 12-class dataset indicate that bi-SiamCA can significantly improve the classification accuracy compared with the prominent traditional and deep learning methods, especially under short data lengths. Feature visualizations show that the similarity between the SSVEP signal and the reference signal in the same frequency gradually improved in our model.

CONCLUSION

The proposed bi-SiamCA model enhances the performance of SSVEP detection and outperforms the compared methods.

SIGNIFICANCE

Due to its high decoding accuracy under short signals, our approach has great potential to implement a high-speed SSVEP-based BCI.

FB-EEGNet: A fusion neural network across multi-stimulus for SSVEP target detection

BACKGROUND

Steady-state visual evoked potential (SSVEP) is a prevalent paradigm of brain-computer interface (BCI). Recently, deep neural networks (DNNs) have been employed for SSVEP target recognition. However, current DNN models can not fully extract information from SSVEP harmonic components, and ignore the influence of non-target stimuli.

NEW METHOD

To employ information of multiple sub-bands and non-target stimulus data, we propose a DNN model for SSVEP target detection, i.e., FB-EEGNet, which fuses features of multiple neural networks. Additionally, we design a multi-label for each sample and optimize the parameters of FB-EEGNet across multi-stimulus to incorporate the information from non-target stimuli.

RESULTS

Under the subject-specific condition, FB-EEGNet achieves the average classification accuracies (information transfer rate (ITR)) of 76.75 % (50.70 bits/min) and 89.14 % (70.45 bits/min) in a time widow of 0.7 s under the public 12-target dataset and our experimental 9-target dataset, respectively. Under the cross-subject condition, FB-EEGNet achieved mean accuracies (ITRs) of 81.72 % (67.99 bits/min) and 92.15 % (76.12 bits/min) on the public and experimental datasets in a time window of 1 s, respectively.

COMPARISON WITH EXISTING METHODS

FB-EEGNet shows superior performance than CCNN, EEGNet, CCA and FBCCA both for subject-dependent and subject-independent SSVEP target recognition.

CONCLUSION

FB-EEGNet can effectively extract information from multiple sub-bands and cross-stimulus targets, providing a promising way for extracting deep features in SSVEP using neural networks.