Cross-Subject Assistance: Inter- and Intra-Subject Maximal Correlation for Enhancing the Performance of SSVEP-Based BCIs

Cross Domain Correlation Maximization for Enhancing the Target Recognition of SSVEP-Based Brain-Computer Interfaces

The target recognition performance of steady-state visual evoked potential (SSVEP)-based brain-computer interfaces can be significantly improved with a training-based approach. However, the training procedure is time consuming and often causes fatigue. Consequently, the number of training data should be limited, which may reduce the classification performance. Thus, how to improve classification accuracy without increasing the training time is crucial to SSVEP-based BCI system. This study proposes a transfer-related component analysis (TransRCA) method for addressing the above issue. In this method, the SSVEP-related components are extracted from a small number of training data of the current individual and combined with those extracted from a large number of existing training data of other individuals. The TransRCA method maximizes not only the inter-trial covariances between the source and target subjects, but also the correlation between the reference signals and SSVEP signals from the source and target subjects. The proposed method was validated on the SSVEP public Benchmark and BETA datasets, and the classification accuracy and information transmission rate of the ensemble version of the proposed TransRCA method were compared with those of the state-of-the-art eCCA, eTRCA, ttCCA, LSTeTRCA, and eIISMC methods on both datasets. The comparison results indicate that the proposed method provides a superior performance compared with these state-of-the-art methods, and thus has high potential for the development of a SSVEP-based brain-computer interface system with high classification performance that only uses a small number of training data.

Cross-Subject Transfer Method Based on Domain Generalization for Facilitating Calibration of SSVEP-Based BCIs

In steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs), various spatial filtering methods based on individual calibration data have been proposed to alleviate the interference of spontaneous activities in SSVEP signals for enhancing the SSVEP detection performance. However, the time-consuming calibration session would increase the visual fatigue of subjects and reduce the usability of the BCI system. The key idea of this study is to propose a cross-subject transfer method based on domain generalization, which transfers the domain-invariant spatial filters and templates learned from source subjects to the target subject with no access to the EEG data from the target subject. The transferred spatial filters and templates are obtained by maximizing the intra- and inter-subject correlations using the SSVEP data corresponding to the target and its neighboring stimuli. For SSVEP detection of the target subject, four types of correlation coefficients are calculated to construct the feature vector. Experimental results estimated with three SSVEP datasets show that the proposed cross-subject transfer method improves the SSVEP detection performance compared to state-of-art methods. The satisfactory results demonstrate that the proposed method provides an effective transfer learning strategy requiring no tedious data collection process for new users, holding the potential of promoting practical applications of SSVEP-based BCI.

Multi-Level Constrained Intra and Inter Subject Feature Representation for Facial Video Based BVP Signal Measurement

Facial video-based blood volume pulse (BVP) signal measurement holds great potential for remote health monitoring, while existing methods have issues with convolutional kernel perceptual field constraints. This article proposes an end-to-end multi-level constrained spatiotemporal representation structure for facial video-based BVP signal measurement. First, an intra- and inter-subject feature representation is proposed to strengthen the BVP-related features generation at high, semantic, and shallow levels, respectively. Second, the global-local association is presented to enhance BVP signal period pattern learning, and the global temporal features are introduced into the local spatial convolution of each frame by adaptive kernel weights. Finally, the multi-dimensional fused features are mapped to one-dimensional BVP signals by the task-oriented signal estimator. The experimental results on the publicly available MMSE-HR dataset demonstrate that the proposed structure overperforms state-of-the-art methods (e.g., AutoHR) in BVP signal measurement, with a 20% and 40% reduction in mean absolute error and root mean squared error, respectively. The proposed structure would be a powerful tool for telemedical and non-contact heart health monitoring.

A Canonical Correlation Analysis-Based Transfer Learning Framework for Enhancing the Performance of SSVEP-Based BCIs

A steady-state visual evoked potential (SSVEP)- based brain-computer interface (BCI) can either achieve high classification accuracy in the case of sufficient training data or suppress the training stage at the cost of low accuracy. Although some researches attempted to conquer the dilemma between performance and practicality, a highly effective approach has not yet been established. In this paper, we propose a canonical correlation analysis (CCA)-based transfer learning framework for improving the performance of an SSVEP BCI and reducing its calibration effort. Three spatial filters are optimized by a CCA algorithm with intra- and inter-subject EEG data (IISCCA), two template signals are estimated separately with the EEG data from the target subject and a set of source subjects and six coefficients are yielded by correlation analysis between a testing signal and each of the two templates after they are filtered by each of the three spatial filters. The feature signal used for classification is extracted by the sum of squared coefficients multiplied by their signs and the frequency of the testing signal is recognized by template matching. To reduce the individual discrepancy between subjects, an accuracy-based subject selection (ASS) algorithm is developed for screening those source subjects whose EEG data are more similar to those of the target subject. The proposed ASS-IISCCA integrates both subject-specific models and subject-independent information for the frequency recognition of SSVEP signals. The performance of ASS-IISCCA was evaluated on a benchmark data set with 35 subjects and compared with the state-of-the-art algorithm task-related component analysis (TRCA). The results show that ASS-IISCCA can significantly improve the performance of SSVEP BCIs with a small number of training trials from a new user, thus helping to facilitate their applications in real world.

An efficient CNN-LSTM Network with spectral normalization and label smoothing technologies for SSVEP frequency recognition

OBJECTIVE

Steady-state visual evoked potentials(SSVEPs) based braincomputer interface(BCI) has received great interests owing to the high information transfer rate(ITR) and available large number of targets. However, the performance of frequency recognition methods heavily depends on the amount of the calibration data for intra-subject classification. Some research adopted the deep learning(DL) algorithm to conduct the inter-subject classification, which could reduce the calculation procedure, but the performance still has large room to improve compared with the intra-subject classification.

APPROACH

To address these issues, we proposed an efficient SSVEP DL NETwork (termed SSVEPNET) based on 1D convolution and long short-term memory (LSTM) module. To enhance the performance of SSVEPNT, we adopted the spectral normalization and label smoothing technologies during implementing the network architecture. We evaluated the SSVEPNET and compared it with other methods for the intra- and inter-subject classification under different conditions, i.e., two datasets, two time-window lengths (1 s and 0.5 s), three sizes of training data.

MAIN RESULTS

Under all the experimental settings, the proposed SSVEPNET achieved the highest average accuracy for the intra- and inter-subject classification on the two SSVEP datasets, when compared with other traditional and DL baseline methods. Signif icance. The extensive experimental results demonstrate that the proposed DL model holds promise to enhance frequency recognition performance in SSVEP-based BCIs. Besides, the mixed network structures with CNN and LSTM, and the spectral normalization and label smoothing could be useful optimization strategies to design efficient models for EEG data.

Facilitating Applications of SSVEP-Based BCIs by Within-Subject Information Transfer

The steady-state visual evoked potential based brain–computer interface (SSVEP–BCI) can provide high-speed alternative and augmentative communication in real-world applications. For individuals using a long-term BCI, within-subject (i.e., cross-day and cross-electrode) transfer learning could improve the BCI performance and reduce the calibration burden. To validate the within-subject transfer learning scheme, this study designs a 40-target SSVEP–BCI. Sixteen subjects are recruited, each of whom has performed experiments on three different days and has undergone the experiments of the SSVEP–BCIs based on the dry and wet electrodes. Several transfer directions, including the cross-day directions in parallel with the cross-electrode directions, are analyzed, and it is found that the transfer learning-based approach can maintain stable performance by zero training. Compared with the fully calibrated approaches, the transfer learning-based approach can achieve significantly better or comparable performance in different transfer directions. This result verifies that the transfer learning-based scheme is well suited for implementing a high-speed zero-training SSVEP–BCI, especially the dry electrode-based SSVEP–BCI system. A validation experiment of the cross-day wet-to-dry transfer, involving nine subjects, has shown that the average accuracy is 85.97 ± 5.60% for the wet-to-dry transfer and 77.69 ± 6.42% for the fully calibrated method with dry electrodes. By leveraging the electroencephalography data acquired on different days by different electrodes via transfer learning, this study lays the foundation for facilitating the long-term usage of the SSVEP–BCI and advancing the frontier of the dry electrode-based SSVEP–BCI in real-world applications.

Riemannian geometry-based transfer learning for reducing training time in c-VEP BCIs

One of the main problems that a brain-computer interface (BCI) face is that a training stage is required for acquiring training data to calibrate its classification model just before every use. Transfer learning is a promising method for addressing the problem. In this paper, we propose a Riemannian geometry-based transfer learning algorithm for code modulated visual evoked potential (c-VEP)-based BCIs, which can effectively reduce the calibration time without sacrificing the classification accuracy. The algorithm includes the main procedures of log-Euclidean data alignment (LEDA), super-trial construction, covariance matrix estimation, training accuracy-based subject selection (TSS) and minimum distance to mean classification. Among them, the LEDA reduces the difference in data distribution between subjects, whereas the TSS promotes the similarity between a target subject and the source subjects. The resulting performance of transfer learning is improved significantly. Sixteen subjects participated in a c-VEP BCI experiment and the recorded data were used in offline analysis. Leave-one subject-out (LOSO) cross-validation was used to evaluate the proposed algorithm on the data set. The results showed that the algorithm achieved much higher classification accuracy than the subject-specific (baseline) algorithm with the same number of training trials. Equivalently, the algorithm reduces the training time of the BCI at the same performance level and thus facilitates its application in real world.

Latency Aligning Task-related Component Analysis Using Wave Propagation for Enhancing SSVEP-based BCIs

Due to the high robustness to artifacts, steady-state visual evoked potential (SSVEP) has been widely applied to construct high-speed brain-computer interfaces (BCIs). Thus far, many spatial filtering methods have been proposed to enhance the target identification performance for SSVEP-based BCIs, and task-related component analysis (TRCA) is among the most effective ones. In this paper, we further extend TRCA and propose a new method called Latency Aligning TRCA (LA-TRCA), which aligns visual latencies on channels to obtain accurate phase information from task-related signals. Based on the SSVEP wave propagation theory, SSVEP spreads from posterior occipital areas over the cortex with a fixed phase velocity. Via estimation of the phase velocity using phase shifts of channels, the visual latencies on different channels can be determined for inter-channel alignment. TRCA is then applied to aligned data epochs for target recognition. For the validation purpose, the classification performance comparison between the proposed LA-TRCA and TRCA-based expansions were performed on two different SSVEP datasets. The experimental results illustrated that the proposed LA-TRCA method outperformed the other TRCA-based expansions, which thus demonstrated the effectiveness of the proposed approach for enhancing the SSVEP detection performance.

cVEP Training Data Validation—Towards Optimal Training Set Composition from Multi-Day Data

This paper investigates the effects of the repetitive block-wise training process on the classification accuracy for a code-modulated visual evoked potentials (cVEP)-based brain–computer interface (BCI). The cVEP-based BCIs are popular thanks to their autocorrelation feature. The cVEP-based stimuli are generated by a specific code pattern, usually the m-sequence, which is phase-shifted between the individual targets. Typically, the cVEP classification requires a subject-specific template (individually created from the user’s own pre-recorded EEG responses to the same stimulus target), which is compared to the incoming electroencephalography (EEG) data, using the correlation algorithms. The amount of the collected user training data determines the accuracy of the system. In this offline study, previously recorded EEG data collected during an online experiment with 10 participants from multiple sessions were used. A template matching target identification, with similar models as the task-related component analysis (TRCA), was used for target classification. The spatial filter was generated by the canonical correlation analysis (CCA). When comparing the training models from one session with the same session’s data (intra-session) and the model from one session with the data from the other session (inter-session), the accuracies were (94.84%, 94.53%) and (76.67%, 77.34%) for intra-sessions and inter-sessions, respectively. In order to investigate the most reliable configuration for accurate classification, the training data blocks from different sessions (days) were compared interchangeably. In the best training set composition, the participants achieved an average accuracy of 82.66% for models based only on two training blocks from two different sessions. Similarly, at least five blocks were necessary for the average accuracy to exceed 90%. The presented method can further improve cVEP-based BCI performance by reusing previously recorded training data.