Similarity-constrained task-related component analysis for enhancing SSVEP detection

Time-frequency feature extraction based on multivariable synchronization index for training-free SSVEP-based BCI

Multivariate synchronization index (MSI), as an effective recognition algorithm for steady-state visual evoked potential (SSVEP) brain-computer interface (BCI), can accurately decode target frequencies without training. To further consider temporal features or extract harmonic components, extended MSI (EMSI), temporally local MSI (TMSI), and filter bank MSI (FBMSI) have been proposed. However, the promotion effects of the above three strategies on MSI have not been compared in detail. In this paper, the performance of EMSI, TMSI, and FBMSI under different time windows was analyzed with the same dataset. The results indicated that the improvement effect of the temporally local method on MSI was better than that of the other two methods under the short time window, and the effect of the filter bank method was better when the time window was greater than 0.8 s. Based on the idea of simultaneously extracting time-frequency features, FBEMSI and FBTMSI were proposed by integrating time delay embedding and temporally local method into FBMSI respectively. The two improved methods, which has no significant difference, can improve the recognition effect of FBMSI. But the computing time of FBEMSI was shorter, which can be a potential method for SSVEP-BCI.

A novel brain-controlled prosthetic hand method integrating AR-SSVEP augmentation, asynchronous control, and machine vision assistance

Background and objective

The brain-computer interface (BCI) system based on steady-state visual evoked potentials (SSVEP) is expected to help disabled patients achieve alternative prosthetic hand assistance. However, the existing study still has some shortcomings in interaction aspects such as stimulus paradigm and control logic. The purpose of this study is to innovate the visual stimulus paradigm and asynchronous decoding/control strategy by integrating augmented reality technology, and propose an asynchronous pattern recognition algorithm, thereby improving the interaction logic and practical application capabilities of the prosthetic hand with the BCI system.

Methods

An asynchronous visual stimulus paradigm based on an augmented reality (AR) interface was proposed in this paper, in which there were 8 control modes, including Grasp, Put down, Pinch, Point, Fist, Palm push, Hold pen, and Initial. According to the attentional orienting characteristics of the paradigm, a novel asynchronous pattern recognition algorithm that combines center extended canonical correlation analysis and support vector machine (Center-ECCA-SVM) was proposed. Then, this study proposed an intelligent BCI system switch based on a deep learning object detection algorithm (YOLOv4) to improve the level of user interaction. Finally, two experiments were designed to test the performance of the brain-controlled prosthetic hand system and its practical performance in real scenarios.

Results

Under the AR paradigm of this study, compared with the liquid crystal display (LCD) paradigm, the average SSVEP spectrum amplitude of multiple subjects increased by 17.41%, and the signal-noise ratio (SNR) increased by 3.52%. The average stimulus pattern recognition accuracy was 96.71 ± 3.91%, which was 2.62% higher than the LCD paradigm. Under the data analysis time of 2s, the Center-ECCA-SVM classifier obtained 94.66 ± 3.87% and 97.40 ± 2.78% asynchronous pattern recognition accuracy under the Normal metric and the Tolerant metric, respectively. And the YOLOv4-tiny model achieves a speed of 25.29fps and a 96.4% confidence in the prosthetic hand in real-time detection. Finally, the brain-controlled prosthetic hand helped the subjects to complete 4 kinds of daily life tasks in the real scene, and the time-consuming were all within an acceptable range, which verified the effectiveness and practicability of the system.

Conclusion

This research is based on improving the user interaction level of the prosthetic hand with the BCI system, and has made improvements in the SSVEP paradigm, asynchronous pattern recognition, interaction, and control logic. Furthermore, it also provides support for BCI areas for alternative prosthetic control, and movement disorder rehabilitation programs.

Almost free of calibration for SSVEP-based brain-computer interfaces

Objective. Steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) is a promising technology that can achieve high information transfer rate (ITR) with supervised algorithms such as ensemble task-related component analysis (eTRCA) and task-discriminant component analysis (TDCA). However, training individual models requires a tedious and time-consuming calibration process, which hinders the real-life use of SSVEP-BCIs. A recent data augmentation method, called source aliasing matrix estimation (SAME), can generate new EEG samples from a few calibration trials. But SAME does not exploit the information across stimuli as well as only reduces the number of calibration trials per command, so it still has some limitations.Approach. This study proposes an extended version of SAME, called multi-stimulus SAME (msSAME), which exploits the similarity of the aliasing matrix across frequencies to enhance the performance of SSVEP-BCI with insufficient calibration trials. We also propose a semi-supervised approach based on msSAME that can further reduce the number of SSVEP frequencies needed for calibration. We evaluate our method on two public datasets, Benchmark and BETA, and an online experiment.Main results. The results show that msSAME outperforms SAME for both eTRCA and TDCA on the public datasets. Moreover, the semi-supervised msSAME-based method achieves comparable performance to the fully calibrated methods and outperforms the conventional free-calibrated methods. Remarkably, our method only needs 24 s to calibrate 40 targets in the online experiment and achieves an average ITR of 213.8 bits min-1with a peak of 242.6 bits min-1.Significance. This study significantly reduces the calibration effort for individual SSVEP-BCIs, which is beneficial for developing practical plug-and-play SSVEP-BCIs.

Phase-Locked Time-Shift Data Augmentation Method for SSVEP Brain-Computer Interfaces

Steady-state visual evoked potential (SSVEP) based brain-computer interfaces (BCIs) have achieved an information transfer rate (ITR) of over 300 bits/min, but abundant training data is required. The performance of SSVEP algorithms deteriorates greatly under limited data, and the existing time-shift data augmentation method fails to improve it because the phase-locked requirement between training samples is violated. To address this issue, this study proposes a novel augmentation method, namely phase-locked time-shift (PLTS), for SSVEP-BCI. The similarity between epochs at different time moments was evaluated, and a unique time-shift step was calculated for each class to augment additional data epochs in each trial. The results showed that the PLTS significantly improved the classification performance of SSVEP algorithms on the BETA SSVEP datasets. Moreover, under the condition of one calibration block, by slightly prolonging the calibration duration (from 48 s to 51.5 s), the ITR increased from 40.88±4.54 bits/min to 122.61±7.05 bits/min with the PLTS. This study provides a new perspective on augmenting data epochs for training-based SSVEP-BCI, promotes the classification accuracy and ITR under limited training data, and thus facilitates the real-life applications of SSVEP-based brain spellers.

Improving the Performance of Individually Calibrated SSVEP-BCI by Task- Discriminant Component Analysis

A brain-computer interface (BCI) provides a direct communication channel between a brain and an external device. Steady-state visual evoked potential based BCI (SSVEP-BCI) has received increasing attention due to its high information transfer rate, which is accomplished by individual calibration for frequency recognition. Task-related component analysis (TRCA) is a recent and state-of-the-art method for individually calibrated SSVEP-BCIs. However, in TRCA, the spatial filter learned from each stimulus may be redundant and temporal information is not fully utilized. To address this issue, this paper proposes a novel method, i.e., task-discriminant component analysis (TDCA), to further improve the performance of individually calibrated SSVEP-BCI. The performance of TDCA was evaluated by two publicly available benchmark datasets, and the results demonstrated that TDCA outperformed ensemble TRCA and other competing methods by a significant margin. An offline and online experiment testing 12 subjects further validated the effectiveness of TDCA. The present study provides a new perspective for designing decoding methods in individually calibrated SSVEP-BCI and presents insight for its implementation in high-speed brain speller applications.