A multi-modal brain-computer interface based on threshold discrimination and its application in wheelchair control

Joint multi-feature extraction and transfer learning in motor imagery brain computer interface

Motor imagery brain computer interface (BCI) systems are considered one of the most crucial paradigms and have received extensive attention from researchers worldwide. However, the non-stationary from subject-to-subject transfer is a substantial challenge for robust BCI operations. To address this issue, this paper proposes a novel approach that integrates joint multi-feature extraction, specifically combining common spatial patterns (CSP) and wavelet packet transforms (WPT), along with transfer learning (TL) in motor imagery BCI systems. This approach leverages the time-frequency characteristics of WPT and the spatial characteristics of CSP while utilizing transfer learning to facilitate EEG identification for target subjects based on knowledge acquired from non-target subjects. Using dataset IVa from BCI Competition III, our proposed approach achieves an impressive average classification accuracy of 93.4%, outperforming five kinds of state-of-the-art approaches. Furthermore, it offers the advantage of enabling the design of various auxiliary problems to learn different aspects of the target problem from unlabeled data through transfer learning, thereby facilitating the implementation of innovative ideas within our proposed approach. Simultaneously, it demonstrates that integrating CSP and WPT while transferring knowledge from other subjects is highly effective in enhancing the average classification accuracy of EEG signals and it provides a novel solution to address subject-to-subject transfer challenges in motor imagery BCI systems.

Multiclass classification of motor imagery tasks based on multi-branch convolutional neural network and temporal convolutional network model

Motor imagery (MI) is a cognitive process wherein an individual mentally rehearses a specific movement without physically executing it. Recently, MI-based brain-computer interface (BCI) has attracted widespread attention. However, accurate decoding of MI and understanding of neural mechanisms still face huge challenges. These seriously hinder the clinical application and development of BCI systems based on MI. Thus, it is very necessary to develop new methods to decode MI tasks. In this work, we propose a multi-branch convolutional neural network (MBCNN) with a temporal convolutional network (TCN), an end-to-end deep learning framework to decode multi-class MI tasks. We first used MBCNN to capture the MI electroencephalography signals information on temporal and spectral domains through different convolutional kernels. Then, we introduce TCN to extract more discriminative features. The within-subject cross-session strategy is used to validate the classification performance on the dataset of BCI Competition IV-2a. The results showed that we achieved 75.08% average accuracy for 4-class MI task classification, outperforming several state-of-the-art approaches. The proposed MBCNN-TCN-Net framework successfully captures discriminative features and decodes MI tasks effectively, improving the performance of MI-BCIs. Our findings could provide significant potential for improving the clinical application and development of MI-based BCI systems.

Effect of 3D paradigm synchronous motion for SSVEP-based hybrid BCI-VR system

A brain-computer interface (BCI) system and virtual reality (VR) are integrated as a more interactive hybrid system (BCI-VR) that allows the user to manipulate the car. A virtual scene in the VR system that is the same as the physical environment is built, and the object's movement can be observed in the VR scene. The four-class three-dimensional (3D) paradigm is designed and moves synchronously in virtual reality. The dynamic paradigm may affect their attention according to the experimenters' feedback. Fifteen subjects in our experiment steered the car according to a specified motion trajectory. According to our online experimental result, different motion trajectories of the paradigm have various effects on the system's performance, and training can mitigate this adverse effect. Moreover, the hybrid system using frequencies between 5 and 10 Hz indicates better performance than those using lower or higher stimulation frequencies. The experiment results show a maximum average accuracy of 0.956 and a maximum information transfer rate (ITR) of 41.033 bits/min. It suggests that a hybrid system provides a high-performance way of brain-computer interaction. This research could encourage more interesting applications involving BCI and VR technologies.