Enhancing the Hybrid BCI Performance With the Common Frequency Pattern in Dual-Channel EEG

Frame-Level Teacher-Student Learning With Data Privacy for EEG Emotion Recognition

Recently, electroencephalogram (EEG) emotion recognition has gradually attracted a lot of attention. This brief designs a novel frame-level teacher-student framework with data privacy (FLTSDP) for EEG emotion recognition. The framework first proposes a teacher-student network without prior professional information for automated filtering of useful frame-level features by a gated mechanism and extracting high-level features by using knowledge distillation to capture the results of EEG emotion recognition from a teacher network and student networks. Then, the results from subnetworks are integrated by using the novel decision module, which, motivated by the voting mechanism, adjusts the composition of feature vectors and improves the weight of accurate prediction to optimize the integration effect. During training, an innovative data privacy protection mechanism is applied for avoiding data sharing, where each student network only inherits weights from all trained networks and does not inherit the training dataset. Here, the framework can be repeatedly optimized and improved by only training the next student subnetwork on new EEG signals. Experimental results show that our framework improves the accuracy of EEG emotion recognition by more than 5% and gets state-of-the-art performance for EEG emotion recognition in the subject-independent mode.

A Combined Virtual Electrode-Based ESA and CNN Method for MI-EEG Signal Feature Extraction and Classification

A Brain-Computer Interface (BCI) is a medium for communication between the human brain and computers, which does not rely on other human neural tissues, but only decodes Electroencephalography (EEG) signals and converts them into commands to control external devices. Motor Imagery (MI) is an important BCI paradigm that generates a spontaneous EEG signal without external stimulation by imagining limb movements to strengthen the brain's compensatory function, and it has a promising future in the field of computer-aided diagnosis and rehabilitation technology for brain diseases. However, there are a series of technical difficulties in the research of motor imagery-based brain-computer interface (MI-BCI) systems, such as: large individual differences in subjects and poor performance of the cross-subject classification model; a low signal-to-noise ratio of EEG signals and poor classification accuracy; and the poor online performance of the MI-BCI system. To address the above problems, this paper proposed a combined virtual electrode-based EEG Source Analysis (ESA) and Convolutional Neural Network (CNN) method for MI-EEG signal feature extraction and classification. The outcomes reveal that the online MI-BCI system developed based on this method can improve the decoding ability of multi-task MI-EEG after training, it can learn generalized features from multiple subjects in cross-subject experiments and has some adaptability to the individual differences of new subjects, and it can decode the EEG intent online and realize the brain control function of the intelligent cart, which provides a new idea for the research of an online MI-BCI system.

A multi-modal brain-computer interface based on threshold discrimination and its application in wheelchair control

In this study, we propose a novel multi-modal brain-computer interface (BCI) system based on the threshold discrimination, which is proposed for the first time to distinguish between SSVEP and MI potentials. The system combines these two heterogeneous signals to increase the number of control commands and improve the performance of asynchronous control of external devices. In this research, an electric wheelchair is controlled as an example. The user can continuously control the wheelchair to turn left/right through motion imagination (MI) by imagining left/right-hand movement and generate another 6 commands for the wheelchair control by focusing on the SSVEP stimulation panel. Ten subjects participated in a MI training session and eight of them completed a mobile obstacle-avoidance experiment in a complex environment requesting high control accuracy for successful manipulation. Comparing with the single-modal BCI-controlled wheelchair system, the results demonstrate that the proposed multi-modal method is effective by providing more satisfactory control accuracy, and show the potential of BCI-controlled systems to be applied in complex daily tasks.

Recognition of Drivers’ Hard and Soft Braking Intentions Based on Hybrid Brain-Computer Interfaces

In this paper, we propose simultaneous and sequential hybrid brain-computer interfaces (hBCIs) that incorporate electroencephalography (EEG) and electromyography (EMG) signals to classify drivers’ hard braking, soft braking, and normal driving intentions to better assist driving for the first time. The simultaneous hBCIs adopt a feature-level fusion strategy (hBCI-FL) and classifier-level fusion strategies (hBCIs-CL). The sequential hBCIs include the hBCI-SE1, where EEG signals are prioritized to detect hard braking, and hBCI-SE2, where EMG signals are prioritized to detect hard braking. Experimental results show that the proposed hBCI-SE1 with spectral features and the one-vs-rest classification strategy performs best with an average system accuracy of 96.37% among hBCIs. This work is valuable for developing human-centric intelligent assistant driving systems to improve driving safety and driving comfort and promote the application of BCIs.

A Novel Hybrid Brain-Computer Interface Combining Motor Imagery and Intermodulation Steady-State Visual Evoked Potential

The hybrid brain-computer interface (hBCI) combining motor imagery (MI) and steady-state visual evoked potential (SSVEP) has been proven to have better performance than a pure MI- or SSVEP-based brain-computer interface (BCI). In most studies on hBCIs, subjects have been required to focus their attention on flickering light-emitting diodes (LEDs) or blocks while imagining body movements. However, these two classical tasks performed concurrently have a poor correlation. Therefore, it is necessary to reduce the task complexity of such a system and improve its user-friendliness. Aiming to achieve this goal, this study proposes a novel hybrid BCI that combines MI and intermodulation SSVEPs. In the proposed system, images of both hands flicker at the same frequency (i.e., 30 Hz) but at different grasp frequencies (i.e., 1 Hz for the left hand, and 1.5 Hz for the right hand), resulting in different intermodulation frequencies for encoding targets. Additionally, movement observation for subjects can help to perform the MI task better. In this study, two types of brain signals are classified independently and then fused by a scoring mechanism based on the probability distribution of relevant parameters. The online verification results showed that the average accuracies of 12 healthy subjects and 11 stroke patients were 92.40 ± 7.45% and 73.07 ± 9.07%, respectively. The average accuracies of 10 healthy subjects in the MI, SSVEP, and hybrid tasks were 84.00 ± 12.81%, 80.75 ± 8.08%, and 89.00 ± 9.94%, respectively. The high recognition accuracy verifies the feasibility and robustness of the proposed system. This study provides a novel and natural paradigm for a hybrid BCI based on MI and SSVEP.

FLDNet: Frame-Level Distilling Neural Network for EEG Emotion Recognition

Based on the current research on EEG emotion recognition, there are some limitations, such as hand-engineered features, redundant and meaningless signal frames and the loss of frame-to-frame correlation. In this paper, a novel deep learning framework is proposed, named the frame-level distilling neural network (FLDNet), for learning distilled features from the correlations of different frames. A layer named the frame gate is designed to integrate weighted semantic information on multiple frames to remove redundant and meaningless signal frames. A triple-net structure is introduced to distill the learned features net by net to replace the hand-engineered features with professional knowledge. Specifically, one neural network is normally trained for several epochs. Then, a second network of the same structure will be initialized again to learn the extracted features from the frame gate of the first neural network based on the output of the first net. Similarly, the third net improves the features based on the frame gate of the second network. To utilize the representation ability of the triple neural network, an ensemble layer is conducted to integrate the discriminative ability of the proposed framework for final decisions. Consequently, the proposed FLDNet provides an effective method for capturing the correlation between different frames and automatically learn distilled high-level features for emotion recognition. The experiments are carried out in a subject-independent emotion recognition task on public emotion datasets of DEAP and DREAMER benchmarks, which have demonstrated the effectiveness and robustness of the proposed FLDNet.

Enhancing EEG-Based Classification of Depression Patients Using Spatial Information

BACKGROUND

Depression has become a leading mental disorder worldwide. Evidence has shown that subjects with depression exhibit different spatial responses in neurophysiological signals from the healthy controls when they are exposed to positive and negative stimuli.

METHODS

We proposed an effective electroencephalogram-based detection method for depression classification using spatial information. A face-in-the-crowd task, including positive and negative emotional facial expressions, was presented to 30 participants, including 16 depression patients and 14 healthy controls. Differential entropy and the genetic algorithm were used for feature extraction and selection, and a support vector machine was used for classification. A task-related common spatial pattern (TCSP) was proposed to enhance the spatial differences before the feature extraction.

RESULTS AND DISCUSSION

We achieved a leave-one-subject-out cross-validation classification result of 84% and 85.7% for positive and negative stimuli, respectively, using TCSP, which is statistically significantly higher than 81.7% and 83.2%, respectively, acquired without the TCSP (p < 0.05). We also evaluated the classification performance using individual frequency bands and found that the contribution of the gamma band was predominant. In addition, we evaluated different classifiers, including k-nearest neighbor and logistic regression, which showed similar trends in the improvement of classification by employing TCSP.

CONCLUSION

The results show that our proposed method, employing spatial information, significantly improves the accuracy of classifying depression patients.

Intralobular and Interlobular Parietal Functional Network Correlated to MI-BCI Performance

Brain-computer interface (BCI) brings hope to patients suffering from neuromuscular diseases, by allowing the control of external devices using neural signals from the central nervous system. However, a portion of individuals was unable to operate BCI with high efficacy. This research aimed to study the brain-wide functional connectivity differences that contributed to BCI performance, and investigate the relationship between task-related connectivity strength and BCI performance. Functional connectivity was estimated using pairwise Pearson's correlation from the EEG of 48 subjects performing left or right hand motor imagery (MI) tasks. The classification accuracy of linear support vector machine (SVM) to distinguish both tasks were used to represent MI-BCI performance. The significant differences in connectivity strengths were examined using Welch's T-test. The association between accuracy and connection strength was studied using correlation model. Three intralobular and fourteen interlobular connections from the parietal lobe showed a correlation of 0.31 and -0.34 respectively. Results indicate that alpha wave connectivity from 8 Hz to 13 Hz was more related to classification performance compared to high-frequency waves. Subject-independent trial-based analysis shows that MI trials executed with stronger intralobular and interlobular parietal connections performed significantly better than trials with weaker connections. Further investigation from an independent MI dataset reveals several similar connections that were correlated with MI-BCI performance. The functional connectivity of the parietal lobe could potentially allow prediction of MI-BCI performance and enable implementation of neurofeedback training for users to improve the usability of MI-BCI.