Optimizing the Performance of the Visual P300-Speller Through Active Mental Tasks Based on Color Distinction and Modulation of Task Difficulty

Improving single-hand open/close motor imagery classification by error-related potentials correction

Objective

The ability of a brain-computer interface (BCI) to classify brain activity in electroencephalograms (EEG) during motor imagery (MI) tasks is an important performance indicator. Because the cortical regions that drive the single-handed open and closed tasks overlap, it is difficult to classify the EEG signals during executing both tasks.

Approach

The addition of special EEG features can improve the accuracy of classifying single-hand open and closed tasks. In this work, we designed a hybrid BCI paradigm based on error-related potentials (ErrP) and motor imagery (MI) and proposed a strategy to correct the classification results of MI by using ErrP information. The ErrP and MI features of EEG data from 11 subjects were superimposed.

Main results

The corrected strategy improved the classification accuracy of single-hand open/close MI tasks from 52.3% to 73.7%, an increase of approximately 21%.

Significance

Our hybrid BCI paradigm improves the classification accuracy of single-hand MI by adding ErrP information, which provides a new approach for improving the classification performance of BCI.

Do numbers make us handy? Behavioral and electrophysiological evidence for number-hand congruency effect

Finger counting facilitates numerical representations and mathematical processing. The current study investigated the association between finger counting habits and number processing by employing behavioral and electrophysiological measures. We explored whether small and large numerical primes influence the recognition of embodied target hand stimuli. Twenty-four right-handed participants that were grouped into right-starters (n = 13) and left-starters (n = 11) for finger counting performed a hand recognition task that consisted of numerical magnitudes as prime and hand recognition as targets. Based on the finger counting habits, congruent (i.e., left-starters: small number/left hand or large number/right hand; right-starters: small number/right hand or large number/left hand) and incongruent (i.e., left-starters: large number/left hand or small number/right hand; right-starters: large number/right hand or small number/left hand) conditions were presented to the participants. The participants were required to indicate whether the targets were left or right hand by simply pressing the left or the right key, respectively. Results indicated faster reaction times (RTs) for congruent as opposed to incongruent trials for all participants. The mean amplitude of the centro-parietal P300 component was significantly increased for the incongruent compared to congruent condition, indicating increased mental effort. Also, analysis of the latency of the P300 in terms of congruency effect in all participants revealed significant results. These combined results provide behavioral and electrophysiological evidence indicating the embodied nature of numbers. The results are interpreted in light of the general findings related to the P300 component. This research supports the association of number-hand representations and corroborates the idea of embodied numerosity.

A flexible speller based on time-space frequency conversion SSVEP stimulation paradigm under dry electrode

Introduction

Speller is the best way to express the performance of the brain-computer interface (BCI) paradigm. Due to its advantages of short analysis time and high accuracy, the SSVEP paradigm has been widely used in the BCI speller system based on the wet electrode. It is widely known that the wet electrode operation is cumbersome and that the subjects have a poor experience. In addition, in the asynchronous SSVEP system based on threshold analysis, the system flickers continuously from the beginning to the end of the experiment, which leads to visual fatigue. The dry electrode has a simple operation and provides a comfortable experience for subjects. The EOG signal can avoid the stimulation of SSVEP for a long time, thus reducing fatigue.

Methods

This study first designed the brain-controlled switch based on continuous blinking EOG signal and SSVEP signal to improve the flexibility of the BCI speller. Second, in order to increase the number of speller instructions, we designed the time-space frequency conversion (TSFC) SSVEP stimulus paradigm by constantly changing the time and space frequency of SSVEP sub-stimulus blocks, and designed a speller in a dry electrode environment.

Results

Seven subjects participated and completed the experiments. The results showed that the accuracy of the brain-controlled switch designed in this study was up to 94.64%, and all the subjects could use the speller flexibly. The designed 60-character speller based on the TSFC-SSVEP stimulus paradigm has an accuracy rate of 90.18% and an information transmission rate (ITR) of 117.05 bits/min. All subjects can output the specified characters in a short time.

Discussion

This study designed and implemented a multi-instruction SSVEP speller based on dry electrode. Through the combination of EOG and SSVEP signals, the speller can be flexibly controlled. The frequency of SSVEP stimulation sub-block is recoded in time and space by TSFC-SSVEP stimulation paradigm, which greatly improves the number of output instructions of BCI system in dry electrode environment. This work only uses FBCCA algorithm to test the stimulus paradigm, which requires a long stimulus time. In the future, we will use trained algorithms to study stimulus paradigm to improve its overall performance.

Advances in P300 brain-computer interface spellers: toward paradigm design and performance evaluation

A brain-computer interface (BCI) is a non-muscular communication technology that provides an information exchange channel for our brains and external devices. During the decades, BCI has made noticeable progress and has been applied in many fields. One of the most traditional BCI applications is the BCI speller. This article primarily discusses the progress of research into P300 BCI spellers and reviews four types of P300 spellers: single-modal P300 spellers, P300 spellers based on multiple brain patterns, P300 spellers with multisensory stimuli, and P300 spellers with multiple intelligent techniques. For each type of P300 speller, we further review several representative P300 spellers, including their design principles, paradigms, algorithms, experimental performance, and corresponding advantages. We particularly emphasized the paradigm design ideas, including the overall layout, individual symbol shapes and stimulus forms. Furthermore, several important issues and research guidance for the P300 speller were identified. We hope that this review can assist researchers in learning the new ideas of these novel P300 spellers and enhance their practical application capability.

Steady-state visually evoked potential collaborative BCI system deep learning classification algorithm based on multi-person feature fusion transfer learning-based convolutional neural network

Objective

The conventional single-person brain–computer interface (BCI) systems have some intrinsic deficiencies such as low signal-to-noise ratio, distinct individual differences, and volatile experimental effect. To solve these problems, a centralized steady-state visually evoked potential collaborative BCI system (SSVEP-cBCI), which characterizes multi-person electroencephalography (EEG) feature fusion was constructed in this paper. Furthermore, three different feature fusion methods compatible with this new system were developed and applied to EEG classification, and a comparative analysis of their classification accuracy was performed with transfer learning-based convolutional neural network (TL-CNN) approach.

Approach

An EEG-based SSVEP-cBCI system was set up to merge different individuals’ EEG features stimulated by the instructions for the same task, and three feature fusion methods were adopted, namely parallel connection, serial connection, and multi-person averaging. The fused features were then input into CNN for classification. Additionally, transfer learning (TL) was applied first to a Tsinghua University (THU) benchmark dataset, and then to a collected dataset, so as to meet the CNN training requirement with a much smaller size of collected dataset and increase the classification accuracy. Ten subjects were recruited for data collection, and both datasets were used to gauge the three fusion algorithms’ performance.

Main results

The results predicted by TL-CNN approach in single-person mode and in multi-person mode with the three feature fusion methods were compared. The experimental results show that each multi-person mode is superior to single-person mode. Within the 3 s time window, the classification accuracy of the single-person CNN is only 90.6%, while the same measure of the two-person parallel connection fusion method can reach 96.6%, achieving better classification effect.

Significance

The results show that the three multi-person feature fusion methods and the deep learning classification algorithm based on TL-CNN can effectively improve the SSVEP-cBCI classification performance. The feature fusion method of multi -person parallel feature connection achieves better classification results. Different feature fusion methods can be selected in different application scenarios to further optimize cBCI.

A P300-Detection Method Based on Logistic Regression and a Convolutional Neural Network

Background

Electroencephalogram (EEG)-based brain-computer interface (BCI) systems are widely utilized in various fields, including health care, intelligent assistance, identity recognition, emotion recognition, and fatigue detection. P300, the main event-related potential, is the primary component detected by EEG-based BCI systems. Existing algorithms for P300 classification in EEG data usually perform well when tested in a single participant, although they exhibit significant decreases in accuracy when tested in new participants. We attempted to address this lack of generalizability associated with existing classification methods using a novel convolutional neural network (CNN) model developed using logistic regression (LR).

Materials and Methods

We proposed an LR-CNN model comprising two parts: a combined LR-based memory model and a CNN-based generalization model. The LR-based memory model can learn the individual features of participants and addresses the decrease in accuracy caused by individual differences when applied to new participants. The CNN-based generalization model can learn the common features among participants, thereby reducing overall classification bias and improving overall classification accuracy.

Results

We compared our method with existing, commonly used classification methods through three different sets of experiments. The experimental results indicated that our method could learn individual differences among participants. Compared with other commonly used classification methods, our method yielded a marked improvement (>90%) in classification among new participants.

Conclusion

The accuracy of the proposed model in the face of new participants is better than that of existing, commonly used classification methods. Such improvements in cross-subject test accuracy will aid in the development of BCI systems.

Reconfiguration of Cortical Brain Network from Searching to Spotting for Dynamic Visual Targets

BACKGROUND

Detecting dynamic targets from complex visual scenes is an important problem in real world. However, the cognitive mechanism accounting for dynamic visual target detection remains unclear.

NEW METHOD

Herein, we aim to explore the cognitive process of dynamic visual target detection from searching to spotting and provide more concrete evidence for cognitive studies related to target detection. Cortical source responses with high spatiotemporal resolution were reconstructed from scalp EEG signals. Then, time-varying cortical networks were built using adaptive directed transfer function to explore the cognitive processes while detecting the dynamic visual target.

RESULTS

The experimental results demonstrated that the dynamic visual target detection enhanced the activation in both the visual and attention networks. Specially, the information flow from the middle occipital gyrus (MOG) mainly contributed to the position function, whereas the activation of the prefrontal cortex (PFC) reflected spatial attention maintenance.

CONCLUSION

The left "frontal-central-parietal" network played as a leading information source in dynamic target detection tasks. These findings provide new insights into cognitive processes of dynamic visual target detection.

DATA AVAILABILITY STATEMENT

The datasets in this study are available on request to the corresponding author.

A Spelling Paradigm With an Added Red Dot Improved the P300 Speller System Performance

The traditional P300 speller system uses the flashing row or column spelling paradigm. However, the classification accuracy and information transfer rate of the P300 speller are not adequate for real-world application. To improve the performance of the P300 speller, we devised a new spelling paradigm in which the flashing row or column of a virtual character matrix is covered by a translucent green circle with a red dot in either the upper or lower half (GC-RD spelling paradigm). We compared the event-related potential (ERP) waveforms with a control paradigm (GC spelling paradigm), in which the flashing row or column of a virtual character matrix was covered by a translucent green circle only. Our experimental results showed that the amplitude of P3a at the parietal area and P3b at the frontal–central–parietal areas evoked by the GC-RD paradigm were significantly greater than those induced by the GC paradigm. Higher classification accuracy and information transmission rates were also obtained in the GC-RD system. Our results indicated that the added red dots increased attention and visuospatial information, resulting in an amplitude increase in both P3a and P3b, thereby improving the performance of the P300 speller system.

Intra- and Inter-subject Variability in EEG-Based Sensorimotor Brain Computer Interface: A Review

Brain computer interfaces (BCI) for the rehabilitation of motor impairments exploit sensorimotor rhythms (SMR) in the electroencephalogram (EEG). However, the neurophysiological processes underpinning the SMR often vary over time and across subjects. Inherent intra- and inter-subject variability causes covariate shift in data distributions that impede the transferability of model parameters amongst sessions/subjects. Transfer learning includes machine learning-based methods to compensate for inter-subject and inter-session (intra-subject) variability manifested in EEG-derived feature distributions as a covariate shift for BCI. Besides transfer learning approaches, recent studies have explored psychological and neurophysiological predictors as well as inter-subject associativity assessment, which may augment transfer learning in EEG-based BCI. Here, we highlight the importance of measuring inter-session/subject performance predictors for generalized BCI frameworks for both normal and motor-impaired people, reducing the necessity for tedious and annoying calibration sessions and BCI training.

The Self-Face Paradigm Improves the Performance of the P300-Speller System

Objective: Previous studies have shown that the performance of the famous face P300-speller was better than that of the classical row/column flashing P300-speller. Furthermore, in some studies, the brain was more active when responding to one's own face than to a famous face, and a self-face stimulus elicited larger amplitude event-related potentials (ERPs) than did a famous face. Thus, we aimed to study the role of the self-face paradigm on further improving the performance of the P300-speller system with the famous face P300-speller paradigm as the control paradigm.

Methods: We designed two facial P300-speller paradigms based on the self-face and a famous face (Ming Yao, a sports star; the famous face spelling paradigm) with a neutral expression.

Results: ERP amplitudes were significantly greater in the self-face than in the famous face spelling paradigm at the parietal area from 340 to 480 ms (P300), from 480 to 600 ms (P600f), and at the fronto-central area from 700 to 800 ms. Offline and online classification results showed that the self-face spelling paradigm accuracies were significantly higher than those of the famous face spelling paradigm at superposing first two times (P < 0.05). Similar results were found for information transfer rates (P < 0.05).

Conclusions: The self-face spelling paradigm significantly improved the performance of the P300-speller system. This has significant practical applications for brain-computer interfaces (BCIs) and could avoid infringement issues caused by using images of other people's faces.

Happy emotion cognition of bimodal audiovisual stimuli optimizes the performance of the P300 speller

OBJECTIVE

Prior studies of emotional cognition have found that emotion-based bimodal face and voice stimuli can elicit larger event-related potential (ERP) amplitudes and enhance neural responses compared with visual-only emotional face stimuli. Recent studies on brain-computer interface have shown that emotional face stimuli have significantly improved the performance of the traditional P300 speller system, but its performance needs to be further improved for practical applications. Therefore, we herein propose a novel audiovisual P300 speller based on bimodal emotional cognition to further improve the performance of the P300 system.

METHODS

The audiovisual P300 speller we proposed is based on happy emotions, with visual and auditory stimuli that consist of several pairs of smiling faces and audible chuckles (E-AV spelling paradigm) of different ages and sexes. The control paradigm was the visual-only emotional face P300 speller (E-V spelling paradigm).

RESULTS

We compared the ERP amplitudes, accuracy, and raw bit rate between the E-AV and E-V spelling paradigms. The target stimuli elicited significantly increased P300 amplitudes (p < .05) and P600 amplitudes (p < .05) in the E-AV spelling paradigm compared with those in the E-V paradigm. The E-AV spelling paradigm also significantly improved the spelling accuracy and the raw bit rate compared with those in the E-V paradigm at one superposition (p < .05) and at two superpositions (p < .05).

SIGNIFICANCE

The proposed emotion-based audiovisual spelling paradigm not only significantly improves the performance of the P300 speller, but also provides a basis for the development of various bimodal P300 speller systems, which is a step forward in the clinical application of brain-computer interfaces.

A Novel Audiovisual P300-Speller Paradigm Based on Cross-Modal Spatial and Semantic Congruence

Objective

Although many studies have attempted to improve the performance of the visual-based P300-speller system, its performance is still not satisfactory. The current system has limitations for patients with neurodegenerative diseases, in which muscular control of the eyes may be impaired or deteriorate over time. Some studies have shown that the audiovisual stimuli with spatial and semantic congruence elicited larger event-related potential (ERP) amplitudes than do unimodal visual stimuli. Therefore, this study proposed a novel multisensory P300-speller based on audiovisual spatial and semantic congruence.

Methods

We designed a novel audiovisual P300-speller paradigm (AV spelling paradigm) in which the pronunciation and visual presentation of characters were matched in spatial position and semantics. We analyzed the ERP waveforms elicited in the AV spelling paradigm and visual-based spelling paradigm (V spelling paradigm) and compared the classification accuracies between these two paradigms.

Results

ERP analysis revealed significant differences in ERP amplitudes between the two paradigms in the following areas (AV > V): the frontal area at 60–140 ms, frontal–central–parietal area at 360–460 ms, frontal area at 700–800 ms, right temporal area at 380–480 and 700–780 ms, and left temporal area at 500–780 ms. Offline classification results showed that the accuracies were significantly higher in the AV spelling paradigm than in the V spelling paradigm after superposing 1, 2, 5, 6, 9, and 10 times (P < 0.05), and there were trends toward improvement in the accuracies at superposing 3, 4, 7, and 8 times (P = 0.06). Similar results were found for information transfer rate between V and AV spelling paradigms at 1, 2, 5, 6, and 10 superposition times (P < 0.05).

Significance

The proposed audiovisual P300-speller paradigm significantly improved the classification accuracies compared with the visual-based P300-speller paradigm. Our novel paradigm combines spatial and semantic features of two sensory modalities, and the present findings provide valuable insights into the development of multimodal ERP-based BCI paradigms.