The Self-Face Paradigm Improves the Performance of the P300-Speller System

Optimization of transfer learning based on source sample selection in Euclidean space for P300-based brain-computer interfaces

Introduction

Event-related potentials (ERPs), such as P300, are widely utilized for non-invasive monitoring of brain activity in brain-computer interfaces (BCIs) via electroencephalogram (EEG). However, the non-stationary nature of EEG signals and different data distributions among subjects create significant challenges for implementing real-time P300-based BCIs. This requires time-consuming calibration and a large number of training samples.

Methods

To address these challenges, this study proposes a transfer learning-based approach that uses a convolutional neural network for high-level feature extraction, followed by Euclidean space data alignment to ensure similar distributions of extracted features. Furthermore, a source selection technique based on the Euclidean distance metric was applied to measure the distance between each source feature sample and a reference point from the target domain. The samples with the lowest distance were then chosen to increase the similarity between source and target datasets. Finally, the transferred features are applied to a discriminative restricted Boltzmann machine classifier for P300 detection.

Results

The proposed method was evaluated on the state-of-the-art BCI Competition III dataset II and rapid serial visual presentation dataset. The results demonstrate that the proposed technique achieves an average accuracy of 97% for both online and offline after 15 repetitions, which is comparable to the state-of-the-art methods. Notably, the proposed approach requires <½ of the training samples needed by previous studies.

Discussion

Therefore, this technique offers an efficient solution for developing ERP-based BCIs with robust performance against reduced a number of training data.

Beauty is in the nose of the beholder: Fragrance modulates attractiveness, confidence and femininity ratings and neural responses to faces of self and others

Previous research investigated cross-modal influence of olfactory stimuli on perception and evaluation of faces. However, little is known about the neural dynamics underpinning this multisensory perception, and no research examined perception for images of oneself, and others, in presence of fragrances. This study investigated the neural mechanisms of olfactory-visual processing using electroencephalography (EEG) and subjective evaluations of self- and other-images. 22 female participants evaluated images of female actors and themselves while being exposed to the fragrance of a commercially available body wash or clean air delivered via olfactometer. Participants rated faces for attractiveness, femininity, confidence and glamorousness on visual analogue scales. EEG data was recorded and event-related potentials (ERPs) associated with onset of face stimuli were analysed to consider effects of fragrance presence on face processing, and interactions between fragrance and self-other image-type. Subjective ratings of confidence, attractiveness and femininity were increased for both image-types in pleasant fragrance relative to clean air condition. ERP components covering early-to-late stages of face processing were modulated by the presence of fragrance. Findings also revealed a cross-modal fragrance-face interaction, with pleasant fragrance particularly affecting ERPs to self-images in mid-latency ERP components. Results showed that the pleasant fragrance of the commercially available body wash impacted how participants perceived faces of self and others. Self- and other-image faces were subjectively rated as more attractive, confident and feminine in the presence of the pleasant fragrance compared to an un-fragranced control. The pleasant fragrance also modulated underlying electrophysiological activity. For the first time, an effect of pleasant fragrance on face perception was observed in the N1 component, suggesting impact within 100 ms. Pleasant fragrance also demonstrated greater impact on subsequent neural processing for self, relative to other-faces. The findings have implications for understanding multisensory integration during evaluations of oneself and others.

Effects of the presentation order of stimulations in sequential ERP/SSVEP Hybrid Brain-Computer Interface

Hybrid Brain-Computer Interface (hBCI) combines multiple neurophysiology modalities or paradigms to speed up the output of a single command or produce multiple ones simultaneously. Concurrent hBCIs that employ endogenous and exogenous paradigms are limited by the reduced set of possible commands. Conversely, the fusion of different exogenous visual evoked potentials demonstrated impressive performances; however, they suffer from limited portability. Yet, sequential hBCIs did not receive much attention mainly due to slower transfer rate and user fatigue during prolonged BCI use (Lorenz et al 2014 J. Neural Eng. 11 035007). Moreover, the crucial factors for optimizing the hybridization remain under-explored. In this paper, we test the feasibility of sequential Event Related-Potentials (ERP) and Steady-State Visual Evoked Potentials (SSVEP) hBCI and study the effect of stimulus order presentation between ERP-SSVEP and SSVEP-ERP for the control of directions and speed of powered wheelchairs or mobile robots with 15 commands. Exploiting the fast single trial face stimulus ERP, SSVEP and modern efficient convolutional neural networks, the configuration with SSVEP presented at first achieved significantly (p < 0.05) higher average accuracy rate with 76.39% ( ± 7.30 standard deviation) hybrid command accuracy and an average Information Transfer Rate (ITR) of 25.05 ( ± 5.32 standard deviation) bits per minute (bpm). The results of the study demonstrate the suitability of a sequential SSVEP-ERP hBCI with challenging dry electroencephalography (EEG) electrodes and low-compute capacity. Although it presents lower ITR than concurrent hBCIs, our system presents an alternative in small screen settings when the conditions for concurrent hBCIs are difficult to satisfy.

Analyzing and computing humans by means of the brain using Brain-Computer Interfaces - understanding the user - previous evidence, self-relevance and the user's self-concept as potential superordinate human factors of relevance

Brain-computer interfaces (BCIs) are well-known instances of how technology can convert a user's brain activity taken from non-invasive electroencephalography (EEG) into computer commands for the purpose of computer-assisted communication and interaction. However, not all users are attaining the accuracy required to use a BCI consistently, despite advancements in technology. Accordingly, previous research suggests that human factors could be responsible for the variance in BCI performance among users. Therefore, the user's internal mental states and traits including motivation, affect or cognition, personality traits, or the user's satisfaction, beliefs or trust in the technology have been investigated. Going a step further, this manuscript aims to discuss which human factors could be potential superordinate factors that influence BCI performance, implicitly, explicitly as well as inter- and intraindividually. Based on the results of previous studies that used comparable protocols to examine the motivational, affective, cognitive state or personality traits of healthy and vulnerable EEG-BCI users within and across well-investigated BCIs (P300-BCIs or SMR-BCIs, respectively), it is proposed that the self-relevance of tasks and stimuli and the user's self-concept provide a huge potential for BCI applications. As potential key human factors self-relevance and the user's self-concept (self-referential knowledge and beliefs about one's self) guide information processing and modulate the user's motivation, attention, or feelings of ownership, agency, and autonomy. Changes in the self-relevance of tasks and stimuli as well as self-referential processing related to one's self (self-concept) trigger changes in neurophysiological activity in specific brain networks relevant to BCI. Accordingly, concrete examples will be provided to discuss how past and future research could incorporate self-relevance and the user's self-concept in the BCI setting - including paradigms, user instructions, and training sessions.

Comparison of Two Paradigms Based on Stimulation with Images in a Spelling Brain-Computer Interface

A P300-based speller can be used to control a home automation system via brain activity. Evaluation of the visual stimuli used in a P300-based speller is a common topic in the field of brain-computer interfaces (BCIs). The aim of the present work is to compare, using the usability approach, two types of stimuli that have provided high performance in previous studies. Twelve participants controlled a BCI under two conditions, which varied in terms of the type of stimulus employed: a red famous face surrounded by a white rectangle (RFW) and a range of neutral pictures (NPs). The usability approach included variables related to effectiveness (accuracy and information transfer rate), efficiency (stress and fatigue), and satisfaction (pleasantness and System Usability Scale and Affect Grid questionnaires). The results indicated that there were no significant differences in effectiveness, but the system that used NPs was reported as significantly more pleasant. Hence, since satisfaction variables should also be considered in systems that potential users are likely to employ regularly, the use of different NPs may be a more suitable option than the use of a single RFW for the development of a home automation system based on a visual P300-based speller.

Advances in P300 brain-computer interface spellers: toward paradigm design and performance evaluation

A brain-computer interface (BCI) is a non-muscular communication technology that provides an information exchange channel for our brains and external devices. During the decades, BCI has made noticeable progress and has been applied in many fields. One of the most traditional BCI applications is the BCI speller. This article primarily discusses the progress of research into P300 BCI spellers and reviews four types of P300 spellers: single-modal P300 spellers, P300 spellers based on multiple brain patterns, P300 spellers with multisensory stimuli, and P300 spellers with multiple intelligent techniques. For each type of P300 speller, we further review several representative P300 spellers, including their design principles, paradigms, algorithms, experimental performance, and corresponding advantages. We particularly emphasized the paradigm design ideas, including the overall layout, individual symbol shapes and stimulus forms. Furthermore, several important issues and research guidance for the P300 speller were identified. We hope that this review can assist researchers in learning the new ideas of these novel P300 spellers and enhance their practical application capability.

Merging Brain-Computer Interface P300 speller datasets: Perspectives and pitfalls

Background

In the last decades, the P300 Speller paradigm was replicated in many experiments, and collected data were released to the public domain to allow research groups, particularly those in the field of machine learning, to test and improve their algorithms for higher performances of brain-computer interface (BCI) systems. Training data is needed to learn the identification of brain activity. The more training data are available, the better the algorithms will perform. The availability of larger datasets is highly desirable, eventually obtained by merging datasets from different repositories. The main obstacle to such merging is that all public datasets are released in various file formats because no standard way is established to share these data. Additionally, all datasets necessitate reading documents or scientific papers to retrieve relevant information, which prevents automating the processing. In this study, we thus adopted a unique file format to demonstrate the importance of having a standard and to propose which information should be stored and why.

Methods

We described our process to convert a dozen of P300 Speller datasets and reported the main encountered problems while converting them into the same file format. All the datasets are characterized by the same 6 × 6 matrix of alphanumeric symbols (characters and numbers or symbols) and by the same subset of acquired signals (8 EEG sensors at the same recording sites).

Results and discussion

Nearly a million stimuli were converted, relative to about 7000 spelled characters and belonging to 127 subjects. The converted stimuli represent the most extensively available platform for training and testing new algorithms on the specific paradigm - the P300 Speller. The platform could potentially allow exploring transfer learning procedures to reduce or eliminate the time needed for training a classifier to improve the performance and accuracy of such BCI systems.

The Impairment of Blood-Brain Barrier in Alzheimer's Disease: Challenges and Opportunities with Stem Cells

Alzheimer’s disease (AD) is the most common neurodegenerative disorder and its prevalence is increasing. Nowadays, very few drugs effectively reduce AD symptoms and thus, a better understanding of its pathophysiology is vital to design new effective schemes. Presymptomatic neuronal damage caused by the accumulation of Amyloid β peptide and Tau protein abnormalities remains a challenge, despite recent efforts in drug development. Importantly, therapeutic targets, biomarkers, and diagnostic techniques have emerged to detect and treat AD. Of note, the compromised blood-brain barrier (BBB) and peripheral inflammation in AD are becoming more evident, being harmful factors that contribute to the development of the disease. Perspectives from different pre-clinical and clinical studies link peripheral inflammation with the onset and progression of AD. This review aims to analyze the main factors and the contribution of impaired BBB in AD development. Additionally, we describe the potential therapeutic strategies using stem cells for AD treatment.

An efficient 3D column-only P300 speller paradigm utilizing few numbers of electrodes and flashings for practical BCI implementation

The event related P300 potentials, positive waveforms in electroencephalography (EEG) signals, are often utilized in brain computer interfaces (BCI). Many studies have been carried out to improve the performance of P300 speller systems either by developing signal processing algorithms and classifiers with different architectures or by designing new paradigms. In this study, a new paradigm is proposed for this purpose. The proposed paradigm combines two remarkable properties of being a 3D animation and utilizing column-only flashings as opposed to classical paradigms which are based on row-column flashings in 2D manner. The new paradigm is utilized in a traditional two-layer artificial neural networks model with a single output neuron, and numerous experiments are conducted to evaluate and compare the performance of the proposed paradigm with that of the classical approach. The experimental results, including statistical significance tests, are presented for single and multiple EEG electrode usage combinations in 1, 3 and 15 flashing repetitions to detect P300 waves as well as to recognize target characters. Using the proposed paradigm, the best average classification accuracy rates on the test data are improved from 89.97% to 93.90% (an improvement of 4.36%) for 1 flashing, from 97.11% to 98.10% (an improvement of 1.01%) for 3 flashings and from 99.70% to 99.81% (an improvement of 0.11%) for 15 flashings when all electrodes, included in the study, are utilized. On the other hand, the accuracy rates are improved by 9.69% for 1 flashing, 4.72% for 3 flashings and 1.73% for 15 flashings when the proposed paradigm is utilized with a single EEG electrode (P8). It is observed that the proposed speller paradigm is especially useful in BCI systems designed for few EEG electrodes usage, and hence, it is more suitable for practical implementations. Moreover, all participants, given a subjective test, declared that the proposed paradigm is more user-friendly than classical ones.

Different effects of using pictures as stimuli in a P300 brain-computer interface under rapid serial visual presentation or row-column paradigm

Previous proposals for controlling a P300-based BCI speller have shown an improvement using alternative images instead of letters as target stimuli under a row-column paradigm (RCP). However, the RCP is not suitable for those patients with a lack of gaze control. To solve that, the rapid serial visual presentation (RSVP) paradigm has been proposed in previous studies. The aim of the present work is to assess if a set of alternative pictures that improved performance in RCP could also improve performance in RSVP. Sixteen participants controlled four conditions in calibration and online tasks: letters in RCP, pictures in RCP, letters in RSVP and pictures in RSVP. The effect given by pictures was greater under RCP than under RSVP, both for performance and event-related potential analyses. Indeed, pictures did not show any improvement under RSVP in comparison to letters. In addition, the condition with pictures under RCP was declared the favourite by most users (68.75%), while the condition with pictures under RSVP was not chosen as favourite by any participant. Therefore, this work shows that the improvement related to the use of pictures as alternative flashing stimuli under RCP may not be transferred to RSVP. Graphical abstract.

Performance Analysis With Different Types of Visual Stimuli in a BCI-Based Speller Under an RSVP Paradigm

Brain-Computer Interface (BCI) systems enable an alternative communication channel for severely-motor disabled patients to interact with their environment using no muscular movements. In recent years, the importance of research into non-gaze dependent brain-computer interface paradigms has been increasing, in contrast to the most frequently studied BCI-based speller paradigm (i.e., row-column presentation, RCP). Several visual modifications that have already been validated under the RCP paradigm for communication purposes have not been validated under the most extended non-gaze dependent rapid serial visual presentation (RSVP) paradigm. Thus, in the present study, three different sets of stimuli were assessed under RSVP, with the following communication features: white letters (WL), famous faces (FF), neutral pictures (NP). Eleven healthy subjects participated in this experiment, in which the subjects had to go through a calibration phase, an online phase and, finally, a subjective questionnaire completion phase. The results showed that the FF and NP stimuli promoted better performance in the calibration and online phases, being slightly better in the FF paradigm. Regarding the subjective questionnaires, again both FF and NP were preferred by the participants in contrast to the WL stimuli, but this time the NP stimuli scored slightly higher. These findings suggest that the use of FF and NP for RSVP-based spellers could be beneficial to increase information transfer rate in comparison to the most frequently used letter-based stimuli and could represent a promising communication system for individuals with altered ocular-motor function.

The Behavioral and ERP Responses to Self- and Other- Referenced Adjectives

The aim was to investigate behavioral reactions and event-related potential (ERP) responses in healthy participants under conditions of personalized attribution of emotional appraisal vocabulary to one-self or to other people. One hundred and fifty emotionally neutral, positive and negative words describing people's traits were used. Subjects were asked to attribute each word to four types of people: one-self, loved, unpleasant and neutral person. The reaction time during adjectives attribution to one-self and a loved person was shorter than during adjectives attribution to neutral and unpleasant people. Self-related adjectives induced higher amplitudes of the N400 ERP peak in the medial cortical areas in comparison with adjectives related to other people. The amplitude of P300 and P600 depended on the emotional valence of assessments, but not on the personalized attribution. The interaction between the attribution effect and the effect of emotional valence of assessments was observed for the N400 peak in the left temporal area. The maximal amplitude of N400 was revealed under self-attributing of emotionally positive adjectives. Our results supported the hypothesis that the emotional valence of assessments and the processing of information about one-self or others were related to the brain processes that differ from each other in a cortical localization or time dynamics.