Toward enhanced P300 speller performance

EEG Baseline Analysis for Effective Extraction of P300 Event-Related Potentials for Welfare Interfaces

Enabling individuals with complete paralysis to operate devices voluntarily requires an effective interface; EEG-based P300 event-related potential (ERP) interfaces are considered a promising approach. P300 is an EEG peak generated in response to specific sensory stimuli recognized by an individual. Accurate detection of this peak necessitates a stable pre-stimulus baseline EEG signal, which serves as the reference for baseline correction. Previous studies have commonly employed either a single-time-point amplitude (e.g., at 100 ms before stimulus onset) or a time-range-averaged amplitude over a specified pre-stimulus period (e.g., 0-200 ms) as a baseline correction method, assuming these provide the most stable EEG reference. However, in assistive P300 interfaces, continuous visual stimuli at 400 ms intervals are typically used to efficiently evoke P300 peaks. Since stimuli are presented before the EEG stabilizes, it remains unclear whether conventional neuroscience baseline correction methods are suitable for such applications. To address this, the present study conducted a P300 induction experiment based on continuous 400 ms interval visual stimuli. Using EEG data recorded from 0 to 1000 ms before each visual stimulus (sampled at 1 ms intervals), we applied three baseline correction methods-single-time-point amplitude, time-range-averaged amplitude, and multi-time-point amplitude-to determine the most effective EEG reference and evaluate the impact on P300 detection performance. The results showed that baseline correction using an amplitude at a single point in time is unstable when the basic EEG rhythm and low-frequency noise remain, while time-range-averaged baseline correction using the 0-200 ms pre-stimulus period led to relatively effective P300 detection. However, it was also found that using only one value averaged over the amplitude from 0 to 200 ms did not result in an accurate EEG reference potential, resulting in an error. Finally, this study confirmed that the multi-time-point baseline correction method, through which the amplitude state from 0 to 200 ms before the visual stimulus is comprehensively evaluated, may be the most effective method for P300 determination.

An auto-segmented multi-time window dual-scale neural network for brain-computer interfaces based on event-related potentials

Objective.Event-related potentials (ERPs) are cerebral responses to cognitive processes, also referred to as cognitive potentials. Accurately decoding ERPs can help to advance research on brain-computer interfaces (BCIs). The spatial pattern of ERP varies with time. In recent years, convolutional neural networks (CNNs) have shown promising results in electroencephalography (EEG) classification, specifically for ERP-based BCIs.Approach.This study proposes an auto-segmented multi-time window dual-scale neural network (AWDSNet). The combination of a multi-window design and a lightweight base network gives AWDSNet good performance at an acceptable cost of computing. For each individual, we create a time window set by calculating the correlation of signedR-squared values, which enables us to determine the length and number of windows automatically. The signal data are segmented based on the obtained window sets in sub-plus-global mode. Then, the multi-window data are fed into a dual-scale CNN model, where the sizes of the convolution kernels are determined by the window sizes. The use of dual-scale spatiotemporal convolution focuses on feature details while also having a large enough receptive length, and the grouping parallelism undermines the increase in the number of parameters that come with dual scaling.Main results.We evaluated the performance of AWDSNet on a public dataset and a self-collected dataset. A comparison was made with four popular methods including EEGNet, DeepConvNet, EEG-Inception, and PPNN. The experimental results show that AWDSNet has excellent classification performance with acceptable computational complexity.Significance.These results indicate that AWDSNet has great potential for applications in ERP decoding.

Multiband task related components enhance rapid cognition decoding for both small and similar objects

The cortically-coupled target recognition system based on rapid serial visual presentation (RSVP) has a wide range of applications in brain computer interface (BCI) fields such as medical and military. However, in the complex natural environment backgrounds, the identification of event-related potentials (ERP) of both small and similar objects that are quickly presented is a research challenge. Therefore, we designed corresponding experimental paradigms and proposed a multi-band task related components matching (MTRCM) method to improve the rapid cognitive decoding of both small and similar objects. We compared the areas under the receiver operating characteristic curve (AUC) between MTRCM and other 9 methods under different numbers of training sample using RSVP-ERP data from 50 subjects. The results showed that MTRCM maintained an overall superiority and achieved the highest average AUC (0.6562 ± 0.0091). We also optimized the frequency band and the time parameters of the method. The verification on public data sets further showed the necessity of designing MTRCM method. The MTRCM method provides a new approach for neural decoding of both small and similar RSVP objects, which is conducive to promote the further development of RSVP-BCI.

Millán J, Sulzer J, Lewis-Peacock JA. Evaluating the Feasibility of Visual Imagery for an EEG-Based Brain-Computer Interface

Visual imagery, or the mental simulation of visual information from memory, could serve as an effective control paradigm for a brain-computer interface (BCI) due to its ability to directly convey the user's intention with many natural ways of envisioning an intended action. However, multiple initial investigations into using visual imagery as a BCI control strategies have been unable to fully evaluate the capabilities of true spontaneous visual mental imagery. One major limitation in these prior works is that the target image is typically displayed immediately preceding the imagery period. This paradigm does not capture spontaneous mental imagery as would be necessary in an actual BCI application but something more akin to short-term retention in visual working memory. Results from the present study show that short-term visual imagery following the presentation of a specific target image provides a stronger, more easily classifiable neural signature in EEG than spontaneous visual imagery from long-term memory following an auditory cue for the image. We also show that short-term visual imagery and visual perception share commonalities in the most predictive electrodes and spectral features. However, visual imagery received greater influence from frontal electrodes whereas perception was mostly confined to occipital electrodes. This suggests that visual perception is primarily driven by sensory information whereas visual imagery has greater contributions from areas associated with memory and attention. This work provides the first direct comparison of short-term and long-term visual imagery tasks and provides greater insight into the feasibility of using visual imagery as a BCI control strategy.

Comprehensive evaluation methods for translating BCI into practical applications: usability, user satisfaction and usage of online BCI systems

Although brain-computer interface (BCI) is considered a revolutionary advancement in human-computer interaction and has achieved significant progress, a considerable gap remains between the current technological capabilities and their practical applications. To promote the translation of BCI into practical applications, the gold standard for online evaluation for classification algorithms of BCI has been proposed in some studies. However, few studies have proposed a more comprehensive evaluation method for the entire online BCI system, and it has not yet received sufficient attention from the BCI research and development community. Therefore, the qualitative leap from analyzing and modeling for offline BCI data to the construction of online BCI systems and optimizing their performance is elaborated, and then user-centred is emphasized, and then the comprehensive evaluation methods for translating BCI into practical applications are detailed and reviewed in the article, including the evaluation of the usability (including effectiveness and efficiency of systems), the evaluation of the user satisfaction (including BCI-related aspects, etc.), and the evaluation of the usage (including the match between the system and user, etc.) of online BCI systems. Finally, the challenges faced in the evaluation of the usability and user satisfaction of online BCI systems, the efficacy of online BCI systems, and the integration of BCI and artificial intelligence (AI) and/or virtual reality (VR) and other technologies to enhance the intelligence and user experience of the system are discussed. It is expected that the evaluation methods for online BCI systems elaborated in this review will promote the translation of BCI into practical applications.

EEG decoding for effects of visual joint attention training on ASD patients with interpretable and lightweight convolutional neural network

Visual joint attention, the ability to track gaze and recognize intent, plays a key role in the development of social and language skills in health humans, which is performed abnormally hard in autism spectrum disorder (ASD). The traditional convolutional neural network, EEGnet, is an effective model for decoding technology, but few studies have utilized this model to address attentional training in ASD patients. In this study, EEGNet was used to decode the P300 signal elicited by training and the saliency map method was used to visualize the cognitive properties of ASD patients during visual attention. The results showed that in the spatial distribution, the parietal lobe was the main region of classification contribution, especially for Pz electrode. In the temporal information, the time period from 300 to 500 ms produced the greatest contribution to the electroencephalogram (EEG) classification, especially around 300 ms. After training for ASD patients, the gradient contribution was significantly enhanced at 300 ms, which was effective only in social scenarios. Meanwhile, with the increase of joint attention training, the P300 latency of ASD patients gradually shifted forward in social scenarios, but this phenomenon was not obvious in non-social scenarios. Our results indicated that joint attention training could improve the cognitive ability and responsiveness of social characteristics in ASD patients.

Post-stroke aphasia rehabilitation using an adapted visual P300 brain-computer interface training: improvement over time, but specificity remains undetermined

Introduction

This study aimed to evaluate the efficacy of visual P300 brain-computer interface use to support rehabilitation of chronic language production deficits commonly experienced by individuals with a left-sided stroke resulting in post-stroke aphasia.

Methods

The study involved twelve participants, but five dropped out. Additionally, data points were missing for three participants in the remaining sample of seven participants. The participants underwent four assessments-a baseline, pre-assessment, post-assessment, and follow-up assessment. Between the pre-and post-assessment, the participants underwent at least 14 sessions of visual spelling using a brain-computer interface. The study aimed to investigate the impact of this intervention on attention, language production, and language comprehension and to determine whether there were any potential effects on quality of life and well-being.

Results

None of the participants showed a consistent improvement in attention. All participants showed an improvement in spontaneous speech production, and three participants experienced a reduction in aphasia severity. We found an improvement in subjective quality of life and daily functioning. However, we cannot rule out the possibility of unspecific effects causing or at least contributing to these results.

Conclusion

Due to challenges in assessing the patient population, resulting in a small sample size and missing data points, the results of using visual P300 brain-computer interfaces for chronic post-stroke aphasia rehabilitation are preliminary. Thus, we cannot decisively judge the potential of this approach.

On the role of generative artificial intelligence in the development of brain-computer interfaces

Since their inception more than 50 years ago, Brain-Computer Interfaces (BCIs) have held promise to compensate for functions lost by people with disabilities through allowing direct communication between the brain and external devices. While research throughout the past decades has demonstrated the feasibility of BCI to act as a successful assistive technology, the widespread use of BCI outside the lab is still beyond reach. This can be attributed to a number of challenges that need to be addressed for BCI to be of practical use including limited data availability, limited temporal and spatial resolutions of brain signals recorded non-invasively and inter-subject variability. In addition, for a very long time, BCI development has been mainly confined to specific simple brain patterns, while developing other BCI applications relying on complex brain patterns has been proven infeasible. Generative Artificial Intelligence (GAI) has recently emerged as an artificial intelligence domain in which trained models can be used to generate new data with properties resembling that of available data. Given the enhancements observed in other domains that possess similar challenges to BCI development, GAI has been recently employed in a multitude of BCI development applications to generate synthetic brain activity; thereby, augmenting the recorded brain activity. Here, a brief review of the recent adoption of GAI techniques to overcome the aforementioned BCI challenges is provided demonstrating the enhancements achieved using GAI techniques in augmenting limited EEG data, enhancing the spatiotemporal resolution of recorded EEG data, enhancing cross-subject performance of BCI systems and implementing end-to-end BCI applications. GAI could represent the means by which BCI would be transformed into a prevalent assistive technology, thereby improving the quality of life of people with disabilities, and helping in adopting BCI as an emerging human-computer interaction technology for general use.

The Effect of Jittered Stimulus Onset Interval on Electrophysiological Markers of Attention in a Brain-Computer Interface Rapid Serial Visual Presentation Paradigm

Brain responses to discrete stimuli are modulated when multiple stimuli are presented in sequence. These alterations are especially pronounced when the time course of an evoked response overlaps with responses to subsequent stimuli, such as in a rapid serial visual presentation (RSVP) paradigm used to control a brain-computer interface (BCI). The present study explored whether the measurement or classification of select brain responses during RSVP would improve through application of an established technique for dealing with overlapping stimulus presentations, known as irregular or "jittered" stimulus onset interval (SOI). EEG data were collected from 24 healthy adult participants across multiple rounds of RSVP calibration and copy phrase tasks with varying degrees of SOI jitter. Analyses measured three separate brain signals sensitive to attention: N200, P300, and occipitoparietal alpha attenuation. Presentation jitter visibly reduced intrusion of the SSVEP, but in general, it did not positively or negatively affect attention effects, classification, or system performance. Though it remains unclear whether stimulus overlap is detrimental to BCI performance overall, the present study demonstrates that single-trial classification approaches may be resilient to rhythmic intrusions like SSVEP that appear in the averaged EEG.

Evaluating Deep Learning Performance for P300 Neural Signal Classification

P300 event-related potential (ERP) signals are useful neurological biomarkers, and their accurate classification is important when studying the cognitive functions in patients with neurological disorders. While many studies have proposed models for classifying these signals, results have been inconsistent. As a result, a consensus has not yet been reached on the optimal model for this classification. In this study, we evaluated the performance of classic machine learning and novel deep learning methods for P300 signal classification in both within and across subject training scenarios across a dataset of 75 subjects. Although the deep learning models attained high attended event classification F1 scores, they did not outperform Stepwise Linear Discriminant Analysis (SWLDA) in the within-subject paradigm. In the across-subject paradigm, however, EEG-Inception was able to significantly outperform SWLDA. These results suggest that deep learning models may provide a general model that do not require subject-specific training and calibration in clinical settings.

Multimodal pre-screening can predict BCI performance variability: A novel subject-specific experimental scheme

BACKGROUND

Brain-computer interface (BCI) systems currently lack the required robustness for long-term daily use due to inter- and intra-subject performance variability. In this study, we propose a novel personalized scheme for a multimodal BCI system, primarily using functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG), to identify, predict, and compensate for factors affecting competence-related and interfering factors associated with performance.

METHOD

11 (out of 13 recruited) participants, including five participants with motor deficits, completed four sessions on average. During the training sessions, the subjects performed a short pre-screening phase, followed by three variations of a novel visou-mental (VM) protocol. Features extracted from the pre-screening phase were used to construct predictive platforms using stepwise multivariate linear regression (MLR) models. In the test sessions, we employed a task-correction phase where our predictive models were used to predict the ideal task variation to maximize performance, followed by an interference-correction phase. We then investigated the associations between predicted and actual performances and evaluated the outcome of correction strategies.

RESULT

The predictive models resulted in respective adjusted R-squared values of 0.942, 0.724, and 0.939 for the first, second, and third variation of the task, respectively. The statistical analyses showed significant associations between the performances predicted by predictive models and the actual performances for the first two task variations, with rhos of 0.7289 (p-value = 0.011) and 0.6970 (p-value = 0.017), respectively. For 81.82 % of the subjects, the task/workload correction stage correctly determined which task variation provided the highest accuracy, with an average performance gain of 5.18 % when applying the correction strategies.

CONCLUSION

Our proposed method can lead to an integrated multimodal predictive framework to compensate for BCI performance variability, particularly, for people with severe motor deficits.

Videoconference fatigue from a neurophysiological perspective: experimental evidence based on electroencephalography (EEG) and electrocardiography (ECG)

In the recent past, many organizations and people have substituted face-to-face meetings with videoconferences. Among others, tools like Zoom, Teams, and Webex have become the "new normal" of human social interaction in many domains (e.g., business, education). However, this radical adoption and extensive use of videoconferencing tools also has a dark side, referred to as videoconference fatigue (VCF). To date only self-report evidence has shown that VCF is a serious issue. However, based on self-reports alone it is hardly possible to provide a comprehensive understanding of a cognitive phenomenon like VCF. Against this background, we examined VCF also from a neurophysiological perspective. Specifically, we collected and analyzed electroencephalography (continuous and event-related) and electrocardiography (heart rate and heart rate variability) data to investigate whether VCF can also be proven on a neurophysiological level. We conducted a laboratory experiment based on a within-subjects design (N = 35). The study context was a university lecture, which was given in a face-to-face and videoconferencing format. In essence, the neurophysiological data-together with questionnaire data that we also collected-show that 50 min videoconferencing, if compared to a face-to-face condition, results in changes in the human nervous system which, based on existing literature, can undoubtedly be interpreted as fatigue. Thus, individuals and organizations must not ignore the fatigue potential of videoconferencing. A major implication of our study is that videoconferencing should be considered as a possible complement to face-to-face interaction, but not as a substitute.

Improving Alzheimer's Disease Classification in Brain MRI Images Using a Neural Network Model Enhanced with PCA and SWLDA

The examination of Alzheimer's disease (AD) using adaptive machine learning algorithms has unveiled promising findings. However, achieving substantial credibility in medical contexts necessitates a combination of notable accuracy, minimal processing time, and universality across diverse populations. Therefore, we have formulated a hybrid methodology in this study to classify AD by employing a brain MRI image dataset. We incorporated an averaging filter during preprocessing in the initial stage to reduce extraneous details. Subsequently, a combined strategy was utilized, involving principal component analysis (PCA) in conjunction with stepwise linear discriminant analysis (SWLDA), followed by an artificial neural network (ANN). SWLDA employs a combination of forward and backward recursion methods to choose a restricted set of features. The forward recursion identifies the most interconnected features based on partial Z-test values. Conversely, the backward recursion method eliminates the least correlated features from the same feature space. After the extraction and selection of features, an optimized artificial neural network (ANN) was utilized to differentiate the various classes of AD. To demonstrate the significance of this hybrid approach, we utilized publicly available brain MRI datasets using a 10-fold cross-validation strategy. The proposed method excelled over existing state-of-the-art systems, attaining weighted average recognition rates of 99.35% and 96.66%, respectively, across all the datasets.

Brain-computer interface: trend, challenges, and threats

Brain-computer interface (BCI), an emerging technology that facilitates communication between brain and computer, has attracted a great deal of research in recent years. Researchers provide experimental results demonstrating that BCI can restore the capabilities of physically challenged people, hence improving the quality of their lives. BCI has revolutionized and positively impacted several industries, including entertainment and gaming, automation and control, education, neuromarketing, and neuroergonomics. Notwithstanding its broad range of applications, the global trend of BCI remains lightly discussed in the literature. Understanding the trend may inform researchers and practitioners on the direction of the field, and on where they should invest their efforts more. Noting this significance, we have analyzed 25,336 metadata of BCI publications from Scopus to determine advancement of the field. The analysis shows an exponential growth of BCI publications in China from 2019 onwards, exceeding those from the United States that started to decline during the same period. Implications and reasons for this trend are discussed. Furthermore, we have extensively discussed challenges and threats limiting exploitation of BCI capabilities. A typical BCI architecture is hypothesized to address two prominent BCI threats, privacy and security, as an attempt to make the technology commercially viable to the society.

Combining brain-computer interfaces and multiplayer video games: an application based on c-VEPs

Introduction and objective

Video games are crucial to the entertainment industry, nonetheless they can be challenging to access for those with severe motor disabilities. Brain-computer interfaces (BCI) systems have the potential to help these individuals by allowing them to control video games using their brain signals. Furthermore, multiplayer BCI-based video games may provide valuable insights into how competitiveness or motivation affects the control of these interfaces. Despite the recent advancement in the development of code-modulated visual evoked potentials (c-VEPs) as control signals for high-performance BCIs, to the best of our knowledge, no studies have been conducted to develop a BCI-driven video game utilizing c-VEPs. However, c-VEPs could enhance user experience as an alternative method. Thus, the main goal of this work was to design, develop, and evaluate a version of the well-known 'Connect 4' video game using a c-VEP-based BCI, allowing 2 users to compete by aligning 4 same-colored coins vertically, horizontally or diagonally.

Methods

The proposed application consists of a multiplayer video game controlled by a real-time BCI system processing 2 electroencephalograms (EEGs) sequentially. To detect user intention, columns in which the coin can be placed was encoded with shifted versions of a pseudorandom binary code, following a traditional circular shifting c-VEP paradigm. To analyze the usability of our application, the experimental protocol comprised an evaluation session by 22 healthy users. Firstly, each user had to perform individual tasks. Afterward, users were matched and the application was used in competitive mode. This was done to assess the accuracy and speed of selection. On the other hand, qualitative data on satisfaction and usability were collected through questionnaires.

Results

The average accuracy achieved was 93.74% ± 1.71%, using 5.25 seconds per selection. The questionnaires showed that users felt a minimal workload. Likewise, high satisfaction values were obtained, highlighting that the application was intuitive and responds quickly and smoothly.

Conclusions

This c-VEP based multiplayer video game has reached suitable performance on 22 users, supported by high motivation and minimal workload. Consequently, compared to other versions of "Connect 4" that utilized different control signals, this version has exhibited superior performance.

SKDCPM algorithm can improve the single-trial decoding performance of very similar error-related potentials. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-4. [PMID: 38083659 DOI: 10.1109/embc40787.2023.10340825] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Error related potential (ErrP) is an effective control signal for the brain-computer interface (BCI). Current ErrP decoding methods can only distinguish right and wrong mental states. However, in real scenarios, error conditions often contain more detailed information, such as the degree of error, which would induce very similar ErrPs. Distinguishing such ErrPs effectively is of vital importance to provide more detailed information for optimizing BCIs. Hereto, a major challenge is the EEG differences of very similar ErrPs are very small. Thus, it is necessary to develop new efficient method for decoding very similar ErrPs. This study newly proposed an algorithm named shrinkage discriminant canonical pattern matching (SKDCPM), and compared its decoding results with the linear discriminant analysis (LDA), shrinkage LDA (SKLDA), stepwise LDA (SWLDA), Bayesian LDA (BLDA) and the DCPM, which were algorithms commonly used for ErrP decoding. A data set of 18 subjects was built, it had four conditions, i.e., right (0°), errors with varying degrees, i.e., 45°, 90°, 180° deviation from the predicted direction. As a result, the SKDCPM had high balanced accuracy (BACC) in right-wrong classification (0° vs. others). More importantly, it achieved a grand averaged BACC of 69.54% with the highest up to 74.25%, which outperformed all the other algorithms in very similar ErrPs decoding (45° vs. 90° vs. 180°) significantly. This study could provide new decoding methods for developing the ErrP-based BCI system.

Identifying markers volatiles in Brazilian virgin oil by multiple headspace solid-phase microextraction, and chemometrics tools

A protocol was optimized to determine the volatile profile from monovarietal virgin olive oil (VOO) by multiple headspace solid-phase microextraction (MHS-SPME) followed by gas chromatography-mass spectrometry (GC-MS) analysis. For this, a Plackett-Burman (PB) and central composite rotational designs (CCRD) were used to define the best condition of extraction. Moreover, fatty acids profile and principal component analysis (PCA) was used to identify markers among the cultivars. The amount of 0.1 g of sample was enough to express the volatile composition of the olive oils by MHS-SPME. Volatile compounds [nonanal, (Z)-3-Hexen-1-ol, (Z)-3-Hexenyl Acetate, Hexyl Acetate, 3-Methylbutyl Acetate, (E)-2-Hexen-1-ol, (E)-2-Hexenyl Acetate] and fatty acids [C17:1, C18, C18:1, C18:2] were those reported such as the markers in the varieties of olive oils. The PCA analysis allowed the classification of the most representative volatiles and fatty acids for each cultivar. Through two principal components was possible to obtain 81.9% of explanation of the variance of the compounds. The compounds were quantified using a validated method. The MHS-SPME combined with multivariate analysis showed a promising tool to identify markers and for the discrimination of olive oil varieties.

Assembling global and local spatial-temporal filters to extract discriminant information of EEG in RSVP task

Objective.Brain-computer interface (BCI) system has developed rapidly in the past decade. And rapid serial visual presentation (RSVP) is an important BCI paradigm to detect the targets in high-speed image streams. For decoding electroencephalography (EEG) in RSVP task, the ensemble-model methods have better performance than the single-model ones.Approach.This study proposed a method based on ensemble learning to extract discriminant information of EEG. An extreme gradient boosting framework was utilized to sequentially generate the sub models, including one global spatial-temporal filter and a group of local ones. EEG was reshaped into a three-dimensional form by remapping the electrode dimension into a 2D array to learn the spatial-temporal features from real local space.Main results.A benchmark RSVP EEG dataset was utilized to evaluate the performance of the proposed method, where EEG data of 63 subjects were analyzed. Compared with several state-of-the-art methods, the spatial-temporal patterns of proposed method were more consistent with P300, and the proposed method can provide significantly better classification performance.Significance.The ensemble model in this study was end-to-end optimized, which can avoid error accumulation. The sub models optimized by gradient boosting theory can extract discriminant information complementarily and non-redundantly.

A dynamical graph-based feature extraction approach to enhance mental task classification in brain-computer interfaces

Graph theoretic approaches in analyzing spatiotemporal dynamics of brain activities are under-studied but could be very promising directions in developing effective brain-computer interfaces (BCIs). Many existing BCI systems use electroencephalogram (EEG) signals to record and decode human neural activities noninvasively. Often, however, the features extracted from the EEG signals ignore the topological information hidden in the EEG temporal dynamics. Moreover, existing graph theoretic approaches are mostly used to reveal the topological patterns of brain functional networks based on synchronization between signals from distinctive spatial regions, instead of interdependence between states at different timestamps. In this study, we present a robust fold-wise hyperparameter optimization framework utilizing a series of conventional graph-based measurements combined with spectral graph features and investigate its discriminative performance on classification of a designed mental task in 6 participants with amyotrophic lateral sclerosis (ALS). Across all of our participants, we reached an average accuracy of 71.1%±4.5% for mental task classification by combining the global graph-based measurements and the spectral graph features, higher than the conventional non-graph based feature performance (67.1%±7.5%). Compared to using either one of the graphic features (66.3%±6.5% for the eigenvalues and 65.9%±5.2% for the global graph features), our feature combination strategy shows considerable improvement in both accuracy and robustness performance. Our results indicate the feasibility and advantage of the presented fold-wise optimization framework utilizing graph-based features in BCI systems targeted at end-users.

Advances in P300 brain-computer interface spellers: toward paradigm design and performance evaluation

A brain-computer interface (BCI) is a non-muscular communication technology that provides an information exchange channel for our brains and external devices. During the decades, BCI has made noticeable progress and has been applied in many fields. One of the most traditional BCI applications is the BCI speller. This article primarily discusses the progress of research into P300 BCI spellers and reviews four types of P300 spellers: single-modal P300 spellers, P300 spellers based on multiple brain patterns, P300 spellers with multisensory stimuli, and P300 spellers with multiple intelligent techniques. For each type of P300 speller, we further review several representative P300 spellers, including their design principles, paradigms, algorithms, experimental performance, and corresponding advantages. We particularly emphasized the paradigm design ideas, including the overall layout, individual symbol shapes and stimulus forms. Furthermore, several important issues and research guidance for the P300 speller were identified. We hope that this review can assist researchers in learning the new ideas of these novel P300 spellers and enhance their practical application capability.

Merging Brain-Computer Interface P300 speller datasets: Perspectives and pitfalls

Background

In the last decades, the P300 Speller paradigm was replicated in many experiments, and collected data were released to the public domain to allow research groups, particularly those in the field of machine learning, to test and improve their algorithms for higher performances of brain-computer interface (BCI) systems. Training data is needed to learn the identification of brain activity. The more training data are available, the better the algorithms will perform. The availability of larger datasets is highly desirable, eventually obtained by merging datasets from different repositories. The main obstacle to such merging is that all public datasets are released in various file formats because no standard way is established to share these data. Additionally, all datasets necessitate reading documents or scientific papers to retrieve relevant information, which prevents automating the processing. In this study, we thus adopted a unique file format to demonstrate the importance of having a standard and to propose which information should be stored and why.

Methods

We described our process to convert a dozen of P300 Speller datasets and reported the main encountered problems while converting them into the same file format. All the datasets are characterized by the same 6 × 6 matrix of alphanumeric symbols (characters and numbers or symbols) and by the same subset of acquired signals (8 EEG sensors at the same recording sites).

Results and discussion

Nearly a million stimuli were converted, relative to about 7000 spelled characters and belonging to 127 subjects. The converted stimuli represent the most extensively available platform for training and testing new algorithms on the specific paradigm - the P300 Speller. The platform could potentially allow exploring transfer learning procedures to reduce or eliminate the time needed for training a classifier to improve the performance and accuracy of such BCI systems.

A BCI painting system using a hybrid control approach based on SSVEP and P300

Brain-computer interfaces (BCIs) can help people with disabilities to communicate with others, express themselves, and even create art. In this paper, a BCI painting system using a hybrid control approach based on steady-state visual evoked potential (SSVEP) and P300 was developed, which can enable simple painting through brain-controlled painting tools. The BCI painting system is composed of two parts: a hybrid stimulus interface and a hybrid electroencephalogram (EEG) signal processing module. The user selects the menus and tools through the SSVEP and P300 stimulus matrices, respectively, and the paintings are displayed in the canvas area of the hybrid stimulus interface in real time. Twenty subjects participated in this study. An offline training experiment was performed to construct the P300 and SSVEP recognition models for each subject; an online painting experiment, which included a copy-painting task and a free-painting task, was performed to evaluate the BCI painting system. The results of the online painting experiment showed that the average tool selection accuracy (88.92 ± 3.94%) of the BCI painting system using the hybrid stimulus interface was slightly higher than that of the traditional brain painting system based on the P300 stimulus interface; the average information transfer rate (ITR) (74.20 ± 5.28 bpm, 71.80 ± 5.15 bpm) in the copy-painting and free-painting tasks of the BCI painting system was significantly higher than that of the traditional brain painting system. Our BCI painting system can effectively help users express their artistic creativity and improve their painting efficiency, and can provide new methods and new ideas for developing BCI-controlled applications.

A novel brain-controlled wheelchair combined with computer vision and augmented reality

BACKGROUND

Brain-controlled wheelchairs (BCWs) are important applications of brain-computer interfaces (BCIs). Currently, most BCWs are semiautomatic. When users want to reach a target of interest in their immediate environment, this semiautomatic interaction strategy is slow.

METHODS

To this end, we combined computer vision (CV) and augmented reality (AR) with a BCW and proposed the CVAR-BCW: a BCW with a novel automatic interaction strategy. The proposed CVAR-BCW uses a translucent head-mounted display (HMD) as the user interface, uses CV to automatically detect environments, and shows the detected targets through AR technology. Once a user has chosen a target, the CVAR-BCW can automatically navigate to it. For a few scenarios, the semiautomatic strategy might be useful. We integrated a semiautomatic interaction framework into the CVAR-BCW. The user can switch between the automatic and semiautomatic strategies.

RESULTS

We recruited 20 non-disabled subjects for this study and used the accuracy, information transfer rate (ITR), and average time required for the CVAR-BCW to reach each designated target as performance metrics. The experimental results showed that our CVAR-BCW performed well in indoor environments: the average accuracies across all subjects were 83.6% (automatic) and 84.1% (semiautomatic), the average ITRs were 8.2 bits/min (automatic) and 8.3 bits/min (semiautomatic), the average times required to reach a target were 42.4 s (automatic) and 93.4 s (semiautomatic), and the average workloads and degrees of fatigue for the two strategies were both approximately 20.

CONCLUSIONS

Our CVAR-BCW provides a user-centric interaction approach and a good framework for integrating more advanced artificial intelligence technologies, which may be useful in the field of disability assistance.

Spaceborne GNSS-R Wind Speed Retrieval Using Machine Learning Methods

This paper focuses on sea surface wind speed estimation using L1B level v3.1 data of reflected GNSS signals from the Cyclone GNSS (CYGNSS) mission and European Centre for Medium-range Weather Forecast Reanalysis (ECMWF) wind speed data. Seven machine learning methods are applied for wind speed retrieval, i.e., Regression trees (Binary Tree (BT), Ensembles of Trees (ET), XGBoost (XGB), LightGBM (LGBM)), ANN (Artificial neural network), Stepwise Linear Regression (SLR), and Gaussian Support Vector Machine (GSVM), and a comparison of their performance is made. The wind speed is divided into two different ranges to study the suitability of the different algorithms. A total of 10 observation variables are considered as input parameters to study the importance of individual variables or combinations thereof. The results show that the LGBM model performs the best with an RMSE of 1.419 and a correlation coefficient of 0.849 in the low wind speed interval (0–15 m/s), while the ET model performs the best with an RMSE of 1.100 and a correlation coefficient of 0.767 in the high wind speed interval (15–30 m/s). The effects of the variables used in wind speed retrieval models are investigated using the XGBoost importance metric, showing that a number of variables play a very significant role in wind speed retrieval. It is expected that these results will provide a useful reference for the development of advanced wind speed retrieval algorithms in the future.

Cross-Platform Implementation of an SSVEP-Based BCI for the Control of a 6-DOF Robotic Arm

Robotics has been successfully applied in the design of collaborative robots for assistance to people with motor disabilities. However, man-machine interaction is difficult for those who suffer severe motor disabilities. The aim of this study was to test the feasibility of a low-cost robotic arm control system with an EEG-based brain-computer interface (BCI). The BCI system relays on the Steady State Visually Evoked Potentials (SSVEP) paradigm. A cross-platform application was obtained in C++. This C++ platform, together with the open-source software Openvibe was used to control a Stäubli robot arm model TX60. Communication between Openvibe and the robot was carried out through the Virtual Reality Peripheral Network (VRPN) protocol. EEG signals were acquired with the 8-channel Enobio amplifier from Neuroelectrics. For the processing of the EEG signals, Common Spatial Pattern (CSP) filters and a Linear Discriminant Analysis classifier (LDA) were used. Five healthy subjects tried the BCI. This work allowed the communication and integration of a well-known BCI development platform such as Openvibe with the specific control software of a robot arm such as Stäubli TX60 using the VRPN protocol. It can be concluded from this study that it is possible to control the robotic arm with an SSVEP-based BCI with a reduced number of dry electrodes to facilitate the use of the system.

A P300-Detection Method Based on Logistic Regression and a Convolutional Neural Network

Background

Electroencephalogram (EEG)-based brain-computer interface (BCI) systems are widely utilized in various fields, including health care, intelligent assistance, identity recognition, emotion recognition, and fatigue detection. P300, the main event-related potential, is the primary component detected by EEG-based BCI systems. Existing algorithms for P300 classification in EEG data usually perform well when tested in a single participant, although they exhibit significant decreases in accuracy when tested in new participants. We attempted to address this lack of generalizability associated with existing classification methods using a novel convolutional neural network (CNN) model developed using logistic regression (LR).

Materials and Methods

We proposed an LR-CNN model comprising two parts: a combined LR-based memory model and a CNN-based generalization model. The LR-based memory model can learn the individual features of participants and addresses the decrease in accuracy caused by individual differences when applied to new participants. The CNN-based generalization model can learn the common features among participants, thereby reducing overall classification bias and improving overall classification accuracy.

Results

We compared our method with existing, commonly used classification methods through three different sets of experiments. The experimental results indicated that our method could learn individual differences among participants. Compared with other commonly used classification methods, our method yielded a marked improvement (>90%) in classification among new participants.

Conclusion

The accuracy of the proposed model in the face of new participants is better than that of existing, commonly used classification methods. Such improvements in cross-subject test accuracy will aid in the development of BCI systems.

A Regional Smoothing Block Sparse Bayesian Learning Method With Temporal Correlation for Channel Selection in P300 Speller

The P300-based brain-computer interfaces (BCIs) enable participants to communicate by decoding the electroencephalography (EEG) signal. Different regions of the brain correspond to various mental activities. Therefore, removing weak task-relevant and noisy channels through channel selection is necessary when decoding a specific type of activity from EEG. It can improve the recognition accuracy and reduce the training time of the subsequent models. This study proposes a novel block sparse Bayesian-based channel selection method for the P300 speller. In this method, we introduce block sparse Bayesian learning (BSBL) into the channel selection of P300 BCI for the first time and propose a regional smoothing BSBL (RSBSBL) by combining the spatial distribution properties of EEG. The RSBSBL can determine the number of channels adaptively. To ensure practicality, we design an automatic selection iteration strategy model to reduce the time cost caused by the inverse operation of the large-size matrix. We verified the proposed method on two public P300 datasets and on our collected datasets. The experimental results show that the proposed method can remove the inferior channels and work with the classifier to obtain high-classification accuracy. Hence, RSBSBL has tremendous potential for channel selection in P300 tasks.

Toward P300-brain-computer interface access to contextual scene displays for AAC: An initial exploration of context and asymmetry processing in healthy adults

Brain-computer interfaces for augmentative and alternative communication (BCI-AAC) may help overcome physical barriers to AAC access. Traditionally, visually based P300-BCI-AAC displays utilize a symmetrical grid layout. Contextual scene displays are composed of context-rich images (e.g., photographs) and may support AAC success. However, contextual scene displays contrast starkly with the standard P300-grid approach. Understanding the neurological processes from which BCI-AAC devices function is crucial to human-centered computing for BCI-AAC. Therefore, the aim of this multidisciplinary investigation is to provide an initial exploration of contextual scene use for BCI-AAC.

METHODS

Participants completed three experimental conditions to evaluate the effects of item arrangement asymmetry and context on P300-based BCI-AAC signals and offline BCI-AAC accuracy, including 1) the full contextual scene condition, 2) asymmetrical item arraignment without context condition and 3) the grid condition. Following each condition, participants completed task-evaluation ratings (e.g., engagement). Offline BCI-AAC accuracy for each condition was evaluated using cross-validation.

RESULTS

Display asymmetry significantly decreased P300 latency in the centro-parietal cluster. P300 amplitudes in the frontal cluster were decreased, though nonsignificantly. Display context significantly increased N170 amplitudes in the occipital cluster, and N400 amplitudes in the centro-parietal and occipital clusters. Scenes were rated as more visually appealing and engaging, and offline BCI-AAC performance for the scene condition was not statistically different from the grid standard.

CONCLUSION

Findings support the feasibility of incorporating scene-based displays for P300-BCI-AAC development to help provide communication for individuals with minimal or emerging language and literacy skills.

IENet: a robust convolutional neural network for EEG based brain-computer interfaces

OBJECTIVE

Brain-computer interfaces (BCIs) based on electroencephalogram (EEG) develop into novel application areas with more complex scenarios, which put forward higher requirements for the robustness of EEG signal processing algorithms. Deep learning can automatically extract discriminative features and potential dependencies via deep structures, demonstrating strong analytical capabilities in numerous domains such as computer vision (CV) and natural language processing (NLP). Making full use of deep learning technology to design a robust algorithm that is capable of analyzing EEG across BCI paradigms is our main work in this paper.

APPROACH

Inspired by InceptionV4 and InceptionTime architecture, we introduce a neural network ensemble named InceptionEEG-Net (IENet), where multi-scale convolutional layer and convolution of length 1 enable model to extract rich high-dimensional features with limited parameters. In addition, we propose the average receptive field gain for convolutional neural networks (CNNs), which optimizes IENet to detect long patterns at a smaller cost. We compare with the current state-of-the-art method across five EEG-BCI paradigms: steady-state visual evoked potentials, epilepsy EEG, overt attention P300 visual-evoked potentials, covert attention P300 visual-evoked potentials and movement-related cortical potentials.

MAIN RESULTS

The classification results show that the generalizability of IENet is on par with the state-of-the-art paradigm-agnostic models on test datasets. Furthermore, the feature explainability analysis of IENet illustrates its capability to extract neurophysiologically interpretable features for different BCI paradigms, ensuring the reliability of algorithm. Significance. It can be seen from our results that IENet can generalize to different BCI paradigms. And it is essential for deep CNNs to increase the receptive field size using average receptive field gain.

SRI-EEG: State-Based Recurrent Imputation for EEG Artifact Correction

Electroencephalogram (EEG) signals are often used as an input modality for Brain Computer Interfaces (BCIs). While EEG signals can be beneficial for numerous types of interaction scenarios in the real world, high levels of noise limits their usage to strictly noise-controlled environments such as a research laboratory. Even in a controlled environment, EEG is susceptible to noise, particularly from user motion, making it highly challenging to use EEG, and consequently BCI, as a ubiquitous user interaction modality. In this work, we address the EEG noise/artifact correction problem. Our goal is to detect physiological artifacts in EEG signal and automatically replace the detected artifacts with imputed values to enable robust EEG sensing overall requiring significantly reduced manual effort than is usual. We present a novel EEG state-based imputation model built upon a recurrent neural network, which we call SRI-EEG, and evaluate the proposed method on three publicly available EEG datasets. From quantitative and qualitative comparisons with six conventional and neural network based approaches, we demonstrate that our method achieves comparable performance to the state-of-the-art methods on the EEG artifact correction task.

Task-Related Component Analysis Combining Paired Character Decoding for Miniature Asymmetric Visual Evoked Potentials

Brain-computer interface (BCI) technology based on event-related potentials (ERP) of electroencephalography (EEG) is widely used in daily life and medical treatment. However, the research of identifying the miniature and more informative asymmetric visual evoked potentials (aVEPs), which belongs to ERP, needs further exploration. Herein, a task-related component analysis combining paired character decoding (TRCA-PCD) method, which can enhance reproducibility of aVEPs in multiple trials and strengthen the features of different samples, was designed to realize fast decoding of aVEPs. The BCI performance and the influence of repetition times between the TRCA-PCD method, the discriminative canonical pattern matching (DCPM) method and traditional task-related component analysis (TRCA) method were compared using a 32-class aVEPs dataset recorded from 32 subjects. The highest average recognition accuracy and information transfer rate (ITR) of TRCA-PCD after parameter selection were 70.37 ± 2.49% (DCPM: 64.91 ± 2.81%, TRCA: 44.01 ± 3.25%) with the peak value of 97.92% and 28.90 ± 3.83 bits/min (DCPM: 21.29 ± 3.35 bits/min, TRCA: 11.54 ± 2.81 bits/min) with the peak value of 94.55 bits/min respectively. Statistical analysis indicated that the highest average recognition rate could be obtained when the repetition time was six, and the highest ITR could be obtained when the repetition time was one. Overall, the results verified the effectiveness and superiority of TRCA-PCD in recognition of aVEPs and provided a reference for parameter selection. Therefore, the TRCA-PCD method can promote the further application of aVEPs in the BCI speller field.

A Precise Medical Imaging Approach for Brain MRI Image Classification

Magnetic resonance imaging (MRI) is an accurate and noninvasive method employed for the diagnosis of various kinds of diseases in medical imaging. Most of the existing systems showed significant performances on small MRI datasets, while their performances decrease against large MRI datasets. Hence, the goal was to design an efficient and robust classification system that sustains a high recognition rate against large MRI dataset. Accordingly, in this study, we have proposed the usage of a novel feature extraction technique that has the ability to extract and select the prominent feature from MRI image. The proposed algorithm selects the best features from the MRI images of various diseases. Further, this approach discriminates various classes based on recursive values such as partial Z-value. The proposed approach only extracts a minor feature set through, respectively, forward and backward recursion models. The most interrelated features are nominated in the forward regression model that depends on the values of partial Z-test, while the minimum interrelated features are diminished from the corresponding feature space under the presence of the backward model. In both cases, the values of Z-test are estimated through the defined labels of the diseases. The proposed model is efficiently looking the localized features, which is one of the benefits of this method. After extracting and selecting the best features, the model is trained by utilizing support vector machine (SVM) to provide the predicted labels to the corresponding MRI images. To show the significance of the proposed model, we utilized a publicly available standard dataset such as Harvard Medical School and Open Access Series of Imaging Studies (OASIS), which contains 24 various brain diseases including normal. The proposed approach achieved the best classification accuracy against existing state-of-the-art systems.

Target-Related Alpha Attenuation in a Brain-Computer Interface Rapid Serial Visual Presentation Calibration

This study evaluated the feasibility of using occipitoparietal alpha activity to drive target/non-target classification in a brain-computer interface (BCI) for communication. EEG data were collected from 12 participants who completed BCI Rapid Serial Visual Presentation (RSVP) calibrations at two different presentation rates: 1 and 4 Hz. Attention-related changes in posterior alpha activity were compared to two event-related potentials (ERPs): N200 and P300. Machine learning approaches evaluated target/non-target classification accuracy using alpha activity. Results indicated significant alpha attenuation following target letters at both 1 and 4 Hz presentation rates, though this effect was significantly reduced in the 4 Hz condition. Target-related alpha attenuation was not correlated with coincident N200 or P300 target effects. Classification using posterior alpha activity was above chance and benefitted from individualized tuning procedures. These findings suggest that target-related posterior alpha attenuation is detectable in a BCI RSVP calibration and that this signal could be leveraged in machine learning algorithms used for RSVP or comparable attention-based BCI paradigms.

Improving the performance of a gaze independent P300-BCI by using the expectancy wave

OBJECTIVE

A P300-BCI conveys a subject's intention through recognition of their ERPs. However, in the case of visual stimuli, its performance depends strongly on eye gaze. When eye movement is impaired, it becomes difficult to focus attention on a target stimulus, and the quality of the ERP declines greatly, thereby affecting recognition efficiency.

APPROACH

In this paper, the expectancy wave (E-wave) is proposed to improve signal quality and thereby improve identification of visual targets under the covert attention. The stimuli of the P300-BCI described here are presented in a fixed sequence, so the subjects can predict the next target stimulus and establish a stable expectancy effect of the target stimulus through training. Features from the E-wave that occurred 0~300ms before a stimulus were added to the post-stimulus ERP components for intention recognition.

MAIN RESULTS

Comparisons of 10 healthy subjects before and after training demonstrated that the expectancy wave generated before target stimulus could be used with the P300 component to improve character recognition accuracy (CRA) from 85% to 92.4%. In addition, CRA using only the expectancy component can reach 68.2%, which is significantly greater than random probability (16.7%). The results of this study indicate that the expectancy wave can be used to improve recognition efficiency for a gaze-independent P300-BCI, and that training contributes to induction and recognition of the potential.

SIGNIFICANCE

This study proposes an effective approach to an efficient gaze-independent P300-BCI system.

Robust asynchronous control of ERP-Based brain-Computer interfaces using deep learning

BACKGROUND AND OBJECTIVE

Brain-computer interfaces (BCI) based on event-related potentials (ERP) are a promising technology for alternative and augmented communication in an assistive context. However, most approaches to date are synchronous, requiring the intervention of a supervisor when the user wishes to turn his attention away from the BCI system. In order to bring these BCIs into real-life applications, a robust asynchronous control of the system is required through monitoring of user attention. Despite the great importance of this limitation, which prevents the deployment of these systems outside the laboratory, it is often overlooked in research articles. This study was aimed to propose a novel method to solve this problem, taking advantage of deep learning for the first time in this context to overcome the limitations of previous strategies based on hand-crafted features.

METHODS

The proposed method, based on EEG-Inception, a novel deep convolutional neural network, divides the problem in 2 stages to achieve the asynchronous control: (i) the model detects user's control state, and (ii) decodes the command only if the user is attending to the stimuli. Additionally, we used transfer learning to reduce the calibration time, even exploring a calibration-less approach.

RESULTS

Our method was evaluated with 22 healthy subjects, analyzing the impact of the calibration time and number of stimulation sequences on the system's performance. For the control state detection stage, we report average accuracies above 91% using only 1 sequence of stimulation and 30 calibration trials, reaching a maximum of 96.95% with 15 sequences. Moreover, our calibration-less approach also achieved suitable results, with a maximum accuracy of 89.36%, showing the benefits of transfer learning. As for the overall asynchronous system, which includes both stages, the maximum information transfer rate was 35.54 bpm, a suitable value for high-speed communication.

CONCLUSIONS

The proposed strategy achieved higher performance with less calibration trials and stimulation sequences than former approaches, representing a promising step forward that paves the way for more practical applications of ERP-based spellers.

Improving Cross-State and Cross-Subject Visual ERP-based BCI with Temporal Modeling and Adversarial Training

Brain-computer interface (BCI) is a useful device for people without relying on peripheral nerves and muscles. However, the performance of the event-related potential (ERP)-based BCI declines when applying it to real environments, especially in cross-state and cross-subject conditions. Here we employ temporal modeling and adversarial training to improve the visual ERP-based BCI under different mental workload states and to alleviate the problems above. The rationality of our method is that the ERP-based BCI is based on electroencephalography (EEG) signals recorded from the scalp's surface, continuously changing with time and somewhat stochastic. In this paper, we propose a hierarchical recurrent network to encode all ERP signals in each repetition at the same time and model them with a temporal manner to predict which visual event elicited an ERP. The hierarchical architecture is a simple yet effective method for organizing recurrent layers in a deep structure to model long sequence signals. Taking a cue from recent advances in adversarial training, we further applied dynamic adversarial perturbations to create adversarial examples to enhance the model performance. We conduct our experiments on one published visual ERP-based BCI task with 15 subjects and 3 different auditory workload states. The results indicate that our hierarchical method can effectively model the long sequence EEG raw data, outperform the baselines on most conditions, including cross-state and cross-subject conditions. Finally, we show how deep learning-based methods with limited EEG data can improve ERP-based BCI with adversarial training. Our code will be released at https://github.com/aispeech-lab/VisBCI.

An FPGA-Embedded Brain-Computer Interface System to Support Individual Autonomy in Locked-In Individuals

Brain-computer interfaces (BCI) can detect specific EEG patterns and translate them into control signals for external devices by providing people suffering from severe motor disabilities with an alternative/additional channel to communicate and interact with the outer world. Many EEG-based BCIs rely on the P300 event-related potentials, mainly because they require training times for the user relatively short and provide higher selection speed. This paper proposes a P300-based portable embedded BCI system realized through an embedded hardware platform based on FPGA (field-programmable gate array), ensuring flexibility, reliability, and high-performance features. The system acquires EEG data during user visual stimulation and processes them in a real-time way to correctly detect and recognize the EEG features. The BCI system is designed to allow to user to perform communication and domotic controls.

Bayesian Inferences on Neural Activity in EEG-Based Brain-Computer Interface

A brain-computer interface (BCI) is a system that translates brain activity into commands to operate technology. A common design for an electroencephalogram (EEG) BCI relies on the classification of the P300 event-related potential (ERP), which is a response elicited by the rare occurrence of target stimuli among common non-target stimuli. Few existing ERP classifiers directly explore the underlying mechanism of the neural activity. To this end, we perform a novel Bayesian analysis of the probability distribution of multi-channel real EEG signals under the P300 ERP-BCI design. We aim to identify relevant spatial temporal differences of the neural activity, which provides statistical evidence of P300 ERP responses and helps design individually efficient and accurate BCIs. As one key finding of our single participant analysis, there is a 90% posterior probability that the target ERPs of the channels around visual cortex reach their negative peaks around 200 milliseconds post-stimulus. Our analysis identifies five important channels (PO7, PO8, Oz, P4, Cz) for the BCI speller leading to a 100% prediction accuracy. From the analyses of nine other participants, we consistently select the identified five channels, and the selection frequencies are robust to small variations of bandpass filters and kernel hyper-parameters.

Investigating the effects of stimulus duration and inter-stimulus interval parameters on P300 based BCI application performance

The main goal of electroencephalography (EEG) based brain-computer interface (BCI) research is to develop a fast and higher classification accuracy (CA) rate method than those of existing ones. Generally, in BCI applications, either motor imagery or event-related P300 based techniques are used for data recording. The stimulus duration (SD) and the inter-stimulus interval (ISI) are crucial two parameters directly affecting the decision speed of the BCI system. In this study, we investigated the performance of the P300 based application in terms of speed and CA for three kinds of protocols which are called fast, medium, and slow included different SD and the ISI values. The training and test data sets were recorded in one week of delay from 8 subjects. The features were extracted by standard deviation, variance, mean, Wavelet Transform and Fourier Transform techniques. Afterwards, they were classified by the k-nearest neighbor algorithm. We obtained 87.08%, 85.41% and 83.95% average CA rate for the fast, medium, and slow protocols, respectively. The obtained results showed that the proposed fast protocol method achieved CA rate between 78.33% and 93.33%. Based on the obtained results, it can be concluded that the fast protocol values can be used for establishing a more accurate and faster P300 based BCI.

Feature selection method based on Menger curvature and LDA theory for a P300 brain-computer interface

Brain-computer interface (BCI) systems decode electroencephalogram signals to establish a channel for direct interaction between the human brain and the external world without the need for muscle or nerve control. The P300 speller, one of the most widely used BCI applications, presents a selection of characters to the user and performs character recognition by identifying P300 event-related potentials from the EEG. Such P300-based BCI systems can reach good levels of accuracy but are difficult to use in day-to-day life due to redundancy and noisy signal. A room for improvement should be considered. We propose a novel hybrid feature selection method for the P300-based BCI system to address the problem of feature redundancy, which combines the Menger curvature and linear discriminant analysis. First, selected strategies are applied separately to a given dataset to estimate the gain for application to each feature. Then, each generated value set is ranked in descending order and judged by a predefined criterion to be suitable in classification models. The intersection of the two approaches is then evaluated to identify an optimal feature subset. The proposed method is evaluated using three public datasets, i.e., BCI Competition III dataset II, BNCI Horizon dataset, and EPFL dataset. Experimental results indicate that compared with other typical feature selection and classification methods, our proposed method has better or comparable performance. Additionally, our proposed method can achieve the best classification accuracy after all epochs in three datasets. In summary, our proposed method provides a new way to enhance the performance of the P300-based BCI speller.

Two brains, one target: Design of a multi-level information fusion model based on dual-subject RSVP

BACKGROUND

Rapid serial visual presentation (RSVP) based brain-computer interface (BCI) is widely used to categorize the target and non-target images. The available information limits the prediction accuracy of single-trial using single-subject electroencephalography (EEG) signals. New Method. Hyperscanning is a new manner to record two or more subjects' signals simultaneously. So we designed a multi-level information fusion model for target image detection based on dual-subject RSVP, namely HyperscanNet. The two modules of this model fuse the data and features of the two subjects at the data and feature layers. A chunked long and short-term memory artificial neural network (LSTM) was used in the time dimension to extract features at different periods separately, completing fine-grained underlying feature extraction. While the feature layer is fused, some plain operations are used to complete the fusion of the data layer to ensure that important information is not missed.

RESULTS

Experimental results show that the F1-score (the harmonic mean of precision and recall) of this method with best group of channels and segment length is 82.76%. Comparison with existing methods. This method improves the F1-score by at least 5% compared to single-subject target detection.

CONCLUSIONS

Target detection can be accomplished by the two subjects' collaboration to achieve a higher and more stable F1-score than a single subject.

Improve P300-speller performance by online tuning stimulus onset asynchrony (SOA)

Objective. The P300-Speller is a classic brain-computer interface paradigm that has been subjected to numerous clinical trials. Some studies have reported that the performance of the P300-Speller is closely related to stimulus onset asynchrony (SOA), but very few studies have attempted to improve the performance of the P300-Speller by optimizing SOA.Approach.In this paper, we designed a P300-Speller system based on a variable SOA and dynamic stop strategy, which can automatically adjust SOA according to real-time operational performance.Main results.The online experiment results of 18 subjects showed that the event-related potential classifier and the dynamic stop algorithm established at 200 ms SOA can maintain the performance at a certain level among 50-300 ms SOA. The system can then reduce the SOA from an initial 200 ms to an average of about 98.5 ms while maintaining letter output accuracy. The average theoretical information transfer rate was significantly improved from 42.4 to 85 bit min^-1(the maximum was 232 bit min^-1).Significance.These results demonstrate that the system established in this paper can automatically optimize the SOA settings, and this personalized SOA adjustment can effectively improve the performance of the P300-Speller.

Mitigating the Impact of Psychophysical Effects During Adaptive Stimulus Selection in the P300 Speller Brain-Computer Interface

Stimulus-driven brain-computer interfaces (BCIs), such as the P300 speller, rely on using sensory stimuli to elicit specific neural signal components called event-related potentials (ERPs) to control external devices. However, psychophysical factors, such as refractory effects and adjacency distractions, may negatively impact ERP elicitation and BCI performance. Although conventional BCI stimulus presentation paradigms usually design stimulus presentation schedules in a pseudo-random manner, recent studies have shown that controlling the stimulus selection process can enhance ERP elicitation. In prior work, we developed an algorithm to adaptively select BCI stimuli using an objective criterion that maximizes the amount of information about the user's intent that can be elicited with the presented stimuli given current data conditions. Here, we enhance this adaptive BCI stimulus selection algorithm to mitigate adjacency distractions and refractory effects by modeling temporal dependencies of ERP elicitation in the objective function and imposing spatial restrictions in the stimulus search space. Results from simulations using synthetic data and human data from a BCI study show that the enhanced adaptive stimulus selection algorithm can improve spelling speeds relative to conventional BCI stimulus presentation paradigms.Clinical relevance-Increased communication rates with our enhanced adaptive stimulus selection algorithm can potentially facilitate the translation of BCIs as viable communication alternatives for individuals with severe neuromuscular limitations.

Breaching Subjects' Thoughts Privacy: A Study with Visual Stimuli and Brain-Computer Interfaces

Brain-computer interfaces (BCIs) started being used in clinical scenarios, reaching nowadays new fields such as entertainment or learning. Using BCIs, neuronal activity can be monitored for various purposes, with the study of the central nervous system response to certain stimuli being one of them, being the case of evoked potentials. However, due to the sensitivity of these data, the transmissions must be protected, with blockchain being an interesting approach to ensure the integrity of the data. This work focuses on the visual sense, and its relationship with the P300 evoked potential, where several open challenges related to the privacy of subjects' information and thoughts appear when using BCI. The first and most important challenge is whether it would be possible to extract sensitive information from evoked potentials. This aspect becomes even more challenging and dangerous if the stimuli are generated when the subject is not aware or conscious that they have occurred. There is an important gap in this regard in the literature, with only one work existing dealing with subliminal stimuli and BCI and having an unclear methodology and experiment setup. As a contribution of this paper, a series of experiments, five in total, have been created to study the impact of visual stimuli on the brain tangibly. These experiments have been applied to a heterogeneous group of ten subjects. The experiments show familiar visual stimuli and gradually reduce the sampling time of known images, from supraliminal to subliminal. The study showed that supraliminal visual stimuli produced P300 potentials about 50% of the time on average across all subjects. Reducing the sample time between images degraded the attack, while the impact of subliminal stimuli was not confirmed. Additionally, younger subjects generally presented a shorter response latency. This work corroborates that subjects' sensitive data can be extracted using visual stimuli and P300.

Review of brain encoding and decoding mechanisms for EEG-based brain-computer interface

A brain-computer interface (BCI) can connect humans and machines directly and has achieved successful applications in the past few decades. Many new BCI paradigms and algorithms have been developed in recent years. Therefore, it is necessary to review new progress in BCIs. This paper summarizes progress for EEG-based BCIs from the perspective of encoding paradigms and decoding algorithms, which are two key elements of BCI systems. Encoding paradigms are grouped by their underlying neural meachanisms, namely sensory- and motor-related, vision-related, cognition-related and hybrid paradigms. Decoding algorithms are reviewed in four categories, namely decomposition algorithms, Riemannian geometry, deep learning and transfer learning. This review will provide a comprehensive overview of both modern primary paradigms and algorithms, making it helpful for those who are developing BCI systems.