P300-Based Asynchronous Brain Computer Interface for Environmental Control System

A model-based brain switch via periodic motor imagery modulation for asynchronous brain-computer interfaces

Objective.Brain switches provide a tangible solution to asynchronized brain-computer interface, which decodes user intention without a pre-programmed structure. However, most brain switches based on electroencephalography signals have high false positive rates (FPRs), resulting in less practicality. This research aims to improve the operating mode and usability of the brain switch.Approach.Here, we propose a novel virtual physical model-based brain switch that leverages periodic active modulation. An optimization problem of minimizing the triggering time subject to a required FPR is formulated, numerical and analytical approximate solutions are obtained based on the model.Main results.Our motor imagery (MI)-based brain switch can reach 0.8FP/h FPR with a median triggering time of 58 s. We evaluated the proposed brain switch during online device control, and their average FPRs substantially outperformed the conventional brain switches in the literature. We further improved the proposed brain switch with the Common Spatial Pattern (CSP) and optimization method. An average FPR of 0.3 FPs/h was obtained for the MI-CSP-based brain switch, and the average triggering time improved to 21.6 s.Significance.This study provides a new approach that could significantly reduce the brain switch's FPR to less than 1 Fps/h, which was less than 10% of the FPR (decreasing by more than a magnitude of order) by other endogenous methods, and the reaction time was comparable to the state-of-the-art approaches. This represents a significant advancement over the current non-invasive asynchronous BCI and will open widespread avenues for translating BCI towards clinical applications.

A Wearable Asynchronous Brain-Computer Interface Based on EEG-EOG Signals With Fewer Channels

OBJECTIVE

Brain-computer interfaces (BCIs) have tremendous application potential in communication, mechatronic control and rehabilitation. However, existing BCI systems are bulky, expensive and require laborious preparation before use. This study proposes a practical and user-friendly BCI system without compromising performance.

METHODS

A hybrid asynchronous BCI system was developed based on an elaborately designed wearable electroencephalography (EEG) amplifier that is compact, easy to use and offers a high signal-to-noise ratio (SNR). The wearable BCI system can detect P300 signals by processing EEG signals from three channels and operates asynchronously by integrating blink detection.

RESULT

The wearable EEG amplifier obtains high quality EEG signals and introduces preprocessing capabilities to BCI systems. The wearable BCI system achieves an average accuracy of 94.03±4.65%, an average information transfer rate (ITR) of 31.42±7.39 bits/min and an average false-positive rate (FPR) of 1.78%.

CONCLUSION

The experimental results demonstrate the feasibility and practicality of the developed wearable EEG amplifier and BCI system.

SIGNIFICANCE

Wearable asynchronous BCI systems with fewer channels are possible, indicating that BCI applications can be transferred from the laboratory to real-world scenarios.

Classification of Targets and Distractors in an Audiovisual Attention Task Based on Electroencephalography

Within the broader context of improving interactions between artificial intelligence and humans, the question has arisen regarding whether auditory and rhythmic support could increase attention for visual stimuli that do not stand out clearly from an information stream. To this end, we designed an experiment inspired by pip-and-pop but more appropriate for eliciting attention and P3a-event-related potentials (ERPs). In this study, the aim was to distinguish between targets and distractors based on the subject's electroencephalography (EEG) data. We achieved this objective by employing different machine learning (ML) methods for both individual-subject (IS) and cross-subject (CS) models. Finally, we investigated which EEG channels and time points were used by the model to make its predictions using saliency maps. We were able to successfully perform the aforementioned classification task for both the IS and CS scenarios, reaching classification accuracies up to 76%. In accordance with the literature, the model primarily used the parietal-occipital electrodes between 200 ms and 300 ms after the stimulus to make its prediction. The findings from this research contribute to the development of more effective P300-based brain-computer interfaces. Furthermore, they validate the EEG data collected in our experiment.

BCI-Utility Metric for Asynchronous P300 Brain-Computer Interface Systems

The Brain-Computer Interface (BCI) was envisioned as an assistive technology option for people with severe movement impairments. The traditional synchronous event-related potential (ERP) BCI design uses a fixed communication speed and is vulnerable to variations in attention. Recent ERP BCI designs have added asynchronous features, including abstention and dynamic stopping, but it remains a open question of how to evaluate asynchronous BCI performance. In this work, we build on the BCI-Utility metric to create the first evaluation metric with special consideration of the asynchronous features of self-paced BCIs. This metric considers accuracy as all of the following three - probability of a correct selection when a selection was intended, probability of making a selection when a selection was intended, and probability of an abstention when an abstention was intended. Further, it considers the average time required for a selection when using dynamic stopping and the proportion of intended selections versus abstentions. We establish the validity of the derived metric via extensive simulations, and illustrate and discuss its practical usage on real-world BCI data. We describe the relative contribution of different inputs with plots of BCI-Utility curves under different parameter settings. Generally, the BCI-Utility metric increases as any of the accuracy values increase and decreases as the expected time for an intended selection increases. Furthermore, in many situations, we find shortening the expected time of an intended selection is the most effective way to improve the BCI-Utility, which necessitates the advancement of asynchronous BCI systems capable of accurate abstention and dynamic stopping.

Robust asynchronous control of ERP-Based brain-Computer interfaces using deep learning

BACKGROUND AND OBJECTIVE

Brain-computer interfaces (BCI) based on event-related potentials (ERP) are a promising technology for alternative and augmented communication in an assistive context. However, most approaches to date are synchronous, requiring the intervention of a supervisor when the user wishes to turn his attention away from the BCI system. In order to bring these BCIs into real-life applications, a robust asynchronous control of the system is required through monitoring of user attention. Despite the great importance of this limitation, which prevents the deployment of these systems outside the laboratory, it is often overlooked in research articles. This study was aimed to propose a novel method to solve this problem, taking advantage of deep learning for the first time in this context to overcome the limitations of previous strategies based on hand-crafted features.

METHODS

The proposed method, based on EEG-Inception, a novel deep convolutional neural network, divides the problem in 2 stages to achieve the asynchronous control: (i) the model detects user's control state, and (ii) decodes the command only if the user is attending to the stimuli. Additionally, we used transfer learning to reduce the calibration time, even exploring a calibration-less approach.

RESULTS

Our method was evaluated with 22 healthy subjects, analyzing the impact of the calibration time and number of stimulation sequences on the system's performance. For the control state detection stage, we report average accuracies above 91% using only 1 sequence of stimulation and 30 calibration trials, reaching a maximum of 96.95% with 15 sequences. Moreover, our calibration-less approach also achieved suitable results, with a maximum accuracy of 89.36%, showing the benefits of transfer learning. As for the overall asynchronous system, which includes both stages, the maximum information transfer rate was 35.54 bpm, a suitable value for high-speed communication.

CONCLUSIONS

The proposed strategy achieved higher performance with less calibration trials and stimulation sequences than former approaches, representing a promising step forward that paves the way for more practical applications of ERP-based spellers.

A high-performance brain switch based on code-modulated visual evoked potentials

OBJECTIVE

Asynchronous brain-computer interfaces (BCIs) are more practical and natural compared to synchronous BCIs. A brain switch is a standard asynchronous BCI, which can automatically detect the specified change of the brain and discriminate between the control state and the idle state. The current brain switches still face challenges on relatively long reaction time (RT) and high false positive rate (FPR).

APPROACH

In this paper, an online electroencephalography-based brain switch is designed to realize a fast reaction and keep long idle time (IDLE) without false positives (FPs) using code-modulated visual evoked potentials (c-VEPs). Two stimulation paradigms were designed and compared in the experiments: multi-code concatenate modulation (concatenation mode) and single-code periodic modulation (periodic mode). Using a task-related component analysis-based detection algorithm, EEG data can be decoded into a series of code indices. Brain states can be detected by a template matching approach with a sliding window on the output series.

MAIN RESULTS

The online experiments achieved an average RT of 1.49 seconds when the average IDLE for each FP was 68.57 minutes (1.46e-2 FP/min) or an average RT of 1.67 seconds without FPs.

SIGNIFICANCE

This study provides a practical c-VEP based brain switch system with both fast reaction and low FPR during idle state, which can be used in various BCI applications.

Interprofessional Practitioners' Opinions on Features and Services for an Augmentative and Alternative Communication Brain-Computer Interface Device

BACKGROUND

Brain-computer interface (BCI) technology is an emerging access method to augmentative and alternative communication (AAC) devices.

OBJECTIVES

To identify, in the early stages of research and development, the perceptions and considerations of interprofessional practice (IPP) team members regarding features and functions for an AAC-BCI device.

DESIGN

Qualitative research methodology applying a grounded theory approach using focus groups with a follow-up survey of participants using NVivo analysis software supporting inductive coding of transcription data.

SETTING

Focus groups held at university, clinic, and industry conference rooms. Discussion was stimulated by a 14-minute video on an AAC-BCI device prototype. The prototype hardware and electroencephalography (EEG) gel and dry electrode headgear were on display.

PARTICIPANTS

Convenience sample of practitioners providing rehabilitation or clinical services to individuals with severe communication disorders and movement impairments who use AAC and/or other assistive technology.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

Descriptive statistics using thematic analysis of participants' opinions, input, and feedback on the ideal design for a noninvasive, EEG-based P300 AAC-BCI device.

RESULTS

Interrater and interjudge reliability were at 98% and 100%, respectively, for transcription and researcher coding. Triangulation of multiple data sources supported theme and subtheme identification that included design features, set-up and calibration, services, and effectiveness. An AAC device with BCI access was unanimously confirmed (100%) as a desirable commercial product. Participants felt that the AAC-BCI prototype appeared effective for meeting daily communication needs (75%). Results showed that participants' preference on headgear types would change based on accuracy (91%) and rate (83%) of performance. A data-logging feature was considered beneficial by 100% of participants.

CONCLUSIONS

IPP teams provided critical impressions on design, services, and features for a commercial AAC-BCI device. Expressed feature and function preferences showed dependence on communication accuracy, rate, and effectiveness. This provides vital guidance for successful clinical deployment.

Proposals and Comparisons from One-Sensor EEG and EOG Human-Machine Interfaces

Human-Machine Interfaces (HMI) allow users to interact with different devices such as computers or home elements. A key part in HMI is the design of simple non-invasive interfaces to capture the signals associated with the user's intentions. In this work, we have designed two different approaches based on Electroencephalography (EEG) and Electrooculography (EOG). For both cases, signal acquisition is performed using only one electrode, which makes placement more comfortable compared to multi-channel systems. We have also developed a Graphical User Interface (GUI) that presents objects to the user using two paradigms-one-by-one objects or rows-columns of objects. Both interfaces and paradigms have been compared for several users considering interactions with home elements.

Multi-View Multi-Scale Optimization of Feature Representation for EEG Classification Improvement

Effectively extracting common space pattern (CSP) features from motor imagery (MI) EEG signals is often highly dependent on the filter band selection. At the same time, optimizing the EEG channel combinations is another key issue that substantially affects the SMR feature representations. Although numerous algorithms have been developed to find channels that record important characteristics of MI, most of them select channels in a cumbersome way with low computational efficiency, thereby limiting the practicality of MI-based BCI systems. In this study, we propose the multi-scale optimization (MSO) of spatial patterns, optimizing filter bands over multiple channel sets within CSPs to further improve the performance of MI-based BCI. Specifically, several channel subsets are first heuristically predefined, and then raw EEG data specific to each of these subsets bandpass-filtered at the overlap between a set of filter bands. Further, instead of solving learning problems for each channel subset independently, we propose a multi-view learning based sparse optimization to jointly extract robust CSP features with L_2,1 -norm regularization, aiming to capture the shared salient information across multiple related spatial patterns for enhanced classification performance. A support vector machine (SVM) classifier is then trained on these optimized EEG features for accurate recognition of MI tasks. Experimental results on three public EEG datasets validate the effectiveness of MSO compared to several other competing methods and their variants. These superior experimental results demonstrate that the proposed MSO method has promising potential in MI-based BCIs.

Evaluation of a P300-Based Brain-Machine Interface for a Robotic Hand-Orthosis Control

This work presents the design, implementation, and evaluation of a P300-based brain-machine interface (BMI) developed to control a robotic hand-orthosis. The purpose of this system is to assist patients with amyotrophic lateral sclerosis (ALS) who cannot open and close their hands by themselves. The user of this interface can select one of six targets, which represent the flexion-extension of one finger independently or the movement of the five fingers simultaneously. We tested offline and online our BMI on eighteen healthy subjects (HS) and eight ALS patients. In the offline test, we used the calibration data of each participant recorded in the experimental sessions to estimate the accuracy of the BMI to classify correctly single epochs as target or non-target trials. On average, the system accuracy was 78.7% for target epochs and 85.7% for non-target trials. Additionally, we observed significant P300 responses in the calibration recordings of all the participants, including the ALS patients. For the BMI online test, each subject performed from 6 to 36 attempts of target selections using the interface. In this case, around 46% of the participants obtained 100% of accuracy, and the average online accuracy was 89.83%. The maximum information transfer rate (ITR) observed in the experiments was 52.83 bit/min, whereas that the average ITR was 18.13 bit/min. The contributions of this work are the following. First, we report the development and evaluation of a mind-controlled robotic hand-orthosis for patients with ALS. To our knowledge, this BMI is one of the first P300-based assistive robotic devices with multiple targets evaluated on people with ALS. Second, we provide a database with calibration data and online EEG recordings obtained in the evaluation of our BMI. This data is useful to develop and compare other BMI systems and test the processing pipelines of similar applications.

Subject-Specific feature selection for near infrared spectroscopy based brain-computer interfaces

BACKGROUND AND OBJECTIVE

Brain-computer interfaces (BCIs) enable people to control an external device by analyzing the brain's neural activity. Functional near-infrared spectroscopy (fNIRS), which is an emerging optical imaging technique, is frequently used in non-invasive BCIs. Determining the subject-specific features is an important concern in enhancing the classification accuracy as well as reducing the complexity of fNIRS based BCI systems. In this study, the effectiveness of subject-specific feature selection on classification accuracy of fNIRS signals is examined.

METHODS

In order to determine the subject-specific optimal feature subsets, stepwise regression analysis based on sequential feature selection (SWR-SFS) and ReliefF methods were employed. Feature selection is applied on time-domain features of fNIRS signals such as mean, slope, peak, skewness and kurtosis values of signals. Linear discriminant analysis, k nearest neighborhood and support vector machines are employed to evaluate the performance of the selected feature subsets. The proposed techniques are validated on benchmark motor imagery (MI) and mental arithmetic (MA) based fNIRS datasets collected from 29 healthy subjects.

RESULTS

Both SWR-SFS and reliefF feature selection methods have significantly improved the classification accuracy. However, the best results (88.67% (HbR) and 86.43% (HbO) for MA dataset and 77.01% (HbR) and 71.32% (HbO) for MI dataset) were achieved using SWR-SFS while feature selection provided extremely high feature reduction rates (89.50% (HbR) and 93.99% (HbO) for MA dataset and 94.04% (HbR) and 97.73% (HbO) for MI dataset).

CONCLUSIONS

The results of the study indicate that employing feature selection improves both MA and MI-based fNIRS signals classification performance significantly.

Error Correction Regression Framework for Enhancing the Decoding Accuracies of Ear-EEG Brain-Computer Interfaces

Ear-electroencephalography (EEG) is a promising tool for practical brain-computer interface (BCI) applications because it is more unobtrusive, comfortable, and mobile than a typical scalp-EEG system. However, an ear-EEG has a natural constraint of electrode location (e.g., limited in or around the ear) for acquiring informative brain signals sufficiently. Achieving reliable performance of ear-EEG in specific BCI paradigms that do not utilize brain signals on the temporal lobe around the ear is difficult. For example, steady-state visual evoked potentials (SSVEPs), which are mainly generated in the occipital area, have a significantly attenuated and distorted amplitude in ear-EEG. Therefore, preserving the high level of decoding accuracy is challenging and essential for SSVEP BCI based on ear-EEG. In this paper, we first investigate linear and nonlinear regression methods to increase the decoding accuracy of ear-EEG regarding SSVEP paradigm by utilizing the estimated target EEG signals on the occipital area. Then, we investigate an ensemble method to consider the prediction variability of the regression methods. Finally, we propose an error correction regression (ECR) framework to reduce the prediction errors by adding an additional nonlinear regression process (i.e., kernel ridge regression). We evaluate the ECR framework in terms of single session, session-to-session transfer, and subject-transfer decoding. We also validate the online decoding ability of the proposed framework with a short-time window size. The average accuracies are observed to be 91.11±9.14%, 90.52±8.67%, 86.96±12.13%, and 78.79±12.59%. This paper demonstrates that SSVEP BCI based on ear-EEG can achieve reliable performance with the proposed ECR framework.

Current Status, Challenges, and Possible Solutions of EEG-Based Brain-Computer Interface: A Comprehensive Review

Brain-Computer Interface (BCI), in essence, aims at controlling different assistive devices through the utilization of brain waves. It is worth noting that the application of BCI is not limited to medical applications, and hence, the research in this field has gained due attention. Moreover, the significant number of related publications over the past two decades further indicates the consistent improvements and breakthroughs that have been made in this particular field. Nonetheless, it is also worth mentioning that with these improvements, new challenges are constantly discovered. This article provides a comprehensive review of the state-of-the-art of a complete BCI system. First, a brief overview of electroencephalogram (EEG)-based BCI systems is given. Secondly, a considerable number of popular BCI applications are reviewed in terms of electrophysiological control signals, feature extraction, classification algorithms, and performance evaluation metrics. Finally, the challenges to the recent BCI systems are discussed, and possible solutions to mitigate the issues are recommended.

An adaptive decoder design based on the receding horizon optimization in BMI system

In a motor brain-machine interface system, since the electroencephalogram signal is changing through out the process of the arm movement, the offline trained decoder with fixed weights is often unable to convert the electroencephalogram signal accurately, resulting in poor recovery of joint motor function. In this paper, a receding horizon optimization strategy is chosen to online update the decoder weights and design an adaptive Wiener-filter-based decoder. Firstly, a classical Wiener-filter-based decoder with fixed weights is brief reviewed. Secondly, the weights in Wiener-filter-based decoder are updated by minimizing the cost function, which is composed by the sum of squared position errors in the given horizon at each sampling time. The simulation shows that the recovery effect of joint motor function and neuron activity in the BMI system with the adaptive decoder are both better than that in the BMI system with the fixed decoder.

Online Home Appliance Control Using EEG-Based Brain–Computer Interfaces

Brain–computer interfaces (BCIs) allow patients with paralysis to control external devices by mental commands. Recent advances in home automation and the Internet of things may extend the horizon of BCI applications into daily living environments at home. In this study, we developed an online BCI based on scalp electroencephalography (EEG) to control home appliances. The BCI users controlled TV channels, a digital door-lock system, and an electric light system in an unshielded environment. The BCI was designed to harness P300 and N200 components of event-related potentials (ERPs). On average, the BCI users could control TV channels with an accuracy of 83.0% ± 17.9%, the digital door-lock with 78.7% ± 16.2% accuracy, and the light with 80.0% ± 15.6% accuracy, respectively. Our study demonstrates a feasibility to control multiple home appliances using EEG-based BCIs.

Asynchronous non-invasive high-speed BCI speller with robust non-control state detection

Brain-Computer Interfaces (BCIs) enable users to control a computer by using pure brain activity. Recent BCIs based on visual evoked potentials (VEPs) have shown to be suitable for high-speed communication. However, all recent high-speed BCIs are synchronous, which means that the system works with fixed time slots so that the user is not able to select a command at his own convenience, which poses a problem in real-world applications. In this paper, we present the first asynchronous high-speed BCI with robust distinction between intentional control (IC) and non-control (NC), with a nearly perfect NC state detection of only 0.075 erroneous classifications per minute. The resulting asynchronous speller achieved an average information transfer rate (ITR) of 122.7 bit/min using a 32 target matrix-keyboard. Since the method is based on random stimulation patterns it allows to use an arbitrary number of targets for any application purpose, which was shown by using an 55 target German QWERTZ-keyboard layout which allowed the participants to write an average of 16.1 (up to 30.7) correct case-sensitive letters per minute. As the presented system is the first asynchronous high-speed BCI speller with a robust non-control state detection, it is an important step for moving BCI applications out of the lab and into real-life.

Asynchronous Control of P300-Based Brain-Computer Interfaces Using Sample Entropy

Brain–computer interfaces (BCI) have traditionally worked using synchronous paradigms. In recent years, much effort has been put into reaching asynchronous management, providing users with the ability to decide when a command should be selected. However, to the best of our knowledge, entropy metrics have not yet been explored. The present study has a twofold purpose: (i) to characterize both control and non-control states by examining the regularity of electroencephalography (EEG) signals; and (ii) to assess the efficacy of a scaled version of the sample entropy algorithm to provide asynchronous control for BCI systems. Ten healthy subjects participated in the study, who were asked to spell words through a visual oddball-based paradigm, attending (i.e., control) and ignoring (i.e., non-control) the stimuli. An optimization stage was performed for determining a common combination of hyperparameters for all subjects. Afterwards, these values were used to discern between both states using a linear classifier. Results show that control signals are more complex and irregular than non-control ones, reaching an average accuracy of 94.40% in classification. In conclusion, the present study demonstrates that the proposed framework is useful in monitoring the attention of a user, and granting the asynchrony of the BCI system.

Sparse Group Representation Model for Motor Imagery EEG Classification

A potential limitation of a motor imagery (MI) based brain-computer interface (BCI) is that it usually requires a relatively long time to record sufficient electroencephalogram (EEG) data for robust classifier training. The calibration burden during data acquisition phase will most probably cause a subject to be reluctant to use a BCI system. To alleviate this issue, we propose a novel sparse group representation model (SGRM) for improving the efficiency of MI-based BCI by exploiting the intersubject information. Specifically, preceded by feature extraction using common spatial pattern, a composite dictionary matrix is constructed with training samples from both the target subject and other subjects. By explicitly exploiting within-group sparse and group-wise sparse constraints, the most compact representation of a test sample of the target subject is then estimated as a linear combination of columns in the dictionary matrix. Classification is implemented by calculating the class-specific representation residual based on the significant training samples corresponding to the nonzero representation coefficients. Accordingly, the proposed SGRM method effectively reduces the required training samples from the target subject due to auxiliary data available from other subjects. With two public EEG data sets, extensive experimental comparisons are carried out between SGRM and other state-of-the-art approaches. Superior classification performance of our method using 40 trials of the target subject for model calibration (Averaged accuracy = 78.2%, Kappa = 0.57 and Averaged accuracy = 77.7%, Kappa = 0.55 for the two data sets, respectively) indicates its promising potential for improving the practicality of MI-based BCI.