A combined brain–computer interface based on P300 potentials and motion-onset visual evoked potentials

Distraction impact of concurrent conversation on event-related potential based brain-computer interfaces

Objective.This study investigates the impact of conversation on the performance of visual event-related potential (ERP)-based brain-computer interfaces (BCIs), considering distractions in real life environment. The research aims to understand how cognitive distractions from speaking and listening activities affect ERP-BCI performance.Approach.The experiment employs a dual-task paradigm where participants control a smart light using visual ERP-BCIs while simultaneously conducting speaking or listening tasks.Main results.The findings reveal that speaking notably degrades BCI accuracy and the amplitude of ERP components, while increases the latency variability of ERP components and occipital alpha power. In contrast, listening and simple syllable repetition tasks have a lesser impact on these variables. The results suggest that speaking activity significantly distracts visual attentional processes critical for BCI operationSignificance. This study highlights the need to take distractions by daily conversation into account of the design and implementation of ERP-BCIs.

Considering preferences, speed and the animation of multiple symbols in developing P300 brain-computer interface for children

PURPOSE

Prior research has begun establishing the efficacy of animation in brain-computer interfaces access to augmentative and alternative communication (BCI-AAC). However, the use of animation in P300-BCI-AAC for children is in the early stages and largely limited to single item highlighting of extended durations. In pursuit of practical application, the present study aims to evaluate children's event-related potential (ERP) characteristics and user experience during a task involving an animated P300-BCI-AAC system.

MATERIALS AND METHODS

The system utilizes multi-item zoom animations to access a 28-pictorial symbols. Participants completed a fast (100 ms) and slow (200 ms) zoom animation highlighting conditions wherein four pictorial symbols were highlighted concurrently.

RESULTS

The proposed display appears feasible, eliciting all targeted ERPs. However, ERP amplitudes may be reduced in comparison to single-item animation highlighting, possibly due to distraction. Ratings of mental effort were significantly higher for the 100 ms condition, though differences in the frontal P200/P300 ERP did not achieve significance. Most participants identified a preference for the 100 ms condition, though age may impact preference.

CONCLUSIONS

Overall, findings support the preliminary feasibility of a proposed 28-item interface that utilises group zoom animation highlighting of pictorial symbols. Further research is needed evaluating ERP characteristics and outcomes from online (real-time) use of animation-based P300-BCI-AAC for children with severe speech and physical impairments across multiple training sessions.

Strategies for highlighting items within visual scene displays to support augmentative and alternative communication access for those with physical impairments

PURPOSE

In contrast to the traditional grid-based display, visual scene displays (VSDs) offer a new paradigm for aided communication. For individuals who cannot select items from an AAC display by direct selection due to physical impairments, AAC access can be supported via methods such as item scanning. Item scanning sequentially highlights items on a display until the individual signals for selection. How items are highlighted or scanned for AAC access can impact performance outcomes. Further, the effectiveness of a VSD interface may be enhanced through consultation with experts in visual communication. Therefore, to support AAC access for those with physical impairments, the aim of this study was to evaluate the perspectives of experts in visual communication regarding effective methods for highlighting VSD elements.

METHODS

Thirteen participants with expertise related to visual communication (e.g., photographers, artists) completed semi-structured interviews regarding techniques for item highlighting.

RESULTS

Study findings identified four main themes to inform how AAC items may be highlighted or scanned, including (1) use of contrast related to light and dark, (2) use of contrast as it relates to colour, (3) outline highlighting, and (4) use of scale and motion.

CONCLUSION

By identifying how compositional techniques can be utilized to highlight VSD elements, study findings may inform current practice for scanning-based AAC access, along with other selection techniques where feedback or highlighting is used (e.g., eye-gaze, brain-computer interface). Further, avenues for just-in-time programming are discussed to support effective implementation for those with physical impairments.IMPLICATIONS FOR REHABILITATIONFindings identify multiple potential techniques to improve scanning through items in a photograph for individuals with severe motor impairments using alternative access strategies.Study findings inform current practice for scanning-based AAC access, along with other selection techniques where feedback or highlighting is used (e.g., eye-gaze, brain-computer interface).Avenues for just in time programming of AAC displays are discussed to decrease programming demands and support effective implementation of study findings.

Promoting Simple and Engaging Brain-Computer Interface Designs for Children by Evaluating Contrasting Motion Techniques

PURPOSE

There is an increasing focus on using motion in augmentative and alternative communication (AAC) systems. In considering brain-computer interface access to AAC (BCI-AAC), motion may provide a simpler or more intuitive avenue for BCI-AAC control. Different motion techniques may be utilized in supporting competency with AAC devices including simple (e.g., zoom) and complex (behaviorally relevant animation) methods. However, how different pictorial symbol animation techniques impact BCI-AAC is unclear.

METHOD

Sixteen healthy children completed two experimental conditions. These conditions included highlighting of pictorial symbols via both functional (complex) and zoom (simple) animation to evaluate the effects of motion techniques on P300-based BCI-AAC signals and offline (predicted) BCI-AAC performance.

RESULTS

Functional (complex) animation significantly increased attentional-related P200/P300 event-related potential (ERP) amplitudes in the parieto-occipital area. Zoom (simple) animation significantly decreased N400 latency. N400 ERP amplitude was significantly greater, and occurred significantly earlier, on the right versus left side for the functional animation condition within the parieto-occipital bin. N200 ERP latency was significantly reduced over the left hemisphere for the zoom condition in the central bin. As hypothesized, elicitation of all targeted ERP components supported offline (predicted) BCI-AAC performance being similar between conditions.

CONCLUSION

Study findings provide continued support for the use of animation in BCI-AAC systems for children and highlight differences in neural and attentional processing between complex and simple animation techniques.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.24085623.

Intracranial brain-computer interface spelling using localized visual motion response

Intracranial brain-computer interfaces (BCIs) can assist severely disabled persons in text communication and environmental control with high precision and speed. Nevertheless, sustainable BCI implants require minimal invasiveness. One of the implantation strategies is to adopt localized and robust cortical activities to drive BCI communication and to make a precise presurgical planning. The visual motion response is a good candidate for inclusion in this strategy because of its focal activity over the middle temporal visual area (MT). Here, we developed an intracranial BCI for spelling, utilizing only three electrodes over the MT area. The best recording electrodes were decided by preoperative functional magnetic resonance imaging (MRI) localization of the MT, and local neural activities were further enhanced by differential rereferencing of these electrodes. The BCI spelling system was validated both offline and online by five epilepsy patients, achieving the fastest speed of 62 bits/min, i.e., 12 characters/min. Moreover, the response patterns of dual-directional visual motion stimuli provided an additional dimension of BCI target encoding and paved the way for a higher information transfer rate of intracranial BCI spelling.

Recommendations for Integrating a P300-Based Brain–Computer Interface in Virtual Reality Environments for Gaming: An Update

The integration of a P300-based brain–computer interface (BCI) into virtual reality (VR) environments is promising for the video games industry. However, it faces several limitations, mainly due to hardware constraints and limitations engendered by the stimulation needed by the BCI. The main restriction is still the low transfer rate that can be achieved by current BCI technology, preventing movement while using VR. The goal of this paper is to review current limitations and to provide application creators with design recommendations to overcome them, thus significantly reducing the development time and making the domain of BCI more accessible to developers. We review the design of video games from the perspective of BCI and VR with the objective of enhancing the user experience. An essential recommendation is to use the BCI only for non-complex and non-critical tasks in the game. Also, the BCI should be used to control actions that are naturally integrated into the virtual world. Finally, adventure and simulation games, especially if cooperative (multi-user), appear to be the best candidates for designing an effective VR game enriched by BCI technology.

Doubling the Speed of N200 Speller via Dual-Directional Motion Encoding

OBJECTIVE

Motion-onset visual evoked potentials (mVEPs)-based spellers, also known as N200 spellers, have been successfully implemented, avoiding flashing stimuli that are common in visual brain-computer interface (BCI). However, their information transfer rates (ITRs), typically below 50 bits/min, are lower than other visual BCI spellers. In this study, we sought to improve the speed of N200 speller to a level above the well-known P300 spellers.

APPROACH

Based on our finding of the spatio-temporal asymmetry of N200 response elicited by leftward and rightward visual motion, a novel dual-directional N200 speller was implemented. By presenting visual stimuli moving in two different directions simultaneously, the new paradigm reduced the stimuli presentation time by half, while ensuring separable N200 features between two visual motion directions. Furthermore, a probability-based dynamic stopping algorithm was also proposed to shorten the decision time for each output further. Both offline and online tests were conducted to evaluate the performance in ten participants.

MAIN RESULTS

Offline results revealed contralateral dominant temporal and spatial patterns in N200 responses when subjects attended to stimuli moving leftward or rightward. In online experiments, the dual-directional paradigm achieved an average ITR of 79.8 bits/min, with the highest ITR of 124.8 bits/min. Compared with the traditional uni-directional N200 speller, the median gain on the ITR was 202%.

SIGNIFICANCE

The proposed dual-directional paradigm managed to double the speed of the N200 speller. Together with its non-flashing characteristics, this dual-directional N200 speller is promising to be a competent candidate for fast and reliable BCI applications.

Brain-computer interface method based on light-flashing and motion hybrid coding

The human best response frequency band for steady-state visual evoked potential stimulus is limited. This results in a reduced number of encoded targets. To circumvent this, we proposed a brain-computer interface (BCI) method based on light-flashing and motion hybrid coding. The hybrid paradigm pattern consisted of a circular light-flashing pattern and a motion pattern located in the inner ring of light-flashing pattern. The motion and light-flashing patterns had different frequencies. This study used five frequencies to encode nine targets. The motion frequency and the light-flashing frequency of the hybrid paradigm consisted of two frequencies in five frequencies. The experimental results showed that the hybrid paradigm could induce stable motion frequency, light-flashing frequency and its harmonic components. Moreover, the modulation between motion and light-flashing was weak. The average accuracy was 92.96% and the information transfer rate was 26.10 bits/min. The experimental results showed that the proposed method could be considered for practical BCI systems.

Effects of Spatial Stimulus Overlap in a Visual P300-based Brain-computer Interface

The rapid serial visual presentation (RSVP) paradigm seems to be one of the most appropriate for patients using P300-based brain-computer interface (BCI) applications, since non-ocular movements are required. However, according to previous works, the use of different locations for each stimulus may improve performance. Thus, the aim of the present work is to explore how spatial overlap between stimuli influences performance in controlling a visual P300-based BCI. Nineteen participants were tested using four levels of overlap between two stimuli: 100%, 66.7%, 33.3% and 0%. Significant differences in accuracy were found between the 0% overlapped condition and all the other conditions, and between 33.3% and higher overlap (66.7% and 100%). These results can be explained due to a modulation in the non-target stimulus amplitude signal caused by the overlapping factor. In short, the stimulus overlap provokes a modulation in performance using a P300-based BCI; this should be considered in future BCI proposals in which an optimal surface exploitation is convenient and potential users have only residual ocular movement.

A novel hybrid BCI speller based on RSVP and SSVEP paradigm

BACKGROUND AND OBJECTIVE

Steady-state visual evoked potential (SSVEP) and rapid serial visual presentation (RSVP) are useful methods in the brain-computer interface (BCI) systems. Hybrid BCI systems that combine these two approaches can enhance the proficiency of the P300 spellers.

METHODS

In this study, a new hybrid RSVP/SSVEP BCI is proposed to increase the classification accuracy and information transfer rate (ITR) as compared with the other RSVP speller paradigms. In this paradigm, RSVP (eliciting a P300 response) and SSVEP stimulations are presented in such a way that the target group of characters is identified by RSVP stimuli, and the target character is recognized by SSVEP stimuli.

RESULTS

The proposed paradigm achieved accuracy of 93.06%, and ITR of 23.41 bit/min averaged across six subjects.

CONCLUSIONS

The new hybrid system demonstrates that by using SSVEP stimulation in Triple RSVP speller paradigm, we could enhance the performance of the system as compared with the traditional Triple RSVP paradigm. Our work is the first hybrid paradigm in RSVP spellers that could obtain the higher classification accuracy and information transfer rate in comparison with the previous RSVP spellers.

Bacomics: a comprehensive cross area originating in the studies of various brain-apparatus conversations

The brain is the most important organ of the human body, and the conversations between the brain and an apparatus can not only reveal a normally functioning or a dysfunctional brain but also can modulate the brain. Here, the apparatus may be a nonbiological instrument, such as a computer, and the consequent brain-computer interface is now a very popular research area with various applications. The apparatus may also be a biological organ or system, such as the gut and muscle, and their efficient conversations with the brain are vital for a healthy life. Are there any common bases that bind these different scenarios? Here, we propose a new comprehensive cross area: Bacomics, which comes from brain-apparatus conversations (BAC) + omics. We take Bacomics to cover at least three situations: (1) The brain is normal, but the conversation channel is disabled, as in amyotrophic lateral sclerosis. The task is to reconstruct or open up new channels to reactivate the brain function. (2) The brain is in disorder, such as in Parkinson's disease, and the work is to utilize existing or open up new channels to intervene, repair and modulate the brain by medications or stimulation. (3) Both the brain and channels are in order, and the goal is to enhance coordinated development between the brain and apparatus. In this paper, we elaborate the connotation of BAC into three aspects according to the information flow: the issue of output to the outside (BAC-1), the issue of input to the brain (BAC-2) and the issue of unity of brain and apparatus (BAC-3). More importantly, there are no less than five principles that may be taken as the cornerstones of Bacomics, such as feedforward and feedback control, brain plasticity, harmony, the unity of opposites and systems principles. Clearly, Bacomics integrates these seemingly disparate domains, but more importantly, opens a much wider door for the research and development of the brain, and the principles further provide the general framework in which to realize or optimize these various conversations.

Online Home Appliance Control Using EEG-Based Brain–Computer Interfaces

Brain–computer interfaces (BCIs) allow patients with paralysis to control external devices by mental commands. Recent advances in home automation and the Internet of things may extend the horizon of BCI applications into daily living environments at home. In this study, we developed an online BCI based on scalp electroencephalography (EEG) to control home appliances. The BCI users controlled TV channels, a digital door-lock system, and an electric light system in an unshielded environment. The BCI was designed to harness P300 and N200 components of event-related potentials (ERPs). On average, the BCI users could control TV channels with an accuracy of 83.0% ± 17.9%, the digital door-lock with 78.7% ± 16.2% accuracy, and the light with 80.0% ± 15.6% accuracy, respectively. Our study demonstrates a feasibility to control multiple home appliances using EEG-based BCIs.

Effect of a combination of flip and zooming stimuli on the performance of a visual brain-computer interface for spelling

Brain-computer interface (BCI) systems can allow their users to communicate with the external world by recognizing intention directly from their brain activity without the assistance of the peripheral motor nervous system. The P300-speller is one of the most widely used visual BCI applications. In previous studies, a flip stimulus (rotating the background area of the character) that was based on apparent motion, suffered from less refractory effects. However, its performance was not improved significantly. In addition, a presentation paradigm that used a "zooming" action (changing the size of the symbol) has been shown to evoke relatively higher P300 amplitudes and obtain a better BCI performance. To extend this method of stimuli presentation within a BCI and, consequently, to improve BCI performance, we present a new paradigm combining both the flip stimulus with a zooming action. This new presentation modality allowed BCI users to focus their attention more easily. We investigated whether such an action could combine the advantages of both types of stimuli presentation to bring a significant improvement in performance compared to the conventional flip stimulus. The experimental results showed that the proposed paradigm could obtain significantly higher classification accuracies and bit rates than the conventional flip paradigm (p<0.01).

A Dual Stimuli Approach Combined with Convolutional Neural Network to Improve Information Transfer Rate of Event-Related Potential-Based Brain-Computer Interface

Increasing command generation rate of an event-related potential-based brain-robot system is challenging, because of limited information transfer rate of a brain-computer interface system. To improve the rate, we propose a dual stimuli approach that is flashing a robot image and is scanning another robot image simultaneously. Two kinds of event-related potentials, N200 and P300 potentials, evoked in this dual stimuli condition are decoded by a convolutional neural network. Compared with the traditional approaches, this proposed approach significantly improves the online information transfer rate from 23.0 or 17.8 to 39.1 bits/min at an accuracy of 91.7%. These results suggest that combining multiple types of stimuli to evoke distinguishable ERPs might be a promising direction to improve the command generation rate in the brain-computer interface.

An Adaptive Calibration Framework for mVEP-Based Brain-Computer Interface

Electroencephalogram signals and the states of subjects are nonstationary. To track changing states effectively, an adaptive calibration framework is proposed for the brain-computer interface (BCI) with the motion-onset visual evoked potential (mVEP) as the control signal. The core of this framework is to update the training set adaptively for classifier training. The updating procedure consists of two operations, that is, adding new samples to the training set and removing old samples from the training set. In the proposed framework, a support vector machine (SVM) and fuzzy C-mean clustering (fCM) are combined to select the reliable samples for the training set from the blocks close to the current blocks to be classified. Because of the complementary information provided by SVM and fCM, they can guarantee the reliability of information fed into classifier training. The removing procedure will aim to remove those old samples recorded a relatively long time before current new blocks. These two operations could yield a new training set, which could be used to calibrate the classifier to track the changing state of the subjects. Experimental results demonstrate that the adaptive calibration framework is effective and efficient and it could improve the performance of online BCI systems.

Scenario Screen: A Dynamic and Context Dependent P300 Stimulator Screen Aimed at Wheelchair Navigation Control

P300 spellers have been widely modified to implement nonspelling tasks. In this work, we propose a "scenario" stimulation screen that is a P300 speller variation to command a wheelchair. Our approach utilized a stimulation screen with an image background (scenario snapshot for a wheelchair) and stimulation markers arranged asymmetrically over relevant landmarks, such as suitable paths, doors, windows, and wall signs. Other scenario stimulation screen features were green/blue stimulation marker color scheme, variable Interstimulus Interval, single marker stimulus mode, and optimized stimulus sequence generator. Eighteen able-bodied subjects participated in the experiment; 78% had no experience in BCI usage. A waveform feature analysis and a Mann-Whitney U test performed over the pairs of target and nontarget coherent averages confirmed that 94% of the subjects elicit P300 (p < .005) on this modified stimulator. Least Absolute Shrinkage and Selection Operator optimization and Linear Discriminant Analysis were utilized for the automatic detection of P300. For evaluation with unseen data, target detection was computed (median sensitivity = 1.00 (0.78-1.00)), together with nontarget discrimination (median specificity = 1.00 (0.98-1.00)). The scenario screen adequately elicits P300 and seems suitable for commanding a wheelchair even when users have no previous experience on the BCI spelling task.

Spelling With a Small Mobile Brain-Computer Interface in a Moving Wheelchair

Research into brain-computer interfaces (BCIs), which spell words using brain signals, has revealed that a desktop version of such a speller, the edges paradigm, offers several advantages: This edges paradigm outperforms the benchmark row-column paradigm in terms of accuracy, bitrate, and user experience. It has remained unknown whether these advantages prevailed with a new version of the edges paradigm designed for a mobile device. This paper investigated and evaluated in a rolling wheelchair a mobile BCI, which implemented the edges paradigm on small displays with which visual crowding tends to occur. How the mobile edge paradigm outperforms the mobile row-column paradigm has implications for understanding how principles of visual neurocognition affect BCI speller use in a mobile context. This investigation revealed that all the advantages of the edges paradigm over the row-column paradigm prevailed in this setting. However, the reduction in adjacent errors for the edges paradigm was unprecedentedly limited to horizontal adjacent errors. The interpretation offered is that dimensional constraints of visual interface design on a smartphone thus affected the neurocognitive processes of crowding.

A systematic review of hybrid brain-computer interfaces: Taxonomy and usability perspectives

A new Brain-Computer Interface (BCI) technique, which is called a hybrid BCI, has recently been proposed to address the limitations of conventional single BCI system. Although some hybrid BCI studies have shown promising results, the field of hybrid BCI is still in its infancy and there is much to be done. Especially, since the hybrid BCI systems are so complicated and complex, it is difficult to understand the constituent and role of a hybrid BCI system at a glance. Also, the complicated and complex systems make it difficult to evaluate the usability of the systems. We systematically reviewed and analyzed the current state-of-the-art hybrid BCI studies, and proposed a systematic taxonomy for classifying the types of hybrid BCIs with multiple taxonomic criteria. After reviewing 74 journal articles, hybrid BCIs could be categorized with respect to 1) the source of brain signals, 2) the characteristics of the brain signal, and 3) the characteristics of operation in each system. In addition, we exhaustively reviewed recent literature on usability of BCIs. To identify the key evaluation dimensions of usability, we focused on task and measurement characteristics of BCI usability. We classified and summarized 31 BCI usability journal articles according to task characteristics (type and description of task) and measurement characteristics (subjective and objective measures). Afterwards, we proposed usability dimensions for BCI and hybrid BCI systems according to three core-constructs: Satisfaction, effectiveness, and efficiency with recommendations for further research. This paper can help BCI researchers, even those who are new to the field, can easily understand the complex structure of the hybrid systems at a glance. Recommendations for future research can also be helpful in establishing research directions and gaining insight in how to solve ergonomics and HCI design issues surrounding BCI and hybrid BCI systems by usability evaluation.

Learning from label proportions in brain-computer interfaces: Online unsupervised learning with guarantees

Objective

Using traditional approaches, a brain-computer interface (BCI) requires the collection of calibration data for new subjects prior to online use. Calibration time can be reduced or eliminated e.g., by subject-to-subject transfer of a pre-trained classifier or unsupervised adaptive classification methods which learn from scratch and adapt over time. While such heuristics work well in practice, none of them can provide theoretical guarantees. Our objective is to modify an event-related potential (ERP) paradigm to work in unison with the machine learning decoder, and thus to achieve a reliable unsupervised calibrationless decoding with a guarantee to recover the true class means.

Method

We introduce learning from label proportions (LLP) to the BCI community as a new unsupervised, and easy-to-implement classification approach for ERP-based BCIs. The LLP estimates the mean target and non-target responses based on known proportions of these two classes in different groups of the data. We present a visual ERP speller to meet the requirements of LLP. For evaluation, we ran simulations on artificially created data sets and conducted an online BCI study with 13 subjects performing a copy-spelling task.

Results

Theoretical considerations show that LLP is guaranteed to minimize the loss function similar to a corresponding supervised classifier. LLP performed well in simulations and in the online application, where 84.5% of characters were spelled correctly on average without prior calibration.

Significance

The continuously adapting LLP classifier is the first unsupervised decoder for ERP BCIs guaranteed to find the optimal decoder. This makes it an ideal solution to avoid tedious calibration sessions. Additionally, LLP works on complementary principles compared to existing unsupervised methods, opening the door for their further enhancement when combined with LLP.

The hybrid BCI system for movement control by combining motor imagery and moving onset visual evoked potential

OBJECTIVE

Movement control is an important application for EEG-BCI (EEG-based brain-computer interface) systems. A single-modality BCI cannot provide an efficient and natural control strategy, but a hybrid BCI system that combines two or more different tasks can effectively overcome the drawbacks encountered in single-modality BCI control.

APPROACH

In the current paper, we developed a new hybrid BCI system by combining MI (motor imagery) and mVEP (motion-onset visual evoked potential), aiming to realize the more efficient 2D movement control of a cursor.

MAIN RESULT

The offline analysis demonstrates that the hybrid BCI system proposed in this paper could evoke the desired MI and mVEP signal features simultaneously, and both are very close to those evoked in the single-modality BCI task. Furthermore, the online 2D movement control experiment reveals that the proposed hybrid BCI system could provide more efficient and natural control commands.

SIGNIFICANCE

The proposed hybrid BCI system is compensative to realize efficient 2D movement control for a practical online system, especially for those situations in which P300 stimuli are not suitable to be applied.

Optimizing the Face Paradigm of BCI System by Modified Mismatch Negative Paradigm

Many recent studies have focused on improving the performance of event-related potential (ERP) based brain computer interfaces (BCIs). The use of a face pattern has been shown to obtain high classification accuracies and information transfer rates (ITRs) by evoking discriminative ERPs (N200 and N400) in addition to P300 potentials. Recently, it has been proved that the performance of traditional P300-based BCIs could be improved through a modification of the mismatch pattern. In this paper, a mismatch inverted face pattern (MIF-pattern) was presented to improve the performance of the inverted face pattern (IF-pattern), one of the state of the art patterns used in visual-based BCI systems. Ten subjects attended in this experiment. The result showed that the mismatch inverted face pattern could evoke significantly larger vertex positive potentials (p < 0.05) and N400s (p < 0.05) compared to the inverted face pattern. The classification accuracy (mean accuracy is 99.58%) and ITRs (mean bit rate is 27.88 bit/min) of the mismatch inverted face pattern was significantly higher than that of the inverted face pattern (p < 0.05).

Asynchronous P300 classification in a reactive brain-computer interface during an outlier detection task

OBJECTIVE

In this study, the feasibility of detecting a P300 via an asynchronous classification mode in a reactive EEG-based brain-computer interface (BCI) was evaluated. The P300 is one of the most popular BCI control signals and therefore used in many applications, mostly for active communication purposes (e.g. P300 speller). As the majority of all systems work with a stimulus-locked mode of classification (synchronous), the field of applications is limited. A new approach needs to be applied in a setting in which a stimulus-locked classification cannot be used due to the fact that the presented stimuli cannot be controlled or predicted by the system.

APPROACH

A continuous observation task requiring the detection of outliers was implemented to test such an approach. The study was divided into an offline and an online part.

MAIN RESULTS

Both parts of the study revealed that an asynchronous detection of the P300 can successfully be used to detect single events with high specificity. It also revealed that no significant difference in performance was found between the synchronous and the asynchronous approach.

SIGNIFICANCE

The results encourage the use of an asynchronous classification approach in suitable applications without a potential loss in performance.

Faster P300 Classifier Training Using Spatiotemporal Beamforming

The linearly-constrained minimum-variance (LCMV) beamformer is traditionally used as a spatial filter for source localization, but here we consider its spatiotemporal extension for P300 classification. We compare two variants and show that the spatiotemporal LCMV beamformer is at par with state-of-the-art P300 classifiers, but several orders of magnitude faster in training the classifier.

Increasing N200 Potentials Via Visual Stimulus Depicting Humanoid Robot Behavior

Achieving recognizable visual event-related potentials plays an important role in improving the success rate in telepresence control of a humanoid robot via N200 or P300 potentials. The aim of this research is to intensively investigate ways to induce N200 potentials with obvious features by flashing robot images (images with meaningful information) and by flashing pictures containing only solid color squares (pictures with incomprehensible information). Comparative studies have shown that robot images evoke N200 potentials with recognizable negative peaks at approximately 260[Formula: see text]ms in the frontal and central areas. The negative peak amplitudes increase, on average, from [Formula: see text]V, induced by flashing the squares, to [Formula: see text]V, induced by flashing the robot images. The data analyses support that the N200 potentials induced by the robot image stimuli exhibit recognizable features. Compared with the square stimuli, the robot image stimuli increase the average accuracy rate by 9.92%, from 83.33% to 93.25%, and the average information transfer rate by 24.56[Formula: see text]bits/min, from 72.18[Formula: see text]bits/min to 96.74 bits/min, in a single repetition. This finding implies that the robot images might provide the subjects with more information to understand the visual stimuli meanings and help them more effectively concentrate on their mental activities.

A P300 brain-computer interface based on a modification of the mismatch negativity paradigm

The P300-based brain-computer interface (BCI) is an extension of the oddball paradigm, and can facilitate communication for people with severe neuromuscular disorders. It has been shown that, in addition to the P300, other event-related potential (ERP) components have been shown to contribute to successful operation of the P300 BCI. Incorporating these components into the classification algorithm can improve the classification accuracy and information transfer rate (ITR). In this paper, a single character presentation paradigm was compared to a presentation paradigm that is based on the visual mismatch negativity. The mismatch negativity paradigm showed significantly higher classification accuracy and ITRs than a single character presentation paradigm. In addition, the mismatch paradigm elicited larger N200 and N400 components than the single character paradigm. The components elicited by the presentation method were consistent with what would be expected from a mismatch paradigm and a typical P300 was also observed. The results show that increasing the signal-to-noise ratio by increasing the amplitude of ERP components can significantly improve BCI speed and accuracy. The mismatch presentation paradigm may be considered a viable option to the traditional P300 BCI paradigm.

Control of humanoid robot via motion-onset visual evoked potentials

This paper investigates controlling humanoid robot behavior via motion-onset specific N200 potentials. In this study, N200 potentials are induced by moving a blue bar through robot images intuitively representing robot behaviors to be controlled with mind. We present the individual impact of each subject on N200 potentials and discuss how to deal with individuality to obtain a high accuracy. The study results document the off-line average accuracy of 93% for hitting targets across over five subjects, so we use this major component of the motion-onset visual evoked potential (mVEP) to code people's mental activities and to perform two types of on-line operation tasks: navigating a humanoid robot in an office environment with an obstacle and picking-up an object. We discuss the factors that affect the on-line control success rate and the total time for completing an on-line operation task.

Application of BCI systems in neurorehabilitation: a scoping review

PURPOSE

To review various types of electroencephalographic activities of the brain and present an overview of brain-computer interface (BCI) systems' history and their applications in rehabilitation.

METHODS

A scoping review of published English literature on BCI application in the field of rehabilitation was undertaken. IEEE Xplore, ScienceDirect, Google Scholar and Scopus databases were searched since inception up to August 2012. All experimental studies published in English and discussed complete cycle of the BCI process was included in the review.

RESULTS AND DISCUSSION

In total, 90 articles met the inclusion criteria and were reviewed. Various approaches that improve the accuracy and performance of BCI systems were discussed. Based on BCI's clinical application, reviewed articles were categorized into three groups: motion rehabilitation, speech rehabilitation and virtual reality control (VRC). Almost half of the reviewed papers (48%) concentrated on VRC. Speech rehabilitation and motion rehabilitation made up 33% and 19% of the reviewed papers, respectively. Among different types of electroencephalography signals, P300, steady state visual evoked potentials and motor imagery signals were the most common.

CONCLUSIONS

This review discussed various applications of BCI in rehabilitation and showed how BCI can be used to improve the quality of life for people with neurological disabilities. It will develop and promote new models of communication and finally, will create an accurate, reliable, online communication between human brain and computer and reduces the negative effects of external stimuli on BCI performance. Implications for Rehabilitation The field of brain-computer interfaces (BCI) is rapidly advancing and it is expected to fulfill a critical role in rehabilitation of neurological disorders and in movement restoration in the forthcoming years. In the near future, BCI has notable potential to become a major tool used by people with disabilities to control locomotion and communicate with surrounding environment and, consequently, improve the quality of life for many affected persons. Electrical field recording at the scalp (i.e. electroencephalography) is the most likely method to be of practical value for clinical use as it is simple and non-invasive. However, some aspects need future improvements, such as the ability to separate close imagery signal (motion of extremities and phalanges that are close together).

AN ERP-BASED BCI USING AN ODDBALL PARADIGM WITH DIFFERENT FACES AND REDUCED ERRORS IN CRITICAL FUNCTIONS

Recent research has shown that a new face paradigm is superior to the conventional "flash only" approach that has dominated P300 brain–computer interfaces (BCIs) for over 20 years. However, these face paradigms did not study the repetition effects and the stability of evoked event related potentials (ERPs), which would decrease the performance of P300 BCI. In this paper, we explored whether a new "multi-faces (MF)" approach would yield more distinct ERPs than the conventional "single face (SF)" approach. To decrease the repetition effects and evoke large ERPs, we introduced a new stimulus approach called the "MF" approach, which shows different familiar faces randomly. Fifteen subjects participated in runs using this new approach and an established "SF" approach. The result showed that the MF pattern enlarged the N200 and N400 components, evoked stable P300 and N400, and yielded better BCI performance than the SF pattern. The MF pattern can evoke larger N200 and N400 components and more stable P300 and N400, which increase the classification accuracy compared to the face pattern.

A survey of the dummy face and human face stimuli used in BCI paradigm

BACKGROUND

It was proved that the human face stimulus were superior to the flash only stimulus in BCI system. However, human face stimulus may lead to copyright infringement problems and was hard to be edited according to the requirement of the BCI study. Recently, it was reported that facial expression changes could be done by changing a curve in a dummy face which could obtain good performance when it was applied to visual-based P300 BCI systems.

NEW METHOD

In this paper, four different paradigms were presented, which were called dummy face pattern, human face pattern, inverted dummy face pattern and inverted human face pattern, to evaluate the performance of the dummy faces stimuli compared with the human faces stimuli.

COMPARISON WITH EXISTING METHOD(S)

The key point that determined the value of dummy faces in BCI systems were whether dummy faces stimuli could obtain as good performance as human faces stimuli. Online and offline results of four different paradigms would have been obtained and comparatively analyzed.

RESULTS

Online and offline results showed that there was no significant difference among dummy faces and human faces in ERPs, classification accuracy and information transfer rate when they were applied in BCI systems.

CONCLUSIONS

Dummy faces stimuli could evoke large ERPs and obtain as high classification accuracy and information transfer rate as the human faces stimuli. Since dummy faces were easy to be edited and had no copyright infringement problems, it would be a good choice for optimizing the stimuli of BCI systems.

Beyond maximum speed--a novel two-stimulus paradigm for brain-computer interfaces based on event-related potentials (P300-BCI)

OBJECTIVE

The speed of brain-computer interfaces (BCI), based on event-related potentials (ERP), is inherently limited by the commonly used one-stimulus paradigm. In this paper, we introduce a novel paradigm that can increase the spelling speed by a factor of 2, thereby extending the one-stimulus paradigm to a two-stimulus paradigm. Two different stimuli (a face and a symbol) are presented at the same time, superimposed on different characters and ERPs are classified using a multi-class classifier. Here, we present the proof-of-principle that is achieved with healthy participants.

APPROACH

Eight participants were confronted with the novel two-stimulus paradigm and, for comparison, with two one-stimulus paradigms that used either one of the stimuli. Classification accuracies (percentage of correctly predicted letters) and elicited ERPs from the three paradigms were compared in a comprehensive offline analysis.

MAIN RESULTS

The accuracies slightly decreased with the novel system compared to the established one-stimulus face paradigm. However, the use of two stimuli allowed for spelling at twice the maximum speed of the one-stimulus paradigms, and participants still achieved an average accuracy of 81.25%. This study introduced an alternative way of increasing the spelling speed in ERP-BCIs and illustrated that ERP-BCIs may not yet have reached their speed limit. Future research is needed in order to improve the reliability of the novel approach, as some participants displayed reduced accuracies. Furthermore, a comparison to the most recent BCI systems with individually adjusted, rapid stimulus timing is needed to draw conclusions about the practical relevance of the proposed paradigm.

SIGNIFICANCE

We introduced a novel two-stimulus paradigm that might be of high value for users who have reached the speed limit with the current one-stimulus ERP-BCI systems.

Language model applications to spelling with Brain-Computer Interfaces

Within the Ambient Assisted Living (AAL) community, Brain-Computer Interfaces (BCIs) have raised great hopes as they provide alternative communication means for persons with disabilities bypassing the need for speech and other motor activities. Although significant advancements have been realized in the last decade, applications of language models (e.g., word prediction, completion) have only recently started to appear in BCI systems. The main goal of this article is to review the language model applications that supplement non-invasive BCI-based communication systems by discussing their potential and limitations, and to discern future trends. First, a brief overview of the most prominent BCI spelling systems is given, followed by an in-depth discussion of the language models applied to them. These language models are classified according to their functionality in the context of BCI-based spelling: the static/dynamic nature of the user interface, the use of error correction and predictive spelling, and the potential to improve their classification performance by using language models. To conclude, the review offers an overview of the advantages and challenges when implementing language models in BCI-based communication systems when implemented in conjunction with other AAL technologies.

A P300-based brain-computer interface with stimuli on moving objects: four-session single-trial and triple-trial tests with a game-like task design

Brain-computer interfaces (BCIs) are tools for controlling computers and other devices without using muscular activity, employing user-controlled variations in signals recorded from the user's brain. One of the most efficient noninvasive BCIs is based on the P300 wave of the brain's response to stimuli and is therefore referred to as the P300 BCI. Many modifications of this BCI have been proposed to further improve the BCI's characteristics or to better adapt the BCI to various applications. However, in the original P300 BCI and in all of its modifications, the spatial positions of stimuli were fixed relative to each other, which can impose constraints on designing applications controlled by this BCI. We designed and tested a P300 BCI with stimuli presented on objects that were freely moving on a screen at a speed of 5.4°/s. Healthy participants practiced a game-like task with this BCI in either single-trial or triple-trial mode within four sessions. At each step, the participants were required to select one of nine moving objects. The mean online accuracy of BCI-based selection was 81% in the triple-trial mode and 65% in the single-trial mode. A relatively high P300 amplitude was observed in response to targets in most participants. Self-rated interest in the task was high and stable over the four sessions (the medians in the 1st/4th sessions were 79/84% and 76/71% in the groups practicing in the single-trial and triple-trial modes, respectively). We conclude that the movement of stimulus positions relative to each other may not prevent the efficient use of the P300 BCI by people controlling their gaze, e.g., in robotic devices and in video games.

Poor performance in SSVEP BCIs: are worse subjects just slower?

Brain-computer interface (BCI) systems translate brain activity into messages or commands. BCI studies that record from a dozen or more subjects typically report substantial variations in performance, as measured by accuracy. Usually, some subjects attain excellent (even perfect) accuracy, while at least one subject performs so poorly that effective communication would not be possible with that BCI. This study aims to further explore the differences between the best and worst performers by studying the changes in estimated accuracy within each trial in an offline simulation of an SSVEP BCI. Results showed that the worst performers not only attained lower accuracies, but needed more time after cue onset before their accuracies improved substantially. This outcome suggests that poor performance may be partly (though not completely) explained by the latency between cue onset and improved accuracy.