Toward enhanced P300 speller performance

A Novel 9-Class Auditory ERP Paradigm Driving a Predictive Text Entry System

Brain–computer interfaces (BCIs) based on event related potentials (ERPs) strive for offering communication pathways which are independent of muscle activity. While most visual ERP-based BCI paradigms require good control of the user's gaze direction, auditory BCI paradigms overcome this restriction. The present work proposes a novel approach using auditory evoked potentials for the example of a multiclass text spelling application. To control the ERP speller, BCI users focus their attention to two-dimensional auditory stimuli that vary in both, pitch (high/medium/low) and direction (left/middle/right) and that are presented via headphones. The resulting nine different control signals are exploited to drive a predictive text entry system. It enables the user to spell a letter by a single nine-class decision plus two additional decisions to confirm a spelled word. This paradigm – called PASS2D – was investigated in an online study with 12 healthy participants. Users spelled with more than 0.8 characters per minute on average (3.4 bits/min) which makes PASS2D a competitive method. It could enrich the toolbox of existing ERP paradigms for BCI end users like people with amyotrophic lateral sclerosis disease in a late stage.

Should the parameters of a BCI translation algorithm be continually adapted?

People with or without motor disabilities can learn to control sensorimotor rhythms (SMRs) recorded from the scalp to move a computer cursor in one or more dimensions or can use the P300 event-related potential as a control signal to make discrete selections. Data collected from individuals using an SMR-based or P300-based BCI were evaluated offline to estimate the impact on performance of continually adapting the parameters of the translation algorithm during BCI operation. The performance of the SMR-based BCI was enhanced by adaptive updating of the feature weights or adaptive normalization of the features. In contrast, P300 performance did not benefit from either of these procedures.

Impact of spatial filters during sensor selection in a visual P300 brain-computer interface

A challenge in designing a Brain-Computer Interface (BCI) is the choice of the channels, e.g. the most relevant sensors. Although a setup with many sensors can be more efficient for the detection of Event-Related Potential (ERP) like the P300, it is relevant to consider only a low number of sensors for a commercial or clinical BCI application. Indeed, a reduced number of sensors can naturally increase the user comfort by reducing the time required for the installation of the EEG (electroencephalogram) cap and can decrease the price of the device. In this study, the influence of spatial filtering during the process of sensor selection is addressed. Two of them maximize the Signal to Signal-plus-Noise Ratio (SSNR) for the different sensor subsets while the third one maximizes the differences between the averaged P300 waveform and the non P300 waveform. We show that the locations of the most relevant sensors subsets for the detection of the P300 are highly dependent on the use of spatial filtering. Applied on data from 20 healthy subjects, this study proves that subsets obtained where sensors are suppressed in relation to their individual SSNR are less efficient than when sensors are suppressed in relation to their contribution once the different selected sensors are combined for enhancing the signal. In other words, it highlights the difference between estimating the P300 projection on the scalp and evaluating the more efficient sensor subsets for a P300-BCI. Finally, this study explores the issue of channel commonality across subjects. The results support the conclusion that spatial filters during the sensor selection procedure allow selecting better sensors for a visual P300 Brain-Computer Interface.

An adaptive P300-based control system

An adaptive P300 brain-computer interface (BCI) using a 12 × 7 matrix explored new paradigms to improve bit rate and accuracy. During online use, the system adaptively selects the number of flashes to average. Five different flash patterns were tested. The 19-flash paradigm represents the typical row/column presentation (i.e. 12 columns and 7 rows). The 9- and 14-flash A and B paradigms present all items of the 12 × 7 matrix three times using either 9 or 14 flashes (instead of 19), decreasing the amount of time to present stimuli. Compared to 9-flash A, 9-flash B decreased the likelihood that neighboring items would flash when the target was not flashing, thereby reducing the interference from items adjacent to targets. 14-flash A also reduced the adjacent item interference and 14-flash B additionally eliminated successive (double) flashes of the same item. Results showed that the accuracy and bit rate of the adaptive system were higher than those of the non-adaptive system. In addition, 9- and 14-flash B produced significantly higher performance than their respective A conditions. The results also show the trend that the 14-flash B paradigm was better than the 19-flash pattern for naive users.

Closing the loop in cortically-coupled computer vision: a brain-computer interface for searching image databases

We describe a closed-loop brain-computer interface that re-ranks an image database by iterating between user generated 'interest' scores and computer vision generated visual similarity measures. The interest scores are based on decoding the electroencephalographic (EEG) correlates of target detection, attentional shifts and self-monitoring processes, which result from the user paying attention to target images interspersed in rapid serial visual presentation (RSVP) sequences. The highest scored images are passed to a semi-supervised computer vision system that reorganizes the image database accordingly, using a graph-based representation that captures visual similarity between images. The system can either query the user for more information, by adaptively resampling the database to create additional RSVP sequences, or it can converge to a 'done' state. The done state includes a final ranking of the image database and also a 'guess' of the user's chosen category of interest. We find that the closed-loop system's re-rankings can substantially expedite database searches for target image categories chosen by the subjects. Furthermore, better reorganizations are achieved than by relying on EEG interest rankings alone, or if the system were simply run in an open loop format without adaptive resampling.

Towards intelligent environments: an augmented reality-brain-machine interface operated with a see-through head-mount display

The brain-machine interface (BMI) or brain-computer interface is a new interface technology that uses neurophysiological signals from the brain to control external machines or computers. This technology is expected to support daily activities, especially for persons with disabilities. To expand the range of activities enabled by this type of interface, here, we added augmented reality (AR) to a P300-based BMI. In this new system, we used a see-through head-mount display (HMD) to create control panels with flicker visual stimuli to support the user in areas close to controllable devices. When the attached camera detects an AR marker, the position and orientation of the marker are calculated, and the control panel for the pre-assigned appliance is created by the AR system and superimposed on the HMD. The participants were required to control system-compatible devices, and they successfully operated them without significant training. Online performance with the HMD was not different from that using an LCD monitor. Posterior and lateral (right or left) channel selections contributed to operation of the AR-BMI with both the HMD and LCD monitor. Our results indicate that AR-BMI systems operated with a see-through HMD may be useful in building advanced intelligent environments.

Suppressing flashes of items surrounding targets during calibration of a P300-based brain-computer interface improves performance

Since the introduction of the P300 brain-computer interface (BCI) speller by Farwell and Donchin in 1988, the speed and accuracy of the system has been significantly improved. Larger electrode montages and various signal processing techniques are responsible for most of the improvement in performance. New presentation paradigms have also led to improvements in bit rate and accuracy (e.g. Townsend et al (2010 Clin. Neurophysiol. 121 1109-20)). In particular, the checkerboard paradigm for online P300 BCI-based spelling performs well, has started to document what makes for a successful paradigm, and is a good platform for further experimentation. The current paper further examines the checkerboard paradigm by suppressing items which surround the target from flashing during calibration (i.e. the suppression condition). In the online feedback mode the standard checkerboard paradigm is used with a stepwise linear discriminant classifier derived from the suppression condition and one classifier derived from the standard checkerboard condition, counter-balanced. The results of this research demonstrate that using suppression during calibration produces significantly more character selections/min ((6.46) time between selections included) than the standard checkerboard condition (5.55), and significantly fewer target flashes are needed per selection in the SUP condition (5.28) as compared to the RCP condition (6.17). Moreover, accuracy in the SUP and RCP conditions remained equivalent (∼90%). Mean theoretical bit rate was 53.62 bits/min in the suppression condition and 46.36 bits/min in the standard checkerboard condition (ns). Waveform morphology also showed significant differences in amplitude and latency.

Control of a brain-computer interface using stereotactic depth electrodes in and adjacent to the hippocampus

A brain-computer interface (BCI) is a device that enables severely disabled people to communicate and interact with their environments using their brain waves. Most research investigating BCI in humans has used scalp-recorded electroencephalography or intracranial electrocorticography. The use of brain signals obtained directly from stereotactic depth electrodes to control a BCI has not previously been explored. In this study, event-related potentials (ERPs) recorded from bilateral stereotactic depth electrodes implanted in and adjacent to the hippocampus were used to control a P300 Speller paradigm. The ERPs were preprocessed and used to train a linear classifier to subsequently predict the intended target letters. The classifier was able to predict the intended target character at or near 100% accuracy using fewer than 15 stimulation sequences in the two subjects tested. Our results demonstrate that ERPs from hippocampal and hippocampal adjacent depth electrodes can be used to reliably control the P300 Speller BCI paradigm.

The P300-based brain-computer interface (BCI): effects of stimulus rate

OBJECTIVE

Brain-computer interface technology can restore communication and control to people who are severely paralyzed. We have developed a non-invasive BCI based on the P300 event-related potential that uses an 8×9 matrix of 72 items that flash in groups of 6. Stimulus presentation rate (i.e., flash rate) is one of several parameters that could affect the speed and accuracy of performance. We studied performance (i.e., accuracy and characters/min) on copy spelling as a function of flash rate.

METHODS

In the first study of six BCI users, stimulus-on and stimulus-off times were equal and flash rate was 4, 8, 16, or 32 Hz. In the second study of five BCI users, flash rate was varied by changing either the stimulus-on or stimulus-off time.

RESULTS

For all users, lower flash rates gave higher accuracy. The flash rate that gave the highest characters/min varied across users, ranging from 8 to 32 Hz. However, variations in stimulus-on and stimulus-off times did not themselves significantly affect accuracy. Providing feedback did not affect results in either study suggesting that offline analyses should readily generalize to online performance. However there do appear to be session-specific effects that can influence the generalizability of classifier results.

CONCLUSIONS

The results show that stimulus presentation (i.e., flash) rate affects the accuracy and speed of P300 BCI performance.

SIGNIFICANCE

These results extend the range over which slower flash rates increase the amplitude of the P300. Considering also presentation time, the optimal rate differs among users, and thus should be set empirically for each user. Optimal flash rate might also vary with other parameters such as the number of items in the matrix.

P300-based brain-computer interface for environmental control: an asynchronous approach

Brain-computer interface (BCI) systems allow people with severe motor disabilities to communicate and interact with the external world. The P300 potential is one of the most used control signals for EEG-based BCIs. Classic P300-based BCIs work in a synchronous mode; the synchronous control assumes that the user is constantly attending to the stimulation, and the number of stimulation sequences is fixed a priori. This issue is an obstacle for the use of these systems in everyday life; users will be engaged in a continuous control state, their distractions will cause misclassification and the speed of selection will not take into account users' current psychophysical condition. An efficient BCI system should be able to understand the user's intentions from the ongoing EEG instead. Also, it has to refrain from making a selection when the user is engaged in a different activity and it should increase or decrease its speed of selection depending on the current user's state. We addressed these issues by introducing an asynchronous BCI and tested its capabilities for effective environmental monitoring, involving 11 volunteers in three recording sessions. Results show that this BCI system can increase the bit rate during control periods while the system is proved to be very efficient in avoiding false negatives when the users are engaged in other tasks.

Manipulating attention via mindfulness induction improves P300-based brain-computer interface performance

In this study, we examined the effects of a short mindfulness meditation induction (MMI) on the performance of a P300-based brain-computer interface (BCI) task. We expected that MMI would harness present-moment attentional resources, resulting in two positive consequences for P300-based BCI use. Specifically, we believed that MMI would facilitate increases in task accuracy and promote the production of robust P300 amplitudes. Sixteen-channel electroencephalographic data were recorded from 18 subjects using a row/column speller task paradigm. Nine subjects participated in a 6 min MMI and an additional nine subjects served as a control group. Subjects were presented with a 6 × 6 matrix of alphanumeric characters on a computer monitor. Stimuli were flashed at a stimulus onset asynchrony (SOA) of 125 ms. Calibration data were collected on 21 items without providing feedback. These data were used to derive a stepwise linear discriminate analysis classifier that was applied to an additional 14 items to evaluate accuracy. Offline performance analyses revealed that MMI subjects were significantly more accurate than control subjects. Likewise, MMI subjects produced significantly larger P300 amplitudes than control subjects at Cz and PO7. The discussion focuses on the potential attentional benefits of MMI for P300-based BCI performance.

A case study on the relation between electroencephalographic and electrocorticographic event-related potentials

This study presents a preliminary analysis of the relationship between electroencephalographic (EEG) and electrocorticographic (ECoG) event-related potentials (ERPs) recorded from from a single patient using a brain-computer interface (BCI) speller. The patient had medically intractable epilepsy and underwent temporary placement of an intracranial ECoG grid electrode array to localize seizure foci. The patient performed one experimental session using the BCI spelling paradigm controlled by scalp-recorded EEG prior to the ECoG grid implantation, and one identical session controlled by ECoG after the grid implantation. The patient was able to achieve near perfect spelling accuracy using EEG and ECoG. An offline analysis of the average ERPs was performed to assess how accurately the average EEG ERPs could be predicted from the ECoG data. The preliminary results indicate that EEG ERPs can be accurately estimated from proximal asynchronous ECoG data using simple linear spatial models.

Two-dimensional auditory p300 speller with predictive text system

P300-based Brain Computer Interfaces offer communication pathways which are independent of muscle activity. Mostly visual stimuli, e.g. blinking of different letters are used as a paradigm of interaction. Neural degenerative diseases like amyotrophic lateral sclerosis (ALS) also cause a decrease in sight, but the ability of hearing is usually unaffected. Therefore, the use of the auditory modality might be preferable. This work presents a multiclass BCI paradigm using two-dimensional auditory stimuli: cues are varying in pitch (high/medium/low) and location (left/middle/right). The resulting nine different classes are embedded in a predictive text system, enabling to spell a letter with a 9-class decision. Moreover, an unbalanced subtrial selection is investigated and compared to the well-established sequence-wise paradigm. Twelve healthy subjects participated in an online study to investigate these approaches.

Convolutional neural networks for P300 detection with application to brain-computer interfaces

A Brain-Computer Interface (BCI) is a specific type of human-computer interface that enables the direct communication between human and computers by analyzing brain measurements. Oddball paradigms are used in BCI to generate event-related potentials (ERPs), like the P300 wave, on targets selected by the user. A P300 speller is based on this principle, where the detection of P300 waves allows the user to write characters. The P300 speller is composed of two classification problems. The first classification is to detect the presence of a P300 in the electroencephalogram (EEG). The second one corresponds to the combination of different P300 responses for determining the right character to spell. A new method for the detection of P300 waves is presented. This model is based on a convolutional neural network (CNN). The topology of the network is adapted to the detection of P300 waves in the time domain. Seven classifiers based on the CNN are proposed: four single classifiers with different features set and three multiclassifiers. These models are tested and compared on the Data set II of the third BCI competition. The best result is obtained with a multiclassifier solution with a recognition rate of 95.5 percent, without channel selection before the classification. The proposed approach provides also a new way for analyzing brain activities due to the receptive field of the CNN models.

Classification of selective attention to auditory stimuli: toward vision-free brain-computer interfacing

Brain-computer interface (BCI) is a developing, novel mode of communication for individuals with severe motor impairments or those who have no other options for communication aside from their brain signals. However, the majority of current BCI systems are based on visual stimuli or visual feedback, which may not be applicable for severe locked-in patients that have lost their eyesight or the ability to control their eye movements. In the present study, we investigated the feasibility of using auditory steady-state responses (ASSRs), elicited by selective attention to a specific sound source, as an electroencephalography (EEG)-based BCI paradigm. In our experiment, two pure tone burst trains with different beat frequencies (37 and 43 Hz) were generated simultaneously from two speakers located at different positions (left and right). Six participants were instructed to close their eyes and concentrate their attention on either auditory stimulus according to the instructions provided randomly through the speakers during the inter-stimulus interval. EEG signals were recorded at multiple electrodes mounted over the temporal, occipital, and parietal cortices. We then extracted feature vectors by combining spectral power densities evaluated at the two beat frequencies. Our experimental results showed high classification accuracies (64.67%, 30 commands/min, information transfer rate (ITR) = 1.89 bits/min; 74.00%, 12 commands/min, ITR = 2.08 bits/min; 82.00%, 6 commands/min, ITR = 1.92 bits/min; 84.33%, 3 commands/min, ITR = 1.12 bits/min; without any artifact rejection, inter-trial interval = 6s), enough to be used for a binary decision. Based on the suggested paradigm, we implemented a first online ASSR-based BCI system that demonstrated the possibility of materializing a totally vision-free BCI system.

Toward a model-based predictive controller design in brain-computer interfaces

A first step in designing a robust and optimal model-based predictive controller (MPC) for brain-computer interface (BCI) applications is presented in this article. An MPC has the potential to achieve improved BCI performance compared to the performance achieved by current ad hoc, nonmodel-based filter applications. The parameters in designing the controller were extracted as model-based features from motor imagery task-related human scalp electroencephalography. Although the parameters can be generated from any model-linear or non-linear, we here adopted a simple autoregressive model that has well-established applications in BCI task discriminations. It was shown that the parameters generated for the controller design can as well be used for motor imagery task discriminations with performance (with 8-23% task discrimination errors) comparable to the discrimination performance of the commonly used features such as frequency specific band powers and the AR model parameters directly used. An optimal MPC has significant implications for high performance BCI applications.

A brain-computer interface for long-term independent home use

Our objective was to develop and validate a new brain-computer interface (BCI) system suitable for long-term independent home use by people with severe motor disabilities. The BCI was used by a 51-year-old male with ALS who could no longer use conventional assistive devices. Caregivers learned to place the electrode cap, add electrode gel, and turn on the BCI. After calibration, the system allowed the user to communicate via EEG. Re-calibration was performed remotely (via the internet), and BCI accuracy assessed in periodic tests. Reports of BCI usefulness by the user and the family were also recorded. Results showed that BCI accuracy remained at 83% (r = -.07, n.s.) for over 2.5 years (1.4% expected by chance). The BCI user and his family state that the BCI had restored his independence in social interactions and at work. He uses the BCI to run his NIH-funded research laboratory and to communicate via e-mail with family, friends, and colleagues. In addition to this first user, several other similarly disabled people are now using the BCI in their daily lives. In conclusion, long-term independent home use of this BCI system is practical for severely disabled people, and can contribute significantly to quality of life and productivity.

Brain-Computer Interfaces for Communication and Control

The brain's electrical signals enable people without muscle control to physically interact with the world.

Predictive spelling with a P300-based brain-computer interface: Increasing the rate of communication

This study compared a conventional P300 speller brain-computer interface (BCI) to one used in conjunction with a predictive spelling program. Performance differences in accuracy, bit rate, selections per minute, and output characters per minute (OCM) were examined. An 8×9 matrix of letters, numbers, and other keyboard commands was used. Participants (n = 24) were required to correctly complete the same 58 character sentence (i.e., correcting for errors) using the predictive speller (PS) and the non-predictive speller (NS), counterbalanced. The PS produced significantly higher OCMs than the NS. Time to complete the task in the PS condition was 12min 43sec as compared to 20min 20sec in the NS condition. Despite the marked improvement in overall output, accuracy was significantly higher in the NS paradigm. P300 amplitudes were significantly larger in the NS than in the PS paradigm; which is attributed to increased workload and task demands. These results demonstrate the potential efficacy of predictive spelling in the context of BCI.

Learning event-related potentials (ERPs) from multichannel EEG recordings: a spatio-temporal modeling framework with a fast estimation algorithm

Extracting event-related potentials (ERPs) from multichannel EEG recordings remains a challenge due to the poor signal-to-noise ratio (SNR). This paper presents a multivariate statistical model of ERPs by exploiting the existing knowledge about their spatio-temporal properties. In particular, a computationally efficient algorithm is derived for fast model estimation. The algorithm, termed SIM, can be intuitively interpreted as maximizing the signal-to-noise ratio in the source space. Using both simulated and real EEG data, we show that the algorithm achieves excellent estimation performance and substantially outperforms a state-of-the-arts algorithm in classification accuracies in a P300 target detection task. The results demonstrate that the proposed modeling framework offers a powerful tool for exploring the spatio-temporal patterns of ERPs as well as learning spatial filters for decoding brain states.

Out of the frying pan into the fire--the P300-based BCI faces real-world challenges

Brain-computer interfaces (BCIs) have been investigated for more than 20 years. Many BCIs use noninvasive electroencephalography as a measurement technique and the P300 event-related potential as an input signal (P300 BCI). Since the first experiment with a P300 BCI system in 1988 by Farwell and Donchin, not only data processing has improved but also stimuli presentation has been varied and a plethora of applications was developed and refined. Nowadays, these applications are facing the challenge of being transferred from the research laboratory into real-life situations to serve motor-impaired people in their homes as assistive technology.

Statistical spatial filtering for a P300-based BCI: tests in able-bodied, and patients with cerebral palsy and amyotrophic lateral sclerosis

The effective use of brain-computer interfaces (BCIs) in real-world environments depends on a satisfactory throughput. In a P300-based BCI, this can be attained by reducing the number of trials needed to detect the P300 signal. However, this task is hampered by the very low signal-to-noise-ratio (SNR) of P300 event related potentials. This paper proposes an efficient methodology that achieves high classification accuracy and high transfer rates for both disabled and able-bodied subjects in a standard P300-based speller system. The system was tested by three subjects with cerebral palsy (CP), two subjects with amyotrophic lateral sclerosis (ALS), and nineteen able-bodied subjects. The paper proposes the application of three statistical spatial filters. The first is a beamformer that maximizes the ratio of signal power and noise power (Max-SNR). The second is a beamformer based on the Fisher criterion (FC). The third approach cascades the FC beamformer with the Max-SNR beamformer satisfying simultaneously sub-optimally both criteria (C-FMS). The calibration process of the BCI system takes about 5 min to collect data and a couple of minutes to obtain spatial filters and classification models. Online results showed that subjects with disabilities have achieved, on average, an accuracy and transfer rate only slightly lower than able-bodied subjects. Taking 23 of the 24 participants, the averaged results achieved a transfer rate of 4.33 symbols per minute with a 91.80% accuracy, corresponding to a bandwidth of 19.18 bits per minute. This study shows the feasibility of the proposed methodology and that effective communication rates are achievable.

A study on EEG quality in physical movements with Steady-State Visual Evoked Potentials

In this paper, we investigated the quality of ElectroEncephaloGraphic (EEG) signals during performing physical movements. By using a portable EEG device, the Steady-State Visual Evoked Potential (SSVEP) was recorded on parietal and occipital locations. The SSVEP induced by flickering stimuli was successfully observed in the self-paced mimic walking conditions as well as in the sitting conditions. To see the dependence of temporal and spatial filters on the potential performance of Brain-Computer Interfaces (BCI) we applied the signal processing of Principal Component Analysis and Linear Discriminant Analysis. The pattern recognition performances in inferring the subject's eye gaze directions from the EEG signals could be perfect even in the self-paced mimic walking conditions. It was found that three electrodes on parieto-occipital and occipital locations were essential in order to have perfect performances. From these results, we conclude that the applications using SSVEP-based BCI can be realized even in the physically moving context.

Brain Painting: First Evaluation of a New Brain-Computer Interface Application with ALS-Patients and Healthy Volunteers

Brain-computer interfaces (BCIs) enable paralyzed patients to communicate; however, up to date, no creative expression was possible. The current study investigated the accuracy and user-friendliness of P300-Brain Painting, a new BCI application developed to paint pictures using brain activity only. Two different versions of the P300-Brain Painting application were tested: A colored matrix tested by a group of ALS-patients (n = 3) and healthy participants (n = 10), and a black and white matrix tested by healthy participants (n = 10). The three ALS-patients achieved high accuracies; two of them reaching above 89% accuracy. In healthy subjects, a comparison between the P300-Brain Painting application (colored matrix) and the P300-Spelling application revealed significantly lower accuracy and P300 amplitudes for the P300-Brain Painting application. This drop in accuracy and P300 amplitudes was not found when comparing the P300-Spelling application to an adapted, black and white matrix of the P300-Brain Painting application. By employing a black and white matrix, the accuracy of the P300-Brain Painting application was significantly enhanced and reached the accuracy of the P300-Spelling application. ALS-patients greatly enjoyed P300-Brain Painting and were able to use the application with the same accuracy as healthy subjects. P300-Brain Painting enables paralyzed patients to express themselves creatively and to participate in the prolific society through exhibitions.

Control of a visual keyboard using an electrocorticographic brain-computer interface

OBJECTIVE

Brain-computer interfaces (BCIs) are devices that enable severely disabled people to communicate and interact with their environments using their brain waves. Most studies investigating BCI in humans have used scalp EEG as the source of electrical signals and focused on motor control of prostheses or computer cursors on a screen. The authors hypothesize that the use of brain signals obtained directly from the cortical surface will more effectively control a communication/spelling task compared to scalp EEG.

METHODS

A total of 6 patients with medically intractable epilepsy were tested for the ability to control a visual keyboard using electrocorticographic (ECOG) signals. ECOG data collected during a P300 visual task paradigm were preprocessed and used to train a linear classifier to subsequently predict the intended target letters.

RESULTS

The classifier was able to predict the intended target character at or near 100% accuracy using fewer than 15 stimulation sequences in 5 of the 6 people tested. ECOG data from electrodes outside the language cortex contributed to the classifier and enabled participants to write words on a visual keyboard.

CONCLUSIONS

This is a novel finding because previous invasive BCI research in humans used signals exclusively from the motor cortex to control a computer cursor or prosthetic device. These results demonstrate that ECOG signals from electrodes both overlying and outside the language cortex can reliably control a visual keyboard to generate language output without voice or limb movements.

Does the 'P300' speller depend on eye gaze?

Many people affected by debilitating neuromuscular disorders such as amyotrophic lateral sclerosis, brainstem stroke or spinal cord injury are impaired in their ability to, or are even unable to, communicate. A brain-computer interface (BCI) uses brain signals, rather than muscles, to re-establish communication with the outside world. One particular BCI approach is the so-called 'P300 matrix speller' that was first described by Farwell and Donchin (1988 Electroencephalogr. Clin. Neurophysiol. 70 510-23). It has been widely assumed that this method does not depend on the ability to focus on the desired character, because it was thought that it relies primarily on the P300-evoked potential and minimally, if at all, on other EEG features such as the visual-evoked potential (VEP). This issue is highly relevant for the clinical application of this BCI method, because eye movements may be impaired or lost in the relevant user population. This study investigated the extent to which the performance in a 'P300' speller BCI depends on eye gaze. We evaluated the performance of 17 healthy subjects using a 'P300' matrix speller under two conditions. Under one condition ('letter'), the subjects focused their eye gaze on the intended letter, while under the second condition ('center'), the subjects focused their eye gaze on a fixation cross that was located in the center of the matrix. The results show that the performance of the 'P300' matrix speller in normal subjects depends in considerable measure on gaze direction. They thereby disprove a widespread assumption in BCI research, and suggest that this BCI might function more effectively for people who retain some eye-movement control. The applicability of these findings to people with severe neuromuscular disabilities (particularly in eye-movements) remains to be determined.

Operation of a P300-based brain-computer interface by individuals with cervical spinal cord injury

OBJECTIVE

This study evaluates the efficacy of a P300-based brain-computer interface (BCI) with green/blue flicker matrices for individuals with cervical spinal cord injury (SCI).

METHODS

Ten individuals with cervical SCI (age 26-53, all male) and 10 age- and sex-matched able-bodied controls (age 27-52, all male) with no prior BCI experience were asked to input hiragana (Japanese alphabet) characters using the P300 BCI with two distinct types of visual stimuli, white/gray and green/blue, in an 8×10 flicker matrix. Both online and offline performance were evaluated.

RESULTS

The mean online accuracy of the SCI subjects was 88.0% for the white/gray and 90.7% for the green/blue flicker matrices. The accuracy of the control subjects was 77.3% and 86.0% for the white/gray and green/blue, respectively. There was a significant difference in online accuracy between the two types of flicker matrix. SCI subjects performed with greater accuracy than controls, but the main effect was not significant.

CONCLUSIONS

Individuals with cervical SCI successfully controlled the P300 BCI, and the green/blue flicker matrices were associated with significantly higher accuracy than the white/gray matrices.

SIGNIFICANCE

The P300 BCI with the green/blue flicker matrices is effective for use not only in able-bodied subjects, but also in individuals with cervical SCI.

Development of speech prostheses: current status and recent advances

Brain-computer interfaces (BCIs) have been developed over the past decade to restore communication to persons with severe paralysis. In the most severe cases of paralysis, known as locked-in syndrome, patients retain cognition and sensation, but are capable of only slight voluntary eye movements. For these patients, no standard communication method is available, although some can use BCIs to communicate by selecting letters or words on a computer. Recent research has sought to improve on existing techniques by using BCIs to create a direct prediction of speech utterances rather than to simply control a spelling device. Such methods are the first steps towards speech prostheses as they are intended to entirely replace the vocal apparatus of paralyzed users. This article outlines many well known methods for restoration of communication by BCI and illustrates the difference between spelling devices and direct speech prediction or speech prosthesis.

The influence of psychological state and motivation on brain-computer interface performance in patients with amyotrophic lateral sclerosis - a longitudinal study

The current study investigated the effects of psychological well-being measured as quality of life (QoL), depression, current mood and motivation on brain–computer interface (BCI) performance in amyotrophic lateral sclerosis (ALS). Six participants with most advanced ALS were trained either for a block of 20 sessions with a BCI based on sensorimotor rhythms (SMR) or a block of 10 sessions with a BCI based on event-related potentials, or both. Questionnaires assessed QoL and severity of depressive symptoms before each training block and mood and motivation before each training session. The SMR-BCI required more training than the P300-BCI. The information transfer rate was higher with the P300-BCI (3.25 bits/min) than with the SMR-BCI (1.16 bits/min). Mood and motivation were related to the number of BCI sessions. Motivational factors, specifically challenge and mastery confidence, were positively related to BCI performance (controlled for the number of sessions) in tow participants, while incompetence fear was negatively related with performance in one participant. BCI performance was not related to motivational factors in three other participants nor to mood in any of the six participants. We conclude that motivational factors may be related to BCI performance in individual subjects and suggest that motivational factors and well-being should be assessed in standard BCI protocols. We also recommend using P300-based BCI as first choice in severely paralyzed patients who present with a P300 evoked potential.

A novel P300-based brain-computer interface stimulus presentation paradigm: moving beyond rows and columns

OBJECTIVE

An electroencephalographic brain-computer interface (BCI) can provide a non-muscular means of communication for people with amyotrophic lateral sclerosis (ALS) or other neuromuscular disorders. We present a novel P300-based BCI stimulus presentation - the checkerboard paradigm (CBP). CBP performance is compared to that of the standard row/column paradigm (RCP) introduced by Farwell and Donchin (1988).

METHODS

Using an 8x9 matrix of alphanumeric characters and keyboard commands, 18 participants used the CBP and RCP in counter-balanced fashion. With approximately 9-12 min of calibration data, we used a stepwise linear discriminant analysis for online classification of subsequent data.

RESULTS

Mean online accuracy was significantly higher for the CBP, 92%, than for the RCP, 77%. Correcting for extra selections due to errors, mean bit rate was also significantly higher for the CBP, 23 bits/min, than for the RCP, 17 bits/min. Moreover, the two paradigms produced significantly different waveforms. Initial tests with three advanced ALS participants produced similar results. Furthermore, these individuals preferred the CBP to the RCP.

CONCLUSIONS

These results suggest that the CBP is markedly superior to the RCP in performance and user acceptability.

SIGNIFICANCE

The CBP has the potential to provide a substantially more effective BCI than the RCP. This is especially important for people with severe neuromuscular disabilities.

N1 wave in the P300 BCI is not sensitive to the physical characteristics of stimuli

One of the widely used paradigms for the brain-computer interface (BCI), the P300 BCI, was proposed by Farwell and Donchin as a variation of the classical visual oddball paradigm, known to elicit the P300 component of the brain event-related potentials (ERP). We show that this paradigm, unlike the standard oddball paradigm, elicit not only the P300 wave but also a strong posterior N1 wave. Moreover, we present evidence that the sensitivity of this ERP component to targets cannot be explained by the variations of the perceived stimuli energy. This evidence is based on comparing the ERP obtained for usual P300 BCI stimuli and for the "inverted" stimulation scheme with low stimulus related variations of light energy (gray letters on the light gray background, "highlighted" by very light darkening). Despite the dramatic difference between the stimuli in the standard and "inverted" schemes, no difference between N1 amplitudes were found, supporting the view that this component's sensitivity to targets cannot be based simply on "foveating" the target, but may be related to spatial attention mechanisms, which involvement is natural for the P300 BCI. Efforts to optimize the P300 BCI should address better use of both P300 and N1 waves.

(C)overt attention and visual speller design in an ERP-based brain-computer interface

BACKGROUND

In a visual oddball paradigm, attention to an event usually modulates the event-related potential (ERP). An ERP-based brain-computer interface (BCI) exploits this neural mechanism for communication. Hitherto, it was unclear to what extent the accuracy of such a BCI requires eye movements (overt attention) or whether it is also feasible for targets in the visual periphery (covert attention). Also unclear was how the visual design of the BCI can be improved to meet peculiarities of peripheral vision such as low spatial acuity and crowding.

METHOD

Healthy participants (N = 13) performed a copy-spelling task wherein they had to count target intensifications. EEG and eye movements were recorded concurrently. First, (c)overt attention was investigated by way of a target fixation condition and a central fixation condition. In the latter, participants had to fixate a dot in the center of the screen and allocate their attention to a target in the visual periphery. Second, the effect of visual speller layout was investigated by comparing the symbol Matrix to an ERP-based Hex-o-Spell, a two-levels speller consisting of six discs arranged on an invisible hexagon.

RESULTS

We assessed counting errors, ERP amplitudes, and offline classification performance. There is an advantage (i.e., less errors, larger ERP amplitude modulation, better classification) of overt attention over covert attention, and there is also an advantage of the Hex-o-Spell over the Matrix. Using overt attention, P1, N1, P2, N2, and P3 components are enhanced by attention. Using covert attention, only N2 and P3 are enhanced for both spellers, and N1 and P2 are modulated when using the Hex-o-Spell but not when using the Matrix. Consequently, classifiers rely mainly on early evoked potentials in overt attention and on later cognitive components in covert attention.

CONCLUSIONS

Both overt and covert attention can be used to drive an ERP-based BCI, but performance is markedly lower for covert attention. The Hex-o-Spell outperforms the Matrix, especially when eye movements are not permitted, illustrating that performance can be increased if one accounts for peculiarities of peripheral vision.

A dictionary-driven P300 speller with a modified interface

P300 spellers are mainly composed of an interface, by which alphanumerical characters are presented to users, and a classification system, which identifies the target character by using acquired EEG data. In this study, we proposed modifications both to the interface and to the classification system, in order to reduce the number of required stimulus repetitions and consequently boost the information transfer rate. We initially incorporated a custom-built dictionary into the classification system, and conducted a study on 14 healthy subjects who copy-spelled 15 four letter words. Incorporating the dictionary, the mean accuracy at five trials increased from 72.86% to 95.71%. To further increase the system performance, we first validated the hypothesis that for a conventional P300 system, most target-error pairs lie on the same row or column. Then based on the validated hypothesis, we adjusted letter positions on the well-known from A to Z interface. The same subjects spelled the same 15 words using the modified interface as well, and the mean information transfer rate at two trials reached 55.32 bits/min.

Single trial independent component analysis for P300 BCI system

A Brain Computer Interface (BCI) is a device that allows the user to communicate with the world without utilizing voluntary muscle activity (i.e., using only the electrical activity of the brain). It makes use of the well-studied observation that the brain reacts differently to different stimuli, as a function of the level of attention allotted to the stimulus stream and the specific processing triggered by the stimulus. In this article we present a single trial independent component analysis (ICA) method that is working with a BCI system proposed by Farwell and Donchin. It can dramatically reduce the signal processing time and improve the data communicating rate. This ICA method achieved 76.67% accuracy on single trial P300 response identification.

P300 Chinese input system based on Bayesian LDA

A brain-computer interface (BCI) is a new communication channel between humans and computers that translates brain activity into recognizable command and control signals. Attended events can evoke P300 potentials in the electroencephalogram. Hence, the P300 has been used in BCI systems to spell, control cursors or robotic devices, and other tasks. This paper introduces a novel P300 BCI to communicate Chinese characters. To improve classification accuracy, an optimization algorithm (particle swarm optimization, PSO) is used for channel selection (i.e., identifying the best electrode configuration). The effects of different electrode configurations on classification accuracy were tested by Bayesian linear discriminant analysis offline. The offline results from 11 subjects show that this new P300 BCI can effectively communicate Chinese characters and that the features extracted from the electrodes obtained by PSO yield good performance.

P300-based brain computer interface experimental setup

A Brain-Computer interface (BCI) is a communication system that enables the generation of a control signal from brain signals such as sensorymotor rhythms and evoked potentials; therefore, it constitutes a novel communication option for people with severe motor disabilities (such as Amyotrophic Lateral Sclerosis patients). This paper presents the development of a P300-based BCI. This prototype uses a homemade six-channel electroencephalograph for the acquisition of the signals, and a visual stimulation matrix; since this matrix contains letters of the alphabet as well as images associated to them, it permits word-writing and the elaboration of messages with the images. To process the signals the software BCI2000 and MATLAB 7.0 were used. The latter was used to program three linear translation algorithms (Stepwise Linear Discriminant Analysis, Lineal Discriminant Analysis and Least Squares) to convert the brain signals into communication signals. These algorithms had a classification accuracy of 90.73 %, 95.75 % and 89.45 % respectively, when using raw data; and of 90.78%, 49.48 % and 53.9 %, when data was previously common-average filtered. The experimental setup was tested in ten healthy volunteers; 5 of them got a 100% success, 1 a 90% success, 2 an around 70% success and 2 a 50% success, in the online free-spelling tests.

Feature selection on movement imagery discrimination and attention detection

Noninvasive brain-computer interfaces (BCI) translate subject's electroencephalogram (EEG) features into device commands. Large feature sets should be down-selected for efficient feature translation. This work proposes two different feature down-selection algorithms for BCI: (a) a sequential forward selection; and (b) an across-group variance. Power rar ratios (PRs) were extracted from the EEG data for movement imagery discrimination. Event-related potentials (ERPs) were employed in the discrimination of cue-evoked responses. While center-out arrows, commonly used in calibration sessions, cued the subjects in the first experiment (for both PR and ERP analyses), less stimulating arrows that were centered in the visual field were employed in the second experiment (for ERP analysis). The proposed algorithms outperformed other three popular feature selection algorithms in movement imagery discrimination. In the first experiment, both algorithms achieved classification errors as low as 12.5% reducing the feature set dimensionality by more than 90%. The classification accuracy of ERPs dropped in the second experiment since centered cues reduced the amplitude of cue-evoked ERPs. The two proposed algorithms effectively reduced feature dimensionality while increasing movement imagery discrimination and detected cue-evoked ERPs that reflect subject attention.

Brain-Computer Interfaces for Speech Communication

This paper briefly reviews current silent speech methodologies for normal and disabled individuals. Current techniques utilizing electromyographic (EMG) recordings of vocal tract movements are useful for physically healthy individuals but fail for tetraplegic individuals who do not have accurate voluntary control over the speech articulators. Alternative methods utilizing EMG from other body parts (e.g., hand, arm, or facial muscles) or electroencephalography (EEG) can provide capable silent communication to severely paralyzed users, though current interfaces are extremely slow relative to normal conversation rates and require constant attention to a computer screen that provides visual feedback and/or cueing. We present a novel approach to the problem of silent speech via an intracortical microelectrode brain computer interface (BCI) to predict intended speech information directly from the activity of neurons involved in speech production. The predicted speech is synthesized and acoustically fed back to the user with a delay under 50 ms. We demonstrate that the Neurotrophic Electrode used in the BCI is capable of providing useful neural recordings for over 4 years, a necessary property for BCIs that need to remain viable over the lifespan of the user. Other design considerations include neural decoding techniques based on previous research involving BCIs for computer cursor or robotic arm control via prediction of intended movement kinematics from motor cortical signals in monkeys and humans. Initial results from a study of continuous speech production with instantaneous acoustic feedback show the BCI user was able to improve his control over an artificial speech synthesizer both within and across recording sessions. The success of this initial trial validates the potential of the intracortical microelectrode-based approach for providing a speech prosthesis that can allow much more rapid communication rates.