Brain Painting: First Evaluation of a New Brain-Computer Interface Application with ALS-Patients and Healthy Volunteers

Long-term independent brain-computer interface home use improves quality of life of a patient in the locked-in state: a case study

OBJECTIVE

Despite intense brain-computer interface (BCI) research for >2 decades, BCIs have hardly been established at patients' homes. The current study aimed at demonstrating expert independent BCI home use by a patient in the locked-in state and the effect it has on quality of life.

DESIGN

In this case study, the P300 BCI-controlled application Brain Painting was facilitated and installed at the patient's home. Family and caregivers were trained in setting up the BCI system. After every BCI session, the end user indicated subjective level of control, loss of control, level of exhaustion, satisfaction, frustration, and enjoyment. To monitor BCI home use, evaluation data of every session were automatically sent and stored on a remote server. Satisfaction with the BCI as an assistive device and subjective workload was indicated by the patient. In accordance with the user-centered design, usability of the BCI was evaluated in terms of its effectiveness, efficiency, and satisfaction. The influence of the BCI on quality of life of the end user was assessed.

SETTING

At the patient's home.

PARTICIPANT

A 73-year-old patient with amyotrophic lateral sclerosis in the locked-in state.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURE

The BCI has been used by the patient independent of experts for >14 months. The patient painted in about 200 BCI sessions (1-3 times per week) with a mean painting duration of 81.86 minutes (SD=52.15, maximum: 230.41). BCI improved quality of life of the patient.

RESULTS

In most of the BCI sessions the end user's satisfaction was high (mean=7.4, SD=3.24; range, 0-10). Dissatisfaction occurred mostly because of technical problems at the beginning of the study or varying BCI control. The subjective workload was moderate (mean=40.61; range, 0-100). The end user was highy satisfied with all components of the BCI (mean 4.42-5.0; range, 1-5). A perfect match between the user and the BCI technology was achieved (mean: 4.8; range, 1-5). Brain Painting had a positive impact on the patient's life on all three dimensions: competence (1.5), adaptability (2.17) and self-esteem (1.5); (range: -3 = maximum negative impact; 3 maximum positive impact). The patient had her first public art exhibition in July 2013; future exhibitions are in preparation.

CONCLUSIONS

Independent BCI home use is possible with high satisfaction for the end user. The BCI indeed positively influenced quality of life of the patient and supports social inclusion. Results demonstrate that visual P300 BCIs can be valuable for patients in the locked-in state even if other means of communication are still available (eye tracker).

Usability and Workload of Access Technology for People With Severe Motor Impairment

Background. Eye trackers are widely used among people with amyotrophic lateral sclerosis, and their benefits to quality of life have been previously shown. On the contrary, Brain-computer interfaces (BCIs) are still quite a novel technology, which also serves as an access technology for people with severe motor impairment. Objective. To compare a visual P300-based BCI and an eye tracker in terms of information transfer rate (ITR), usability, and cognitive workload in users with motor impairments. Methods. Each participant performed 3 spelling tasks, over 4 total sessions, using an Internet browser, which was controlled by a spelling interface that was suitable for use with either the BCI or the eye tracker. At the end of each session, participants evaluated usability and cognitive workload of the system. Results. ITR and System Usability Scale (SUS) score were higher for the eye tracker (Wilcoxon signed-rank test: ITR T = 9, P = .016; SUS T = 12.50, P = .035). Cognitive workload was higher for the BCI ( T = 4; P = .003). Conclusions. Although BCIs could be potentially useful for people with severe physical disabilities, we showed that the usability of BCIs based on the visual P300 remains inferior to eye tracking. We suggest that future research on visual BCIs should use eye tracking–based control as a comparison to evaluate performance or focus on nonvisual paradigms for persons who have lost gaze control.

The user-centered design as novel perspective for evaluating the usability of BCI-controlled applications

Albeit research on brain-computer interfaces (BCI) for controlling applications has expanded tremendously, we still face a translational gap when bringing BCI to end-users. To bridge this gap, we adapted the user-centered design (UCD) to BCI research and development which implies a shift from focusing on single aspects, such as accuracy and information transfer rate (ITR), to a more holistic user experience. The UCD implements an iterative process between end-users and developers based on a valid evaluation procedure. Within the UCD framework usability of a device can be defined with regard to its effectiveness, efficiency, and satisfaction. We operationalized these aspects to evaluate BCI-controlled applications. Effectiveness was regarded equivalent to accuracy of selections and efficiency to the amount of information transferred per time unit and the effort invested (workload). Satisfaction was assessed with questionnaires and visual-analogue scales. These metrics have been successfully applied to several BCI-controlled applications for communication and entertainment, which were evaluated by end-users with severe motor impairment. Results of four studies, involving a total of N = 19 end-users revealed: effectiveness was moderate to high; efficiency in terms of ITR was low to high and workload low to medium; depending on the match between user and technology, and type of application satisfaction was moderate to high. The here suggested evaluation metrics within the framework of the UCD proved to be an applicable and informative approach to evaluate BCI controlled applications, and end-users with severe impairment and in the locked-in state were able to participate in this process.

Brain–Computer Interface after Nervous System Injury

Brain–computer interface (BCI) has proven to be a useful tool for providing alternative communication and mobility to patients suffering from nervous system injury. BCI has been and will continue to be implemented into rehabilitation practices for more interactive and speedy neurological recovery. The most exciting BCI technology is evolving to provide therapeutic benefits by inducing cortical reorganization via neuronal plasticity. This article presents a state-of-the-art review of BCI technology used after nervous system injuries, specifically: amyotrophic lateral sclerosis, Parkinson’s disease, spinal cord injury, stroke, and disorders of consciousness. Also presented is transcending, innovative research involving new treatment of neurological disorders.

A P300-based brain-computer interface aimed at operating electronic devices at home for severely disabled people

The present study aims at developing and assessing an assistive tool for operating electronic devices at home by means of a P300-based brain-computer interface (BCI). Fifteen severely impaired subjects participated in the study. The developed tool allows users to interact with their usual environment fulfilling their main needs. It allows for navigation through ten menus and to manage up to 113 control commands from eight electronic devices. Ten out of the fifteen subjects were able to operate the proposed tool with accuracy above 77 %. Eight out of them reached accuracies higher than 95 %. Moreover, bitrates up to 20.1 bit/min were achieved. The novelty of this study lies in the use of an environment control application in a real scenario: real devices managed by potential BCI end-users. Although impaired users might not be able to set up this system without aid of others, this study takes a significant step to evaluate the degree to which such populations could eventually operate a stand-alone system. Our results suggest that neither the type nor the degree of disability is a relevant issue to suitably operate a P300-based BCI. Hence, it could be useful to assist disabled people at home improving their personal autonomy.

Beyond maximum speed--a novel two-stimulus paradigm for brain-computer interfaces based on event-related potentials (P300-BCI)

OBJECTIVE

The speed of brain-computer interfaces (BCI), based on event-related potentials (ERP), is inherently limited by the commonly used one-stimulus paradigm. In this paper, we introduce a novel paradigm that can increase the spelling speed by a factor of 2, thereby extending the one-stimulus paradigm to a two-stimulus paradigm. Two different stimuli (a face and a symbol) are presented at the same time, superimposed on different characters and ERPs are classified using a multi-class classifier. Here, we present the proof-of-principle that is achieved with healthy participants.

APPROACH

Eight participants were confronted with the novel two-stimulus paradigm and, for comparison, with two one-stimulus paradigms that used either one of the stimuli. Classification accuracies (percentage of correctly predicted letters) and elicited ERPs from the three paradigms were compared in a comprehensive offline analysis.

MAIN RESULTS

The accuracies slightly decreased with the novel system compared to the established one-stimulus face paradigm. However, the use of two stimuli allowed for spelling at twice the maximum speed of the one-stimulus paradigms, and participants still achieved an average accuracy of 81.25%. This study introduced an alternative way of increasing the spelling speed in ERP-BCIs and illustrated that ERP-BCIs may not yet have reached their speed limit. Future research is needed in order to improve the reliability of the novel approach, as some participants displayed reduced accuracies. Furthermore, a comparison to the most recent BCI systems with individually adjusted, rapid stimulus timing is needed to draw conclusions about the practical relevance of the proposed paradigm.

SIGNIFICANCE

We introduced a novel two-stimulus paradigm that might be of high value for users who have reached the speed limit with the current one-stimulus ERP-BCI systems.

Brain-computer interface (BCI) evaluation in people with amyotrophic lateral sclerosis

Brain-computer interfaces (BCIs) might restore communication to people severely disabled by amyotrophic lateral sclerosis (ALS) or other disorders. We sought to: 1) define a protocol for determining whether a person with ALS can use a visual P300-based BCI; 2) determine what proportion of this population can use the BCI; and 3) identify factors affecting BCI performance. Twenty-five individuals with ALS completed an evaluation protocol using a standard 6 × 6 matrix and parameters selected by stepwise linear discrimination. With an 8-channel EEG montage, the subjects fell into two groups in BCI accuracy (chance accuracy 3%). Seventeen averaged 92 (± 3)% (range 71-100%), which is adequate for communication (G70 group). Eight averaged 12 (± 6)% (range 0-36%), inadequate for communication (L40 subject group). Performance did not correlate with disability: 11/17 (65%) of G70 subjects were severely disabled (i.e. ALSFRS-R < 5). All L40 subjects had visual impairments (e.g. nystagmus, diplopia, ptosis). P300 was larger and more anterior in G70 subjects. A 16-channel montage did not significantly improve accuracy. In conclusion, most people severely disabled by ALS could use a visual P300-based BCI for communication. In those who could not, visual impairment was the principal obstacle. For these individuals, auditory P300-based BCIs might be effective.

Effectiveness of the P3-speller in brain-computer interfaces for amyotrophic lateral sclerosis patients: a systematic review and meta-analysis

A quarter of century ago, Farwell and Donchin (1988) described their mental prosthesis for “talking off the top of your head.” This innovative communication system, later named P3-speller, has been the most investigated and tested brain–computer interface (BCI) system, to date. A main goal of the research on P3-spellers was the development of an effective assistive device for patients with severe motor diseases. Among these patients are those affected by amyotrophic lateral sclerosis (ALS). ALS patients have become a target population in P3-speller (and more generally in BCI) research. The P3-speller relies on the visual sensory modality, and it can be controlled by requiring users to actively move their eyes. Unfortunately, eye-movement control is usually not spared in the last stages of ALS, and, then, it is definitively lost in the case of complete paralysis. We reviewed the literature on ALS patients tested by means of P3-speller systems. Our aim was to investigate the evidence available to date of the P3-spellers effectiveness in ALS patients. To address this goal, a meta-analytic approach was adopted. The pooled classification accuracy performance, among retrieved studies, was about 74%. This estimation, however, was affected by significant heterogeneity and inconsistency among studies. This fact makes this percentage estimation (i.e., 74%) unreliable. Nowadays, the conclusion is that the initial hopes posed on P3-speller for ALS patients have not been met yet. In addition, no trials in which the P3-speller has been compared to current assistive technologies for communication (e.g., eye-trackers) are available. In conclusion, further studies are required to obtain a reliable index of P3-speller effectiveness in ALS. Furthermore, comparisons of P3-speller systems with the available assistive technologies are needed to assess the P3-speller usefulness with non-completely paralyzed ALS-patients.

A multimenu system based on the P300 component as a time saving procedure for communication with a brain-computer interface

The present study investigates a Brain-Computer Interface (BCI) spelling procedure based on the P300 evoked potential. It uses a small matrix of words arranged in a tree-shaped organization ("multimenu"), and allows the user to build phrases one word at a time, instead of letter by letter. Experiments were performed in two sessions on a group of seven healthy volunteers. In the former, the "multimenu" was tested with a total of 60 choices: 30 "externally-imposed" selections and 30 "free-choice" selections. In the latter, 3 × 3 matrices were compared with 6 × 6 matrices. Each matrix was composed of letters or words, for a total of four matrices. Differences in classifier accuracy, bit rate and amplitude of the evoked P300 were evaluated. Average accuracy in all subjects was 87% with no differences between the selection methods. The 3 × 3 "multimenu" obtained the same level of classifier accuracy as the 6 × 6 matrices, even with a significantly lower amplitude of the P300. Bit rate was increased when using the 3 × 3 matrices compared to the 6 × 6 ones. The "multimenu" system was equally effective, but faster than conventional, letter-based matrices. By improving the speed of communication, this method can be of help to patients with severe difficulties in communication.

Amyotrophic lateral sclerosis progression and stability of brain-computer interface communication

Our objective was to investigate the relationship between brain-computer interface (BCI) communication skill and disease progression in amyotrophic lateral sclerosis (ALS). We sought also to assess stability of BCI communication performance over time and whether it is related to the progression of neurological impairment before entering the locked-in state. A three years follow-up, BCI evaluation in a group of ALS patients (n = 24) was conducted. For a variety of reasons only three patients completed the three years follow-up. BCI communication skill and disability level, using the Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised, were assessed at admission and at each of the three follow-ups. Multiple non-parametric statistical methods were used to ensure reliability of the dependent variables: correlations, paired test and factor analysis of variance. Results demonstrated no significant relationship between BCI communication skill (BCI-CS) and disease evolution. The patients who performed the follow-up evaluations preserved their BCI-CS over time. Patients' age at admission correlated positively with the ability to achieve control over a BCI. In conclusion, disease evolution in ALS does not affect the ability to control a BCI for communication. BCI performance can be maintained in the different stages of the illness.

Prediction of auditory and visual p300 brain-computer interface aptitude

OBJECTIVE

Brain-computer interfaces (BCIs) provide a non-muscular communication channel for patients with late-stage motoneuron disease (e.g., amyotrophic lateral sclerosis (ALS)) or otherwise motor impaired people and are also used for motor rehabilitation in chronic stroke. Differences in the ability to use a BCI vary from person to person and from session to session. A reliable predictor of aptitude would allow for the selection of suitable BCI paradigms. For this reason, we investigated whether P300 BCI aptitude could be predicted from a short experiment with a standard auditory oddball.

METHODS

Forty healthy participants performed an electroencephalography (EEG) based visual and auditory P300-BCI spelling task in a single session. In addition, prior to each session an auditory oddball was presented. Features extracted from the auditory oddball were analyzed with respect to predictive power for BCI aptitude.

RESULTS

Correlation between auditory oddball response and P300 BCI accuracy revealed a strong relationship between accuracy and N2 amplitude and the amplitude of a late ERP component between 400 and 600 ms. Interestingly, the P3 amplitude of the auditory oddball response was not correlated with accuracy.

CONCLUSIONS

Event-related potentials recorded during a standard auditory oddball session moderately predict aptitude in an audiory and highly in a visual P300 BCI. The predictor will allow for faster paradigm selection.

SIGNIFICANCE

Our method will reduce strain on patients because unsuccessful training may be avoided, provided the results can be generalized to the patient population.

The changing face of P300 BCIs: a comparison of stimulus changes in a P300 BCI involving faces, emotion, and movement

BACKGROUND

One of the most common types of brain-computer interfaces (BCIs) is called a P300 BCI, since it relies on the P300 and other event-related potentials (ERPs). In the canonical P300 BCI approach, items on a monitor flash briefly to elicit the necessary ERPs. Very recent work has shown that this approach may yield lower performance than alternate paradigms in which the items do not flash but instead change in other ways, such as moving, changing colour or changing to characters overlaid with faces.

METHODOLOGY/PRINCIPAL FINDINGS

The present study sought to extend this research direction by parametrically comparing different ways to change items in a P300 BCI. Healthy subjects used a P300 BCI across six different conditions. Three conditions were similar to our prior work, providing the first direct comparison of characters flashing, moving, and changing to faces. Three new conditions also explored facial motion and emotional expression. The six conditions were compared across objective measures such as classification accuracy and bit rate as well as subjective measures such as perceived difficulty. In line with recent studies, our results indicated that the character flash condition resulted in the lowest accuracy and bit rate. All four face conditions (mean accuracy >91%) yielded significantly better performance than the flash condition (mean accuracy = 75%).

CONCLUSIONS/SIGNIFICANCE

Objective results reaffirmed that the face paradigm is superior to the canonical flash approach that has dominated P300 BCIs for over 20 years. The subjective reports indicated that the conditions that yielded better performance were not considered especially burdensome. Therefore, although further work is needed to identify which face paradigm is best, it is clear that the canonical flash approach should be replaced with a face paradigm when aiming at increasing bit rate. However, the face paradigm has to be further explored with practical applications particularly with locked-in patients.

A portable auditory P300 brain-computer interface with directional cues

OBJECTIVES

The main objective of the current study was to implement and evaluate a P300 based brain-computer interface (BCI) speller that uses directional cues of auditory stimuli, which are presented over headphones. The interstimulus interval (ISI) was successively reduced to determine the optimal combination of speed and accuracy. The study further aimed at quantifying the differences in subjective workload between the auditory and the visual P300 spelling application. The influence of workload, mood and motivation on BCI performance and P300 amplitude was investigated.

METHODS

Twenty healthy participants performed auditory and visual spelling tasks in an EEG experiment with online feedback.

RESULTS

Sixteen of twenty participants performed at or above a level necessary for satisfactory communication (≥70% spelling accuracy) with the auditory BCI. Average bit rates of up to 2.76 bits/min (best subject 7.43 bits/min) were achieved. A significantly higher workload was reported for the auditory speller compared to the visual paradigm. Motivation significantly influenced P300 amplitude at Pz in the auditory condition.

CONCLUSIONS

The results of the online study suggest that the proposed paradigm offers a means of communication for most healthy users.

SIGNIFICANCE

The described auditory BCI can serve as a communication channel for completely paralyzed patients.

P300 brain computer interface: current challenges and emerging trends

A brain-computer interface (BCI) enables communication without movement based on brain signals measured with electroencephalography (EEG). BCIs usually rely on one of three types of signals: the P300 and other components of the event-related potential (ERP), steady state visual evoked potential (SSVEP), or event related desynchronization (ERD). Although P300 BCIs were introduced over twenty years ago, the past few years have seen a strong increase in P300 BCI research. This closed-loop BCI approach relies on the P300 and other components of the ERP, based on an oddball paradigm presented to the subject. In this paper, we overview the current status of P300 BCI technology, and then discuss new directions: paradigms for eliciting P300s; signal processing methods; applications; and hybrid BCIs. We conclude that P300 BCIs are quite promising, as several emerging directions have not yet been fully explored and could lead to improvements in bit rate, reliability, usability, and flexibility.

Comparison of dry and gel based electrodes for p300 brain-computer interfaces

Most brain–computer interfaces (BCIs) rely on one of three types of signals in the electroencephalogram (EEG): P300s, steady-state visually evoked potentials, and event-related desynchronization. EEG is typically recorded non-invasively with electrodes mounted on the human scalp using conductive electrode gel for optimal impedance and data quality. The use of electrode gel entails serious problems that are especially pronounced in real-world settings when experts are not available. Some recent work has introduced dry electrode systems that do not require gel, but often introduce new problems such as comfort and signal quality. The principal goal of this study was to assess a new dry electrode BCI system in a very common task: spelling with a P300 BCI. A total of 23 subjects used a P300 BCI to spell the word “LUCAS” while receiving real-time, closed-loop feedback. The dry system yielded classification accuracies that were similar to those obtained with gel systems. All subjects completed a questionnaire after data recording, and all subjects stated that the dry system was not uncomfortable. This is the first field validation of a dry electrode P300 BCI system, and paves the way for new research and development with EEG recording systems that are much more practical and convenient in field settings than conventional systems.

Listen, You are Writing! Speeding up Online Spelling with a Dynamic Auditory BCI

Representing an intuitive spelling interface for brain-computer interfaces (BCI) in the auditory domain is not straight-forward. In consequence, all existing approaches based on event-related potentials (ERP) rely at least partially on a visual representation of the interface. This online study introduces an auditory spelling interface that eliminates the necessity for such a visualization. In up to two sessions, a group of healthy subjects (N = 21) was asked to use a text entry application, utilizing the spatial cues of the AMUSE paradigm (Auditory Multi-class Spatial ERP). The speller relies on the auditory sense both for stimulation and the core feedback. Without prior BCI experience, 76% of the participants were able to write a full sentence during the first session. By exploiting the advantages of a newly introduced dynamic stopping method, a maximum writing speed of 1.41 char/min (7.55 bits/min) could be reached during the second session (average: 0.94 char/min, 5.26 bits/min). For the first time, the presented work shows that an auditory BCI can reach performances similar to state-of-the-art visual BCIs based on covert attention. These results represent an important step toward a purely auditory BCI.

A brain-computer interface as input channel for a standard assistive technology software

Recently brain-computer interface (BCI) control was integrated into the commercial assistive technology product QualiWORLD (QualiLife Inc., Paradiso-Lugano, CH). Usability of the first prototype was evaluated in terms of effectiveness (accuracy), efficiency (information transfer rate and subjective workload/NASA Task Load Index) and user satisfaction (Quebec User Evaluation of Satisfaction with assistive Technology, QUEST 2.0) by four end-users with severe disabilities. Three assistive technology experts evaluated the device from a third person perspective. The results revealed high performance levels in communication and internet tasks. Users and assistive technology experts were quite satisfied with the device. However, none could imagine using the device in daily life without improvements. Main obstacles were the EEG-cap and low speed.