Combining Brain-Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges

Brain implants for substituting lost motor function: state of the art and potential impact on the lives of motor-impaired seniors

Recent scientific achievements bring the concept of neural prosthetics for reinstating lost motor function closer to medical application. Current research involves severely paralyzed people under the age of 65, but implications for seniors with stroke or trauma-induced impairments are clearly on the horizon. Demographic changes will lead to a shortage of personnel to care for an increasing population of senior citizens, threatening maintenance of an acceptable level of care and urging ways for people to live longer at their home independent from personal assistance. This is particularly challenging when people suffer from disabilities such as partial paralysis after stroke or trauma, where daily personal assistance is required. For some of these people, neural prosthetics can reinstate some lost motor function and/or lost communication, thereby increasing independence and possibly quality of life. In this viewpoint article, we present the state of the art in decoding brain activity in the service of brain-computer interfacing. Although some noninvasive applications produce good results, we focus on brain implants that benefit from better quality brain signals. Fully implantable neural prostheses for home use are not available yet, but clinical trials are being prepared. More sophisticated systems are expected to follow in the years to come, with capabilities of interest for less severe paralysis. Eventually the combination of smart robotics and brain implants is expected to enable people to interact well enough with their environment to live an independent life in spite of motor disabilities.

SCoT: a Python toolbox for EEG source connectivity

Analysis of brain connectivity has become an important research tool in neuroscience. Connectivity can be estimated between cortical sources reconstructed from the electroencephalogram (EEG). Such analysis often relies on trial averaging to obtain reliable results. However, some applications such as brain-computer interfaces (BCIs) require single-trial estimation methods. In this paper, we present SCoT-a source connectivity toolbox for Python. This toolbox implements routines for blind source decomposition and connectivity estimation with the MVARICA approach. Additionally, a novel extension called CSPVARICA is available for labeled data. SCoT estimates connectivity from various spectral measures relying on vector autoregressive (VAR) models. Optionally, these VAR models can be regularized to facilitate ill posed applications such as single-trial fitting. We demonstrate basic usage of SCoT on motor imagery (MI) data. Furthermore, we show simulation results of utilizing SCoT for feature extraction in a BCI application. These results indicate that CSPVARICA and correct regularization can significantly improve MI classification. While SCoT was mainly designed for application in BCIs, it contains useful tools for other areas of neuroscience. SCoT is a software package that (1) brings combined source decomposition and connectivtiy estimation to the open Python platform, and (2) offers tools for single-trial connectivity estimation. The source code is released under the MIT license and is available online at github.com/SCoT-dev/SCoT.

Feasibility of a hybrid brain-computer interface for advanced functional electrical therapy

We present a feasibility study of a novel hybrid brain-computer interface (BCI) system for advanced functional electrical therapy (FET) of grasp. FET procedure is improved with both automated stimulation pattern selection and stimulation triggering. The proposed hybrid BCI comprises the two BCI control signals: steady-state visual evoked potentials (SSVEP) and event-related desynchronization (ERD). The sequence of the two stages, SSVEP-BCI and ERD-BCI, runs in a closed-loop architecture. The first stage, SSVEP-BCI, acts as a selector of electrical stimulation pattern that corresponds to one of the three basic types of grasp: palmar, lateral, or precision. In the second stage, ERD-BCI operates as a brain switch which activates the stimulation pattern selected in the previous stage. The system was tested in 6 healthy subjects who were all able to control the device with accuracy in a range of 0.64–0.96. The results provided the reference data needed for the planned clinical study. This novel BCI may promote further restoration of the impaired motor function by closing the loop between the “will to move” and contingent temporally synchronized sensory feedback.

Toward brain-computer interface based wheelchair control utilizing tactually-evoked event-related potentials

BACKGROUND

People with severe disabilities, e.g., due to neurodegenerative disease, depend on technology that allows for accurate wheelchair control. For those who cannot operate a wheelchair with a joystick, brain-computer interfaces (BCI) may offer a valuable option. Technology depending on visual or auditory input may not be feasible as these modalities are dedicated to processing of environmental stimuli (e.g., recognition of obstacles, ambient noise). Herein we thus validated the feasibility of a BCI based on tactually-evoked event-related potentials (ERP) for wheelchair control. Furthermore, we investigated use of a dynamic stopping method to improve speed of the tactile BCI system.

METHODS

Positions of four tactile stimulators represented navigation directions (left thigh: move left; right thigh: move right; abdomen: move forward; lower neck: move backward) and N = 15 participants delivered navigation commands by focusing their attention on the desired tactile stimulus in an oddball-paradigm.

RESULTS

Participants navigated a virtual wheelchair through a building and eleven participants successfully completed the task of reaching 4 checkpoints in the building. The virtual wheelchair was equipped with simulated shared-control sensors (collision avoidance), yet these sensors were rarely needed.

CONCLUSION

We conclude that most participants achieved tactile ERP-BCI control sufficient to reliably operate a wheelchair and dynamic stopping was of high value for tactile ERP classification. Finally, this paper discusses feasibility of tactile ERPs for BCI based wheelchair control.

Noninvasive brain-computer interfaces for augmentative and alternative communication

Brain-computer interfaces (BCIs) promise to provide a novel access channel for assistive technologies, including augmentative and alternative communication (AAC) systems, to people with severe speech and physical impairments (SSPI). Research on the subject has been accelerating significantly in the last decade and the research community took great strides toward making BCI-AAC a practical reality to individuals with SSPI. Nevertheless, the end goal has still not been reached and there is much work to be done to produce real-world-worthy systems that can be comfortably, conveniently, and reliably used by individuals with SSPI with help from their families and care givers who will need to maintain, setup, and debug the systems at home. This paper reviews reports in the BCI field that aim at AAC as the application domain with a consideration on both technical and clinical aspects.

Workshops of the Fifth International Brain-Computer Interface Meeting: Defining the Future

The Fifth International Brain-Computer Interface (BCI) Meeting met June 3-7th, 2013 at the Asilomar Conference Grounds, Pacific Grove, California. The conference included 19 workshops covering topics in brain-computer interface and brain-machine interface research. Topics included translation of BCIs into clinical use, standardization and certification, types of brain activity to use for BCI, recording methods, the effects of plasticity, special interest topics in BCIs applications, and future BCI directions. BCI research is well established and transitioning to practical use to benefit people with physical impairments. At the same time, new applications are being explored, both for people with physical impairments and beyond. Here we provide summaries of each workshop, illustrating the breadth and depth of BCI research and high-lighting important issues for future research and development.

Brain connectivity in continuous error tasks

Error-related potentials (ErrP) have been recently incorporated in brain-machine interfaces (BMIs) due to its ability to adapt and correct both the output of the BMI or the behavior of the machine. Most of these applications rely on synchronous tasks with different user's evaluations associated to correct and wrong events. Asynchronous detection during the continuous evaluation of the task, however, has to cope with background noise and an increased number of misdetections common in event-related potential detection. This paper studies a different characteristic that may carry additional information to be exploited by asynchronous ErrP detectors: brain connectivity coherence patterns appearing while the user monitors the continuous operation of a device. The results obtained with five subject revealed the presence of an error potential in an asynchronous reaching task an showed an increase in the coherency within the theta band.

Towards effective non-invasive brain-computer interfaces dedicated to gait rehabilitation systems

In the last few years, significant progress has been made in the field of walk rehabilitation. Motor cortex signals in bipedal monkeys have been interpreted to predict walk kinematics. Epidural electrical stimulation in rats and in one young paraplegic has been realized to partially restore motor control after spinal cord injury. However, these experimental trials are far from being applicable to all patients suffering from motor impairments. Therefore, it is thought that more simple rehabilitation systems are desirable in the meanwhile. The goal of this review is to describe and summarize the progress made in the development of non-invasive brain-computer interfaces dedicated to motor rehabilitation systems. In the first part, the main principles of human locomotion control are presented. The paper then focuses on the mechanisms of supra-spinal centers active during gait, including results from electroencephalography, functional brain imaging technologies [near-infrared spectroscopy (NIRS), functional magnetic resonance imaging (fMRI), positron-emission tomography (PET), single-photon emission-computed tomography (SPECT)] and invasive studies. The first brain-computer interface (BCI) applications to gait rehabilitation are then presented, with a discussion about the different strategies developed in the field. The challenges to raise for future systems are identified and discussed. Finally, we present some proposals to address these challenges, in order to contribute to the improvement of BCI for gait rehabilitation.

Demonstration of a semi-autonomous hybrid brain-machine interface using human intracranial EEG, eye tracking, and computer vision to control a robotic upper limb prosthetic

To increase the ability of brain-machine interfaces (BMIs) to control advanced prostheses such as the modular prosthetic limb (MPL), we are developing a novel system: the Hybrid Augmented Reality Multimodal Operation Neural Integration Environment (HARMONIE). This system utilizes hybrid input, supervisory control, and intelligent robotics to allow users to identify an object (via eye tracking and computer vision) and initiate (via brain-control) a semi-autonomous reach-grasp-and-drop of the object by the MPL. Sequential iterations of HARMONIE were tested in two pilot subjects implanted with electrocorticographic (ECoG) and depth electrodes within motor areas. The subjects performed the complex task in 71.4% (20/28) and 67.7% (21/31) of trials after minimal training. Balanced accuracy for detecting movements was 91.1% and 92.9%, significantly greater than chance accuracies (p < 0.05). After BMI-based initiation, the MPL completed the entire task 100% (one object) and 70% (three objects) of the time. The MPL took approximately 12.2 s for task completion after system improvements implemented for the second subject. Our hybrid-BMI design prevented all but one baseline false positive from initiating the system. The novel approach demonstrated in this proof-of-principle study, using hybrid input, supervisory control, and intelligent robotics, addresses limitations of current BMIs.

Offline analysis of context contribution to ERP-based typing BCI performance

OBJECTIVE

We aim to increase the symbol rate of electroencephalography (EEG) based brain-computer interface (BCI) typing systems by utilizing context information.

APPROACH

Event related potentials (ERP) corresponding to a stimulus in EEG can be used to detect the intended target of a person for BCI. This paradigm is widely utilized to build letter-by-letter BCI typing systems. Nevertheless currently available BCI typing systems still require improvement due to low typing speeds. This is mainly due to the reliance on multiple repetitions before making a decision to achieve higher typing accuracy. Another possible approach to increase the speed of typing while not significantly reducing the accuracy of typing is to use additional context information. In this paper, we study the effect of using a language model (LM) as additional evidence for intent detection. Bayesian fusion of an n-gram symbol model with EEG features is proposed, and a specifically regularized discriminant analysis ERP discriminant is used to obtain EEG-based features. The target detection accuracies are rigorously evaluated for varying LM orders, as well as the number of ERP-inducing repetitions.

MAIN RESULTS

The results demonstrate that the LMs contribute significantly to letter classification accuracy. For instance, we find that a single-trial ERP detection supported by a 4-gram LM may achieve the same performance as using 3-trial ERP classification for the non-initial letters of words.

SIGNIFICANCE

Overall, the fusion of evidence from EEG and LMs yields a significant opportunity to increase the symbol rate of a BCI typing system.

A low-cost EEG system-based hybrid brain-computer interface for humanoid robot navigation and recognition

This paper describes a hybrid brain-computer interface (BCI) technique that combines the P300 potential, the steady state visually evoked potential (SSVEP), and event related de-synchronization (ERD) to solve a complicated multi-task problem consisting of humanoid robot navigation and control along with object recognition using a low-cost BCI system. Our approach enables subjects to control the navigation and exploration of a humanoid robot and recognize a desired object among candidates. This study aims to demonstrate the possibility of a hybrid BCI based on a low-cost system for a realistic and complex task. It also shows that the use of a simple image processing technique, combined with BCI, can further aid in making these complex tasks simpler. An experimental scenario is proposed in which a subject remotely controls a humanoid robot in a properly sized maze. The subject sees what the surrogate robot sees through visual feedback and can navigate the surrogate robot. While navigating, the robot encounters objects located in the maze. It then recognizes if the encountered object is of interest to the subject. The subject communicates with the robot through SSVEP and ERD-based BCIs to navigate and explore with the robot, and P300-based BCI to allow the surrogate robot recognize their favorites. Using several evaluation metrics, the performances of five subjects navigating the robot were quite comparable to manual keyboard control. During object recognition mode, favorite objects were successfully selected from two to four choices. Subjects conducted humanoid navigation and recognition tasks as if they embodied the robot. Analysis of the data supports the potential usefulness of the proposed hybrid BCI system for extended applications. This work presents an important implication for the future work that a hybridization of simple BCI protocols provide extended controllability to carry out complicated tasks even with a low-cost system.

Continuous decoding of motor attempt and motor imagery from EEG activity in spinal cord injury patients

Spinal cord injury (SCI) associates brain reorganization with a loss of cortical representation of paralyzed limbs. This effect is more pronounced in the chronic state, which can be reached approximately 6 months after the lesion. As many of the brain-computer interfaces (BCI) developed to date rely on the user motor activity, loss of this activity hinders the application of BCI technology for rehabilitation or motor compensation in these patients. This work is a preliminary study with three quadriplegic patients close to reaching the chronic state, addressing two questions: (i) whether it is still possible to use BCI technology to detect motor intention of the paralyzed hand at this state of chronicity; and (ii) whether it is better for the BCI decoding to rely on the motor attempt or the motor imagery of the hand as mental paradigm. The results show that one of the three patients had already lost the motor programs related to the hand, so it was not possible to build a motor-related BCI for him. For the other patients it was suitable to design a BCI based on both paradigms, but the results were better using motor attempt as it has broader activation associated patterns that are easier to recognize.

Upper alpha neurofeedback training over the motor cortex increases SMR desynchronization in motor tasks

Desynchronization of sensorimotor rhythms (SMR) is a distinctive feature that provides a discriminative pattern for BCI operation. However, individuals such as BCI illiterates can not produce these discriminable patterns with sufficient reliability. Additionally, SMR desynchronization can become deteriorated or extinct in patients with spinal cord injury or a cerebrovascular accident. In all these situations BCI usage is compromised. This paper proposes an intervention based on neurofeedback training of the upper alpha band to improve SMR desynchronization. The feasibility of this intervention is demonstrated in a preliminary study in which five healthy subjects were trained to increase their upper alpha band power. Such increases produced higher SMR desynchronization and better discrimination between rest and execution states of a motor task.

Performance of the Emotiv Epoc headset for P300-based applications

Background

For two decades, EEG-based Brain-Computer Interface (BCI) systems have been widely studied in research labs. Now, researchers want to consider out-of-the-lab applications and make this technology available to everybody. However, medical-grade EEG recording devices are still much too expensive for end-users, especially disabled people. Therefore, several low-cost alternatives have appeared on the market. The Emotiv Epoc headset is one of them. Although some previous work showed this device could suit the customer’s needs in terms of performance, no quantitative classification-based assessments compared to a medical system are available.

Methods

This paper aims at statistically comparing a medical-grade system, the ANT device, and the Emotiv Epoc headset by determining their respective performances in a P300 BCI using the same electrodes. On top of that, a review of previous Emotiv studies and a discussion on practical considerations regarding both systems are proposed. Nine healthy subjects participated in this experiment during which the ANT and the Emotiv systems are used in two different conditions: sitting on a chair and walking on a treadmill at constant speed.

Results

The Emotiv headset performs significantly worse than the medical device; observed effect sizes vary from medium to large. The Emotiv headset has higher relative operational and maintenance costs than its medical-grade competitor.

Conclusions

Although this low-cost headset is able to record EEG data in a satisfying manner, it should only be chosen for non critical applications such as games, communication systems, etc. For rehabilitation or prosthesis control, this lack of reliability may lead to serious consequences. For research purposes, the medical system should be chosen except if a lot of trials are available or when the Signal-to-Noise Ratio is high. This also suggests that the design of a specific low-cost EEG recording system for critical applications and research is still required.

Investigating brief motor imagery for an ERD/ERS based BCI

This study establishes the effectiveness of event related synchronisation (ERS) features for a system paced brain computer interface (BCI). In particular, the relationship between the duration of motor imagery (MI) and the quality of the features extracted from the ERS is investigated. To this end, two groups of users performed brief (2s) or sustained (4s) MI, and offline single trial BCIs were validated on each group based on features extracted from the EEG before, during and after MI. The BCIs were designed to recognise two intentional control tasks and a no-control state. Cross-validated results indicate that brief MI leads to more informative ERS features than sustained MI.

Effects of performing two visual tasks on single-trial detection of event-related potentials

The detection of event-related potentials (ERPs) in brain-computer interface (BCI) depends on the ability of the subject to pay attention to specific stimuli presented during the BCI task. For healthy users, a BCI shall be used as a complement to other existing devices, which involve the response to other tasks. Those tasks may impair selective attention, particularly if the stimuli have the same modality e.g. visual. It is therefore critical to analyze how single-trial detection of brain evoked response is impaired by the addition of tasks concerning the same modality. We tested 10 healthy participants using an application that has two visual target detection tasks. The first one corresponds to a rapid serial visual presentation paradigm where target detection is achieved by brain-evoked single-trial detection in the recorded electroencephalogram (EEG) signal. The second task is the detection of a visual event on a tactical map by a behavioral response. These tasks were tested individually (single task) and in parallel (dual-task). Whereas the performance of single-trial detection was not impaired between single and dual-task conditions, the behavioral performance decreased during the dual-task condition. These results quantify the performance drop that can occur in a dual-task system using both brain-evoked responses and behavioral responses.

Using frequency-domain features for the generalization of EEG error-related potentials among different tasks

EEG brain-computer interfaces (BCI) require a calibration phase prior to the on-line control of the device, which is a difficulty for the practical development of this technology as it is user-, session- and task-specific. The large body of research in BCIs based on event-related potentials (ERP) use temporal features, which have demonstrated to be stable for each user along time, but do not generalize well among tasks different from the calibration task. This paper explores the use of low frequency features to improve the generalization capabilities of the BCIs using error-potentials. The results show that there exists a stable pattern in the frequency domain that allows a classifier to generalize among the tasks. Furthermore, the study also shows that it is possible to combine temporal and frequency features to obtain the best of both domains.

Toward binary brain computer interface using steady-state visually evoked potential under eyes closed condition

It is highly difficult for severely amyotrophic lateral sclerosis and heavily spinal cord injury patients to use the brain computer interfaces (BCIs) based on the steady-state visual evoked potential (SSVEP) which need to control the direction of their eye gaze. We investigated amplitude change of the SSVEP associated with mental concentration on flicker to develop the SSVEP-based BCI usable under eyes-closed condition. Under the stimulus conditions of the flickering frequency of 10 Hz and the stimulus intensity of 5 lx, significant difference between the SSVEP amplitude in relaxed state and that in concentrated state was observed in the wide region of the scalp except the left frontal region, while such significance was also seen in the bilateral occipital lobes and left parietal region under the conditions of 14 Hz and 5 lx. Such an impact of mental concentration on the SSVEP amplitude was reproducible.

Hybrid EEG and eye movement interface to multi-directional target selection

This work addresses the development of a low-cost hybrid interface with eye tracking and brain signals. Eye movement detection is used for search task and EEG-based brain computer interface (BCI) for selection task. Multi-directional target selection experiments with the hybrid interface device were conducted with five subjects to evaluate the proposed hybrid interface scheme. The task asked each user to move a cursor onto a circular target among twelve possible positions and select it. Using the Fitts' law, the interface performance was compared with the computer mouse. With two BCI selection confirmation schemes, the hybrid interface attained 2-2.7 bit/s overall. Based on the results, the potential of the proposed hybrid interface was discussed.

P300-based brain-neuronal computer interaction for spelling applications

A brain-neuronal computer interaction system can provide a communication channel for severely disabled people or a supplementary control channel for able-bodied subjects. In this paper, a physiological hybrid P300-based speller that uses a modified stimulus presentation paradigm-the half checkerboard paradigm (HCBP)-is evaluated. The speller uses electrooculography and electroencephalography signals for selecting alphanumeric characters or commands arranged in an 8 × 9 matrix. In this study a group of subjects, who can voluntarily gaze at a target, used the checkerboard paradigm- and HCBP-based spellers in a counterbalanced fashion for comparing their performances under a series of online tests. A 16-character-long text was spelled by each subject, while a 13-character-long text was used for calibrating the system. By using the HCBP, the time required for spelling one character is reduced, resulting in higher information transfer rates. The results suggest that the HCBP has the potential to provide a more effective P300 paradigm with a major importance for people with neuromuscular diseases and also for healthy people as a supplementary communication channel.

A wireless multichannel EEG recording platform

A wireless multichannel data acquisition system is being designed for ElectroEncephaloGraphy (EEG) recording. The system is based on a custom integrated circuit (ASIC) for signal conditioning, amplification and digitization and also on commercial components for RF transmission. It supports the RF transmission of a 32-channel EEG recording sampled at 1 kHz with a 12-bit resolution. The RF communication uses the MICS band (Medical Implant Communication Service) at 402-405 Mhz. This integration is a first step towards a lightweight EEG cap for Brain Computer Interface (BCI) studies. Here, we present the platform architecture and its submodules. In vivo validations are presented with noise characterization and wireless data transfer measurements.

Review of the BCI Competition IV

The BCI competition IV stands in the tradition of prior BCI competitions that aim to provide high quality neuroscientific data for open access to the scientific community. As experienced already in prior competitions not only scientists from the narrow field of BCI compete, but scholars with a broad variety of backgrounds and nationalities. They include high specialists as well as students. The goals of all BCI competitions have always been to challenge with respect to novel paradigms and complex data. We report on the following challenges: (1) asynchronous data, (2) synthetic, (3) multi-class continuous data, (4) session-to-session transfer, (5) directionally modulated MEG, (6) finger movements recorded by ECoG. As after past competitions, our hope is that winning entries may enhance the analysis methods of future BCIs.

Stop-event-related potentials from intracranial electrodes reveal a key role of premotor and motor cortices in stopping ongoing movements

In humans, the ability to withhold manual motor responses seems to rely on a right-lateralized frontal–basal ganglia–thalamic network, including the pre-supplementary motor area and the inferior frontal gyrus (IFG). These areas should drive subthalamic nuclei to implement movement inhibition via the hyperdirect pathway. The output of this network is expected to influence those cortical areas underlying limb movement preparation and initiation, i.e., premotor (PMA) and primary motor (M1) cortices. Electroencephalographic (EEG) studies have shown an enhancement of the N200/P300 complex in the event-related potentials (ERPs) when a planned reaching movement is successfully stopped after the presentation of an infrequent stop-signal. PMA and M1 have been suggested as possible neural sources of this ERP complex but, due to the limited spatial resolution of scalp EEG, it is not yet clear which cortical areas contribute to its generation. To elucidate the role of motor cortices, we recorded epicortical ERPs from the lateral surface of the fronto-temporal lobes of five pharmacoresistant epileptic patients performing a reaching version of the countermanding task while undergoing presurgical monitoring. We consistently found a stereotyped ERP complex on a single-trial level when a movement was successfully cancelled. These ERPs were selectively expressed in M1, PMA, and Brodmann's area (BA) 9 and their onsets preceded the end of the stop process, suggesting a causal involvement in this executive function. Such ERPs also occurred in unsuccessful-stop (US) trials, that is, when subjects moved despite the occurrence of a stop-signal, mostly when they had long reaction times (RTs). These findings support the hypothesis that motor cortices are the final target of the inhibitory command elaborated by the frontal–basal ganglia–thalamic network.

Command detection and classification in tongue drive assistive technology

Tongue Drive System (TDS) is a new assistive technology that enables individuals with severe disabilities such as those with spinal cord injury (SCI) to regain environmental control using their tongue motion. We have developed a new sensor signal processing (SSP) algorithm which uses four 3-axial magneto-resistive sensor outputs to accurately detect and classify between seven different user-control commands in stationary as well as mobile conditions. The new algorithm employs a two-stage classification method with a combination of 9 classifiers to discriminate between 4 commands on the left or right side of the oral cavity (one neutral command shared on both sides). Evaluation of the new SSP algorithm on five able-bodied subjects resulted in true positive rates in the range of 70-99% with corresponding false positive rates in the range of 5-7%, showing a notable improvement in the resulted true-false (TF) differences when compared to the previous algorithm.

A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair

Brain-computer interfaces (BCIs) are used to translate brain activity signals into control signals for external devices. Currently, it is difficult for BCI systems to provide the multiple independent control signals necessary for the multi-degree continuous control of a wheelchair. In this paper, we address this challenge by introducing a hybrid BCI that uses the motor imagery-based mu rhythm and the P300 potential to control a brain-actuated simulated or real wheelchair. The objective of the hybrid BCI is to provide a greater number of commands with increased accuracy to the BCI user. Our paradigm allows the user to control the direction (left or right turn) of the simulated or real wheelchair using left- or right-hand imagery. Furthermore, a hybrid manner can be used to control speed. To decelerate, the user imagines foot movement while ignoring the flashing buttons on the graphical user interface (GUI). If the user wishes to accelerate, then he/she pays attention to a specific flashing button without performing any motor imagery. Two experiments were conducted to assess the BCI control; both a simulated wheelchair in a virtual environment and a real wheelchair were tested. Subjects steered both the simulated and real wheelchairs effectively by controlling the direction and speed with our hybrid BCI system. Data analysis validated the use of our hybrid BCI system to control the direction and speed of a wheelchair.

Decoding with limited neural data: a mixture of time-warped trajectory models for directional reaches

Neuroprosthetic devices promise to allow paralyzed patients to perform the necessary functions of everyday life. However, to allow patients to use such tools it is necessary to decode their intent from neural signals such as electromyograms (EMGs). Because these signals are noisy, state of the art decoders integrate information over time. One systematic way of doing this is by taking into account the natural evolution of the state of the body--by using a so-called trajectory model. Here we use two insights about movements to enhance our trajectory model: (1) at any given time, there is a small set of likely movement targets, potentially identified by gaze; (2) reaches are produced at varying speeds. We decoded natural reaching movements using EMGs of muscles that might be available from an individual with spinal cord injury. Target estimates found from tracking eye movements were incorporated into the trajectory model, while a mixture model accounted for the inherent uncertainty in these estimates. Warping the trajectory model in time using a continuous estimate of the reach speed enabled accurate decoding of faster reaches. We found that the choice of richer trajectory models, such as those incorporating target or speed, improves decoding particularly when there is a small number of EMGs available.

Recent trends in assistive technology for mobility

Loss of physical mobility makes maximal participation in desired activities more difficult and in the worst case fully prevents participation. This paper surveys recent work in assistive technology to improve mobility for persons with a disability, drawing on examples observed during a tour of academic and industrial research sites in Europe. The underlying theme of this recent work is a more seamless integration of the capabilities of the user and the assistive technology. This improved integration spans diverse technologies, including powered wheelchairs, prosthetic limbs, functional electrical stimulation, and wearable exoskeletons. Improved integration is being accomplished in three ways: 1) improving the assistive technology mechanics; 2) improving the user-technology physical interface; and 3) sharing of control between the user and the technology. We provide an overview of these improvements in user-technology integration and discuss whether such improvements have the potential to be transformative for people with mobility impairments.

From spinal central pattern generators to cortical network: integrated BCI for walking rehabilitation

Success in locomotor rehabilitation programs can be improved with the use of brain-computer interfaces (BCIs). Although a wealth of research has demonstrated that locomotion is largely controlled by spinal mechanisms, the brain is of utmost importance in monitoring locomotor patterns and therefore contains information regarding central pattern generation functioning. In addition, there is also a tight coordination between the upper and lower limbs, which can also be useful in controlling locomotion. The current paper critically investigates different approaches that are applicable to this field: the use of electroencephalogram (EEG), upper limb electromyogram (EMG), or a hybrid of the two neurophysiological signals to control assistive exoskeletons used in locomotion based on programmable central pattern generators (PCPGs) or dynamic recurrent neural networks (DRNNs). Plantar surface tactile stimulation devices combined with virtual reality may provide the sensation of walking while in a supine position for use of training brain signals generated during locomotion. These methods may exploit mechanisms of brain plasticity and assist in the neurorehabilitation of gait in a variety of clinical conditions, including stroke, spinal trauma, multiple sclerosis, and cerebral palsy.

Latency correction of error potentials between different experiments reduces calibration time for single-trial classification

One fundamental limitation of EEG-based brain-computer interfaces is the time needed to calibrate the system prior to the detection of signals, due to the wide variety of issues affecting the EEG measurements. For event-related potentials (ERP), one of these sources of variability is the application performed: Protocols with different cognitive workloads might yield to different latencies of the ERPs. In this sense, it is still not clear the effect that these latency variations have on the single-trial classification. This work studies the differences in the latencies of error potentials across three experiments with increasing cognitive workloads. A delay-correction algorithm based on the cross-correlation of the averaged signals is presented, and tested with a single-trial classification of the signals. The results showed that latency variations exist between different protocols, and that it is feasible to re-use data from previous experiments to calibrate a classifier able to detect the signals of a new experiment, thus reducing the calibration time.

A telepresence mobile robot controlled with a noninvasive brain-computer interface

This paper reports an electroencephalogram-based brain-actuated telepresence system to provide a user with presence in remote environments through a mobile robot, with access to the Internet. This system relies on a P300-based brain-computer interface (BCI) and a mobile robot with autonomous navigation and camera orientation capabilities. The shared-control strategy is built by the BCI decoding of task-related orders (selection of visible target destinations or exploration areas), which can be autonomously executed by the robot. The system was evaluated using five healthy participants in two consecutive steps: 1) screening and training of participants and 2) preestablished navigation and visual exploration telepresence tasks. On the basis of the results, the following evaluation studies are reported: 1) technical evaluation of the device and its main functionalities and 2) the users' behavior study. The overall result was that all participants were able to complete the designed tasks, reporting no failures, which shows the robustness of the system and its feasibility to solve tasks in real settings where joint navigation and visual exploration were needed. Furthermore, the participants showed great adaptation to the telepresence system.

Tools for Brain-Computer Interaction: A General Concept for a Hybrid BCI

The aim of this work is to present the development of a hybrid Brain-Computer Interface (hBCI) which combines existing input devices with a BCI. Thereby, the BCI should be available if the user wishes to extend the types of inputs available to an assistive technology system, but the user can also choose not to use the BCI at all; the BCI is active in the background. The hBCI might decide on the one hand which input channel(s) offer the most reliable signal(s) and switch between input channels to improve information transfer rate, usability, or other factors, or on the other hand fuse various input channels. One major goal therefore is to bring the BCI technology to a level where it can be used in a maximum number of scenarios in a simple way. To achieve this, it is of great importance that the hBCI is able to operate reliably for long periods, recognizing and adapting to changes as it does so. This goal is only possible if many different subsystems in the hBCI can work together. Since one research institute alone cannot provide such different functionality, collaboration between institutes is necessary. To allow for such a collaboration, a new concept and common software framework is introduced. It consists of four interfaces connecting the classical BCI modules: signal acquisition, preprocessing, feature extraction, classification, and the application. But it provides also the concept of fusion and shared control. In a proof of concept, the functionality of the proposed system was demonstrated.

A Novel 9-Class Auditory ERP Paradigm Driving a Predictive Text Entry System

Brain–computer interfaces (BCIs) based on event related potentials (ERPs) strive for offering communication pathways which are independent of muscle activity. While most visual ERP-based BCI paradigms require good control of the user's gaze direction, auditory BCI paradigms overcome this restriction. The present work proposes a novel approach using auditory evoked potentials for the example of a multiclass text spelling application. To control the ERP speller, BCI users focus their attention to two-dimensional auditory stimuli that vary in both, pitch (high/medium/low) and direction (left/middle/right) and that are presented via headphones. The resulting nine different control signals are exploited to drive a predictive text entry system. It enables the user to spell a letter by a single nine-class decision plus two additional decisions to confirm a spelled word. This paradigm – called PASS2D – was investigated in an online study with 12 healthy participants. Users spelled with more than 0.8 characters per minute on average (3.4 bits/min) which makes PASS2D a competitive method. It could enrich the toolbox of existing ERP paradigms for BCI end users like people with amyotrophic lateral sclerosis disease in a late stage.

The Asilomar Survey: Stakeholders' Opinions on Ethical Issues Related to Brain-Computer Interfacing

Brain-Computer Interface (BCI) research and (future) applications raise important ethical issues that need to be addressed to promote societal acceptance and adequate policies. Here we report on a survey we conducted among 145 BCI researchers at the 4^th International BCI conference, which took place in May-June 2010 in Asilomar, California. We assessed respondents' opinions about a number of topics. First, we investigated preferences for terminology and definitions relating to BCIs. Second, we assessed respondents' expectations on the marketability of different BCI applications (BCIs for healthy people, BCIs for assistive technology, BCIs-controlled neuroprostheses and BCIs as therapy tools). Third, we investigated opinions about ethical issues related to BCI research for the development of assistive technology: informed consent process with locked-in patients, risk-benefit analyses, team responsibility, consequences of BCI on patients' and families' lives, liability and personal identity and interaction with the media. Finally, we asked respondents which issues are urgent in BCI research.

Automatic classification of artifactual ICA-components for artifact removal in EEG signals

BACKGROUND

Artifacts contained in EEG recordings hamper both, the visual interpretation by experts as well as the algorithmic processing and analysis (e.g. for Brain-Computer Interfaces (BCI) or for Mental State Monitoring). While hand-optimized selection of source components derived from Independent Component Analysis (ICA) to clean EEG data is widespread, the field could greatly profit from automated solutions based on Machine Learning methods. Existing ICA-based removal strategies depend on explicit recordings of an individual's artifacts or have not been shown to reliably identify muscle artifacts.

METHODS

We propose an automatic method for the classification of general artifactual source components. They are estimated by TDSEP, an ICA method that takes temporal correlations into account. The linear classifier is based on an optimized feature subset determined by a Linear Programming Machine (LPM). The subset is composed of features from the frequency-, the spatial- and temporal domain. A subject independent classifier was trained on 640 TDSEP components (reaction time (RT) study, n = 12) that were hand labeled by experts as artifactual or brain sources and tested on 1080 new components of RT data of the same study. Generalization was tested on new data from two studies (auditory Event Related Potential (ERP) paradigm, n = 18; motor imagery BCI paradigm, n = 80) that used data with different channel setups and from new subjects.

RESULTS

Based on six features only, the optimized linear classifier performed on level with the inter-expert disagreement (<10% Mean Squared Error (MSE)) on the RT data. On data of the auditory ERP study, the same pre-calculated classifier generalized well and achieved 15% MSE. On data of the motor imagery paradigm, we demonstrate that the discriminant information used for BCI is preserved when removing up to 60% of the most artifactual source components.

CONCLUSIONS

We propose a universal and efficient classifier of ICA components for the subject independent removal of artifacts from EEG data. Based on linear methods, it is applicable for different electrode placements and supports the introspection of results. Trained on expert ratings of large data sets, it is not restricted to the detection of eye- and muscle artifacts. Its performance and generalization ability is demonstrated on data of different EEG studies.

Brain-computer interface based on generation of visual images

This paper examines the task of recognizing EEG patterns that correspond to performing three mental tasks: relaxation and imagining of two types of pictures: faces and houses. The experiments were performed using two EEG headsets: BrainProducts ActiCap and Emotiv EPOC. The Emotiv headset becomes widely used in consumer BCI application allowing for conducting large-scale EEG experiments in the future. Since classification accuracy significantly exceeded the level of random classification during the first three days of the experiment with EPOC headset, a control experiment was performed on the fourth day using ActiCap. The control experiment has shown that utilization of high-quality research equipment can enhance classification accuracy (up to 68% in some subjects) and that the accuracy is independent of the presence of EEG artifacts related to blinking and eye movement. This study also shows that computationally-inexpensive bayesian classifier based on covariance matrix analysis yields similar classification accuracy in this problem as a more sophisticated Multi-class Common Spatial Patterns (MCSP) classifier.

BCI demographics II: how many (and what kinds of) people can use a high-frequency SSVEP BCI?

Brain-computer interface (BCI) systems use brain activity as an input signal and enable communication without movement. This study is a successor of our previous study (BCI demographics I) and examines correlations among BCI performance, personal preferences, and different subject factors such as age or gender for two sets of steady-state visual evoked potential (SSVEP) stimuli: one in the medium frequency range (13, 14, 15 and 16 Hz) and another in the high-frequency range (34, 36, 38, 40 Hz). High-frequency SSVEPs (above 30 Hz) diminish user fatigue and risk of photosensitive epileptic seizures. Results showed that most people, despite having no prior BCI experience, could use the SSVEP-based Bremen-BCI system in a very noisy field setting at a fair. Results showed that demographic parameters as well as handedness, tiredness, alcohol and caffeine consumption, etc., have no significant effect on the performance of SSVEP-based BCI. Most subjects did not consider the flickering stimuli annoying, only five out of total 86 participants indicated change in fatigue during the experiment. 84 subjects performed with a mean information transfer rate of 17.24 ±6.99 bit/min and an accuracy of 92.26 ±7.82% with the medium frequency set, whereas only 56 subjects performed with a mean information transfer rate of 12.10 ±7.31 bit/min and accuracy of 89.16 ±9.29% with the high-frequency set. These and other demographic analyses may help identify the best BCI for each user.

Multimodal fusion of muscle and brain signals for a hybrid-BCI

Practical Brain-Computer Interfaces (BCIs) for disabled people should allow them to use all their remaining functionalities as control possibilities. Sometimes these people have residual activity of their muscles, most likely in the morning when they are not exhausted. In this work we fuse electromyographic (EMG) with electroencephalographic (EEG) activity in the framework of a so called "Hybrid-BCI" (hBCI) approach. Thereby, subjects could achieve a good control of their hBCI independently of their level of muscular fatigue. Furthermore, although EMG alone yields good performance, it is outperformed by the hybrid fusing of EEG and EMG. Two different fusion techniques are explored showing graceful performance degradation in the case of signal attenuation. Such a system allows a very reliable control and a smooth handover if the subjects get exhausted or fatigued during the day.