Brain–computer interface and semantic classical conditioning of communication in paralysis

Brain-computer interfaces and human factors: the role of language and cultural differences-Still a missing gap?

Brain-computer interfaces (BCIs) aim at the non-invasive investigation of brain activity for supporting communication and interaction of the users with their environment by means of brain-machine assisted technologies. Despite technological progress and promising research aimed at understanding the influence of human factors on BCI effectiveness, some topics still remain unexplored. The aim of this article is to discuss why it is important to consider the language of the user, its embodied grounding in perception, action and emotions, and its interaction with cultural differences in information processing in future BCI research. Based on evidence from recent studies, it is proposed that detection of language abilities and language training are two main topics of enquiry of future BCI studies to extend communication among vulnerable and healthy BCI users from bench to bedside and real world applications. In addition, cultural differences shape perception, actions, cognition, language and emotions subjectively, behaviorally as well as neuronally. Therefore, BCI applications should consider cultural differences in information processing to develop culture- and language-sensitive BCI applications for different user groups and BCIs, and investigate the linguistic and cultural contexts in which the BCI will be used.

Optical brain imaging and its application to neurofeedback

Besides passive recording of brain electric or magnetic activity, also non-ionizing electromagnetic or optical radiation can be used for real-time brain imaging. Here, changes in the radiation's absorption or scattering allow for continuous in vivo assessment of regional neurometabolic and neurovascular activity. Besides magnetic resonance imaging (MRI), over the last years, also functional near-infrared spectroscopy (fNIRS) was successfully established in real-time metabolic brain imaging. In contrast to MRI, fNIRS is portable and can be applied at bedside or in everyday life environments, e.g., to restore communication and movement. Here we provide a comprehensive overview of the history and state-of-the-art of real-time optical brain imaging with a special emphasis on its clinical use towards neurofeedback and brain-computer interface (BCI) applications. Besides pointing to the most critical challenges in clinical use, also novel approaches that combine real-time optical neuroimaging with other recording modalities (e.g. electro- or magnetoencephalography) are described, and their use in the context of neuroergonomics, neuroenhancement or neuroadaptive systems discussed.

Covert Intention to Answer "Yes" or "No" Can Be Decoded from Single-Trial Electroencephalograms (EEGs)

Interpersonal communication is based on questions and answers, and the most useful and simplest case is the binary “yes or no” question and answer. The purpose of this study is to show that it is possible to decode intentions on “yes” or “no” answers from multichannel single-trial electroencephalograms, which were recorded while covertly answering to self-referential questions with either “yes” or “no.” The intention decoding algorithm consists of a common spatial pattern and support vector machine, which are employed for the feature extraction and pattern classification, respectively, after dividing the overall time-frequency range into subwindows of 200 ms × 2 Hz. The decoding accuracy using the information within each subwindow was investigated to find useful temporal and spectral ranges and found to be the highest for 800–1200 ms in the alpha band or 200–400 ms in the theta band. When the features from multiple subwindows were utilized together, the accuracy was significantly increased up to ∼86%. The most useful features for the “yes/no” discrimination was found to be focused in the right frontal region in the theta band and right centroparietal region in the alpha band, which may reflect the violation of autobiographic facts and higher cognitive load for “no” compared to “yes.” Our task requires the subjects to answer self-referential questions just as in interpersonal conversation without any self-regulation of the brain signals or high cognitive efforts, and the “yes” and “no” answers are decoded directly from the brain activities. This implies that the “mind reading” in a true sense is feasible. Beyond its contribution in fundamental understanding of the neural mechanism of human intention, the decoding of “yes” or “no” from brain activities may eventually lead to a natural brain-computer interface.

Semantic and BCI-performance in completely paralyzed patients: Possibility of language attrition in completely locked in syndrome

Patients with completely locked-in syndrome (CLIS) are incapable of any voluntary muscle movement and do not have any means of communication. Recently functional near infrared spectroscopy (fNIRS) based brain computer interface (BCI) has been successfully used to enable communication with these patients. The developed fNIRS-BCI system relies on the intactness of language comprehension in these patients in all dimensions of language. Interwoven language and motor cortex in brain, and lack of muscular activity in long run, can cause language attrition due to complete immobility in CLIS patients. In this study we have investigated effects of semantic content of sentences presented to a CLIS patient on the performance of the BCI system during a YES/NO paradigm. Comparison of communication success rate in BCI classification between different semantic categories indicate that semantic content of sentences presented to a CLIS patient can affect the BCI performance. Affected concepts are mostly associated with executive words. These findings can be beneficial towards development of more reliable communication device for patients in CLIS. In addition, these results may assist in elucidating the cognitive changes in completely paralyzed patients with the passage of time since the onset of total immovability.

Covert Intention to Answer to Self-Referential Questions Is Represented in Alpha-Band Local and Interregional Neural Synchronies

The most fundamental and simplest intention for interpersonal communication may be the intentions to answer “yes” or “no” to a question, based on a binary decision. However, the neural mechanism of this type of intention has not been investigated in detail. The main purpose of this study was to investigate cortical processing of the “yes/no” intentions to answer self-referential questions. Multichannel electroencephalograms (EEGs) were recorded while covertly answering self-referential questions with either “yes” or “no”. Event-related spectral perturbation (ERSP) and interregional phase synchrony (PS) were investigated to identify the differences in local and global neural synchronies between two intentions. We found that the local and interregional neural synchronies in the alpha-band were significantly different between “yes” and “no,” especially at the period of retaining the intention in mind, which was greater for “no” than for “yes.” These results can be interpreted to signify that a higher cognitive load during working memory retention or higher attentional demand is required for the “no” intention compared to “yes.” Our findings suggest that both local and global neural synchronies in the alpha-band may be significantly differentiated during a critical temporal epoch, according to the contents of the mental representation of the intention.

Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation

Brain-machine interfaces (BMIs) combine methods, approaches, and concepts derived from neurophysiology, computer science, and engineering in an effort to establish real-time bidirectional links between living brains and artificial actuators. Although theoretical propositions and some proof of concept experiments on directly linking the brains with machines date back to the early 1960s, BMI research only took off in earnest at the end of the 1990s, when this approach became intimately linked to new neurophysiological methods for sampling large-scale brain activity. The classic goals of BMIs are 1) to unveil and utilize principles of operation and plastic properties of the distributed and dynamic circuits of the brain and 2) to create new therapies to restore mobility and sensations to severely disabled patients. Over the past decade, a wide range of BMI applications have emerged, which considerably expanded these original goals. BMI studies have shown neural control over the movements of robotic and virtual actuators that enact both upper and lower limb functions. Furthermore, BMIs have also incorporated ways to deliver sensory feedback, generated from external actuators, back to the brain. BMI research has been at the forefront of many neurophysiological discoveries, including the demonstration that, through continuous use, artificial tools can be assimilated by the primate brain's body schema. Work on BMIs has also led to the introduction of novel neurorehabilitation strategies. As a result of these efforts, long-term continuous BMI use has been recently implicated with the induction of partial neurological recovery in spinal cord injury patients.

Comparison of eye tracking, electrooculography and an auditory brain-computer interface for binary communication: a case study with a participant in the locked-in state

BACKGROUND

In this study, we evaluated electrooculography (EOG), an eye tracker and an auditory brain-computer interface (BCI) as access methods to augmentative and alternative communication (AAC). The participant of the study has been in the locked-in state (LIS) for 6 years due to amyotrophic lateral sclerosis. He was able to communicate with slow residual eye movements, but had no means of partner independent communication. We discuss the usability of all tested access methods and the prospects of using BCIs as an assistive technology.

METHODS

Within four days, we tested whether EOG, eye tracking and a BCI would allow the participant in LIS to make simple selections. We optimized the parameters in an iterative procedure for all systems.

RESULTS

The participant was able to gain control over all three systems. Nonetheless, due to the level of proficiency previously achieved with his low-tech AAC method, he did not consider using any of the tested systems as an additional communication channel. However, he would consider using the BCI once control over his eye muscles would no longer be possible. He rated the ease of use of the BCI as the highest among the tested systems, because no precise eye movements were required; but also as the most tiring, due to the high level of attention needed to operate the BCI.

CONCLUSIONS

In this case study, the partner based communication was possible due to the good care provided and the proficiency achieved by the interlocutors. To ease the transition from a low-tech AAC method to a BCI once control over all muscles is lost, it must be simple to operate. For persons, who rely on AAC and are affected by a progressive neuromuscular disease, we argue that a complementary approach, combining BCIs and standard assistive technology, can prove valuable to achieve partner independent communication and ease the transition to a purely BCI based approach. Finally, we provide further evidence for the importance of a user-centered approach in the design of new assistive devices.

Developing brain-computer interfaces from a user-centered perspective: Assessing the needs of persons with amyotrophic lateral sclerosis, caregivers, and professionals

By focus group methodology, we examined the opinions and requirements of persons with ALS, their caregivers, and health care assistants with regard to developing a brain-computer interface (BCI) system that fulfills the user's needs. Four overarching topics emerged from this analysis: 1) lack of information on BCI and its everyday applications; 2) importance of a customizable system that supports individuals throughout the various stages of the disease; 3) relationship between affectivity and technology use; and 4) importance of individuals retaining a sense of agency. These findings should be considered when developing new assistive technology. Moreover, the BCI community should acknowledge the need to bridge experimental results and its everyday application.

Insula and inferior frontal triangularis activations distinguish between conditioned brain responses using emotional sounds for basic BCI communication

In order to enable communication through a brain-computer interface (BCI), it is necessary to discriminate between distinct brain responses. As a first step, we probed the possibility to discriminate between affirmative (“yes”) and negative (“no”) responses using a semantic classical conditioning paradigm, within an fMRI setting. Subjects were presented with congruent and incongruent word-pairs as conditioned stimuli (CS), respectively eliciting affirmative and negative responses. Incongruent word-pairs were associated to an unpleasant unconditioned stimulus (scream, US1) and congruent word-pairs were associated to a pleasant unconditioned stimulus (baby-laughter, US2), in order to elicit emotional conditioned responses (CR). The aim was to discriminate between affirmative and negative responses, enabled by their association with the positive and negative affective stimuli. In the late acquisition phase, when the US were not present anymore, there was a strong significant differential activation for incongruent and congruent word-pairs in a cluster comprising the left insula and the inferior frontal triangularis. This association was not found in the habituation phase. These results suggest that the difference in affirmative and negative brain responses was established as an effect of conditioning, allowing to further investigate the possibility of using this paradigm for a binary choice BCI.

A feasibility study of an improved procedure for using EEG to detect brain responses to imagery instruction in patients with disorders of consciousness

One of the major concerns of recent studies is the correct discrimination between vegetative and minimally conscious state as the distinction between these two conditions has major implications for subsequent patient rehabilitation. In particular, it would be advantageous to establish communication with these patients. This work describes a procedure using EEG to detect brain responses to imagery instruction in patients with disorders of consciousness. Five healthy subjects and five patients with different disorders of consciousness took part in the study. A support vector machine classifier applied to EEG data was used to distinguish two mental tasks (Imagery Trial) and to detect answers to simple yes or no questions (pre-Communication Trial). The proposed procedure uses feature selection based on a nested-leave-one-out algorithm to reduce the number of electrodes required. We obtained a mean classification accuracy of 82.0% (SD 5.1%) for healthy subjects and 84.6% (SD 9.1%) for patients in the Imagery Trial, and a mean classification accuracy of 80.7% (SD 11.5%) for healthy subjects and 91.7% (SD 7.4%) for patients in the pre-Communication Trial. The subset of electrodes selected was subject and session dependent.

Prediction of P300 BCI aptitude in severe motor impairment

Brain-computer interfaces (BCIs) provide a non-muscular communication channel for persons with severe motor impairments. Previous studies have shown that the aptitude with which a BCI can be controlled varies from person to person. A reliable predictor of performance could facilitate selection of a suitable BCI paradigm. Eleven severely motor impaired participants performed three sessions of a P300 BCI web browsing task. Before each session auditory oddball data were collected to predict the BCI aptitude of the participants exhibited in the current session. We found a strong relationship of early positive and negative potentials around 200 ms (elicited with the auditory oddball task) with performance. The amplitude of the P2 (r  =  −0.77) and of the N2 (r  =  −0.86) had the strongest correlations. Aptitude prediction using an auditory oddball was successful. The finding that the N2 amplitude is a stronger predictor of performance than P3 amplitude was reproduced after initially showing this effect with a healthy sample of BCI users. This will reduce strain on the end-users by minimizing the time needed to find suitable paradigms and inspire new approaches to improve performance.

Brain communication in the locked-in state

Patients in the completely locked-in state have no means of communication and they represent the target population for brain-computer interface research in the last 15 years. Although different paradigms have been tested and different physiological signals used, to date no sufficiently documented completely locked-in state patient was able to control a brain-computer interface over an extended time period. We introduce Pavlovian semantic conditioning to enable basic communication in completely locked-in state. This novel paradigm is based on semantic conditioning for online classification of neuroelectric or any other physiological signals to discriminate between covert (cognitive) 'yes' and 'no' responses. The paradigm comprised the presentation of affirmative and negative statements used as conditioned stimuli, while the unconditioned stimulus consisted of electrical stimulation of the skin paired with affirmative statements. Three patients with advanced amyotrophic lateral sclerosis participated over an extended time period, one of which was in a completely locked-in state, the other two in the locked-in state. The patients' level of vigilance was assessed through auditory oddball procedures to study the correlation between vigilance level and the classifier's performance. The average online classification accuracies of slow cortical components of electroencephalographic signals were around chance level for all the patients. The use of a non-linear classifier in the offline classification procedure resulted in a substantial improvement of the accuracy in one locked-in state patient achieving 70% correct classification. A reliable level of performance in the completely locked-in state patient was not achieved uniformly throughout the 37 sessions despite intact cognitive processing capacity, but in some sessions communication accuracies up to 70% were achieved. Paradigm modifications are proposed. Rapid drop of vigilance was detected suggesting attentional variations or variations of circadian period as important factors in brain-computer interface communication with locked-in state and completely locked-in state.

Prediction of brain-computer interface aptitude from individual brain structure

OBJECTIVE

Brain-computer interface (BCI) provide a non-muscular communication channel for patients with impairments of the motor system. A significant number of BCI users is unable to obtain voluntary control of a BCI-system in proper time. This makes methods that can be used to determine the aptitude of a user necessary.

METHODS

We hypothesized that integrity and connectivity of involved white matter connections may serve as a predictor of individual BCI-performance. Therefore, we analyzed structural data from anatomical scans and DTI of motor imagery BCI-users differentiated into high and low BCI-aptitude groups based on their overall performance.

RESULTS

Using a machine learning classification method we identified discriminating structural brain trait features and correlated the best features with a continuous measure of individual BCI-performance. Prediction of the aptitude group of each participant was possible with near perfect accuracy (one error).

CONCLUSIONS

Tissue volumetric analysis yielded only poor classification results. In contrast, the structural integrity and myelination quality of deep white matter structures such as the Corpus Callosum, Cingulum, and Superior Fronto-Occipital Fascicle were positively correlated with individual BCI-performance.

SIGNIFICANCE

This confirms that structural brain traits contribute to individual performance in BCI use.

Encode the "STOP" command by photo-stimulation for precise control of rat-robot

Studies on behavior control are important for bio-robots designation. For auto or manual navigation of the bio-robots, the accuracy of the command execution is especially critical. In this paper, we reported a precise "STOP" command for the rat-robots by optical stimulation of the central nervous system (CNS). We labeled dorsolateral periaqueductal gray (dlPAG) neurons with light sensitive channelrhodopsin-2 (ChR2) and directly probed the optical fiber to reactivate these neurons. The rats showed freezing behavior only upon the optical stimulation with an appropriate range of laser intensity and stimulation frequency. Neuron spikes and local field potential (LFP) were simultaneously recorded with optical stimulation by optrodes on free moving rat-robots. Together, our findings demonstrated the utility of deep brain optical stimulation for the stopping behavior of rat-robot control and indicated a potential application of optogenetics for precise control of bio-robots in further work.