Long-term evaluation of a 4-class imagery-based brain-computer interface

Evaluating the perspectives of those with severe physical impairments while learning BCI control of a commercial augmentative and alternative communication paradigm

Augmentative and alternative communication (AAC) techniques can provide access to communication for individuals with severe physical impairments. Brain-computer interface (BCI) access techniques may serve alongside existing AAC access methods to provide communication device control. However, there is limited information available about how individual perspectives change with motor-based BCI-AAC learning. Four individuals with ALS completed 12 BCI-AAC training sessions in which they made letter selections during an automatic row-column scanning pattern via a motor-based BCI-AAC. Recurring measures were taken before and after each BCI-AAC training session to evaluate changes associated with BCI-AAC performance, and included measures of fatigue, frustration, mental effort, physical effort, device satisfaction, and overall ease of device control. Levels of pre- to post-fatigue were low for use of the BCI-AAC system. However, participants indicated different perceptions of the term fatigue, with three participants discussing fatigue to be generally synonymous with physical effort, and one mental effort. Satisfaction with the BCI-AAC system was related to BCI-AAC performance for two participants, and levels of frustration for two participants. Considering a range of person-centered measures in future clinical BCI-AAC applications is important for optimizing and standardizing BCI-AAC assessment procedures.

Mental imagery of whole-body motion along the sagittal-anteroposterior axis

Whole-body motor imagery is conceptualised as a mental symbolisation directly and indirectly associated with neural oscillations similar to whole-body motor execution. Motor and somatosensory activity, including vestibular activity, is a typical corticocortical substrate of body motion. Yet, it is not clear how this neural substrate is organised when participants are instructed to imagine moving their body forward or backward along the sagittal-anteroposterior axis. It is the aim of the current study to identify the fingerprint of the neural substrate by recording the cortical activity of 39 participants via a 32 electroencephalography (EEG) device. The participants were instructed to imagine moving their body forward or backward from a first-person perspective. Principal Component Analysis (i.e. PCA) applied to the neural activity of whole-body motor imagery revealed neural interconnections mirroring between forward and backward conditions: beta pre-motor and motor oscillations in the left and right hemisphere overshadowed beta parietal oscillations in forward condition, and beta parietal oscillations in the left and right hemisphere overshadowed beta pre-motor and motor oscillations in backward condition. Although functional significance needs to be discerned, beta pre-motor, motor and somatosensory oscillations might represent specific settings within the corticocortical network and provide meaningful information regarding the neural dynamics of continuous whole-body motion. It was concluded that the evoked multimodal fronto-parietal neural activity would correspond to the neural activity that could be expected if the participants were physically enacting movement of the whole-body in sagittal-anteroposterior plane as they would in their everyday environment.

Convolutional neural networks for decoding electroencephalography responses and visualizing trial by trial changes in discriminant features

BACKGROUND

Deep learning has revolutionized the field of computer vision, where convolutional neural networks (CNNs) extract complex patterns of information from large datasets. The use of deep networks in neuroscience is mainly focused to neuroimaging or brain computer interface -BCI- applications. In electroencephalography (EEG) research, multivariate pattern analysis (MVPA) mainly relies on linear algorithms, which require a homogeneous dataset and assume that discriminant features appear at consistent latencies and electrodes across trials. However, neural responses may shift in time or space during an experiment, resulting in under-estimation of discriminant features. Here, we aimed at using CNNs to classify EEG responses to external stimuli, by taking advantage of time- and space- unlocked neural activity, and at examining how discriminant features change over the course of an experiment, on a trial by trial basis.

NEW METHOD

We present a novel pipeline, consisting of data augmentation, CNN training, and feature visualization techniques, fine-tuned for MVPA on EEG data.

RESULTS

Our pipeline provides high classification performance and generalizes to new datasets. Additionally, we show that the features identified by the CNN for classification are electrophysiologically interpretable and can be reconstructed at the single-trial level to study trial-by-trial evolution of class-specific discriminant activity.

COMPARISON WITH EXISTING TECHNIQUES

The developed pipeline was compared to commonly used MVPA algorithms like logistic regression and support vector machines, as well as to shallow and deep convolutional neural networks. Our approach yielded significantly higher classification performance than existing MVPA techniques (p = 0.006) and comparable results to other CNNs for EEG data.

CONCLUSION

In summary, we present a novel deep learning pipeline for MVPA of EEG data, that can extract trial-by-trial discriminative activity in a data-driven way.

Neural Kinesthetic Contribution to Motor Imagery of Body Parts: Tongue, Hands, and Feet

Motor imagery (MI) is assimilated to a perception-action process, which is mentally represented. Although several models suggest that MI, and its equivalent motor execution, engage very similar brain areas, the mechanisms underlying MI and their associated components are still under investigation today. Using 22 Ag/AgCl EEG electrodes, 19 healthy participants (nine males and 10 females) with an average age of 25.8 years old (sd = 3.5 years) were required to imagine moving several parts of their body (i.e., first-person perspective) one by one: left and right hand, tongue, and feet. Network connectivity analysis based on graph theory, together with a correlational analysis, were performed on the data. The findings suggest evidence for motor and somesthetic neural synchronization and underline the role of the parietofrontal network for the tongue imagery task only. At both unilateral and bilateral cortical levels, only the tongue imagery task appears to be associated with motor and somatosensory representations, that is, kinesthetic representations, which might contribute to verbal actions. As such, the present findings suggest the idea that imagined tongue movements, involving segmentary kinesthetic actions, could be the prerequisite of language.

Decoding covert speech for intuitive control of brain-computer interfaces based on single-trial EEG: a feasibility study

For individuals with severe motor deficiencies, controlling external devices such as robotic arms or wheelchairs can be challenging, as many devices require some degree of motor control to be operated, e.g. when controlled using a joystick. A brain-computer interface (BCI) relies only on signals from the brain and may be used as a controller instead of muscles. Motor imagery (MI) has been used in many studies as a control signal for BCIs. However, MI may not be suitable for all control purposes, and several people cannot obtain BCI control with MI. In this study, the aim was to investigate the feasibility of decoding covert speech from single-trial EEG and compare and combine it with MI. In seven healthy subjects, EEG was recorded with twenty-five channels during six different actions: Speaking three words (both covert and overt speech), two arm movements (both motor imagery and execution), and one idle class. Temporal and spectral features were derived from the epochs and classified with a random forest classifier. The average classification accuracy was $67 \pm 9$ % and $75\pm 7$ % for covert and overt speech, respectively; this was 5-10 % lower than the movement classification. The performance of the combined movement-speech decoder was $61 \pm 9$ % and $67\pm 7$ % (covert and overt), but it is possible to have more classes available for control. The possibility of using covert speech for controlling a BCI was outlined; this is a step towards a multimodal BCI system for improved usability.

The Sensitivity of Single-Trial Mu-Suppression Detection for Motor Imagery Performance as Compared to Motor Execution and Motor Observation Performance

Motor imagery (MI) has been widely used to operate brain-computer interface (BCI) systems for rehabilitation and some life assistive devices. However, the current performance of an MI-based BCI cannot fully meet the needs of its in-field applications. Most of the BCIs utilizing a generalized feature for all participants have been found to greatly hamper the efficacy of the BCI system. Hence, some attempts have made on the exploration of subject-dependent parameters, but it remains challenging to enhance BCI performance as expected. To this end, in this study, we used the independent component analysis (ICA), which has been proved capable of isolating the pure motor-related component from non-motor-related brain processes and artifacts and extracting the common motor-related component across MI, motor execution (ME), and motor observation (MO) conditions. Then, a sliding window approach was used to detect significant mu-suppression from the baseline using the electroencephalographic (EEG) alpha power time course and, thus, the success rate of the mu-suppression detection could be assessed on a single-trial basis. By comparing the success rates using different parameters, we further quantified the extent of the improvement in each motor condition to evaluate the effectiveness of both generalized and individualized parameters. The results showed that in ME condition, the success rate under individualized latency and that under generalized latency was 90.0% and 77.75%, respectively; in MI condition, the success rate was 74.14% for individual latency and 58.47% for generalized latency, and in MO condition, the success rate was 67.89% and 61.26% for individual and generalized latency, respectively. As can be seen, the success rate in each motor condition was significantly improved by utilizing an individualized latency compared to that using the generalized latency. Moreover, the comparison of the individualized window latencies for the mu-suppression detection across different runs of the same participant as well as across different participants showed that the window latency was significantly more consistent in the intra-subject than in the inter-subject settings. As a result, we proposed that individualizing the latency for detecting the mu-suppression feature for each participant might be a promising attempt to improve the MI-based BCI performance.

Using brain-computer interfaces: a scoping review of studies employing social research methods

Background

The rapid expansion of research on Brain-Computer Interfaces (BCIs) is not only due to the promising solutions offered for persons with physical impairments. There is also a heightened need for understanding BCIs due to the challenges regarding ethics presented by new technology, especially in its impact on the relationship between man and machine. Here we endeavor to present a scoping review of current studies in the field to gain insight into the complexity of BCI use. By examining studies related to BCIs that employ social research methods, we seek to demonstrate the multitude of approaches and concerns from various angles in considering the social and human impact of BCI technology.

Methods

For this scoping review of research on BCIs’ social and ethical implications, we systematically analyzed six databases, encompassing the fields of medicine, psychology, and the social sciences, in order to identify empirical studies on BCIs. The search yielded 73 publications that employ quantitative, qualitative, or mixed methods.

Results

Of the 73 publications, 71 studies address the user perspective. Some studies extend to consideration of other BCI stakeholders such as medical technology experts, caregivers, or health care professionals. The majority of the studies employ quantitative methods. Recurring themes across the studies examined were general user opinion towards BCI, central technical or social issues reported, requests/demands made by users of the technology, the potential/future of BCIs, and ethical aspects of BCIs.

Conclusions

Our findings indicate that while technical aspects of BCIs such as usability or feasibility are being studied extensively, comparatively little in-depth research has been done on the self-image and self-experience of the BCI user. In general there is also a lack of focus or examination of the caregiver’s perspective.

Electronic supplementary material

The online version of this article (10.1186/s12910-019-0354-1) contains supplementary material, which is available to authorized users.

Mental Task Evaluation for Hybrid NIRS-EEG Brain-Computer Interfaces

Based on recent electroencephalography (EEG) and near-infrared spectroscopy (NIRS) studies that showed that tasks such as motor imagery and mental arithmetic induce specific neural response patterns, we propose a hybrid brain-computer interface (hBCI) paradigm in which EEG and NIRS data are fused to improve binary classification performance. We recorded simultaneous NIRS-EEG data from nine participants performing seven mental tasks (word generation, mental rotation, subtraction, singing and navigation, and motor and face imagery). Classifiers were trained for each possible pair of tasks using (1) EEG features alone, (2) NIRS features alone, and (3) EEG and NIRS features combined, to identify the best task pairs and assess the usefulness of a multimodal approach. The NIRS-EEG approach led to an average increase in peak kappa of 0.03 when using features extracted from one-second windows (equivalent to an increase of 1.5% in classification accuracy for balanced classes). The increase was much stronger (0.20, corresponding to an 10% accuracy increase) when focusing on time windows of high NIRS performance. The EEG and NIRS analyses further unveiled relevant brain regions and important feature types. This work provides a basis for future NIRS-EEG hBCI studies aiming to improve classification performance toward more efficient and flexible BCIs.

Single-trial effective brain connectivity patterns enhance discriminability of mental imagery tasks

OBJECTIVE

The majority of the current approaches of connectivity based brain-computer interface (BCI) systems focus on distinguishing between different motor imagery (MI) tasks. Brain regions associated with MI are anatomically close to each other, hence these BCI systems suffer from low performances. Our objective is to introduce single-trial connectivity feature based BCI system for cognition imagery (CI) based tasks wherein the associated brain regions are located relatively far away as compared to those for MI.

APPROACH

We implemented time-domain partial Granger causality (PGC) for the estimation of the connectivity features in a BCI setting. The proposed hypothesis has been verified with two publically available datasets involving MI and CI tasks.

MAIN RESULTS

The results support the conclusion that connectivity based features can provide a better performance than a classical signal processing framework based on bandpass features coupled with spatial filtering for CI tasks, including word generation, subtraction, and spatial navigation. These results show for the first time that connectivity features can provide a reliable performance for imagery-based BCI system.

SIGNIFICANCE

We show that single-trial connectivity features for mixed imagery tasks (i.e. combination of CI and MI) can outperform the features obtained by current state-of-the-art method and hence can be successfully applied for BCI applications.

Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation

Brain-machine interfaces (BMIs) combine methods, approaches, and concepts derived from neurophysiology, computer science, and engineering in an effort to establish real-time bidirectional links between living brains and artificial actuators. Although theoretical propositions and some proof of concept experiments on directly linking the brains with machines date back to the early 1960s, BMI research only took off in earnest at the end of the 1990s, when this approach became intimately linked to new neurophysiological methods for sampling large-scale brain activity. The classic goals of BMIs are 1) to unveil and utilize principles of operation and plastic properties of the distributed and dynamic circuits of the brain and 2) to create new therapies to restore mobility and sensations to severely disabled patients. Over the past decade, a wide range of BMI applications have emerged, which considerably expanded these original goals. BMI studies have shown neural control over the movements of robotic and virtual actuators that enact both upper and lower limb functions. Furthermore, BMIs have also incorporated ways to deliver sensory feedback, generated from external actuators, back to the brain. BMI research has been at the forefront of many neurophysiological discoveries, including the demonstration that, through continuous use, artificial tools can be assimilated by the primate brain's body schema. Work on BMIs has also led to the introduction of novel neurorehabilitation strategies. As a result of these efforts, long-term continuous BMI use has been recently implicated with the induction of partial neurological recovery in spinal cord injury patients.

Evaluation of induced and evoked changes in EEG during selective attention to verbal stimuli

BACKGROUND

Two challenges need to be addressed before bringing non-motor mental tasks for brain-computer interface (BCI) control to persons in a minimally conscious state (MCS), who can be behaviorally unresponsive even when proven to be consciously aware: first, keeping the cognitive demands as low as possible so that they could be fulfilled by persons with MCS. Second, increasing the control of experimental protocol (i.e. type and timing of the task performance).

NEW METHOD

The goal of this study is twofold: first goal is to develop an experimental paradigm that can facilitate the performance of brain-teasers (e.g. mental subtraction and word generation) on the one hand, and can increase the control of experimental protocol on the other hand. The second goal of this study is to exploit the similar findings for mentally attending to someone else's verbal performance of brain-teaser tasks and self-performing the same tasks to setup an online BCI, and to compare it in healthy participants to the current "state-of-the-art" motor imagery (MI, sports).

RESULTS

The response accuracies for the best performing healthy participants indicate that selective attention to verbal performance of mental subtraction (SUB) is a viable alternative to the MI. Time-frequency analysis of the SUB task in one participant with MCS did not reveal any significant (p<0.05) EEG changes, whereas imagined performance of one sport of participants' choice (SPORT) revealed task-related EEG changes over neurophysiological plausible cortical areas.

COMPARISON WITH EXISTING METHODS

We found that mentally attending to someone else's verbal performance of brain-teaser tasks leads to similar results as in self-performing the same tasks.

CONCLUSIONS

In this work we demonstrated that a single auditory selective attention task (i.e. mentally attending to someone else's verbal performance of mental subtraction) can modulate both induced and evoked changes in EEG, and be used for yes/no communication in an auditory scanning paradigm.

Brain-computer interfaces for patients with disorders of consciousness

The disorders of consciousness refer to clinical conditions that follow a severe head injury. Patients diagnosed as in a vegetative state lack awareness, while patients diagnosed as in a minimally conscious state retain fluctuating awareness. However, it is a challenge to accurately diagnose these disorders with clinical assessments of behavior. To improve diagnostic accuracy, neuroimaging-based approaches have been developed to detect the presence or absence of awareness in patients who lack overt responsiveness. For the small subset of patients who retain awareness, brain-computer interfaces could serve as tools for communication and environmental control. Here we review the existing literature concerning the sensory and cognitive abilities of patients with disorders of consciousness with respect to existing brain-computer interface designs. We highlight the challenges of device development for this special population and address some of the most promising approaches for future investigations.

Structural and functional correlates of motor imagery BCI performance: Insights from the patterns of fronto-parietal attention network

Motor imagery (MI)-based brain-computer interfaces (BCIs) have been widely used for rehabilitation of motor abilities and prosthesis control for patients with motor impairments. However, MI-BCI performance exhibits a wide variability across subjects, and the underlying neural mechanism remains unclear. Several studies have demonstrated that both the fronto-parietal attention network (FPAN) and MI are involved in high-level cognitive processes that are crucial for the control of BCIs. Therefore, we hypothesized that the FPAN may play an important role in MI-BCI performance. In our study, we recorded multi-modal datasets consisting of MI electroencephalography (EEG) signals, T1-weighted structural and resting-state functional MRI data for each subject. MI-BCI performance was evaluated using the common spatial pattern to extract the MI features from EEG signals. One cortical structural feature (cortical thickness (CT)) and two measurements (degree centrality (DC) and eigenvector centrality (EC)) of node centrality were derived from the structural and functional MRI data, respectively. Based on the information extracted from the EEG and MRI, a correlation analysis was used to elucidate the relationships between the FPAN and MI-BCI performance. Our results show that the DC of the right ventral intraparietal sulcus, the EC and CT of the left inferior parietal lobe, and the CT of the right dorsolateral prefrontal cortex were significantly associated with MI-BCI performance. Moreover, the receiver operating characteristic analysis and machine learning classification revealed that the EC and CT of the left IPL could effectively predict the low-aptitude BCI users from the high-aptitude BCI users with 83.3% accuracy. Those findings consistently reveal that the individuals who have efficient FPAN would perform better on MI-BCI. Our findings may deepen the understanding of individual variability in MI-BCI performance, and also may provide a new biomarker to predict individual MI-BCI performance.

Endogenous Sensory Discrimination and Selection by a Fast Brain Switch for a High Transfer Rate Brain-Computer Interface

In this study, we present a novel multi-class brain-computer interface (BCI) for communication and control. In this system, the information processing is shared by the algorithm (computer) and the user (human). Specifically, an electro-tactile cycle was presented to the user, providing the choice (class) by delivering timely sensory input. The user discriminated these choices by his/her endogenous sensory ability and selected the desired choice with an intuitive motor task. This selection was detected by a fast brain switch based on real-time detection of movement-related cortical potentials from scalp EEG. We demonstrated the feasibility of such a system with a four-class BCI, yielding a true positive rate of  ∼ 80% and  ∼ 70%, and an information transfer rate of  ∼ 7 bits/min and  ∼ 5 bits/min, for the movement and imagination selection command, respectively. Furthermore, when the system was extended to eight classes, the throughput of the system was improved, demonstrating the capability of accommodating a large number of classes. Combining the endogenous sensory discrimination with the fast brain switch, the proposed system could be an effective, multi-class, gaze-independent BCI system for communication and control applications.

Classification of mental tasks in the prefrontal cortex using fNIRS

Functional near infrared spectroscopy (fNIRS) is rapidly gaining interest in both the Neuroscience, as well as the Brain-Computer-Interface (BCI) community. Despite these efforts, most single-trial analysis of fNIRS data is focused on motor-imagery, or mental arithmetics. In this study, we investigate the suitability of different mental tasks, namely mental arithmetics, word generation and mental rotation for fNIRS based BCIs. We provide the first systematic comparison of classification accuracies achieved in a sample study. Data was collected from 10 subjects performing these three tasks.

Noninvasive Brain-Computer Interfaces Based on Sensorimotor Rhythms

Brain-computer interfaces (BCIs) have been explored in the field of neuroengineering to investigate how the brain can use these systems to control external devices. We review the principles and approaches we have taken to develop a sensorimotor rhythm EEG based brain-computer interface (BCI). The methods include developing BCI systems incorporating the control of physical devices to increase user engagement, improving BCI systems by inversely mapping scalp-recorded EEG signals to the cortical source domain, integrating BCI with noninvasive neuromodulation strategies to improve learning, and incorporating mind-body awareness training to enhance BCI learning and performance. The challenges and merits of these strategies are discussed, together with recent findings. Our work indicates that the sensorimotor-rhythm-based noninvasive BCI has the potential to provide communication and control capabilities as an alternative to physiological motor pathways.

Individually adapted imagery improves brain-computer interface performance in end-users with disability

Brain-computer interfaces (BCIs) translate oscillatory electroencephalogram (EEG) patterns into action. Different mental activities modulate spontaneous EEG rhythms in various ways. Non-stationarity and inherent variability of EEG signals, however, make reliable recognition of modulated EEG patterns challenging. Able-bodied individuals who use a BCI for the first time achieve - on average - binary classification performance of about 75%. Performance in users with central nervous system (CNS) tissue damage is typically lower. User training generally enhances reliability of EEG pattern generation and thus also robustness of pattern recognition. In this study, we investigated the impact of mental tasks on binary classification performance in BCI users with central nervous system (CNS) tissue damage such as persons with stroke or spinal cord injury (SCI). Motor imagery (MI), that is the kinesthetic imagination of movement (e.g. squeezing a rubber ball with the right hand), is the "gold standard" and mainly used to modulate EEG patterns. Based on our recent results in able-bodied users, we hypothesized that pair-wise combination of "brain-teaser" (e.g. mental subtraction and mental word association) and "dynamic imagery" (e.g. hand and feet MI) tasks significantly increases classification performance of induced EEG patterns in the selected end-user group. Within-day (How stable is the classification within a day?) and between-day (How well does a model trained on day one perform on unseen data of day two?) analysis of variability of mental task pair classification in nine individuals confirmed the hypothesis. We found that the use of the classical MI task pair hand vs. feed leads to significantly lower classification accuracy - in average up to 15% less - in most users with stroke or SCI. User-specific selection of task pairs was again essential to enhance performance. We expect that the gained evidence will significantly contribute to make imagery-based BCI technology become accessible to a larger population of users including individuals with special needs due to CNS damage.

A hybrid BCI based on EEG and fNIRS signals improves the performance of decoding motor imagery of both force and speed of hand clenching

OBJECTIVE

In order to increase the number of states classified by a brain-computer interface (BCI), we utilized a motor imagery task where subjects imagined both force and speed of hand clenching.

APPROACH

The BCI utilized simultaneously recorded electroencephalographic (EEG) and functional near-infrared spectroscopy (fNIRS) signals. The time-phase-frequency feature was extracted from EEG, whereas the HbD [the difference of oxy-hemoglobin (HbO) and deoxy-hemoglobin (Hb)] feature was used to improve the classification accuracy of fNIRS. The EEG and fNIRS features were combined and optimized using the joint mutual information (JMI) feature selection criterion; then the extracted features were classified with the extreme learning machines (ELMs).

MAIN RESULTS

In this study, the averaged classification accuracy of EEG signals achieved by the time-phase-frequency feature improved by 7%, to 18%, more than the single-type feature, and improved by 15% more than common spatial pattern (CSP) feature. The HbD feature of fNIRS signals improved the accuracy by 1%, to 4%, more than Hb, HbO, or HbT (total hemoglobin). The EEG-fNIRS feature for decoding motor imagery of both force and speed of hand clenching achieved an accuracy of 89% ± 2%, and improved the accuracy by 1% to 5% more than the sole EEG or fNIRS feature.

SIGNIFICANCE

Our novel motor imagery paradigm improves BCI performance by increasing the number of extracted commands. Both the time-phase-frequency and the HbD feature improve the classification accuracy of EEG and fNIRS signals, respectively, and the hybrid EEG-fNIRS technique achieves a higher decoding accuracy for two-class motor imagery, which may provide the framework for future multi-modal online BCI systems.

An approach to improve the performance of subject-independent BCIs-based on motor imagery allocating subjects by gender

Background

One of the difficulties for the implementation of Brain-Computer Interface (BCI) systems for motor impaired patients is the time consumed in the system design process, since patients do not have the adequate physical nor psychological conditions to complete the process. For this reason most of BCIs are designed in a subject-dependent approach using data of healthy subjects. The developing of subject-independent systems is an option to decrease the required training sessions to design a BCI with patient functionality. This paper presents a proof-of-concept study to evaluate subject-independent system based on hand motor imagery taking gender into account.

Methods

Subject-Independent BCIs are proposed using Common Spatial Patterns and log variance features of two groups of healthy subjects; one of the groups was composed by people of male gender and the other one by people of female gender. The performance of the developed gender-specific BCI designs was evaluated with respect to a subject-independent BCI designed without taking gender into account, and afterwards its performance was evaluated with data of two healthy subjects that were not included in the initial sample. As an additional test to probe the potential use for subcortical stroke patients we applied the methodology to two patients with right hand weakness. T-test was employed to determine the significance of the difference between traditional approach and the proposed gender-specific approach.

Results

For most of the tested conditions, the gender-specific BCIs have a statistically significant better performance than those that did not take gender into account. It was also observed that with a BCI designed with log-variance features in the alpha and beta band of healthy subjects’ data, it was possible to classify hand motor imagery of subcortical stroke patients above the practical level of chance.

Conclusions

A larger subjects’ sample test may be necessary to improve the performances of the gender-specific BCIs and to further test this methodology on different patients. The reduction of complexity in the implementation of BCI systems could bring these systems closer to applications such as controlling devices for the motor rehabilitation of stroke patients, and therefore, contribute to a more effective neurological rehabilitation.

The user-centered design as novel perspective for evaluating the usability of BCI-controlled applications

Albeit research on brain-computer interfaces (BCI) for controlling applications has expanded tremendously, we still face a translational gap when bringing BCI to end-users. To bridge this gap, we adapted the user-centered design (UCD) to BCI research and development which implies a shift from focusing on single aspects, such as accuracy and information transfer rate (ITR), to a more holistic user experience. The UCD implements an iterative process between end-users and developers based on a valid evaluation procedure. Within the UCD framework usability of a device can be defined with regard to its effectiveness, efficiency, and satisfaction. We operationalized these aspects to evaluate BCI-controlled applications. Effectiveness was regarded equivalent to accuracy of selections and efficiency to the amount of information transferred per time unit and the effort invested (workload). Satisfaction was assessed with questionnaires and visual-analogue scales. These metrics have been successfully applied to several BCI-controlled applications for communication and entertainment, which were evaluated by end-users with severe motor impairment. Results of four studies, involving a total of N = 19 end-users revealed: effectiveness was moderate to high; efficiency in terms of ITR was low to high and workload low to medium; depending on the match between user and technology, and type of application satisfaction was moderate to high. The here suggested evaluation metrics within the framework of the UCD proved to be an applicable and informative approach to evaluate BCI controlled applications, and end-users with severe impairment and in the locked-in state were able to participate in this process.

Enhanced performance by time-frequency-phase feature for EEG-based BCI systems

We introduce a new motor parameter imagery paradigm using clench speed and clench force motor imagery. The time-frequency-phase features are extracted from mu rhythm and beta rhythms, and the features are optimized using three process methods: no-scaled feature using “MIFS” feature selection criterion, scaled feature using “MIFS” feature selection criterion, and scaled feature using “mRMR” feature selection criterion. Support vector machines (SVMs) and extreme learning machines (ELMs) are compared for classification between clench speed and clench force motor imagery using the optimized feature. Our results show that no significant difference in the classification rate between SVMs and ELMs is found. The scaled feature combinations can get higher classification accuracy than the no-scaled feature combinations at significant level of 0.01, and the “mRMR” feature selection criterion can get higher classification rate than the “MIFS” feature selection criterion at significant level of 0.01. The time-frequency-phase feature can improve the classification rate by about 20% more than the time-frequency feature, and the best classification rate between clench speed motor imagery and clench force motor imagery is 92%. In conclusion, the motor parameter imagery paradigm has the potential to increase the direct control commands for BCI control and the time-frequency-phase feature has the ability to improve BCI classification accuracy.

Wireless EEG with individualized channel layout enables efficient motor imagery training

OBJECTIVE

The study compared two channel-reduction approaches in order to investigate the effects of systematic motor imagery (MI) neurofeedback practice in an everyday environment using a very user-friendly EEG system consisting of individualized caps and highly portable hardware.

METHODS

Sixteen BCI novices were trained over four consecutive days to imagine left and right hand movements while receiving feedback. The most informative bipolar channels for use on the subsequent days were identified on the first day for each individual based on a high-density online MI recording.

RESULTS

Online classification accuracy on the first day was 85.1% on average (range: 64.7-97.7%). Offline an individually-selected bipolar channel pair based on common spatial patterns significantly outperformed a pair informed by independent component analysis and a standard 10-20 pair. From day 2 to day 4 online MI accuracy increased significantly (day 2: 69.1%; day 4: 73.3%), which was mostly caused by a reduction in ipsilateral event-related desynchronization of sensorimotor rhythms.

CONCLUSION

The present study demonstrates that systematic MI practice in an everyday environment with a user-friendly EEG system results in MI learning effects.

SIGNIFICANCE

These findings help to bridge the gap between elaborate laboratory studies with healthy participants and efficient home or hospital based MI neurofeedback protocols.

Learning to modulate one's own brain activity: the effect of spontaneous mental strategies

Using neurofeedback (NF), individuals can learn to modulate their own brain activity, in most cases electroencephalographic (EEG) rhythms. Although a large body of literature reports positive effects of NF training on behavior and cognitive functions, there are hardly any reports on how participants can successfully learn to gain control over their own brain activity. About one third of people fail to gain significant control over their brain signals even after repeated training sessions. The reasons for this failure are still largely unknown. In this context, we investigated the effects of spontaneous mental strategies on NF performance. Twenty healthy participants performed either a SMR (sensorimotor rhythm, 12–15 Hz) based or a Gamma (40–43 Hz) based NF training over ten sessions. After the first and the last training session, they were asked to write down which mental strategy they have used for self-regulating their EEG. After the first session, all participants reported the use of various types of mental strategies such as visual strategies, concentration, or relaxation. After the last NF training session, four participants of the SMR group reported to employ no specific strategy. These four participants showed linear improvements in NF performance over the ten training sessions. In contrast, participants still reporting the use of specific mental strategies in the last NF session showed no changes in SMR based NF performance over the ten sessions. This effect could not be observed in the Gamma group. The Gamma group showed no prominent changes in Gamma power over the NF training sessions, regardless of the mental strategies used. These results indicate that successful SMR based NF performance is associated with implicit learning mechanisms. Participants stating vivid reports on strategies to control their SMR probably overload cognitive resources, which might be counterproductive in terms of increasing SMR power.

Whatever works: a systematic user-centered training protocol to optimize brain-computer interfacing individually

This study implemented a systematic user-centered training protocol for a 4-class brain-computer interface (BCI). The goal was to optimize the BCI individually in order to achieve high performance within few sessions for all users. Eight able-bodied volunteers, who were initially naïve to the use of a BCI, participated in 10 sessions over a period of about 5 weeks. In an initial screening session, users were asked to perform the following seven mental tasks while multi-channel EEG was recorded: mental rotation, word association, auditory imagery, mental subtraction, spatial navigation, motor imagery of the left hand and motor imagery of both feet. Out of these seven mental tasks, the best 4-class combination as well as most reactive frequency band (between 8-30 Hz) was selected individually for online control. Classification was based on common spatial patterns and Fisher's linear discriminant analysis. The number and time of classifier updates varied individually. Selection speed was increased by reducing trial length. To minimize differences in brain activity between sessions with and without feedback, sham feedback was provided in the screening and calibration runs in which usually no real-time feedback is shown. Selected task combinations and frequency ranges differed between users. The tasks that were included in the 4-class combination most often were (1) motor imagery of the left hand (2), one brain-teaser task (word association or mental subtraction) (3), mental rotation task and (4) one more dynamic imagery task (auditory imagery, spatial navigation, imagery of the feet). Participants achieved mean performances over sessions of 44-84% and peak performances in single-sessions of 58-93% in this user-centered 4-class BCI protocol. This protocol is highly adjustable to individual users and thus could increase the percentage of users who can gain and maintain BCI control. A high priority for future work is to examine this protocol with severely disabled users.

Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design

While recent research on Brain-Computer Interfaces (BCI) has highlighted their potential for many applications, they remain barely used outside laboratories. The main reason is their lack of robustness. Indeed, with current BCI, mental state recognition is usually slow and often incorrect. Spontaneous BCI (i.e., mental imagery-based BCI) often rely on mutual learning efforts by the user and the machine, with BCI users learning to produce stable ElectroEncephaloGraphy (EEG) patterns (spontaneous BCI control being widely acknowledged as a skill) while the computer learns to automatically recognize these EEG patterns, using signal processing. Most research so far was focused on signal processing, mostly neglecting the human in the loop. However, how well the user masters the BCI skill is also a key element explaining BCI robustness. Indeed, if the user is not able to produce stable and distinct EEG patterns, then no signal processing algorithm would be able to recognize them. Unfortunately, despite the importance of BCI training protocols, they have been scarcely studied so far, and used mostly unchanged for years. In this paper, we advocate that current human training approaches for spontaneous BCI are most likely inappropriate. We notably study instructional design literature in order to identify the key requirements and guidelines for a successful training procedure that promotes a good and efficient skill learning. This literature study highlights that current spontaneous BCI user training procedures satisfy very few of these requirements and hence are likely to be suboptimal. We therefore identify the flaws in BCI training protocols according to instructional design principles, at several levels: in the instructions provided to the user, in the tasks he/she has to perform, and in the feedback provided. For each level, we propose new research directions that are theoretically expected to address some of these flaws and to help users learn the BCI skill more efficiently.