Toward a hybrid brain-computer interface based on imagined movement and visual attention

Beta bursts question the ruling power for brain-computer interfaces

Objective: Current efforts to build reliable brain-computer interfaces (BCI) span multiple axes from hardware, to software, to more sophisticated experimental protocols, and personalized approaches. However, despite these abundant efforts, there is still room for significant improvement. We argue that a rather overlooked direction lies in linking BCI protocols with recent advances in fundamental neuroscience.Approach: In light of these advances, and particularly the characterization of the burst-like nature of beta frequency band activity and the diversity of beta bursts, we revisit the role of beta activity in 'left vs. right hand' motor imagery (MI) tasks. Current decoding approaches for such tasks take advantage of the fact that MI generates time-locked changes in induced power in the sensorimotor cortex and rely on band-passed power changes in single or multiple channels. Although little is known about the dynamics of beta burst activity during MI, we hypothesized that beta bursts should be modulated in a way analogous to their activity during performance of real upper limb movements.Main results and Significance: We show that classification features based on patterns of beta burst modulations yield decoding results that are equivalent to or better than typically used beta power across multiple open electroencephalography datasets, thus providing insights into the specificity of these bio-markers.

The clinical effects of brain-computer interface with robot on upper-limb function for post-stroke rehabilitation: a meta-analysis and systematic review

PURPOSE

Many recent clinical studies have suggested that the combination of brain-computer interfaces (BCIs) can induce neurological recovery and improvement in motor function. In this review, we performed a systematic review and meta-analysis to evaluate the clinical effects of BCI-robot systems.

METHODS

The articles published from January 2010 to December 2020 have been searched by using the databases (EMBASE, PubMed, CINAHL, EBSCO, Web of Science and manual search). The single-group studies were qualitatively described, and only the controlled-trial studies were included for the meta-analysis. The mean difference (MD) of Fugl-Meyer Assessment (FMA) scores were pooled and the random-effects model method was used to perform the meta-analysis. The PRISMA criteria were followed in current review.

RESULTS

A total of 897 records were identified, eight single-group studies and 11 controlled-trial studies were included in our review. The systematic analysis indicated that the BCI-robot systems had a significant improvement on motor function recovery. The meta-analysis showed there were no statistic differences between BCI-robot groups and robot groups, neither in the immediate effects nor long-term effects (p > 0.05).

CONCLUSION

The use of BCI-robot systems has significant improvement on the motor function recovery of hemiparetic upper-limb, and there is a sustaining effect. The meta-analysis showed no statistical difference between the experimental group (BCI-robot) and the control group (robot). However, there are a few shortcomings in the experimental design of existing studies, more clinical trials need to be conducted, and the experimental design needs to be more rigorous.Implications for RehabilitationIn this review, we evaluated the clinical effects of brain-computer interface with robot on upper-limb function for post-stroke rehabilitation. After we screened the databases, 19 articles were included in this review. These articles all clinical trial research, they all used non-invasive brain-computer interfaces and upper-limb robot.We conducted the systematic review with nine articles, the result indicated that the BCI-robot system had a significant improvement on motor function recovery. Eleven articles were included for the meta-analysis, the result showed there were no statistic differences between BCI-robot groups and robot groups, neither in the immediate effects nor long-term effects.We thought the result of meta-analysis which showed no statistic difference was probably caused by the heterogenicity of clinical trial designs of these articles.We thought the BCI-robot systems are promising strategies for post-stroke rehabilitation. And we gave several suggestions for further research: (1) The experimental design should be more rigorous, and describe the experimental designs in detail, especially the control group intervention, to make the experiment replicability. (2) New evaluation criteria need to be established, more objective assessment such as biomechanical assessment, fMRI should be utilised as the primary outcome. (3) More clinical studies with larger sample size, novel external devices, and BCI systems need to be conducted to investigate the differences between BCI-robot system and other interventions. (4) Further research could shift the focus to the patients who are in subacute stage, to explore if the early BCI training can make a positive impact on cerebral cortical recovery.

Quantification of Active Visual Attention using RGB camera

Active visual attention (AVA) is the cognitive ability that helps to focus on important visual information while responding to a stimulus and is important for human-behavior and psychophysiological research. Existing eye-trackers/camera-based methods are either expensive or impose privacy issues as face videos are recorded for analysis. Proposed approach using blink-rate variability (BRV), is inexpensive, easy to implement, efficient and handles privacy issues, making it amenable to real-time applications. Our solution uses laptop camera/webcams and a single blink feature, namely BRV. First, we estimated participant's head pose to check camera alignment and detect if he is looking at the screen. Next, subject-specific threshold is computed using eye aspect ratio (EAR) to detect blinks from which BRV signal is constructed. Only EAR values are saved, and participant's face video is NOT saved or transmitted. Finally, a novel AVA score is computed. Results shows that the proposed score is robust across participants, ambient light conditions and occlusions like spectacles.

Motor Imagery Classification for Brain Computer Interface Using Deep Convolutional Neural Networks and Mixup Augmentation

Goal: Building a DL model that can be trained on small EEG training set of a single subject presents an interesting challenge that this work is trying to address. In particular, this study is trying to avoid the need for long EEG data collection sessions, and without combining multiple subjects training datasets, which has a detrimental effect on the classification performance due to the inter-individual variability among subjects. Methods: A customized Convolutional Neural Network with mixup augmentation was trained with [Formula: see text]120 EEG trials for only one subject per model. Results: Modified ResNet18 and DenseNet121 models with mixup augmentation achieved 0.920 (95% Confidence Interval: 0.908, 0.933) and 0.933 (95% Confidence Interval: 0.922, 0.945) classification accuracy, respectively. Conclusions: We show that the designed classifiers resulted in a higher classification performance in comparison to other DL classifiers of previous studies on the same dataset, despite the limited training dataset used in this work.

EEG hybrid brain-computer interfaces: A scoping review applying an existing hybrid-BCI taxonomy and considerations for pediatric applications

Most hybrid brain-computer interfaces (hBCI) aim at improving the performance of single-input BCI. Many combinations are possible to configure an hBCI, such as using multiple brain input signals, different stimuli or more than one input system. Multiple studies have been done since 2010 where such interfaces have been tested and analyzed. Results and conclusions are promising but little has been discussed as to what is the best approach for the pediatric population, should they use hBCI as an assistive technology. Children might face greater challenges when using BCI and might benefit from less complex interfaces. Hence, in this scoping review we included 42 papers that developed hBCI systems for the purpose of control of assistive devices or communication software, and we analyzed them through the lenses of potential use in clinical settings and for children. We extracted taxonomic categories proposed in previous studies to describe the types of interfaces that have been developed. We also proposed interface characteristics that could be observed in different hBCI, such as type of target, number of targets and number of steps before selection. Then, we discussed how each of the extracted characteristics could influence the overall complexity of the system and what might be the best options for applications for children. Effectiveness and efficiency were also collected and included in the analysis. We concluded that the least complex hBCI interfaces might involve having a brain inputs and an external input, with a sequential role of operation, and visual stimuli. Those interfaces might also use a minimal number of targets of the strobic type, with one or two steps before the final selection. We hope this review can be used as a guideline for future hBCI developments and as an incentive to the design of interfaces that can also serve children who have motor impairments.

A Novel Hybrid Brain-Computer Interface Combining Motor Imagery and Intermodulation Steady-State Visual Evoked Potential

The hybrid brain-computer interface (hBCI) combining motor imagery (MI) and steady-state visual evoked potential (SSVEP) has been proven to have better performance than a pure MI- or SSVEP-based brain-computer interface (BCI). In most studies on hBCIs, subjects have been required to focus their attention on flickering light-emitting diodes (LEDs) or blocks while imagining body movements. However, these two classical tasks performed concurrently have a poor correlation. Therefore, it is necessary to reduce the task complexity of such a system and improve its user-friendliness. Aiming to achieve this goal, this study proposes a novel hybrid BCI that combines MI and intermodulation SSVEPs. In the proposed system, images of both hands flicker at the same frequency (i.e., 30 Hz) but at different grasp frequencies (i.e., 1 Hz for the left hand, and 1.5 Hz for the right hand), resulting in different intermodulation frequencies for encoding targets. Additionally, movement observation for subjects can help to perform the MI task better. In this study, two types of brain signals are classified independently and then fused by a scoring mechanism based on the probability distribution of relevant parameters. The online verification results showed that the average accuracies of 12 healthy subjects and 11 stroke patients were 92.40 ± 7.45% and 73.07 ± 9.07%, respectively. The average accuracies of 10 healthy subjects in the MI, SSVEP, and hybrid tasks were 84.00 ± 12.81%, 80.75 ± 8.08%, and 89.00 ± 9.94%, respectively. The high recognition accuracy verifies the feasibility and robustness of the proposed system. This study provides a novel and natural paradigm for a hybrid BCI based on MI and SSVEP.

Influence of Auditory Cues on the Neuronal Response to Naturalistic Visual Stimuli in a Virtual Reality Setting

Virtual reality environments offer great opportunities to study the performance of brain-computer interfaces (BCIs) in real-world contexts. As real-world stimuli are typically multimodal, their neuronal integration elicits complex response patterns. To investigate the effect of additional auditory cues on the processing of visual information, we used virtual reality to mimic safety-related events in an industrial environment while we concomitantly recorded electroencephalography (EEG) signals. We simulated a box traveling on a conveyor belt system where two types of stimuli – an exploding and a burning box – interrupt regular operation. The recordings from 16 subjects were divided into two subsets, a visual-only and an audio-visual experiment. In the visual-only experiment, the response patterns for both stimuli elicited a similar pattern – a visual evoked potential (VEP) followed by an event-related potential (ERP) over the occipital-parietal lobe. Moreover, we found the perceived severity of the event to be reflected in the signal amplitude. Interestingly, the additional auditory cues had a twofold effect on the previous findings: The P1 component was significantly suppressed in the case of the exploding box stimulus, whereas the N2c showed an enhancement for the burning box stimulus. This result highlights the impact of multisensory integration on the performance of realistic BCI applications. Indeed, we observed alterations in the offline classification accuracy for a detection task based on a mixed feature extraction (variance, power spectral density, and discrete wavelet transform) and a support vector machine classifier. In the case of the explosion, the accuracy slightly decreased by –1.64% p. in an audio-visual experiment compared to the visual-only. Contrarily, the classification accuracy for the burning box increased by 5.58% p. when additional auditory cues were present. Hence, we conclude, that especially in challenging detection tasks, it is favorable to consider the potential of multisensory integration when BCIs are supposed to operate under (multimodal) real-world conditions.

Continuous Hybrid BCI Control for Robotic Arm Using Noninvasive Electroencephalogram, Computer Vision, and Eye Tracking

The controlling of robotic arms based on brain–computer interface (BCI) can revolutionize the quality of life and living conditions for individuals with physical disabilities. Invasive electroencephalography (EEG)-based BCI has been able to control multiple degrees of freedom (DOFs) robotic arms in three dimensions. However, it is still hard to control a multi-DOF robotic arm to reach and grasp the desired target accurately in complex three-dimensional (3D) space by a noninvasive system mainly due to the limitation of EEG decoding performance. In this study, we propose a noninvasive EEG-based BCI for a robotic arm control system that enables users to complete multitarget reach and grasp tasks and avoid obstacles by hybrid control. The results obtained from seven subjects demonstrated that motor imagery (MI) training could modulate brain rhythms, and six of them completed the online tasks using the hybrid-control-based robotic arm system. The proposed system shows effective performance due to the combination of MI-based EEG, computer vision, gaze detection, and partially autonomous guidance, which drastically improve the accuracy of online tasks and reduce the brain burden caused by long-term mental activities.

An Impending Paradigm Shift in Motor Imagery Based Brain-Computer Interfaces

The development of reliable assistive devices for patients that suffer from motor impairments following central nervous system lesions remains a major challenge in the field of non-invasive Brain-Computer Interfaces (BCIs). These approaches are predominated by electroencephalography and rely on advanced signal processing and machine learning methods to extract neural correlates of motor activity. However, despite tremendous and still ongoing efforts, their value as effective clinical tools remains limited. We advocate that a rather overlooked research avenue lies in efforts to question neurophysiological markers traditionally targeted in non-invasive motor BCIs. We propose an alternative approach grounded by recent fundamental advances in non-invasive neurophysiology, specifically subject-specific feature extraction of sensorimotor bursts of activity recorded via (possibly magnetoencephalography-optimized) electroencephalography. This path holds promise in overcoming a significant proportion of existing limitations, and could foster the wider adoption of online BCIs in rehabilitation protocols.

Phase-spatial beamforming renders a visual brain computer interface capable of exploiting EEG electrode phase shifts in motion-onset target responses

Brain-computer interfaces (BCIs) provide communication facilities that do not rely on the brains usual pathways. Visual BCIs are based on changes in EEG activity in response to attended flashing or flickering targets. A less taxing way to encode such targets is with briefly moving stimuli, the onset of which elicits a lateralized EEG potential over the parieto-occipital scalp area called the motion-onset visual evoked potential (mVEP). We recruited 21 healthy subjects for an experiment in which motion-onset stimulations translating leftwards (LT) or rightwards (RT) were encoding 9 displayed targets. We propose a novel algorithm that exploits the phase-shift between EEG electrodes to improve target decoding performance. We hereto extend the spatiotemporal beamformer (stBF) with a phase extracting procedure, leading to the phase-spatial beamformer (psBF). We show that psBF performs significantly better than the stBF (p<0.001 for 1 and 2 stimulus repetitions and p<0.01 for 3 to 5 stimulus repetitions), as well as the previously validated linear support-vector machines (p<0.001 for 5 stimulus repetitions and p<0.01 for 1,2 and 6 stimulus repetitions) and stepwise linear discriminant analysis decoders (p<0.001 for all repetitions) when simultaneously addressing timing and translation direction.

Reducing False Triggering Caused by Irrelevant Mental Activities in Brain-Computer Interface Based on Motor Imagery

In recent years, the brain-computer interface (BCI) based on motor imagery (MI) has been considered as a potential post-stroke rehabilitation technology. However, the recognition of MI relies on the event-related desynchronization (ERD) feature, which has poor task specificity. Further, there is the problem of false triggering (irrelevant mental activities recognized as the MI of the target limb). In this paper, we discuss the feasibility of reducing the false triggering rate using a novel paradigm, in which the steady-state somatosensory evoked potential (SSSEP) is combined with the MI (MI-SSSEP). Data from the target (right hand MI) and nontarget task (rest) were used to establish the recognition model, and three kinds of interference tasks were used to test the false triggering performance. In the MI-SSSEP paradigm, ERD and SSSEP features modulated by MI could be used for recognition, while in the MI paradigm, only ERD features could be used. The results showed that the false triggering rate of interference tasks with SSSEP features was reduced to 29.3%, which was far lower than the 55.5% seen under the MI paradigm with ERD features. Moreover, in the MI-SSSEP paradigm, the recognition rate of the target and nontarget task was also significantly improved. Further analysis showed that the specificity of SSSEP was significantly higher than that of ERD (p < 0.05), but the sensitivity was not significantly different. These results indicated that SSSEP modulated by MI could more specifically decode the target task MI, and thereby may have potential in achieving more accurate rehabilitation training.

Review of brain encoding and decoding mechanisms for EEG-based brain-computer interface

A brain-computer interface (BCI) can connect humans and machines directly and has achieved successful applications in the past few decades. Many new BCI paradigms and algorithms have been developed in recent years. Therefore, it is necessary to review new progress in BCIs. This paper summarizes progress for EEG-based BCIs from the perspective of encoding paradigms and decoding algorithms, which are two key elements of BCI systems. Encoding paradigms are grouped by their underlying neural meachanisms, namely sensory- and motor-related, vision-related, cognition-related and hybrid paradigms. Decoding algorithms are reviewed in four categories, namely decomposition algorithms, Riemannian geometry, deep learning and transfer learning. This review will provide a comprehensive overview of both modern primary paradigms and algorithms, making it helpful for those who are developing BCI systems.

A Multi-DoF Prosthetic Hand Finger Joint Controller for Wearable sEMG Sensors by Nonlinear Autoregressive Exogenous Model

The loss of mobility function and sensory information from the arm, hand, and fingertips hampers the activities of daily living (ADL) of patients. A modern bionic prosthetic hand can compensate for the lost functions and realize multiple degree of freedom (DoF) movements. However, the commercially available prosthetic hands usually have limited DoFs due to limited sensors and lack of stable classification algorithms. This study aimed to propose a controller for finger joint angle estimation by surface electromyography (sEMG). The sEMG data used for training were gathered with the Myo armband, which is a commercial EMG sensor. Two features in the time domain were extracted and fed into a nonlinear autoregressive model with exogenous inputs (NARX). The NARX model was trained with pre-selected parameters using the Levenberg-Marquardt algorithm. Comparing with the targets, the regression correlation coefficient (R) of the model outputs was more than 0.982 over all test subjects, and the mean square error was less than 10.02 for a signal range in arbitrary units equal to [0, 255]. The study also demonstrated that the proposed model could be used in daily life movements with good accuracy and generalization abilities.

Exploration of illusory visual motion stimuli: An EEG-based brain-computer interface for practical assistive communication systems

This paper presents an illusory visual motion stimulus-based brain-computer interface (BCI). We aim to use the proposed system to enhance the motor imagery (MI) modality. Since motor imagery requires a long time for training, a stimulation method with external stimuli through the sensory system is an alternative method for increasing efficiency. The research is divided into two parts. First, we observed the visual motion illusion pattern based on brain topographic maps for the novel BCI modality. Second, we implemented the illusory visual motion stimulus-based BCI system. Arrow and moving-arrow patterns were used to modulate alpha rhythms at the visual and motor cortex. The arrow pattern had an average classification accuracy of approximately 78.5%. Additionally, illusory visual motion stimulus-based BCI systems are proposed using the proposed feature extraction and decision-making algorithm. This proposed BCI system can control the cursor moving in the left or right direction with the designed algorithm to create five commands for assistive communication. Ten volunteers participated in the experiment, and a brain-computer interface system with motor imagery and an illusory visual motion stimulus were used to compare efficiencies. The results showed that the proposed method achieved approximately 4% higher accuracy than motor imagery. The accuracy of the proposed illusory visual motion stimulus and algorithm was approximately 80.3%. Therefore, an illusory visual motion stimulus hybrid BCI system can be incorporated into the MI-based BCI system for beginner motor imagery. Based on the results, the proposed assistive communication system can be used to enhance communication in people with severe disabilities.

Progress in Brain Computer Interface: Challenges and Opportunities

Brain computer interfaces (BCI) provide a direct communication link between the brain and a computer or other external devices. They offer an extended degree of freedom either by strengthening or by substituting human peripheral working capacity and have potential applications in various fields such as rehabilitation, affective computing, robotics, gaming, and neuroscience. Significant research efforts on a global scale have delivered common platforms for technology standardization and help tackle highly complex and non-linear brain dynamics and related feature extraction and classification challenges. Time-variant psycho-neurophysiological fluctuations and their impact on brain signals impose another challenge for BCI researchers to transform the technology from laboratory experiments to plug-and-play daily life. This review summarizes state-of-the-art progress in the BCI field over the last decades and highlights critical challenges.

EEG-based hybrid QWERTY mental speller with high information transfer rate

BACKGROUND

Brain-computer interface (BCI) spellers detect variations in brain waves to help subjects communicate with the world. This study introduces a P300-SSVEP hybrid BCI-based QWERTY speller.

METHODS

The proposed hybrid speller, combines SSVEP and P300 features using a hybrid paradigm. P300 was used as time division multiplexing index which results in the use of lesser number of assumed frequencies for SSVEP elicitation. Each flickering frequency was also assigned a unique colour, to enhance system accuracy.

RESULTS

On the basis of 20 subjects, an average accuracy of classification of 96.42% and a mean information transfer rate (ITR) of 131.0 bits per min. (BPM) was achieved during the free spelling trial (trial-F).

COMPARISON

The t test results revealed that the hybrid QWERTY speller performed significantly better (on the basis of mean classification accuracy and ITR) as compared to the traditional P300 speller) and the QWERTY SSVEP speller. Also, the amount of time taken to spell a word was significantly lesser in the case of hybrid QWERTY speller in contrast to traditional P300 speller while it was almost the same as compared to QWERTY SSVEP speller.

CONCLUSION

QWERTY speller outperformed the stereotypical P300 speller as well as QWERTY SSVEP speller.

Closed-Loop Phase-Dependent Vibration Stimulation Improves Motor Imagery-Based Brain-Computer Interface Performance

The motor imagery (MI) paradigm has been wildly used in brain-computer interface (BCI), but the difficulties in performing imagery tasks limit its application. Mechanical vibration stimulus has been increasingly used to enhance the MI performance, but its improvement consistence is still under debate. To develop more effective vibration stimulus methods for consistently enhancing MI, this study proposes an EEG phase-dependent closed-loop mechanical vibration stimulation method. The subject's index finger of the non-dominant hand was given 4 different vibration stimulation conditions (i.e., continuous open-loop vibration stimulus, two different phase-dependent closed-loop vibration stimuli and no stimulus) when performing two tasks of imagining movement and rest of the index finger from his/her dominant hand. We compared MI performance and brain oscillatory patterns under different conditions to verify the effectiveness of this method. The subjects performed 80 trials of each type in a random order, and the average phase-lock value of closed-loop stimulus conditions was 0.71. It was found that the closed-loop vibration stimulus applied in the falling phase helped the subjects to produce stronger event-related desynchronization (ERD) and sustain longer. Moreover, the classification accuracy was improved by about 9% compared with MI without any vibration stimulation (p = 0.012, paired t-test). This method helps to modulate the mu rhythm and make subjects more concentrated on the imagery and without negative enhancement during rest tasks, ultimately improves MI-based BCI performance. Participants reported that the tactile fatigue under closed-loop stimulation conditions was significantly less than continuous stimulation. This novel method is an improvement to the traditional vibration stimulation enhancement research and helps to make stimulation more precise and efficient.

Hybrid control approaches for hands-free high level human-computer interface-a review

For more than a decade, more number of human-machine interfaces had been developed by various combination of user inputs such as speech, hand and head gestures, eye gaze and body movements, etc. And many research issues have been addressed, including facial expression recognition, human emotion analysis, speech recognition/synthesis, human-computer interaction, virtual reality and augmented reality interaction, etc. As a result, the development of a hybrid approach becomes a central issue for hands-free high-level human computer, to help elderly and disabled people. They characterise the user's preferred communication style and support user's ability to flexibly combine modes or to switch from one input mode to another that may be better suited to a particular task or setting. This review discusses the various hybrid control approaches of hands-free high level human-computer interface.

MI3DNet: A Compact CNN for Motor Imagery EEG Classification with Visualizable Dense Layer Parameters

Electroencephalography (EEG) based Brain Computer Interface (BCI) attracts more and more attention. Motor Imagery (MI) is a popular one among all the EEG paradigms. Building a subject-independent MI EEG classification procedure is a main challenge in practical applications. Recently, Convolutional Neural Network (CNN) has been introduced and achieved state-of-the-art performance in related areas. To extract subject-independent features in MI EEG classification, we propose the MI3DNet, using a remapped signal cubic as the input. Experiments show that MI3DNet has a higher performance with fewer parameters and layers. We also give a method to plot the parameters of the dense layer, and explain its effect.

Learning in brain-computer interface control evidenced by joint decomposition of brain and behavior

OBJECTIVE

Motor imagery-based brain-computer interfaces (BCIs) use an individual's ability to volitionally modulate localized brain activity, often as a therapy for motor dysfunction or to probe causal relations between brain activity and behavior. However, many individuals cannot learn to successfully modulate their brain activity, greatly limiting the efficacy of BCI for therapy and for basic scientific inquiry. Formal experiments designed to probe the nature of BCI learning have offered initial evidence that coherent activity across spatially distributed and functionally diverse cognitive systems is a hallmark of individuals who can successfully learn to control the BCI. However, little is known about how these distributed networks interact through time to support learning.

APPROACH

Here, we address this gap in knowledge by constructing and applying a multimodal network approach to decipher brain-behavior relations in motor imagery-based brain-computer interface learning using magnetoencephalography. Specifically, we employ a minimally constrained matrix decomposition method - non-negative matrix factorization - to simultaneously identify regularized, covarying subgraphs of functional connectivity, to assess their similarity to task performance, and to detect their time-varying expression.

MAIN RESULTS

We find that learning is marked by diffuse brain-behavior relations: good learners displayed many subgraphs whose temporal expression tracked performance. Individuals also displayed marked variation in the spatial properties of subgraphs such as the connectivity between the frontal lobe and the rest of the brain, and in the temporal properties of subgraphs such as the stage of learning at which they reached maximum expression. From these observations, we posit a conceptual model in which certain subgraphs support learning by modulating brain activity in sensors near regions important for sustaining attention. To test this model, we use tools that stipulate regional dynamics on a networked system (network control theory), and find that good learners display a single subgraph whose temporal expression tracked performance and whose architecture supports easy modulation of sensors located near brain regions important for attention.

SIGNIFICANCE

The nature of our contribution to the neuroscience of BCI learning is therefore both computational and theoretical; we first use a minimally-constrained, individual specific method of identifying mesoscale structure in dynamic brain activity to show how global connectivity and interactions between distributed networks supports BCI learning, and then we use a formal network model of control to lend theoretical support to the hypothesis that these identified subgraphs are well suited to modulate attention.

Review on motor imagery based BCI systems for upper limb post-stroke neurorehabilitation: From designing to application

Strokes are a growing cause of mortality and many stroke survivors suffer from motor impairment as well as other types of disabilities in their daily life activities. To treat these sequelae, motor imagery (MI) based brain-computer interface (BCI) systems have shown potential to serve as an effective neurorehabilitation tool for post-stroke rehabilitation therapy. In this review, different MI-BCI based strategies, including "Functional Electric Stimulation, Robotics Assistance and Hybrid Virtual Reality based Models," have been comprehensively reported for upper-limb neurorehabilitation. Each of these approaches have been presented to illustrate the in-depth advantages and challenges of the respective BCI systems. Additionally, the current state-of-the-art and main concerns regarding BCI based post-stroke neurorehabilitation devices have also been discussed. Finally, recommendations for future developments have been proposed while discussing the BCI neurorehabilitation systems.

Alpha Frequency Intervention by Electrical Stimulation to Improve Performance in Mu-Based BCI

The accuracy of brain-computer interfaces (BCIs) is important for effective communication and control. The mu-based BCI is one of the most widely used systems, of which the related methods to improve users' accuracy are still poorly studied, especially for the BCI illiteracy. Here, we examined a way to enhance mu-based BCI performance by electrically stimulating the ulnar nerve of the contralateral wrist at the alpha frequency (10 Hz) during left- and right-hand motor imagination in two BCI groups (literate and illiterate). We demonstrate that this alpha frequency intervention enhances the classification accuracy between left- and right-hand motor imagery from 66.41% to 81.57% immediately after intervention and to 75.28% two days after intervention in the BCI illiteracy group, while classification accuracy improves from 82.12% to 91.84% immediately after intervention and to 89.03% two days after intervention in the BCI literacy group. However, the classification accuracy did not change before and after the sham intervention (no electrical stimulation). Furthermore, the ERD on the primary sensorimotor cortex during left- or right-hand motor imagery tasks was more visible at the mu-rhythm (8-13 Hz) after alpha frequency intervention. Alpha frequency intervention increases the mu-rhythm power difference between left- and right-hand motor imagery tasks. These results provide evidence that alpha frequency intervention is an effective way to improve BCI performance by regulating the mu-rhythm which might provide a way to reduce BCI illiteracy.

Mental state space visualization for interactive modeling of personalized BCI control strategies

OBJECTIVE

Numerous studies in the area of BCI are focused on the search for a better experimental paradigm-a set of mental actions that a user can evoke consistently and a machine can discriminate reliably. Examples of such mental activities are motor imagery, mental computations, etc. We propose a technique that instead allows the user to try different mental actions in the search for the ones that will work best.

APPROACH

The system is based on a modification of the self-organizing map (SOM) algorithm and enables interactive communication between the user and the learning system through a visualization of user's mental state space. During the interaction with the system the user converges on the paradigm that is most efficient and intuitive for that particular user.

MAIN RESULTS

Results of the two experiments, one allowing muscular activity, another permitting mental activity only, demonstrate soundness of the proposed method and offer preliminary validation of the performance improvement over the traditional closed-loop feedback approach.

SIGNIFICANCE

The proposed method allows a user to visually explore their mental state space in real time, opening new opportunities for scientific inquiry. The application of this method to the area of brain-computer interfaces enables more efficient search for the mental states that will allow a user to reliably control a BCI system.

Classification of EEG Motor Imagery Tasks Using Convolution Neural Networks

Electroencephalograph (EEG) is a highly nonlinear data and very difficult to be classified. The EEG signal is commonly used in the area of Brain-Computer Interface (BCI). The signal can be used as an operative command for directional movements for a powered wheelchair to assist people with disability in performing the daily activity.In this paper, we aim to classify Electroencephalograph EEG signals extracted from subjects which had been trained to perform four Motoric Imagery (MI) tasks for two classes. The classification will be processed via a Convolutional Neural Network (CNN) utilising all 22 electrodes based on 10-20 system placement. The EEG datasets will be transformed into scaleogram using Continuous Wavelet Transform (CWT) method.We evaluated two different types of image configuration, i.e. layered and stacked input datasets. Our procedure starts from denoising the EEG signals, employing Bump CWT from 8-32 Hz brain wave. Our CNN architecture is based on the Visual Geometry Group (VGG-16) network. Our results show that layered image dataset yields a high accuracy with an average of 68.33% for two classes classification.

Physiological Responses During Hybrid BNCI Control of an Upper-Limb Exoskeleton

When combined with assistive robotic devices, such as wearable robotics, brain/neural-computer interfaces (BNCI) have the potential to restore the capabilities of handicapped people to carry out activities of daily living. To improve applicability of such systems, workload and stress should be reduced to a minimal level. Here, we investigated the user's physiological reactions during the exhaustive use of the interfaces of a hybrid control interface. Eleven BNCI-naive healthy volunteers participated in the experiments. All participants sat in a comfortable chair in front of a desk and wore a whole-arm exoskeleton as well as wearable devices for monitoring physiological, electroencephalographic (EEG) and electrooculographic (EoG) signals. The experimental protocol consisted of three phases: (i) Set-up, calibration and BNCI training; (ii) Familiarization phase; and (iii) Experimental phase during which each subject had to perform EEG and EoG tasks. After completing each task, the NASA-TLX questionnaire and self-assessment manikin (SAM) were completed by the user. We found significant differences (p-value < 0.05) in heart rate variability (HRV) and skin conductance level (SCL) between participants during the use of the two different biosignal modalities (EEG, EoG) of the BNCI. This indicates that EEG control is associated with a higher level of stress (associated with a decrease in HRV) and mental work load (associated with a higher level of SCL) when compared to EoG control. In addition, HRV and SCL modulations correlated with the subject's workload perception and emotional responses assessed through NASA-TLX questionnaires and SAM.

Modeling and classification of voluntary and imagery movements for brain-computer interface from fNIR and EEG signals through convolutional neural network

Practical brain-computer interface (BCI) demands the learning-based adaptive model that can handle diverse problems. To implement a BCI, usually functional near-infrared spectroscopy (fNIR) is used for measuring functional changes in brain oxygenation and electroencephalography (EEG) for evaluating the neuronal electric potential regarding the psychophysiological activity. Since the fNIR modality has an issue of temporal resolution, fNIR alone is not enough to achieve satisfactory classification accuracy as multiple neural stimuli are produced by voluntary and imagery movements. This leads us to make a combination of fNIR and EEG with a view to developing a BCI model for the classification of the brain signals of the voluntary and imagery movements. This work proposes a novel approach to prepare functional neuroimages from the fNIR and EEG using eight different movement-related stimuli. The neuroimages are used to train a convolutional neural network (CNN) to formulate a predictive model for classifying the combined fNIR-EEG data. The results reveal that the combined fNIR-EEG modality approach along with a CNN provides improved classification accuracy compared to a single modality and conventional classifiers. So, the outcomes of the proposed research work will be very helpful in the implementation of the finer BCI system.

A Comparison between BCI Simulation and Neurofeedback for Forward/Backward Navigation in Virtual Reality

A brain-computer interface (BCI) decodes the brain signals representing a desire to do something and transforms those signals into a control command. However, only a limited number of mental tasks have been previously investigated and classified. This study aimed to investigate two motor imagery (MI) commands, moving forward and moving backward, using a small number of EEG channels, to be used in a neurofeedback context. This study also aimed to simulate a BCI and investigate the offline classification between MI movements in forward and backward directions, using different features and classification methods. Ten healthy people participated in a two-session (48 min each) experiment. This experiment investigated neurofeedback of navigation in a virtual tunnel. Each session consisted of 320 trials where subjects were asked to imagine themselves moving in the tunnel in a forward or backward motion after a randomly presented (forward versus backward) command on the screen. Three electrodes were mounted bilaterally over the motor cortex. Trials were conducted with feedback. Data from session 1 were analyzed offline to train classifiers and to calculate thresholds for both tasks. These thresholds were used to form control signals that were later used online in session 2 in neurofeedback training to trigger the virtual tunnel to move in the direction requested by the user's brain signals. After 96 min of training, the online band-power neurofeedback training achieved an average classification of 76%, while the offline BCI simulation using power spectral density asymmetrical ratio and AR-modeled band power as features, and using LDA and SVM as classifiers, achieved an average classification of 80%.

Combined Lateral and Longitudinal Control of EEG Signals-Based Brain-Controlled Vehicles

Using brain signals rather than limbs to control a vehicle may not only help persons with disabilities to acquire driving ability, but also provide healthy persons with a new alternative way to drive. In this paper, we propose a combined lateral and longitudinal control system for electroencephalogram (EEG) signals-based brain-controlled vehicles (BCVs). The proposed system is designed by integrating a user interface, a brain-computer interface (BCI), a control interface model, a lateral controller, and a longitudinal controller. We conduct driver-and-hardware-in-the-loop experiments under two control conditions (i.e., the brain- and manual-control conditions) with different subjects and three driving tests (i.e., the lane-changing, path-selection, and car-following tests). Experimental results show the feasibility of using brain signals to continuously perform both the lateral and longitudinal control of a vehicle. This study not only promotes the development of BCVs, but also provides some insights on how to apply BCIs in conjunction with assistant controllers to control other dynamic systems.

Enhancing the Hybrid BCI Performance With the Common Frequency Pattern in Dual-Channel EEG

The brain-computer interface establishes a direct communication pathway between the human brain and an external device by recognizing specific patterns in cortical activities. The principle of hybridization stands for combining at least two different BCI modalities into a single interface with the aim of improving the information transfer rate by increasing the recognition accuracy and number of choices available for the user. This study proposes a simultaneous hybrid BCI system that recognizes the motor imagery (MI) and the steady-state visually evoked potentials (SSVEP) using the EEG signals from a dual-channel EEG setting with sensors placed over the central area (C3 and C4 channels). The data processing implements a supervised optimization algorithm for the feature extraction, named the common frequency pattern, which finds the optimal spectral filter that maximizes the separability of the data by classes. The experiment compares the classification accuracy in a two-class task using the MI, SSVEP and hybrid approaches on seventeen healthy 18-29 years old subjects with various dual-channel setups and complete set of thirty EEG electrodes. The designed system reaches a high accuracy of 97.4 ± 1.1% in the hybrid task using the C3-C4 channel configuration, which is marginally lower than the 98.8 ± 0.5% accuracy achieved with the complete set of channels while applying the support vector classifier; in the plain SSVEP task the accuracy drops from 91.3 ± 3.9% to 86.0 ± 2.5% while moving from the occipital to central area under the dual-channel condition. The results demonstrate that by combining the principles of hybridization and data-driven spectral filtering for the feature selection it is feasible to compensate a lack of spatial information and implement the proposed BCI using a portable few channel EEG device even under sub-optimal conditions for the sensors placement.

Exploring Cognitive Flexibility With a Noninvasive BCI Using Simultaneous Steady-State Visual Evoked Potentials and Sensorimotor Rhythms

EEG-based brain-computer interface (BCI) technology creates non-biological pathways for conveying a user's mental intent solely through noninvasively measured neural signals. While optimizing the performance of a single task has long been the focus of BCI research, in order to translate this technology into everyday life, realistic situations, in which multiple tasks are performed simultaneously, must be investigated. In this paper, we explore the concept of cognitive flexibility, or multitasking, within the BCI framework by utilizing a 2-D cursor control task, using sensorimotor rhythms (SMRs), and a four-target visual attention task, using steady-state visual evoked potentials (SSVEPs), both individually and simultaneously. We found no significant difference between the accuracy of the tasks when executing them alone (SMR-57.9% ± 15.4% and SSVEP-59.0% ± 14.2%) and simultaneously (SMR-54.9% ± 17.2% and SSVEP-57.5% ± 15.4%). These modest decreases in performance were supported by similar, non-significant changes in the electrophysiology of the SSVEP and SMR signals. In this sense, we report that multiple BCI tasks can be performed simultaneously without a significant deterioration in performance; this finding will help drive these systems toward realistic daily use in which a user's cognition will need to be involved in multiple tasks at once.

Quadcopter Flight Control Using a Non-invasive Multi-Modal Brain Computer Interface

Brain-Computer Interfaces (BCIs) translate neuronal information into commands to control external software or hardware, which can improve the quality of life for both healthy and disabled individuals. Here, a multi-modal BCI which combines motor imagery (MI) and steady-state visual evoked potential (SSVEP) is proposed to achieve stable control of a quadcopter in three-dimensional physical space. The complete information common spatial pattern (CICSP) method is used to extract two MI features to control the quadcopter to fly left-forward and right-forward, and canonical correlation analysis (CCA) is employed to perform the SSVEP classification for rise and fall. Eye blinking is designed to switch these two modes while hovering. Real-time feedback is provided to subjects by a global camera. Two flight tasks were conducted in physical space in order to certify the reliability of the BCI system. Subjects were asked to control the quadcopter to fly forward along the zig-zag pattern to pass through a gate in the relatively simple task. For the other complex task, the quadcopter was controlled to pass through two gates successively according to an S-shaped route. The performance of the BCI system is quantified using suitable metrics and subjects are able to acquire 86.5% accuracy for the complicated flight task. It is demonstrated that the multi-modal BCI has the ability to increase the accuracy rate, reduce the task burden, and improve the performance of the BCI system in the real world.

A Review of EEG-Based Brain-Computer Interface Systems Design

A brain-computer interface (BCI) system can recognize the mental activities pattern by computer algorithms to control the external devices. Electroencephalogram (EEG) is one of the most common used approach for BCI due to the convenience and non-invasive implement. Therefore, more and more BCIs have been designed for the disabled people that suffer from stroke or spinal cord injury to help them for rehabilitation and life. We introduce the common BCI paradigms, the signal processing, and feature extraction methods. Then, we survey the different combined modes of hybrids BCIs and review the design of the synchronous/asynchronous BCIs. Finally, the shared control methods are discussed.

An EEG/EMG/EOG-Based Multimodal Human-Machine Interface to Real-Time Control of a Soft Robot Hand

Brain-computer interface (BCI) technology shows potential for application to motor rehabilitation therapies that use neural plasticity to restore motor function and improve quality of life of stroke survivors. However, it is often difficult for BCI systems to provide the variety of control commands necessary for multi-task real-time control of soft robot naturally. In this study, a novel multimodal human-machine interface system (mHMI) is developed using combinations of electrooculography (EOG), electroencephalography (EEG), and electromyogram (EMG) to generate numerous control instructions. Moreover, we also explore subject acceptance of an affordable wearable soft robot to move basic hand actions during robot-assisted movement. Six healthy subjects separately perform left and right hand motor imagery, looking-left and looking-right eye movements, and different hand gestures in different modes to control a soft robot in a variety of actions. The results indicate that the number of mHMI control instructions is significantly greater than achievable with any individual mode. Furthermore, the mHMI can achieve an average classification accuracy of 93.83% with the average information transfer rate of 47.41 bits/min, which is entirely equivalent to a control speed of 17 actions per minute. The study is expected to construct a more user-friendly mHMI for real-time control of soft robot to help healthy or disabled persons perform basic hand movements in friendly and convenient way.

Recent Advances in Hybrid Brain-Computer Interface Systems: A Technological and Quantitative Review

Brain-Computer Interface (BCI) is a system that enables users to transmit commands to the computer using their brain activity recorded by electroencephalography. In a Hybrid Brain-Computer Interface (HBCI), a BCI control signal combines with one or more BCI control signals or with Human-Machine Interface (HMI) biosignals to increase classification accuracy, boost system speed, and improve user’s satisfaction. HBCI systems are categorized according to the type of combined signals and the combination technique (simultaneous or sequential). They have been used in several applications such as cursor control, target selection, and spellers. Increasing the number of articles published in this field indicates the significance of these systems. In this paper, different HBCI combinations, their important features, and potential applications are discussed. In most cases, the combination of a BCI control signal with a HMI biosignal yields higher information transfer rate than two BCI control signals.

Performance of Brain-Computer Interfacing Based on Tactile Selective Sensation and Motor Imagery

A large proportion of users do not achieve adequate control using current non-invasive brain-computer interfaces (BCIs). This issue has being coined "BCI-Illiteracy" and is observed among different BCI modalities. Here, we compare the performance and the BCI-illiteracy rate of a tactile selective sensation (SS) and motor imagery (MI) BCI, for a large subject samples. We analyzed 80 experimental sessions from 57 subjects with two-class SS protocols. For SS, the group average performance was 79.8 ± 10.6%, with 43 out of the 57 subjects (75.4%) exceeding the 70% BCI-illiteracy threshold for left- and right-hand SS discrimination. When compared with previous results, this tactile BCI outperformed all other tactile BCIs currently available. We also analyzed 63 experimental sessions from 43 subjects with two-class MI BCI protocols, where the group average performance was 77.2 ± 13.3%, with 69.7% of the subjects exceeding the 70% performance threshold for left- and right-hand MI. For within-subject comparison, the 24 subjects who participated to both the SS and MI experiments, the BCI performance was superior with SS than MI especially in beta frequency band (p < 0.05), with enhanced R² discriminative information in the somatosensory cortex for the SS modality. Both SS and MI showed a functional dissociation between lower alpha ([8 10] Hz) and upper alpha ([10 13] Hz) bands, with BCI performance significantly better in the upper alpha than the lower alpha (p < 0.05) band. In summary, we demonstrated that SS is a promising BCI modality with low BCI illiteracy issue and has great potential in practical applications reaching large population.

Multiple-command single-frequency SSVEP-based BCI system using flickering action video

BACKGROUND

The number of commands in a brain-computer interface (BCI) system is important. This study proposes a new BCI technique to increase the number of commands in a single BCI system without loss of accuracy.

NEW METHOD

We expected that a flickering action video with left and right elbow movements could simultaneously activate the different pattern of event-related desynchronization (ERD) according to the video contents (e.g., left or right) and steady-state visually evoked potential (SSVEP). The classification accuracy to discriminate left, right, and rest states was compared under the three following feature combinations: SSVEP power (19-21 Hz), Mu power (8-13 Hz), and simultaneous SSVEP and Mu power.

RESULTS

The SSVEP feature could discriminate the stimulus condition, regardless of left or right, from the rest condition, while the Mu feature discriminated left or right, but was relatively poor in discriminating stimulus from rest. However, combining the SSVEP and Mu features, which were evoked by the stimulus with a single frequency, showed superior performance for discriminating all the stimuli among rest, left, or right.

COMPARISON WITH THE EXISTING METHOD

The video contents could activate the ERD differently, and the flickering component increased its accuracy, such that it revealed a better performance to discriminate when considering together.

CONCLUSIONS

This paradigm showed possibility of increasing performance in terms of accuracy and number of commands with a single frequency by applying flickering action video paradigm and applicability to rehabilitation systems used by patients to facilitate their mirror neuron systems while training.

EEG-Based Brain-Computer Interfaces

Brain-computer interfaces (BCIs) provide a direct communication channel between human brain and output devices. Due to advantages such as non-invasiveness, ease of use, and low cost, electroencephalography (EEG) is the most popular method for current BCIs. This chapter gives an overview of the current EEG-based BCIs for the main purpose of communication and control. This chapter first provides a taxonomy of the EEG-based BCI systems by categorizing them into three major groups: (1) BCIs based on event-related potentials (ERPs), (2) BCIs based on sensorimotor rhythms, and (3) hybrid BCIs. Next, this chapter describes challenges and potential solutions in developing practical BCI systems toward high communication speed, convenient system use, and low user variation. Then this chapter briefly reviews both medical and non-medical applications of current BCIs. Finally, this chapter concludes with a summary of current stage and future perspectives of the EEG-based BCI technology.

Quasi-periodic fluctuation in Donchin's speller signals and its potential use for asynchronous control

When we examine the event-related potential (ERP) responses of Donchin's brain-computer interface (BCI) speller, a type of quasi-periodic fluctuation (FLUC) overlapping with the ERP components can be observed; this fluctuation is traditionally treated as interference. However, if the FLUC is detectable in a working BCI, it can be used for asynchronous control, i.e. to indicate whether the BCI is under the control state (CS) or under the non-control idle state (NC). Asynchronous control is an important issue to address to enable BCI's practical use. In this paper, we examine the characteristics of the FLUC and explore the possibility of using the FLUC for asynchronous control of the BCI. For detecting the FLUC, we propose a method based on the power spectrum and evaluate the detection rates in a simulation. As a result, high true positive rates (TPRs) and low false positive rates (FPRs) are obtained. Our work reveals that the FLUC is of great value for implementing an asynchronous BCI.

Frequency Specific Cortical Dynamics During Motor Imagery Are Influenced by Prior Physical Activity

Motor imagery is often used inducing changes in electroencephalographic (EEG) signals for imagery-based brain-computer interfacing (BCI). A BCI is a device translating brain signals into control signals providing severely motor-impaired persons with an additional, non-muscular channel for communication and control. In the last years, there is increasing interest using BCIs also for healthy people in terms of enhancement or gaming. Most studies focusing on improving signal processing feature extraction and classification methods, but the performance of a BCI can also be improved by optimizing the user's control strategies, e.g., using more vivid and engaging mental tasks for control. We used multichannel EEG to investigate neural correlates of a sports imagery task (playing tennis) compared to a simple motor imagery task (squeezing a ball). To enhance the vividness of both tasks participants performed a short physical exercise between two imagery sessions. EEG was recorded from 60 closely spaced electrodes placed over frontal, central, and parietal areas of 30 healthy volunteers divided in two groups. Whereas Group 1 (EG) performed a physical exercise between the two imagery sessions, Group 2 (CG) watched a landscape movie without physical activity. Spatiotemporal event-related desynchronization (ERD) and event-related synchronization (ERS) patterns during motor imagery (MI) tasks were evaluated. The results of the EG showed significant stronger ERD patterns in the alpha frequency band (8-13 Hz) during MI of tennis after training. Our results are in evidence with previous findings that MI in combination with motor execution has beneficial effects. We conclude that sports MI combined with an interactive game environment could be a future promising task in motor learning and rehabilitation improving motor functions in late therapy processes or support neuroplasticity.

Neurophysiological predictors and spectro-spatial discriminative features for enhancing SMR-BCI

Neural engineering research is actively engaged in optimizing the robustness of sensorimotor rhythms (SMR)-brain-computer interface (BCI) to boost its potential real-world use.

OBJECTIVE

This paper investigates two vital factors in efficient and robust SMR-BCI design-algorithms that address subject-variability of optimal features and neurophysiological factors that correlate with BCI performance. Existing SMR-BCI research using electroencephalogram (EEG) to classify bilateral motor imagery (MI) focus on identifying subject-specific frequency bands with most discriminative motor patterns localized to sensorimotor region.

APPROACH

A novel strategy to further optimize BCI performance by taking into account the variability of discriminative spectral regions across various EEG channels is proposed in this paper.

MAIN RESULTS

The proposed technique results in a significant ([Formula: see text]) increase in average ([Formula: see text]) classification accuracy by [Formula: see text] accompanied by a considerable reduction in number of channels and bands. The session-to-session transfer variation in spectro-spatial patterns using proposed algorithm is investigated offline and classification performance of the optimized BCI model is successfully evaluated in an online SMR-BCI. Further, the effective prediction of SMR-BCI performance with physiological indicators derived from multi-channel resting-state EEG is demonstrated. The results indicate that the resting state activation patterns such as entropy and gamma power from pre-motor (fronto-central) and posterior (parietal and centro-parietal) areas, and beta power from posterior (centro-parietal) areas estimate BCI performance with minimum error. These patterns, strongly related to BCI performance, may represent certain cognitive states during rest.

SIGNIFICANCE

Findings reported in this paper imply the need for subject-specific modelling of BCI and the prediction of BCI performance using multi-channel rest-state parameters, to ensure enhanced BCI performance.

Self-Paced Operation of a Wheelchair Based on a Hybrid Brain-Computer Interface Combining Motor Imagery and P300 Potential

This paper presents a hybrid brain-computer interface (BCI) that combines motor imagery (MI) and P300 potential for the asynchronous operation of a brain-controlled wheelchair whose design is based on a Mecanum wheel. This paradigm is completely user-centric. By sequentially performing MI tasks or paying attention to P300 flashing, the user can use eleven functions to control the wheelchair: move forward/backward, move left/right, move left45/right45, accelerate/decelerate, turn left/right, and stop. The practicality and effectiveness of the proposed approach were validated in eight subjects, all of whom achieved good performance. The preliminary results indicated that the proposed hybrid BCI system with different mental strategies operating sequentially is feasible and has potential applications for practical self-paced control.

The control of a virtual automatic car based on multiple patterns of motor imagery BCI

Multiple degrees of freedom (DOF) commands are required for a brain-actuated virtual automatic car, which makes the brain-computer interface (BCI) control strategy a big challenge. In order to solve the challenging issue, a mixed model of BCI combining P300 potentials and motor imagery had been realized in our previous study. However, compared with single model BCI, more training procedures are needed for the mixed model and more mental workload for users to bear. In the present study, we propose a multiple patterns of motor imagery (MPMI) BCI method, which is based on the traditional two patterns of motor imagery. Our motor imagery BCI approach had been extended to multiple patterns: right-hand motor imagery, left-hand motor imagery, foot motor imagery, and both hands motor imagery resulting in turning right, turning left, acceleration, and deceleration for a virtual automatic car control. Ten healthy subjects participated in online experiments, the experimental results not only show the efficiency of our proposed MPMI-BCI strategy but also indicate that those users can control the virtual automatic car spontaneously and efficiently without any other visual attention. Furthermore, the metric of path length optimality ratio (1.23) is very encouraging and the time optimality ratio (1.28) is especially remarkable. Graphical Abstract The paradigm of multiple patterns of motor imagery detection and the relevant topographies of CSP weights for different MI patterns.

Wireless Stimulus-on-Device Design for Novel P300 Hybrid Brain-Computer Interface Applications

Improving the independent living ability of people who have suffered spinal cord injuries (SCIs) is essential for their quality of life. Brain-computer interfaces (BCIs) provide promising solutions for people with high-level SCIs. This paper proposes a novel and practical P300-based hybrid stimulus-on-device (SoD) BCI architecture for wireless networking applications. Instead of a stimulus-on-panel architecture (SoP), the proposed SoD architecture provides an intuitive control scheme. However, because P300 recognitions rely on the synchronization between stimuli and response potentials, the variation of latency between target stimuli and elicited P300 is a concern when applying a P300-based BCI to wireless applications. In addition, the subject-dependent variation of elicited P300 affects the performance of the BCI. Thus, an adaptive model that determines an appropriate interval for P300 feature extraction was proposed in this paper. Hence, this paper employed the artificial bee colony- (ABC-) based interval type-2 fuzzy logic system (IT2FLS) to deal with the variation of latency between target stimuli and elicited P300 so that the proposed P300-based SoD approach would be feasible. Furthermore, the target and nontarget stimuli were identified in terms of a support vector machine (SVM) classifier. Experimental results showed that, from five subjects, the performance of classification and information transfer rate were improved after calibrations (86.00% and 24.2 bits/ min before calibrations; 90.25% and 27.9 bits/ min after calibrations).