Increasing accessibility to a large brain-computer interface dataset: Curation of physionet EEG motor movement/imagery dataset for decoding and classification

A reliable motor imagery (MI) brain-computer interface (BCI) requires accurate decoding, which in turn requires model calibration using electroencephalography (EEG) signals from subjects executing or imagining the execution of movements. Although the PhysioNet EEG Motor Movement/Imagery Dataset is currently the largest EEG dataset in the literature, relatively few studies have used it to decode MI trials. In the present study, we curated and cleaned this dataset to store it in an accessible format that is convenient for quick exploitation, decoding, and classification using recent integrated development environments. We dropped six subjects owing to anomalies in EEG recordings and pre-possessed the rest, resulting in 103 subjects spanning four MI and four motor execution tasks. The annotations were coded to correspond to different tasks using numerical values. The resulting dataset is stored in both MATLAB structure and CSV files to ensure ease of access and organization. We believe that improving the accessibility of this dataset will help EEG-based MI-BCI decoding and classification, enabling more reliable real-life applications. The convenience and ease of access of this dataset may therefore lead to improvements in cross-subject classification and transfer learning.

A generative adaptive convolutional neural network with attention mechanism for driver fatigue detection with class-imbalanced and insufficient data

Over the past few years, fatigue driving has emerged as one of the main causes of traffic accidents, necessitating the development of driver fatigue detection systems. However, many existing methods involves tedious manual parameter tunings, a process that is both time-consuming and results in task-specific models. On the other hand, most of the researches on fatigue recognition are based on class-balanced and sufficient data, and effectively "mine" meaningful information from class-imbalanced and insufficient data for fatigue recognition is still a challenge. In this paper, we proposed two novel models, the attention-based residual adaptive multiscale fully convolutional network-long short term memory network (ARMFCN-LSTM), and the Generative ARMFCN-LSTM (GARMFCN-LSTM) aiming to address this issue. ARMFCN-LSTM excels at automatically extracting multiscale representations through adaptive multiscale temporal convolutions, while capturing temporal dependency features through LSTM. GARMFCN-LSTM integrates Wasserstein GAN with gradient penalty (WGAN-GP) into ARMFCN-LSTM to improve driver fatigue detection performance by alleviating data scarcity and addressing class imbalances. Experimental results show that ARMFCN-LSTM achieves the highest classification accuracy of 95.84% in driver fatigue detection on the class-balanced EEG dataset (binary classification), and GARMFCN-LSTM attained an improved classification accuracy of 84.70% on the class-imbalanced EOG dataset (triple classification), surpassing the competing methods. Therefore, the proposed models are promising for further implementations in online driver fatigue detection systems.

Dynamic decomposition graph convolutional neural network for SSVEP-based brain-computer interface

The SSVEP-based paradigm serves as a prevalent approach in the realm of brain-computer interface (BCI). However, the processing of multi-channel electroencephalogram (EEG) data introduces challenges due to its non-Euclidean characteristic, necessitating methodologies that account for inter-channel topological relations. In this paper, we introduce the Dynamic Decomposition Graph Convolutional Neural Network (DDGCNN) designed for the classification of SSVEP EEG signals. Our approach incorporates layerwise dynamic graphs to address the oversmoothing issue in Graph Convolutional Networks (GCNs), employing a dense connection mechanism to mitigate the gradient vanishing problem. Furthermore, we enhance the traditional linear transformation inherent in GCNs with graph dynamic fusion, thereby elevating feature extraction and adaptive aggregation capabilities. Our experimental results demonstrate the effectiveness of proposed approach in learning and extracting features from EEG topological structure. The results shown that DDGCNN outperforms other state-of-the-art (SOTA) algorithms reported on two datasets (Dataset 1: 54 subjects, 4 targets, 2 sessions; Dataset 2: 35 subjects, 40 targets). Additionally, we showcase the implementation of DDGCNN in the context of synchronized BCI robotic fish control. This work represents a significant advancement in the field of EEG signal processing for SSVEP-based BCIs. Our proposed method processes SSVEP time domain signals directly as an end-to-end system, making it easy to deploy. The code is available at https://github.com/zshubin/DDGCNN.

Design and verification of a wearable wireless 64-channel high-resolution EEG acquisition system with wi-fi transmission

Brain-computer interfaces (BCIs) allow communication between the brain and the external world. This type of technology has been extensively studied. However, BCI instruments with high signal quality are typically heavy and large. Thus, recording electroencephalography (EEG) signals is an inconvenient task. In recent years, system-on-chip (SoC) approaches have been integrated into BCI research, and sensors for wireless portable devices have been developed; however, there is still considerable work to be done. As neuroscience research has advanced, EEG signal analyses have come to require more accurate data. Due to the limited bandwidth of Bluetooth wireless transmission technology, EEG measurement systems with more than 16 channels must be used to reduce the sampling rate and prevent data loss. Therefore, the goal of this study was to develop a multichannel, high-resolution (24-bit), high-sampling-rate EEG BCI device that transmits signals via Wi-Fi. We believe that this system can be used in neuroscience research. The EEG acquisition system proposed in this work is based on a Cortex-M4 microcontroller with a Wi-Fi subsystem, providing a multichannel design and improved signal quality. This system is compatible with wet sensors, Ag/AgCl electrodes, and dry sensors. A LabVIEW-based user interface receives EEG data via Wi-Fi transmission and saves the raw data for offline analysis. In previous cognitive experiments, event tags have been communicated using Recommended Standard 232 (RS-232). The developed system was validated through event-related potential (ERP) and steady-state visually evoked potential (SSVEP) experiments. Our experimental results demonstrate that this system is suitable for recording EEG measurements and has potential in practical applications. The advantages of the developed system include its high sampling rate and high amplification. Additionally, in the future, Internet of Things (IoT) technology can be integrated into the system for remote real-time analysis or edge computing.

MI-DAGSC: A domain adaptation approach incorporating comprehensive information from MI-EEG signals

Non-stationarity of EEG signals leads to high variability between subjects, making it challenging to directly use data from other subjects (source domain) for the classifier in the current subject (target domain). In this study, we propose MI-DAGSC to address domain adaptation challenges in EEG-based motor imagery (MI) decoding. By combining domain-level information, class-level information, and inter-sample structure information, our model effectively aligns the feature distributions of source and target domains. This work is an extension of our previous domain adaptation work MI-DABAN (Li et al., 2023). Based on MI-DABAN, MI-DAGSC designs Sample-Feature Blocks (SFBs) and Graph Convolution Blocks (GCBs) to focus on intra-sample and inter-sample information. The synergistic integration of SFBs and GCBs enable the model to capture comprehensive information and understand the relationship between samples, thus improving representation learning. Furthermore, we introduce a triplet loss to enhance the alignment and compactness of feature representations. Extensive experiments on real EEG datasets demonstrate the effectiveness of MI-DAGSC, confirming that our method makes a valuable contribution to the MI-EEG decoding. Moreover, it holds great potential for various applications in brain-computer interface systems and neuroscience research. And the code of the proposed architecture in this study is available under https://github.com/zhangdx21/MI-DAGSC.

A novel noninvasive brain-computer interface by imagining isometric force levels

Physiological circuits differ across increasing isometric force levels during unilateral contraction. Therefore, we first explored the possibility of predicting the force level based on electroencephalogram (EEG) activity recorded during a single trial of unilateral 5% or 40% of maximal isometric voluntary contraction (MVC) in right-hand grip imagination. Nine healthy subjects were involved in this study. The subjects were required to randomly perform 20 trials for each force level while imagining a right-hand grip. We proposed the use of common spatial patterns (CSPs) and coherence between EEG signals as features in a support vector machine for force level prediction. The results showed that the force levels could be predicted through single-trial EEGs while imagining the grip (mean accuracy = 81.4 ± 13.29%). Additionally, we tested the possibility of online control of a ball game using the above paradigm through unilateral grip imagination at different force levels (i.e., 5% of MVC imagination and 40% of MVC imagination for right-hand movement control). Subjects played the ball games effectively by controlling direction with our novel BCI system (n = 9, mean accuracy = 76.67 ± 9.35%). Data analysis validated the use of our BCI system in the online control of a ball game. This information may provide additional commands for the control of robots by users through combinations with other traditional brain-computer interfaces, e.g., different limb imaginations.

Cross-modal guiding and reweighting network for multi-modal RSVP-based target detection

Rapid Serial Visual Presentation (RSVP) based Brain-Computer Interface (BCI) facilities the high-throughput detection of rare target images by detecting evoked event-related potentials (ERPs). At present, the decoding accuracy of the RSVP-based BCI system limits its practical applications. This study introduces eye movements (gaze and pupil information), referred to as EYE modality, as another useful source of information to combine with EEG-based BCI and forms a novel target detection system to detect target images in RSVP tasks. We performed an RSVP experiment, recorded the EEG signals and eye movements simultaneously during a target detection task, and constructed a multi-modal dataset including 20 subjects. Also, we proposed a cross-modal guiding and fusion network to fully utilize EEG and EYE modalities and fuse them for better RSVP decoding performance. In this network, a two-branch backbone was built to extract features from these two modalities. A Cross-Modal Feature Guiding (CMFG) module was proposed to guide EYE modality features to complement the EEG modality for better feature extraction. A Multi-scale Multi-modal Reweighting (MMR) module was proposed to enhance the multi-modal features by exploring intra- and inter-modal interactions. And, a Dual Activation Fusion (DAF) was proposed to modulate the enhanced multi-modal features for effective fusion. Our proposed network achieved a balanced accuracy of 88.00% (±2.29) on the collected dataset. The ablation studies and visualizations revealed the effectiveness of the proposed modules. This work implies the effectiveness of introducing the EYE modality in RSVP tasks. And, our proposed network is a promising method for RSVP decoding and further improves the performance of RSVP-based target detection systems.

A spatial-temporal linear feature learning algorithm for P300-based brain-computer interfaces

Speller brain-computer interface (BCI) systems can help neuromuscular disorders patients write their thoughts by using the electroencephalogram (EEG) signals by just focusing on the speller tasks. For practical speller-based BCI systems, the P300 event-related brain potential is measured by using the EEG signal. In this paper, we design a robust machine-learning algorithm for P300 target detection. The novel spatial-temporal linear feature learning (STLFL) algorithm is proposed to extract high-level P300 features. The STLFL method is a modified linear discriminant analysis technique focusing on the spatial-temporal aspects of information extraction. A new P300 detection structure is then proposed based on the combination of the novel STLFL feature extraction and discriminative restricted Boltzmann machine (DRBM) for the classification approach (STLFL + DRBM). The effectiveness of the proposed technique is evaluated using two state-of-the-art P300 BCI datasets. Across the two available databases, we show that in terms of average target recognition accuracy and standard deviation values, the proposed STLFL + DRBM method outperforms traditional methods by 33.5, 78.5, 93.5, and 98.5% for 1, 5, 10, and 15 repetitions, respectively, in BCI competition III datasets II and by 71.3, 100, 100, and 100% for 1, 5, 10, and 15 repetitions, respectively, in BCI competition II datasets II and by 67.5 ± 4, 84.2 ± 2.5, 93.5 ± 1, 96.3 ± 1, and 98.4 ± 0.5% for rapid serial visual presentation (RSVP) based dataset in repetitions 1-5. The method has some advantages over the existing variants including its efficiency, robustness with a small number of training samples, and a high ability to create discriminative features between classes.

The Berlin Bimanual Test for Tetraplegia (BeBiTT): development, psychometric properties, and sensitivity to change in assistive hand exoskeleton application

BACKGROUND

Assistive hand exoskeletons are promising tools to restore hand function after cervical spinal cord injury (SCI) but assessing their specific impact on bimanual hand and arm function is limited due to lack of reliable and valid clinical tests. Here, we introduce the Berlin Bimanual Test for Tetraplegia (BeBiTT) and demonstrate its psychometric properties and sensitivity to assistive hand exoskeleton-related improvements in bimanual task performance.

METHODS

Fourteen study participants with subacute cervical SCI performed the BeBiTT unassisted (baseline). Thereafter, participants repeated the BeBiTT while wearing a brain/neural hand exoskeleton (B/NHE) (intervention). Online control of the B/NHE was established via a hybrid sensorimotor rhythm-based brain-computer interface (BCI) translating electroencephalographic (EEG) and electrooculographic (EOG) signals into open/close commands. For reliability assessment, BeBiTT scores were obtained by four independent observers. Besides internal consistency analysis, construct validity was assessed by correlating baseline BeBiTT scores with the Spinal Cord Independence Measure III (SCIM III) and Quadriplegia Index of Function (QIF). Sensitivity to differences in bimanual task performance was assessed with a bootstrapped paired t-test.

RESULTS

The BeBiTT showed excellent interrater reliability (intraclass correlation coefficients > 0.9) and internal consistency (α = 0.91). Validity of the BeBiTT was evidenced by strong correlations between BeBiTT scores and SCIM III as well as QIF. Wearing a B/NHE (intervention) improved the BeBiTT score significantly (p < 0.05) with high effect size (d = 1.063), documenting high sensitivity to intervention-related differences in bimanual task performance.

CONCLUSION

The BeBiTT is a reliable and valid test for evaluating bimanual task performance in persons with tetraplegia, suitable to assess the impact of assistive hand exoskeletons on bimanual function.

MI-DABAN: A dual-attention-based adversarial network for motor imagery classification

The brain-computer interface (BCI) based on motor imagery electroencephalography (EEG) is widely used because of its convenience and safety. However, due to the distributional disparity between EEG signals, data from other subjects cannot be used directly to train a subject-specific classifier. For efficient use of the labeled data, domain transfer learning and adversarial learning are gradually applied to BCI classification tasks. While these methods improve classification performance, they only align globally and ignore task-specific class boundaries, which may lead to the blurring of features near the classification boundaries. Simultaneously, they employ fully shared generators to extract features, resulting in the loss of domain-specific information and the destruction of performance. To address these issues, we propose a novel dual-attention-based adversarial network for motor imagery classification (MI-DABAN). Our framework leverages multiple subjects' knowledge to improve a single subject's motor imagery classification performance by cleverly using a novel adversarial learning method and two unshared attention blocks. Specifically, without introducing additional domain discriminators, we iteratively maximize and minimize the output difference between the two classifiers to implement adversarial learning to ensure accurate domain alignment. Among them, maximization is used to identify easily confused samples near the decision boundary, and minimization is used to align the source and target domain distributions. Moreover, for the shallow features from source and target domains, we use two non-shared attention blocks to preserve domain-specific information, which can prevent the negative transfer of domain information and further improve the classification performance on test data. We conduct extensive experiments on two publicly available EEG datasets, namely BCI Competition IV Datasets 2a and 2b. The experiment results demonstrate our method's effectiveness and superiority.

A multi-modal brain-computer interface based on threshold discrimination and its application in wheelchair control

In this study, we propose a novel multi-modal brain-computer interface (BCI) system based on the threshold discrimination, which is proposed for the first time to distinguish between SSVEP and MI potentials. The system combines these two heterogeneous signals to increase the number of control commands and improve the performance of asynchronous control of external devices. In this research, an electric wheelchair is controlled as an example. The user can continuously control the wheelchair to turn left/right through motion imagination (MI) by imagining left/right-hand movement and generate another 6 commands for the wheelchair control by focusing on the SSVEP stimulation panel. Ten subjects participated in a MI training session and eight of them completed a mobile obstacle-avoidance experiment in a complex environment requesting high control accuracy for successful manipulation. Comparing with the single-modal BCI-controlled wheelchair system, the results demonstrate that the proposed multi-modal method is effective by providing more satisfactory control accuracy, and show the potential of BCI-controlled systems to be applied in complex daily tasks.

Competing at the Cybathlon championship for people with disabilities: long-term motor imagery brain-computer interface training of a cybathlete who has tetraplegia

Background

The brain–computer interface (BCI) race at the Cybathlon championship, for people with disabilities, challenges teams (BCI researchers, developers and pilots with spinal cord injury) to control an avatar on a virtual racetrack without movement. Here we describe the training regime and results of the Ulster University BCI Team pilot who has tetraplegia and was trained to use an electroencephalography (EEG)-based BCI intermittently over 10 years, to compete in three Cybathlon events.

Methods

A multi-class, multiple binary classifier framework was used to decode three kinesthetically imagined movements (motor imagery of left arm, right arm, and feet), and relaxed state. Three game paradigms were used for training i.e., NeuroSensi, Triad, and Cybathlon Race: BrainDriver. An evaluation of the pilot’s performance is presented for two Cybathlon competition training periods—spanning 20 sessions over 5 weeks prior to the 2019 competition, and 25 sessions over 5 weeks in the run up to the 2020 competition.

Results

Having participated in BCI training in 2009 and competed in Cybathlon 2016, the experienced pilot achieved high two-class accuracy on all class pairs when training began in 2019 (decoding accuracy > 90%, resulting in efficient NeuroSensi and Triad game control). The BrainDriver performance (i.e., Cybathlon race completion time) improved significantly during the training period, leading up to the competition day, ranging from 274–156 s (255 ± 24 s to 191 ± 14 s mean ± std), over 17 days (10 sessions) in 2019, and from 230–168 s (214 ± 14 s to 181 ± 4 s), over 18 days (13 sessions) in 2020. However, on both competition occasions, towards the race date, the performance deteriorated significantly.

Conclusions

The training regime and framework applied were highly effective in achieving competitive race completion times. The BCI framework did not cope with significant deviation in electroencephalography (EEG) observed in the sessions occurring shortly before and during the race day. Changes in cognitive state as a result of stress, arousal level, and fatigue, associated with the competition challenge and performance pressure, were likely contributing factors to the non-stationary effects that resulted in the BCI and pilot achieving suboptimal performance on race day.

Trial registration not registered

Supplementary Information

The online version contains supplementary material available at 10.1186/s12984-022-01073-9.

Predicting motor behavior: an efficient EEG signal processing pipeline to detect brain states with potential therapeutic relevance for VR-based neurorehabilitation

Virtual reality (VR)-based motor therapy is an emerging approach in neurorehabilitation. The combination of VR with electroencephalography (EEG) presents further opportunities to improve therapeutic efficacy by personalizing the paradigm. Specifically, the idea is to synchronize the choice and timing of stimuli in the perceived virtual world with fluctuating brain states relevant to motor behavior. Here, we present an open source EEG single-trial based classification pipeline that is designed to identify ongoing brain states predictive of the planning and execution of movements. 9 healthy volunteers each performed 1080 trials of a repetitive reaching task with an implicit two-alternative forced choice, i.e., use of the right or left hand, in response to the appearance of a visual target. The performance of the EEG decoding pipeline was assessed with respect to classification accuracy of right vs. left arm use, based on the EEG signal at the time of the stimulus. Different features, feature extraction methods, and classifiers were compared at different time windows; the number and location of informative EEG channels and the number of calibration trials needed were also quantified, as well as any benefits from individual-level optimization of pipeline parameters. This resulted in a set of recommended parameters that achieved an average 83.3% correct prediction on never-before-seen testing data, and a state-of-the-art 77.1% in a real-time simulation. Neurophysiological plausibility of the resulting classifiers was assessed by time-frequency and event-related potential analyses, as well as by Independent Component Analysis topographies and cortical source localization. We expect that this pipeline will facilitate the identification of relevant brain states as prospective therapeutic targets in closed-loop EEG-VR motor neurorehabilitation.

The influence of psychological and cognitive states on error-related negativity evoked during post-stroke rehabilitation movements

Background

Recently, error-related negativity (ERN) signals are proposed to develop an assist-as-needed robotic stroke rehabilitation program. Stroke patients’ state-of-mind, such as motivation to participate and active involvement in the rehabilitation program, affects their rate of recovery from motor disability. If the characteristics of the robotic stroke rehabilitation program can be altered based on the state-of-mind of the patients, such that the patients remain engaged in the program, the rate of recovery from their motor disability can be improved. However, before that, it is imperative to understand how the states-of-mind of a participant affect their ERN signal.

Methods

This study aimed to determine the association between the ERN signal and the psychological and cognitive states of the participants. Experiments were conducted on stroke patients, which involved performing a physical rehabilitation exercise and a questionnaire to measure participants' subjective experience on four factors: motivation in participating in the experiment, perceived effort, perceived pressure, awareness of uncompleted exercise trials while performing the rehabilitation exercise. Statistical correlation analysis, EEG time-series and topographical analysis were used to assess the association between the ERN signals and the psychological and cognitive states of the participants.

Results

A strong correlation between the amplitude of the ERN signal and the psychological and cognitive states of the participants was observed, which indicate the possibility of estimating the said states using the amplitudes of the novel ERN signal.

Conclusions

The findings pave the way for the development of an ERN based dynamically adaptive assist-as-needed robotic stroke rehabilitation program of which characteristics can be altered to keep the participants’ motivation, effort, engagement in the rehabilitation program high. In future, the single-trial prediction ability of the novel ERN signals to predict the state-of-mind of stroke patients will be evaluated.

A robust and reliable online P300-based BCI system using Emotiv EPOC + headset

Brain-computer interface (BCI) system aims to enable interaction with people and therefore the environment without muscular activation, using changes in brain signals due to the execution of cognitive tasks. The target of the presented work is to investigate the power of Emotiv EPOC + headset to detect and record the P300 wave. Moreover, the effect of preprocessing the acquired signal was studied. Five participants were asked to attend different sessions to an equivalent 6x6 matrix while the rows and columns were randomly flashed at a rate of 200 ms. The acquired EEG data were sent wirelessly to OpenViBE software, which is employed to run the P300 speller. Two classification methods were tried: Linear discriminate analysis (LDA) and support vector machine (SVM). The capability of the headset to detect the P300 signals is proven by the results. Additionally, results show that participants reached accuracy up to 90 and 70% after only two training sessions for Linear discriminate analysis (LDA) and support vector machine (SVM) classifiers, respectively. The significance of this work is to demonstrate that such a portable and affordable headset might be useful to design and implement a robust and reliable online P300-based BCI system.

An EEG channel selection method for motor imagery based brain-computer interface and neurofeedback using Granger causality

Motor imagery (MI) brain-computer interface (BCI) and neurofeedback (NF) with electroencephalogram (EEG) signals are commonly used for motor function improvement in healthy subjects and to restore neurological functions in stroke patients. Generally, in order to decrease noisy and redundant information in unrelated EEG channels, channel selection methods are used which provide feasible BCI and NF implementations with better performances. Our assumption is that there are causal interactions between the channels of EEG signal in MI tasks that are repeated in different trials of a BCI and NF experiment. Therefore, a novel method for EEG channel selection is proposed which is based on Granger causality (GC) analysis. Additionally, the machine-learning approach is used to cluster independent component analysis (ICA) components of the EEG signal into artifact and normal EEG clusters. After channel selection, using the common spatial pattern (CSP) and regularized CSP (RCSP), features are extracted and with the k-nearest neighbor (k-NN), support vector machine (SVM) and linear discriminant analysis (LDA) classifiers, MI tasks are classified into left and right hand MI. The goal of this study is to achieve a method resulting in lower EEG channels with higher classification performance in MI-based BCI and NF by causal constraint. The proposed method based on GC, with only eight selected channels, results in 93.03% accuracy, 92.93% sensitivity, and 93.12% specificity, with RCSP feature extractor and best classifier for each subject, after being applied on Physionet MI dataset, which is increased by 3.95%, 3.73%, and 4.13%, in comparison with correlation-based channel selection method.

Filter bank temporally local canonical correlation analysis for short time window SSVEPs classification

Canonical correlation analysis (CCA) method and its extended methods have been widely and successfully applied to the frequency recognition in SSVEP-based BCI systems. As a state-of-the-art extended method, filter bank canonical correlation analysis has higher accuracy and information transmission rate (ITR) than CCA. However, in the CCA method, the temporally local structure of samples has not been well considered. In this correspondence, we proposed termed temporally local canonical correlation analysis (TCCA). In this new method, the original covariance matrix was replaced by the temporally local covariance matrix. Furthermore, we proposed an improved frequency identification method of filter bank based on TCCA, named filter bank temporally local canonical correlation analysis (FBTCCA). In the offline environment, we used a leave-one-subject-out validation strategy on datasets of ten testees to optimize the parameters of TCCA and FBTCCA and evaluate the two algorithms. The experimental results affirm that TCCA markedly outperformed CCA, and FBTCCA obtained the highest accuracy among the four methods. This study corroborates that TCCA-based approaches have great potential for implementing short time window SSVEP-based BCI systems.

Multiclass EEG signal classification utilizing Rényi min-entropy-based feature selection from wavelet packet transformation

This paper proposes a novel feature selection method utilizing Rényi min-entropy-based algorithm for achieving a highly efficient brain–computer interface (BCI). Usually, wavelet packet transformation (WPT) is extensively used for feature extraction from electro-encephalogram (EEG) signals. For the case of multiple-class problem, classification accuracy solely depends on the effective feature selection from the WPT features. In conventional approaches, Shannon entropy and mutual information methods are often used to select the features. In this work, we have shown that our proposed Rényi min-entropy-based approach outperforms the conventional methods for multiple EEG signal classification. The dataset of BCI competition-IV (contains 4-class motor imagery EEG signal) is used for this experiment. The data are preprocessed and separated as the classes and used for the feature extraction using WPT. Then, for feature selection Shannon entropy, mutual information, and Rényi min-entropy methods are applied. With the selected features, four-class motor imagery EEG signals are classified using several machine learning algorithms. The results suggest that the proposed method is better than the conventional approaches for multiple-class BCI.

Multiclass covert speech classification using extreme learning machine

The objective of the proposed research is to classify electroencephalography (EEG) data of covert speech words. Six subjects were asked to perform covert speech tasks i.e mental repetition of four different words i.e 'left', 'right', 'up' and 'down'. Fifty trials for each word recorded for every subject. Kernel-based Extreme Learning Machine (kernel ELM) was used for multiclass and binary classification of EEG signals of covert speech words. We achieved a maximum multiclass and binary classification accuracy of (49.77%) and (85.57%) respectively. The kernel ELM achieves significantly higher accuracy compared to some of the most commonly used classification algorithms in Brain-Computer Interfaces (BCIs). Our findings suggested that covert speech EEG signals could be successfully classified using kernel ELM. This research involving the classification of covert speech words potentially leading to real-time silent speech BCI research.

Modeling and classification of voluntary and imagery movements for brain-computer interface from fNIR and EEG signals through convolutional neural network

Practical brain-computer interface (BCI) demands the learning-based adaptive model that can handle diverse problems. To implement a BCI, usually functional near-infrared spectroscopy (fNIR) is used for measuring functional changes in brain oxygenation and electroencephalography (EEG) for evaluating the neuronal electric potential regarding the psychophysiological activity. Since the fNIR modality has an issue of temporal resolution, fNIR alone is not enough to achieve satisfactory classification accuracy as multiple neural stimuli are produced by voluntary and imagery movements. This leads us to make a combination of fNIR and EEG with a view to developing a BCI model for the classification of the brain signals of the voluntary and imagery movements. This work proposes a novel approach to prepare functional neuroimages from the fNIR and EEG using eight different movement-related stimuli. The neuroimages are used to train a convolutional neural network (CNN) to formulate a predictive model for classifying the combined fNIR-EEG data. The results reveal that the combined fNIR-EEG modality approach along with a CNN provides improved classification accuracy compared to a single modality and conventional classifiers. So, the outcomes of the proposed research work will be very helpful in the implementation of the finer BCI system.

Direct comparison of supervised and semi-supervised retraining approaches for co-adaptive BCIs

For Brain-Computer interfaces (BCIs), system calibration is a lengthy but necessary process for successful operation. Co-adaptive BCIs aim to shorten training and imply positive motivation to users by presenting feedback already at early stages: After just 5 min of gathering calibration data, the systems are able to provide feedback and engage users in a mutual learning process. In this work, we investigate whether the retraining stage of co-adaptive BCIs can be adapted to a semi-supervised concept, where only a small amount of labeled data is available and all additional data needs to be labeled by the BCI itself. The aim of the current work was to evaluate whether a semi-supervised co-adaptive BCI could successfully compete with a supervised co-adaptive BCI model. In a supporting two-class (190 trials per condition) BCI study based on motor imagery tasks, we evaluated both approaches in two separate groups of 10 participants online, while we simulated the other approach in each group offline. Our results indicate that despite the lack of true labeled data, the semi-supervised driven BCI did not perform significantly worse (p > 0.05) than the supervised counterpart. We believe that these findings contribute to developing BCIs for long-term use, where continuous adaptation becomes imperative for maintaining meaningful BCI performance.

Graphical abstract

A dynamic and self-adaptive classification algorithm for motor imagery EEG signals

BACKGROUND

Brain-computer interface (BCI) is a communication pathway applied for pathological analysis or functional substitution. BCI based on functional substitution enables the recognition of a subject's intention to control devices such as prosthesis and wheelchairs. Discrimination of electroencephalography (EEG) trials related to left- and right-hand movements requires complex EEG signal processing to achieve good system performance.

NEW METHOD

In this study, a novel dynamic and self-adaptive algorithm (DSAA) based on the least-squares method is proposed to select the most appropriate feature extraction and classification algorithms couple for each subject. Specifically, the best couple identified during the training of the system is updated during online testing in order to check the stability of the selected couple and maintain high system accuracy.

RESULTS

Extensive and systematic experiments were conducted on public datasets of 17 subjects in the BCI-competition and the results show an improved performance for DSAA over other selected state-of-the-art methods.

COMPARISON WITH EXISTING METHODS

The results show that the proposed system enhanced the classification accuracy for the three chosen public datasets by 8% compared to other approaches. Moreover, the proposed system was successful in selecting the best path despite the unavailability of reference labels.

CONCLUSIONS

Performing dynamic and self-adaptive selection for the best feature extraction and classification algorithm couple increases the recognition rate of trials despite the unavailability of reference trial labels. This approach allows the development of a complete BCI system with excellent accuracy.

Correlation-based channel selection and regularized feature optimization for MI-based BCI

Multi-channel EEG data are usually necessary for spatial pattern identification in motor imagery (MI)-based brain computer interfaces (BCIs). To some extent, signals from some channels containing redundant information and noise may degrade BCI performance. We assume that the channels related to MI should contain common information when participants are executing the MI tasks. Based on this hypothesis, a correlation-based channel selection (CCS) method is proposed to select the channels that contained more correlated information in this study. The aim is to improve the classification performance of MI-based BCIs. Furthermore, a novel regularized common spatial pattern (RCSP) method is used to extract effective features. Finally, a support vector machine (SVM) classifier with the Radial Basis Function (RBF) kernel is trained to accurately identify the MI tasks. An experimental study is implemented on three public EEG datasets (BCI competition IV dataset 1, BCI competition III dataset IVa and BCI competition III dataset IIIa) to validate the effectiveness of the proposed methods. The results show that the CCS algorithm obtained superior classification accuracy (78% versus 56.4% for dataset1, 86.6% versus 76.5% for dataset 2 and 91.3% versus 85.1% for dataset 3) compared to the algorithm using all channels (AC), when CSP is used to extract the features. Furthermore, RCSP could further improve the classification accuracy (81.6% for dataset1, 87.4% for dataset2 and 91.9% for dataset 3), when CCS is used to select the channels.

The Potential for a Speech Brain-Computer Interface Using Chronic Electrocorticography

A brain-computer interface (BCI) is a technology that uses neural features to restore or augment the capabilities of its user. A BCI for speech would enable communication in real time via neural correlates of attempted or imagined speech. Such a technology would potentially restore communication and improve quality of life for locked-in patients and other patients with severe communication disorders. There have been many recent developments in neural decoders, neural feature extraction, and brain recording modalities facilitating BCI for the control of prosthetics and in automatic speech recognition (ASR). Indeed, ASR and related fields have developed significantly over the past years, and many lend many insights into the requirements, goals, and strategies for speech BCI. Neural speech decoding is a comparatively new field but has shown much promise with recent studies demonstrating semantic, auditory, and articulatory decoding using electrocorticography (ECoG) and other neural recording modalities. Because the neural representations for speech and language are widely distributed over cortical regions spanning the frontal, parietal, and temporal lobes, the mesoscopic scale of population activity captured by ECoG surface electrode arrays may have distinct advantages for speech BCI, in contrast to the advantages of microelectrode arrays for upper-limb BCI. Nevertheless, there remain many challenges for the translation of speech BCIs to clinical populations. This review discusses and outlines the current state-of-the-art for speech BCI and explores what a speech BCI using chronic ECoG might entail.

Ethical Considerations in Ending Exploratory Brain-Computer Interface Research Studies in Locked-in Syndrome

Brain-computer interface (BCI) is a promising technology for restoring communication in individuals with locked-in syndrome (LIS). BCI technology offers a potential tool for individuals with impaired or absent means of effective communication to use brain activity to control an output device such as a computer keyboard. Exploratory studies of BCI devices for communication in people with LIS are underway. Research with individuals with LIS presents not only technological challenges, but ethical challenges as well. Whereas recent attention has been focused on ethical issues that arise at the initiation of studies, such as how to obtain valid consent, relatively little attention has been given to issues at the conclusion of studies. BCI research in LIS highlights one such challenge: How to decide when an exploratory BCI research study should end. In this article, we present the case of an individual with presumed LIS enrolled in an exploratory BCI study. We consider whether two common ethical frameworks for stopping randomized clinical trials-equipoise and nonexploitation-can be usefully applied to elucidating researcher obligations to end exploratory BCI research. We argue that neither framework is a good fit for exploratory BCI research. Instead, we apply recent work on clinician-researcher fiduciary obligations and in turn offer some preliminary recommendations for BCI researchers on how to end exploratory BCI studies.

An efficient scheme for mental task classification utilizing reflection coefficients obtained from autocorrelation function of EEG signal

Classification of different mental tasks using electroencephalogram (EEG) signal plays an imperative part in various brain–computer interface (BCI) applications. In the design of BCI systems, features extracted from lower frequency bands of scalp-recorded EEG signals are generally considered to classify mental tasks and higher frequency bands are mostly ignored as noise. However, in this paper, it is demonstrated that high frequency components of EEG signal can provide accommodating data for enhancing the classification performance of the mental task-based BCI. Instead of using autoregressive (AR) parameters considering AR modeling of EEG data, reflection coefficients obtained from EEG signal are proposed as potential features. From a given frame of EEG data, reflection coefficients are directly extracted by using the autocorrelation values in a recursive fashion, which avoids matrix inversion and computation of AR parameters. Use of reflection coefficients not only provides an effective feature vector for EEG signal classification but also offers very low computational burden. Support vector machine classifier is deployed in leave-one-out cross-validation manner to carry out classification process. Extensive simulation is done on an openly accessible dataset containing five different mental tasks. It is found that the proposed scheme can classify mental tasks with a very high level of accuracy as well as low time complexity in contrast with some of the existing strategies.

EEG Classification with a Sequential Decision-Making Method in Motor Imagery BCI

To develop subject-specific classifier to recognize mental states fast and reliably is an important issue in brain-computer interfaces (BCI), particularly in practical real-time applications such as wheelchair or neuroprosthetic control. In this paper, a sequential decision-making strategy is explored in conjunction with an optimal wavelet analysis for EEG classification. The subject-specific wavelet parameters based on a grid-search method were first developed to determine evidence accumulative curve for the sequential classifier. Then we proposed a new method to set the two constrained thresholds in the sequential probability ratio test (SPRT) based on the cumulative curve and a desired expected stopping time. As a result, it balanced the decision time of each class, and we term it balanced threshold SPRT (BTSPRT). The properties of the method were illustrated on 14 subjects' recordings from offline and online tests. Results showed the average maximum accuracy of the proposed method to be 83.4% and the average decision time of 2.77[Formula: see text]s, when compared with 79.2% accuracy and a decision time of 3.01[Formula: see text]s for the sequential Bayesian (SB) method. The BTSPRT method not only improves the classification accuracy and decision speed comparing with the other nonsequential or SB methods, but also provides an explicit relationship between stopping time, thresholds and error, which is important for balancing the speed-accuracy tradeoff. These results suggest that BTSPRT would be useful in explicitly adjusting the tradeoff between rapid decision-making and error-free device control.

A Novel Multilayer Correlation Maximization Model for Improving CCA-Based Frequency Recognition in SSVEP Brain-Computer Interface

Multiset canonical correlation analysis (MsetCCA) has been successfully applied to optimize the reference signals by extracting common features from multiple sets of electroencephalogram (EEG) for steady-state visual evoked potential (SSVEP) recognition in brain-computer interface application. To avoid extracting the possible noise components as common features, this study proposes a sophisticated extension of MsetCCA, called multilayer correlation maximization (MCM) model for further improving SSVEP recognition accuracy. MCM combines advantages of both CCA and MsetCCA by carrying out three layers of correlation maximization processes. The first layer is to extract the stimulus frequency-related information in using CCA between EEG samples and sine-cosine reference signals. The second layer is to learn reference signals by extracting the common features with MsetCCA. The third layer is to re-optimize the reference signals set in using CCA with sine-cosine reference signals again. Experimental study is implemented to validate effectiveness of the proposed MCM model in comparison with the standard CCA and MsetCCA algorithms. Superior performance of MCM demonstrates its promising potential for the development of an improved SSVEP-based brain-computer interface.

Use of phase-locking value in sensorimotor rhythm-based brain-computer interface: zero-phase coupling and effects of spatial filters

Phase-locking value (PLV) is a potentially useful feature in sensorimotor rhythm-based brain-computer interface (BCI). However, volume conduction may cause spurious zero-phase coupling between two EEG signals and it is not clear whether PLV effects are independent of spectral amplitude. Volume conduction might be reduced by spatial filtering, but it is uncertain what impact this might have on PLV. Therefore, the goal of this study was to explore whether zero-phase PLV is meaningful and how it is affected by spatial filtering. Both amplitude and PLV feature were extracted in the frequency band of 10-15 Hz by classical methods using archival EEG data of 18 subjects trained on a two-target BCI task. The results show that with right ear-referenced data, there is meaningful long-range zero-phase synchronization likely involving the primary motor area and the supplementary motor area that cannot be explained by volume conduction. Another novel finding is that the large Laplacian spatial filter enhances the amplitude feature but eliminates most of the phase information seen in ear-referenced data. A bipolar channel using phase-coupled areas also includes both phase and amplitude information and has a significant practical advantage since fewer channels required.

Music and natural sounds in an auditory steady-state response based brain-computer interface to increase user acceptance

Patients with total locked-in syndrome are conscious; however, they cannot express themselves because most of their voluntary muscles are paralyzed, and many of these patients have lost their eyesight. To improve the quality of life of these patients, there is an increasing need for communication-supporting technologies that leverage the remaining senses of the patient along with physiological signals. The auditory steady-state response (ASSR) is an electro-physiologic response to auditory stimulation that is amplitude-modulated by a specific frequency. By leveraging the phenomenon whereby ASSR is modulated by mind concentration, a brain-computer interface paradigm was proposed to classify the selective attention of the patient. In this paper, we propose an auditory stimulation method to minimize auditory stress by replacing the monotone carrier with familiar music and natural sounds for an ergonomic system. Piano and violin instrumentals were employed in the music sessions; the sounds of water streaming and cicadas singing were used in the natural sound sessions. Six healthy subjects participated in the experiment. Electroencephalograms were recorded using four electrodes (Cz, Oz, T7 and T8). Seven sessions were performed using different stimuli. The spectral power at 38 and 42Hz and their ratio for each electrode were extracted as features. Linear discriminant analysis was utilized to classify the selections for each subject. In offline analysis, the average classification accuracies with a modulation index of 1.0 were 89.67% and 87.67% using music and natural sounds, respectively. In online experiments, the average classification accuracies were 88.3% and 80.0% using music and natural sounds, respectively. Using the proposed method, we obtained significantly higher user-acceptance scores, while maintaining a high average classification accuracy.

Informed Consent in Implantable BCI Research: Identifying Risks and Exploring Meaning

Implantable brain-computer interface (BCI) technology is an expanding area of engineering research now moving into clinical application. Ensuring meaningful informed consent in implantable BCI research is an ethical imperative. The emerging and rapidly evolving nature of implantable BCI research makes identification of risks, a critical component of informed consent, a challenge. In this paper, 6 core risk domains relevant to implantable BCI research are identified-short and long term safety, cognitive and communicative impairment, inappropriate expectations, involuntariness, affective impairment, and privacy and security. Work in deep brain stimulation provides a useful starting point for understanding this core set of risks in implantable BCI. Three further risk domains-risks pertaining to identity, agency, and stigma-are identified. These risks are not typically part of formalized consent processes. It is important as informed consent practices are further developed for implantable BCI research that attention be paid not just to disclosing core research risks but exploring the meaning of BCI research with potential participants.

Beyond p-values in the evaluation of brain-computer interfaces: A Bayesian estimation approach

BACKGROUND

To statistically evaluate the performance of brain-computer interfaces (BCIs), researchers usually rely on null hypothesis significance testing (NHST), i.e. p-values. However, over-reliance on NHST is often identified as one of the causes of the recent reproducibility crisis in psychology and neuroscience.

NEW METHOD

In this paper we propose Bayesian estimation as an alternative to NHST in the analysis of BCI performance data. For the three most common experimental designs in BCI research - which would usually be analyzed using a t-test, a linear regression, or an ANOVA - we develop hierarchical models and estimate their parameters using Bayesian inference. Furthermore, we show that the described models are special cases of the hierarchical generalized linear model (HGLM), which we propose as a general framework for the analysis of BCI performance.

RESULTS

We demonstrate the effectiveness of the proposed models on three real datasets and show how the results obtained with Bayesian estimation can give a nuanced insight into BCI performance data. Additionally, we provide all the data and code necessary to reproduce the presented results.

COMPARISON WITH EXISTING METHOD(S)

Compared to NHST, Bayesian estimation with the HGLM allows more flexibility in the analysis of BCI performance data from nested experimental designs, and the obtained results have a more straightforward interpretation.

CONCLUSIONS

Besides gains in flexibility and interpretability, a wider adoption of the Bayesian estimation approach in BCI studies could bring about greater transparency in data analysis, allow accumulation of knowledge across studies, and reduce questionable practices such as "p-hacking".

Subject-based feature extraction by using fisher WPD-CSP in brain-computer interfaces

BACKGROUND AND OBJECTIVE

Feature extraction of electroencephalogram (EEG) plays a vital role in brain-computer interfaces (BCIs). In recent years, common spatial pattern (CSP) has been proven to be an effective feature extraction method. However, the traditional CSP has disadvantages of requiring a lot of input channels and the lack of frequency information. In order to remedy the defects of CSP, wavelet packet decomposition (WPD) and CSP are combined to extract effective features. But WPD-CSP method considers less about extracting specific features that are fitted for the specific subject. So a subject-based feature extraction method using fisher WPD-CSP is proposed in this paper.

METHODS

The idea of proposed method is to adapt fisher WPD-CSP to each subject separately. It mainly includes the following six steps: (1) original EEG signals from all channels are decomposed into a series of sub-bands using WPD; (2) average power values of obtained sub-bands are computed; (3) the specified sub-bands with larger values of fisher distance according to average power are selected for that particular subject; (4) each selected sub-band is reconstructed to be regarded as a new EEG channel; (5) all new EEG channels are used as input of the CSP and a six-dimensional feature vector is obtained by the CSP. The subject-based feature extraction model is so formed; (6) the probabilistic neural network (PNN) is used as the classifier and the classification accuracy is obtained.

RESULTS

Data from six subjects are processed by the subject-based fisher WPD-CSP, the non-subject-based fisher WPD-CSP and WPD-CSP, respectively. Compared with non-subject-based fisher WPD-CSP and WPD-CSP, the results show that the proposed method yields better performance (sensitivity: 88.7±0.9%, and specificity: 91±1%) and the classification accuracy from subject-based fisher WPD-CSP is increased by 6-12% and 14%, respectively.

CONCLUSIONS

The proposed subject-based fisher WPD-CSP method can not only remedy disadvantages of CSP by WPD but also discriminate helpless sub-bands for each subject and make remaining fewer sub-bands keep better separability by fisher distance, which leads to a higher classification accuracy than WPD-CSP method.

Analysis of EEG signal by flicker-noise spectroscopy: identification of right-/left-hand movement imagination

Flicker-noise spectroscopy (FNS) is a general phenomenological approach to analyzing dynamics of complex nonlinear systems by extracting information contained in chaotic signals. The main idea of FNS is to describe an information hidden in correlation links, which are present in the chaotic component of the signal, by a set of parameters. In the paper, FNS is used for the analysis of electroencephalography signal related to the hand movement imagination. The signal has been parametrized in accordance with the FNS method, and significant changes in the FNS parameters have been observed, at the time when the subject imagines the movement. For the right-hand movement imagination, abrupt changes (visible as a peak) of the parameters, calculated for the data recorded from the left hemisphere, appear at the time corresponding to the initial moment of the imagination. In contrary, for the left-hand movement imagination, the meaningful changes in the parameters are observed for the data recorded from the right hemisphere. As the motor cortex is activated mainly contralaterally to the hand, the analysis of the FNS parameters allows to distinguish between the imagination of the right- and left-hand movement. This opens its potential application in the brain–computer interface.

Eliciting dual-frequency SSVEP using a hybrid SSVEP-P300 BCI

BACKGROUND

Steady-state visual-evoked potential (SSVEP)-based brain-computer interfaces (BCIs) generate weak SSVEP with a monitor and cannot use harmonic frequencies, whereas P300-based BCIs need multiple stimulation sequences. These issues can decrease the information transfer rate (ITR).

NEW METHOD

In this paper, we introduce a novel hybrid SSVEP-P300 speller that generates dual-frequency SSVEP, allowing it to overcome the abovementioned limitations and improve the performance. The hybrid speller consists of nine panels flickering at different frequencies. Each panel contains four different characters that appear in a random sequence. The flickering panel and the periodically updating character evoke the dual-frequency SSVEP, while the oddball stimulus of the target character evokes the P300. A canonical correlation analysis (CCA) and a step-wise linear discriminant analysis (SWLDA) classified SSVEP and P300, respectively. Ten subjects participated in offline and online experiments, in which accuracy and ITR were compared with those of conventional SSVEP and P300 spellers.

RESULTS

The offline analysis revealed not only the P300 potential but also SSVEP with peaks at sub-harmonic frequencies, demonstrating that the proposed speller elicited dual-frequency SSVEP. This dual-frequency stimulation improved SSVEP recognition, increased the number of targets by employing harmonic frequencies, reduced the stimulation time for P300, and consequently improved ITR as compared to the conventional spellers.

COMPARISON WITH EXISTING METHODS

The new method reduces the stimulation time and allows harmonic frequencies to be employed for different stimuli.

CONCLUSIONS

The results indicate that this study provides a promising approach to make the BCI speller more reliable and efficient.

A novel Morse code-inspired method for multiclass motor imagery brain-computer interface (BCI) design

Motor imagery (MI)-based brain-computer interfaces (BCIs) allow disabled individuals to control external devices voluntarily, helping us to restore lost motor functions. However, the number of control commands available in MI-based BCIs remains limited, limiting the usability of BCI systems in control applications involving multiple degrees of freedom (DOF), such as control of a robot arm. To address this problem, we developed a novel Morse code-inspired method for MI-based BCI design to increase the number of output commands. Using this method, brain activities are modulated by sequences of MI (sMI) tasks, which are constructed by alternately imagining movements of the left or right hand or no motion. The codes of the sMI task was detected from EEG signals and mapped to special commands. According to permutation theory, an sMI task with N-length allows 2 × (2(N)-1) possible commands with the left and right MI tasks under self-paced conditions. To verify its feasibility, the new method was used to construct a six-class BCI system to control the arm of a humanoid robot. Four subjects participated in our experiment and the averaged accuracy of the six-class sMI tasks was 89.4%. The Cohen's kappa coefficient and the throughput of our BCI paradigm are 0.88 ± 0.060 and 23.5bits per minute (bpm), respectively. Furthermore, all of the subjects could operate an actual three-joint robot arm to grasp an object in around 49.1s using our approach. These promising results suggest that the Morse code-inspired method could be used in the design of BCIs for multi-DOF control.

Optimizing spatial patterns with sparse filter bands for motor-imagery based brain-computer interface

BACKGROUND

Common spatial pattern (CSP) has been most popularly applied to motor-imagery (MI) feature extraction for classification in brain-computer interface (BCI) application. Successful application of CSP depends on the filter band selection to a large degree. However, the most proper band is typically subject-specific and can hardly be determined manually.

NEW METHOD

This study proposes a sparse filter band common spatial pattern (SFBCSP) for optimizing the spatial patterns. SFBCSP estimates CSP features on multiple signals that are filtered from raw EEG data at a set of overlapping bands. The filter bands that result in significant CSP features are then selected in a supervised way by exploiting sparse regression. A support vector machine (SVM) is implemented on the selected features for MI classification.

RESULTS

Two public EEG datasets (BCI Competition III dataset IVa and BCI Competition IV IIb) are used to validate the proposed SFBCSP method. Experimental results demonstrate that SFBCSP help improve the classification performance of MI.

COMPARISON WITH EXISTING METHODS

The optimized spatial patterns by SFBCSP give overall better MI classification accuracy in comparison with several competing methods.

CONCLUSIONS

The proposed SFBCSP is a potential method for improving the performance of MI-based BCI.

Audio-visual synchrony and feature-selective attention co-amplify early visual processing

Our brain relies on neural mechanisms of selective attention and converging sensory processing to efficiently cope with rich and unceasing multisensory inputs. One prominent assumption holds that audio-visual synchrony can act as a strong attractor for spatial attention. Here, we tested for a similar effect of audio-visual synchrony on feature-selective attention. We presented two superimposed Gabor patches that differed in colour and orientation. On each trial, participants were cued to selectively attend to one of the two patches. Over time, spatial frequencies of both patches varied sinusoidally at distinct rates (3.14 and 3.63 Hz), giving rise to pulse-like percepts. A simultaneously presented pure tone carried a frequency modulation at the pulse rate of one of the two visual stimuli to introduce audio-visual synchrony. Pulsed stimulation elicited distinct time-locked oscillatory electrophysiological brain responses. These steady-state responses were quantified in the spectral domain to examine individual stimulus processing under conditions of synchronous versus asynchronous tone presentation and when respective stimuli were attended versus unattended. We found that both, attending to the colour of a stimulus and its synchrony with the tone, enhanced its processing. Moreover, both gain effects combined linearly for attended in-sync stimuli. Our results suggest that audio-visual synchrony can attract attention to specific stimulus features when stimuli overlap in space.

NIRS-based classification of clench force and speed motor imagery with the use of empirical mode decomposition for BCI

Near-infrared spectroscopy (NIRS) is a non-invasive optical technique used for brain-computer interface (BCI). This study aims to investigate the brain hemodynamic responses of clench force and speed motor imagery and extract task-relevant features to obtain better classification performance. Given the non-stationary characteristics of real hemodynamic measurements, empirical mode decomposition (EMD) was applied to reduce the physiological noise overwhelmed in the task-relevant NIRS signals. Compared with continuous wavelet decomposition, EMD does not require a pre-determined basis function. EMD decomposes the original signals into a set of intrinsic mode functions (IMFs). In this study, joint mutual information was applied to select the optimal features, and support vector machine was used as a classifier. Offline and pseudo-online analyses showed that the most feasible classification accuracy can be obtained using IMFs as input features. Accordingly, an alternative feature is provided to develop the NIRS-BCI system.

Frequency-modulated steady-state visual evoked potentials: a new stimulation method for brain-computer interfaces

BACKGROUND

Steady-state visual evoked potentials (SSVEPs) are widely used for brain-computer interfaces. However, users experience fatigue due to exposure to flickering stimuli. High-frequency stimulation has been proposed to reduce this problem. We adapt frequency-modulated (FM) stimulation from the auditory domain, where it is commonly used to evoke steady-state responses, and compare the EEG as well as behavioral flicker perceptibility ratings.

NEW METHOD

We evoke SSVEPs with a green light-emitting diode (LED) driven by FM signals.

RESULTS

FM-SSVEPs with different carrier and modulation frequencies can reliably be evoked with spectral peaks at the lower FM sideband. Subjective perceptibility ratings decrease with increasing FM carrier frequencies, while the peak amplitude and signal-to-noise ratio (SNR) remain the same.

COMPARISON WITH EXISTING METHOD

There are neither amplitude nor SNR differences between SSVEPs evoked rectangularly, sinusoidally or via FM. Perceptibility ratings were lower for FM-SSVEPs with carrier frequencies of 20Hz and above than for sinusoidally evoked SSVEPs.

CONCLUSIONS

FM-SSVEPs seem to be beneficial for BCI usage. Reduced flicker perceptibility in FM-SSVEPs suggests reduced fatigue, which leads to an enhanced user experience and performance.

Modified CC-LR algorithm with three diverse feature sets for motor imagery tasks classification in EEG based brain-computer interface

Motor imagery (MI) tasks classification provides an important basis for designing brain-computer interface (BCI) systems. If the MI tasks are reliably distinguished through identifying typical patterns in electroencephalography (EEG) data, a motor disabled people could communicate with a device by composing sequences of these mental states. In our earlier study, we developed a cross-correlation based logistic regression (CC-LR) algorithm for the classification of MI tasks for BCI applications, but its performance was not satisfactory. This study develops a modified version of the CC-LR algorithm exploring a suitable feature set that can improve the performance. The modified CC-LR algorithm uses the C3 electrode channel (in the international 10-20 system) as a reference channel for the cross-correlation (CC) technique and applies three diverse feature sets separately, as the input to the logistic regression (LR) classifier. The present algorithm investigates which feature set is the best to characterize the distribution of MI tasks based EEG data. This study also provides an insight into how to select a reference channel for the CC technique with EEG signals considering the anatomical structure of the human brain. The proposed algorithm is compared with eight of the most recently reported well-known methods including the BCI III Winner algorithm. The findings of this study indicate that the modified CC-LR algorithm has potential to improve the identification performance of MI tasks in BCI systems. The results demonstrate that the proposed technique provides a classification improvement over the existing methods tested.

Transferring brain-computer interfaces beyond the laboratory: successful application control for motor-disabled users

OBJECTIVES

Brain-computer interfaces (BCIs) are no longer only used by healthy participants under controlled conditions in laboratory environments, but also by patients and end-users, controlling applications in their homes or clinics, without the BCI experts around. But are the technology and the field mature enough for this? Especially the successful operation of applications - like text entry systems or assistive mobility devices such as tele-presence robots - requires a good level of BCI control. How much training is needed to achieve such a level? Is it possible to train naïve end-users in 10 days to successfully control such applications?

MATERIALS AND METHODS

In this work, we report our experiences of training 24 motor-disabled participants at rehabilitation clinics or at the end-users' homes, without BCI experts present. We also share the lessons that we have learned through transferring BCI technologies from the lab to the user's home or clinics.

RESULTS

The most important outcome is that 50% of the participants achieved good BCI performance and could successfully control the applications (tele-presence robot and text-entry system). In the case of the tele-presence robot the participants achieved an average performance ratio of 0.87 (max. 0.97) and for the text entry application a mean of 0.93 (max. 1.0). The lessons learned and the gathered user feedback range from pure BCI problems (technical and handling), to common communication issues among the different people involved, and issues encountered while controlling the applications.

CONCLUSION

The points raised in this paper are very widely applicable and we anticipate that they might be faced similarly by other groups, if they move on to bringing the BCI technology to the end-user, to home environments and towards application prototype control.

EEG-based classification of imaginary left and right foot movements using beta rebound

OBJECTIVE

The purpose of this study was to investigate cortical lateralization of event-related (de)synchronization during left and right foot motor imagery tasks and to determine classification accuracy of the two imaginary movements in a brain-computer interface (BCI) paradigm.

METHODS

We recorded 31-channel scalp electroencephalograms (EEGs) from nine healthy subjects during brisk imagery tasks of left and right foot movements. EEG was analyzed with time-frequency maps and topographies, and the accuracy rate of classification between left and right foot movements was calculated.

RESULTS

Beta rebound at the end of imagination (increase of EEG beta rhythm amplitude) was identified from the two EEGs derived from the right-shift and left-shift bipolar pairs at the vertex. This process enabled discrimination between right or left foot imagery at a high accuracy rate (maximum 81.6% in single trial analysis).

CONCLUSION

These data suggest that foot motor imagery has potential to elicit left-right differences in EEG, while BCI using the unilateral foot imagery can achieve high classification accuracy, similar to ordinary BCI, based on hand motor imagery.

SIGNIFICANCE

By combining conventional discrimination techniques, the left-right discrimination of unilateral foot motor imagery provides a novel BCI system that could control a foot neuroprosthesis or a robotic foot.

Cortical effects of user training in a motor imagery based brain-computer interface measured by fNIRS and EEG

The present study aims to gain insights into the effects of training with a motor imagery (MI)-based brain-computer interface (BCI) on activation patterns of the sensorimotor cortex. We used functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) to investigate long-term training effects across 10 sessions using a 2-class (right hand and feet) MI-based BCI in fifteen subjects. In the course of the training a significant enhancement of activation pattern emerges, represented by an [oxy-Hb] increase in fNIRS and a stronger event-related desynchronization in the upper β-frequency band in the EEG. These effects were only visible in participants with relatively low BCI performance (mean accuracy ≤ 70%). We found that training with an MI-based BCI affects cortical activation patterns especially in users with low BCI performance. Our results may serve as a valuable contribution to the field of BCI research and provide information about the effects that training with an MI-based BCI has on cortical activation patterns. This might be useful for clinical applications of BCI which aim at promoting and guiding neuroplasticity.

A comparison study of two P300 speller paradigms for brain-computer interface

In this paper, a comparison of two existing P300 spellers is conducted. In the first speller, the visual stimuli of characters are presented in a single character (SC) paradigm and each button corresponding to a character flashes individually in a random order. The second speller is based on a region-based (RB) paradigm. In the first level, all characters are grouped and each button corresponding to a group flashes individually in a random order. Once a group is selected, the characters in it will appear on the flashing buttons of the second level for the selection of desired character. In a spelling experiment involving 12 subjects, higher online accuracy was obtained on the RB paradigm-based P300 speller than the SC paradigm-based P300 speller. Furthermore, we analyzed P300 detection performance, the P300 waveforms and Fisher ratios using the data collected by the two spellers. It was found that the stimuli display paradigm of the RB speller enhances P300 potential and is more suitable for P300 detection.