Adaptive semi-supervised classification to reduce intersession non-stationarity in multiclass motor imagery-based brain–computer interfaces

Joint multi-feature extraction and transfer learning in motor imagery brain computer interface

Motor imagery brain computer interface (BCI) systems are considered one of the most crucial paradigms and have received extensive attention from researchers worldwide. However, the non-stationary from subject-to-subject transfer is a substantial challenge for robust BCI operations. To address this issue, this paper proposes a novel approach that integrates joint multi-feature extraction, specifically combining common spatial patterns (CSP) and wavelet packet transforms (WPT), along with transfer learning (TL) in motor imagery BCI systems. This approach leverages the time-frequency characteristics of WPT and the spatial characteristics of CSP while utilizing transfer learning to facilitate EEG identification for target subjects based on knowledge acquired from non-target subjects. Using dataset IVa from BCI Competition III, our proposed approach achieves an impressive average classification accuracy of 93.4%, outperforming five kinds of state-of-the-art approaches. Furthermore, it offers the advantage of enabling the design of various auxiliary problems to learn different aspects of the target problem from unlabeled data through transfer learning, thereby facilitating the implementation of innovative ideas within our proposed approach. Simultaneously, it demonstrates that integrating CSP and WPT while transferring knowledge from other subjects is highly effective in enhancing the average classification accuracy of EEG signals and it provides a novel solution to address subject-to-subject transfer challenges in motor imagery BCI systems.

An end-to-end multi-task motor imagery EEG classification neural network based on dynamic fusion of spectral-temporal features

Electroencephalograph (EEG) brain-computer interfaces (BCI) have potential to provide new paradigms for controlling computers and devices. The accuracy of brain pattern classification in EEG BCI is directly affected by the quality of features extracted from EEG signals. Currently, feature extraction heavily relies on prior knowledge to engineer features (for example from specific frequency bands); therefore, better extraction of EEG features is an important research direction. In this work, we propose an end-to-end deep neural network that automatically finds and combines features for motor imagery (MI) based EEG BCI with 4 or more imagery classes (multi-task). First, spectral domain features of EEG signals are learned by compact convolutional neural network (CCNN) layers. Then, gated recurrent unit (GRU) neural network layers automatically learn temporal patterns. Lastly, an attention mechanism dynamically combines (across EEG channels) the extracted spectral-temporal features, reducing redundancy. We test our method using BCI Competition IV-2a and a data set we collected. The average classification accuracy on 4-class BCI Competition IV-2a was 85.1 % ± 6.19 %, comparable to recent work in the field and showing low variability among participants; average classification accuracy on our 6-class data was 64.4 % ± 8.35 %. Our dynamic fusion of spectral-temporal features is end-to-end and has relatively few network parameters, and the experimental results show its effectiveness and potential.

Recruiting neural field theory for data augmentation in a motor imagery brain-computer interface

We introduce a novel approach to training data augmentation in brain-computer interfaces (BCIs) using neural field theory (NFT) applied to EEG data from motor imagery tasks. BCIs often suffer from limited accuracy due to a limited amount of training data. To address this, we leveraged a corticothalamic NFT model to generate artificial EEG time series as supplemental training data. We employed the BCI competition IV '2a' dataset to evaluate this augmentation technique. For each individual, we fitted the model to common spatial patterns of each motor imagery class, jittered the fitted parameters, and generated time series for data augmentation. Our method led to significant accuracy improvements of over 2% in classifying the "total power" feature, but not in the case of the "Higuchi fractal dimension" feature. This suggests that the fit NFT model may more favorably represent one feature than the other. These findings pave the way for further exploration of NFT-based data augmentation, highlighting the benefits of biophysically accurate artificial data.

Feature optimization based on improved novel global harmony search algorithm for motor imagery electroencephalogram classification

Background

Effectively decoding electroencephalogram (EEG) pattern for specific mental tasks is a crucial topic in the development of brain-computer interface (BCI). Extracting common spatial pattern (CSP) features from motor imagery EEG signals is often highly dependent on the selection of frequency band and time interval. Therefore, optimizing frequency band and time interval would contribute to effective feature extraction and accurate EEG decoding.

Objective

This study proposes an approach based on an improved novel global harmony search (INGHS) to optimize frequency-time parameters for effective CSP feature extraction.

Methods

The INGHS algorithm is applied to find the optimal frequency band and temporal interval. The linear discriminant analysis and support vector machine are used for EEG pattern decoding. Extensive experimental studies are conducted on three EEG datasets to assess the effectiveness of our proposed method.

Results

The average test accuracy obtained by the time-frequency parameters selected by the proposed INGHS method is slightly better than artificial bee colony (ABC) and particle swarm optimization (PSO) algorithms. Furthermore, the INGHS algorithm is superior to PSO and ABC in running time.

Conclusion

These superior experimental results demonstrate that the optimal frequency band and time interval selected by the INGHS algorithm could significantly improve the decoding accuracy compared with the traditional CSP method. This method has a potential to improve the performance of MI-based BCI systems.

Simultaneous multiple-stimulus auditory brain-computer interface with semi-supervised learning and prior probability distribution tuning

Objective.Auditory brain-computer interfaces (BCIs) enable users to select commands based on the brain activity elicited by auditory stimuli. However, existing auditory BCI paradigms cannot increase the number of available commands without decreasing the selection speed, because each stimulus needs to be presented independently and sequentially under the standard oddball paradigm. To solve this problem, we propose a double-stimulus paradigm that simultaneously presents multiple auditory stimuli.Approach.For addition to an existing auditory BCI paradigm, the best discriminable sound was chosen following a subjective assessment. The new sound was located on the right-hand side and presented simultaneously with an existing sound from the left-hand side. A total of six sounds were used for implementing the auditory BCI with a 6 × 6 letter matrix. We employ semi-supervised learning (SSL) and prior probability distribution tuning to improve the accuracy of the paradigm. The SSL method involved updating of the classifier weights, and their prior probability distributions were adjusted using the following three types of distributions: uniform, empirical, and extended empirical (e-empirical). The performance was evaluated based on the BCI accuracy and information transfer rate (ITR).Main results.The double-stimulus paradigm resulted in a BCI accuracy of 67.89 ± 11.46% and an ITR of 2.67 ± 1.09 bits min^-1, in the absence of SSL and with uniform distribution. The proposed combination of SSL with e-empirical distribution improved the BCI accuracy and ITR to 74.59 ± 12.12% and 3.37 ± 1.27 bits min^-1, respectively. The event-related potential analysis revealed that contralateral and right-hemispheric dominances contributed to the BCI performance improvement.Significance.Our study demonstrated that a BCI based on multiple simultaneous auditory stimuli, incorporating SSL and e-empirical prior distribution, can increase the number of commands without sacrificing typing speed beyond the acceptable level of accuracy.

Machine Learning for Motor Imagery Wrist Dorsiflexion Prediction in Brain-Computer Interface Assisted Stroke Rehabilitation

Stroke is a life-changing event that can affect the survivors' physical, cognitive and emotional state. Stroke care focuses on helping the survivors to regain their strength; recover as much functionality as possible and return to independent living through rehabilitation therapies. Automated training protocols have been reported to improve the efficiency of the rehabilitation process. These protocols also decrease the dependency of the process on a professional trainer. Brain-Computer Interface (BCI) based systems are examples of such systems where they make use of the motor imagery (MI) based electroencephalogram (EEG) signals to drive the rehabilitation protocols. In this paper, we have proposed the use of well-known machine learning (ML) algorithms, such as, the decision tree (DT), Naive Bayesian (NB), linear discriminant analysis (LDA), support vector machine (SVM), ensemble learning classifier (ELC), and artificial neural network (ANN) for MI wrist dorsiflexion prediction in a BCI assisted stroke rehabilitation study conducted on eleven stroke survivors with either the left or right paresis. The doubling sub-band selection filter bank common spatial pattern (DSBS-FBCSP) has been proposed as feature extractor and it is observed that the ANN based classifier produces the best results.

Sequential Learning on sEMGs in Short- and Long-term Situations via Self-training Semi-supervised Support Vector Machine

The purpose of this study it to assess the effect of sequential learning of self-training support vector machine (ST-S3VM) on short- and long-term surface electromyogram (sEMG) datasets. A machine learning-based supervised classi-fier is enabling stable, complex, and high-performance motion control. Unlabeled sEMG measurements are easy by the devel-opment of wearable sensing technology. Thus, semi-supervised learning methods are attracted attention to utilize unlabeled sEMG data for supervised classifier with a small amount of labeled data. To evaluate robustness of ST-S3VM in realistic conditions, two public datasets which respectively contain a short- and long-term dataset were used. We compared the performance of ST-S3VM with four-kinds of SVM classifiers. In both short- and long-term situations, ST combined classifiers (ST-SVM and ST-S3VM) showed higher performances than the methods without ST (SVM and S3VM). In some cases, ST-S3VM had the best performance, but in other cases, ST-SVM had better performance than ST-S3VM. In order to make better use of unlabeled data, we will develop ST-S3VM to reduce the impact of harmful unlabeled data.

An adaptive closed-loop ECoG decoder for long-term and stable bimanual control of an exoskeleton by a tetraplegic

OBJECTIVE

The article aims at addressing 2 challenges to step motor BCI out of laboratories: asynchronous control of complex bimanual effectors with large numbers of degrees of freedom, using chronic and safe recorders, and the decoding performance stability over time without frequent decoder recalibration.

APPROACH

Closed-loop adaptive/incremental decoder training is one strategy to create a model stable over time. Adaptive decoders update their parameters with new incoming data, optimizing the model parameters in real time. It allows cross-session training with multiple recording conditions during closed loop BCI experiments. In the article, an adaptive tensor-based Recursive Exponentially Weighted Markov-Switching multi-Linear Model (REW-MSLM) decoder is proposed. REW-MSLM uses a Mixture of Expert (ME) architecture, mixing or switching independent decoders (experts) according to the probability estimated by a "gating" model. A Hidden Markov model approach is employed as gating model to improve the decoding robustness and to provide strong idle state support. The ME architecture fits the multi-limb paradigm associating an expert to a particular limb or action.

MAIN RESULTS

Asynchronous control of an exoskeleton by a tetraplegic patient using a chronically implanted epidural electrocorticography (EpiCoG) recorder is reported. The stable over a period of 6 months (without decoder recalibration) 8-dimensional alternative bimanual control of the exoskeleton and its virtual avatar is demonstrated.

SIGNIFICANCE

Based on the long-term (>36 months) chronic bilateral epidural ECoG recordings in a tetraplegic (ClinicalTrials.gov, NCT02550522), we addressed the poorly explored field of asynchronous bimanual BCI. The new decoder was designed to meet to several challenges: the high-dimensional control of a complex effector in experiments closer to real-world behaviour (point-to-point pursuit versus conventional center-out tasks), with the ability of the BCI system to act as a stand-alone device switching between idle and control states, and a stable performance over a long period of time without decoder recalibration.

Electroencephalography-Based Brain-Computer Interface Motor Imagery Classification

Background

Advances in the medical applications of brain-computer interface, like the motor imagery systems, are highly contributed to making the disabled live better. One of the challenges with such systems is to achieve high classification accuracy.

Methods

A highly accurate classification algorithm with low computational complexity is proposed here to classify different motor imageries and execution tasks. An experimental study is performed on two electroencephalography datasets (Iranian Brain-Computer Interface competition [iBCIC] dataset and the world BCI Competition IV dataset 2a) to validate the effectiveness of the proposed method. For lower complexity, the common spatial pattern is applied to decrease the 64 channel signal to four components, in addition to increase the class separability. From these components, first, some features are extracted in the time and time-frequency domains, and next, the best linear combination of these is selected by adopting the stepwise linear discriminant analysis (LDA) method, which are then applied in training and testing the classifiers including LDA, random forest, support vector machine, and K nearest neighbors. The classification strategy is of majority voting among the results of the binary classifiers.

Results

The experimental results indicate that the proposed algorithm accuracy is much higher than that of the winner of the first iBCIC. As to dataset 2a of the world BCI competition IV, the obtained results for subjects 6 and 9 outperform their counterparts. Moreover, this algorithm yields a mean kappa value of 0.53, which is higher than that of the second winner of the competition.

Conclusion

The results indicate that this method is able to classify motor imagery and execution tasks in both effective and automatic manners.

A Novel Classification Framework Using the Graph Representations of Electroencephalogram for Motor Imagery based Brain-Computer Interface

The motor imagery (MI) based brain-computer interfaces (BCIs) have been proposed as a potential physical rehabilitation technology. However, the low classification accuracy achievable with MI tasks is still a challenge when building effective BCI systems. We propose a novel MI classification model based on measurement of functional connectivity between brain regions and graph theory. Specifically, motifs describing local network structures in the brain are extracted from functional connectivity graphs. A graph embedding model called Ego-CNNs is then used to build a classifier, which can convert the graph from a structural representation to a fixed-dimensional vector for detecting critical structure in the graph. We validate our proposed method on four datasets, and the results show that our proposed method produces high classification accuracies in two-class classification tasks (92.8% for dataset 1, 93.4% for dataset 2, 96.5% for dataset 3, and 80.2% for dataset 4) and multiclass classification tasks (90.33% for dataset 1). Our proposed method achieves a mean Kappa value of 0.88 across nine participants, which is superior to other methods we compared it to. These results indicate that there is a local structural difference in functional connectivity graphs extracted under different motor imagery tasks. Our proposed method has great potential for motor imagery classification in future studies.

Recognizing motor imagery tasks from EEG oscillations through a novel ensemble-based neural network architecture

Brain-Computer Interfaces (BCI) provide effective tools aimed at recognizing different brain activities, translate them into actions, and enable humans to directly communicate through them. In this context, the need for strong recognition performances results in increasingly sophisticated machine learning (ML) techniques, which may result in poor performance in a real application (e.g., limiting a real-time implementation). Here, we propose an ensemble approach to effectively balance between ML performance and computational costs in a BCI framework. The proposed model builds a classifier by combining different ML models (base-models) that are specialized to different classification sub-problems. More specifically, we employ this strategy with an ensemble-based architecture consisting of multi-layer perceptrons, and test its performance on a publicly available electroencephalography-based BCI dataset with four-class motor imagery tasks. Compared to previously proposed models tested on the same dataset, the proposed approach provides greater average classification performances and lower inter-subject variability.

A novel method for classification of multi-class motor imagery tasks based on feature fusion

Motor imagery based brain computer interface (MI-BCI) has the advantage of strong independence that can rely on the spontaneous brain activity of the user to operate external devices. However, MI-BCI still has the problem of poor control effect, which requires more effective feature extraction algorithms and classification methods to extract distinctly separable features from electroencephalogram (EEG) signals. This paper proposes a novel framework based on Bispectrum, Entropy and common spatial pattern (BECSP). Here we use three methods of bispectrum in higher order spectra, entropy and CSP to extract MI-EEG signal features, and then select the most contributing features through tree-based feature selection algorithm. By comparing the classification results of SVM, Random Forest, Naive Bayes, LDA, KNN, Xgboost and Adaboost, we finally decide to use the SVM algorithm based on RBF kernel function which obtained the best result among them for classification. The proposed method is applied to the BCI competition IV data set 2a and BCI competition III data set IVa. On data set 2a, the highest accuracy on the evaluation data set reaches 85%. The experiment on data set IVa can also achieve good results. Compared with other algorithms that use the same data set, the performance of our algorithm has also been improved.

A Review on Signal Processing Approaches to Reduce Calibration Time in EEG-Based Brain-Computer Interface

In an electroencephalogram- (EEG-) based brain–computer interface (BCI), a subject can directly communicate with an electronic device using his EEG signals in a safe and convenient way. However, the sensitivity to noise/artifact and the non-stationarity of EEG signals result in high inter-subject/session variability. Therefore, each subject usually spends long and tedious calibration time in building a subject-specific classifier. To solve this problem, we review existing signal processing approaches, including transfer learning (TL), semi-supervised learning (SSL), and a combination of TL and SSL. Cross-subject TL can transfer amounts of labeled samples from different source subjects for the target subject. Moreover, Cross-session/task/device TL can reduce the calibration time of the subject for the target session, task, or device by importing the labeled samples from the source sessions, tasks, or devices. SSL simultaneously utilizes the labeled and unlabeled samples from the target subject. The combination of TL and SSL can take advantage of each other. For each kind of signal processing approaches, we introduce their concepts and representative methods. The experimental results show that TL, SSL, and their combination can obtain good classification performance by effectively utilizing the samples available. In the end, we draw a conclusion and point to research directions in the future.

Optimization of Model Training Based on Iterative Minimum Covariance Determinant In Motor-Imagery BCI

The common spatial patterns (CSP) algorithm is one of the most frequently used and effective spatial filtering methods for extracting relevant features for use in motor imagery brain-computer interfaces (MI-BCIs). However, the inherent defect of the traditional CSP algorithm is that it is highly sensitive to potential outliers, which adversely affects its performance in practical applications. In this work, we propose a novel feature optimization and outlier detection method for the CSP algorithm. Specifically, we use the minimum covariance determinant (MCD) to detect and remove outliers in the dataset, then we use the Fisher score to evaluate and select features. In addition, in order to prevent the emergence of new outliers, we propose an iterative minimum covariance determinant (IMCD) algorithm. We evaluate our proposed algorithm in terms of iteration times, classification accuracy and feature distribution using two BCI competition datasets. The experimental results show that the average classification performance of our proposed method is 12% and 22.9% higher than that of the traditional CSP method in two datasets ([Formula: see text]), and our proposed method obtains better performance in comparison with other competing methods. The results show that our method improves the performance of MI-BCI systems.

A Novel Time-Incremental End-to-End Shared Neural Network with Attention-Based Feature Fusion for Multiclass Motor Imagery Recognition

In the research of motor imagery brain-computer interface (MI-BCI), traditional electroencephalogram (EEG) signal recognition algorithms appear to be inefficient in extracting EEG signal features and improving classification accuracy. In this paper, we discuss a solution to this problem based on a novel step-by-step method of feature extraction and pattern classification for multiclass MI-EEG signals. First, the training data from all subjects is merged and enlarged through autoencoder to meet the need for massive amounts of data while reducing the bad effect on signal recognition because of randomness, instability, and individual variability of EEG data. Second, an end-to-end sharing structure with attention-based time-incremental shallow convolution neural network is proposed. Shallow convolution neural network (SCNN) and bidirectional long short-term memory (BiLSTM) network are used to extract frequency-spatial domain features and time-series features of EEG signals, respectively. Then, the attention model is introduced into the feature fusion layer to dynamically weight these extracted temporal-frequency-spatial domain features, which greatly contributes to the reduction of feature redundancy and the improvement of classification accuracy. At last, validation tests using BCI Competition IV 2a data sets show that classification accuracy and kappa coefficient have reached 82.7 ± 5.57% and 0.78 ± 0.074, which can strongly prove its advantages in improving classification accuracy and reducing individual difference among different subjects from the same network.

Early classification of motor tasks using dynamic functional connectivity graphs from EEG

OBJECTIVE

Classification of electroencephalography (EEG) signals with high accuracy using short recording intervals has been a challenging problem in developing brain computer interfaces (BCIs). This paper presents a novel feature extraction method for EEG recordings to tackle this problem.

APPROACH

The proposed approach is based on the concept that the brain functions in a dynamic manner, and utilizes dynamic functional connectivity graphs. The EEG data is first segmented into intervals during which functional networks sustain their connectivity. Functional connectivity networks for each identified segment are then localized, and graphs are constructed, which will be used as features. To take advantage of the dynamic nature of the generated graphs, a Long Short Term Memory (LSTM) classifier is employed for classification.

MAIN RESULTS

Features extracted from various durations of post-stimulus EEG data associated with motor execution and imagery tasks are used to test the performance of the classifier. Results show an average accuracy of 85.32% using features extracted from only 500 ms of the post-stimulus data.

SIGNIFICANCE

Our results demonstrate, for the first time, that using the proposed feature extraction method, it is possible to classify motor tasks from EEG recordings using a short interval of the data in the order of hundreds of milliseconds (e.g. 500 ms). This duration is considerably shorter than what has been reported before. These results will have significant implications for improving the effectiveness and the speed of BCIs, particularly for those used in assistive technologies.

Classification of error-related potentials from single-trial EEG in association with executed and imagined movements: a feature and classifier investigation

Error-related potentials (ErrPs) have been proposed for designing adaptive brain-computer interfaces (BCIs). Therefore, ErrPs must be decoded. The aim of this study was to evaluate ErrP decoding using combinations of different feature types and classifiers in BCI paradigms involving motor execution (ME) and imagination (MI). Fifteen healthy subjects performed 510 (ME) and 390 (MI) trials of right/left wrist extensions and foot dorsiflexions. Sham BCI feedback was delivered with an accuracy of 80% (ME) and 70% (MI). Continuous EEG was recorded and divided into ErrP and NonErrP epochs. Temporal, spectral, and discrete wavelet transform (DWT) marginals and template matching features were extracted, and all combinations of feature types were classified using linear discriminant analysis, support vector machine, and random forest classifiers. ErrPs were elicited for both ME and MI paradigms, and the average classification accuracies were significantly higher than the chance level. The highest average classification accuracy was obtained using temporal features and a combination of temporal + DWT features classified with random forest; 89 ± 9% and 83 ± 9% for ME and MI, respectively. These results generally indicate that temporal features should be used when detecting ErrPs, but there is great inter-subject variability, which means that user-specific features should be derived to maximize the performance. Graphical abstract.

Current Status, Challenges, and Possible Solutions of EEG-Based Brain-Computer Interface: A Comprehensive Review

Brain-Computer Interface (BCI), in essence, aims at controlling different assistive devices through the utilization of brain waves. It is worth noting that the application of BCI is not limited to medical applications, and hence, the research in this field has gained due attention. Moreover, the significant number of related publications over the past two decades further indicates the consistent improvements and breakthroughs that have been made in this particular field. Nonetheless, it is also worth mentioning that with these improvements, new challenges are constantly discovered. This article provides a comprehensive review of the state-of-the-art of a complete BCI system. First, a brief overview of electroencephalogram (EEG)-based BCI systems is given. Secondly, a considerable number of popular BCI applications are reviewed in terms of electrophysiological control signals, feature extraction, classification algorithms, and performance evaluation metrics. Finally, the challenges to the recent BCI systems are discussed, and possible solutions to mitigate the issues are recommended.

EEG-based Classification of Lower Limb Motor Imagery with Brain Network Analysis

This study aims to investigate the difference in cortical signal characteristics between the left and right foot imaginary movements and to improve the classification accuracy of the experimental tasks. Raw signals were gathered from 64-channel scalp electroencephalograms of 11 healthy participants. Firstly, the cortical source model was defined with 62 regions of interest over the sensorimotor cortex (nine Brodmann areas). Secondly, functional connectivity was calculated by phase lock value for α and β rhythm networks. Thirdly, network-based statistics were applied to identify whether there existed stable and significant subnetworks that formed between the two types of motor imagery tasks. Meanwhile, ten graph theory indices were investigated for each network by t-test to determine statistical significance between tasks. Finally, sparse multinomial logistic regression (SMLR)-support vector machine (SVM), as a feature selection and classification model, was used to analyze the graph theory features. The specific time-frequency (α event-related desynchronization and β event-related synchronization) difference network between the two tasks was congregated at the midline and demonstrated significant connections in the premotor areas and primary somatosensory cortex. A few of statistically significant differences in the network properties were observed between tasks in the α and β rhythm. The SMLR-SVM classification model achieved fair discrimination accuracy between imaginary movements of the two feet (maximum 75% accuracy rate in single-trial analyses). This study reveals the network mechanism of the discrimination of the left and right foot motor imagery, which can provide a novel avenue for the BCI system by unilateral lower limb motor imagery.

Adaptive feature extraction in EEG-based motor imagery BCI: tracking mental fatigue

OBJECTIVE

Electroencephalogram (EEG) signals are non-stationary. This could be due to internal fluctuation of brain states such as fatigue, frustration, etc. This necessitates the development of adaptive brain-computer interfaces (BCI) whose performance does not deteriorate significantly with the adversary change in the cognitive state. In this paper, we put forward an unsupervised adaptive scheme to adapt the feature extractor of motor imagery (MI) BCIs by tracking the fatigue level of the user.

APPROACH

Eleven subjects participated in the study during which they accomplished MI tasks while self-reporting their perceived levels of mental fatigue. Out of the 11 subjects, only six completed the whole experiment, while the others quit in the middle because of experiencing high fatigue. The adaptive feature extractor is attained through the adaptation of the common spatial patterns (CSP), one of the most popular feature extraction algorithms in EEG-based BCIs. The proposed method was analyzed in two ways: offline and in near real-time. The separability of the MI EEG features extracted by the proposed adaptive CSP (ADCSP) has been compared with that by the conventional CSP (C-CSP) and another CSP based adaptive method (ACSP) in terms of: Davies Bouldin index (DBI), Fisher score (FS) and Dunn's index (DI).

MAIN RESULTS

Experimental results show significant improvement in the separability of MI EEG features extracted by ADCSP as compared to that by C-CSP and ACSP.

SIGNIFICANCE

Collectively, the results of the experiments in this study suggest that adapting CSP based on mental fatigue can improve the class separability of MI EEG features.

Data Space Adaptation for Multiclass Motor Imagery-based BCI

Various adaptation techniques have been proposed to address the non-stationarity issue faced by electroencephalogram (EEG)-based brain-computer interfaces (BCIs). However, most of these adaptation techniques are only suitable for binary-class BCIs. This paper proposes a supervised multiclass data space adaptation technique (MDSA) to transform the test data using a linear transformation such that the distribution difference between the multiclass train and test data is minimized. The results of using the proposed MDSA on BCI Competition IV dataset 2a improved the classification accuracy by an average of 4.3% when 20 trials per class were used from the test session to estimate adaptation transformation. The results also showed that the proposed MDSA algorithm outperformed the multi pooled mean linear discrimination (MPMLDA) technique with as few as 10 trials per class used for calculating the transformation matrix. Hence the results showed the effectiveness of the proposed MDSA algorithm in addressing non-stationarity issue for multiclass EEG-based BCI.

A novel hybrid deep learning scheme for four-class motor imagery classification

OBJECTIVE

Learning the structures and unknown correlations of a motor imagery electroencephalogram (MI-EEG) signal is important for its classification. It is also a major challenge to obtain good classification accuracy from the increased number of classes and increased variability from different people. In this study, a four-class MI task is investigated.

APPROACH

An end-to-end novel hybrid deep learning scheme is developed to decode the MI task from EEG data. The proposed algorithm consists of two parts: a. A one-versus-rest filter bank common spatial pattern is adopted to preprocess and pre-extract the features of the four-class MI signal. b. A hybrid deep network based on the convolutional neural network and long-term short-term memory network is proposed to extract and learn the spatial and temporal features of the MI signal simultaneously.

MAIN RESULTS

The main contribution of this paper is to propose a hybrid deep network framework to improve the classification accuracy of the four-class MI-EEG signal. The hybrid deep network is a subject-independent shared neural network, which means it can be trained by using the training data from all subjects to form one model.

SIGNIFICANCE

The classification performance obtained by the proposed algorithm on brain-computer interface (BCI) competition IV dataset 2a in terms of accuracy is 83% and Cohen's kappa value is 0.80. Finally, the shared hybrid deep network is evaluated by every subject respectively, and the experimental results illustrate that the shared neural network has satisfactory accuracy. Thus, the proposed algorithm could be of great interest for real-life BCIs.

Direct comparison of supervised and semi-supervised retraining approaches for co-adaptive BCIs

For Brain-Computer interfaces (BCIs), system calibration is a lengthy but necessary process for successful operation. Co-adaptive BCIs aim to shorten training and imply positive motivation to users by presenting feedback already at early stages: After just 5 min of gathering calibration data, the systems are able to provide feedback and engage users in a mutual learning process. In this work, we investigate whether the retraining stage of co-adaptive BCIs can be adapted to a semi-supervised concept, where only a small amount of labeled data is available and all additional data needs to be labeled by the BCI itself. The aim of the current work was to evaluate whether a semi-supervised co-adaptive BCI could successfully compete with a supervised co-adaptive BCI model. In a supporting two-class (190 trials per condition) BCI study based on motor imagery tasks, we evaluated both approaches in two separate groups of 10 participants online, while we simulated the other approach in each group offline. Our results indicate that despite the lack of true labeled data, the semi-supervised driven BCI did not perform significantly worse (p > 0.05) than the supervised counterpart. We believe that these findings contribute to developing BCIs for long-term use, where continuous adaptation becomes imperative for maintaining meaningful BCI performance.

Graphical abstract

Adaptive classification to reduce non-stationarity in visual evoked potential brain-computer interfaces

Non-stationarity of electroencephalogram (EEG) signals greatly affect classifier performance in brain-computer interface (BCI). To overcome this problem we propose an adaptive classifier model known as extended multi-class pooled mean linear discriminant analysis (EMPMLDA). Here, we update the average class pair co-variance matrix along with pooled mean values. Evaluation of classifiers are done on visual evoked cortical potential data-sets. We demonstrate that EMPMLDA can significantly outperform other static classifiers such as MLDA and adaptive classifiers (MPMLDA). Furthermore an optimal update coefficient can be achieved using different datasets.

An Optimized Channel Selection Method Based on Multifrequency CSP-Rank for Motor Imagery-Based BCI System

BACKGROUND

Due to the redundant information contained in multichannel electroencephalogram (EEG) signals, the classification accuracy of brain-computer interface (BCI) systems may deteriorate to a large extent. Channel selection methods can help to remove task-independent electroencephalogram (EEG) signals and hence improve the performance of BCI systems. However, in different frequency bands, brain areas associated with motor imagery are not exactly the same, which will result in the inability of traditional channel selection methods to extract effective EEG features.

NEW METHOD

To address the above problem, this paper proposes a novel method based on common spatial pattern- (CSP-) rank channel selection for multifrequency band EEG (CSP-R-MF). It combines the multiband signal decomposition filtering and the CSP-rank channel selection methods to select significant channels, and then linear discriminant analysis (LDA) was used to calculate the classification accuracy.

RESULTS

The results showed that our proposed CSP-R-MF method could significantly improve the average classification accuracy compared with the CSP-rank channel selection method.

Adaptive multi-degree of freedom Brain Computer Interface using online feedback: Towards novel methods and metrics of mutual adaptation between humans and machines for BCI

This paper proposes a novel adaptive online-feedback methodology for Brain Computer Interfaces (BCI). The method uses ElectroEncephaloGraphic (EEG) signals and combines motor with speech imagery to allow for tasks that involve multiple degrees of freedom (DoF). The main approach utilizes the covariance matrix descriptor as feature, and the Relevance Vector Machines (RVM) classifier. The novel contributions include, (1) a new method to select representative data to update the RVM model, and (2) an online classifier which is an adaptively-weighted mixture of RVM models to account for the users' exploration and exploitation processes during the learning phase. Instead of evaluating the subjects' performance solely based on the conventional metric of accuracy, we analyze their skill's improvement based on 3 other criteria, namely the confusion matrix's quality, the separability of the data, and their instability. After collecting calibration data for 8 minutes in the first run, 8 participants were able to control the system while receiving visual feedback in the subsequent runs. We observed significant improvement in all subjects, including two of them who fell into the BCI illiteracy category. Our proposed BCI system complements the existing approaches in several aspects. First, the co-adaptation paradigm not only adapts the classifiers, but also allows the users to actively discover their own way to use the BCI through their exploration and exploitation processes. Furthermore, the auto-calibrating system can be used immediately with a minimal calibration time. Finally, this is the first work to combine motor and speech imagery in an online feedback experiment to provide multiple DoF for BCI control applications.

Towards correlation-based time window selection method for motor imagery BCIs

The start of the cue is often used to initiate the feature window used to control motor imagery (MI)-based brain-computer interface (BCI) systems. However, the time latency during an MI period varies between trials for each participant. Fixing the starting time point of MI features can lead to decreased system performance in MI-based BCI systems. To address this issue, we propose a novel correlation-based time window selection (CTWS) algorithm for MI-based BCIs. Specifically, the optimized reference signals for each class were selected based on correlation analysis and performance evaluation. Furthermore, the starting points of time windows for both training and testing samples were adjusted using correlation analysis. Finally, the feature extraction and classification algorithms were used to calculate the classification accuracy. With two datasets, the results demonstrate that the CTWS algorithm significantly improved the system performance when compared to directly using feature extraction approaches. Importantly, the average improvement in accuracy of the CTWS algorithm on the datasets of healthy participants and stroke patients was 16.72% and 5.24%, respectively when compared to traditional common spatial pattern (CSP) algorithm. In addition, the average accuracy increased 7.36% and 9.29%, respectively when the CTWS was used in conjunction with Sub-Alpha-Beta Log-Det Divergences (Sub-ABLD) algorithm. These findings suggest that the proposed CTWS algorithm holds promise as a general feature extraction approach for MI-based BCIs.

Identification of Anisomerous Motor Imagery EEG Signals Based on Complex Algorithms

Motor imagery (MI) electroencephalograph (EEG) signals are widely applied in brain-computer interface (BCI). However, classified MI states are limited, and their classification accuracy rates are low because of the characteristics of nonlinearity and nonstationarity. This study proposes a novel MI pattern recognition system that is based on complex algorithms for classifying MI EEG signals. In electrooculogram (EOG) artifact preprocessing, band-pass filtering is performed to obtain the frequency band of MI-related signals, and then, canonical correlation analysis (CCA) combined with wavelet threshold denoising (WTD) is used for EOG artifact preprocessing. We propose a regularized common spatial pattern (R-CSP) algorithm for EEG feature extraction by incorporating the principle of generic learning. A new classifier combining the K-nearest neighbor (KNN) and support vector machine (SVM) approaches is used to classify four anisomerous states, namely, imaginary movements with the left hand, right foot, and right shoulder and the resting state. The highest classification accuracy rate is 92.5%, and the average classification accuracy rate is 87%. The proposed complex algorithm identification method can significantly improve the identification rate of the minority samples and the overall classification performance.

Towards psychologically adaptive brain-computer interfaces

OBJECTIVE

Brain-computer interface (BCI) performance is sensitive to short-term changes in psychological states such as fatigue, frustration, and attention. This paper explores the design of a BCI that can adapt to these short-term changes.

APPROACH

Eleven able-bodied individuals participated in a study during which they used a mental task-based EEG-BCI to play a simple maze navigation game while self-reporting their perceived levels of fatigue, frustration, and attention. In an offline analysis, a regression algorithm was trained to predict changes in these states, yielding Pearson correlation coefficients in excess of 0.45 between the self-reported and predicted states. Two means of fusing the resultant mental state predictions with mental task classification were investigated. First, single-trial mental state predictions were used to predict correct classification by the BCI during each trial. Second, an adaptive BCI was designed that retrained a new classifier for each testing sample using only those training samples for which predicted mental state was similar to that predicted for the current testing sample.

MAIN RESULTS

Mental state-based prediction of BCI reliability exceeded chance levels. The adaptive BCI exhibited significant, but practically modest, increases in classification accuracy for five of 11 participants and no significant difference for the remaining six despite a smaller average training set size.

SIGNIFICANCE

Collectively, these findings indicate that adaptation to psychological state may allow the design of more accurate BCIs.

An Asynchronous P300-Based Brain-Computer Interface Web Browser for Severely Disabled People

This paper presents an electroencephalographic (EEG) P300-based brain-computer interface (BCI) Internet browser. The system uses the "odd-ball" row-col paradigm for generating the P300 evoked potentials on the scalp of the user, which are immediately processed and translated into web browser commands. There were previous approaches for controlling a BCI web browser. However, to the best of our knowledge, none of them was focused on an assistive context, failing to test their applications with a suitable number of end users. In addition, all of them were synchronous applications, where it was necessary to introduce a "read-mode" command in order to avoid a continuous command selection. Thus, the aim of this study is twofold: 1) to test our web browser with a population of multiple sclerosis (MS) patients in order to assess the usefulness of our proposal to meet their daily communication needs; and 2) to overcome the aforementioned limitation by adding a threshold that discerns between control and non-control states, allowing the user to calmly read the web page without undesirable selections. The browser was tested with sixteen MS patients and five healthy volunteers. Both quantitative and qualitative metrics were obtained. MS participants reached an average accuracy of 84.14%, whereas 95.75% was achieved by control subjects. Results show that MS patients can successfully control the BCI web browser, improving their personal autonomy.

Assembling A Multi-Feature EEG Classifier for Left–Right Motor Imagery Data Using Wavelet-Based Fuzzy Approximate Entropy for Improved Accuracy

An EEG classifier is proposed for application in the analysis of motor imagery (MI) EEG data from a brain–computer interface (BCI) competition in this study. Applying subject-action-related brainwave data acquired from the sensorimotor cortices, the system primarily consists of artifact and background removal, feature extraction, feature selection and classification. In addition to background noise, the electrooculographic (EOG) artifacts are also automatically removed to further improve the analysis of EEG signals. Several potential features, including amplitude modulation, spectral power and asymmetry ratio, adaptive autoregressive model, and wavelet fuzzy approximate entropy (wfApEn) that can measure and quantify the complexity or irregularity of EEG signals, are then extracted for subsequent classification. Finally, the significant sub-features are selected from feature combination by quantum-behaved particle swarm optimization and then classified by support vector machine (SVM). Compared with feature extraction without wfApEn on MI data from two data sets for nine subjects, the results indicate that the proposed system including wfApEn obtains better performance in average classification accuracy of 88.2% and average number of commands per minute of 12.1, which is promising in the BCI work applications.