Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review

Machine-learning model predicting postoperative delirium in older patients using intraoperative frontal electroencephalographic signatures

Objective

In older patients receiving general anesthesia, postoperative delirium (POD) is the most frequent form of cerebral dysfunction. Early identification of patients at higher risk to develop POD could provide the opportunity to adapt intraoperative and postoperative therapy. We, therefore, propose a machine learning approach to predict the risk of POD in elderly patients, using routine intraoperative electroencephalography (EEG) and clinical data that are readily available in the operating room.

Methods

We conducted a retrospective analysis of the data of a single-center study at the Charité-Universitätsmedizin Berlin, Department of Anesthesiology [ISRCTN 36437985], including 1,277 patients, older than 60 years with planned surgery and general anesthesia. To deal with the class imbalance, we used balanced ensemble methods, specifically Bagging and Random Forests and as a performance measure, the area under the ROC curve (AUC-ROC). We trained our models including basic clinical parameters and intraoperative EEG features in particular classical spectral and burst suppression signatures as well as multi-band covariance matrices, which were classified, taking advantage of the geometry of a Riemannian manifold. The models were validated with 10 repeats of a 10-fold cross-validation.

Results

Including EEG data in the classification resulted in a robust and reliable risk evaluation for POD. The clinical parameters alone achieved an AUC-ROC score of 0.75. Including EEG signatures improved the classification when the patients were grouped by anesthetic agents and evaluated separately for each group. The spectral features alone showed an AUC-ROC score of 0.66; the covariance features showed an AUC-ROC score of 0.68. The AUC-ROC scores of EEG features relative to patient data differed by anesthetic group. The best performance was reached, combining both the EEG features and the clinical parameters. Overall, the AUC-ROC score was 0.77, for patients receiving Propofol it was 0.78, for those receiving Sevoflurane it was 0.8 and for those receiving Desflurane 0.73. Applying the trained prediction model to an independent data set of a different clinical study confirmed these results for the combined classification, while the classifier on clinical parameters alone did not generalize.

Conclusion

A machine learning approach combining intraoperative frontal EEG signatures with clinical parameters could be an easily applicable tool to early identify patients at risk to develop POD.

Riemannian distance based channel selection and feature extraction combining discriminative time-frequency bands and Riemannian tangent space for MI-BCIs

OBJECTIVE

Motor imagery-based brain computer interfaces (MI-BCIs) have been widely researched because they do not demand external stimuli and have a high degree of maneuverability. In most scenarios, superabundant selected channels, fixed time windows, and frequency bands would certainly affect the performance of MI-BCIs due to the neurophysiological diversities between different individuals. In this study, we tended to effectively use the Riemannian geometry of spatial covariance matrix to extract more robust features and thus enhance the decoding efficiency.

APPROACH

First, we propose a Riemannian distance-based EEG channel selection method (RDCS), which preliminarily reduces the information redundancy in the first stage. Second, we extract discriminative Riemannian Tangent Space features of EEG signals of selected channels from the most discriminant time-frequency bands (DTFRTS) to further enhance decoding accuracy for MI-BCIs. Finally, we trained a support vector machine (SVM) model with a linear kernel to classify our extracted discriminative Riemannian features and evaluated our proposed method using publicly available BCI Competition Ⅳ dataset Ⅰ (DS1) and Competition Ⅲ dataset Ⅲa (DS2).

MAIN RESULTS

The experimental results showed that the average classification accuracy with the selected 10-channel EEG signals of our method is 88.1% and 91.6% in DS1 and DS2 respectively. The average improvements are 24.3% & 27.1% on DS1 and 4.4% & 14.2% on DS2 for 10 & 20 selected channels, respectively.

SIGNIFICANCE

These results showed that our proposed method is a promising candidate for performance improvement of MI-BCIs.

RNeuMark: A Riemannian EEG Analysis Framework for Neuromarketing

Neuromarketing exploits neuroimaging techniques so as to reinforce the predictive power of conventional marketing tools, like questionnaires and focus groups. Electroencephalography (EEG) is the most commonly encountered neuroimaging technique due to its non-invasiveness, low-cost, and its very recent embedding in wearable devices. The transcription of brainwave patterns to consumer attitude is supported by various signal descriptors, while the quest for profitable novel ways is still an open research question. Here, we suggest the use of sample covariance matrices as alternative descriptors, that encapsulate the coordinated neural activity from distinct brain areas, and the adoption of Riemannian geometry for their handling. We first establish the suitability of Riemannian approach for neuromarketing-related problems and then suggest a relevant decoding scheme for predicting consumers' choices (e.g., willing to buy or not a specific product). Since the decision-making process involves the concurrent interaction of various cognitive processes and consequently of distinct brain rhythms, the proposed decoder takes the form of an ensemble classifier that builds upon a multi-view perspective, with each view dedicated to a specific frequency band. Adopting a standard machine learning procedure, and using a set of trials (training data) in conjunction with the associated behavior labels ("buy"/ "not buy"), we train a battery of classifiers accordingly. Each classifier is designed to operate in the space recovered from the inter-trial distances of SCMs and to cast a rhythm-depended decision that is eventually combined with the predictions of the rest ones. The demonstration and evaluation of the proposed approach are performed in 2 neuromarketing-related datasets of different nature. The first is employed to showcase the potential of the suggested descriptor, while the second to showcase the decoder's superiority against popular alternatives in the field.

Decoding EEG rhythms offline and online during motor imagery for standing and sitting based on a brain-computer interface

Motor imagery (MI)-based brain-computer interface (BCI) systems have shown promising advances for lower limb motor rehabilitation. The purpose of this study was to develop an MI-based BCI for the actions of standing and sitting. Thirty-two healthy subjects participated in the study using 17 active EEG electrodes. We used a combination of the filter bank common spatial pattern (FBCSP) method and the regularized linear discriminant analysis (RLDA) technique for decoding EEG rhythms offline and online during motor imagery for standing and sitting. The offline analysis indicated the classification of motor imagery and idle state provided a mean accuracy of 88.51 ± 1.43% and 85.29 ± 1.83% for the sit-to-stand and stand-to-sit transitions, respectively. The mean accuracies of the sit-to-stand and stand-to-sit online experiments were 94.69 ± 1.29% and 96.56 ± 0.83%, respectively. From these results, we believe that the MI-based BCI may be useful to future brain-controlled standing systems.

Decoding of Turning Intention during Walking Based on EEG Biomarkers

In the EEG literature, there is a lack of asynchronous intention models that realistically propose interfaces for applications that must operate in real time. In this work, a novel BMI approach to detect in real time the intention to turn is proposed. For this purpose, an offline, pseudo-online and online analysis is presented to validate the EEG as a biomarker for the intention to turn. This article presents a methodology for the creation of a BMI that could differentiate two classes: monotonous walk and intention to turn. A comparison of some of the most popular algorithms in the literature is conducted. To filter the signal, two relevant algorithms are used: H∞ filter and ASR. For processing and classification, the mean of the covariance matrices in the Riemannian space was calculated and then, with various classifiers of different types, the distance of the test samples to each class in the Riemannian space was estimated. This dispenses with power-based models and the necessary baseline correction, which is a problem in realistic scenarios. In the cross-validation for a generic selection (valid for any subject) and a personalized one, the results were, on average, 66.2% and 69.6% with the best filter H∞. For the pseudo-online, the custom configuration for each subject was an average of 40.2% TP and 9.3 FP/min; the best subject obtained 43.9% TP and 2.9 FP/min. In the final validation test, this subject obtained 2.5 FP/min and an accuracy rate of 71.43%, and the turn anticipation was 0.21 s on average.

EEGSym: Overcoming Inter-Subject Variability in Motor Imagery Based BCIs With Deep Learning

In this study, we present a new Deep Learning (DL) architecture for Motor Imagery (MI) based Brain Computer Interfaces (BCIs) called EEGSym. Our implementation aims to improve previous state-of-the-art performances on MI classification by overcoming inter-subject variability and reducing BCI inefficiency, which has been estimated to affect 10-50% of the population. This convolutional neural network includes the use of inception modules, residual connections and a design that introduces the symmetry of the brain through the mid-sagittal plane into the network architecture. It is complemented with a data augmentation technique that improves the generalization of the model and with the use of transfer learning across different datasets. We compare EEGSym's performance on inter-subject MI classification with ShallowConvNet, DeepConvNet, EEGNet and EEG-Inception. This comparison is performed on 5 publicly available datasets that include left or right hand motor imagery of 280 subjects. This population is the largest that has been evaluated in similar studies to date. EEGSym significantly outperforms the baseline models reaching accuracies of 88.6±9.0 on Physionet, 83.3±9.3 on OpenBMI, 85.1±9.5 on Kaya2018, 87.4±8.0 on Meng2019 and 90.2±6.5 on Stieger2021. At the same time, it allows 95.7% of the tested population (268 out of 280 users) to reach BCI control (≥70% accuracy). Furthermore, these results are achieved using only 16 electrodes of the more than 60 available on some datasets. Our implementation of EEGSym, which includes new advances for EEG processing with DL, outperforms previous state-of-the-art approaches on inter-subject MI classification.

Mental State Classification Using Multi-Graph Features

We consider the problem of extracting features from passive, multi-channel electroencephalogram (EEG) devices for downstream inference tasks related to high-level mental states such as stress and cognitive load. Our proposed feature extraction method uses recently developed spectral-based multi-graph tools and applies them to the time series of graphs implied by the statistical dependence structure (e.g., correlation) amongst the multiple sensors. We study the features in the context of two datasets each consisting of at least 30 participants and recorded using multi-channel EEG systems. We compare the classification performance of a classifier trained on the proposed features to a classifier trained on the traditional band power-based features in three settings and find that the two feature sets offer complementary predictive information. We conclude by showing that the importance of particular channels and pairs of channels for classification when using the proposed features is neuroscientifically valid.

Geometrical modelling of neuronal clustering and development

The dynamic geometry of neuronal development is an essential concept in theoretical neuroscience. We aimed to design a mathematical model which outlines stepwise in an innovative form and designed to model neuronal development geometrically and modelling spatially the neuronal-electrical field interaction. We demonstrated flexibility in forming the cell and its nucleus to show neuronal growth from inside to outside that uses a fractal cylinder to generate neurons (pyramidal/sphere) in form of mathematically called ‘surface of revolution’. Furthermore, we verified the effect of the adjacent neurons on a free branch from one-side, by modelling a ‘normal vector surface’ that represented a group of neurons. Our model also indicated how the geometrical shapes and clustering of the neurons can be transformed mathematically in the form of vector field that is equivalent to the neuronal electromagnetic activity/electric flux. We further simulated neuronal-electrical field interaction that was implemented spatially using Van der Pol oscillator and taking Laplacian vector field as it reflects biophysical mechanism of neuronal activity and geometrical change. In brief, our study would be considered a proper platform and inspiring modelling for next more complicated geometrical and electrical constructions.

Class-distinctiveness-based frequency band selection on the Riemannian manifold for oscillatory activity-based BCIs: preliminary results

Considering user-specific settings is known to enhance Brain-Computer Interface (BCI) performances. In particular, the optimal frequency band for oscillatory activity classification is highly user-dependent and many frequency band selection methods have been developed in the past two decades. However, it is not well studied whether those conventional methods can be efficiently applied to the Riemannian BCIs, a recent family of BCI systems that utilize the non-Euclidean nature of the data unlike conventional BCI pipelines. In this paper, we proposed a novel frequency band selection method working on the Riemannian manifold. The frequency band is selected considering the class distinctiveness as quantified based on the inter-class distance and the intra-class variance on the manifold. An advantage of this method is that the frequency bandwidth can be adjusted for each individual without intensive optimization steps. In a comparative experiment using a public dataset of motor imagery-based BCI, our method showed a substantial improvement in average accuracy over both a fixed broad frequency band and a popular conventional frequency band selection method. In particular, our method substantially improved performances for subjects with initially low accuracies. This preliminary result suggests the importance of developing new user-specific setting algorithms considering the manifold properties, rather than directly applying methods developed prior to the rise of the Riemannian BCIs.

Pseudo-online detection and classification for upper-limb movements

Objective. This study analyzed detection (movement vs. non-movement) and classification (different types of movements) to decode upper-limb movement volitions in a pseudo-online fashion.Approach. Nine healthy subjects executed four self-initiated movements: left wrist extension, right wrist extension, left index finger extension, and right index finger extension. For detection, we investigated the performance of three individual classifiers (support vector machine (SVM), EEGNET, and Riemannian geometry featured SVM) on three frequency bands (0.05-5 Hz, 5-40 Hz, 0.05-40 Hz). The best frequency band and the best classifier combinations were constructed to realize an ensemble processing pipeline using majority voting. For classification, we used adaptive boosted Riemannian geometry model to differentiate contra-lateral and ipsilateral movements.Main results. The ensemble model achieved 79.6 ± 8.8% true positive rate and 3.1 ± 1.2 false positives per minute with 75.3 ± 112.6 ms latency on a pseudo-online detection task. The following classification gave around 67% accuracy to differentiate contralateral movements.Significance. The newly proposed ensemble method and pseudo-online testing procedure could provide a robust brain-computer interface design for movement decoding.

Epilepsy Detection Based on Riemann Potato in Noisy Environment

Epilepsy detection based on electroencephalogram (EEG) is important for the diagnosis and treatment of epilepsy. The existing feature extraction method not only consumes a lot of time but also leads to epilepsy information loss because of nonideal denoising. Therefore, the paper proposes to use noisy EEG signals to detect epilepsy. The original EEG signal is divided into normal signal and abnormal signal by Riemann potato, and the epilepsy detection model is established based on the normal signal and abnormal signal, respectively. Finally, the 2 detection results are combined as a final result. The detection performance of 94.84%, 83.03% sensitivity, and 97.67% specificity is achieved. The experimental results show that the original noisy signal which is separated by the Riemann potato can have high epilepsy detection performance.

EEG emotion recognition based on enhanced SPD matrix and manifold dimensionality reduction

Recently, Riemannian geometry-based pattern recognition has been widely employed to brain computer interface (BCI) researches, providing new idea for emotion recognition based on electroencephalogram (EEG) signals. Although the symmetric positive definite (SPD) matrix manifold constructed from the traditional covariance matrix contains large amount of spatial information, these methods do not perform well to classify and recognize emotions, and the high dimensionality problem still unsolved. Therefore, this paper proposes a new strategy for EEG emotion recognition utilizing Riemannian geometry with the aim of achieving better classification performance. The emotional EEG signals of 32 healthy subjects were from an open-source dataset (DEAP). The wavelet packets were first applied to extract the time-frequency features of the EEG signals, and then the features were used to construct the enhanced SPD matrix. A supervised dimensionality reduction algorithm was then designed on the Riemannian manifold to reduce the high dimensionality of the SPD matrices, gather samples of the same labels together, and separate samples of different labels as much as possible. Finally, the samples were mapped to the tangent space, and the K-nearest neighbors (KNN), Random Forest (RF) and Support Vector Machine (SVM) method were employed for classification. The proposed method achieved an average accuracy of 91.86%, 91.84% on the valence and arousal recognition tasks. Furthermore, we also obtained the superior accuracy of 86.71% on the four-class recognition task, demonstrated the superiority over state-of-the-art emotion recognition methods.

Harmonized-Multinational qEEG norms (HarMNqEEG)

This paper extends frequency domain quantitative electroencephalography (qEEG) methods pursuing higher sensitivity to detect Brain Developmental Disorders. Prior qEEG work lacked integration of cross-spectral information omitting important functional connectivity descriptors. Lack of geographical diversity precluded accounting for site-specific variance, increasing qEEG nuisance variance. We ameliorate these weaknesses. (i) Create lifespan Riemannian multinational qEEG norms for cross-spectral tensors. These norms result from the HarMNqEEG project fostered by the Global Brain Consortium. We calculate the norms with data from 9 countries, 12 devices, and 14 studies, including 1564 subjects. Instead of raw data, only anonymized metadata and EEG cross-spectral tensors were shared. After visual and automatic quality control, developmental equations for the mean and standard deviation of qEEG traditional and Riemannian DPs were calculated using additive mixed-effects models. We demonstrate qEEG "batch effects" and provide methods to calculate harmonized z-scores. (ii) We also show that harmonized Riemannian norms produce z-scores with increased diagnostic accuracy predicting brain dysfunction produced by malnutrition in the first year of life and detecting COVID induced brain dysfunction. (iii) We offer open code and data to calculate different individual z-scores from the HarMNqEEG dataset. These results contribute to developing bias-free, low-cost neuroimaging technologies applicable in various health settings.

Retrospective on the First Passive Brain-Computer Interface Competition on Cross-Session Workload Estimation

As is the case in several research domains, data sharing is still scarce in the field of Brain-Computer Interfaces (BCI), and particularly in that of passive BCIs-i.e., systems that enable implicit interaction or task adaptation based on a user's mental state(s) estimated from brain measures. Moreover, research in this field is currently hindered by a major challenge, which is tackling brain signal variability such as cross-session variability. Hence, with a view to develop good research practices in this field and to enable the whole community to join forces in working on cross-session estimation, we created the first passive brain-computer interface competition on cross-session workload estimation. This competition was part of the 3rd International Neuroergonomics conference. The data were electroencephalographic recordings acquired from 15 volunteers (6 females; average 25 y.o.) who performed 3 sessions-separated by 7 days-of the Multi-Attribute Task Battery-II (MATB-II) with 3 levels of difficulty per session (pseudo-randomized order). The data -training and testing sets-were made publicly available on Zenodo along with Matlab and Python toy code (https://doi.org/10.5281/zenodo.5055046). To this day, the database was downloaded more than 900 times (unique downloads of all version on the 10th of December 2021: 911). Eleven teams from 3 continents (31 participants) submitted their work. The best achieving processing pipelines included a Riemannian geometry-based method. Although better than the adjusted chance level (38% with an α at 0.05 for a 3-class classification problem), the results still remained under 60% of accuracy. These results clearly underline the real challenge that is cross-session estimation. Moreover, they confirmed once more the robustness and effectiveness of Riemannian methods for BCI. On the contrary, chance level results were obtained by one third of the methods-4 teams- based on Deep Learning. These methods have not demonstrated superior results in this contest compared to traditional methods, which may be due to severe overfitting. Yet this competition is the first step toward a joint effort to tackle BCI variability and to promote good research practices including reproducibility.

Security Requirements and Challenges of 6G Technologies and Applications

After implementing 5G technology, academia and industry started researching 6th generation wireless network technology (6G). 6G is expected to be implemented around the year 2030. It will offer a significant experience for everyone by enabling hyper-connectivity between people and everything. In addition, it is expected to extend mobile communication possibilities where earlier generations could not have developed. Several potential technologies are predicted to serve as the foundation of 6G networks. These include upcoming and current technologies such as post-quantum cryptography, artificial intelligence (AI), machine learning (ML), enhanced edge computing, molecular communication, THz, visible light communication (VLC), and distributed ledger (DL) technologies such as blockchain. From a security and privacy perspective, these developments need a reconsideration of prior security traditional methods. New novel authentication, encryption, access control, communication, and malicious activity detection must satisfy the higher significant requirements of future networks. In addition, new security approaches are necessary to ensure trustworthiness and privacy. This paper provides insights into the critical problems and difficulties related to the security, privacy, and trust issues of 6G networks. Moreover, the standard technologies and security challenges per each technology are clarified. This paper introduces the 6G security architecture and improvements over the 5G architecture. We also introduce the security issues and challenges of the 6G physical layer. In addition, the AI/ML layers and the proposed security solution in each layer are studied. The paper summarizes the security evolution in legacy mobile networks and concludes with their security problems and the most essential 6G application services and their security requirements. Finally, this paper provides a complete discussion of 6G networks’ trustworthiness and solutions.

Functional connectivity ensemble method to enhance BCI performance (FUCONE)

OBJECTIVE

Relying on the idea that functional connectivity provides important insights on the underlying dynamic of neuronal interactions, we propose a novel framework that combines functional connectivity estimators and covariance-based pipelines to improve the classification of mental states, such as motor imagery.

METHODS

A Riemannian classifier is trained for each estimator and an ensemble classifier combines the decisions in each feature space. A thorough assessment of the functional connectivity estimators is provided and the best performing pipeline among those tested, called FUCONE, is evaluated on different conditions and datasets.

RESULTS

Using a meta-analysis to aggregate results across datasets, FUCONE performed significantly better than all state-of-the-art methods.

CONCLUSION

The performance gain is mostly imputable to the improved diversity of the feature spaces, increasing the robustness of the ensemble classifier with respect to the inter- and intra-subject variability.

SIGNIFICANCE

Our results offer new insights into the need to consider functional connectivity-based methods to improve the BCI performance.

Experimental Validation of the Cumulative MDRM in theP300 Speller Machine

The P300 speller Machine is among the leading applications of the electroencephalography (EEG)-based brain computer interfaces (BCIs), it is still a benchmark and a persistent challenge for the BCI Community. EEG signal classification represents the key piece of a BCI chain. The minimum distance to Riemannian mean (MDRM) belongs to these classification methods emerging in different BCI applications such as text spelling by thought. Based on a binary classification of each covariance matrix separately, character prediction is done according to the highest score across the whole set of all repetitions. Minimum cumulative distance to Riemannian mean (MCDRM) is a Cumulative variant of the MDRM, perfectly adapted to the P300 Speller Machine. The power of this variant is that prediction takes a more global proceeding involving the n repetitions together. Indeed, thanks to cumulative distances selected row and column are those having the covariance matrices both closer to the Target barycenter and farther from the non-Target one. This variant overcomes the main MDRM limitations as it improves inter-sessional generalization, allows optimal use of all repetitions and reduces considerably the risk of conflict appearing during the selection of rows and columns leading to character prediction. We applied this variant to the raw signals of Data set II-b of Berlin BCI and compared to the published results the MCDRM offers significantly higher results: 97.5% of correct predictions compared to the 96.5% of the competition winner. The MCDRM fits best with the P300 Speller machine, especially when dealing with noisy signals that requires intelligent and optimal usage of the n repetitions.

Robust learning from corrupted EEG with dynamic spatial filtering

Building machine learning models using EEG recorded outside of the laboratory setting requires methods robust to noisy data and randomly missing channels. This need is particularly great when working with sparse EEG montages (1-6 channels), often encountered in consumer-grade or mobile EEG devices. Neither classical machine learning models nor deep neural networks trained end-to-end on EEG are typically designed or tested for robustness to corruption, and especially to randomly missing channels. While some studies have proposed strategies for using data with missing channels, these approaches are not practical when sparse montages are used and computing power is limited (e.g., wearables, cell phones). To tackle this problem, we propose dynamic spatial filtering (DSF), a multi-head attention module that can be plugged in before the first layer of a neural network to handle missing EEG channels by learning to focus on good channels and to ignore bad ones. We tested DSF on public EEG data encompassing ∼4,000 recordings with simulated channel corruption and on a private dataset of ∼100 at-home recordings of mobile EEG with natural corruption. Our proposed approach achieves the same performance as baseline models when no noise is applied, but outperforms baselines by as much as 29.4% accuracy when significant channel corruption is present. Moreover, DSF outputs are interpretable, making it possible to monitor the effective channel importance in real-time. This approach has the potential to enable the analysis of EEG in challenging settings where channel corruption hampers the reading of brain signals.

Workshops of the Eighth International Brain-Computer Interface Meeting: BCIs: The Next Frontier

The Eighth International Brain-Computer Interface (BCI) Meeting was held June 7-9th, 2021 in a virtual format. The conference continued the BCI Meeting series' interactive nature with 21 workshops covering topics in BCI (also called brain-machine interface) research. As in the past, workshops covered the breadth of topics in BCI. Some workshops provided detailed examinations of specific methods, hardware, or processes. Others focused on specific BCI applications or user groups. Several workshops continued consensus building efforts designed to create BCI standards and increase the ease of comparisons between studies and the potential for meta-analysis and large multi-site clinical trials. Ethical and translational considerations were both the primary topic for some workshops or an important secondary consideration for others. The range of BCI applications continues to expand, with more workshops focusing on approaches that can extend beyond the needs of those with physical impairments. This paper summarizes each workshop, provides background information and references for further study, presents an overview of the discussion topics, and describes the conclusion, challenges, or initiatives that resulted from the interactions and discussion at the workshop.

Customizing skills for assistive robotic manipulators, an inverse reinforcement learning approach with error-related potentials

Robotic assistance via motorized robotic arm manipulators can be of valuable assistance to individuals with upper-limb motor disabilities. Brain-computer interfaces (BCI) offer an intuitive means to control such assistive robotic manipulators. However, BCI performance may vary due to the non-stationary nature of the electroencephalogram (EEG) signals. It, hence, cannot be used safely for controlling tasks where errors may be detrimental to the user. Avoiding obstacles is one such task. As there exist many techniques to avoid obstacles in robotics, we propose to give the control to the robot to avoid obstacles and to leave to the user the choice of the robot behavior to do so a matter of personal preference as some users may be more daring while others more careful. We enable the users to train the robot controller to adapt its way to approach obstacles relying on BCI that detects error-related potentials (ErrP), indicative of the user’s error expectation of the robot’s current strategy to meet their preferences. Gaussian process-based inverse reinforcement learning, in combination with the ErrP-BCI, infers the user’s preference and updates the obstacle avoidance controller so as to generate personalized robot trajectories. We validate the approach in experiments with thirteen able-bodied subjects using a robotic arm that picks up, places and avoids real-life objects. Results show that the algorithm can learn user’s preference and adapt the robot behavior rapidly using less than five demonstrations not necessarily optimal.

Teaching an assistive robotic manipulator to move objects in a cluttered table requires demonstrations from expert operators, but what if the experts are individuals with motor disabilities? Batzianoulis et al. propose a learning approach which combines robot autonomy and a brain-computer interfacing that decodes whether the generated trajectories meet the user’s criteria, and show how their system enables the robot to learn individual user’s preferred behaviors using less than five demonstrations that are not necessarily optimal.

Mental State Detection Using Riemannian Geometry on Electroencephalogram Brain Signals

The goal of this study was to implement a Riemannian geometry (RG)-based algorithm to detect high mental workload (MWL) and mental fatigue (MF) using task-induced electroencephalogram (EEG) signals. In order to elicit high MWL and MF, the participants performed a cognitively demanding task in the form of the letter n-back task. We analyzed the time-varying characteristics of the EEG band power (BP) features in the theta and alpha frequency band at different task conditions and cortical areas by employing a RG-based framework. MWL and MF were considered as too high, when the Riemannian distances of the task-run EEG reached or surpassed the threshold of the baseline EEG. The results of this study showed a BP increase in the theta and alpha frequency bands with increasing experiment duration, indicating elevated MWL and MF that impedes/hinders the task performance of the participants. High MWL and MF was detected in 8 out of 20 participants. The Riemannian distances also showed a steady increase toward the threshold with increasing experiment duration, with the most detections occurring toward the end of the experiment. To support our findings, subjective ratings (questionnaires concerning fatigue and workload levels) and behavioral measures (performance accuracies and response times) were also considered.

Sub-100 μW Multispectral Riemannian Classification for EEG-Based Brain-Machine Interfaces

Motor imagery (MI) brain-machine interfaces (BMIs) enable us to control machines by merely thinking of performing a motor action. Practical use cases require a wearable solution where the classification of the brain signals is done locally near the sensor using machine learning models embedded on energy-efficient microcontroller units (MCUs), for assured privacy, user comfort, and long-term usage. In this work, we provide practical insights on the accuracy-cost trade-off for embedded BMI solutions. Our multispectral Riemannian classifier reaches 75.1% accuracy on a 4-class MI task. The accuracy is further improved by tuning different types of classifiers to each subject, achieving 76.4%. We further scale down the model by quantizing it to mixed-precision representations with a minimal accuracy loss of 1% and 1.4%, respectively, which is still up to 4.1% more accurate than the state-of-the-art embedded convolutional neural network. We implement the model on a low-power MCU within an energy budget of merely 198 μJ and taking only 16.9 ms per classification. Classifying samples continuously, overlapping the 3.5 s samples by 50% to avoid missing user inputs allows for operation at just 85 μW. Compared to related works in embedded MI-BMIs, our solution sets the new state-of-the-art in terms of accuracy-energy trade-off for near-sensor classification.

On the interpretation of linear Riemannian tangent space model parameters in M/EEG

Riemannian tangent space methods offer state-of-the-art performance in magnetoencephalography (MEG) and electroencephalography (EEG) based applications such as brain-computer interfaces and biomarker development. One limitation, particularly relevant for biomarker development, is limited model interpretability compared to established component-based methods. Here, we propose a method to transform the parameters of linear tangent space models into interpretable patterns. Using typical assumptions, we show that this approach identifies the true patterns of latent sources, encoding a target signal. In simulations and two real MEG and EEG datasets, we demonstrate the validity of the proposed approach and investigate its behavior when the model assumptions are violated. Our results confirm that Riemannian tangent space methods are robust to differences in the source patterns across observations. We found that this robustness property also transfers to the associated patterns.

A novel method to reduce the motor imagery BCI illiteracy

To reduce the motor imagery brain-computer interface (MI-BCI) illiteracy phenomenon and improve the classification accuracy, this paper proposed a novel method combining paradigm selection and Riemann distance classification. Firstly, a novel sensitivity-based paradigm selection (SPS) algorithm is designed for the optimization of classification to find the best classification pattern through a sensitive indicator. Then, a generalized Riemann minimum distance mean (GRMDM) classifier is proposed by introducing a weight factor to fuse the Log-Euclidean Metric classifier and the Riemannian Stein divergence classifier. The experimental results show that the proposed method achieves a better performance for multi-class motor imagery tasks. The average classification accuracy on the BCI competition IV dataset2a is 80.98%, which is 11.04% higher than Stein divergence classifier on the original two-class paradigm. Furthermore, the proposed method demonstrates its capacity on reducing MI-BCI illiteracy. Graphical abstract Here we investigate whether the BCI illiteracy phenomenon can be reduced through sensitivity-based paradigm selection (SPS) method and generalized Riemann minimum distance mean (GRMDM) classifier when performing motor imagery tasks.

Fatigue Detection with Spatial-Temporal Fusion Method on Covariance Manifolds of Electroencephalography

With the increasing pressure of current life, fatigue caused by high-pressure work has deeply affected people and even threatened their lives. In particular, fatigue driving has become a leading cause of traffic accidents and deaths. This paper investigates electroencephalography (EEG)-based fatigue detection for driving by mining the latent information through the spatial-temporal changes in the relations between EEG channels. First, EEG data are partitioned into several segments to calculate the covariance matrices of each segment, and then we feed these matrices into a recurrent neural network to obtain high-level temporal information. Second, the covariance matrices of whole signals are leveraged to extract two kinds of spatial features, which will be fused with temporal characteristics to obtain comprehensive spatial-temporal information. Experiments on an open benchmark showed that our method achieved an excellent classification accuracy of 93.834% and performed better than several novel methods. These experimental results indicate that our method enables better reliability and feasibility in the detection of fatigued driving.

Classification of motor imagery using multisource joint transfer learning

As an important way for human-computer interaction, the motor imagery brain-computer interface (MI-BCI) can decode personal motor intention directly by analyzing electroencephalogram (EEG) signals. However, a large amount of labeled data has to be collected for each new subject since EEG patterns vary between individuals. The long calibration phase severely limits the further development of MI-BCI. To tackle this problem, multi-source joint domain adaption (MJDA) and multi-source joint Riemannian adaption (MJRA) algorithms are proposed in this paper. Both methods aim to transfer knowledge from other subjects to the current subject who has only a small amount of labeled data. First, the common spatial pattern with Euclidean alignment is used to select source subjects who have similar spatial patterns to the target subject. Second, the covariance matrices of EEG trials are aligned in Riemannian space by removing subject-specific baselines. These two steps are shared by MJDA and MJRA. In the last step, MJDA attempts to minimize the feature distribution mismatch in the Riemannian tangent space, while MJRA attempts to find an adaptive Riemannian classifier. Finally, the proposed methods are validated on two datasets: BCI Competition IV 2a and online event-related desynchronization (ERD)-BCI. The experimental results demonstrate that both MJDA and MJRA outperform the state-of-the-art approaches. The MJDA provides a new idea for the offline analysis of MI-BCI, while MJRA could make a big difference to the online calibration of MI-BCI.

Estimating the probability of coincidental similarity between atomic displacement parameters with machine learning

High-resolution diffraction studies of macromolecules incorporate the tensor form of the anisotropic displacement parameter (ADP) of atoms from their mean position. The comparison of these parameters requires a statistical framework that can handle the experimental and modeling errors linked to structure determination. Here, a Bayesian machine learning model is introduced that approximates ADPs with the random Wishart distribution. This model allows for the comparison of random samples from a distribution that is trained on experimental structures. The comparison revealed that the experimental similarity between atoms is larger than predicted by the random model for a substantial fraction of the comparisons. Different metrics between ADPs were evaluated and categorized based on how useful they are at detecting non-accidental similarity and whether they can be replaced by other metrics. The most complementary comparisons were provided by Euclidean, Riemann and Wasserstein metrics. The analysis of ADP similarity and the positional distance of atoms in bovine trypsin revealed a set of atoms with striking ADP similarity over a long physical distance, and generally the physical distance between atoms and their ADP similarity do not correlate strongly. A substantial fraction of long- and short-range ADP similarities does not form by coincidence and are reproducibly observed in different crystal structures of the same protein.

Enhancing transfer performance across datasets for brain-computer interfaces using a combination of alignment strategies and adaptive batch normalization

Objective. Recently, transfer learning (TL) and deep learning (DL) have been introduced to solve intra- and inter-subject variability problems in brain-computer interfaces (BCIs). However, current TL and DL algorithms are usually validated within a single dataset, assuming that data of the test subjects are acquired under the same condition as that of training (source) subjects. This assumption is generally violated in practice because of different acquisition systems and experimental settings across studies and datasets. Thus, the generalization ability of these algorithms needs further validations in a cross-dataset scenario, which is closer to the actual situation. This study compared the transfer performance of pre-trained deep-learning models with different preprocessing strategies in a cross-dataset scenario.Approach. This study used four publicly available motor imagery datasets, each was successively selected as a source dataset, and the others were used as target datasets. EEGNet and ShallowConvNet with four preprocessing strategies, namely channel normalization, trial normalization, Euclidean alignment, and Riemannian alignment, were trained with the source dataset. The transfer performance of pre-trained models was validated on the target datasets. This study also used adaptive batch normalization (AdaBN) for reducing interval covariate shift across datasets. This study compared the transfer performance of using the four preprocessing strategies and that of a baseline approach based on manifold embedded knowledge transfer (MEKT). This study also explored the possibility and performance of fusing MEKT and EEGNet.Main results. The results show that DL models with alignment strategies had significantly better transfer performance than the other two preprocessing strategies. As an unsupervised domain adaptation method, AdaBN could also significantly improve the transfer performance of DL models. The transfer performance of DL models that combined AdaBN and alignment strategies significantly outperformed MEKT. Moreover, the generalizability of EEGNet models that combined AdaBN and alignment strategies could be further improved via the domain adaptation step in MEKT, achieving the best generalization ability among multiple datasets (BNCI2014001: 0.788, PhysionetMI: 0.679, Weibo2014: 0.753, Cho2017: 0.650).Significance. The combination of alignment strategies and AdaBN could easily improve the generalizability of DL models without fine-tuning. This study may provide new insights into the design of transfer neural networks for BCIs by separating source and target batch normalization layers in the domain adaptation process.

BioPyC, an Open-Source Python Toolbox for Offline Electroencephalographic and Physiological Signals Classification

Research on brain–computer interfaces (BCIs) has become more democratic in recent decades, and experiments using electroencephalography (EEG)-based BCIs has dramatically increased. The variety of protocol designs and the growing interest in physiological computing require parallel improvements in processing and classification of both EEG signals and bio signals, such as electrodermal activity (EDA), heart rate (HR) or breathing. If some EEG-based analysis tools are already available for online BCIs with a number of online BCI platforms (e.g., BCI2000 or OpenViBE), it remains crucial to perform offline analyses in order to design, select, tune, validate and test algorithms before using them online. Moreover, studying and comparing those algorithms usually requires expertise in programming, signal processing and machine learning, whereas numerous BCI researchers come from other backgrounds with limited or no training in such skills. Finally, existing BCI toolboxes are focused on EEG and other brain signals but usually do not include processing tools for other bio signals. Therefore, in this paper, we describe BioPyC, a free, open-source and easy-to-use Python platform for offline EEG and biosignal processing and classification. Based on an intuitive and well-guided graphical interface, four main modules allow the user to follow the standard steps of the BCI process without any programming skills: (1) reading different neurophysiological signal data formats, (2) filtering and representing EEG and bio signals, (3) classifying them, and (4) visualizing and performing statistical tests on the results. We illustrate BioPyC use on four studies, namely classifying mental tasks, the cognitive workload, emotions and attention states from EEG signals.

Mental workload classification based on ignored auditory probes and spatial covariance

Objective.Estimation of mental workload (MWL) levels by electroencephalography (EEG)-based mental state monitoring systems has been widely explored. Using event-related potentials (ERPs), elicited by ignored auditory probes, minimizes intrusiveness and has shown high performance for estimating MWL level when tested in laboratory situations. However, when facing real-world applications, the characteristics of ERP waveforms, like latency and amplitude, are often affected by noise, which leads to a decrease in classification performance. One approach to mitigating this is using spatial covariance, which is less sensitive to latency and amplitude distortion. In this study, we used ignored auditory probes in a single-stimulus paradigm and tested Riemannian processed covariance-based features for MWL level estimation in a realistic flight-control task.Approach.We recorded EEG data with an eight-channel system from participants while they performed a simulated drone-control task and manipulated MWL levels (high and low) by task difficulty. We compared support vector machine classification performance based on frequency band power features versus features generated via the Riemannian log map operator from spatial covariance matrices. We also compared accuracy of using data segmented as auditory ERPs versus non-ERPs, for which data windows did not overlap with the ERPs.Main results.Classification accuracy of both types of features showed no significant difference between ERPs and non-ERPs. When we ignore auditory stimuli to perform continuous decoding, covariance-based features in the gamma band had area under the receiver-operating-characteristic curve (AUC) of 0.883, which was significantly higher than band power features (AUC = 0.749).Significance.This study demonstrates that Riemannian-processed covariance features are viable for MWL classification under a realistic experimental scenario.

Boosting automated sleep staging performance in big datasets using population subgrouping

Current approaches to automated sleep staging from the electroencephalogram (EEG) rely on constructing a large labeled training and test corpora by aggregating data from different individuals. However, many of the subjects in the training set may exhibit changes in the EEG that are very different from the subjects in the test set. Training an algorithm on such data without accounting for this diversity can cause underperformance. Moreover, test data may have unexpected sensor misplacement or different instrument noise and spectral responses. This work proposes a novel method to learn relevant individuals based on their similarities effectively. The proposed method embeds all training patients into a shared and robust feature space. Individuals who share strong statistical relationships and are similar based on their EEG signals are clustered in this feature space before being passed to a deep learning framework for classification. Using 994 patient EEGs from the 2018 Physionet Challenge (≈6,561 h of recording), we demonstrate that the clustering approach significantly boosts performance compared to state-of-the-art deep learning approaches. The proposed method improves, on average, a precision score from 0.72 to 0.81, a sensitivity score from 0.74 to 0.82, and a Cohen's Kappa coefficient from 0.64 to 0.75 under 10-fold cross-validation.

Review of brain encoding and decoding mechanisms for EEG-based brain-computer interface

A brain-computer interface (BCI) can connect humans and machines directly and has achieved successful applications in the past few decades. Many new BCI paradigms and algorithms have been developed in recent years. Therefore, it is necessary to review new progress in BCIs. This paper summarizes progress for EEG-based BCIs from the perspective of encoding paradigms and decoding algorithms, which are two key elements of BCI systems. Encoding paradigms are grouped by their underlying neural meachanisms, namely sensory- and motor-related, vision-related, cognition-related and hybrid paradigms. Decoding algorithms are reviewed in four categories, namely decomposition algorithms, Riemannian geometry, deep learning and transfer learning. This review will provide a comprehensive overview of both modern primary paradigms and algorithms, making it helpful for those who are developing BCI systems.

Using distance on the Riemannian manifold to compare representations in brain and in models

Representational similarity analysis (RSA) summarizes activity patterns for a set of experimental conditions into a matrix composed of pairwise comparisons between activity patterns. Two examples of such matrices are the condition-by-condition inner product and correlation matrix. These representational matrices reside on the manifold of positive semidefinite matrices, called the Riemannian manifold. We hypothesize that representational similarities would be more accurately quantified by considering the underlying manifold of the representational matrices. Thus, we introduce the distance on the Riemannian manifold as a metric for comparing representations. Analyzing simulated and real fMRI data and considering a wide range of metrics, we show that the Riemannian distance is least susceptible to sampling bias, results in larger intra-subject reliability, and affords searchlight mapping with high sensitivity and specificity. Furthermore, we show that the Riemannian distance can be used for measuring multi-dimensional connectivity. This measure captures both univariate and multivariate connectivity and is also more sensitive to nonlinear regional interactions compared to the state-of-the-art measures. Applying our proposed metric to neural network representations of natural images, we demonstrate that it also possesses outstanding performance in quantifying similarity in models. Taken together, our results lend credence to the proposition that RSA should consider the manifold of the representational matrices to summarize response patterns in the brain and in models.

An Online Data Visualization Feedback Protocol for Motor Imagery-Based BCI Training

Brain-computer interface (BCI) has developed rapidly over the past two decades, mainly due to advancements in machine learning. Subjects must learn to modulate their brain activities to ensure a successful BCI. Feedback training is a practical approach to this learning process; however, the commonly used classifier-dependent approaches have inherent limitations such as the need for calibration and a lack of continuous feedback over long periods of time. This paper proposes an online data visualization feedback protocol that intuitively reflects the EEG distribution in Riemannian geometry in real time. Rather than learning a hyperplane, the Riemannian geometry formulation allows iterative learning of prototypical covariance matrices that are translated into visualized feedback through diffusion map process. Ten subjects were recruited for MI-BCI (motor imagery-BCI) training experiments. The subjects learned to modulate their sensorimotor rhythm to centralize the points within one category and to separate points belonging to different categories. The results show favorable overall training effects in terms of the class distinctiveness and EEG feature discriminancy over a 3-day training with 30% learners. A steadily increased class distinctiveness in the last three sessions suggests that the advanced training protocol is effective. The optimal frequency band was consistent during the 3-day training, and the difference between subjects with good or low MI-BCI performance could be clearly observed. We believe that the proposed feedback protocol has promising application prospect.

Low-Dimensional Subject Representation-Based Transfer Learning in EEG Decoding

Recently, the advances in passive brain-computer interfaces (BCIs) based on electroencephalogram (EEG) have shed light on real-world neuromonitoring technologies. However, human variability in the EEG activities hinders the development of practical applications of EEG-based BCI. To tackle this problem, many transfer-learning techniques perform supervised calibration. This kind of calibration approach requires task-relevant data, which is impractical in real-life scenarios such as drowsiness during driving. This study presents a transfer-learning framework for EEG decoding based on the low-dimensional representations of subjects learned from the pre-trial EEG. Tensor decomposition was applied to the pre-trial EEG of subjects to extract the underlying characteristics in subject, spatial, and spectral domains. Then, the proposed framework assessed the characteristics to obtain the low-dimensional subject representations such that the subjects with similar brain dynamics can be identified. This method can leverage the existing data from other users, and a small number of data from a rapid, non-task, unsupervised calibration from a new user to build an accurate BCI. Our results demonstrated that, in terms of prediction accuracy, the proposed low-dimensional subject representation-based transfer learning (LDSR-TL) framework outperformed the random selection, and the Riemannian manifold approach in cognitive-state tracking, while requiring fewer training data. The results can greatly improve the practicability, and usability of EEG-based BCI in the real world.

Probabilistic learning vector quantization on manifold of symmetric positive definite matrices

In this paper, we develop a new classification method for manifold-valued data in the framework of probabilistic learning vector quantization. In many classification scenarios, the data can be naturally represented by symmetric positive definite matrices, which are inherently points that live on a curved Riemannian manifold. Due to the non-Euclidean geometry of Riemannian manifolds, traditional Euclidean machine learning algorithms yield poor results on such data. In this paper, we generalize the probabilistic learning vector quantization algorithm for data points living on the manifold of symmetric positive definite matrices equipped with Riemannian natural metric (affine-invariant metric). By exploiting the induced Riemannian distance, we derive the probabilistic learning Riemannian space quantization algorithm, obtaining the learning rule through Riemannian gradient descent. Empirical investigations on synthetic data, image data , and motor imagery electroencephalogram (EEG) data demonstrate the superior performance of the proposed method.

Online Mental Fatigue Monitoring via Indirect Brain Dynamics Evaluation

Driver mental fatigue leads to thousands of traffic accidents. The increasing quality and availability of low-cost electroencephalogram (EEG) systems offer possibilities for practical fatigue monitoring. However, non-data-driven methods, designed for practical, complex situations, usually rely on handcrafted data statistics of EEG signals. To reduce human involvement, we introduce a data-driven methodology for online mental fatigue detection: self-weight ordinal regression (SWORE). Reaction time (RT), referring to the length of time people take to react to an emergency, is widely considered an objective behavioral measure for mental fatigue state. Since regression methods are sensitive to extreme RTs, we propose an indirect RT estimation based on preferences to explore the relationship between EEG and RT, which generalizes to any scenario when an objective fatigue indicator is available. In particular, SWORE evaluates the noisy EEG signals from multiple channels in terms of two states: shaking state and steady state. Modeling the shaking state can discriminate the reliable channels from the uninformative ones, while modeling the steady state can suppress the task-nonrelevant fluctuation within each channel. In addition, an online generalized Bayesian moment matching (online GBMM) algorithm is proposed to online-calibrate SWORE efficiently per participant. Experimental results with 40 participants show that SWORE can maximally achieve consistent with RT, demonstrating the feasibility and adaptability of our proposed framework in practical mental fatigue estimation.

Review of Riemannian Distances and Divergences, Applied to SSVEP-based BCI

The firstgeneration of brain-computer interfaces (BCI) classifies multi-channel electroencephalographic (EEG) signals, enhanced by optimized spatial filters.The second generation directly classifies covariance matrices estimated on EEG signals, based on straightforward algorithms such as the minimum-distance-to-Riemannian-mean (MDRM). Classification results vary greatly depending on the chosen Riemannian distance or divergence, whose definitions and reference implementations are spread across a wide mathematical literature. This paper reviews all the Riemannian distances and divergences to process covariance matrices, with an implementation compatible with BCI constraints. The impact of using different metrics is assessed on a steady-state visually evoked potentials (SSVEP) dataset, evaluating centers of classes and classification accuracy. Riemannian approaches embed crucial properties to process EEG data. The Riemannian centers of classes outperform Euclidean ones both in offline and online setups. Some Riemannian distances and divergences have better performances in terms of classification accuracy, while others have appealing computational efficiency.

The Riemannian Spatial Pattern method: mapping and clustering movement imagery using Riemannian geometry

OBJECTIVE

Over the last decade, Riemannian geometry has shown promising results for motor imagery classification. However, extracting the underlying spatial features is not as straightforward as for applying Common Spatial Pattern (CSP) filtering prior to classification. In this article, we propose a simple way to extract the spatial patterns obtained from Riemannian classification: the Riemannian Spatial Pattern (RSP) method, which is based on the backward channel selection procedure.

APPROACH

The RSP method was compared to the CSP approach on ECoG data obtained from a quadriplegic patient while performing imagined movements of arm articulations and fingers.

MAIN RESULTS

Similar results were found between the RSP and CSP methods for mapping each motor imagery task with activations following the classical somatotopic organization. Clustering obtained by pairwise comparisons of imagined motor movements however, revealed higher differentiation for the RSP method compared to the CSP approach. Importantly, the RSP approach could provide a precise comparison of the imagined finger flexions which added supplementary information to the mapping results.

SIGNIFICANCE

Our new RSP method illustrates the interest of the Riemannian framework in the spatial domain and as such offers new avenues for the neuroimaging community. This study is part of an ongoing clinical trial registered with ClinicalTrials.gov, NCT02550522.

Long-Term BCI Training of a Tetraplegic User: Adaptive Riemannian Classifiers and User Training

While often presented as promising assistive technologies for motor-impaired users, electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs) remain barely used outside laboratories due to low reliability in real-life conditions. There is thus a need to design long-term reliable BCIs that can be used outside-of-the-lab by end-users, e.g., severely motor-impaired ones. Therefore, we propose and evaluate the design of a multi-class Mental Task (MT)-based BCI for longitudinal training (20 sessions over 3 months) of a tetraplegic user for the CYBATHLON BCI series 2019. In this BCI championship, tetraplegic pilots are mentally driving a virtual car in a racing video game. We aimed at combining a progressive user MT-BCI training with a newly designed machine learning pipeline based on adaptive Riemannian classifiers shown to be promising for real-life applications. We followed a two step training process: the first 11 sessions served to train the user to control a 2-class MT-BCI by performing either two cognitive tasks (REST and MENTAL SUBTRACTION) or two motor-imagery tasks (LEFT-HAND and RIGHT-HAND). The second training step (9 remaining sessions) applied an adaptive, session-independent Riemannian classifier that combined all 4 MT classes used before. Moreover, as our Riemannian classifier was incrementally updated in an unsupervised way it would capture both within and between-session non-stationarity. Experimental evidences confirm the effectiveness of this approach. Namely, the classification accuracy improved by about 30% at the end of the training compared to initial sessions. We also studied the neural correlates of this performance improvement. Using a newly proposed BCI user learning metric, we could show our user learned to improve his BCI control by producing EEG signals matching increasingly more the BCI classifier training data distribution, rather than by improving his EEG class discrimination. However, the resulting improvement was effective only on synchronous (cue-based) BCI and it did not translate into improved CYBATHLON BCI game performances. For the sake of overcoming this in the future, we unveil possible reasons for these limited gaming performances and identify a number of promising future research directions. Importantly, we also report on the evolution of the user's neurophysiological patterns and user experience throughout the BCI training and competition.

A New - Discriminative and Multi- Filter Bank Tangent Space Mapping Method for Recognition of Multiclass Motor

Objective: Tangent Space Mapping (TSM) using the geometric structure of the covariance matrices is an effective method to recognize multiclass motor imagery (MI). Compared with the traditional CSP method, the Riemann geometric method based on TSM takes into account the nonlinear information contained in the covariance matrix, and can extract more abundant and effective features. Moreover, the method is an unsupervised operation, which can reduce the time of feature extraction. However, EEG features induced by MI mental activities of different subjects are not the same, so selection of subject-specific discriminative EEG frequency components play a vital role in the recognition of multiclass MI. In order to solve the problem, a discriminative and multi-scale filter bank tangent space mapping (DMFBTSM) algorithm is proposed in this article to design the subject-specific Filter Bank (FB) so as to effectively recognize multiclass MI tasks. Methods: On the 4-class BCI competition IV-2a dataset, first, a non-parametric method of multivariate analysis of variance (MANOVA) based on the sum of squared distances is used to select discriminative frequency bands for a subject; next, a multi-scale FB is generated according to the range of these frequency bands, and then decompose multi-channel EEG of the subject into multiple sub-bands combined with several time windows. Then TSM algorithm is used to estimate Riemannian tangent space features in each sub-band and finally a liner Support Vector Machines (SVM) is used for classification. Main Results: The analysis results show that the proposed discriminative FB enhances the multi-scale TSM algorithm, improves the classification accuracy and reduces the execution time during training and testing. On the 4-class BCI competition IV-2a dataset, the average session to session classification accuracy of nine subjects reached 77.33 ± 12.3%. When the training time and the test time are similar, the average classification accuracy is 2.56% higher than the latest TSM method based on multi-scale filter bank analysis technology. When the classification accuracy is similar, the training speed is increased by more than three times, and the test speed is increased two times more. Compared with Supervised Fisher Geodesic Minimum Distance to the Mean (Supervised FGMDRM), another new variant based on Riemann geometry classifier, the average accuracy is 3.36% higher, we also compared with the latest Deep Learning method, and the average accuracy of 10-fold cross validation improved by 2.58%. Conclusion: Research shows that the proposed DMFBTSM algorithm can improve the classification accuracy of MI tasks. Significance: Compared with the MFBTSM algorithm, the algorithm proposed in this article is expected to select frequency bands with good separability for specific subject to improve the classification accuracy of multiclass MI tasks and reduce the feature dimension to reduce training time and testing time.

Progress in Brain Computer Interface: Challenges and Opportunities

Brain computer interfaces (BCI) provide a direct communication link between the brain and a computer or other external devices. They offer an extended degree of freedom either by strengthening or by substituting human peripheral working capacity and have potential applications in various fields such as rehabilitation, affective computing, robotics, gaming, and neuroscience. Significant research efforts on a global scale have delivered common platforms for technology standardization and help tackle highly complex and non-linear brain dynamics and related feature extraction and classification challenges. Time-variant psycho-neurophysiological fluctuations and their impact on brain signals impose another challenge for BCI researchers to transform the technology from laboratory experiments to plug-and-play daily life. This review summarizes state-of-the-art progress in the BCI field over the last decades and highlights critical challenges.

Dual stream neural networks for brain signal classification

Objective. The primary objective of this work is to develop a neural nework classifier for arbitrary collections of functional neuroimaging signals to be used in brain–computer interfaces (BCIs). Approach. We propose a dual stream neural network (DSNN) for the classification problem. The first stream is an end-to-end classifier taking raw time-dependent signals as input and generating feature identification signatures from them. The second stream enhances the identified features from the first stream by adjoining a dynamic functional connectivity matrix aimed at incorporating nuanced multi-channel information during specified BCI tasks. Main results. The proposed DSNN classifier is benchmarked against three publicly available datasets, where the classifier demonstrates performance comparable to, or better than the state-of-art in each instance. An information theoretic examination of the trained network is also performed, utilizing various tools, to demonstrate how to glean interpretive insight into how the hidden layers of the network parse the underlying biological signals. Significance . The resulting DSNN is a subject-independent classifier that works for any collection of 1D functional neuroimaging signals, with the option of integrating domain specific information in the design.