Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review

EEG-based emergency braking intention detection during simulated driving

BACKGROUND

Current research related to electroencephalogram (EEG)-based driver's emergency braking intention detection focuses on recognizing emergency braking from normal driving, with little attention to differentiating emergency braking from normal braking. Moreover, the classification algorithms used are mainly traditional machine learning methods, and the inputs to the algorithms are manually extracted features.

METHODS

To this end, a novel EEG-based driver's emergency braking intention detection strategy is proposed in this paper. The experiment was conducted on a simulated driving platform with three different scenarios: normal driving, normal braking and emergency braking. We compared and analyzed the EEG feature maps of the two braking modes, and explored the use of traditional methods, Riemannian geometry-based methods, and deep learning-based methods to predict the emergency braking intention, all using the raw EEG signals rather than manually extracted features as input.

RESULTS

We recruited 10 subjects for the experiment and used the area under the receiver operating characteristic curve (AUC) and F1 score as evaluation metrics. The results showed that both the Riemannian geometry-based method and the deep learning-based method outperform the traditional method. At 200 ms before the start of real braking, the AUC and F1 score of the deep learning-based EEGNet algorithm were 0.94 and 0.65 for emergency braking vs. normal driving, and 0.91 and 0.85 for emergency braking vs. normal braking, respectively. The EEG feature maps also showed a significant difference between emergency braking and normal braking. Overall, based on EEG signals, it was feasible to detect emergency braking from normal driving and normal braking.

CONCLUSIONS

The study provides a user-centered framework for human-vehicle co-driving. If the driver's intention to brake in an emergency can be accurately identified, the vehicle's automatic braking system can be activated hundreds of milliseconds earlier than the driver's real braking action, potentially avoiding some serious collisions.

Mind wandering state detection during video-based learning via EEG

The aim of this study is to explore the potential of technology for detecting mind wandering, particularly during video-based distance learning, with the ultimate benefit of improving learning outcomes. To overcome the challenges of previous mind wandering research in ecological validity, sample balance, and dataset size, this study utilized practical electroencephalography (EEG) recording hardware and designed a paradigm consisting of viewing short-duration video lectures under a focused learning condition and a future planning condition. Participants estimated statistics of their attentional state at the end of each video, and we combined this rating scale feedback with self-caught key press responses during video watching to obtain binary labels for classifier training. EEG was recorded using an 8-channel system, and spatial covariance features processed by Riemannian geometry were employed. The results demonstrate that a radial basis function kernel support vector machine classifier, using Riemannian-processed covariance features from delta, theta, alpha, and beta bands, can detect mind wandering with a mean area under the receiver operating characteristic curve (AUC) of 0.876 for within-participant classification and AUC of 0.703 for cross-lecture classification. Furthermore, our results suggest that a short duration of training data is sufficient to train a classifier for online decoding, as cross-lecture classification remained at an average AUC of 0.689 when using 70% of the training set (about 9 min). The findings highlight the potential for practical EEG hardware in detecting mind wandering with high accuracy, which has potential application to improving learning outcomes during video-based distance learning.

Decoding Multi-Class Motor Imagery and Motor Execution Tasks Using Riemannian Geometry Algorithms on Large EEG Datasets

The use of Riemannian geometry decoding algorithms in classifying electroencephalography-based motor-imagery brain-computer interfaces (BCIs) trials is relatively new and promises to outperform the current state-of-the-art methods by overcoming the noise and nonstationarity of electroencephalography signals. However, the related literature shows high classification accuracy on only relatively small BCI datasets. The aim of this paper is to provide a study of the performance of a novel implementation of the Riemannian geometry decoding algorithm using large BCI datasets. In this study, we apply several Riemannian geometry decoding algorithms on a large offline dataset using four adaptation strategies: baseline, rebias, supervised, and unsupervised. Each of these adaptation strategies is applied in motor execution and motor imagery for both scenarios 64 electrodes and 29 electrodes. The dataset is composed of four-class bilateral and unilateral motor imagery and motor execution of 109 subjects. We run several classification experiments and the results show that the best classification accuracy is obtained for the scenario where the baseline minimum distance to Riemannian mean has been used. The mean accuracy values up to 81.5% for motor execution, and up to 76.4% for motor imagery. The accurate classification of EEG trials helps to realize successful BCI applications that allow effective control of devices.

A novel semi-supervised meta learning method for subject-transfer brain-computer interface

The brain-computer interface (BCI) provides a direct communication pathway between the human brain and external devices. However, the models trained for existing subjects perform poorly on new subjects, which is termed the subject calibration problem. In this paper, we propose a semi-supervised meta learning (SSML) method for subject-transfer calibration. The proposed SSML learns a model-agnostic meta learner with existing subjects and then fine-tunes the meta learner in a semi-supervised learning manner, i.e. using a few labelled samples and many unlabelled samples of the target subject for calibration. It is significant for BCI applications in which labelled data are scarce or expensive while unlabelled data are readily available. Three different BCI paradigms are tested: event-related potential detection, emotion recognition and sleep staging. The SSML achieved classification accuracies of 0.95, 0.89 and 0.83 in the benchmark datasets of three paradigms. The runtime complexity of SSML grows linearly as the number of samples of target subject increases so that is possible to apply it in real-time systems. This study is the first attempt to apply semi-supervised model-agnostic meta learning methodology for subject calibration. The experimental results demonstrated the effectiveness and potential of the SSML method for subject-transfer BCI applications.

Bio-robotics research for non-invasive myoelectric neural interfaces for upper-limb prosthetic control: a 10-year perspective review

A decade ago, a group of researchers from academia and industry identified a dichotomy between the industrial and academic state-of-the-art in upper-limb prosthesis control, a widely used bio-robotics application. They proposed that four key technical challenges, if addressed, could bridge this gap and translate academic research into clinically and commercially viable products. These challenges are unintuitive control schemes, lack of sensory feedback, poor robustness and single sensor modality. Here, we provide a perspective review on the research effort that occurred in the last decade, aiming at addressing these challenges. In addition, we discuss three research areas essential to the recent development in upper-limb prosthetic control research but were not envisioned in the review 10 years ago: deep learning methods, surface electromyogram decomposition and open-source databases. To conclude the review, we provide an outlook into the near future of the research and development in upper-limb prosthetic control and beyond.

Riemannian geometry-based metrics to measure and reinforce user performance changes during brain-computer interface user training

Despite growing interest and research into brain-computer interfaces (BCI), their usage remains limited outside of research laboratories. One reason for this is BCI inefficiency, the phenomenon where a significant number of potential users are unable to produce machine-discernible brain signal patterns to control the devices. To reduce the prevalence of BCI inefficiency, some have advocated for novel user-training protocols that enable users to more effectively modulate their neural activity. Important considerations for the design of these protocols are the assessment measures that are used for evaluating user performance and for providing feedback that guides skill acquisition. Herein, we present three trial-wise adaptations (running, sliding window and weighted average) of Riemannian geometry-based user-performance metrics (classDistinct reflecting the degree of class separability and classStability reflecting the level of within-class consistency) to enable feedback to the user following each individual trial. We evaluated these metrics, along with conventional classifier feedback, using simulated and previously recorded sensorimotor rhythm-BCI data to assess their correlation with and discrimination of broader trends in user performance. Analysis revealed that the sliding window and weighted average variants of our proposed trial-wise Riemannian geometry-based metrics more accurately reflected performance changes during BCI sessions compared to conventional classifier output. The results indicate the metrics are a viable method for evaluating and tracking user performance changes during BCI-user training and, therefore, further investigation into how these metrics may be presented to users during training is warranted.

Case-Based and Quantum Classification for ERP-Based Brain-Computer Interfaces

Low transfer rates are a major bottleneck for brain-computer interfaces based on electroencephalography (EEG). This problem has led to the development of more robust and accurate classifiers. In this study, we investigated the performance of variational quantum, quantum-enhanced support vector, and hypergraph case-based reasoning classifiers in the binary classification of EEG data from a P300 experiment. On the one hand, quantum classification is a promising technology to reduce computational time and improve learning outcomes. On the other hand, case-based reasoning has an excellent potential to simplify the preprocessing steps of EEG analysis. We found that the balanced training (prediction) accuracy of each of these three classifiers was 56.95 (51.83), 83.17 (50.25), and 71.10% (52.04%), respectively. In addition, case-based reasoning performed significantly lower with a simplified (49.78%) preprocessing pipeline. These results demonstrated that all classifiers were able to learn from the data and that quantum classification of EEG data was implementable; however, more research is required to enable a greater prediction accuracy because none of the classifiers were able to generalize from the data. This could be achieved by improving the configuration of the quantum classifiers (e.g., increasing the number of shots) and increasing the number of trials for hypergraph case-based reasoning classifiers through transfer learning.

A transfer learning-based feedback training motivates the performance of SMR-BCI

Objective. Feedback training is a practical approach to brain-computer interface (BCI) end-users learning to modulate their sensorimotor rhythms (SMRs). BCI self-regulation learning has been shown to be influenced by subjective psychological factors, such as motivation. However, few studies have taken into account the users' self-motivation as additional guidance for the cognitive process involved in BCI learning. In this study we tested a transfer learning (TL) feedback method designed to increase self-motivation by providing information about past performance.Approach. Electroencephalography (EEG) signals from the previous runs were affine transformed and displayed as points on the screen, along with the newly recorded EEG signals in the current run, giving the subjects a context for self-motivation. Subjects were asked to separate the feedback points for the current run under the display of the separability of prior training. We conducted a between-subject feedback training experiment, in which 24 healthy SMR-BCI naive subjects were trained to imagine left- and right-hand movements. The participants were provided with either TL feedback or typical cursor-bar (CB) feedback (control condition), for three sessions on separate days.Main results. The behavioral results showed an increased challenge and stable mastery confidence, suggesting that subjects' motivation grew as the feedback training went on. The EEG results showed favorable overall training effects with TL feedback in terms of the class distinctiveness and EEG discriminancy. Performance was 28.5% higher in the third session than in the first. About 41.7% of the subjects were 'learners' including not only low-performance subjects, but also good-performance subjects who might be affected by the ceiling effect. Subjects were able to control BCI with TL feedback with a higher performance of 60.5% during the last session compared to CB feedback.Significance. The present study demonstrated that the proposed TL feedback method boosted psychological engagement through the self-motivated context, and further allowed subjects to modulate SMR effectively. The proposed TL feedback method also provided an alternative to typical CB feedback.

Identifying Thematics in a Brain-Computer Interface Research

This umbrella review is motivated to understand the shift in research themes on brain-computer interfacing (BCI) and it determined that a shift away from themes that focus on medical advancement and system development to applications that included education, marketing, gaming, safety, and security has occurred. The background of this review examined aspects of BCI categorisation, neuroimaging methods, brain control signal classification, applications, and ethics. The specific area of BCI software and hardware development was not examined. A search using One Search was undertaken and 92 BCI reviews were selected for inclusion. Publication demographics indicate the average number of authors on review papers considered was 4.2 ± 1.8. The results also indicate a rapid increase in the number of BCI reviews from 2003, with only three reviews before that period, two in 1972, and one in 1996. While BCI authors were predominantly Euro-American in early reviews, this shifted to a more global authorship, which China dominated by 2020-2022. The review revealed six disciplines associated with BCI systems: life sciences and biomedicine (n = 42), neurosciences and neurology (n = 35), and rehabilitation (n = 20); (2) the second domain centred on the theme of functionality: computer science (n = 20), engineering (n = 28) and technology (n = 38). There was a thematic shift from understanding brain function and modes of interfacing BCI systems to more applied research novel areas of research-identified surround artificial intelligence, including machine learning, pre-processing, and deep learning. As BCI systems become more invasive in the lives of "normal" individuals, it is expected that there will be a refocus and thematic shift towards increased research into ethical issues and the need for legal oversight in BCI application.

Revisiting Riemannian geometry-based EEG decoding through approximate joint diagonalization

Objective.The wider adoption of Riemannian geometry in electroencephalography (EEG) processing is hindered by two factors: (a) it involves the manipulation of complex mathematical formulations and, (b) it leads to computationally demanding tasks. The main scope of this work is to simplify particular notions of Riemannian geometry and provide an efficient and comprehensible scheme for neuroscientific explorations.Approach.To overcome the aforementioned shortcomings, we exploit the concept of approximate joint diagonalization in order to reconstruct the spatial covariance matrices assuming the existence of (and identifying) a common eigenspace in which the application of Riemannian geometry is significantly simplified.Main results.The employed reconstruction process abides to physiologically plausible assumptions, reduces the computational complexity in Riemannian geometry schemes and bridges the gap between rigorous mathematical procedures and computational neuroscience. Our approach is both formally established and experimentally validated by employing real and synthetic EEG data.Significance.The implications of the introduced reconstruction process are highlighted by reformulating and re-introducing two signal processing methodologies, namely the 'Symmetric Positive Definite (SPD) Matrix Quantization' and the 'Coding over SPD Atoms'. The presented approach paves the way for robust and efficient neuroscientific explorations that exploit Riemannian geometry schemes.

Tangent space alignment: Transfer learning for Brain-Computer Interface

Statistical variability of electroencephalography (EEG) between subjects and between sessions is a common problem faced in the field of Brain-Computer Interface (BCI). Such variability prevents the usage of pre-trained machine learning models and requires the use of a calibration for every new session. This paper presents a new transfer learning (TL) method that deals with this variability. This method aims to reduce calibration time and even improve accuracy of BCI systems by aligning EEG data from one subject to the other in the tangent space of the positive definite matrices Riemannian manifold. We tested the method on 18 BCI databases comprising a total of 349 subjects pertaining to three BCI paradigms, namely, event related potentials (ERP), motor imagery (MI), and steady state visually evoked potentials (SSVEP). We employ a support vector classifier for feature classification. The results demonstrate a significant improvement of classification accuracy, as compared to a classical training-test pipeline, in the case of the ERP paradigm, whereas for both the MI and SSVEP paradigm no deterioration of performance is observed. A global 2.7% accuracy improvement is obtained compared to a previously published Riemannian method, Riemannian Procrustes Analysis (RPA). Interestingly, tangent space alignment has an intrinsic ability to deal with transfer learning for sets of data that have different number of channels, naturally applying to inter-dataset transfer learning.

An Efficient Model-Compressed EEGNet Accelerator for Generalized Brain-Computer Interfaces With Near Sensor Intelligence

Brain-computer interfaces (BCIs) is promising in interacting with machines through electroencephalogram (EEG) signal. The compact end-to-end neural network model for generalized BCIs, EEGNet, has been implemented in hardware to get near sensor intelligence, but without enough efficiency. To utilize EEGNet in low-power wearable device for long-term use, this paper proposes an efficient EEGNet inference accelerator. Firstly, the EEGNet model is compressed by embedded channel selection, normalization merging, and product quantization. The customized accelerator based on the compressed model is then designed. The multilayer convolutions are achieved by reusing multiplying-accumulators and processing elements (PEs) to minimize area of logic circuits, and the weights and intermediate results are quantized to minimize memory sizes. The PEs are clock-gated to save power. Experimental results in FPGA on three datasets show the good generalizing ability of the proposed design across three BCI diagrams, which only consumes 3.31% area and 1.35% power compared to the one-to-one parallel design. The speedup factors of 1.4, 3.5, and 3.7 are achieved by embedded channel selection with negligible loss of accuracy (-0.80%). The presented accelerator is also synthesized in 65 nm CMOS low power (LP) process and consumes 0.23M gates, 24.4 ms/inference, 0.267 mJ/inference, which is 87.22% more efficient than the implementation of EEGNet in a RISC-V MCU realized in 40 nm CMOS LP process in terms of area, and 20.77% more efficient in terms of energy efficiency on BCIC-IV-2a dataset.

Review of brain–computer interface based on steady‐state visual evoked potential

The brain–computer interface (BCI) technology has received lots of attention in the field of scientific research because it can help disabled people improve their quality of life. Steady‐state visual evoked potential (SSVEP) is the most researched BCI experimental paradigm, which offers the advantages of high signal‐to‐noise ratio and short training‐time requirement by users. In a complete BCI system, the two most critical components are the experimental paradigm and decoding algorithm. However, a systematic combination of the SSVEP experimental paradigm and decoding algorithms is missing in existing studies. In the present study, the transient visual evoked potential, SSVEP, and various improved SSVEP paradigms are compared and analyzed, and the problems and development bottlenecks in the experimental paradigm are finally pointed out. Subsequently, the canonical correlation analysis and various improved decoding algorithms are introduced, and the opportunities and challenges of the SSVEP decoding algorithm are discussed.

Machine-learning model predicting postoperative delirium in older patients using intraoperative frontal electroencephalographic signatures

Objective

In older patients receiving general anesthesia, postoperative delirium (POD) is the most frequent form of cerebral dysfunction. Early identification of patients at higher risk to develop POD could provide the opportunity to adapt intraoperative and postoperative therapy. We, therefore, propose a machine learning approach to predict the risk of POD in elderly patients, using routine intraoperative electroencephalography (EEG) and clinical data that are readily available in the operating room.

Methods

We conducted a retrospective analysis of the data of a single-center study at the Charité-Universitätsmedizin Berlin, Department of Anesthesiology [ISRCTN 36437985], including 1,277 patients, older than 60 years with planned surgery and general anesthesia. To deal with the class imbalance, we used balanced ensemble methods, specifically Bagging and Random Forests and as a performance measure, the area under the ROC curve (AUC-ROC). We trained our models including basic clinical parameters and intraoperative EEG features in particular classical spectral and burst suppression signatures as well as multi-band covariance matrices, which were classified, taking advantage of the geometry of a Riemannian manifold. The models were validated with 10 repeats of a 10-fold cross-validation.

Results

Including EEG data in the classification resulted in a robust and reliable risk evaluation for POD. The clinical parameters alone achieved an AUC-ROC score of 0.75. Including EEG signatures improved the classification when the patients were grouped by anesthetic agents and evaluated separately for each group. The spectral features alone showed an AUC-ROC score of 0.66; the covariance features showed an AUC-ROC score of 0.68. The AUC-ROC scores of EEG features relative to patient data differed by anesthetic group. The best performance was reached, combining both the EEG features and the clinical parameters. Overall, the AUC-ROC score was 0.77, for patients receiving Propofol it was 0.78, for those receiving Sevoflurane it was 0.8 and for those receiving Desflurane 0.73. Applying the trained prediction model to an independent data set of a different clinical study confirmed these results for the combined classification, while the classifier on clinical parameters alone did not generalize.

Conclusion

A machine learning approach combining intraoperative frontal EEG signatures with clinical parameters could be an easily applicable tool to early identify patients at risk to develop POD.

Riemannian distance based channel selection and feature extraction combining discriminative time-frequency bands and Riemannian tangent space for MI-BCIs

OBJECTIVE

Motor imagery-based brain computer interfaces (MI-BCIs) have been widely researched because they do not demand external stimuli and have a high degree of maneuverability. In most scenarios, superabundant selected channels, fixed time windows, and frequency bands would certainly affect the performance of MI-BCIs due to the neurophysiological diversities between different individuals. In this study, we tended to effectively use the Riemannian geometry of spatial covariance matrix to extract more robust features and thus enhance the decoding efficiency.

APPROACH

First, we propose a Riemannian distance-based EEG channel selection method (RDCS), which preliminarily reduces the information redundancy in the first stage. Second, we extract discriminative Riemannian Tangent Space features of EEG signals of selected channels from the most discriminant time-frequency bands (DTFRTS) to further enhance decoding accuracy for MI-BCIs. Finally, we trained a support vector machine (SVM) model with a linear kernel to classify our extracted discriminative Riemannian features and evaluated our proposed method using publicly available BCI Competition Ⅳ dataset Ⅰ (DS1) and Competition Ⅲ dataset Ⅲa (DS2).

MAIN RESULTS

The experimental results showed that the average classification accuracy with the selected 10-channel EEG signals of our method is 88.1% and 91.6% in DS1 and DS2 respectively. The average improvements are 24.3% & 27.1% on DS1 and 4.4% & 14.2% on DS2 for 10 & 20 selected channels, respectively.

SIGNIFICANCE

These results showed that our proposed method is a promising candidate for performance improvement of MI-BCIs.

RNeuMark: A Riemannian EEG Analysis Framework for Neuromarketing

Neuromarketing exploits neuroimaging techniques so as to reinforce the predictive power of conventional marketing tools, like questionnaires and focus groups. Electroencephalography (EEG) is the most commonly encountered neuroimaging technique due to its non-invasiveness, low-cost, and its very recent embedding in wearable devices. The transcription of brainwave patterns to consumer attitude is supported by various signal descriptors, while the quest for profitable novel ways is still an open research question. Here, we suggest the use of sample covariance matrices as alternative descriptors, that encapsulate the coordinated neural activity from distinct brain areas, and the adoption of Riemannian geometry for their handling. We first establish the suitability of Riemannian approach for neuromarketing-related problems and then suggest a relevant decoding scheme for predicting consumers' choices (e.g., willing to buy or not a specific product). Since the decision-making process involves the concurrent interaction of various cognitive processes and consequently of distinct brain rhythms, the proposed decoder takes the form of an ensemble classifier that builds upon a multi-view perspective, with each view dedicated to a specific frequency band. Adopting a standard machine learning procedure, and using a set of trials (training data) in conjunction with the associated behavior labels ("buy"/ "not buy"), we train a battery of classifiers accordingly. Each classifier is designed to operate in the space recovered from the inter-trial distances of SCMs and to cast a rhythm-depended decision that is eventually combined with the predictions of the rest ones. The demonstration and evaluation of the proposed approach are performed in 2 neuromarketing-related datasets of different nature. The first is employed to showcase the potential of the suggested descriptor, while the second to showcase the decoder's superiority against popular alternatives in the field.

Decoding EEG rhythms offline and online during motor imagery for standing and sitting based on a brain-computer interface

Motor imagery (MI)-based brain-computer interface (BCI) systems have shown promising advances for lower limb motor rehabilitation. The purpose of this study was to develop an MI-based BCI for the actions of standing and sitting. Thirty-two healthy subjects participated in the study using 17 active EEG electrodes. We used a combination of the filter bank common spatial pattern (FBCSP) method and the regularized linear discriminant analysis (RLDA) technique for decoding EEG rhythms offline and online during motor imagery for standing and sitting. The offline analysis indicated the classification of motor imagery and idle state provided a mean accuracy of 88.51 ± 1.43% and 85.29 ± 1.83% for the sit-to-stand and stand-to-sit transitions, respectively. The mean accuracies of the sit-to-stand and stand-to-sit online experiments were 94.69 ± 1.29% and 96.56 ± 0.83%, respectively. From these results, we believe that the MI-based BCI may be useful to future brain-controlled standing systems.

A novel deep learning model based on the ICA and Riemannian manifold for EEG-based emotion recognition

BACKGROUND

The EEG-based emotion recognition is one of the primary research orientations in the field of emotional intelligence and human-computer interaction (HCI).

NEW METHOD

We proposed a novel model, denoted as ICRM-LSTM, for EEG-based emotion recognition by combining the independent component analysis (ICA), the Riemannian manifold (RM), and the long short-term memory network (LSTM). The SEED and MAHNOB-HCI dataset were employed to verify the performance of the proposed model. Firstly, ICA was used to perform blind source separation (BSS) for the preprocessed EEG signals provided by the two datasets. Then, a series of the covariance matrices according to time order were extracted from the blind source signals, and the symmetric positive definiteness of covariance matrix allowed us to project them from RM space to Euclid space by logarithmic mapping. Finally, the transformed covariance matrices were taken as inputs of the LSTM network to perform the emotion recognition.

RESULTS

To validate the effect of the ICRM method on the performance of the proposed model, we designed three groups of comparative experiments. The average accuracy of the ICRM-LSTM model were 96.75 % and 98.09 % in SEED and MAHNOB-HCI, respectively. Then we compared our model with the models didn't perform the ICA or RM method, where we found that the proposed model had better performance. Furthermore, to verify the robustness, we added three groups of Gaussian noise with different signal-to-noise ratios (SNR) to the preprocessed EEG signals, and the proposed model achieved a good performance in both the two datasets.

COMPARISON WITH EXISTING METHOD

The performance of our model was superior to most of existing methods which also employed the SEED or MAHNOB-HCI dataset.

CONCLUSION

This paper demonstrated that the ICA and RM were helpful for EEG-based emotion recognition, and provided the evidence that the RM method could effectively alleviate the problem of the uncertain ordering of ICA.

Decoding of Turning Intention during Walking Based on EEG Biomarkers

In the EEG literature, there is a lack of asynchronous intention models that realistically propose interfaces for applications that must operate in real time. In this work, a novel BMI approach to detect in real time the intention to turn is proposed. For this purpose, an offline, pseudo-online and online analysis is presented to validate the EEG as a biomarker for the intention to turn. This article presents a methodology for the creation of a BMI that could differentiate two classes: monotonous walk and intention to turn. A comparison of some of the most popular algorithms in the literature is conducted. To filter the signal, two relevant algorithms are used: H∞ filter and ASR. For processing and classification, the mean of the covariance matrices in the Riemannian space was calculated and then, with various classifiers of different types, the distance of the test samples to each class in the Riemannian space was estimated. This dispenses with power-based models and the necessary baseline correction, which is a problem in realistic scenarios. In the cross-validation for a generic selection (valid for any subject) and a personalized one, the results were, on average, 66.2% and 69.6% with the best filter H∞. For the pseudo-online, the custom configuration for each subject was an average of 40.2% TP and 9.3 FP/min; the best subject obtained 43.9% TP and 2.9 FP/min. In the final validation test, this subject obtained 2.5 FP/min and an accuracy rate of 71.43%, and the turn anticipation was 0.21 s on average.

EEGSym: Overcoming Inter-Subject Variability in Motor Imagery Based BCIs With Deep Learning

In this study, we present a new Deep Learning (DL) architecture for Motor Imagery (MI) based Brain Computer Interfaces (BCIs) called EEGSym. Our implementation aims to improve previous state-of-the-art performances on MI classification by overcoming inter-subject variability and reducing BCI inefficiency, which has been estimated to affect 10-50% of the population. This convolutional neural network includes the use of inception modules, residual connections and a design that introduces the symmetry of the brain through the mid-sagittal plane into the network architecture. It is complemented with a data augmentation technique that improves the generalization of the model and with the use of transfer learning across different datasets. We compare EEGSym's performance on inter-subject MI classification with ShallowConvNet, DeepConvNet, EEGNet and EEG-Inception. This comparison is performed on 5 publicly available datasets that include left or right hand motor imagery of 280 subjects. This population is the largest that has been evaluated in similar studies to date. EEGSym significantly outperforms the baseline models reaching accuracies of 88.6±9.0 on Physionet, 83.3±9.3 on OpenBMI, 85.1±9.5 on Kaya2018, 87.4±8.0 on Meng2019 and 90.2±6.5 on Stieger2021. At the same time, it allows 95.7% of the tested population (268 out of 280 users) to reach BCI control (≥70% accuracy). Furthermore, these results are achieved using only 16 electrodes of the more than 60 available on some datasets. Our implementation of EEGSym, which includes new advances for EEG processing with DL, outperforms previous state-of-the-art approaches on inter-subject MI classification.

Mental State Classification Using Multi-Graph Features

We consider the problem of extracting features from passive, multi-channel electroencephalogram (EEG) devices for downstream inference tasks related to high-level mental states such as stress and cognitive load. Our proposed feature extraction method uses recently developed spectral-based multi-graph tools and applies them to the time series of graphs implied by the statistical dependence structure (e.g., correlation) amongst the multiple sensors. We study the features in the context of two datasets each consisting of at least 30 participants and recorded using multi-channel EEG systems. We compare the classification performance of a classifier trained on the proposed features to a classifier trained on the traditional band power-based features in three settings and find that the two feature sets offer complementary predictive information. We conclude by showing that the importance of particular channels and pairs of channels for classification when using the proposed features is neuroscientifically valid.

Geometrical modelling of neuronal clustering and development

The dynamic geometry of neuronal development is an essential concept in theoretical neuroscience. We aimed to design a mathematical model which outlines stepwise in an innovative form and designed to model neuronal development geometrically and modelling spatially the neuronal-electrical field interaction. We demonstrated flexibility in forming the cell and its nucleus to show neuronal growth from inside to outside that uses a fractal cylinder to generate neurons (pyramidal/sphere) in form of mathematically called ‘surface of revolution’. Furthermore, we verified the effect of the adjacent neurons on a free branch from one-side, by modelling a ‘normal vector surface’ that represented a group of neurons. Our model also indicated how the geometrical shapes and clustering of the neurons can be transformed mathematically in the form of vector field that is equivalent to the neuronal electromagnetic activity/electric flux. We further simulated neuronal-electrical field interaction that was implemented spatially using Van der Pol oscillator and taking Laplacian vector field as it reflects biophysical mechanism of neuronal activity and geometrical change. In brief, our study would be considered a proper platform and inspiring modelling for next more complicated geometrical and electrical constructions.

Class-distinctiveness-based frequency band selection on the Riemannian manifold for oscillatory activity-based BCIs: preliminary results

Considering user-specific settings is known to enhance Brain-Computer Interface (BCI) performances. In particular, the optimal frequency band for oscillatory activity classification is highly user-dependent and many frequency band selection methods have been developed in the past two decades. However, it is not well studied whether those conventional methods can be efficiently applied to the Riemannian BCIs, a recent family of BCI systems that utilize the non-Euclidean nature of the data unlike conventional BCI pipelines. In this paper, we proposed a novel frequency band selection method working on the Riemannian manifold. The frequency band is selected considering the class distinctiveness as quantified based on the inter-class distance and the intra-class variance on the manifold. An advantage of this method is that the frequency bandwidth can be adjusted for each individual without intensive optimization steps. In a comparative experiment using a public dataset of motor imagery-based BCI, our method showed a substantial improvement in average accuracy over both a fixed broad frequency band and a popular conventional frequency band selection method. In particular, our method substantially improved performances for subjects with initially low accuracies. This preliminary result suggests the importance of developing new user-specific setting algorithms considering the manifold properties, rather than directly applying methods developed prior to the rise of the Riemannian BCIs.

Pseudo-online detection and classification for upper-limb movements

Objective. This study analyzed detection (movement vs. non-movement) and classification (different types of movements) to decode upper-limb movement volitions in a pseudo-online fashion.Approach. Nine healthy subjects executed four self-initiated movements: left wrist extension, right wrist extension, left index finger extension, and right index finger extension. For detection, we investigated the performance of three individual classifiers (support vector machine (SVM), EEGNET, and Riemannian geometry featured SVM) on three frequency bands (0.05-5 Hz, 5-40 Hz, 0.05-40 Hz). The best frequency band and the best classifier combinations were constructed to realize an ensemble processing pipeline using majority voting. For classification, we used adaptive boosted Riemannian geometry model to differentiate contra-lateral and ipsilateral movements.Main results. The ensemble model achieved 79.6 ± 8.8% true positive rate and 3.1 ± 1.2 false positives per minute with 75.3 ± 112.6 ms latency on a pseudo-online detection task. The following classification gave around 67% accuracy to differentiate contralateral movements.Significance. The newly proposed ensemble method and pseudo-online testing procedure could provide a robust brain-computer interface design for movement decoding.

Epilepsy Detection Based on Riemann Potato in Noisy Environment

Epilepsy detection based on electroencephalogram (EEG) is important for the diagnosis and treatment of epilepsy. The existing feature extraction method not only consumes a lot of time but also leads to epilepsy information loss because of nonideal denoising. Therefore, the paper proposes to use noisy EEG signals to detect epilepsy. The original EEG signal is divided into normal signal and abnormal signal by Riemann potato, and the epilepsy detection model is established based on the normal signal and abnormal signal, respectively. Finally, the 2 detection results are combined as a final result. The detection performance of 94.84%, 83.03% sensitivity, and 97.67% specificity is achieved. The experimental results show that the original noisy signal which is separated by the Riemann potato can have high epilepsy detection performance.

EEG emotion recognition based on enhanced SPD matrix and manifold dimensionality reduction

Recently, Riemannian geometry-based pattern recognition has been widely employed to brain computer interface (BCI) researches, providing new idea for emotion recognition based on electroencephalogram (EEG) signals. Although the symmetric positive definite (SPD) matrix manifold constructed from the traditional covariance matrix contains large amount of spatial information, these methods do not perform well to classify and recognize emotions, and the high dimensionality problem still unsolved. Therefore, this paper proposes a new strategy for EEG emotion recognition utilizing Riemannian geometry with the aim of achieving better classification performance. The emotional EEG signals of 32 healthy subjects were from an open-source dataset (DEAP). The wavelet packets were first applied to extract the time-frequency features of the EEG signals, and then the features were used to construct the enhanced SPD matrix. A supervised dimensionality reduction algorithm was then designed on the Riemannian manifold to reduce the high dimensionality of the SPD matrices, gather samples of the same labels together, and separate samples of different labels as much as possible. Finally, the samples were mapped to the tangent space, and the K-nearest neighbors (KNN), Random Forest (RF) and Support Vector Machine (SVM) method were employed for classification. The proposed method achieved an average accuracy of 91.86%, 91.84% on the valence and arousal recognition tasks. Furthermore, we also obtained the superior accuracy of 86.71% on the four-class recognition task, demonstrated the superiority over state-of-the-art emotion recognition methods.

Harmonized-Multinational qEEG norms (HarMNqEEG)

This paper extends frequency domain quantitative electroencephalography (qEEG) methods pursuing higher sensitivity to detect Brain Developmental Disorders. Prior qEEG work lacked integration of cross-spectral information omitting important functional connectivity descriptors. Lack of geographical diversity precluded accounting for site-specific variance, increasing qEEG nuisance variance. We ameliorate these weaknesses. (i) Create lifespan Riemannian multinational qEEG norms for cross-spectral tensors. These norms result from the HarMNqEEG project fostered by the Global Brain Consortium. We calculate the norms with data from 9 countries, 12 devices, and 14 studies, including 1564 subjects. Instead of raw data, only anonymized metadata and EEG cross-spectral tensors were shared. After visual and automatic quality control, developmental equations for the mean and standard deviation of qEEG traditional and Riemannian DPs were calculated using additive mixed-effects models. We demonstrate qEEG "batch effects" and provide methods to calculate harmonized z-scores. (ii) We also show that harmonized Riemannian norms produce z-scores with increased diagnostic accuracy predicting brain dysfunction produced by malnutrition in the first year of life and detecting COVID induced brain dysfunction. (iii) We offer open code and data to calculate different individual z-scores from the HarMNqEEG dataset. These results contribute to developing bias-free, low-cost neuroimaging technologies applicable in various health settings.

Retrospective on the First Passive Brain-Computer Interface Competition on Cross-Session Workload Estimation

As is the case in several research domains, data sharing is still scarce in the field of Brain-Computer Interfaces (BCI), and particularly in that of passive BCIs-i.e., systems that enable implicit interaction or task adaptation based on a user's mental state(s) estimated from brain measures. Moreover, research in this field is currently hindered by a major challenge, which is tackling brain signal variability such as cross-session variability. Hence, with a view to develop good research practices in this field and to enable the whole community to join forces in working on cross-session estimation, we created the first passive brain-computer interface competition on cross-session workload estimation. This competition was part of the 3rd International Neuroergonomics conference. The data were electroencephalographic recordings acquired from 15 volunteers (6 females; average 25 y.o.) who performed 3 sessions-separated by 7 days-of the Multi-Attribute Task Battery-II (MATB-II) with 3 levels of difficulty per session (pseudo-randomized order). The data -training and testing sets-were made publicly available on Zenodo along with Matlab and Python toy code (https://doi.org/10.5281/zenodo.5055046). To this day, the database was downloaded more than 900 times (unique downloads of all version on the 10th of December 2021: 911). Eleven teams from 3 continents (31 participants) submitted their work. The best achieving processing pipelines included a Riemannian geometry-based method. Although better than the adjusted chance level (38% with an α at 0.05 for a 3-class classification problem), the results still remained under 60% of accuracy. These results clearly underline the real challenge that is cross-session estimation. Moreover, they confirmed once more the robustness and effectiveness of Riemannian methods for BCI. On the contrary, chance level results were obtained by one third of the methods-4 teams- based on Deep Learning. These methods have not demonstrated superior results in this contest compared to traditional methods, which may be due to severe overfitting. Yet this competition is the first step toward a joint effort to tackle BCI variability and to promote good research practices including reproducibility.

Security Requirements and Challenges of 6G Technologies and Applications

After implementing 5G technology, academia and industry started researching 6th generation wireless network technology (6G). 6G is expected to be implemented around the year 2030. It will offer a significant experience for everyone by enabling hyper-connectivity between people and everything. In addition, it is expected to extend mobile communication possibilities where earlier generations could not have developed. Several potential technologies are predicted to serve as the foundation of 6G networks. These include upcoming and current technologies such as post-quantum cryptography, artificial intelligence (AI), machine learning (ML), enhanced edge computing, molecular communication, THz, visible light communication (VLC), and distributed ledger (DL) technologies such as blockchain. From a security and privacy perspective, these developments need a reconsideration of prior security traditional methods. New novel authentication, encryption, access control, communication, and malicious activity detection must satisfy the higher significant requirements of future networks. In addition, new security approaches are necessary to ensure trustworthiness and privacy. This paper provides insights into the critical problems and difficulties related to the security, privacy, and trust issues of 6G networks. Moreover, the standard technologies and security challenges per each technology are clarified. This paper introduces the 6G security architecture and improvements over the 5G architecture. We also introduce the security issues and challenges of the 6G physical layer. In addition, the AI/ML layers and the proposed security solution in each layer are studied. The paper summarizes the security evolution in legacy mobile networks and concludes with their security problems and the most essential 6G application services and their security requirements. Finally, this paper provides a complete discussion of 6G networks’ trustworthiness and solutions.

Functional connectivity ensemble method to enhance BCI performance (FUCONE)

OBJECTIVE

Relying on the idea that functional connectivity provides important insights on the underlying dynamic of neuronal interactions, we propose a novel framework that combines functional connectivity estimators and covariance-based pipelines to improve the classification of mental states, such as motor imagery.

METHODS

A Riemannian classifier is trained for each estimator and an ensemble classifier combines the decisions in each feature space. A thorough assessment of the functional connectivity estimators is provided and the best performing pipeline among those tested, called FUCONE, is evaluated on different conditions and datasets.

RESULTS

Using a meta-analysis to aggregate results across datasets, FUCONE performed significantly better than all state-of-the-art methods.

CONCLUSION

The performance gain is mostly imputable to the improved diversity of the feature spaces, increasing the robustness of the ensemble classifier with respect to the inter- and intra-subject variability.

SIGNIFICANCE

Our results offer new insights into the need to consider functional connectivity-based methods to improve the BCI performance.

Experimental Validation of the Cumulative MDRM in theP300 Speller Machine

The P300 speller Machine is among the leading applications of the electroencephalography (EEG)-based brain computer interfaces (BCIs), it is still a benchmark and a persistent challenge for the BCI Community. EEG signal classification represents the key piece of a BCI chain. The minimum distance to Riemannian mean (MDRM) belongs to these classification methods emerging in different BCI applications such as text spelling by thought. Based on a binary classification of each covariance matrix separately, character prediction is done according to the highest score across the whole set of all repetitions. Minimum cumulative distance to Riemannian mean (MCDRM) is a Cumulative variant of the MDRM, perfectly adapted to the P300 Speller Machine. The power of this variant is that prediction takes a more global proceeding involving the n repetitions together. Indeed, thanks to cumulative distances selected row and column are those having the covariance matrices both closer to the Target barycenter and farther from the non-Target one. This variant overcomes the main MDRM limitations as it improves inter-sessional generalization, allows optimal use of all repetitions and reduces considerably the risk of conflict appearing during the selection of rows and columns leading to character prediction. We applied this variant to the raw signals of Data set II-b of Berlin BCI and compared to the published results the MCDRM offers significantly higher results: 97.5% of correct predictions compared to the 96.5% of the competition winner. The MCDRM fits best with the P300 Speller machine, especially when dealing with noisy signals that requires intelligent and optimal usage of the n repetitions.

Robust learning from corrupted EEG with dynamic spatial filtering

Building machine learning models using EEG recorded outside of the laboratory setting requires methods robust to noisy data and randomly missing channels. This need is particularly great when working with sparse EEG montages (1-6 channels), often encountered in consumer-grade or mobile EEG devices. Neither classical machine learning models nor deep neural networks trained end-to-end on EEG are typically designed or tested for robustness to corruption, and especially to randomly missing channels. While some studies have proposed strategies for using data with missing channels, these approaches are not practical when sparse montages are used and computing power is limited (e.g., wearables, cell phones). To tackle this problem, we propose dynamic spatial filtering (DSF), a multi-head attention module that can be plugged in before the first layer of a neural network to handle missing EEG channels by learning to focus on good channels and to ignore bad ones. We tested DSF on public EEG data encompassing ∼4,000 recordings with simulated channel corruption and on a private dataset of ∼100 at-home recordings of mobile EEG with natural corruption. Our proposed approach achieves the same performance as baseline models when no noise is applied, but outperforms baselines by as much as 29.4% accuracy when significant channel corruption is present. Moreover, DSF outputs are interpretable, making it possible to monitor the effective channel importance in real-time. This approach has the potential to enable the analysis of EEG in challenging settings where channel corruption hampers the reading of brain signals.

Workshops of the Eighth International Brain-Computer Interface Meeting: BCIs: The Next Frontier

The Eighth International Brain-Computer Interface (BCI) Meeting was held June 7-9th, 2021 in a virtual format. The conference continued the BCI Meeting series' interactive nature with 21 workshops covering topics in BCI (also called brain-machine interface) research. As in the past, workshops covered the breadth of topics in BCI. Some workshops provided detailed examinations of specific methods, hardware, or processes. Others focused on specific BCI applications or user groups. Several workshops continued consensus building efforts designed to create BCI standards and increase the ease of comparisons between studies and the potential for meta-analysis and large multi-site clinical trials. Ethical and translational considerations were both the primary topic for some workshops or an important secondary consideration for others. The range of BCI applications continues to expand, with more workshops focusing on approaches that can extend beyond the needs of those with physical impairments. This paper summarizes each workshop, provides background information and references for further study, presents an overview of the discussion topics, and describes the conclusion, challenges, or initiatives that resulted from the interactions and discussion at the workshop.

Customizing skills for assistive robotic manipulators, an inverse reinforcement learning approach with error-related potentials

Robotic assistance via motorized robotic arm manipulators can be of valuable assistance to individuals with upper-limb motor disabilities. Brain-computer interfaces (BCI) offer an intuitive means to control such assistive robotic manipulators. However, BCI performance may vary due to the non-stationary nature of the electroencephalogram (EEG) signals. It, hence, cannot be used safely for controlling tasks where errors may be detrimental to the user. Avoiding obstacles is one such task. As there exist many techniques to avoid obstacles in robotics, we propose to give the control to the robot to avoid obstacles and to leave to the user the choice of the robot behavior to do so a matter of personal preference as some users may be more daring while others more careful. We enable the users to train the robot controller to adapt its way to approach obstacles relying on BCI that detects error-related potentials (ErrP), indicative of the user’s error expectation of the robot’s current strategy to meet their preferences. Gaussian process-based inverse reinforcement learning, in combination with the ErrP-BCI, infers the user’s preference and updates the obstacle avoidance controller so as to generate personalized robot trajectories. We validate the approach in experiments with thirteen able-bodied subjects using a robotic arm that picks up, places and avoids real-life objects. Results show that the algorithm can learn user’s preference and adapt the robot behavior rapidly using less than five demonstrations not necessarily optimal.

Teaching an assistive robotic manipulator to move objects in a cluttered table requires demonstrations from expert operators, but what if the experts are individuals with motor disabilities? Batzianoulis et al. propose a learning approach which combines robot autonomy and a brain-computer interfacing that decodes whether the generated trajectories meet the user’s criteria, and show how their system enables the robot to learn individual user’s preferred behaviors using less than five demonstrations that are not necessarily optimal.

Mental State Detection Using Riemannian Geometry on Electroencephalogram Brain Signals

The goal of this study was to implement a Riemannian geometry (RG)-based algorithm to detect high mental workload (MWL) and mental fatigue (MF) using task-induced electroencephalogram (EEG) signals. In order to elicit high MWL and MF, the participants performed a cognitively demanding task in the form of the letter n-back task. We analyzed the time-varying characteristics of the EEG band power (BP) features in the theta and alpha frequency band at different task conditions and cortical areas by employing a RG-based framework. MWL and MF were considered as too high, when the Riemannian distances of the task-run EEG reached or surpassed the threshold of the baseline EEG. The results of this study showed a BP increase in the theta and alpha frequency bands with increasing experiment duration, indicating elevated MWL and MF that impedes/hinders the task performance of the participants. High MWL and MF was detected in 8 out of 20 participants. The Riemannian distances also showed a steady increase toward the threshold with increasing experiment duration, with the most detections occurring toward the end of the experiment. To support our findings, subjective ratings (questionnaires concerning fatigue and workload levels) and behavioral measures (performance accuracies and response times) were also considered.

Sub-100 μW Multispectral Riemannian Classification for EEG-Based Brain-Machine Interfaces

Motor imagery (MI) brain-machine interfaces (BMIs) enable us to control machines by merely thinking of performing a motor action. Practical use cases require a wearable solution where the classification of the brain signals is done locally near the sensor using machine learning models embedded on energy-efficient microcontroller units (MCUs), for assured privacy, user comfort, and long-term usage. In this work, we provide practical insights on the accuracy-cost trade-off for embedded BMI solutions. Our multispectral Riemannian classifier reaches 75.1% accuracy on a 4-class MI task. The accuracy is further improved by tuning different types of classifiers to each subject, achieving 76.4%. We further scale down the model by quantizing it to mixed-precision representations with a minimal accuracy loss of 1% and 1.4%, respectively, which is still up to 4.1% more accurate than the state-of-the-art embedded convolutional neural network. We implement the model on a low-power MCU within an energy budget of merely 198 μJ and taking only 16.9 ms per classification. Classifying samples continuously, overlapping the 3.5 s samples by 50% to avoid missing user inputs allows for operation at just 85 μW. Compared to related works in embedded MI-BMIs, our solution sets the new state-of-the-art in terms of accuracy-energy trade-off for near-sensor classification.

On the interpretation of linear Riemannian tangent space model parameters in M/EEG

Riemannian tangent space methods offer state-of-the-art performance in magnetoencephalography (MEG) and electroencephalography (EEG) based applications such as brain-computer interfaces and biomarker development. One limitation, particularly relevant for biomarker development, is limited model interpretability compared to established component-based methods. Here, we propose a method to transform the parameters of linear tangent space models into interpretable patterns. Using typical assumptions, we show that this approach identifies the true patterns of latent sources, encoding a target signal. In simulations and two real MEG and EEG datasets, we demonstrate the validity of the proposed approach and investigate its behavior when the model assumptions are violated. Our results confirm that Riemannian tangent space methods are robust to differences in the source patterns across observations. We found that this robustness property also transfers to the associated patterns.

A novel method to reduce the motor imagery BCI illiteracy

To reduce the motor imagery brain-computer interface (MI-BCI) illiteracy phenomenon and improve the classification accuracy, this paper proposed a novel method combining paradigm selection and Riemann distance classification. Firstly, a novel sensitivity-based paradigm selection (SPS) algorithm is designed for the optimization of classification to find the best classification pattern through a sensitive indicator. Then, a generalized Riemann minimum distance mean (GRMDM) classifier is proposed by introducing a weight factor to fuse the Log-Euclidean Metric classifier and the Riemannian Stein divergence classifier. The experimental results show that the proposed method achieves a better performance for multi-class motor imagery tasks. The average classification accuracy on the BCI competition IV dataset2a is 80.98%, which is 11.04% higher than Stein divergence classifier on the original two-class paradigm. Furthermore, the proposed method demonstrates its capacity on reducing MI-BCI illiteracy. Graphical abstract Here we investigate whether the BCI illiteracy phenomenon can be reduced through sensitivity-based paradigm selection (SPS) method and generalized Riemann minimum distance mean (GRMDM) classifier when performing motor imagery tasks.

Fatigue Detection with Spatial-Temporal Fusion Method on Covariance Manifolds of Electroencephalography

With the increasing pressure of current life, fatigue caused by high-pressure work has deeply affected people and even threatened their lives. In particular, fatigue driving has become a leading cause of traffic accidents and deaths. This paper investigates electroencephalography (EEG)-based fatigue detection for driving by mining the latent information through the spatial-temporal changes in the relations between EEG channels. First, EEG data are partitioned into several segments to calculate the covariance matrices of each segment, and then we feed these matrices into a recurrent neural network to obtain high-level temporal information. Second, the covariance matrices of whole signals are leveraged to extract two kinds of spatial features, which will be fused with temporal characteristics to obtain comprehensive spatial-temporal information. Experiments on an open benchmark showed that our method achieved an excellent classification accuracy of 93.834% and performed better than several novel methods. These experimental results indicate that our method enables better reliability and feasibility in the detection of fatigued driving.

Classification of motor imagery using multisource joint transfer learning

As an important way for human-computer interaction, the motor imagery brain-computer interface (MI-BCI) can decode personal motor intention directly by analyzing electroencephalogram (EEG) signals. However, a large amount of labeled data has to be collected for each new subject since EEG patterns vary between individuals. The long calibration phase severely limits the further development of MI-BCI. To tackle this problem, multi-source joint domain adaption (MJDA) and multi-source joint Riemannian adaption (MJRA) algorithms are proposed in this paper. Both methods aim to transfer knowledge from other subjects to the current subject who has only a small amount of labeled data. First, the common spatial pattern with Euclidean alignment is used to select source subjects who have similar spatial patterns to the target subject. Second, the covariance matrices of EEG trials are aligned in Riemannian space by removing subject-specific baselines. These two steps are shared by MJDA and MJRA. In the last step, MJDA attempts to minimize the feature distribution mismatch in the Riemannian tangent space, while MJRA attempts to find an adaptive Riemannian classifier. Finally, the proposed methods are validated on two datasets: BCI Competition IV 2a and online event-related desynchronization (ERD)-BCI. The experimental results demonstrate that both MJDA and MJRA outperform the state-of-the-art approaches. The MJDA provides a new idea for the offline analysis of MI-BCI, while MJRA could make a big difference to the online calibration of MI-BCI.

Estimating the probability of coincidental similarity between atomic displacement parameters with machine learning

High-resolution diffraction studies of macromolecules incorporate the tensor form of the anisotropic displacement parameter (ADP) of atoms from their mean position. The comparison of these parameters requires a statistical framework that can handle the experimental and modeling errors linked to structure determination. Here, a Bayesian machine learning model is introduced that approximates ADPs with the random Wishart distribution. This model allows for the comparison of random samples from a distribution that is trained on experimental structures. The comparison revealed that the experimental similarity between atoms is larger than predicted by the random model for a substantial fraction of the comparisons. Different metrics between ADPs were evaluated and categorized based on how useful they are at detecting non-accidental similarity and whether they can be replaced by other metrics. The most complementary comparisons were provided by Euclidean, Riemann and Wasserstein metrics. The analysis of ADP similarity and the positional distance of atoms in bovine trypsin revealed a set of atoms with striking ADP similarity over a long physical distance, and generally the physical distance between atoms and their ADP similarity do not correlate strongly. A substantial fraction of long- and short-range ADP similarities does not form by coincidence and are reproducibly observed in different crystal structures of the same protein.

Enhancing transfer performance across datasets for brain-computer interfaces using a combination of alignment strategies and adaptive batch normalization

Objective. Recently, transfer learning (TL) and deep learning (DL) have been introduced to solve intra- and inter-subject variability problems in brain-computer interfaces (BCIs). However, current TL and DL algorithms are usually validated within a single dataset, assuming that data of the test subjects are acquired under the same condition as that of training (source) subjects. This assumption is generally violated in practice because of different acquisition systems and experimental settings across studies and datasets. Thus, the generalization ability of these algorithms needs further validations in a cross-dataset scenario, which is closer to the actual situation. This study compared the transfer performance of pre-trained deep-learning models with different preprocessing strategies in a cross-dataset scenario.Approach. This study used four publicly available motor imagery datasets, each was successively selected as a source dataset, and the others were used as target datasets. EEGNet and ShallowConvNet with four preprocessing strategies, namely channel normalization, trial normalization, Euclidean alignment, and Riemannian alignment, were trained with the source dataset. The transfer performance of pre-trained models was validated on the target datasets. This study also used adaptive batch normalization (AdaBN) for reducing interval covariate shift across datasets. This study compared the transfer performance of using the four preprocessing strategies and that of a baseline approach based on manifold embedded knowledge transfer (MEKT). This study also explored the possibility and performance of fusing MEKT and EEGNet.Main results. The results show that DL models with alignment strategies had significantly better transfer performance than the other two preprocessing strategies. As an unsupervised domain adaptation method, AdaBN could also significantly improve the transfer performance of DL models. The transfer performance of DL models that combined AdaBN and alignment strategies significantly outperformed MEKT. Moreover, the generalizability of EEGNet models that combined AdaBN and alignment strategies could be further improved via the domain adaptation step in MEKT, achieving the best generalization ability among multiple datasets (BNCI2014001: 0.788, PhysionetMI: 0.679, Weibo2014: 0.753, Cho2017: 0.650).Significance. The combination of alignment strategies and AdaBN could easily improve the generalizability of DL models without fine-tuning. This study may provide new insights into the design of transfer neural networks for BCIs by separating source and target batch normalization layers in the domain adaptation process.

BioPyC, an Open-Source Python Toolbox for Offline Electroencephalographic and Physiological Signals Classification

Research on brain–computer interfaces (BCIs) has become more democratic in recent decades, and experiments using electroencephalography (EEG)-based BCIs has dramatically increased. The variety of protocol designs and the growing interest in physiological computing require parallel improvements in processing and classification of both EEG signals and bio signals, such as electrodermal activity (EDA), heart rate (HR) or breathing. If some EEG-based analysis tools are already available for online BCIs with a number of online BCI platforms (e.g., BCI2000 or OpenViBE), it remains crucial to perform offline analyses in order to design, select, tune, validate and test algorithms before using them online. Moreover, studying and comparing those algorithms usually requires expertise in programming, signal processing and machine learning, whereas numerous BCI researchers come from other backgrounds with limited or no training in such skills. Finally, existing BCI toolboxes are focused on EEG and other brain signals but usually do not include processing tools for other bio signals. Therefore, in this paper, we describe BioPyC, a free, open-source and easy-to-use Python platform for offline EEG and biosignal processing and classification. Based on an intuitive and well-guided graphical interface, four main modules allow the user to follow the standard steps of the BCI process without any programming skills: (1) reading different neurophysiological signal data formats, (2) filtering and representing EEG and bio signals, (3) classifying them, and (4) visualizing and performing statistical tests on the results. We illustrate BioPyC use on four studies, namely classifying mental tasks, the cognitive workload, emotions and attention states from EEG signals.