Adaptive multi-degree of freedom Brain Computer Interface using online feedback: Towards novel methods and metrics of mutual adaptation between humans and machines for BCI

Riemannian geometry-based metrics to measure and reinforce user performance changes during brain-computer interface user training

Despite growing interest and research into brain-computer interfaces (BCI), their usage remains limited outside of research laboratories. One reason for this is BCI inefficiency, the phenomenon where a significant number of potential users are unable to produce machine-discernible brain signal patterns to control the devices. To reduce the prevalence of BCI inefficiency, some have advocated for novel user-training protocols that enable users to more effectively modulate their neural activity. Important considerations for the design of these protocols are the assessment measures that are used for evaluating user performance and for providing feedback that guides skill acquisition. Herein, we present three trial-wise adaptations (running, sliding window and weighted average) of Riemannian geometry-based user-performance metrics (classDistinct reflecting the degree of class separability and classStability reflecting the level of within-class consistency) to enable feedback to the user following each individual trial. We evaluated these metrics, along with conventional classifier feedback, using simulated and previously recorded sensorimotor rhythm-BCI data to assess their correlation with and discrimination of broader trends in user performance. Analysis revealed that the sliding window and weighted average variants of our proposed trial-wise Riemannian geometry-based metrics more accurately reflected performance changes during BCI sessions compared to conventional classifier output. The results indicate the metrics are a viable method for evaluating and tracking user performance changes during BCI-user training and, therefore, further investigation into how these metrics may be presented to users during training is warranted.

An adaptive closed-loop ECoG decoder for long-term and stable bimanual control of an exoskeleton by a tetraplegic

OBJECTIVE

The article aims at addressing 2 challenges to step motor BCI out of laboratories: asynchronous control of complex bimanual effectors with large numbers of degrees of freedom, using chronic and safe recorders, and the decoding performance stability over time without frequent decoder recalibration.

APPROACH

Closed-loop adaptive/incremental decoder training is one strategy to create a model stable over time. Adaptive decoders update their parameters with new incoming data, optimizing the model parameters in real time. It allows cross-session training with multiple recording conditions during closed loop BCI experiments. In the article, an adaptive tensor-based Recursive Exponentially Weighted Markov-Switching multi-Linear Model (REW-MSLM) decoder is proposed. REW-MSLM uses a Mixture of Expert (ME) architecture, mixing or switching independent decoders (experts) according to the probability estimated by a "gating" model. A Hidden Markov model approach is employed as gating model to improve the decoding robustness and to provide strong idle state support. The ME architecture fits the multi-limb paradigm associating an expert to a particular limb or action.

MAIN RESULTS

Asynchronous control of an exoskeleton by a tetraplegic patient using a chronically implanted epidural electrocorticography (EpiCoG) recorder is reported. The stable over a period of 6 months (without decoder recalibration) 8-dimensional alternative bimanual control of the exoskeleton and its virtual avatar is demonstrated.

SIGNIFICANCE

Based on the long-term (>36 months) chronic bilateral epidural ECoG recordings in a tetraplegic (ClinicalTrials.gov, NCT02550522), we addressed the poorly explored field of asynchronous bimanual BCI. The new decoder was designed to meet to several challenges: the high-dimensional control of a complex effector in experiments closer to real-world behaviour (point-to-point pursuit versus conventional center-out tasks), with the ability of the BCI system to act as a stand-alone device switching between idle and control states, and a stable performance over a long period of time without decoder recalibration.

Co-adaptive control of bionic limbs via unsupervised adaptation of muscle synergies

OBJECTIVE

In this work, we present a myoelectric interface that extracts natural motor synergies from multi-muscle signals and adapts in real-time with new user inputs. With this unsupervised adaptive myocontrol (UAM) system, optimal synergies for control are continuously co-adapted with changes in user motor control, or as a function of perturbed conditions via online non-negative matrix factorization guided by physiologically informed sparseness constraints in lieu of explicit data labelling.

METHODS

UAM was tested in a set of virtual target reaching tasks completed by able-bodied and amputee subjects. Tests were conducted under normative and electrode perturbed conditions to gauge control robustness with comparisons to non-adaptive and supervised adaptive myocontrol schemes. Furthermore, UAM was used to interface an amputee with a multi-functional powered hand prosthesis during standardized Clothespin Relocation Tests, also conducted in normative and perturbed conditions.

RESULTS

In virtual tests, UAM effectively mitigated performance degradation caused by electrode displacement, affording greater resilience over an existing supervised adaptive system for amputee subjects. Induced electrode shifts also had negligible effect on the real world control performance of UAM with consistent completion times (23.91±1.33 s) achieved across Clothespin Relocation Tests in the normative and electrode perturbed conditions.

CONCLUSION

UAM affords comparable robustness improvements to existing supervised adaptive myocontrol interfaces whilst providing additional practical advantages for clinical deployment.

SIGNIFICANCE

The proposed system uniquely incorporates neuromuscular control principles with unsupervised online learning methods and presents a working example of a freely co-adaptive bionic interface.

Functional Complex Networks Based on Operational Architectonics: Application on Electroencephalography-Brain-computer Interface for Imagined Speech

A new method for analyzing brain complex dynamics and states is presented. This method constructs functional brain graphs and is comprised of two pylons: (a) Operational architectonics (OA) concept of brain and mind functioning. (b) Network neuroscience. In particular, the algorithm utilizes OA framework for a non-parametric segmentation of EEGs, which leads to the identification of change points, namely abrupt jumps in EEG amplitude, called Rapid Transition Processes (RTPs). Subsequently, the time coordinates of RTPs are used for the generation of undirected weighted complex networks fulfilling a scale-free topology criterion, from which various network metrics of brain connectivity are estimated. These metrics form feature vectors, which can be used in machine learning algorithms for classification and/or prediction. The method is tested in classification problems on an EEG-based BCI data set, acquired from individuals during imagery pronunciation tasks of various words/vowels. The classification results, based on a Naïve Bayes classifier, show that the overall accuracies were found to be above chance level in all tested cases. This method was also compared with other state-of-the-art computational approaches commonly used for functional network generation, exhibiting competitive performance. The method can be useful to neuroscientists wishing to enhance their repository of brain research algorithms.

A neurophysiological approach to spatial filter selection for adaptive brain–computer interfaces

Objective. The common spatial patterns (CSP) algorithm is an effective method to extract discriminatory features from electroencephalography (EEG) to be used by a brain–computer interface (BCI). However, informed selection of CSP filters typically requires oversight from a BCI expert to accept or reject filters based on the neurophysiological plausibility of their activation patterns. Our goal was to identify, analyze and automatically classify prototypical CSP patterns to enhance the prediction of motor imagery states in a BCI. Approach. A data-driven approach that used four publicly available EEG datasets was adopted. Cluster analysis revealed recurring, visually similar CSP patterns and a convolutional neural network was developed to distinguish between established CSP pattern classes. Furthermore, adaptive spatial filtering schemes that utilize the categorization of CSP patterns were proposed and evaluated. Main results. Classes of common neurophysiologically probable and improbable CSP patterns were established. Analysis of the relationship between these categories of CSP patterns and classification performance revealed discarding neurophysiologically improbable filters can decrease decoder performance. Further analysis revealed that the spatial orientation of EEG modulations can evolve over time, and that the features extracted from the original CSP filters can become inseparable. Importantly, it was shown through a novel adaptive CSP technique that adaptation in response to these emerging patterns can restore feature separability. Significance. These findings highlight the importance of considering and reporting on spatial filter activation patterns in both online and offline studies. They also emphasize to researchers in the field the importance of spatial filter adaptation in BCI decoder design, particularly for online studies with a focus on training users to develop stable and suitable brain patterns.

Channel Selection for Optimal EEG Measurement in Motor Imagery-Based Brain-Computer Interfaces

A method for selecting electroencephalographic (EEG) signals in motor imagery-based brain-computer interfaces (MI-BCI) is proposed for enhancing the online interoperability and portability of BCI systems, as well as user comfort. The attempt is also to reduce variability and noise of MI-BCI, which could be affected by a large number of EEG channels. The relation between selected channels and MI-BCI performance is therefore analyzed. The proposed method is able to select acquisition channels common to all subjects, while achieving a performance compatible with the use of all the channels. Results are reported with reference to a standard benchmark dataset, the BCI competition IV dataset 2a. They prove that a performance compatible with the best state-of-the-art approaches can be achieved, while adopting a significantly smaller number of channels, both in two and in four tasks classification. In particular, classification accuracy is about 77-83% in binary classification with down to 6 EEG channels, and above 60% for the four-classes case when 10 channels are employed. This gives a contribution in optimizing the EEG measurement while developing non-invasive and wearable MI-based brain-computer interfaces.

Vocal Imagery vs Intention: Viability of Vocal-Based EEG-BCI Paradigms

The viability of electroencephalogram (EEG) based vocal imagery (VIm) and vocal intention (VInt) Brain-Computer Interface (BCI) systems has been investigated in this study. Four different types of experimental tasks related to humming has been designed and exploited here. They are: (i) non-task specific (NTS), (ii) motor task (MT), (iii) VIm task, and (iv) VInt task. EEG signals from seventeen participants for each of these tasks were recorded from 16 electrode locations on the scalp and its features were extracted and analysed using common spatial pattern (CSP) filter. These features were subsequently fed into a support vector machine (SVM) classifier for classification. This analysis aimed to perform a binary classification, predicting whether the subject was performing one task or the other. Results from an extensive analysis showed a mean classification accuracy of 88.9% for VIm task and 91.1% for VInt task. This study clearly shows that VIm can be classified with ease and is a viable paradigm to integrate in BCIs. Such systems are not only useful for people with speech problems, but in general for people who use BCI systems to help them out in their everyday life, giving them another dimension of system control.