Functional disconnection of associative cortical areas predicts performance during BCI training

Intentional binding for noninvasive BCI control

Objective. Noninvasive brain-computer interfaces (BCIs) allow to interact with the external environment by naturally bypassing the musculoskeletal system. Making BCIs efficient and accurate is paramount to improve the reliability of real-life and clinical applications, from open-loop device control to closed-loop neurorehabilitation.Approach. By promoting sense of agency and embodiment, realistic setups including multimodal channels of communication, such as eye-gaze, and robotic prostheses aim to improve BCI performance. However, how the mental imagery command should be integrated in those hybrid systems so as to ensure the best interaction is still poorly understood. To address this question, we performed a hybrid EEG-based BCI training involving healthy volunteers enrolled in a reach-and-grasp action operated by a robotic arm.Main results. Showed that the hand grasping motor imagery timing significantly affects the BCI accuracy evolution as well as the spatiotemporal brain dynamics. Larger accuracy improvement was obtained when motor imagery is performed just after the robot reaching, as compared to before or during the movement. The proximity with the subsequent robot grasping favored intentional binding, led to stronger motor-related brain activity, and primed the ability of sensorimotor areas to integrate information from regions implicated in higher-order cognitive functions.Significance. Taken together, these findings provided fresh evidence about the effects of intentional binding on human behavior and cortical network dynamics that can be exploited to design a new generation of efficient brain-machine interfaces.

EEG decoding with spatiotemporal convolutional neural network for visualization and closed-loop control of sensorimotor activities: A simultaneous EEG-fMRI study

Closed-loop neurofeedback training utilizes neural signals such as scalp electroencephalograms (EEG) to manipulate specific neural activities and the associated behavioral performance. A spatiotemporal filter for high-density whole-head scalp EEG using a convolutional neural network can overcome the ambiguity of the signaling source because each EEG signal includes information on the remote regions. We simultaneously acquired EEG and functional magnetic resonance images in humans during the brain-computer interface (BCI) based neurofeedback training and compared the reconstructed and modeled hemodynamic responses of the sensorimotor network. Filters constructed with a convolutional neural network captured activities in the targeted network with spatial precision and specificity superior to those of the EEG signals preprocessed with standard pipelines used in BCI-based neurofeedback paradigms. The middle layers of the trained model were examined to characterize the neuronal oscillatory features that contributed to the reconstruction. Analysis of the layers for spatial convolution revealed the contribution of distributed cortical circuitries to reconstruction, including the frontoparietal and sensorimotor areas, and those of temporal convolution layers that successfully reconstructed the hemodynamic response function. Employing a spatiotemporal filter and leveraging the electrophysiological signatures of the sensorimotor excitability identified in our middle layer analysis would contribute to the development of a further effective neurofeedback intervention.

Measuring neuronal avalanches to inform brain-computer interfaces

Large-scale interactions among multiple brain regions manifest as bursts of activations called neuronal avalanches, which reconfigure according to the task at hand and, hence, might constitute natural candidates to design brain-computer interfaces (BCIs). To test this hypothesis, we used source-reconstructed magneto/electroencephalography during resting state and a motor imagery task performed within a BCI protocol. To track the probability that an avalanche would spread across any two regions, we built an avalanche transition matrix (ATM) and demonstrated that the edges whose transition probabilities significantly differed between conditions hinged selectively on premotor regions in all subjects. Furthermore, we showed that the topology of the ATMs allows task-decoding above the current gold standard. Hence, our results suggest that neuronal avalanches might capture interpretable differences between tasks that can be used to inform brain-computer interfaces.

Examining Neural Connectivity in Schizophrenia Using Task-Based EEG: A Graph Theory Approach

Schizophrenia (SZ) is a complex disorder characterized by a range of symptoms and behaviors that have significant consequences for individuals, families, and society in general. Electroencephalography (EEG) is a valuable tool for understanding the neural dynamics and functional abnormalities associated with schizophrenia. Research studies utilizing EEG have identified specific patterns of brain activity in individuals diagnosed with schizophrenia that may reflect disturbances in neural synchronization and information processing in cortical circuits. Considering the temporal dynamics of functional connectivity provides a more comprehensive understanding of brain networks' organization and how they change during different cognitive states. This temporal perspective would enhance our understanding of the underlying mechanisms of schizophrenia. In the present study, we will use measures based on graph theory to obtain dynamic and static indicators in order to evaluate differences in the functional connectivity of individuals diagnosed with SZ and healthy controls using an ecologically valid task. At the static level, patients showed alterations in their ability to segregate information, particularly in the default mode network (DMN). As for dynamic measures, patients showed reduced values in most metrics (segregation, integration, centrality, and resilience), reflecting a reduced number of dynamic states of brain networks. Our results show the utility of combining static and dynamic indicators of functional connectivity from EEG sensors.

High-density scalp electroencephalogram dataset during sensorimotor rhythm-based brain-computer interfacing

Real-time functional imaging of human neural activity and its closed-loop feedback enable voluntary control of targeted brain regions. In particular, a brain-computer interface (BCI), a direct bridge of neural activities and machine actuation is one promising clinical application of neurofeedback. Although a variety of studies reported successful self-regulation of motor cortical activities probed by scalp electroencephalogram (EEG), it remains unclear how neurophysiological, experimental conditions or BCI designs influence variability in BCI learning. Here, we provide the EEG data during using BCIs based on sensorimotor rhythm (SMR), consisting of 4 separate datasets. All EEG data were acquired with a high-density scalp EEG setup containing 128 channels covering the whole head. All participants were instructed to perform motor imagery of right-hand movement as the strategy to control BCIs based on the task-related power attenuation of SMR magnitude, that is event-related desynchronization. This dataset would allow researchers to explore the potential source of variability in BCI learning efficiency and facilitate follow-up studies to test the explicit hypotheses explored by the dataset.

De Novo Brain-Computer Interfacing Deforms Manifold of Populational Neural Activity Patterns in Human Cerebral Cortex

Human brains are capable of modulating innate activities to adapt to novel environments and tasks; for sensorimotor neural system this means acquisition of a rich repertoire of activity patterns that improve behavioral performance. To directly map the process of acquiring the neural repertoire during tasks onto performance improvement, we analyzed net neural populational activity during the learning of its voluntary modulation by brain-computer interface (BCI) operation in female and male humans. The recorded whole-head high-density scalp electroencephalograms (EEGs) were subjected to dimensionality reduction algorithm to capture changes in cortical activity patterns represented by the synchronization of neuronal oscillations during adaptation. Although the preserved variance of targeted features in the reduced dimensions was 20%, we found systematic interactions between the activity patterns and BCI classifiers that detected motor attempt; the neural manifold derived in the embedded space was stretched along with motor-related features of EEG by model-based fixed classifiers but not with adaptive classifiers that were constantly recalibrated to user activity. Moreover, the manifold was deformed to be orthogonal to the boundary by de novo classifiers with a fixed decision boundary based on biologically unnatural features. Collectively, the flexibility of human cortical signaling patterns (i.e., neural plasticity) is only induced by operation of a BCI whose classifier required fixed activities, and the adaptation could be induced even the requirement is not consistent with biologically natural responses. These principles of neural adaptation at a macroscopic level may underlie the ability of humans to learn wide-ranging behavioral repertoires and adapt to novel environments.

An Effective Fusing Approach by Combining Connectivity Network Pattern and Temporal-Spatial Analysis for EEG-Based BCI Rehabilitation

Motor-modality-based brain computer interface (BCI) could promote the neural rehabilitation for stroke patients. Temporal-spatial analysis was commonly used for pattern recognition in this task. This paper introduced a novel connectivity network analysis for EEG-based feature selection. The network features of connectivity pattern not only captured the spatial activities responding to motor task, but also mined the interactive pattern among these cerebral regions. Furthermore, the effective combination between temporal-spatial analysis and network analysis was evaluated for improving the performance of BCI classification (81.7%). And the results demonstrated that it could raise the classification accuracies for most of patients (6 of 7 patients). This proposed method was meaningful for developing the effective BCI training program for stroke rehabilitation.

Anesthetic modulations dissociate neuroelectric characteristics between sensory-evoked and spontaneous activities across bilateral rat somatosensory cortical laminae

Spontaneous neural activity has been widely adopted to construct functional connectivity (FC) amongst distant brain regions. Although informative, the functional role and signaling mechanism of the resting state FC are not intuitive as those in stimulus/task-evoked activity. In order to bridge the gap, we investigated anesthetic modulation of both resting-state and sensory-evoked activities. We used two well-studied GABAergic anesthetics of varying dose (isoflurane: 0.5–2.0% and α-chloralose: 30 and 60 mg/kg∙h) and recorded changes in electrophysiology using a pair of laminar electrode arrays that encompass the entire depth of the bilateral somatosensory cortices (S1fl) in rats. Specifically, the study focused to describe how varying anesthesia conditions affect the resting state activities and resultant FC between bilateral hemispheres in comparison to those obtained by evoked responses. As results, isoflurane decreased the amplitude of evoked responses in a dose-dependent manner mostly due to the habituation of repetitive responses. However, α-chloralose rather intensified the amplitude without exhibiting habituation. No such diverging trend was observed for the spontaneous activity, in which both anesthetics increased the signal power. For α-chloralose, overall FC was similar to that obtained with the lowest dose of isoflurane at 0.5% while higher doses of isoflurane displayed increased FC. Interestingly, only α-chloralose elicited relatively much greater increases in the ipsi-stimulus evoked response (i.e., in S1fl ipsilateral to the stimulated forelimb) than those associated with the contra-stimulus response, suggesting enhanced neuronal excitability. Taken together, the findings demonstrate modulation of the FC profiles by anesthesia is highly non-linear, possibly with a distinct underlying mechanism that affects either resting state or evoked activities differently. Further, the current study warrants thorough investigation of the basal neuronal states prior to the interpretation of resting state FC and evoked activities for accurate understanding of neural signal processing and circuitry.

Functional connectivity ensemble method to enhance BCI performance (FUCONE)

OBJECTIVE

Relying on the idea that functional connectivity provides important insights on the underlying dynamic of neuronal interactions, we propose a novel framework that combines functional connectivity estimators and covariance-based pipelines to improve the classification of mental states, such as motor imagery.

METHODS

A Riemannian classifier is trained for each estimator and an ensemble classifier combines the decisions in each feature space. A thorough assessment of the functional connectivity estimators is provided and the best performing pipeline among those tested, called FUCONE, is evaluated on different conditions and datasets.

RESULTS

Using a meta-analysis to aggregate results across datasets, FUCONE performed significantly better than all state-of-the-art methods.

CONCLUSION

The performance gain is mostly imputable to the improved diversity of the feature spaces, increasing the robustness of the ensemble classifier with respect to the inter- and intra-subject variability.

SIGNIFICANCE

Our results offer new insights into the need to consider functional connectivity-based methods to improve the BCI performance.

The effect of visual and proprioceptive feedback on sensorimotor rhythms during BCI training

Brain–computer interfaces (BCI) can be designed with several feedback modalities. To promote appropriate brain plasticity in therapeutic applications, the feedback should guide the user to elicit the desired brain activity and preferably be similar to the imagined action. In this study, we employed magnetoencephalography (MEG) to measure neurophysiological changes in healthy subjects performing motor imagery (MI) -based BCI training with two different feedback modalities. The MI-BCI task used in this study lasted 40–60 min and involved imagery of right- or left-hand movements. 8 subjects performed the task with visual and 14 subjects with proprioceptive feedback. We analysed power changes across the session at multiple frequencies in the range of 4–40 Hz with a generalized linear model to find those frequencies at which the power increased significantly during training. In addition, the power increase was analysed for each gradiometer, separately for alpha (8–13 Hz), beta (14–30 Hz) and gamma (30–40 Hz) bands, to find channels showing significant linear power increase over the session. These analyses were applied during three different conditions: rest, preparation, and MI. Visual feedback enhanced the amplitude of mainly high beta and gamma bands (24–40 Hz) in all conditions in occipital and left temporal channels. During proprioceptive feedback, in contrast, power increased mainly in alpha and beta bands. The alpha-band enhancement was found in multiple parietal, occipital, and temporal channels in all conditions, whereas the beta-band increase occurred during rest and preparation mainly in the parieto-occipital region and during MI in the parietal channels above hand motor regions. Our results show that BCI training with proprioceptive feedback increases the power of sensorimotor rhythms in the motor cortex, whereas visual feedback causes mainly a gamma-band increase in the visual cortex. MI-BCIs should involve proprioceptive feedback to facilitate plasticity in the motor cortex.

Gymnasts' Ability to Modulate Sensorimotor Rhythms During Kinesthetic Motor Imagery of Sports Non-specific Movements Superior to Non-gymnasts

Previous psychological studies using questionnaires have consistently reported that athletes have superior motor imagery ability, both for sports-specific and for sports-non-specific movements. However, regarding motor imagery of sports-non-specific movements, no physiological studies have demonstrated differences in neural activity between athletes and non-athletes. The purpose of this study was to examine the differences in sensorimotor rhythms during kinesthetic motor imagery (KMI) of sports-non-specific movements between gymnasts and non-gymnasts. We selected gymnasts as an example population because they are likely to have particularly superior motor imagery ability due to frequent usage of motor imagery, including KMI as part of daily practice. Healthy young participants (16 gymnasts and 16 non-gymnasts) performed repeated motor execution and KMI of sports-non-specific movements (wrist dorsiflexion and shoulder abduction of the dominant hand). Scalp electroencephalogram (EEG) was recorded over the contralateral sensorimotor cortex. During motor execution and KMI, sensorimotor EEG power is known to decrease in the α- (8–15 Hz) and β-bands (16–35 Hz), referred to as event-related desynchronization (ERD). We calculated the maximal peak of ERD both in the α- (αERDmax) and β-bands (βERDmax) as a measure of changes in corticospinal excitability. αERDmax was significantly greater in gymnasts, who subjectively evaluated their KMI as being more vivid in the psychological questionnaire. On the other hand, βERDmax was greater in gymnasts only for shoulder abduction KMI. These findings suggest gymnasts' signature of flexibly modulating sensorimotor rhythms with no movements, which may be the basis of their superior ability of KMI for sports-non-specific movements.

Effects of Frontal Theta Rhythms in a Prior Resting State on the Subsequent Motor Imagery Brain-Computer Interface Performance

Dealing with subjects who are unable to attain a proper level of performance, that is, those with brain–computer interface (BCI) illiteracy or BCI inefficients, is still a major issue in human electroencephalography (EEG) BCI systems. The most suitable approach to address this issue is to analyze the EEG signals of individual subjects independently recorded before the main BCI tasks, to evaluate their performance on these tasks. This study mainly focused on non-linear analyses and deep learning techniques to investigate the significant relationship between the intrinsic characteristics of a prior idle resting state and the subsequent BCI performance. To achieve this main objective, a public EEG motor/movement imagery dataset that constituted two individual EEG signals recorded from an idle resting state and a motor imagery BCI task was used in this study. For the EEG processing in the prior resting state, spectral analysis but also non-linear analyses, such as sample entropy, permutation entropy, and recurrent quantification analyses (RQA), were performed to obtain individual groups of EEG features to represent intrinsic EEG characteristics in the subject. For the EEG signals in the BCI tasks, four individual decoding methods, as a filter-bank common spatial pattern-based classifier and three types of convolution neural network-based classifiers, quantified the subsequent BCI performance in the subject. Statistical linear regression and ANOVA with post hoc analyses verified the significant relationship between non-linear EEG features in the prior resting state and three types of BCI performance as low-, intermediate-, and high-performance groups that were statistically discriminated by the subsequent BCI performance. As a result, we found that the frontal theta rhythm ranging from 4 to 8 Hz during the eyes open condition was highly associated with the subsequent BCI performance. The RQA findings that higher determinism and lower mean recurrent time were mainly observed in higher-performance groups indicate that more regular and stable properties in the EEG signals over the frontal regions during the prior resting state would provide a critical clue to assess an individual BCI ability in the following motor imagery task.

Relative Power Correlates With the Decoding Performance of Motor Imagery Both Across Time and Subjects

One of the most significant challenges in the application of brain-computer interfaces (BCI) is the large performance variation, which often occurs over time or across users. Recent evidence suggests that the physiological states may explain this performance variation in BCI, however, the underlying neurophysiological mechanism is unclear. In this study, we conducted a seven-session motor-imagery (MI) experiment on 20 healthy subjects to investigate the neurophysiological mechanism on the performance variation. The classification accuracy was calculated offline by common spatial pattern (CSP) and support vector machine (SVM) algorithms to measure the MI performance of each subject and session. Relative Power (RP) values from different rhythms and task stages were used to reflect the physiological states and their correlation with the BCI performance was investigated. Results showed that the alpha band RP from the supplementary motor area (SMA) within a few seconds before MI was positively correlated with performance. Besides, the changes of RP between task and pre-task stage from theta, alpha, and gamma band were also found to be correlated with performance both across time and subjects. These findings reveal a neurophysiological manifestation of the performance variations, and would further provide a way to improve the BCI performance.

BCI learning induces core-periphery reorganization in M/EEG multiplex brain networks

Brain-computer interfaces (BCIs) constitute a promising tool for communication and control. However, mastering non-invasive closed-loop systems remains a learned skill that is difficult to develop for a non-negligible proportion of users. The involved learning process induces neural changes associated with a brain network reorganization that remains poorly understood. To address this inter-subject variability, we adopted a multilayer approach to integrate brain network properties from electroencephalographic (EEG) and magnetoencephalographic (MEG) data resulting from a four-session BCI training program followed by a group of healthy subjects. Our method gives access to the contribution of each layer to multilayer network that tends to be equal with time. We show that regardless the chosen modality, a progressive increase in the integration of somatosensory areas in the α band was paralleled by a decrease of the integration of visual processing and working memory areas in the β band. Notably, only brain network properties in multilayer network correlated with future BCI scores in the α2 band: positively in somatosensory and decision-making related areas and negatively in associative areas. Our findings cast new light on neural processes underlying BCI training. Integrating multimodal brain network properties provides new information that correlates with behavioral performance and could be considered as a potential marker of BCI learning.

Deep Neural Regression Prediction of Motor Imagery Skills Using EEG Functional Connectivity Indicators

Motor imaging (MI) induces recovery and neuroplasticity in neurophysical regulation. However, a non-negligible portion of users presents insufficient coordination skills of sensorimotor cortex control. Assessments of the relationship between wakefulness and tasks states are conducted to foster neurophysiological and mechanistic interpretation in MI-related applications. Thus, to understand the organization of information processing, measures of functional connectivity are used. Also, models of neural network regression prediction are becoming popular, These intend to reduce the need for extracting features manually. However, predicting MI practicing's neurophysiological inefficiency raises several problems, like enhancing network regression performance because of the overfitting risk. Here, to increase the prediction performance, we develop a deep network regression model that includes three procedures: leave-one-out cross-validation combined with Monte Carlo dropout layers, subject clustering of MI inefficiency, and transfer learning between neighboring runs. Validation is performed using functional connectivity predictors extracted from two electroencephalographic databases acquired in conditions close to real MI applications (150 users), resulting in a high prediction of pretraining desynchronization and initial training synchronization with adequate physiological interpretability.

Network-based brain computer interfaces: principles and applications

Brain-computer interfaces (BCIs) make possible to interact with the external environment by decoding the mental intention of individuals. BCIs can therefore be used to address basic neuroscience questions but also to unlock a variety of applications from exoskeleton control to neurofeedback (NFB) rehabilitation. In general, BCI usability critically depends on the ability to comprehensively characterize brain functioning and correctly identify the user's mental state. To this end, much of the efforts have focused on improving the classification algorithms taking into account localized brain activities as input features. Despite considerable improvement BCI performance is still unstable and, as a matter of fact, current features represent oversimplified descriptors of brain functioning. In the last decade, growing evidence has shown that the brain works as a networked system composed of multiple specialized and spatially distributed areas that dynamically integrate information. While more complex, looking at how remote brain regions functionally interact represents a grounded alternative to better describe brain functioning. Thanks to recent advances in network science, i.e. a modern field that draws on graph theory, statistical mechanics, data mining and inferential modelling, scientists have now powerful means to characterize complex brain networks derived from neuroimaging data. Notably, summary features can be extracted from these networks to quantitatively measure specific organizational properties across a variety of topological scales. In this topical review, we aim to provide the state-of-the-art supporting the development of a network theoretic approach as a promising tool for understanding BCIs and improve usability.

Regression Networks for Neurophysiological Indicator Evaluation in Practicing Motor Imagery Tasks

Motor Imagery (MI) promotes motor learning in activities, like developing professional motor skills, sports gestures, and patient rehabilitation. However, up to 30% of users may not develop enough coordination skills after training sessions because of inter and intra-subject variability. Here, we develop a data-driven estimator, termed Deep Regression Network (DRN), which jointly extracts and performs the regression analysis in order to assess the efficiency of the individual brain networks in practicing MI tasks. The proposed double-stage estimator initially learns a pool of deep patterns, extracted from the input data, in order to feed a neural regression model, allowing for infering the distinctiveness between subject assemblies having similar variability. The results, which were obtained on real-world MI data, prove that the DRN estimator fosters pre-training neural desynchronization and initial training synchronization to predict the bi-class accuracy response, thus providing a better understanding of the Brain-Computer Interface inefficiency of subjects.

Learning in brain-computer interface control evidenced by joint decomposition of brain and behavior

OBJECTIVE

Motor imagery-based brain-computer interfaces (BCIs) use an individual's ability to volitionally modulate localized brain activity, often as a therapy for motor dysfunction or to probe causal relations between brain activity and behavior. However, many individuals cannot learn to successfully modulate their brain activity, greatly limiting the efficacy of BCI for therapy and for basic scientific inquiry. Formal experiments designed to probe the nature of BCI learning have offered initial evidence that coherent activity across spatially distributed and functionally diverse cognitive systems is a hallmark of individuals who can successfully learn to control the BCI. However, little is known about how these distributed networks interact through time to support learning.

APPROACH

Here, we address this gap in knowledge by constructing and applying a multimodal network approach to decipher brain-behavior relations in motor imagery-based brain-computer interface learning using magnetoencephalography. Specifically, we employ a minimally constrained matrix decomposition method - non-negative matrix factorization - to simultaneously identify regularized, covarying subgraphs of functional connectivity, to assess their similarity to task performance, and to detect their time-varying expression.

MAIN RESULTS

We find that learning is marked by diffuse brain-behavior relations: good learners displayed many subgraphs whose temporal expression tracked performance. Individuals also displayed marked variation in the spatial properties of subgraphs such as the connectivity between the frontal lobe and the rest of the brain, and in the temporal properties of subgraphs such as the stage of learning at which they reached maximum expression. From these observations, we posit a conceptual model in which certain subgraphs support learning by modulating brain activity in sensors near regions important for sustaining attention. To test this model, we use tools that stipulate regional dynamics on a networked system (network control theory), and find that good learners display a single subgraph whose temporal expression tracked performance and whose architecture supports easy modulation of sensors located near brain regions important for attention.

SIGNIFICANCE

The nature of our contribution to the neuroscience of BCI learning is therefore both computational and theoretical; we first use a minimally-constrained, individual specific method of identifying mesoscale structure in dynamic brain activity to show how global connectivity and interactions between distributed networks supports BCI learning, and then we use a formal network model of control to lend theoretical support to the hypothesis that these identified subgraphs are well suited to modulate attention.