Real-time EEG feedback during simultaneous EEG–fMRI identifies the cortical signature of motor imagery

EEG-based motor imagery channel selection and classification using hybrid optimization and two-tier deep learning

Brain-computer interface (BCI) technology holds promise for individuals with profound motor impairments, offering the potential for communication and control. Motor imagery (MI)-based BCI systems are particularly relevant in this context. Despite their potential, achieving accurate and robust classification of MI tasks using electroencephalography (EEG) data remains a significant challenge. In this paper, we employed the Minimum Redundancy Maximum Relevance (MRMR) algorithm to optimize channel selection. Furthermore, we introduced a hybrid optimization approach that combines the War Strategy Optimization (WSO) and Chimp Optimization Algorithm (ChOA). This hybridization significantly enhances the classification model's overall performance and adaptability. A two-tier deep learning architecture is proposed for classification, consisting of a Convolutional Neural Network (CNN) and a modified Deep Neural Network (M-DNN). The CNN focuses on capturing temporal correlations within EEG data, while the M-DNN is designed to extract high-level spatial characteristics from selected EEG channels. Integrating optimal channel selection, hybrid optimization, and the two-tier deep learning methodology in our BCI framework presents an enhanced approach for precise and effective BCI control. Our model got 95.06% accuracy with high precision. This advancement has the potential to significantly impact neurorehabilitation and assistive technology applications, facilitating improved communication and control for individuals with motor impairments.

A delayed matching task-based study on action sequence of motor imagery

The way people imagine greatly affects performance of brain-computer interface (BCI) based on motion imagery (MI). Action sequence is a basic unit of imitation, learning, and memory for motor behavior. Whether it influences the MI-BCI is unknown, and how to manifest this influence is difficult since the MI is a spontaneous brain activity. To investigate the influence of the action sequence, this study proposes a novel paradigm named action sequences observing and delayed matching task to use images and videos to guide people to observe, match and reinforce the memory of sequence. Seven subjects' ERPs and MI performance are analyzed under four different levels of complexities or orders of the sequence. Results demonstrated that the action sequence in terms of complexity and sequence order significantly affects the MI. The complex action in positive order obtains stronger ERD/ERS and more pronounced MI feature distributions, and yields an MI classification accuracy that is 12.3% higher than complex action in negative order (p < 0.05). In addition, the ERP amplitudes derived from the supplementary motor area show a positive correlation to the MI. This study demonstrates a new perspective of improving imagery in the MI-BCI by considering the complexity and order of the action sequences, and provides a novel index for manifesting the MI performance by ERP.

Neuroimaging of motor recovery after ischemic stroke - functional reorganization of motor network

The long-term motor outcome of acute stroke patients may be correlated to the reorganization of brain motor network. Abundant neuroimaging studies contribute to understand the pathological changes and recovery of motor networks after stroke. In this review, we summarized how current neuroimaging studies have increased understanding of reorganization and plasticity in post stroke motor recovery. Firstly, we discussed the changes in the motor network over time during the motor-activation and resting states, as well as the overall functional integration trend of the motor network. These studies indicate that the motor network undergoes dynamic bilateral hemispheric functional reorganization, as well as a trend towards network randomization. In the second part, we summarized the current study progress in the application of neuroimaging technology to early predict the post-stroke motor outcome. In the third part, we discuss the neuroimaging techniques commonly used in the post-stroke recovery. These methods provide direct or indirect visualization patterns to understand the neural mechanisms of post-stroke motor recovery, opening up new avenues for studying spontaneous and treatment-induced recovery and plasticity after stroke.

Attentional ERPs in consumers of smoked and insufflated cocaine associated with neuropsychological performance

BACKGROUND

Cocaine consumption is associated with reduced attentional event-related potentials (ERPs), namely P3a and P3b, indicating bottom-up and top-down deficits respectively. At cognitive level, these impairments are larger for faster routes of administration (e.g., smoked cocaine [SC]) than slower routes (e.g., insufflated cocaine [IC]). Here we assess these ERPs considering the route of cocaine administration. We hypothesized that SC dependent (SCD) would exhibit reduced amplitude of the P3a, while both SCD and IC dependent (ICD) would show reduced amplitude of the P3b.

METHODS

We examined 25 SCD, 22 ICD matched by poly-consumption profiles, and 25 controls matched by demographic variables. We combined EEG data from the Global-Local task with behavioral data from attentional cognitive tasks.

RESULTS

At the behavioral level, SCD exhibited attentional deficits in both bottom-up and top-down processes, while ICD only showed a tendency for top-down deficits. The amplitude of P3a and P3b was lower in Users groups. We observed subtle route-based differences, with larger differences in the P3a for SCD and in the P3b for ICD. Neurophysiological and behavioral data converged, with the P3a associated to bottom-up performance and P3b to top-down.

CONCLUSIONS

Different routes of administration lead to distinct attentional neurocognitive profiles. Specifically, SCD showed greater attentional impairment, mainly at bottom-up/P3a, while ICD showed a trend of top-down/P3b deficits. These findings emphasize the crucial role of considering the route of administration in both clinical and research settings and support the use of attentional ERPs as valid measures for assessing attentional deficits in substance Dependence.

Enhancing Brain-Computer Interface Performance by Incorporating Brain-to-Brain Coupling

Human cooperation relies on key features of social interaction in order to reach desirable outcomes. Similarly, human-robot interaction may benefit from integration with human-human interaction factors. In this paper, we aim to investigate brain-to-brain coupling during motor imagery (MI)-based brain-computer interface (BCI) training using eye-contact and hand-touch interaction. Twelve pairs of friends (experimental group) and 10 pairs of strangers (control group) were recruited for MI-based BCI tests concurrent with electroencephalography (EEG) hyperscanning. Event-related desynchronization (ERD) was estimated to measure cortical activation, and interbrain functional connectivity was assessed using multilevel statistical analysis. Furthermore, we compared BCI classification performance under different social interaction conditions. In the experimental group, greater ERD was found around the contralateral sensorimotor cortex under social interaction conditions compared with MI without any social interaction. Notably, EEG channels with decreased power were mainly distributed around the frontal, central, and occipital regions. A significant increase in interbrain coupling was also found under social interaction conditions. BCI decoding accuracies were significantly improved in the eye contact condition and eye and hand contact condition compared with the no-interaction condition. However, for the strangers' group, no positive effects were observed in comparisons of cortical activations between interaction and no-interaction conditions. These findings indicate that social interaction can improve the neural synchronization between familiar partners with enhanced brain activations and brain-to-brain coupling. This study may provide a novel method for enhancing MI-based BCI performance in conjunction with neural synchronization between users.

Noise-Factorized Disentangled Representation Learning for Generalizable Motor Imagery EEG Classification

Motor Imagery (MI) Electroencephalography (EEG) is one of the most common Brain-Computer Interface (BCI) paradigms that has been widely used in neural rehabilitation and gaming. Although considerable research efforts have been dedicated to developing MI EEG classification algorithms, they are mostly limited in handling scenarios where the training and testing data are not from the same subject or session. Such poor generalization capability significantly limits the realization of BCI in real-world applications. In this paper, we proposed a novel framework to disentangle the representation of raw EEG data into three components, subject/session-specific, MI-task-specific, and random noises, so that the subject/session-specific feature extends the generalization capability of the system. This is realized by a joint discriminative and generative framework, supported by a series of fundamental training losses and training strategies. We evaluated our framework on three public MI EEG datasets, and detailed experimental results show that our method can achieve superior performance by a large margin compared to current state-of-the-art benchmark algorithms.

Towards user-centric BCI design: Markov chain-based user assessment for mental imagery EEG-BCIs

Objective.While electroencephalography (EEG)-based brain-computer interfaces (BCIs) have many potential clinical applications, their use is impeded by poor performance for many users. To improve BCI performance, either via enhanced signal processing or user training, it is critical to understand and describe each user's ability to perform mental control tasks and produce discernible EEG patterns. While classification accuracy has predominantly been used to assess user performance, limitations and criticisms of this approach have emerged, thus prompting the need to develop novel user assessment approaches with greater descriptive capability. Here, we propose a combination of unsupervised clustering and Markov chain models to assess and describe user skill.Approach.Using unsupervisedK-means clustering, we segmented the EEG signal space into regions representing pattern states that users could produce. A user's movement through these pattern states while performing different tasks was modeled using Markov chains. Finally, using the steady-state distributions and entropy rates of the Markov chains, we proposed two metricstaskDistinctandrelativeTaskInconsistencyto assess, respectively, a user's ability to (i) produce distinct task-specific patterns for each mental task and (ii) maintain consistent patterns during individual tasks.Main results.Analysis of data from 14 adolescents using a three-class BCI revealed significant correlations between thetaskDistinctandrelativeTaskInconsistencymetrics and classification F1 score. Moreover, analysis of the pattern states and Markov chain models yielded descriptive information regarding user performance not immediately apparent from classification accuracy.Significance.Our proposed user assessment method can be used in concert with classifier-based analysis to further understand the extent to which users produce task-specific, time-evolving EEG patterns. In turn, this information could be used to enhance user training or classifier design.

Predictors of neurofeedback treatment outcome in binge-eating disorder: An exploratory study

OBJECTIVE

Knowledge on predictors for treatment response to psychotherapy in binge-eating disorder (BED) is mixed and not yet available for increasingly popular neurofeedback (NF) treatment targeting self-regulation of aberrant brain activity. This study examined eating disorder- and psychopathology-related predictors for NF treatment success in BED.

METHOD

Patients with BED (N = 78) were randomized to 12 sessions of real-time functional near-infrared spectroscopy (rtfNIRS)-NF, targeting individual prefrontal cortex signal up-regulation, electroencephalography (EEG)-NF, targeting down-regulation of fronto-central beta activity, or waitlist (WL). The few studies assessing predictors for clinical outcomes after NF and evidenced predictors for psychotherapy guided the selection of baseline eating disorder-related predictors, including objective binge-eating (OBE) frequency, eating disorder psychopathology (EDP), food cravings, and body mass index (BMI), and general psychopathology-related predictors, including depressive and anxiety symptoms, impulsivity, emotion dysregulation, and self-efficacy. These questionnaire-based or objectively assessed (BMI) predictors were regressed on outcomes OBE frequency and EDP as key features of BED at post-treatment (t1) and 6-month follow-up (t2) in preregistered generalized mixed models (https://osf.io/4aktp).

RESULTS

Higher EDP, food cravings, and BMI predicted worse outcomes across all groups at t1 and t2. General psychopathology-related predictors did not predict outcomes at t1 and t2. Explorative analyses indicated that lower OBE frequency and higher self-efficacy predicted lower OBE frequency, and lower EDP predicted lower EDP after the waiting period in WL.

DISCUSSION

Consistent with findings for psychotherapy, higher eating disorder-related predictors were associated with higher EDP and OBE frequency. The specificity of psychopathological predictors for NF treatment success warrants further examination.

PUBLIC SIGNIFICANCE

This exploratory study firstly assessed eating disorder- and psychopathology-related predictors for neurofeedback treatment outcome in binge-eating disorder and overweight. Findings showed an association between higher eating disorder symptoms and worse neurofeedback outcomes, indicating special needs to be considered in neurofeedback treatment for patients with a higher binge-eating disorder symptom burden. In general, outcomes and assignment to neurofeedback treatment may be improved upon consideration of baseline psychological variables.

Two is better? combining EEG and fMRI for BCI and neurofeedback: a systematic review

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) are two commonly used non-invasive techniques for measuring brain activity in neuroscience and brain-computer interfaces (BCI).Objective. In this review, we focus on the use of EEG and fMRI in neurofeedback (NF) and discuss the challenges of combining the two modalities to improve understanding of brain activity and achieve more effective clinical outcomes. Advanced technologies have been developed to simultaneously record EEG and fMRI signals to provide a better understanding of the relationship between the two modalities. However, the complexity of brain processes and the heterogeneous nature of EEG and fMRI present challenges in extracting useful information from the combined data.Approach. We will survey existing EEG-fMRI combinations and recent studies that exploit EEG-fMRI in NF, highlighting the experimental and technical challenges.Main results. We made a classification of the different combination of EEG-fMRI for NF, we provide a review of multimodal analysis methods for EEG-fMRI features. We also survey the current state of research on EEG-fMRI in the different existing NF paradigms. Finally, we also identify some of the remaining challenges in this field.Significance. By exploring EEG-fMRI combinations in NF, we are advancing our knowledge of brain function and its applications in clinical settings. As such, this review serves as a valuable resource for researchers, clinicians, and engineers working in the field of neural engineering and rehabilitation, highlighting the promising future of EEG-fMRI-based NF.

Favoring the cognitive-motor process in the closed-loop of BCI mediated post stroke motor function recovery: challenges and approaches

The brain-computer interface (BCI)-mediated rehabilitation is emerging as a solution to restore motor skills in paretic patients after stroke. In the human brain, cortical motor neurons not only fire when actions are carried out but are also activated in a wired manner through many cognitive processes related to movement such as imagining, perceiving, and observing the actions. Moreover, the recruitment of motor cortexes can usually be regulated by environmental conditions, forming a closed-loop through neurofeedback. However, this cognitive-motor control loop is often interrupted by the impairment of stroke. The requirement to bridge the stroke-induced gap in the motor control loop is promoting the evolution of the BCI-based motor rehabilitation system and, notably posing many challenges regarding the disease-specific process of post stroke motor function recovery. This review aimed to map the current literature surrounding the new progress in BCI-mediated post stroke motor function recovery involved with cognitive aspect, particularly in how it refired and rewired the neural circuit of motor control through motor learning along with the BCI-centric closed-loop.

Neural modulation enhancement using connectivity-based EEG neurofeedback with simultaneous fMRI for emotion regulation

Emotion regulation plays a key role in human behavior and overall well-being. Neurofeedback is a non-invasive self-brain training technique used for emotion regulation to enhance brain function and treatment of mental disorders through behavioral changes. Previous neurofeedback research often focused on using activity from a single brain region as measured by fMRI or power from one or two EEG electrodes. In a new study, we employed connectivity-based EEG neurofeedback through recalling positive autobiographical memories and simultaneous fMRI to upregulate positive emotion. In our novel approach, the feedback was determined by the coherence of EEG electrodes rather than the power of one or two electrodes. We compared the efficiency of this connectivity-based neurofeedback to traditional activity-based neurofeedback through multiple experiments. The results showed that connectivity-based neurofeedback effectively improved BOLD signal change and connectivity in key emotion regulation regions such as the amygdala, thalamus, and insula, and increased EEG frontal asymmetry, which is a biomarker for emotion regulation and treatment of mental disorders such as PTSD, anxiety, and depression and coherence among EEG channels. The psychometric evaluations conducted both before and after the neurofeedback experiments revealed that participants demonstrated improvements in enhancing positive emotions and reducing negative emotions when utilizing connectivity-based neurofeedback, as compared to traditional activity-based and sham neurofeedback approaches. These findings suggest that connectivity-based neurofeedback may be a superior method for regulating emotions and could be a useful alternative therapy for mental disorders, providing individuals with greater control over their brain and mental functions.

High-density scalp electroencephalogram dataset during sensorimotor rhythm-based brain-computer interfacing

Real-time functional imaging of human neural activity and its closed-loop feedback enable voluntary control of targeted brain regions. In particular, a brain-computer interface (BCI), a direct bridge of neural activities and machine actuation is one promising clinical application of neurofeedback. Although a variety of studies reported successful self-regulation of motor cortical activities probed by scalp electroencephalogram (EEG), it remains unclear how neurophysiological, experimental conditions or BCI designs influence variability in BCI learning. Here, we provide the EEG data during using BCIs based on sensorimotor rhythm (SMR), consisting of 4 separate datasets. All EEG data were acquired with a high-density scalp EEG setup containing 128 channels covering the whole head. All participants were instructed to perform motor imagery of right-hand movement as the strategy to control BCIs based on the task-related power attenuation of SMR magnitude, that is event-related desynchronization. This dataset would allow researchers to explore the potential source of variability in BCI learning efficiency and facilitate follow-up studies to test the explicit hypotheses explored by the dataset.

EEG-LLAMAS: A low-latency neurofeedback platform for artifact reduction in EEG-fMRI

Simultaneous EEG-fMRI is a powerful multimodal technique for imaging the brain, but its use in neurofeedback experiments has been limited by EEG noise caused by the MRI environment. Neurofeedback studies typically require analysis of EEG in real time, but EEG acquired inside the scanner is heavily contaminated with ballistocardiogram (BCG) artifact, a high-amplitude artifact locked to the cardiac cycle. Although techniques for removing BCG artifacts do exist, they are either not suited to real-time, low-latency applications, such as neurofeedback, or have limited efficacy. We propose and validate a new open-source artifact removal software called EEG-LLAMAS (Low Latency Artifact Mitigation Acquisition Software), which adapts and advances existing artifact removal techniques for low-latency experiments. We first used simulations to validate LLAMAS in data with known ground truth. We found that LLAMAS performed better than the best publicly-available real-time BCG removal technique, optimal basis sets (OBS), in terms of its ability to recover EEG waveforms, power spectra, and slow wave phase. To determine whether LLAMAS would be effective in practice, we then used it to conduct real-time EEG-fMRI recordings in healthy adults, using a steady state visual evoked potential (SSVEP) task. We found that LLAMAS was able to recover the SSVEP in real time, and recovered the power spectra collected outside the scanner better than OBS. We also measured the latency of LLAMAS during live recordings, and found that it introduced a lag of less than 50 ms on average. The low latency of LLAMAS, coupled with its improved artifact reduction, can thus be effectively used for EEG-fMRI neurofeedback. A limitation of the method is its use of a reference layer, a piece of EEG equipment which is not commercially available, but can be assembled in-house. This platform enables closed-loop experiments which previously would have been prohibitively difficult, such as those that target short-duration EEG events, and is shared openly with the neuroscience community.

Tailoring brain-machine interface rehabilitation training based on neural reorganization: towards personalized treatment for stroke patients

Electroencephalogram (EEG)-based brain-machine interface (BMI) has the potential to enhance rehabilitation training efficiency, but it still remains elusive regarding how to design BMI training for heterogeneous stroke patients with varied neural reorganization. Here, we hypothesize that tailoring BMI training according to different patterns of neural reorganization can contribute to a personalized rehabilitation trajectory. Thirteen stroke patients were recruited in a 2-week personalized BMI training experiment. Clinical and behavioral measurements, as well as cortical and muscular activities, were assessed before and after training. Following treatment, significant improvements were found in motor function assessment. Three types of brain activation patterns were identified during BMI tasks, namely, bilateral widespread activation, ipsilesional focusing activation, and contralesional recruitment activation. Patients with either ipsilesional dominance or contralesional dominance can achieve recovery through personalized BMI training. Results indicate that personalized BMI training tends to connect the potentially reorganized brain areas with event-contingent proprioceptive feedback. It can also be inferred that personalization plays an important role in establishing the sensorimotor loop in BMI training. With further understanding of neural rehabilitation mechanisms, personalized treatment strategy is a promising way to improve the rehabilitation efficacy and promote the clinical use of rehabilitation robots and other neurotechnologies.

Probing fMRI brain connectivity and activity changes during emotion regulation by EEG neurofeedback

Despite the existence of several emotion regulation studies using neurofeedback, interactions among a small number of regions were evaluated, and therefore, further investigation is needed to understand the interactions of the brain regions involved in emotion regulation. We implemented electroencephalography (EEG) neurofeedback with simultaneous functional magnetic resonance imaging (fMRI) using a modified happiness-inducing task through autobiographical memories to upregulate positive emotion. Then, an explorative analysis of whole brain regions was done to understand the effect of neurofeedback on brain activity and the interaction of whole brain regions involved in emotion regulation. The participants in the control and experimental groups were asked to do emotion regulation while viewing positive images of autobiographical memories and getting sham or real (based on alpha asymmetry) EEG neurofeedback, respectively. The proposed multimodal approach quantified the effects of EEG neurofeedback in changing EEG alpha power, fMRI blood oxygenation level-dependent (BOLD) activity of prefrontal, occipital, parietal, and limbic regions (up to 1.9% increase), and functional connectivity in/between prefrontal, parietal, limbic system, and insula in the experimental group. New connectivity links were identified by comparing the brain functional connectivity between experimental conditions (Upregulation and View blocks) and also by comparing the brain connectivity of the experimental and control groups. Psychometric assessments confirmed significant changes in positive and negative mood states in the experimental group by neurofeedback. Based on the exploratory analysis of activity and connectivity among all brain regions involved in emotion regions, we found significant BOLD and functional connectivity increases due to EEG neurofeedback in the experimental group, but no learning effect was observed in the control group. The results reveal several new connections among brain regions as a result of EEG neurofeedback which can be justified according to emotion regulation models and the role of those regions in emotion regulation and recalling positive autobiographical memories.

Thirty-minute motor imagery exercise aided by EEG sensorimotor rhythm neurofeedback enhances morphing of sensorimotor cortices: a double-blind sham-controlled study

Neurofeedback training using electroencephalogram (EEG)-based brain-computer interfaces (BCIs) combined with mental rehearsals of motor behavior has demonstrated successful self-regulation of motor cortical excitability. However, it remains unclear whether the acquisition of skills to voluntarily control neural excitability is accompanied by structural plasticity boosted by neurofeedback. Here, we sought short-term changes in cortical structures induced by 30 min of BCI-based neurofeedback training, which aimed at the regulation of sensorimotor rhythm (SMR) in scalp EEG. When participants performed kinesthetic motor imagery of right finger movement with online feedback of either event-related desynchronisation (ERD) of SMR magnitude from the contralateral sensorimotor cortex (SM1) or those from other participants (i.e. placebo), the learning rate of SMR-ERD control was significantly different. Although overlapped structural changes in gray matter volumes were found in both groups, significant differences revealed by group-by-group comparison were spatially different; whereas the veritable neurofeedback group exhibited sensorimotor area-specific changes, the placebo exhibited spatially distributed changes. The white matter change indicated a significant decrease in the corpus callosum in the verum group. Furthermore, the learning rate of SMR regulation was correlated with the volume changes in the ipsilateral SM1, suggesting the involvement of interhemispheric motor control circuitries in BCI control tasks.

Simultaneous real-time EEG-fMRI neurofeedback: A systematic review

Neurofeedback (NF) is a biofeedback technique that teaches individuals self-control of brain functions by measuring brain activations and providing an online feedback signal to modify emotional, cognitive, and behavioral functions. NF approaches typically rely on a single modality, such as electroencephalography (EEG-NF) or a brain imaging technique, such as functional magnetic resonance imaging (fMRI-NF). The introduction of simultaneous EEG-fMRI tools has opened up the possibility of combining the high temporal resolution of EEG with the high spatial resolution of fMRI, thereby increasing the accuracy of NF. However, only a few studies have actively combined both techniques. In this study, we conducted a systematic review of EEG-fMRI-NF studies (N = 17) to identify the potential and effectiveness of this non-invasive treatment for neurological conditions. The systematic review revealed a lack of homogeneity among the studies, including sample sizes, acquisition methods in terms of simultaneity of the two procedures (unimodal EEG-NF and fMRI-NF), therapeutic targets field, and the number of sessions. Indeed, because most studies are based on a single session of NF, it is difficult to draw any conclusions regarding the therapeutic efficacy of NF. Therefore, further research is needed to fully understand non-clinical and clinical potential of EEG-fMRI-NF.

Dynamic functional connectivity estimation for neurofeedback emotion regulation paradigm with simultaneous EEG-fMRI analysis

Joint Analysis of EEG and fMRI datasets can bring new insight into brain mechanisms. In this paper, we employed the recently introduced Correlated Coupled Tensor Matrix Factorization (CCMTF) method for analysis of the emotion regulation paradigm based on EEG frontal asymmetry neurofeedback in the alpha frequency band with simultaneous fMRI. CCMTF method assumes that the co-variations of the common dimension (temporal dimension) between EEG and fMRI are correlated and not necessarily identical. The results of the CCMTF method suggested that EEG and fMRI had similar covariations during the transition of brain activities from resting states to task (view and upregulation) states and these covariations followed an increasing trend. The fMRI shared spatial component showed activations in the limbic system, DLPFC, OFC, and VLPC regions, which were consistent with the previous studies and were linked to EEG frequency patterns in the range of 1–15 Hz with a correlation value close to 0.75. The estimated regions from the CCMTF method were then used as the candidate nodes for dynamic functional connectivity (dFC) analysis, in which the changes in connectivity from view to upregulation states were examined. The results of the dFC analysis were compared with a Normalized Mutual information (NMI) based approach in two different frequency ranges (1–15 and 15–40 Hz) as the NMI method was applied to the vectors of dFC nodes of EEG and fMRI data. The results of the two methods illustrated that the relation between EEG and fMRI datasets was mostly in the frequency range of 1–15 Hz. These relations were both in the brain activations and the dFCs between the two modalities. This paper suggests that the CCMTF method is a capable approach for extracting the shared information between EEG and fMRI data and can reveal new information about brain functions and their connectivity without solving the EEG inverse problem or analyzing different frequency bands.

Neurofeedback for the Education of Children with ADHD and Specific Learning Disorders: A Review

Neurofeedback (NF) is a type of biofeedback in which an individual’s brain activity is measured and presented to them to support self-regulation of ongoing brain oscillations and achieve specific behavioral and neurophysiological outcomes. NF training induces changes in neurophysiological circuits that are associated with behavioral changes. Recent evidence suggests that the NF technique can be used to train electrical brain activity and facilitate learning among children with learning disorders. Toward this aim, this review first presents a generalized model for NF systems, and then studies involving NF training for children with disorders such as dyslexia, attention-deficit/hyperactivity disorder (ADHD), and other specific learning disorders such as dyscalculia and dysgraphia are reviewed. The discussion elaborates on the potential for translational applications of NF in educational and learning settings with details. This review also addresses some issues concerning the role of NF in education, and it concludes with some solutions and future directions. In order to provide the best learning environment for children with ADHD and other learning disorders, it is critical to better understand the role of NF in educational settings. The review provides the potential challenges of the current systems to aid in highlighting the issues undermining the efficacy of current systems and identifying solutions to address them. The review focuses on the use of NF technology in education for the development of adaptive teaching methods and the best learning environment for children with learning disabilities.

Sources of independent mu components reveal different brain areas involved in motor imagery, motor execution, and movement observation

To answer the question of whether the same brain circuit(s) facilitates motor imagery (MI), motor execution (ME), and movement observation (MO), we conducted electroencephalography (EEG) experiment combining the three motor conditions in the same experimental runs. The EEG data were analyzed using two different independent component analysis (ICA) decomposition approaches: a single ICA decomposition on all EEG data combined and separate ICA decomposition on the EEG data obtained from the separate conditions. The results indicated that the separate ICA approach may provide a better fit to the EEG data obtained from the separate conditions to deliver specific independent right mu components with distinct topographies for each of the motor conditions. The topography of the MI condition covered the brain regions posterior to the central sulcus (P4 EEG channel); the ME condition covered the brain regions anterior to the central sulcus (C4 EEG channel), and the MO condition had broader coverage with the main activation in the premotor region (CP4 EEG channel). The source localization results also exhibited significant differences among the motor conditions. In addition, the result of single ICA decomposition resembled the result of separate ICA decomposition on the EEG data of ME with similar topographies and closely located EEG sources. This finding may further indicate that the result of single ICA decomposition may be dominated by the ME motor condition because it manifests higher data variance than the other two motor conditions.

Simultaneous electroencephalography-functional magnetic resonance imaging for assessment of human brain function

Electroencephalography (EEG) and functional Magnetic Resonance Imaging (MRI) have long been used as tools to examine brain activity. Since both methods are very sensitive to changes of synaptic activity, simultaneous recording of EEG and fMRI can provide both high temporal and spatial resolution. Therefore, the two modalities are now integrated into a hybrid tool, EEG-fMRI, which encapsulates the useful properties of the two. Among other benefits, EEG-fMRI can contribute to a better understanding of brain connectivity and networks. This review lays its focus on the methodologies applied in performing EEG-fMRI studies, namely techniques used for the recording of EEG inside the scanner, artifact removal, and statistical analysis of the fMRI signal. We will investigate simultaneous resting-state and task-based EEG-fMRI studies and discuss their clinical and technological perspectives. Moreover, it is established that the brain regions affected by a task-based neural activity might not be limited to the regions in which they have been initiated. Advanced methods can help reveal the regions responsible for or affected by a developed neural network. Therefore, we have also looked into studies related to characterization of structure and dynamics of brain networks. The reviewed literature suggests that EEG-fMRI can provide valuable complementary information about brain neural networks and functions.

Basal ganglia-cortical connectivity underlies self-regulation of brain oscillations in humans

Brain-computer interfaces provide an artificial link by which the brain can directly interact with the environment. To achieve fine brain-computer interface control, participants must modulate the patterns of the cortical oscillations generated from the motor and somatosensory cortices. However, it remains unclear how humans regulate cortical oscillations, the controllability of which substantially varies across individuals. Here, we performed simultaneous electroencephalography (to assess brain-computer interface control) and functional magnetic resonance imaging (to measure brain activity) in healthy participants. Self-regulation of cortical oscillations induced activity in the basal ganglia-cortical network and the neurofeedback control network. Successful self-regulation correlated with striatal activity in the basal ganglia-cortical network, through which patterns of cortical oscillations were likely modulated. Moreover, basal ganglia-cortical network and neurofeedback control network connectivity correlated with strong and weak self-regulation, respectively. The findings indicate that the basal ganglia-cortical network is important for self-regulation, the understanding of which should help advance brain-computer interface technology.

Simultaneous fMRI-EEG in 26 healthy participants indicate that the basal ganglia cortical network and the neurofeedback control network play different roles in self-regulation, providing further insight into the neural correlates for brain-machine interface control and feedback.

Allostatic-Interoceptive Overload in Frontotemporal Dementia

BACKGROUND

The predictive coding theory of allostatic-interoceptive load states that brain networks mediating autonomic regulation and interoceptive-exteroceptive balance regulate the internal milieu to anticipate future needs and environmental demands. These functions seem to be distinctly compromised in behavioral variant frontotemporal dementia (bvFTD), including alterations of the allostatic-interoceptive network (AIN). Here, we hypothesize that bvFTD is typified by an allostatic-interoceptive overload.

METHODS

We assessed resting-state heartbeat evoked potential (rsHEP) modulation as well as its behavioral and multimodal neuroimaging correlates in patients with bvFTD relative to healthy control subjects and patients with Alzheimer's disease (N = 94). We measured 1) resting-state electroencephalography (to assess the rsHEP, prompted by visceral inputs and modulated by internal body sensing), 2) associations between rsHEP and its neural generators (source location), 3) cognitive disturbances (cognitive state, executive functions, facial emotion recognition), 4) brain atrophy, and 5) resting-state functional magnetic resonance imaging functional connectivity (AIN vs. control networks).

RESULTS

Relative to healthy control subjects and patients with Alzheimer's disease, patients with bvFTD presented more negative rsHEP amplitudes with sources in critical hubs of the AIN (insula, amygdala, somatosensory cortex, hippocampus, anterior cingulate cortex). This exacerbated rsHEP modulation selectively predicted the patients' cognitive profile (including cognitive decline, executive dysfunction, and emotional impairments). In addition, increased rsHEP modulation in bvFTD was associated with decreased brain volume and connectivity of the AIN. Machine learning results confirmed AIN specificity in predicting the bvFTD group.

CONCLUSIONS

Altogether, these results suggest that bvFTD may be characterized by an allostatic-interoceptive overload manifested in ongoing electrophysiological markers, brain atrophy, functional networks, and cognition.

Using causal methods to map symptoms to brain circuits in neurodevelopment disorders: moving from identifying correlates to developing treatments

A wide variety of model systems and experimental techniques can provide insight into the structure and function of the human brain in typical development and in neurodevelopmental disorders. Unfortunately, this work, whether based on manipulation of animal models or observational and correlational methods in humans, has a high attrition rate in translating scientific discovery into practicable treatments and therapies for neurodevelopmental disorders.With new computational and neuromodulatory approaches to interrogating brain networks, opportunities exist for "bedside-to bedside-translation" with a potentially shorter path to therapeutic options. Specifically, methods like lesion network mapping can identify brain networks involved in the generation of complex symptomatology, both from acute onset lesion-related symptoms and from focal developmental anomalies. Traditional neuroimaging can examine the generalizability of these findings to idiopathic populations, while non-invasive neuromodulation techniques such as transcranial magnetic stimulation provide the ability to do targeted activation or inhibition of these specific brain regions and networks. In parallel, real-time functional MRI neurofeedback also allow for endogenous neuromodulation of specific targets that may be out of reach for transcranial exogenous methods.Discovery of novel neuroanatomical circuits for transdiagnostic symptoms and neuroimaging-based endophenotypes may now be feasible for neurodevelopmental disorders using data from cohorts with focal brain anomalies. These novel circuits, after validation in large-scale highly characterized research cohorts and tested prospectively using noninvasive neuromodulation and neurofeedback techniques, may represent a new pathway for symptom-based targeted therapy.

Sensorimotor Rhythm-Brain Computer Interface With Audio-Cue, Motor Observation and Multisensory Feedback for Upper-Limb Stroke Rehabilitation: A Controlled Study

Several studies have shown the positive clinical effect of brain computer interface (BCI) training for stroke rehabilitation. This study investigated the efficacy of the sensorimotor rhythm (SMR)-based BCI with audio-cue, motor observation and multisensory feedback for post-stroke rehabilitation. Furthermore, we discussed the interaction between training intensity and training duration in BCI training. Twenty-four stroke patients with severe upper limb (UL) motor deficits were randomly assigned to two groups: 2-week SMR-BCI training combined with conventional treatment (BCI Group, BG, n = 12) and 2-week conventional treatment without SMR-BCI intervention (Control Group, CG, n = 12). Motor function was measured using clinical measurement scales, including Fugl-Meyer Assessment-Upper Extremities (FMA-UE; primary outcome measure), Wolf Motor Functional Test (WMFT), and Modified Barthel Index (MBI), at baseline (Week 0), post-intervention (Week 2), and follow-up week (Week 4). EEG data from patients allocated to the BG was recorded at Week 0 and Week 2 and quantified by mu suppression means event-related desynchronization (ERD) in mu rhythm (8–12 Hz). All functional assessment scores (FMA-UE, WMFT, and MBI) significantly improved at Week 2 for both groups (p < 0.05). The BG had significantly higher FMA-UE and WMFT improvement at Week 4 compared to the CG. The mu suppression of bilateral hemisphere both had a positive trend with the motor function scores at Week 2. This study proposes a new effective SMR-BCI system and demonstrates that the SMR-BCI training with audio-cue, motor observation and multisensory feedback, together with conventional therapy may promote long-lasting UL motor improvement.

Clinical Trial Registration: [http://www.chictr.org.cn], identifier [ChiCTR2000041119].

Real-Time Detection and Feedback of Canonical Electroencephalogram Microstates: Validating a Neurofeedback System as a Function of Delay

Recent neurotechnology has developed various methods for neurofeedback (NF), in which participants observe their own neural activity to be regulated in an ideal direction. EEG-microstates (EEGms) are spatially featured states that can be regulated through NF training, given that they have recently been indicated as biomarkers for some disorders. The current study was conducted to develop an EEG-NF system for detecting “canonical 4 EEGms” in real time. There are four representative EEG states, regardless of the number of channels, preprocessing procedures, or participants. Accordingly, our 10 Hz NF system was implemented to detect them (msA, B, C, and D) and audio-visually inform participants of its detection. To validate the real-time effect of this system on participants’ performance, the NF was intentionally delayed for participants to prevent their cognitive control in learning. Our results suggest that the feedback effect was observed only under the no-delay condition. The number of Hits increased significantly from the baseline period and increased from the 1- or 20-s delay conditions. In addition, when the Hits were compared among the msABCD, each cognitive or perceptual function could be characterized, though the correspondence between each microstate and psychological ability might not be that simple. For example, msD should be generally task-positive and less affected by the inserted delay, whereas msC is more delay-sensitive. In this study, we developed and validated a new EEGms-NF system as a function of delay. Although the participants were naive to the inserted delay, the real-time NF successfully increased their Hit performance, even within a single-day experiment, although target specificity remains unclear. Future research should examine long-term training effects using this NF system.

5 Hz rTMS improves motor-imagery based BCI classification performance

Brain-computer interface (BCI) based rehabilitation has been proven a promising method facilitating motor recovery. Recognizing motor intention is crucial for realizing BCI rehabilitation training. Event-related desynchronization (ERD) is a kind of electroencephalogram (EEG) inherent characteristics associated with motor intention. However, due to brain deficits poststroke, some patients are not able to generate ERD, which discourages them to be involved in BCI rehabilitation training. To boost ERD during motor imagery (MI), this paper investigates the effects of high-frequency repetitive transcranial magnetic stimulation (rTMS) on BCI classification performance. Eleven subjects participated in this study. The experiment consisted of two conditions: rTMS + MI versus sham rTMS + MI, which were arranged on different days. MI tests with 64-channel EEG recording were arranged immediately before and after rTMS and sham rTMS. Time-frequency analysis were utilized to measure ERD changes. Common spatial pattern was used to extract features and linear discriminant analysis was used to calculate offline classification accuracies. Paired-sample t-test and Wilcoxon signed rank tests with post-hoc analysis were used to compare performance before and after stimulation. Statistically stronger ERD (-13.93±12.99%) was found after real rTMS compared with ERD (-5.71±21.25%) before real rTMS (p<0.05). Classification accuracy after real rTMS (70.71±10.32%) tended to be higher than that before real rTMS (66.50±8.48%) (p<0.1). However, no statistical differences were found after sham stimulation. This research provides an effective method in improving BCI performance by utilizing neural modulation.Clinical Relevance- This study offers a promising treatment for patients who cannot be recruited in BCI rehabilitation training due to poor BCI classification performance.

The impact of neurofeedback on effective connectivity networks in chronic stroke patients: an exploratory study

Objective.In this study, we assessed the impact of electroencephalography-functional magnetic resonance imaging (EEG-fMRI) neurofeedback (NF) on connectivity strength and direction in bilateral motor cortices in chronic stroke patients. Most of the studies using NF or brain computer interfaces for stroke rehabilitation have assessed treatment effects focusing on successful activation of targeted cortical regions. However, given the crucial role of brain network reorganization for stroke recovery, our broader aim was to assess connectivity changes after an NF training protocol targeting localized motor areas.Approach.We considered changes in fMRI connectivity after a multisession EEG-fMRI NF training targeting ipsilesional motor areas in nine stroke patients. We applied the dynamic causal modeling and parametric empirical Bayes frameworks for the estimation of effective connectivity changes. We considered a motor network including both ipsilesional and contralesional premotor, supplementary and primary motor areas.Main results.Our results indicate that NF upregulation of targeted areas (ipsilesional supplementary and primary motor areas) not only modulated activation patterns, but also had a more widespread impact on fMRI bilateral motor networks. In particular, inter-hemispheric connectivity between premotor and primary motor regions decreased, and ipsilesional self-inhibitory connections were reduced in strength, indicating an increase in activation during the NF motor task.Significance.To the best of our knowledge, this is the first work that investigates fMRI connectivity changes elicited by training of localized motor targets in stroke. Our results open new perspectives in the understanding of large-scale effects of NF training and the design of more effective NF strategies, based on the pathophysiology underlying stroke-induced deficits.

Multimodal mechanisms of human socially reinforced learning across neurodegenerative diseases

Social feedback can selectively enhance learning in diverse domains. Relevant neurocognitive mechanisms have been studied mainly in healthy persons, yielding correlational findings. Neurodegenerative lesion models, coupled with multimodal brain measures, can complement standard approaches by revealing direct multidimensional correlates of the phenomenon.

To this end, we assessed socially reinforced and non-socially reinforced learning in 40 healthy participants as well as persons with behavioural variant frontotemporal dementia (n = 21), Parkinson’s disease (n = 31) and Alzheimer’s disease (n = 20). These conditions are typified by predominant deficits in social cognition, feedback-based learning and associative learning, respectively, although all three domains may be partly compromised in the other conditions. We combined a validated behavioural task with ongoing EEG signatures of implicit learning (medial frontal negativity) and offline MRI measures (voxel-based morphometry).

In healthy participants, learning was facilitated by social feedback relative to non-social feedback. In comparison with controls, this effect was specifically impaired in behavioural variant frontotemporal dementia and Parkinson’s disease, while unspecific learning deficits (across social and non-social conditions) were observed in Alzheimer’s disease. EEG results showed increased medial frontal negativity in healthy controls during social feedback and learning. Such a modulation was selectively disrupted in behavioural variant frontotemporal dementia. Neuroanatomical results revealed extended temporo-parietal and fronto-limbic correlates of socially reinforced learning, with specific temporo-parietal associations in behavioural variant frontotemporal dementia and predominantly fronto-limbic regions in Alzheimer’s disease. In contrast, non-socially reinforced learning was consistently linked to medial temporal/hippocampal regions. No associations with cortical volume were found in Parkinson’s disease. Results are consistent with core social deficits in behavioural variant frontotemporal dementia, subtle disruptions in ongoing feedback-mechanisms and social processes in Parkinson’s disease and generalized learning alterations in Alzheimer’s disease. This multimodal approach highlights the impact of different neurodegenerative profiles on learning and social feedback.

Our findings inform a promising theoretical and clinical agenda in the fields of social learning, socially reinforced learning and neurodegeneration.

An end-to-end CNN with attentional mechanism applied to raw EEG in a BCI classification task

Objective.Motor-imagery (MI) classification base on electroencephalography (EEG) has been long studied in neuroscience and more recently widely used in healthcare applications such as mobile assistive robots and neurorehabilitation. In particular, EEG-based MI classification methods that rely on convolutional neural networks (CNNs) have achieved relatively high classification accuracy. However, naively training CNNs to classify raw EEG data from all channels, especially for high-density EEG, is computationally demanding and requires huge training sets. It often also introduces many irrelevant input features, making it difficult for the CNN to extract the informative ones. This problem is compounded by a dearth of training data, which is particularly acute for MI tasks, because these are cognitively demanding and thus fatigue inducing.Approach.To address these issues, we proposed an end-to-end CNN-based neural network with attentional mechanism together with different data augmentation (DA) techniques. We tested it on two benchmark MI datasets, brain-computer interface (BCI) competition IV 2a and 2b. In addition, we collected a new dataset, recorded using high-density EEG, and containing both MI and motor execution (ME) tasks, which we share with the community.Main results.Our proposed neural-network architecture outperformed all state-of-the-art methods that we found in the literature, with and without DA, reaching an average classification accuracy of 93.6% and 87.83% on BCI 2a and 2b, respectively. We also directly compare decoding of MI and ME tasks. Focusing on MI classification, we find optimal channel configurations and the best DA techniques as well as investigate combining data across participants and the role of transfer learning.Significance.Our proposed approach improves the classification accuracy for MI in the benchmark datasets. In addition, collecting our own dataset enables us to compare MI and ME and investigate various aspects of EEG decoding critical for neuroscience and BCI.

The Impact of Context on EEG Motor Imagery Neurofeedback and Related Motor Domains

Neurofeedback (NF) is a versatile non-invasive neuromodulation technique. In combination with motor imagery (MI), NF has considerable potential for enhancing motor performance or supplementing motor rehabilitation. However, not all users achieve reliable NF control. While research has focused on various brain signal properties and the optimisation of signal processing to solve this issue, the impact of context, i.e. the conditions in which NF motor tasks occur, is comparatively unknown. We review current research on the impact of context on MI NF and related motor domains. We identify long-term factors that act at the level of the individual or of the intervention, and short-term factors, with levels before/after and during a session. The reviewed literature indicates that context plays a significant role. We propose considering context factors as well as within-level and across-level interactions when studying MI NF.

Multi-scale neural decoding and analysis

Objective.

Complex spatiotemporal neural activity encodes rich information related to behavior and cognition. Conventional research has focused on neural activity acquired using one of many different measurement modalities, each of which provides useful but incomplete assessment of the neural code. Multi-modal techniques can overcome tradeoffs in the spatial and temporal resolution of a single modality to reveal deeper and more comprehensive understanding of system-level neural mechanisms. Uncovering multi-scale dynamics is essential for a mechanistic understanding of brain function and for harnessing neuroscientific insights to develop more effective clinical treatment.

Approach.

We discuss conventional methodologies used for characterizing neural activity at different scales and review contemporary examples of how these approaches have been combined. Then we present our case for integrating activity across multiple scales to benefit from the combined strengths of each approach and elucidate a more holistic understanding of neural processes.

Main results.

We examine various combinations of neural activity at different scales and analytical techniques that can be used to integrate or illuminate information across scales, as well the technologies that enable such exciting studies. We conclude with challenges facing future multi-scale studies, and a discussion of the power and potential of these approaches.

Significance.

This roadmap will lead the readers toward a broad range of multi-scale neural decoding techniques and their benefits over single-modality analyses. This Review article highlights the importance of multi-scale analyses for systematically interrogating complex spatiotemporal mechanisms underlying cognition and behavior.

When Is Simultaneous Recording Necessary? A Guide for Researchers Considering Combined EEG-fMRI

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) provide non-invasive measures of brain activity at varying spatial and temporal scales, offering different views on brain function for both clinical and experimental applications. Simultaneous recording of these measures attempts to maximize the respective strengths of each method, while compensating for their weaknesses. However, combined recording is not necessary to address all research questions of interest, and experiments may have greater statistical power to detect effects by maximizing the signal-to-noise ratio in separate recording sessions. While several existing papers discuss the reasons for or against combined recording, this article aims to synthesize these arguments into a flow chart of questions that researchers can consider when deciding whether to record EEG and fMRI separately or simultaneously. Given the potential advantages of simultaneous EEG-fMRI, the aim is to provide an initial overview of the most important concepts and to direct readers to relevant literature that will aid them in this decision.

Changes in EEG Brain Connectivity Caused by Short-Term BCI Neurofeedback-Rehabilitation Training: A Case Study

Background

In combined with neurofeedback, Motor Imagery (MI) based Brain-Computer Interface (BCI) has been an effective long-term treatment therapy for motor dysfunction caused by neurological injury in the brain (e.g., post-stroke hemiplegia). However, individual neurological differences have led to variability in the single sessions of rehabilitation training. Research on the impact of short training sessions on brain functioning patterns can help evaluate and standardize the short duration of rehabilitation training. In this paper, we use the electroencephalogram (EEG) signals to explore the brain patterns’ changes after a short-term rehabilitation training.

Materials and Methods

Using an EEG-BCI system, we analyzed the changes in short-term (about 1-h) MI training data with and without visual feedback, respectively. We first examined the EEG signal’s Mu band power’s attenuation caused by Event-Related Desynchronization (ERD). Then we use the EEG’s Event-Related Potentials (ERP) features to construct brain networks and evaluate the training from multiple perspectives: small-scale based on single nodes, medium-scale based on hemispheres, and large-scale based on all-brain.

Results

Results showed no significant difference in the ERD power attenuation estimation in both groups. But the neurofeedback group’s ERP brain network parameters had substantial changes and trend properties compared to the group without feedback. The neurofeedback group’s Mu band power’s attenuation increased but not significantly (fitting line slope = 0.2, t-test value p > 0.05) after the short-term MI training, while the non-feedback group occurred an insignificant decrease (fitting line slope = −0.4, t-test value p > 0.05). In the ERP-based brain network analysis, the neurofeedback group’s network parameters were attenuated in all scales significantly (t-test value: p < 0.01); while the non-feedback group’s most network parameters didn’t change significantly (t-test value: p > 0.05).

Conclusion

The MI-BCI training’s short-term effects does not show up in the ERD analysis significantly but can be detected by ERP-based network analysis significantly. Results inspire the efficient evaluation of short-term rehabilitation training and provide a useful reference for subsequent studies.

Interoception Primes Emotional Processing: Multimodal Evidence from Neurodegeneration

Recent frameworks in cognitive neuroscience and behavioral neurology underscore interoceptive priors as core modulators of negative emotions. However, the field lacks experimental designs manipulating the priming of emotions via interoception and exploring their multimodal signatures in neurodegenerative models. Here, we designed a novel task that involves interoceptive and control-exteroceptive priming conditions followed by post-interoception and post-exteroception facial emotion recognition (FER). We recruited 114 participants, including healthy controls (HCs) as well as patients with behavioral variant frontotemporal dementia (bvFTD), Parkinson's disease (PD), and Alzheimer's disease (AD). We measured online EEG modulations of the heart-evoked potential (HEP), and associations with both brain structural and resting-state functional connectivity patterns. Behaviorally, post-interoception negative FER was enhanced in HCs but selectively disrupted in bvFTD and PD, with AD presenting generalized disruptions across emotion types. Only bvFTD presented impaired interoceptive accuracy. Increased HEP modulations during post-interoception negative FER was observed in HCs and AD, but not in bvFTD or PD patients. Across all groups, post-interoception negative FER correlated with the volume of the insula and the ACC. Also, negative FER was associated with functional connectivity along the (a) salience network in the post-interoception condition, and along the (b) executive network in the post-exteroception condition. These patterns were selectively disrupted in bvFTD (a) and PD (b), respectively. Our approach underscores the multidimensional impact of interoception on emotion, while revealing a specific pathophysiological marker of bvFTD. These findings inform a promising theoretical and clinical agenda in the fields of nteroception, emotion, allostasis, and neurodegeneration.SIGNIFICANCE STATEMENT We examined whether and how emotions are primed by interoceptive states combining multimodal measures in healthy controls and neurodegenerative models. In controls, negative emotion recognition and ongoing HEP modulations were increased after interoception. These patterns were selectively disrupted in patients with atrophy across key interoceptive-emotional regions (e.g., the insula and the cingulate in frontotemporal dementia, frontostriatal networks in Parkinson's disease), whereas persons with Alzheimer's disease presented generalized emotional processing abnormalities with preserved interoceptive mechanisms. The integration of both domains was associated with the volume and connectivity (salience network) of canonical interoceptive-emotional hubs, critically involving the insula and the anterior cingulate. Our study reveals multimodal markers of interoceptive-emotional priming, laying the groundwork for new agendas in cognitive neuroscience and behavioral neurology.

Time-resolved estimation of strength of motor imagery representation by multivariate EEG decoding

Objective. Multivariate decoding enables access to information encoded in multiple brain activity features with high temporal resolution. However, whether the strength, of which this information is represented in the brain, can be extracted across time within single trials remains largely unexplored.Approach.In this study, we addressed this question by applying a support vector machine (SVM) to extract motor imagery (MI) representations, from electroencephalogram (EEG) data, and by performing time-resolved single-trial analyses of the multivariate decoding. EEG was recorded from a group of healthy participants during MI of opening and closing of the same hand.Main results.Cross-temporal decoding revealed both dynamic and stationary MI-relevant features during the task. Specifically, features representing MI evolved dynamically early in the trial and later stabilized into a stationary network of MI features. Using a hierarchical genetic algorithm for selection of MI-relevant features, we identified primarily contralateral alpha and beta frequency features over the sensorimotor and parieto-occipital cortices as stationary which extended into a bilateral pattern in the later part of the trial. During the stationary encoding of MI, by extracting the SVM prediction scores, we analyzed MI-relevant EEG activity patterns with respect to the temporal dynamics within single trials. We show that the SVM prediction score correlates to the amplitude of univariate MI-relevant features (as documented from an extensive repertoire of previous MI studies) within single trials, strongly suggesting that these are functional variations of MI strength hidden in trial averages.Significance.Our work demonstrates a powerful approach for estimating MI strength continually within single trials, having far-reaching impact for single-trial analyses. In terms of MI neurofeedback for motor rehabilitation, these results set the ground for more refined neurofeedback reflecting the strength of MI that can be provided to patients continually in time.

Subject inefficiency phenomenon of motor imagery brain-computer interface: Influence factors and potential solutions

Motor imagery brain–computer interfaces (MI‐BCIs) have great potential value in prosthetics control, neurorehabilitation, and gaming; however, currently, most such systems only operate in controlled laboratory environments. One of the most important obstacles is the MI‐BCI inefficiency phenomenon. The accuracy of MI‐BCI control varies significantly (from chance level to 100% accuracy) across subjects due to the not easily induced and unstable MI‐related EEG features. An MI‐BCI inefficient subject is defined as a subject who cannot achieve greater than 70% accuracy after sufficient training time, and multiple survey results indicate that inefficient subjects account for 10%–50% of the experimental population. The widespread use of MI‐BCI has been seriously limited due to these large percentages of inefficient subjects. In this review, we summarize recent findings of the cause of MI‐BCI inefficiency from resting‐state brain function, task‐related brain activity, brain structure, and psychological perspectives. These factors help understand the reasons for inter‐subject MI‐BCI control performance variability, and it can be concluded that the lower resting‐state sensorimotor rhythm (SMR) is the key factor in MI‐BCI inefficiency, which has been confirmed by multiple independent laboratories. We then propose to divide MI‐BCI inefficient subjects into three categories according to the resting‐state SMR and offline/online accuracy to apply more accurate approaches to solve the inefficiency problem. The potential solutions include developing transfer learning algorithms, new experimental paradigms, mindfulness meditation practice, novel training strategies, and identifying new motor imagery‐related EEG features. To date, few studies have focused on improving the control accuracy of MI‐BCI inefficient subjects; thus, we appeal to the BCI community to focus more on this research area. Only by reducing the percentage of inefficient subjects can we create the opportunity to expand the value and influence of MI‐BCI.

A Dynamic Multi-Scale Network for EEG Signal Classification

Accurate and automatic classification of the speech imagery electroencephalography (EEG) signals from a Brain-Computer Interface (BCI) system is highly demanded in clinical diagnosis. The key factor in designing an automatic classification system is to extract essential features from the original input; though many methods have achieved great success in this domain, they may fail to process the multi-scale representations from different receptive fields and thus hinder the model from achieving a higher performance. To address this challenge, in this paper, we propose a novel dynamic multi-scale network to achieve the EEG signal classification. The whole classification network is based on ResNet, and the input signal first encodes the features by the Short-time Fourier Transform (STFT); then, to further improve the multi-scale feature extraction ability, we incorporate a dynamic multi-scale (DMS) layer, which allows the network to learn multi-scale features from different receptive fields at a more granular level. To validate the effectiveness of our designed network, we conduct extensive experiments on public dataset III of BCI competition II, and the experimental results demonstrate that our proposed dynamic multi-scale network could achieve promising classification performance in this task.

Dynamic Modeling of Common Brain Neural Activity in Motor Imagery Tasks

Evaluation of brain dynamics elicited by motor imagery (MI) tasks can contribute to clinical and learning applications. The multi-subject analysis is to make inferences on the group/population level about the properties of MI brain activity. However, intrinsic neurophysiological variability of neural dynamics poses a challenge for devising efficient MI systems. Here, we develop a time-frequency model for estimating the spatial relevance of common neural activity across subjects employing an introduced statistical thresholding rule. In deriving multi-subject spatial maps, we present a comparative analysis of three feature extraction methods: Common Spatial Patterns, Functional Connectivity, and Event-Related De/Synchronization. In terms of interpretability, we evaluate the effectiveness in gathering MI data from collective populations by introducing two assumptions: (i) Non-linear assessment of the similarity between multi-subject data originating the subject-level dynamics; (ii) Assessment of time-varying brain network responses according to the ranking of individual accuracy performed in distinguishing distinct motor imagery tasks (left-hand vs. right-hand). The obtained validation results indicate that the estimated collective dynamics differently reflect the flow of sensorimotor cortex activation, providing new insights into the evolution of MI responses.

Neural component analysis: source localisation for motor imagery classification

The electroencephalogram (EEG) records a summed mixture of multiple sources of neural activity distributed throughout the brain. Source separation methods aim to un-mix the EEG in order to recover activity generated by the original sources. However, most current state-of-the-art source separation methods do not take into account the physical locations of sources of EEG activity.We present a new source separation method which uses an accurate model of the head to un-mix the EEG into individual sources based on their physical locations.We apply our method to an EEG dataset recorded during motor imagery and show that it is able to identify sources that are located in distinct physical regions of the brain. We compare our method to independent component analysis and show that our sources have higher spatial specificity and, furthermore, allow higher classification accuracies (a mean improvement in accuracy of 8.6% was achieved p =0.039).

Entropy-Based Estimation of Event-Related De/Synchronization in Motor Imagery Using Vector-Quantized Patterns

Assessment of brain dynamics elicited by motor imagery (MI) tasks contributes to clinical and learning applications. In this regard, Event-Related Desynchronization/Synchronization (ERD/S) is computed from Electroencephalographic signals, which show considerable variations in complexity. We present an Entropy-based method, termed VQEnt, for estimation of ERD/S using quantized stochastic patterns as a symbolic space, aiming to improve their discriminability and physiological interpretability. The proposed method builds the probabilistic priors by assessing the Gaussian similarity between the input measured data and their reduced vector-quantized representation. The validating results of a bi-class imagine task database (left and right hand) prove that VQEnt holds symbols that encode several neighboring samples, providing similar or even better accuracy than the other baseline sample-based algorithms of Entropy estimation. Besides, the performed ERD/S time-series are close enough to the trajectories extracted by the variational percentage of EEG signal power and fulfill the physiological MI paradigm. In BCI literate individuals, the VQEnt estimator presents the most accurate outcomes at a lower amount of electrodes placed in the sensorimotor cortex so that reduced channel set directly involved with the MI paradigm is enough to discriminate between tasks, providing an accuracy similar to the performed by the whole electrode set.

Pushing the Limits of EEG: Estimation of Large-Scale Functional Brain Networks and Their Dynamics Validated by Simultaneous fMRI

Functional magnetic resonance imaging (fMRI) is the technique of choice for detecting large-scale functional brain networks and to investigate their dynamics. Because fMRI measures brain activity indirectly, electroencephalography (EEG) has been recently considered a feasible tool for detecting such networks, particularly the resting-state networks (RSNs). However, a truly unbiased validation of such claims is still missing, which can only be accomplished by using simultaneously acquired EEG and fMRI data, due to the spontaneous nature of the activity underlying the RSNs. Additionally, EEG is still poorly explored for the purpose of mapping task-specific networks, and no studies so far have been focused on investigating networks' dynamic functional connectivity (dFC) with EEG. Here, we started by validating RSNs derived from the continuous reconstruction of EEG sources by directly comparing them with those derived from simultaneous fMRI data of 10 healthy participants, and obtaining an average overlap (quantified by the Dice coefficient) of 0.4. We also showed the ability of EEG to map the facial expressions processing network (FEPN), highlighting regions near the posterior superior temporal sulcus, where the FEPN is anchored. Then, we measured the dFC using EEG for the first time in this context, estimated dFC brain states using dictionary learning, and compared such states with those obtained from the fMRI. We found a statistically significant match between fMRI and EEG dFC states, and determined the existence of two matched dFC states which contribution over time was associated with the brain activity at the FEPN, showing that the dynamics of FEPN can be captured by both fMRI and EEG. Our results push the limits of EEG toward being used as a brain imaging tool, while supporting the growing literature on EEG correlates of (dynamic) functional connectivity measured with fMRI, and providing novel insights into the coupling mechanisms underlying the two imaging techniques.

A multidimensional and multi-feature framework for cardiac interoception

Interoception (the sensing of inner-body signals) is a multi-faceted construct with major relevance for basic and clinical neuroscience research. However, the neurocognitive signatures of this domain (cutting across behavioral, electrophysiological, and fMRI connectivity levels) are rarely reported in convergent or systematic fashion. Additionally, various controversies in the field might reflect the caveats of standard interoceptive accuracy (IA) indexes, mainly based on heartbeat detection (HBD) tasks. Here we profit from a novel IA index (md) to provide a convergent multidimensional and multi-feature approach to cardiac interoception. We found that outcomes from our IA-md index are associated with -and predicted by- canonical markers of interoception, including the hd-EEG-derived heart-evoked potential (HEP), fMRI functional connectivity within interoceptive hubs (insular, somatosensory, and frontal networks), and socio-emotional skills. Importantly, these associations proved more robust than those involving current IA indexes. Furthermore, this pattern of results persisted when taking into consideration confounding variables (gender, age, years of education, and executive functioning). This work has relevant theoretical and clinical implications concerning the characterization of cardiac interoception and its assessment in heterogeneous samples, such as those composed of neuropsychiatric patients.

A Multi-Target Motor Imagery Training Using Bimodal EEG-fMRI Neurofeedback: A Pilot Study in Chronic Stroke Patients

Traditional rehabilitation techniques present limitations and the majority of patients show poor 1-year post-stroke recovery. Thus, Neurofeedback (NF) or Brain-Computer-Interface applications for stroke rehabilitation purposes are gaining increased attention. Indeed, NF has the potential to enhance volitional control of targeted cortical areas and thus impact on motor function recovery. However, current implementations are limited by temporal, spatial or practical constraints of the specific imaging modality used. In this pilot work and for the first time in literature, we applied bimodal EEG-fMRI NF for upper limb stroke recovery on four stroke-patients with different stroke characteristics and motor impairment severity. We also propose a novel, multi-target training approach that guides the training towards the activation of the ipsilesional primary motor cortex. In addition to fMRI and EEG outcomes, we assess the integrity of the corticospinal tract (CST) with tractography. Preliminary results suggest the feasibility of our approach and show its potential to induce an augmented activation of ipsilesional motor areas, depending on the severity of the stroke deficit. Only the two patients with a preserved CST and subcortical lesions succeeded in upregulating the ipsilesional primary motor cortex and exhibited a functional improvement of upper limb motricity. These findings highlight the importance of taking into account the variability of the stroke patients' population and enabled to identify inclusion criteria for the design of future clinical studies.

Investigating Priming Effects of Physical Practice on Motor Imagery-Induced Event-Related Desynchronization

For motor imagery (MI) to be effective, an internal representation of the to-be-imagined movement may be required. A representation can be achieved through prior motor execution (ME), but the neural correlates of MI that are primed by ME practice are currently unknown. In this study, young healthy adults performed MI practice of a unimanual visuo-motor task (Group MI, n = 19) or ME practice combined with subsequent MI practice (Group ME&MI, n = 18) while electroencephalography (EEG) was recorded. Data analysis focused on the MI-induced event-related desynchronization (ERD). Specifically, changes in the ERD and movement times (MT) between a short familiarization block of ME (Block pre-ME), conducted before the MI or the ME combined with MI practice phase, and a short block of ME conducted after the practice phase (Block post-ME) were analyzed. Neither priming effects of ME practice on MI-induced ERD were found nor performance-enhancing effects of MI practice in general. We found enhancements of the ERD and MT in Block post-ME compared to Block pre-ME, but only for Group ME&MI. A comparison of ME performance measures before and after the MI phase indicated however that these changes could not be attributed to the combination of ME and MI practice. The mixed results of this study may be a consequence of the considerable intra- and inter-individual differences in the ERD, introduced by specifics of the experimental setup, in particular the individual and variable task duration, and suggest that task and experimental setup can affect the interplay of ME and MI.