Targeted reinforcement of neural oscillatory activity with real-time neuroimaging feedback

CTNet: a convolutional transformer network for EEG-based motor imagery classification

Brain-computer interface (BCI) technology bridges the direct communication between the brain and machines, unlocking new possibilities for human interaction and rehabilitation. EEG-based motor imagery (MI) plays a pivotal role in BCI, enabling the translation of thought into actionable commands for interactive and assistive technologies. However, the constrained decoding performance of brain signals poses a limitation to the broader application and development of BCI systems. In this study, we introduce a convolutional Transformer network (CTNet) designed for EEG-based MI classification. Firstly, CTNet employs a convolutional module analogous to EEGNet, dedicated to extracting local and spatial features from EEG time series. Subsequently, it incorporates a Transformer encoder module, leveraging a multi-head attention mechanism to discern the global dependencies of EEG's high-level features. Finally, a straightforward classifier module comprising fully connected layers is followed to categorize EEG signals. In subject-specific evaluations, CTNet achieved remarkable decoding accuracies of 82.52% and 88.49% on the BCI IV-2a and IV-2b datasets, respectively. Furthermore, in the challenging cross-subject assessments, CTNet achieved recognition accuracies of 58.64% on the BCI IV-2a dataset and 76.27% on the BCI IV-2b dataset. In both subject-specific and cross-subject evaluations, CTNet holds a leading position when compared to some of the state-of-the-art methods. This underscores the exceptional efficacy of our approach and its potential to set a new benchmark in EEG decoding.

How time shapes cognitive control: A high-density EEG study of task-switching

Task-switching is one of the most popular paradigms to investigate cognitive control. The main finding of interest is the switch cost: RTs in switch trials are longer than RTs in repetition trials. Despite the massive amount of research in these topics, little is known about the underlying temporal dynamics of the cortical regions involved in these phenomena. Here we used high density EEG to unveil the spatiotemporal neural dynamics associated with both the switch cost and to its modulation over time (time-on-task effect), as two markers of cognitive control reflecting effortful and procedural mechanisms, respectively. We found that, as a function of task practice, the switch cost decreased and both the switch-positivity and the switch-negativity event-related responses increased, although the latter showed a larger modulatory effect. At a source level, this effect was revealed by a progressively higher activation of the left middle and superior frontal gyrus.

Control freaks: Towards optimal selection of control conditions for fMRI neurofeedback studies

fMRI Neurofeedback research employs many different control conditions. Currently, there is no consensus as to which control condition is best, and the answer depends on what aspects of the neurofeedback-training design one is trying to control for. These aspects can range from determining whether participants can learn to control brain activity via neurofeedback to determining whether there are clinically significant effects of the neurofeedback intervention. Lack of consensus over criteria for control conditions has hampered the design and interpretation of studies employing neurofeedback protocols. This paper presents an overview of the most commonly employed control conditions currently used in neurofeedback studies and discusses their advantages and disadvantages. Control conditions covered include no control, treatment-as-usual, bidirectional-regulation control, feedback of an alternative brain signal, sham feedback, and mental-rehearsal control. We conclude that the selection of the control condition(s) should be determined by the specific research goal of the study and best procedures that effectively control for relevant confounding factors.

Differential Subjective Experiences in Learners and Non-learners in Frontal Alpha Neurofeedback: Piloting a Mixed-Method Approach

In a neurofeedback paradigm, trainees learn to willfully control their brain dynamics. How this is realized remains an open question. We evaluate the hypothesis that learning success is associated with a specific phenomenology. To address this proposal, we combined quantitative and qualitative analyses of a short neurofeedback training (NFT) session during which participants enhanced mid-frontal alpha power and were then subsequently interviewed about their experiences. We analyzed the electrophysiological data to determine learning success and classify trainees as learners and non-learners. The subjective experiences differed between the two groups and are best described along a trying-sensing continuum, with non-learners engaging effortfully with the task (e.g., “I will it [the bar] to move”) whereas learners reported more sensing of their inner (e.g., “Something inside my stomach”) and outer environment (e.g., “I was aware of the sound of the beeps”). In the process of piloting this mixed-method approach, we developed a classification system for the verbal reports. This system provides an explicit analytic framework which might guide future studies that aim to investigate the association between subjective experiences and NFT protocols.

Endogenously generated gamma-band oscillations in early visual cortex: A neurofeedback study

Human subjects were trained with neurofeedback (NFB) to enhance the power of narrow-band gamma oscillations in circumscribed regions of early visual cortex. To select the region and the oscillation frequency for NFB training, gamma oscillations were induced with locally presented drifting gratings. The source and frequency of these induced oscillations were determined using beamforming methods. During NFB training the power of narrow band gamma oscillations was continuously extracted from this source with online beamforming and converted into the pitch of a tone signal. We found that seven out of ten subjects were able to selectively increase the amplitude of gamma oscillations in the absence of visual stimulation. One subject however failed completely and two subjects succeeded to manipulate the feedback signal by contraction of muscles. In all subjects the attempts to enhance visual gamma oscillations were associated with an increase of beta oscillations over precentral/frontal regions. Only successful subjects exhibited an additional marked increase of theta oscillations over precentral/prefrontal and temporal regions whereas unsuccessful subjects showed an increase of alpha band oscillations over occipital regions. We argue that spatially confined networks in early visual cortex can be entrained to engage in narrow band gamma oscillations not only by visual stimuli but also by top down signals. We interpret the concomitant increase in beta oscillations as indication for an engagement of the fronto-parietal attention network and the increase of theta oscillations as a correlate of imagery. Our finding support the application of NFB in disease conditions associated with impaired gamma synchronization.

Random Feedback Makes Listeners Tone-Deaf

The mental representation of pitch structure (tonal knowledge) is a core component of musical experience and is learned implicitly through exposure to music. One theory of congenital amusia (tone deafness) posits that conscious access to tonal knowledge is disrupted, leading to a severe deficit of music cognition. We tested this idea by providing random performance feedback to neurotypical listeners while they listened to melodies for tonal incongruities and had their electrical brain activity monitored. The introduction of random feedback was associated with a reduction of accuracy and confidence, and a suppression of the late positive brain response usually elicited by conscious detection of a tonal violation. These effects mirror the behavioural and neurophysiological profile of amusia. In contrast, random feedback was associated with an increase in the amplitude of the early right anterior negativity, possibly due to heightened attention to the experimental task. This successful simulation of amusia in a normal brain highlights the key role of feedback in learning, and thereby provides a new avenue for the rehabilitation of learning disorders.

A Comparative Study on the Detection of Covert Attention in Event-Related EEG and MEG Signals to Control a BCI

In brain-computer interface (BCI) applications the detection of neural processing as revealed by event-related potentials (ERPs) is a frequently used approach to regain communication for people unable to interact through any peripheral muscle control. However, the commonly used electroencephalography (EEG) provides signals of low signal-to-noise ratio, making the systems slow and inaccurate. As an alternative noninvasive recording technique, the magnetoencephalography (MEG) could provide more advantageous electrophysiological signals due to a higher number of sensors and the magnetic fields not being influenced by volume conduction. We investigated whether MEG provides higher accuracy in detecting event-related fields (ERFs) compared to detecting ERPs in simultaneously recorded EEG, both evoked by a covert attention task, and whether a combination of the modalities is advantageous. In our approach, a detection algorithm based on spatial filtering is used to identify ERP/ERF components in a data-driven manner. We found that MEG achieves higher decoding accuracy (DA) compared to EEG and that the combination of both further improves the performance significantly. However, MEG data showed poor performance in cross-subject classification, indicating that the algorithm's ability for transfer learning across subjects is better in EEG. Here we show that BCI control by covert attention is feasible with EEG and MEG using a data-driven spatial filter approach with a clear advantage of the MEG regarding DA but with a better transfer learning in EEG.

Defining the Neural Substrate of the Adult Outcome of Childhood ADHD: A Multimodal Neuroimaging Study of Response Inhibition

OBJECTIVE

Understanding the neural processes tied to the adult outcome of childhood attention deficit hyperactivity disorder (ADHD) could guide novel interventions to improve its clinical course. It has been argued that normalization of prefrontal cortical activity drives remission from ADHD, while anomalies in subcortical processes are "fixed," present even in remission. Using multimodal neuroimaging of inhibitory processes, the authors tested these hypotheses in adults followed since childhood, contrasting remitted against persistent ADHD.

METHOD

Adult participants (persistent ADHD, N=35; remit-ted ADHD, N=47; never affected, N=99) were scanned with functional MRI (fMRI) (N=85), magnetoencephalography (N=33), or both (N=63) during a response inhibition task.

RESULTS

In fMRI analyses, during inhibition, right caudate anomalies reflected a childhood ADHD history and were present even among those who remitted. By contrast, differences related to adult outcome emerged in cortical (right inferior frontal and inferior parietal/precuneus) and cerebellar regions. The persistent ADHD group showed under-activation, whereas the remitted ADHD group did not differ significantly from the never-affected group. Magnetoencephalography showed that the association between adult symptom severity and prefrontal neuronal activity was confined to the time window covering the act of inhibition (300 ms-350 ms). Group differences in cerebellar and parietal neuronal activity occurred during the time window of performance monitoring processes (500 ms-600 ms).

CONCLUSIONS

By combining fMRI and magnetoencephalography, the location and time window of neuronal activity that underpins the adult outcome of ADHD was pinpointed. Thus, the cortico-cerebellar processes tied to the clinical course of ADHD are separated from the subcortical processes that are not.

Neurofeedback learning modifies the incidence rate of alpha spindles, but not their duration and amplitude

Although the first experiments on alpha-neurofeedback date back nearly six decades ago, when Joseph Kamiya reported successful operant conditioning of alpha-rhythm in humans, the effectiveness of this paradigm in various experimental and clinical settings is still a matter of debate. Here, we investigated the changes in EEG patterns during a continuously administered neurofeedback of P4 alpha activity. Two days of neurofeedback training were sufficient for a significant increase in the alpha power to occur. A detailed analysis of these EEG changes showed that the alpha power rose because of an increase in the incidence rate of alpha episodes, whereas the amplitude and the duration of alpha oscillations remained unchanged. These findings suggest that neurofeedback facilitates volitional control of alpha activity onset, but alpha episodes themselves appear to be maintained automatically with no volitional control – a property overlooked by previous studies that employed continuous alpha-power neurofeedback. We propose that future research on alpha neurofeedback should explore reinforcement schedules based on detection of onsets and offsets of alpha waves, and employ these statistics for exploration and quantification of neurofeedback induced effects.

Proprioceptive Feedback Facilitates Motor Imagery-Related Operant Learning of Sensorimotor β-Band Modulation

Motor imagery (MI) activates the sensorimotor system independent of actual movements and might be facilitated by neurofeedback. Knowledge on the interaction between feedback modality and the involved frequency bands during MI-related brain self-regulation is still scarce. Previous studies compared the cortical activity during the MI task with concurrent feedback (MI with feedback condition) to cortical activity during the relaxation task where no feedback was provided (relaxation without feedback condition). The observed differences might, therefore, be related to either the task or the feedback. A proper comparison would necessitate studying a relaxation condition with feedback and a MI task condition without feedback as well. Right-handed healthy subjects performed two tasks, i.e., MI and relaxation, in alternating order. Each of the tasks (MI vs. relaxation) was studied with and without feedback. The respective event-driven oscillatory activity, i.e., sensorimotor desynchronization (during MI) or synchronization (during relaxation), was rewarded with contingent feedback. Importantly, feedback onset was delayed to study the task-related cortical activity in the absence of feedback provision during the delay period. The reward modality was alternated every 15 trials between proprioceptive and visual feedback. Proprioceptive input was superior to visual input to increase the range of task-related spectral perturbations in the α- and β-band, and was necessary to consistently achieve MI-related sensorimotor desynchronization (ERD) significantly below baseline. These effects occurred in task periods without feedback as well. The increased accuracy and duration of learned brain self-regulation achieved in the proprioceptive condition was specific to the β-band. MI-related operant learning of brain self-regulation is facilitated by proprioceptive feedback and mediated in the sensorimotor β-band.

Comparing Features for Classification of MEG Responses to Motor Imagery

BACKGROUND

Motor imagery (MI) with real-time neurofeedback could be a viable approach, e.g., in rehabilitation of cerebral stroke. Magnetoencephalography (MEG) noninvasively measures electric brain activity at high temporal resolution and is well-suited for recording oscillatory brain signals. MI is known to modulate 10- and 20-Hz oscillations in the somatomotor system. In order to provide accurate feedback to the subject, the most relevant MI-related features should be extracted from MEG data. In this study, we evaluated several MEG signal features for discriminating between left- and right-hand MI and between MI and rest.

METHODS

MEG was measured from nine healthy participants imagining either left- or right-hand finger tapping according to visual cues. Data preprocessing, feature extraction and classification were performed offline. The evaluated MI-related features were power spectral density (PSD), Morlet wavelets, short-time Fourier transform (STFT), common spatial patterns (CSP), filter-bank common spatial patterns (FBCSP), spatio-spectral decomposition (SSD), and combined SSD+CSP, CSP+PSD, CSP+Morlet, and CSP+STFT. We also compared four classifiers applied to single trials using 5-fold cross-validation for evaluating the classification accuracy and its possible dependence on the classification algorithm. In addition, we estimated the inter-session left-vs-right accuracy for each subject.

RESULTS

The SSD+CSP combination yielded the best accuracy in both left-vs-right (mean 73.7%) and MI-vs-rest (mean 81.3%) classification. CSP+Morlet yielded the best mean accuracy in inter-session left-vs-right classification (mean 69.1%). There were large inter-subject differences in classification accuracy, and the level of the 20-Hz suppression correlated significantly with the subjective MI-vs-rest accuracy. Selection of the classification algorithm had only a minor effect on the results.

CONCLUSIONS

We obtained good accuracy in sensor-level decoding of MI from single-trial MEG data. Feature extraction methods utilizing both the spatial and spectral profile of MI-related signals provided the best classification results, suggesting good performance of these methods in an online MEG neurofeedback system.

Removing Cardiac Artefacts in Magnetoencephalography with Resampled Moving Average Subtraction

Magnetoencephalography (MEG) signals are commonly contaminated by cardiac artefacts (CAs). Principle component analysis and independent component analysis have been widely used for removing CAs, but they typically require a complex procedure for the identification of CA-related components. We propose a simple and efficient method, resampled moving average subtraction (RMAS), to remove CAs from MEG data. Based on an electrocardiogram (ECG) channel, a template for each cardiac cycle was estimated by a weighted average of epochs of MEG data over consecutive cardiac cycles, combined with a resampling technique for accurate alignment of the time waveforms. The template was subtracted from the corresponding epoch of the MEG data. The resampling reduced distortions due to asynchrony between the cardiac cycle and the MEG sampling times. The RMAS method successfully suppressed CAs while preserving both event-related responses and high-frequency (>45 Hz) components in the MEG data.

Parkinson subtype-specific Granger-causal coupling and coherence frequency in the subthalamic area

Previous work on Parkinson's disease (PD) has indicated a predominantly afferent coupling between affected arm muscle activity and electrophysiological activity within the subthalamic nucleus (STN). So far, no information is available indicating which frequency components drive the afferent information flow in PD patients. Non-directional coupling e.g. by measuring coherence is primarily established in the beta band as well as at tremor frequency. Based on previous evidence it is likely that different subtypes of the disease are associated with different connectivity patterns. Therefore, we determined coherence and causality between local field potentials (LFPs) in the STN and surface electromyograms (EMGs) from the contralateral arm in 18 akinetic-rigid (AR) PD patients and 8 tremor-dominant (TD) PD patients. During the intraoperative recording, patients were asked to lift their forearm contralateral to the recording side. Significantly more afferent connections were detected for the TD patients for tremor-periods and non-tremor-periods combined as well as for only tremor periods. Within the STN 74% and 63% of the afferent connections are associated with coherence from 4-8Hz and 8-12Hz, respectively. However, when considering only tremor-periods significantly more afferent than efferent connections were associated with coherence from 12 to 20Hz across all recording heights. No difference between efferent and afferent connections is seen in the frequency range from 4 to 12Hz for all recording heights. For the AR patients, no significant difference in afferent and efferent connections within the STN was found for the different frequency bands. Still, for the AR patients dorsal of the STN significantly more afferent than efferent connections were associated with coherence in the frequency range from 12 to 16Hz. These results provide further evidence for the differential pathological oscillations and pathways present in AR and TD Parkinson patients.

MEG-based neurofeedback for hand rehabilitation

BACKGROUND

Providing neurofeedback (NF) of motor-related brain activity in a biologically-relevant and intuitive way could maximize the utility of a brain-computer interface (BCI) for promoting therapeutic plasticity. We present a BCI capable of providing intuitive and direct control of a video-based grasp.

METHODS

Utilizing magnetoencephalography's (MEG) high temporal and spatial resolution, we recorded sensorimotor rhythms (SMR) that were modulated by grasp or rest intentions. SMR modulation controlled the grasp aperture of a stop motion video of a human hand. The displayed hand grasp position was driven incrementally towards a closed or opened state and subjects were required to hold the targeted position for a time that was adjusted to change the task difficulty.

RESULTS

We demonstrated that three individuals with complete hand paralysis due to spinal cord injury (SCI) were able to maintain brain-control of closing and opening a virtual hand with an average of 63 % success which was significantly above the average chance rate of 19 %. This level of performance was achieved without pre-training and less than 4 min of calibration. In addition, successful grasp targets were reached in 1.96 ± 0.15 s. Subjects performed 200 brain-controlled trials in approximately 30 min excluding breaks. Two of the three participants showed a significant improvement in SMR indicating that they had learned to change their brain activity within a single session of NF.

CONCLUSIONS

This study demonstrated the utility of a MEG-based BCI system to provide realistic, efficient, and focused NF to individuals with paralysis with the goal of using NF to induce neuroplasticity.

Brain computer interface learning for systems based on electrocorticography and intracortical microelectrode arrays

A brain-computer interface (BCI) system transforms neural activity into control signals for external devices in real time. A BCI user needs to learn to generate specific cortical activity patterns to control external devices effectively. We call this process BCI learning, and it often requires significant effort and time. Therefore, it is important to study this process and develop novel and efficient approaches to accelerate BCI learning. This article reviews major approaches that have been used for BCI learning, including computer-assisted learning, co-adaptive learning, operant conditioning, and sensory feedback. We focus on BCIs based on electrocorticography and intracortical microelectrode arrays for restoring motor function. This article also explores the possibility of brain modulation techniques in promoting BCI learning, such as electrical cortical stimulation, transcranial magnetic stimulation, and optogenetics. Furthermore, as proposed by recent BCI studies, we suggest that BCI learning is in many ways analogous to motor and cognitive skill learning, and therefore skill learning should be a useful metaphor to model BCI learning.

Source-based neurofeedback methods using EEG recordings: training altered brain activity in a functional brain source derived from blind source separation

A developing literature explores the use of neurofeedback in the treatment of a range of clinical conditions, particularly ADHD and epilepsy, whilst neurofeedback also provides an experimental tool for studying the functional significance of endogenous brain activity. A critical component of any neurofeedback method is the underlying physiological signal which forms the basis for the feedback. While the past decade has seen the emergence of fMRI-based protocols training spatially confined BOLD activity, traditional neurofeedback has utilized a small number of electrode sites on the scalp. As scalp EEG at a given electrode site reflects a linear mixture of activity from multiple brain sources and artifacts, efforts to successfully acquire some level of control over the signal may be confounded by these extraneous sources. Further, in the event of successful training, these traditional neurofeedback methods are likely influencing multiple brain regions and processes. The present work describes the use of source-based signal processing methods in EEG neurofeedback. The feasibility and potential utility of such methods were explored in an experiment training increased theta oscillatory activity in a source derived from Blind Source Separation (BSS) of EEG data obtained during completion of a complex cognitive task (spatial navigation). Learned increases in theta activity were observed in two of the four participants to complete 20 sessions of neurofeedback targeting this individually defined functional brain source. Source-based EEG neurofeedback methods using BSS may offer important advantages over traditional neurofeedback, by targeting the desired physiological signal in a more functionally and spatially specific manner. Having provided preliminary evidence of the feasibility of these methods, future work may study a range of clinically and experimentally relevant brain processes where individual brain sources may be targeted by source-based EEG neurofeedback.

Brain–Computer Interface after Nervous System Injury

Brain–computer interface (BCI) has proven to be a useful tool for providing alternative communication and mobility to patients suffering from nervous system injury. BCI has been and will continue to be implemented into rehabilitation practices for more interactive and speedy neurological recovery. The most exciting BCI technology is evolving to provide therapeutic benefits by inducing cortical reorganization via neuronal plasticity. This article presents a state-of-the-art review of BCI technology used after nervous system injuries, specifically: amyotrophic lateral sclerosis, Parkinson’s disease, spinal cord injury, stroke, and disorders of consciousness. Also presented is transcending, innovative research involving new treatment of neurological disorders.

How to use fMRI functional localizers to improve EEG/MEG source estimation

EEG and MEG have excellent temporal resolution, but the estimation of the neural sources that generate the signals recorded by the sensors is a difficult, ill-posed problem. The high spatial resolution of functional MRI makes it an ideal tool to improve the localization of the EEG/MEG sources using data fusion. However, the combination of the two techniques remains challenging, as the neural generators of the EEG/MEG and BOLD signals might in some cases be very different. Here we describe a data fusion approach that was developed by our team over the last decade in which fMRI is used to provide source constraints that are based on functional areas defined individually for each subject. This mini-review describes the different steps that are necessary to perform source estimation using this approach. It also provides a list of pitfalls that should be avoided when doing fMRI-informed EEG/MEG source imaging. Finally, it describes the advantages of using a ROI-based approach for group-level analysis and for the study of sensory systems.