EEG-Informed fMRI: A Review of Data Analysis Methods

Trimodal brain imaging: A novel approach for simultaneous investigation of human brain function

While advancements have improved the extent to which individual brain imaging approaches capture information regarding spatial or temporal dynamics of brain activity, the connections between these aspects and their relation to psychological functioning remain only partially understood. Acquisition and integration across multiple brain imaging modalities allows for the possible clarification of these connections. The present review provides an overview of three complementary modalities - functional magnetic resonance imaging (fMRI), electroencephalography/event-related potentials (EEG/ERP), and event-related optical signals (EROS) - and discusses progress and considerations for each modality, along with a summary of a novel protocol for acquiring them simultaneously. Initial evidence points to the feasibility of acquiring and integrating multiple measures of brain function that allows for addressing questions in ways not otherwise possible using traditional approaches. Simultaneous trimodal brain imaging in humans provides new possibilities for clarifying spatiotemporal dynamics of brain activity and for identifying multifaceted associations with measures of individual differences and important health outcomes.

The hemodynamic response to co-occurring interictal epileptiform discharges and high-frequency oscillations localizes the seizure-onset zone

OBJECTIVE

To use intracranial electroencephalography (EEG) to characterize functional magnetic resonance imaging (fMRI) activation maps associated with high-frequency oscillations (HFOs) (80-250 Hz) and examine their proximity to HFO- and seizure-generating tissue.

METHODS

Forty-five patients implanted with intracranial depth electrodes underwent a simultaneous EEG-fMRI study at 3 T. HFOs were detected algorithmically from cleaned EEG and visually confirmed by an experienced electroencephalographer. HFOs that co-occurred with interictal epileptiform discharges (IEDs) were subsequently identified. fMRI activation maps associated with HFOs were generated that occurred either independently of IEDs or within ±200 ms of an IED. For all significant analyses, the Maximum, Second Maximum, and Closest activation clusters were identified, and distances were measured to both the electrodes where the HFOs were observed and the electrodes involved in seizure onset.

RESULTS

We identified 108 distinct groups of HFOs from 45 patients. We found that HFOs with IEDs produced fMRI clusters that were closer to the local field potentials of the corresponding HFOs observed within the EEG than HFOs without IEDs. In addition to the fMRI clusters being closer to the location of the EEG correlate, HFOs with IEDs generated Maximum clusters with greater z-scores and larger volumes than HFOs without IEDs. We also observed that HFOs with IEDs resulted in more discrete activation maps.

SIGNIFICANCE

Intracranial EEG-fMRI can be used to probe the hemodynamic response to HFOs. The hemodynamic response associated with HFOs that co-occur with IEDs better identifies known epileptic tissue than HFOs that occur independently.

Conscious but not thinking-Mind-blanks during visuomotor tracking: An fMRI study of endogenous attention lapses

Attention lapses (ALs) are complete lapses of responsiveness in which performance is briefly but completely disrupted and during which, as opposed to microsleeps, the eyes remain open. Although the phenomenon of ALs has been investigated by behavioural and physiological means, the underlying cause of an AL has largely remained elusive. This study aimed to investigate the underlying physiological substrates of behaviourally identified endogenous ALs during a continuous visuomotor task, primarily to answer the question: Were the ALs during this task due to extreme mind-wandering or mind-blanks? The data from two studies were combined, resulting in data from 40 healthy non-sleep-deprived subjects (20M/20F; mean age 27.1 years, 20-45). Only 17 of the 40 subjects were used in the analysis due to a need for a minimum of two ALs per subject. Subjects performed a random 2-D continuous visuomotor tracking task for 50 and 20 min in Studies 1 and 2, respectively. Tracking performance, eye-video, and functional magnetic resonance imaging (fMRI) were recorded simultaneously. A human expert visually inspected the tracking performance and eye-video recordings to identify and categorise lapses of responsiveness as microsleeps or ALs. Changes in neural activity during 85 ALs (17 subjects) relative to responsive tracking were estimated by whole-brain voxel-wise fMRI and by haemodynamic response (HR) analysis in regions of interest (ROIs) from seven key networks to reveal the neural signature of ALs. Changes in functional connectivity (FC) within and between the key ROIs were also estimated. Networks explored were the default mode network, dorsal attention network, frontoparietal network, sensorimotor network, salience network, visual network, and working memory network. Voxel-wise analysis revealed a significant increase in blood-oxygen-level-dependent activity in the overlapping dorsal anterior cingulate cortex and supplementary motor area region but no significant decreases in activity; the increased activity is considered to represent a recovery-of-responsiveness process following an AL. This increased activity was also seen in the HR of the corresponding ROI. Importantly, HR analysis revealed no trend of increased activity in the posterior cingulate of the default mode network, which has been repeatedly demonstrated to be a strong biomarker of mind-wandering. FC analysis showed decoupling of external attention, which supports the involuntary nature of ALs, in addition to the neural recovery processes. Other findings were a decrease in HR in the frontoparietal network before the onset of ALs, and a decrease in FC between default mode network and working memory network. These findings converge to our conclusion that the ALs observed during our task were involuntary mind-blanks. This is further supported behaviourally by the short duration of the ALs (mean 1.7 s), which is considered too brief to be instances of extreme mind-wandering. This is the first study to demonstrate that at least the majority of complete losses of responsiveness on a continuous visuomotor task are, if not due to microsleeps, due to involuntary mind-blanks.

Odor and taste characteristics, transduction mechanism, and perceptual interaction in fermented foods: a review

Fermentation is a critical technological process for flavor development in fermented foods. The combination of odor and taste, known as flavor, is crucial in enhancing people's perception and psychology toward fermented foods, thereby increasing their acceptance among consumers. This review summarized the determination and key flavor compound screening methods in fermented foods and analyzed the flavor perception, perceptual interactions, and evaluation methods. The flavor compounds in fermented foods could be separated, purified, and identified by instrument techniques, and a molecular sensory science approach could identify the key flavor compounds. How flavor compounds bind to their respective receptors determines flavor perception, which is influenced by their perceptual interactions, including odor-odor, taste-taste, and odor-taste. Evaluation methods of flavor perception mainly include human sensory evaluation, electronic sensors and biosensors, and neuroimaging techniques. Among them, the biosensor-based evaluation methods could facilitate the investigation of the flavor transduction mechanism and the neuroimaging technique could explain the brain's signals that relate to the perception of flavor and how they compare to signals from other senses. This review aims to elucidate the flavor profile of fermented foods and highlight the significance of comprehending the interactions between various flavor compounds, thus improving the healthiness and sensory attributes.

EEG decoding with spatiotemporal convolutional neural network for visualization and closed-loop control of sensorimotor activities: A simultaneous EEG-fMRI study

Closed-loop neurofeedback training utilizes neural signals such as scalp electroencephalograms (EEG) to manipulate specific neural activities and the associated behavioral performance. A spatiotemporal filter for high-density whole-head scalp EEG using a convolutional neural network can overcome the ambiguity of the signaling source because each EEG signal includes information on the remote regions. We simultaneously acquired EEG and functional magnetic resonance images in humans during the brain-computer interface (BCI) based neurofeedback training and compared the reconstructed and modeled hemodynamic responses of the sensorimotor network. Filters constructed with a convolutional neural network captured activities in the targeted network with spatial precision and specificity superior to those of the EEG signals preprocessed with standard pipelines used in BCI-based neurofeedback paradigms. The middle layers of the trained model were examined to characterize the neuronal oscillatory features that contributed to the reconstruction. Analysis of the layers for spatial convolution revealed the contribution of distributed cortical circuitries to reconstruction, including the frontoparietal and sensorimotor areas, and those of temporal convolution layers that successfully reconstructed the hemodynamic response function. Employing a spatiotemporal filter and leveraging the electrophysiological signatures of the sensorimotor excitability identified in our middle layer analysis would contribute to the development of a further effective neurofeedback intervention.

Low frequency oscillations reflect neurovascular coupling and disappear after cerebral death

Spectrum power analysis in the low frequency oscillations (LFO) region of functional near infrared spectroscopy (fNIRS) is a promising method to deliver information about brain activation and therefore might be used for prognostication in patients with disorders of consciousness in the neurocritical care unit alongside with established methods. In this study, we measure the cortical hemodynamic response measured by fNIRS in the LFO region following auditory and somatosensory stimulation in healthy subjects. The significant hemodynamic reaction in the contralateral hemisphere correlation with the physiologic electric response suggests neurovascular coupling. In addition, we investigate power spectrum changes in steady state measurements of cerebral death patients and healthy subjects in the LFO region, the frequency of the heartbeat and respiration. The spectral power within the LFO region was lower in the patients with cerebral death compared to the healthy subjects, whereas there were no differences in spectral power for physiological activities such as heartbeat and respiration rate. This finding indicates the cerebral origin of our low frequency measurements. Therefore, LFO measurements are a potential method to detect brain activation in patients with disorders of consciousness and cerebral death. However, further studies in patients are needed to investigate its potential clinical use.

EEG as a potential ground truth for the assessment of cognitive state in software development activities: A multimodal imaging study

Cognitive human error and recent cognitive taxonomy on human error causes of software defects support the intuitive idea that, for instance, mental overload, attention slips, and working memory overload are important human causes for software bugs. In this paper, we approach the EEG as a reliable surrogate to MRI-based reference of the programmer's cognitive state to be used in situations where heavy imaging techniques are infeasible. The idea is to use EEG biomarkers to validate other less intrusive physiological measures, that can be easily recorded by wearable devices and useful in the assessment of the developer's cognitive state during software development tasks. Herein, our EEG study, with the support of fMRI, presents an extensive and systematic analysis by inspecting metrics and extracting relevant information about the most robust features, best EEG channels and the best hemodynamic time delay in the context of software development tasks. From the EEG-fMRI similarity analysis performed, we found significant correlations between a subset of EEG features and the Insula region of the brain, which has been reported as a region highly related to high cognitive tasks, such as software development tasks. We concluded that despite a clear inter-subject variability of the best EEG features and hemodynamic time delay used, the most robust and predominant EEG features, across all the subjects, are related to the Hjorth parameter Activity and Total Power features, from the EEG channels F4, FC4 and C4, and considering in most of the cases a hemodynamic time delay of 4 seconds used on the hemodynamic response function. These findings should be taken into account in future EEG-fMRI studies in the context of software debugging.

The time-evolving epileptic brain network: concepts, definitions, accomplishments, perspectives

Epilepsy is now considered a network disease that affects the brain across multiple levels of spatial and temporal scales. The paradigm shift from an epileptic focus-a discrete cortical area from which seizures originate-to a widespread epileptic network-spanning lobes and hemispheres-considerably advanced our understanding of epilepsy and continues to influence both research and clinical treatment of this multi-faceted high-impact neurological disorder. The epileptic network, however, is not static but evolves in time which requires novel approaches for an in-depth characterization. In this review, we discuss conceptual basics of network theory and critically examine state-of-the-art recording techniques and analysis tools used to assess and characterize a time-evolving human epileptic brain network. We give an account on current shortcomings and highlight potential developments towards an improved clinical management of epilepsy.

Is There a Difference in EEG Characteristics in Acute, Chronic, and Experimentally Induced Musculoskeletal Pain States? a Systematic Review

Electroencephalographic (EEG) alterations have been demonstrated in acute, chronic, and experimentally induced musculoskeletal (MSK) pain conditions. However, there is no cumulative evidence on the associated EEG characteristics differentiating acute, chronic, and experimentally induced musculoskeletal pain states, especially compared to healthy controls. The present systematic review was performed according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines (PRISMA) to review and summarize available evidence for cortical brain activity and connectivity alterations in acute, chronic, and experimentally induced MSK pain states. Five electronic databases were systematically searched from their inception to 2022. A total of 3471 articles were screened, and 26 full articles (five studies on chronic pain and 21 studies on experimentally induced pain) were included for the final synthesis. Using the Downs and Black risk of assessment tool, 92% of the studies were assessed as low to moderate quality. The review identified a 'very low' level of evidence for the changes in EEG and subjective outcome measures for both chronic and experimentally induced MSK pain based on the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) criteria. Overall, the findings of this review indicate a trend toward decreased alpha and beta EEG power in evoked chronic clinical pain conditions and increased theta and alpha power in resting-state EEG recorded from chronic MSK pain conditions. EEG characteristics are unclear under experimentally induced pain conditions.

Two is better? combining EEG and fMRI for BCI and neurofeedback: a systematic review

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) are two commonly used non-invasive techniques for measuring brain activity in neuroscience and brain-computer interfaces (BCI).Objective. In this review, we focus on the use of EEG and fMRI in neurofeedback (NF) and discuss the challenges of combining the two modalities to improve understanding of brain activity and achieve more effective clinical outcomes. Advanced technologies have been developed to simultaneously record EEG and fMRI signals to provide a better understanding of the relationship between the two modalities. However, the complexity of brain processes and the heterogeneous nature of EEG and fMRI present challenges in extracting useful information from the combined data.Approach. We will survey existing EEG-fMRI combinations and recent studies that exploit EEG-fMRI in NF, highlighting the experimental and technical challenges.Main results. We made a classification of the different combination of EEG-fMRI for NF, we provide a review of multimodal analysis methods for EEG-fMRI features. We also survey the current state of research on EEG-fMRI in the different existing NF paradigms. Finally, we also identify some of the remaining challenges in this field.Significance. By exploring EEG-fMRI combinations in NF, we are advancing our knowledge of brain function and its applications in clinical settings. As such, this review serves as a valuable resource for researchers, clinicians, and engineers working in the field of neural engineering and rehabilitation, highlighting the promising future of EEG-fMRI-based NF.

NeuXus open-source tool for real-time artifact reduction in simultaneous EEG-fMRI

The simultaneous acquisition of electroencephalography and functional magnetic resonance imaging (EEG-fMRI) allows the complementary study of the brain's electrophysiology and hemodynamics with high temporal and spatial resolution. One application with great potential is neurofeedback training of targeted brain activity, based on the real-time analysis of the EEG and/or fMRI signals. This depends on the ability to reduce in real time the severe artifacts affecting the EEG signal acquired with fMRI, mainly the gradient and pulse artifacts. A few methods have been proposed for this purpose, but they are either slow, hardware-dependent, publicly unavailable, or proprietary software. Here, we present a fully open-source and publicly available tool for real-time EEG artifact reduction in simultaneous EEG-fMRI recordings that is fast and applicable to any hardware. Our tool is integrated in the Python toolbox NeuXus for real-time EEG processing and adapts to a real-time scenario well-established artifact average subtraction methods combined with a long short-term memory network for R peak detection. We benchmarked NeuXus on three different datasets, in terms of artifact power reduction and background signal preservation in resting state, alpha-band power reactivity to eyes closure, and event-related desynchronization during motor imagery. We showed that NeuXus performed at least as well as the only available real-time tool for conventional hardware setups (BrainVision's RecView) and a well-established offline tool (EEGLAB's FMRIB plugin). We also demonstrated NeuXus' real-time ability by reporting execution times under 250 ms. In conclusion, we present and validate the first fully open-source and hardware-independent solution for real-time artifact reduction in simultaneous EEG-fMRI studies.

Corticostriatal beta oscillation changes associated with cognitive function in Parkinson's disease

Cognitive impairment is the most frequent non-motor symptom in Parkinson's disease and is associated with deficits in a number of cognitive functions including working memory. However, the pathophysiology of Parkinson's disease cognitive impairment is poorly understood. Beta oscillations have previously been shown to play an important role in cognitive functions including working memory encoding. Decreased dopamine in motor cortico-striato-thalamo-cortical (CSTC) circuits increases the spectral power of beta oscillations and results in Parkinson's disease motor symptoms. Analogous changes in parallel cognitive CSTC circuits involving the caudate and dorsolateral prefrontal cortex (DLPFC) may contribute to Parkinson's disease cognitive impairment. The objective of our study is to evaluate whether changes in beta oscillations in the caudate and DLPFC contribute to cognitive impairment in Parkinson's disease patients. To investigate this, we used local field potential recordings during deep brain stimulation surgery in 15 patients with Parkinson's disease. Local field potentials were recorded from DLPFC and caudate at rest and during a working memory task. We examined changes in beta oscillatory power during the working memory task as well as the relationship of beta oscillatory activity to preoperative cognitive status, as determined from neuropsychological testing results. We additionally conducted exploratory analyses on the relationship between cognitive impairment and task-based changes in spectral power in additional frequency bands. Spectral power of beta oscillations decreased in both DLPFC and caudate during working memory encoding and increased in these structures during feedback. Subjects with cognitive impairment had smaller decreases in caudate and DLPFC beta oscillatory power during encoding. In our exploratory analysis, we found that similar differences occurred in alpha frequencies in caudate and theta and alpha in DLPFC. Our findings suggest that oscillatory power changes in cognitive CSTC circuits may contribute to cognitive symptoms in patients with Parkinson's disease. These findings may inform the future development of novel neuromodulatory treatments for cognitive impairment in Parkinson's disease.

The effects of attention in auditory-visual integration revealed by time-varying networks

Attention and audiovisual integration are crucial subjects in the field of brain information processing. A large number of previous studies have sought to determine the relationship between them through specific experiments, but failed to reach a unified conclusion. The reported studies explored the relationship through the frameworks of early, late, and parallel integration, though network analysis has been employed sparingly. In this study, we employed time-varying network analysis, which offers a comprehensive and dynamic insight into cognitive processing, to explore the relationship between attention and auditory-visual integration. The combination of high spatial resolution functional magnetic resonance imaging (fMRI) and high temporal resolution electroencephalography (EEG) was used. Firstly, a generalized linear model (GLM) was employed to find the task-related fMRI activations, which was selected as regions of interesting (ROIs) for nodes of time-varying network. Then the electrical activity of the auditory-visual cortex was estimated via the normalized minimum norm estimation (MNE) source localization method. Finally, the time-varying network was constructed using the adaptive directed transfer function (ADTF) technology. Notably, Task-related fMRI activations were mainly observed in the bilateral temporoparietal junction (TPJ), superior temporal gyrus (STG), primary visual and auditory areas. And the time-varying network analysis revealed that V1/A1↔STG occurred before TPJ↔STG. Therefore, the results supported the theory that auditory-visual integration occurred before attention, aligning with the early integration framework.

Development and validation of an fMRI-informed EEG model of reward-related ventral striatum activation

Reward processing is essential for our mental-health and well-being. In the current study, we developed and validated a scalable, fMRI-informed EEG model for monitoring reward processing related to activation in the ventral-striatum (VS), a significant node in the brain's reward system. To develop this EEG-based model of VS-related activation, we collected simultaneous EEG/fMRI data from 17 healthy individuals while listening to individually-tailored pleasurable music - a highly rewarding stimulus known to engage the VS. Using these cross-modal data, we constructed a generic regression model for predicting the concurrently acquired Blood-Oxygen-Level-Dependent (BOLD) signal from the VS using spectro-temporal features from the EEG signal (termed hereby VS-related-Electrical Finger Print; VS-EFP). The performance of the extracted model was examined using a series of tests that were applied on the original dataset and, importantly, an external validation dataset collected from a different group of 14 healthy individuals who underwent the same EEG/FMRI procedure. Our results showed that the VS-EFP model, as measured by simultaneous EEG, predicted BOLD activation in the VS and additional functionally relevant regions to a greater extent than an EFP model derived from a different anatomical region. The developed VS-EFP was also modulated by musical pleasure and predictive of the VS-BOLD during a monetary reward task, further indicating its functional relevance. These findings provide compelling evidence for the feasibility of using EEG alone to model neural activation related to the VS, paving the way for future use of this scalable neural probing approach in neural monitoring and self-guided neuromodulation.

A revisit of the amygdala theory of autism: Twenty years after

The human amygdala has long been implicated to play a key role in autism spectrum disorder (ASD). Yet it remains unclear to what extent the amygdala accounts for the social dysfunctions in ASD. Here, we review studies that investigate the relationship between amygdala function and ASD. We focus on studies that employ the same task and stimuli to directly compare people with ASD and patients with focal amygdala lesions, and we also discuss functional data associated with these studies. We show that the amygdala can only account for a limited number of deficits in ASD (primarily face perception tasks but not social attention tasks), a network view is, therefore, more appropriate. We next discuss atypical brain connectivity in ASD, factors that can explain such atypical brain connectivity, and novel tools to analyze brain connectivity. Lastly, we discuss new opportunities from multimodal neuroimaging with data fusion and human single-neuron recordings that can enable us to better understand the neural underpinnings of social dysfunctions in ASD. Together, the influential amygdala theory of autism should be extended with emerging data-driven scientific discoveries such as machine learning-based surrogate models to a broader framework that considers brain connectivity at the global scale.

Cognition is entangled with metabolism: relevance for resting-state EEG-fMRI

The brain is a living organ with distinct metabolic constraints. However, these constraints are typically considered as secondary or supportive of information processing which is primarily performed by neurons. The default operational definition of neural information processing is that (1) it is ultimately encoded as a change in individual neuronal firing rate as this correlates with the presentation of a peripheral stimulus, motor action or cognitive task. Two additional assumptions are associated with this default interpretation: (2) that the incessant background firing activity against which changes in activity are measured plays no role in assigning significance to the extrinsically evoked change in neural firing, and (3) that the metabolic energy that sustains this background activity and which correlates with differences in neuronal firing rate is merely a response to an evoked change in neuronal activity. These assumptions underlie the design, implementation, and interpretation of neuroimaging studies, particularly fMRI, which relies on changes in blood oxygen as an indirect measure of neural activity. In this article we reconsider all three of these assumptions in light of recent evidence. We suggest that by combining EEG with fMRI, new experimental work can reconcile emerging controversies in neurovascular coupling and the significance of ongoing, background activity during resting-state paradigms. A new conceptual framework for neuroimaging paradigms is developed to investigate how ongoing neural activity is "entangled" with metabolism. That is, in addition to being recruited to support locally evoked neuronal activity (the traditional hemodynamic response), changes in metabolic support may be independently "invoked" by non-local brain regions, yielding flexible neurovascular coupling dynamics that inform the cognitive context. This framework demonstrates how multimodal neuroimaging is necessary to probe the neurometabolic foundations of cognition, with implications for the study of neuropsychiatric disorders.

Hemodynamic functional connectivity optimization of frequency EEG microstates enables attention LSTM framework to classify distinct temporal cortical communications of different cognitive tasks

Temporal analysis of global cortical communication of cognitive tasks in coarse EEG information is still challenging due to the underlying complex neural mechanisms. This study proposes an attention-based time-series deep learning framework that processes fMRI functional connectivity optimized quasi-stable frequency microstates for classifying distinct temporal cortical communications of the cognitive task. Seventy volunteers were subjected to visual target detection tasks, and their electroencephalogram (EEG) and functional MRI (fMRI) were acquired simultaneously. At first, the acquired EEG information was preprocessed and bandpass to delta, theta, alpha, beta, and gamma bands and then subjected to quasi-stable frequency-microstate estimation. Subsequently, time-series elicitation of each frequency microstates is optimized with graph theory measures of simultaneously eliciting fMRI functional connectivity between frontal, parietal, and temporal cortices. The distinct neural mechanisms associated with each optimized frequency-microstate were analyzed using microstate-informed fMRI. Finally, these optimized, quasi-stable frequency microstates were employed to train and validate the attention-based Long Short-Term Memory (LSTM) time-series architecture for classifying distinct temporal cortical communications of the target from other cognitive tasks. The temporal, sliding input sampling windows were chosen between 180 to 750 ms/segment based on the stability of transition probabilities of the optimized microstates. The results revealed 12 distinct frequency microstates capable of deciphering target detections' temporal cortical communications from other task engagements. Particularly, fMRI functional connectivity measures of target engagement were observed significantly correlated with the right-diagonal delta (r = 0.31), anterior-posterior theta (r = 0.35), left-right theta (r = - 0.32), alpha (r = - 0.31) microstates. Further, neuro-vascular information of microstate-informed fMRI analysis revealed the association of delta/theta and alpha/beta microstates with cortical communications and local neural processing, respectively. The classification accuracies of the attention-based LSTM were higher than the traditional LSTM architectures, particularly the frameworks that sampled the EEG data with a temporal width of 300 ms/segment. In conclusion, the study demonstrates reliable temporal classifications of global cortical communication of distinct tasks using an attention-based LSTM utilizing fMRI functional connectivity optimized quasi-stable frequency microstates.

Extracting electrophysiological correlates of functional magnetic resonance imaging data using the canonical polyadic decomposition

The relation between electrophysiology and BOLD-fMRI requires further elucidation. One approach for studying this relation is to find time-frequency features from electrophysiology that explain the variance of BOLD time-series. Convolution of these features with a canonical hemodynamic response function (HRF) is often required to model neurovascular coupling mechanisms and thus account for time shifts between electrophysiological and BOLD-fMRI data. We propose a framework for extracting the spatial distribution of these time-frequency features while also estimating more flexible, region-specific HRFs. The core component of this method is the decomposition of a tensor containing impulse response functions using the Canonical Polyadic Decomposition. The outputs of this decomposition provide insight into the relation between electrophysiology and BOLD-fMRI and can be used to construct estimates of BOLD time-series. We demonstrated the performance of this method on simulated data while also examining the effects of simulated measurement noise and physiological confounds. Afterwards, we validated our method on publicly available task-based and resting-state EEG-fMRI data. We adjusted our method to accommodate the multisubject nature of these datasets, enabling the investigation of inter-subject variability with regards to EEG-to-BOLD neurovascular coupling mechanisms. We thus also demonstrate how EEG features for modelling the BOLD signal differ across subjects.

Simultaneous electroencephalography-functional magnetic resonance imaging for assessment of human brain function

Electroencephalography (EEG) and functional Magnetic Resonance Imaging (MRI) have long been used as tools to examine brain activity. Since both methods are very sensitive to changes of synaptic activity, simultaneous recording of EEG and fMRI can provide both high temporal and spatial resolution. Therefore, the two modalities are now integrated into a hybrid tool, EEG-fMRI, which encapsulates the useful properties of the two. Among other benefits, EEG-fMRI can contribute to a better understanding of brain connectivity and networks. This review lays its focus on the methodologies applied in performing EEG-fMRI studies, namely techniques used for the recording of EEG inside the scanner, artifact removal, and statistical analysis of the fMRI signal. We will investigate simultaneous resting-state and task-based EEG-fMRI studies and discuss their clinical and technological perspectives. Moreover, it is established that the brain regions affected by a task-based neural activity might not be limited to the regions in which they have been initiated. Advanced methods can help reveal the regions responsible for or affected by a developed neural network. Therefore, we have also looked into studies related to characterization of structure and dynamics of brain networks. The reviewed literature suggests that EEG-fMRI can provide valuable complementary information about brain neural networks and functions.

EEG-fMRI of epileptiform discharges: non-invasive investigation of the whole brain

Simultaneous EEG-fMRI is a unique and non-invasive method for investigating epileptic activity. Interictal epileptiform discharge-related EEG-fMRI provides cortical and subcortical blood oxygen level-dependent (BOLD) signal changes specific to epileptic discharges. As a result, EEG-fMRI has revealed insights into generators and networks involved in epileptic activity in different types of epilepsy, demonstrating-for instance-the implication of the thalamus in human generalized spike and wave discharges and the role of the Default Mode Network (DMN) in absences and focal epilepsy, and proposed a mechanism for the cortico-subcortical interactions in Lennox-Gastaut syndrome discharges. EEG-fMRI can find deep sources of epileptic activity not available to scalp EEG or MEG and provides critical new information to delineate the epileptic focus when considering surgical treatment or electrode implantation. In recent years, methodological advances, such as artifact removal and automatic detection of events have rendered this method easier to implement, and its clinical potential has since been established by evidence of the impact of BOLD response on clinical decision-making and of the relationship between concordance of BOLD responses with extent of resection and surgical outcome. This review presents the recent developments in EEG-fMRI methodology and EEG-fMRI studies in different types of epileptic disorders as follows: EEG-fMRI acquisition, gradient and pulse artifact removal, statistical analysis, clinical applications, pre-surgical evaluation, altered physiological state in generalized genetic epilepsy, and pediatric EEG-fMRI studies.

Forecasting brain activity based on models of spatiotemporal brain dynamics: A comparison of graph neural network architectures

Comprehending the interplay between spatial and temporal characteristics of neural dynamics can contribute to our understanding of information processing in the human brain. Graph neural networks (GNNs) provide a new possibility to interpret graph-structured signals like those observed in complex brain networks. In our study we compare different spatiotemporal GNN architectures and study their ability to model neural activity distributions obtained in functional MRI (fMRI) studies. We evaluate the performance of the GNN models on a variety of scenarios in MRI studies and also compare it to a VAR model, which is currently often used for directed functional connectivity analysis. We show that by learning localized functional interactions on the anatomical substrate, GNN-based approaches are able to robustly scale to large network studies, even when available data are scarce. By including anatomical connectivity as the physical substrate for information propagation, such GNNs also provide a multimodal perspective on directed connectivity analysis, offering a novel possibility to investigate the spatiotemporal dynamics in brain networks.

High-Density Electroencephalography-Informed Multiband Functional Magnetic Resonance Imaging Reveals Rhythm-Specific Activations Within the Trigeminal Nociceptive Network

The interest in exploring trigeminal pain processing has grown in recent years, mainly due to various pathologies (such as migraine) related to this system. However, research efforts have mainly focused on understanding molecular mechanisms or studying pathological states. On the contrary, non-invasive imaging studies are limited by either spatial or temporal resolution depending on the modality used. This can be overcome by using multimodal imaging techniques such as simultaneous functional magnetic resonance imaging (fMRI) and electroencephalography (EEG). Although this technique has already been applied to neuroscientific research areas and consequently gained insights into diverse sensory systems and pathologies, only a few studies have applied EEG-fMRI in the field of pain processing and none in the trigeminal system. Focusing on trigeminal nociception, we used a trigeminal pain paradigm, which has been well-studied in either modality. For validation, we first acquired stand-alone measures with each imaging modality before fusing them in a simultaneous session. Furthermore, we introduced a new, yet simple, non-parametric correlation technique, which exploits trial-to-trial variance of both measurement techniques with Spearman’s correlations, to consolidate the results gained by the two modalities. This new technique does not presume a linear relationship and needs a few repetitions per subject. We also showed cross-validation by analyzing visual stimulations. Using these techniques, we showed that EEG power changes in the theta-band induced by trigeminal pain correlate with fMRI activation within the brainstem, whereas those of gamma-band oscillations correlate with BOLD signals in higher cortical areas.

Gustatory Cortex Is Involved in Evidence Accumulation during Food Choice

Food choice is one of the most fundamental and most frequent value-based decisions for all animals including humans. However, the neural circuitry involved in food-based decisions is only recently being addressed. Given the relatively fast dynamics of decision formation, electroencephalography (EEG)-informed fMRI analysis is highly beneficial for localizing this circuitry in humans. Here, by using the EEG correlates of evidence accumulation in a simultaneously recorded EEG-fMRI dataset, we found a significant role for the right temporal-parietal operculum (PO) and medial insula including gustatory cortex (GC) in binary choice between food items. These activations were uncovered by using the “EEG energy” (power 2 of EEG) as the BOLD regressor and were missed if conventional analysis with the EEG signal itself were to be used, in agreement with theoretical predictions for EEG and BOLD relations. No significant positive correlations were found with higher powers of EEG (powers 3 or 4) pointing to specificity and sufficiency of EEG energy as the main correlate of the BOLD response. This finding extends the role of cortical areas traditionally involved in palatability processing to value-based decision-making and offers the “EEG energy” as a key regressor of BOLD response in simultaneous EEG-fMRI designs.

Modeling the Hemodynamic Response Function Using EEG-fMRI Data During Eyes-Open Resting-State Conditions and Motor Task Execution

Being able to accurately quantify the hemodynamic response function (HRF) that links the blood oxygen level dependent functional magnetic resonance imaging (BOLD-fMRI) signal to the underlying neural activity is important both for elucidating neurovascular coupling mechanisms and improving the accuracy of fMRI-based functional connectivity analyses. In particular, HRF estimation using BOLD-fMRI is challenging particularly in the case of resting-state data, due to the absence of information about the underlying neuronal dynamics. To this end, using simultaneously recorded electroencephalography (EEG) and fMRI data is a promising approach, as EEG provides a more direct measure of neural activations. In the present work, we employ simultaneous EEG-fMRI to investigate the regional characteristics of the HRF using measurements acquired during resting conditions. We propose a novel methodological approach based on combining distributed EEG source space reconstruction, which improves the spatial resolution of HRF estimation and using block-structured linear and nonlinear models, which enables us to simultaneously obtain HRF estimates and the contribution of different EEG frequency bands. Our results suggest that the dynamics of the resting-state BOLD signal can be sufficiently described using linear models and that the contribution of each band is region specific. Specifically, it was found that sensory-motor cortices exhibit positive HRF shapes, whereas the lateral occipital cortex and areas in the parietal cortex, such as the inferior and superior parietal lobule exhibit negative HRF shapes. To validate the proposed method, we repeated the analysis using simultaneous EEG-fMRI measurements acquired during execution of a unimanual hand-grip task. Our results reveal significant associations between BOLD signal variations and electrophysiological power fluctuations in the ipsilateral primary motor cortex, particularly for the EEG beta band, in agreement with previous studies in the literature.

Simultaneous EEG-fMRI: What Have We Learned and What Does the Future Hold?

Simultaneous EEG-fMRI has developed into a mature measurement technique in the past 25 years. During this time considerable technical and analytical advances have been made, enabling valuable scientific contributions to a range of research fields. This review will begin with an introduction to the measurement principles involved in EEG and fMRI and the advantages of combining these methods. The challenges faced when combining the two techniques will then be considered. An overview of the leading application fields where EEG-fMRI has made a significant contribution to the scientific literature and emerging applications in EEG-fMRI research trends is then presented.

EEG-fMRI: Ballistocardiogram Artifact Reduction by Surrogate Method for Improved Source Localization

For the analysis of simultaneous EEG-fMRI recordings, it is vital to use effective artifact removal tools. This applies in particular to the ballistocardiogram (BCG) artifact which is difficult to remove without distorting signals of interest related to brain activity. Here, we documented the use of surrogate source models to separate the artifact-related signals from brain signals with minimal distortion of the brain activity of interest. The artifact topographies used for surrogate separation were created automatically using principal components analysis (PCA-S) or by manual selection of artifact components utilizing independent components analysis (ICA-S). Using real resting-state data from 55 subjects superimposed with simulated auditory evoked potentials (AEP), both approaches were compared with three established BCG artifact removal methods: Blind Source Separation (BSS), Optimal Basis Set (OBS), and a mixture of both (OBS-ICA). Each method was evaluated for its applicability for ERP and source analysis using the following criteria: the number of events surviving artifact threshold scans, signal-to-noise ratio (SNR), error of source localization, and signal variance explained by the dipolar model. Using these criteria, PCA-S and ICA-S fared best overall, with highly significant differences to the established methods, especially in source localization. The PCA-S approach was also applied to a single subject Berger experiment performed in the MRI scanner. Overall, the removal of BCG artifacts by the surrogate methods provides a substantial improvement for the analysis of simultaneous EEG-fMRI data compared to the established methods.

Optimizing EEG Source Reconstruction with Concurrent fMRI-Derived Spatial Priors

Reconstructing EEG sources involves a complex pipeline, with the inverse problem being the most challenging. Multiple inversion algorithms are being continuously developed, aiming to tackle the non-uniqueness of this problem, which has been shown to be partially circumvented by including prior information in the inverse models. Despite a few efforts, there are still current and persistent controversies regarding the inversion algorithm of choice and the optimal set of spatial priors to be included in the inversion models. The use of simultaneous EEG-fMRI data is one approach to tackle this problem. The spatial resolution of fMRI makes fMRI derived spatial priors very convenient for EEG reconstruction, however, only task activation maps and resting-state networks (RSNs) have been explored so far, overlooking the recent, but already accepted, notion that brain networks exhibit dynamic functional connectivity fluctuations. The lack of a systematic comparison between different source reconstruction algorithms, considering potentially more brain-informative priors such as fMRI, motivates the search for better reconstruction models. Using simultaneous EEG-fMRI data, here we compared four different inversion algorithms (minimum norm, MN; low resolution electromagnetic tomography, LORETA; empirical Bayes beamformer, EBB; and multiple sparse priors, MSP) under a Bayesian framework (as implemented in SPM), each with three different sets of priors consisting of: (1) those specific to the algorithm; (2) those specific to the algorithm plus fMRI task activation maps and RSNs; and (3) those specific to the algorithm plus fMRI task activation maps and RSNs and network modules of task-related dFC states estimated from the dFC fluctuations. The quality of the reconstructed EEG sources was quantified in terms of model-based metrics, namely the expectation of the posterior probability P(model|data) and variance explained of the inversion models, and the overlap/proportion of brain regions known to be involved in the visual perception tasks that the participants were submitted to, and RSN templates, with/within EEG source components. Model-based metrics suggested that model parsimony is preferred, with the combination MSP and priors specific to this algorithm exhibiting the best performance. However, optimal overlap/proportion values were found using EBB and priors specific to this algorithm and fMRI task activation maps and RSNs or MSP and considering all the priors (algorithm priors, fMRI task activation maps and RSNs and dFC state modules), respectively, indicating that fMRI spatial priors, including dFC state modules, might contain useful information to recover EEG source components reflecting neuronal activity of interest. Our main results show that providing fMRI spatial derived priors that reflect the dynamics of the brain might be useful to map neuronal activity more accurately from EEG-fMRI. Furthermore, this work paves the way towards a more informative selection of the optimal EEG source reconstruction approach, which may be critical in future studies.

MRI magnetic compatible electrical neural interface: From materials to application

Neural electrical interfaces are important tools for local neural stimulation and recording, which potentially have wide application in the diagnosis and treatment of neural diseases, as well as in the transmission of neural activity for brain-computer interface (BCI) systems. At the same time, magnetic resonance imaging (MRI) is one of the effective and non-invasive techniques for recording whole-brain signals, providing details of brain structures and also activation pattern maps. Simultaneous recording of extracellular neural signals and MRI combines two expressions of the same neural activity and is believed to be of great importance for the understanding of brain function. However, this combination makes requests on the magnetic and electronic performance of neural interface devices. MRI-compatibility refers here to a technological approach to simultaneous MRI and electrode recording or stimulation without artifacts in imaging. Trade-offs between materials magnetic susceptibility selection and electrical function should be considered. Herein, prominent trends in selecting materials of suitable magnetic properties are analyzed and material design, function and application of neural interfaces are outlined together with the remaining challenge to fabricate MRI-compatible neural interface.

Oscillatory electroencephalographic patterns of arithmetic problem solving in fourth graders

Numerous studies have identified neurophysiological correlates of performing arithmetic in adults. For example, oscillatory electroencephalographic (EEG) patterns associated with retrieval and procedural strategies are well established. Whereas fact retrieval has been linked to enhanced left-hemispheric theta ERS (event-related synchronization), procedural strategies are accompanied by increased bilateral alpha ERD (event-related desynchronization). It is currently not clear if these findings generalize to children. Our study is the first to investigate oscillatory EEG activity related to strategy use and arithmetic operations in children. We assessed ERD/ERS correlates of 31 children in fourth grade (aged between nine and ten years) during arithmetic problem solving. We presented multiplication and subtraction problems, which children solved with fact retrieval or a procedure. We analyzed these four problem categories (retrieved multiplications, retrieved subtractions, procedural multiplications, and procedural subtractions) in our study. In summary, we found similar strategy-related patterns to those reported in previous studies with adults. That is, retrieval problems elicited stronger left-hemispheric theta ERS and weaker alpha ERD as compared to procedural problems. Interestingly, we observed neurophysiological differences between multiplications and subtractions within retrieval problems. Although there were no response time or accuracy differences, retrieved multiplications were accompanied by larger theta ERS than retrieved subtractions. This finding could indicate that retrieval of multiplication and subtraction facts are distinct processes, and/or that multiplications are more frequently retrieved than subtractions in this age group.

Stationary EEG pattern relates to large-scale resting state networks - An EEG-fMRI study connecting brain networks across time-scales

Relating brain dynamics acting on time scales that differ by at least an order of magnitude is a fundamental issue in brain research. The same is true for the observation of stable dynamical structures in otherwise highly non-stationary signals. The present study addresses both problems by the analysis of simultaneous resting state EEG-fMRI recordings of 53 patients with epilepsy. Confirming previous findings, we observe a generic and temporally stable average correlation pattern in EEG recordings. We design a predictor for the General Linear Model describing fluctuations around the stationary EEG correlation pattern and detect resting state networks in fMRI data. The acquired statistical maps are contrasted to several surrogate tests and compared with maps derived by spatial Independent Component Analysis of the fMRI data. By means of the proposed EEG-predictor we observe core nodes of known fMRI resting state networks with high specificity in the default mode, the executive control and the salience network. Our results suggest that both, the stationary EEG pattern as well as resting state fMRI networks are different expressions of the same brain activity. This activity is interpreted as the dynamics on (or close to) a stable attractor in phase space that is necessary to maintain the brain in an efficient operational mode. We discuss that this interpretation is congruent with the theoretical framework of complex systems as well as with the brain's energy balance.

EEG microstate analysis of emotion regulation reveals no sequential processing of valence and emotional arousal

In electroencephalography (EEG), microstates are distributions of activity across the scalp that persist for several tens of milliseconds before changing into a different pattern. Microstate analysis is a way of utilizing EEG as both temporal and spatial imaging tool, but has rarely been applied to task-based data. This study aimed to conceptually replicate microstate findings of valence and emotional arousal processing and investigate the effects of emotion regulation on microstates, using data of an EEG paradigm with 107 healthy adults who actively viewed emotional pictures, cognitively detached from them, or suppressed facial reactions. Within the first 600 ms after stimulus onset only the comparison of viewing positive and negative pictures yielded significant results, caused by different electrodes depending on the microstate. Since the microstates associated with more and less emotionally arousing pictures did not differ, sequential processing could not be replicated. When extending the analysis to 2000 ms after stimulus onset, differences were exclusive to the comparison of viewing and detaching from negative pictures. Intriguingly, we observed the novel phenomenon of a microstate difference that could not be attributed to single electrodes. This suggests that microstate analysis can detect differences beyond those detected by event-related potential analysis.

Student's Learning Strategies and Academic Emotions: Their Influence on Learning Satisfaction During the COVID-19 Pandemic

Background: Based on the control-value theory (CVT), learning strategies and academic emotions are closely related to learning achievement, and have been considered as important factors influencing student's learning satisfaction and learning performance in the online learning context. However, only a few studies have focused on the influence of learning strategies on academic emotions and the interaction of learning strategies with behavioral engagement and social interaction on learning satisfaction. Methods: The participants were 363 pre-service teachers in China, and we used structural equation modeling (SEM) to analyze the mediating and moderating effects of the data. Results: The main findings of the current study showed that learning strategies influence students' online learning satisfaction through academic emotions. The interaction between learning strategies and behavioral engagement was also an important factor influencing online learning satisfaction. Conclusions: We explored the internal mechanism and boundary conditions of how learning strategies influenced learning satisfaction to provide intellectual guarantee and theoretical support for the online teaching design and online learning platform. This study provides theoretical contributions to the CVT and practical value for massive open online courses (MOOCs), flipped classrooms and blended learning in the future.

Association of EEG-fMRI Responses and Outcome After Epilepsy Surgery

BACKGROUND AND OBJECTIVES

To assess the utility of EEG-fMRI for epilepsy surgery, we evaluated surgical outcome in relation to the resection of the most significant EEG-fMRI response.

METHODS

Patients with postoperative neuroimaging and follow-up of at least 1 year were included. In EEG-fMRI responses, we defined as primary the cluster with the highest absolute t value located in the cortex and evaluated 3 levels of confidence for the results. The threshold for low confidence was t ≥ 3.1 (p < 0.005); the one for medium confidence corresponded to correction for multiple comparisons with a false discovery rate of 0.05; and a result reached high confidence when the primary cluster was much more significant than the next highest cluster. Concordance with the resection was determined by comparison to postoperative neuroimaging.

RESULTS

We evaluated 106 epilepsy surgeries in 84 patients. An increasing association between concordance and surgical outcome with higher levels of confidence was demonstrated. If the peak response was not resected, the surgical outcome was likely to be poor: for the high confidence level, no patient had a good outcome; for the medium and low levels, only 18% and 28% had a good outcome. The positive predictive value remained low for all confidence levels, indicating that removing the maximum cluster did not ensure seizure freedom.

DISCUSSION

Resection of the primary EEG-fMRI cluster, especially in high confidence cases, is necessary to obtain a good outcome but not sufficient.

CLASSIFICATION OF EVIDENCE

This study provides Class II evidence that failure to resect the primary EEG-fMRI cluster is associated with poorer epilepsy surgery outcomes.

Unified Retrospective EEG Motion Educated Artefact Suppression for EEG-fMRI to Suppress Magnetic Field Gradient Artefacts During Motion

The data quality of simultaneously acquired electroencephalography and functional magnetic resonance imaging (EEG-fMRI) can be strongly affected by motion. Recent work has shown that the quality of fMRI data can be improved by using a Moiré-Phase-Tracker (MPT)-camera system for prospective motion correction. The use of the head position acquired by the MPT-camera-system has also been shown to correct motion-induced voltages, ballistocardiogram (BCG) and gradient artefact residuals separately. In this work we show the concept of an integrated framework based on the general linear model to provide a unified motion informed model of in-MRI artefacts. This model (retrospective EEG motion educated gradient artefact suppression, REEG-MEGAS) is capable of correcting voltage-induced, BCG and gradient artefact residuals of EEG data acquired simultaneously with prospective motion corrected fMRI. In our results, we have verified that applying REEG-MEGAS correction to EEG data acquired during subject motion improves the data quality in terms of motion induced voltages and also GA residuals in comparison to standard Artefact Averaging Subtraction and Retrospective EEG Motion Artefact Suppression. Besides that, we provide preliminary evidence that although adding more regressors to a model may slightly affect the power of physiological signals such as the alpha-rhythm, its application may increase the overall quality of a dataset, particularly when strongly affected by motion. This was verified by analysing the EEG traces, power spectra density and the topographic distribution from two healthy subjects. We also have verified that the correction by REEG-MEGAS improves higher frequency artefact correction by decreasing the power of Gradient Artefact harmonics. Our method showed promising results for decreasing the power of artefacts for frequencies up to 250 Hz. Additionally, REEG-MEGAS is a hybrid framework that can be implemented for real time prospective motion correction of EEG and fMRI data. Among other EEG-fMRI applications, the approach described here may benefit applications such as EEG-fMRI neurofeedback and brain computer interface, which strongly rely on the prospective acquisition and application of motion artefact removal.

Removal of physiological artifacts from simultaneous EEG and fMRI recordings

OBJECTIVE

Simultaneous recording of the electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI) allows a combination of eletrophysiological and haemodynamic information to be used to form a more complete picture of cerebral dynamics. However, EEG recorded within the MRI scanner is contaminated by both imaging artifacts and physiological artifacts. The majority of the techniques used to pre-process such EEG focus on removal of the imaging and balistocardiogram artifacts, with some success, but don't remove all other physiological artifacts.

METHODS

We propose a new offline EEG artifact removal method based upon a combination of independent component analysis and fMRI-based head movement estimation to aid the removal of physiological artifacts from EEG recorded during EEG-fMRI recordings. Our method makes novel use of head movement trajectories estimated from the fMRI recording in order to assist with identifying physiological artifacts in the EEG and is designed to be used after removal of the fMRI imaging artifact from the EEG.

RESULTS

We evaluate our method on EEG recorded during a joint EEG-fMRI session from healthy adult participants. Our method significantly reduces the influence of all types of physiological artifacts on the EEG. We also compare our method with a state-of-the-art physiological artifact removal method and demonstrate superior performance removing physiological artifacts.

CONCLUSIONS

Our proposed method is able to remove significantly more physiological artifact components from the EEG, recorded during a joint EEG-fMRI session, than other state-of-the-art methods.

SIGNIFICANCE

Our proposed method represents a marked improvement over current processing pipelines for removing physiological noise from EEG recorded during a joint EEG-fMRI session.

Recent Advances and Future Directions in Brain MR Imaging Studies in Schizophrenia: Toward Elucidating Brain Pathology and Developing Clinical Tools

Schizophrenia is a common severe psychiatric disorder that affects approximately 1% of general population through the life course. Historically, in Kraepelin's time, schizophrenia was a disease unit conceptualized as dementia praecox; however, since then, the disease concept has changed. Recent MRI studies had shown that the neuropathology of the brain in this disorder was characterized by mild progression before and after the onset of the disease, and that the brain alterations were relatively smaller than assumed. Although genetic factors contribute to the brain alterations in schizophrenia, which are thought to be trait differences, other changes include factors that are common in psychiatric diseases. Furthermore, it has been shown that the brain differences specific to schizophrenia were relatively small compared to other changes, such as those caused by brain development, aging, and gender. In addition, compared to the disease and participant factors, machine and imaging protocol differences could affect MRI signals, which should be addressed in multi-site studies. Recent advances in MRI modalities, such as multi-shell diffusion-weighted imaging, magnetic resonance spectroscopy, and multimodal brain imaging analysis, may be candidates to sharpen the characterization of schizophrenia-specific factors and provide new insights. The Brain/MINDS Beyond Human Brain MRI (BMB-HBM) project has been launched considering the differences and noises irrespective of the disease pathologies and includes the future perspectives of MRI studies for various psychiatric and neurological disorders. The sites use restricted MRI machines and harmonized multi-modal protocols, standardized image preprocessing, and traveling subject harmonization. Data sharing to the public will be planned in FY 2024. In the future, we believe that combining a high-quality human MRI dataset with genetic data, randomized controlled trials, and MRI for non-human primates and animal models will enable us to understand schizophrenia, elucidate its neural bases and therapeutic targets, and provide tools for clinical application at bedside.

Epilepsy under the scope of ultra-high field MRI

Ultra-high field magnetic resonance imaging (UHF-MRI) is capable of unraveling anatomical structures in a submillimeter range. In addition, its high resonance regime allows the quantification of constitutive molecules in a spatially sensitive manner, a crucial capability for determining the extent and localization of a probable epileptogenic region or the severity of the epilepsy. The main technical challenges for data acquisition under UHF are to produce a strong, homogeneous transverse field, while keeping the tissue power deposition within the safe regulatory guidelines. The nonuniformities caused by destructive and constructive interferences at UHFs required new technologies to accelerate and increase yield regarding time spent and quality achieved. Image quality is the paramount contribution of UHF high-resolution imaging, which is capable to disclose fine details of the hippocampal formation and its surroundings and their changes in the course of epilepsy. Other sequences like diffusion tensor imaging (DTI) and multiecho susceptibility imaging at 7 T in vivo can assist the creation of normative atlases of the hippocampal subfields or the reconstruction of the highly arborized cerebral blood vessels. In our review, we specify the impact of these advanced relevant techniques onto the study of epilepsy. In this context, we focused onto high field high-resolution scanners and clinically-enriched decision-making. Studies on focal dysplasias correlating ex vivo high-resolution imaging with specific histological and ultrastructural patterns showed that white matter hyperintensities were related to a demyelination process and other alterations. Preliminary results correlating thick serial sections through bioptic epileptogenic tissue could extend the strategy to localize degenerated tissue sectors, correlate nature and extent of tissue loss with preoperative diagnosis and postoperative outcome. Finally, this protocol will provide the neurosurgeon with a detailed depiction of the removed pathologic tissue and possible adverse effects by the pathologic tissue left in situ. This article is part of the special issue "NEWroscience 2018".

Screening Tools and Assessment Methods of Cognitive Decline Associated With Age-Related Hearing Loss: A Review

Strong links between hearing and cognitive function have been confirmed by a growing number of cross-sectional and longitudinal studies. Seniors with age-related hearing loss (ARHL) have a significantly higher cognitive impairment incidence than those with normal hearing. The correlation mechanism between ARHL and cognitive decline is not fully elucidated to date. However, auditory intervention for patients with ARHL may reduce the risk of cognitive decline, as early cognitive screening may improve related treatment strategies. Currently, clinical audiology examinations rarely include cognitive screening tests, partly due to the lack of objective quantitative indicators with high sensitivity and specificity. Questionnaires are currently widely used as a cognitive screening tool, but the subject's performance may be negatively affected by hearing loss. Numerous electroencephalogram (EEG) and magnetic resonance imaging (MRI) studies analyzed brain structure and function changes in patients with ARHL. These objective electrophysiological tools can be employed to reveal the association mechanism between auditory and cognitive functions, which may also find biological markers to be more extensively applied in assessing the progression towards cognitive decline and observing the effects of rehabilitation training for patients with ARHL. In this study, we reviewed clinical manifestations, pathological changes, and causes of ARHL and discussed their cognitive function effects. Specifically, we focused on current cognitive screening tools and assessment methods and analyzed their limitations and potential integration.

When Is Simultaneous Recording Necessary? A Guide for Researchers Considering Combined EEG-fMRI

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) provide non-invasive measures of brain activity at varying spatial and temporal scales, offering different views on brain function for both clinical and experimental applications. Simultaneous recording of these measures attempts to maximize the respective strengths of each method, while compensating for their weaknesses. However, combined recording is not necessary to address all research questions of interest, and experiments may have greater statistical power to detect effects by maximizing the signal-to-noise ratio in separate recording sessions. While several existing papers discuss the reasons for or against combined recording, this article aims to synthesize these arguments into a flow chart of questions that researchers can consider when deciding whether to record EEG and fMRI separately or simultaneously. Given the potential advantages of simultaneous EEG-fMRI, the aim is to provide an initial overview of the most important concepts and to direct readers to relevant literature that will aid them in this decision.

Neural Correlates of Motor Recovery after Robot-Assisted Training in Chronic Stroke: A Multimodal Neuroimaging Study

Stroke is a leading cause of motor disability worldwide, and robot-assisted therapies have been increasingly applied to facilitate the recovery process. However, the underlying mechanism and induced neuroplasticity change remain partially understood, and few studies have investigated this from a multimodality neuroimaging perspective. The current study adopted BCI-guided robot hand therapy as the training intervention and combined multiple neuroimaging modalities to comprehensively understand the potential association between motor function alteration and various neural correlates. We adopted EEG-informed fMRI technique to understand the functional regions sensitive to training intervention. Additionally, correlation analysis among training effects, nonlinear property change quantified by fractal dimension (FD), and integrity of M1-M1 (M1: primary motor cortex) anatomical connection were performed. EEG-informed fMRI analysis indicated that for iM1 (iM1: ipsilesional M1) regressors, regions with significantly increased partial correlation were mainly located in contralesional parietal, prefrontal, and sensorimotor areas and regions with significantly decreased partial correlation were mainly observed in the ipsilesional supramarginal gyrus and superior temporal gyrus. Pearson's correlations revealed that the interhemispheric asymmetry change significantly correlated with the training effect as well as the integrity of M1-M1 anatomical connection. In summary, our study suggested that multiple functional brain regions not limited to motor areas were involved during the recovery process from multimodality perspective. The correlation analyses suggested the essential role of interhemispheric interaction in motor rehabilitation. Besides, the underlying structural substrate of the bilateral M1-M1 connection might relate to the interhemispheric change. This study might give some insights in understanding the neuroplasticity induced by the integrated BCI-guided robot hand training intervention and further facilitate the design of therapies for chronic stroke patients.

Data-driven beamforming technique to attenuate ballistocardiogram artefacts in electroencephalography-functional magnetic resonance imaging without detecting cardiac pulses in electrocardiography recordings

Simultaneous recording of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) is a very promising non‐invasive neuroimaging technique. However, EEG data obtained from the simultaneous EEG–fMRI are strongly influenced by MRI‐related artefacts, namely gradient artefacts (GA) and ballistocardiogram (BCG) artefacts. When compared to the GA correction, the BCG correction is more challenging to remove due to its inherent variabilities and dynamic changes over time. The standard BCG correction (i.e., average artefact subtraction [AAS]), require detecting cardiac pulses from simultaneous electrocardiography (ECG) recording. However, ECG signals are also distorted and will become problematic for detecting reliable cardiac peaks. In this study, we focused on a beamforming spatial filtering technique to attenuate all unwanted source activities outside of the brain. Specifically, we applied the beamforming technique to attenuate the BCG artefact in EEG–fMRI, and also to recover meaningful task‐based neural signals during an attentional network task (ANT) which required participants to identify visual cues and respond accurately. We analysed EEG–fMRI data in 20 healthy participants during the ANT, and compared four different BCG corrections (non‐BCG corrected, AAS BCG corrected, beamforming + AAS BCG corrected, beamforming BCG corrected). We demonstrated that the beamforming approach did not only significantly reduce the BCG artefacts, but also significantly recovered the expected task‐based brain activity when compared to the standard AAS correction. This data‐driven beamforming technique appears promising especially for longer data acquisition of sleep and resting EEG–fMRI. Our findings extend previous work regarding the recovery of meaningful EEG signals by an optimized suppression of MRI‐related artefacts.

Interictal Epileptiform Discharge Dynamics in Peri-sylvian Polymicrogyria Using EEG-fMRI

Polymicrogyria (PMG) is a common malformation of cortical development associated with a higher susceptibility to epileptic seizures. Seizures secondary to PMG are characterized by difficult-to-localize cerebral sources due to the complex and widespread lesion structure. Tracing the dynamics of interictal epileptiform discharges (IEDs) in patients with epilepsy has been shown to reveal the location of epileptic activity sources, crucial for successful treatment in cases of focal drug-resistant epilepsy. In this case series IED dynamics were evaluated with simultaneous EEG-fMRI recordings in four patients with unilateral peri-sylvian polymicrogyria (PSPMG) by tracking BOLD activations over time: before, during and following IED appearance on scalp EEG. In all cases, focal BOLD activations within the lesion itself preceded the activity associated with the time of IED appearance on EEG, which showed stronger and more widespread activations. We therefore propose that early hemodynamic activity corresponding to IEDs may hold important localizing information potentially leading to the cerebral sources of epileptic activity. IEDs are suggested to develop within a small area in the PSPMG lesion with structural properties obscuring the appearance of their electric field on the scalp and only later engage widespread structures which allow the production of large currents which are recognized as IEDs on EEG.

Brain Connectivity Studies on Structure-Function Relationships: A Short Survey with an Emphasis on Machine Learning

This short survey reviews the recent literature on the relationship between the brain structure and its functional dynamics. Imaging techniques such as diffusion tensor imaging (DTI) make it possible to reconstruct axonal fiber tracks and describe the structural connectivity (SC) between brain regions. By measuring fluctuations in neuronal activity, functional magnetic resonance imaging (fMRI) provides insights into the dynamics within this structural network. One key for a better understanding of brain mechanisms is to investigate how these fast dynamics emerge on a relatively stable structural backbone. So far, computational simulations and methods from graph theory have been mainly used for modeling this relationship. Machine learning techniques have already been established in neuroimaging for identifying functionally independent brain networks and classifying pathological brain states. This survey focuses on methods from machine learning, which contribute to our understanding of functional interactions between brain regions and their relation to the underlying anatomical substrate.

On the Role of Bilateral Brain Hypofunction and Abnormal Lateralization of Cortical Information Flow as Neural Underpinnings of Conventional Metaphor Processing Impairment in Schizophrenia: An fMRI and EEG Study

Figurative language processing (e.g. metaphors) is commonly impaired in schizophrenia. In the present study, we investigated the neural activity and propagation of information within neural circuits related to the figurative speech, as a neural substrate of impaired conventional metaphor processing in schizophrenia. The study included 30 schizophrenia outpatients and 30 healthy controls, all of whom were assessed with a functional Magnetic Resonance Imaging (fMRI) and electroencephalography (EEG) punchline-based metaphor comprehension task including literal (neutral), figurative (metaphorical) and nonsense (absurd) endings. The blood oxygenation level-dependent signal was recorded with 3T MRI scanner and direction and strength of cortical information flow in the time course of task processing was estimated with a 64-channel EEG input for directed transfer function. The presented results revealed that the behavioral manifestation of impaired figurative language in schizophrenia is related to the hypofunction in the bilateral fronto-temporo-parietal brain regions (fMRI) and various differences in effective connectivity in the fronto-temporo-parietal circuit (EEG). Schizophrenia outpatients showed an abnormal pattern of connectivity during metaphor processing which was related to bilateral (but more pronounced at the left hemisphere) hypoactivation of the brain. Moreover, we found reversed lateralization patterns, i.e. a rightward-shifted pattern during metaphor processing in schizophrenia compared to the control group. In conclusion, the presented findings revealed that the impairment of the conventional metaphor processing in schizophrenia is related to the bilateral brain hypofunction, which supports the evidence on reversed lateralization of the language neural network and the existence of compensatory recruitment of alternative neural circuits in schizophrenia.

fMRI and transcranial electrical stimulation (tES): A systematic review of parameter space and outcomes

The combination of non-invasive brain stimulation interventions with human brain mapping methods have supported research beyond correlational associations between brain activity and behavior. Functional MRI (fMRI) partnered with transcranial electrical stimulation (tES) methods, i.e., transcranial direct current (tDCS), transcranial alternating current (tACS), and transcranial random noise (tRNS) stimulation, explore the neuromodulatory effects of tES in the targeted brain regions and their interconnected networks and provide opportunities for individualized interventions. Advances in the field of tES-fMRI can be hampered by the methodological variability between studies that confounds comparability/replicability. In order to explore variability in the tES-fMRI methodological parameter space (MPS), we conducted a systematic review of 222 tES-fMRI experiments (181 tDCS, 39 tACS and 2 tRNS) published before February 1, 2019, and suggested a framework to systematically report main elements of MPS across studies. Publications dedicated to tRNS-fMRI were not considered in this systematic review. We have organized main findings in terms of fMRI modulation by tES. tES modulates activation and connectivity beyond the stimulated areas particularly with prefrontal stimulation. There were no two studies with the same MPS to replicate findings. We discuss how to harmonize the MPS to promote replication in future studies.

Inferring Macroscale Brain Dynamics via Fusion of Simultaneous EEG-fMRI

Advances in the instrumentation and signal processing for simultaneously acquired electroencephalography and functional magnetic resonance imaging (EEG-fMRI) have enabled new ways to observe the spatiotemporal neural dynamics of the human brain. Central to the utility of EEG-fMRI neuroimaging systems are the methods for fusing the two data streams, with machine learning playing a key role. These methods can be dichotomized into those that are symmetric and asymmetric in terms of how the two modalities inform the fusion. Studies using these methods have shown that fusion yields new insights into brain function that are not possible when each modality is acquired separately. As technology improves and methods for fusion become more sophisticated, the future of EEG-fMRI for noninvasive measurement of brain dynamics includes mesoscale mapping at ultrahigh magnetic resonance fields, targeted perturbation-based neuroimaging, and using deep learning to uncover nonlinear representations that link the electrophysiological and hemodynamic measurements.

Multimodal EEG-fMRI: advancing insight into large-scale human brain dynamics

Advances in the acquisition and analysis of functional magnetic resonance imaging (fMRI) data are revealing increasingly rich spatiotemporal structure across the human brain. Nonetheless, uncertainty surrounding the origins of fMRI hemodynamic signals, and in the link between large-scale fMRI patterns and ongoing functional states, presently limits the neurobiological conclusions one can draw from fMRI alone. Electroencephalography (EEG) provides complementary information about neural electrical activity and state change, and simultaneously acquiring EEG together with fMRI presents unique opportunities for studying large-scale brain activity and gaining more information from fMRI itself. Here, we discuss recent progress in the use of concurrent EEG-fMRI to enrich the investigation of neural and physiological states and clarify the origins of fMRI hemodynamic signals. Throughout, we outline perspectives on future directions and open challenges.

Artifact Reduction in Simultaneous EEG-fMRI: A Systematic Review of Methods and Contemporary Usage

Simultaneous electroencephalography-functional MRI (EEG-fMRI) is a technique that combines temporal (largely from EEG) and spatial (largely from fMRI) indicators of brain dynamics. It is useful for understanding neuronal activity during many different event types, including spontaneous epileptic discharges, the activity of sleep stages, and activity evoked by external stimuli and decision-making tasks. However, EEG recorded during fMRI is subject to imaging, pulse, environment and motion artifact, causing noise many times greater than the neuronal signals of interest. Therefore, artifact removal methods are essential to ensure that artifacts are accurately removed, and EEG of interest is retained. This paper presents a systematic review of methods for artifact reduction in simultaneous EEG-fMRI from literature published since 1998, and an additional systematic review of EEG-fMRI studies published since 2016. The aim of the first review is to distill the literature into clear guidelines for use of simultaneous EEG-fMRI artifact reduction methods, and the aim of the second review is to determine the prevalence of artifact reduction method use in contemporary studies. We find that there are many published artifact reduction techniques available, including hardware, model based, and data-driven methods, but there are few studies published that adequately compare these methods. In contrast, recent EEG-fMRI studies show overwhelming use of just one or two artifact reduction methods based on literature published 15–20 years ago, with newer methods rarely gaining use outside the group that developed them. Surprisingly, almost 15% of EEG-fMRI studies published since 2016 fail to adequately describe the methods of artifact reduction utilized. We recommend minimum standards for reporting artifact reduction techniques in simultaneous EEG-fMRI studies and suggest that more needs to be done to make new artifact reduction techniques more accessible for the researchers and clinicians using simultaneous EEG-fMRI.

The relationship between EEG and fMRI connectomes is reproducible across simultaneous EEG-fMRI studies from 1.5T to 7T

Both electroencephalography (EEG) and functional Magnetic Resonance Imaging (fMRI) are non-invasive methods that show complementary aspects of human brain activity. Despite measuring different proxies of brain activity, both the measured blood-oxygenation (fMRI) and neurophysiological recordings (EEG) are indirectly coupled. The electrophysiological and BOLD signal can map the underlying functional connectivity structure at the whole brain scale at different timescales. Previous work demonstrated a moderate but significant correlation between resting-state functional connectivity of both modalities, however there is a wide range of technical setups to measure simultaneous EEG-fMRI and the reliability of those measures between different setups remains unknown. This is true notably with respect to different magnetic field strengths (low and high field) and different spatial sampling of EEG (medium to high-density electrode coverage). Here, we investigated the reproducibility of the bimodal EEG-fMRI functional connectome in the most comprehensive resting-state simultaneous EEG-fMRI dataset compiled to date including a total of 72 subjects from four different imaging centers. Data was acquired from 1.5T, 3T and 7T scanners with simultaneously recorded EEG using 64 or 256 electrodes. We demonstrate that the whole-brain monomodal connectivity reproducibly correlates across different datasets and that a moderate crossmodal correlation between EEG and fMRI connectivity of r ≈ 0.3 can be reproducibly extracted in low- and high-field scanners. The crossmodal correlation was strongest in the EEG-β frequency band but exists across all frequency bands. Both homotopic and within intrinsic connectivity network (ICN) connections contributed the most to the crossmodal relationship. This study confirms, using a considerably diverse range of recording setups, that simultaneous EEG-fMRI offers a consistent estimate of multimodal functional connectomes in healthy subjects that are dominantly linked through a functional core of ICNs across spanning across the different timescales measured by EEG and fMRI. This opens new avenues for estimating the dynamics of brain function and provides a better understanding of interactions between EEG and fMRI measures. This observed level of reproducibility also defines a baseline for the study of alterations of this coupling in pathological conditions and their role as potential clinical markers.