EEG-fMRI integration: a critical review of biophysical modeling and data analysis approaches

Distinct basal ganglia contributions to learning from implicit and explicit value signals in perceptual decision-making

Metacognitive evaluations of confidence provide an estimate of decision accuracy that could guide learning in the absence of explicit feedback. We examine how humans might learn from this implicit feedback in direct comparison with that of explicit feedback, using simultaneous EEG-fMRI. Participants performed a motion direction discrimination task where stimulus difficulty was increased to maintain performance, with intermixed explicit- and no-feedback trials. We isolate single-trial estimates of post-decision confidence using EEG decoding, and find these neural signatures re-emerge at the time of feedback together with separable signatures of explicit feedback. We identified these signatures of implicit versus explicit feedback along a dorsal-ventral gradient in the striatum, a finding uniquely enabled by an EEG-fMRI fusion. These two signals appear to integrate into an aggregate representation in the external globus pallidus, which could broadcast updates to improve cortical decision processing via the thalamus and insular cortex, irrespective of the source of feedback.

Deep EEG source localization via EMD-based fMRI high spatial frequency

Brain imaging with a high-spatiotemporal resolution is crucial for accurate brain-function mapping. Electroencephalography (EEG) and functional Magnetic Resonance Imaging (fMRI) are two popular neuroimaging modalities with complementary features that record brain function with high temporal and spatial resolution, respectively. One popular non-invasive way to obtain data with both high spatial and temporal resolutions is to combine the fMRI activation map and EEG data to improve the spatial resolution of the EEG source localization. However, using the whole fMRI map may cause spurious results for the EEG source localization, especially for deep brain regions. Considering the head's conductivity, deep regions' sources with low activity are unlikely to be detected by the EEG electrodes at the scalp. In this study, we use fMRI's high spatial-frequency component to identify the local high-intensity activations that are most likely to be captured by the EEG. The 3D Empirical Mode Decomposition (3D-EMD), a data-driven method, is used to decompose the fMRI map into its spatial-frequency components. Different validation measurements for EEG source localization show improved performance for the EEG inverse-modeling informed by the fMRI's high-frequency spatial component compared to the fMRI-informed EEG source-localization methods. The level of improvement varies depending on the voxels' intensity and their distribution. Our experimental results also support this conclusion.

A revisit of the amygdala theory of autism: Twenty years after

The human amygdala has long been implicated to play a key role in autism spectrum disorder (ASD). Yet it remains unclear to what extent the amygdala accounts for the social dysfunctions in ASD. Here, we review studies that investigate the relationship between amygdala function and ASD. We focus on studies that employ the same task and stimuli to directly compare people with ASD and patients with focal amygdala lesions, and we also discuss functional data associated with these studies. We show that the amygdala can only account for a limited number of deficits in ASD (primarily face perception tasks but not social attention tasks), a network view is, therefore, more appropriate. We next discuss atypical brain connectivity in ASD, factors that can explain such atypical brain connectivity, and novel tools to analyze brain connectivity. Lastly, we discuss new opportunities from multimodal neuroimaging with data fusion and human single-neuron recordings that can enable us to better understand the neural underpinnings of social dysfunctions in ASD. Together, the influential amygdala theory of autism should be extended with emerging data-driven scientific discoveries such as machine learning-based surrogate models to a broader framework that considers brain connectivity at the global scale.

Minimizing the distortions in electrophysiological source imaging of cortical oscillatory activity via Spectral Structured Sparse Bayesian Learning

Oscillatory processes at all spatial scales and on all frequencies underpin brain function. Electrophysiological Source Imaging (ESI) is the data-driven brain imaging modality that provides the inverse solutions to the source processes of the EEG, MEG, or ECoG data. This study aimed to carry out an ESI of the source cross-spectrum while controlling common distortions of the estimates. As with all ESI-related problems under realistic settings, the main obstacle we faced is a severely ill-conditioned and high-dimensional inverse problem. Therefore, we opted for Bayesian inverse solutions that posited a priori probabilities on the source process. Indeed, rigorously specifying both the likelihoods and a priori probabilities of the problem leads to the proper Bayesian inverse problem of cross-spectral matrices. These inverse solutions are our formal definition for cross-spectral ESI (cESI), which requires a priori of the source cross-spectrum to counter the severe ill-condition and high-dimensionality of matrices. However, inverse solutions for this problem were NP-hard to tackle or approximated within iterations with bad-conditioned matrices in the standard ESI setup. We introduce cESI with a joint a priori probability upon the source cross-spectrum to avoid these problems. cESI inverse solutions are low-dimensional ones for the set of random vector instances and not random matrices. We achieved cESI inverse solutions through the variational approximations via our Spectral Structured Sparse Bayesian Learning (ssSBL) algorithm https://github.com/CCC-members/Spectral-Structured-Sparse-Bayesian-Learning. We compared low-density EEG (10-20 system) ssSBL inverse solutions with reference cESIs for two experiments: (a) high-density MEG that were used to simulate EEG and (b) high-density macaque ECoG that were recorded simultaneously with EEG. The ssSBL resulted in two orders of magnitude with less distortion than the state-of-the-art ESI methods. Our cESI toolbox, including the ssSBL method, is available at https://github.com/CCC-members/BC-VARETA_Toolbox.

Source localization and functional network analysis in emotion cognitive reappraisal with EEG-fMRI integration

Background

The neural activity and functional networks of emotion-based cognitive reappraisal have been widely investigated using electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). However, single-mode neuroimaging techniques are limited in exploring the regulation process with high temporal and spatial resolution.

Objectives

We proposed a source localization method with multimodal integration of EEG and fMRI and tested it in the source-level functional network analysis of emotion cognitive reappraisal.

Methods

EEG and fMRI data were simultaneously recorded when 15 subjects were performing the emotional cognitive reappraisal task. Fused priori weighted minimum norm estimation (FWMNE) with sliding windows was proposed to trace the dynamics of EEG source activities, and the phase lag index (PLI) was used to construct the functional brain network associated with the process of downregulating negative affect using the reappraisal strategy.

Results

The functional networks were constructed with the measure of PLI, in which the important regions were indicated. In the gamma band source-level network analysis, the cuneus, the lateral orbitofrontal cortex, the superior parietal cortex, the postcentral gyrus, and the pars opercularis were identified as important regions in reappraisal with high betweenness centrality.

Conclusion

The proposed multimodal integration method for source localization identified the key cortices involved in emotion regulation, and the network analysis demonstrated the important brain regions involved in the cognitive control of reappraisal. It shows promise in the utility in the clinical setting for affective disorders.

Multiple-Kernel Support Vector Machine for Predicting Internet Gaming Disorder Using Multimodal Fusion of PET, EEG, and Clinical Features

Internet gaming disorder (IGD) has become an important social and psychiatric issue in recent years. To prevent IGD and provide the appropriate intervention, an accurate prediction method for identifying IGD is necessary. In this study, we investigated machine learning methods of multimodal neuroimaging data including Positron Emission Tomography (PET), Electroencephalography (EEG), and clinical features to enhance prediction accuracy. Unlike the conventional methods which usually concatenate all features into one feature vector, we adopted a multiple-kernel support vector machine (MK-SVM) to classify IGD. We compared the prediction performance of standard machine learning methods such as SVM, random forest, and boosting with the proposed method in patients with IGD (N = 28) and healthy controls (N = 24). We showed that the prediction accuracy of the optimal MK-SVM using three kinds of modalities was much higher than other conventional machine learning methods, with the highest accuracy being 86.5%, the sensitivity 89.3%, and the specificity 83.3%. Furthermore, we deduced that clinical variables had the highest contribution to the optimal IGD prediction model and that the other two modalities were also indispensable. We found that more efficient integration of multimodal data through kernel combination could contribute to better performance of the prediction model. This study is a novel attempt to integrate each method from different sources and suggests that integrating each method, such as self-administrated reports, PET, and EEG, improves the prediction of IGD.

Modeling the Hemodynamic Response Function Using EEG-fMRI Data During Eyes-Open Resting-State Conditions and Motor Task Execution

Being able to accurately quantify the hemodynamic response function (HRF) that links the blood oxygen level dependent functional magnetic resonance imaging (BOLD-fMRI) signal to the underlying neural activity is important both for elucidating neurovascular coupling mechanisms and improving the accuracy of fMRI-based functional connectivity analyses. In particular, HRF estimation using BOLD-fMRI is challenging particularly in the case of resting-state data, due to the absence of information about the underlying neuronal dynamics. To this end, using simultaneously recorded electroencephalography (EEG) and fMRI data is a promising approach, as EEG provides a more direct measure of neural activations. In the present work, we employ simultaneous EEG-fMRI to investigate the regional characteristics of the HRF using measurements acquired during resting conditions. We propose a novel methodological approach based on combining distributed EEG source space reconstruction, which improves the spatial resolution of HRF estimation and using block-structured linear and nonlinear models, which enables us to simultaneously obtain HRF estimates and the contribution of different EEG frequency bands. Our results suggest that the dynamics of the resting-state BOLD signal can be sufficiently described using linear models and that the contribution of each band is region specific. Specifically, it was found that sensory-motor cortices exhibit positive HRF shapes, whereas the lateral occipital cortex and areas in the parietal cortex, such as the inferior and superior parietal lobule exhibit negative HRF shapes. To validate the proposed method, we repeated the analysis using simultaneous EEG-fMRI measurements acquired during execution of a unimanual hand-grip task. Our results reveal significant associations between BOLD signal variations and electrophysiological power fluctuations in the ipsilateral primary motor cortex, particularly for the EEG beta band, in agreement with previous studies in the literature.

When Is Simultaneous Recording Necessary? A Guide for Researchers Considering Combined EEG-fMRI

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) provide non-invasive measures of brain activity at varying spatial and temporal scales, offering different views on brain function for both clinical and experimental applications. Simultaneous recording of these measures attempts to maximize the respective strengths of each method, while compensating for their weaknesses. However, combined recording is not necessary to address all research questions of interest, and experiments may have greater statistical power to detect effects by maximizing the signal-to-noise ratio in separate recording sessions. While several existing papers discuss the reasons for or against combined recording, this article aims to synthesize these arguments into a flow chart of questions that researchers can consider when deciding whether to record EEG and fMRI separately or simultaneously. Given the potential advantages of simultaneous EEG-fMRI, the aim is to provide an initial overview of the most important concepts and to direct readers to relevant literature that will aid them in this decision.

Inferring Macroscale Brain Dynamics via Fusion of Simultaneous EEG-fMRI

Advances in the instrumentation and signal processing for simultaneously acquired electroencephalography and functional magnetic resonance imaging (EEG-fMRI) have enabled new ways to observe the spatiotemporal neural dynamics of the human brain. Central to the utility of EEG-fMRI neuroimaging systems are the methods for fusing the two data streams, with machine learning playing a key role. These methods can be dichotomized into those that are symmetric and asymmetric in terms of how the two modalities inform the fusion. Studies using these methods have shown that fusion yields new insights into brain function that are not possible when each modality is acquired separately. As technology improves and methods for fusion become more sophisticated, the future of EEG-fMRI for noninvasive measurement of brain dynamics includes mesoscale mapping at ultrahigh magnetic resonance fields, targeted perturbation-based neuroimaging, and using deep learning to uncover nonlinear representations that link the electrophysiological and hemodynamic measurements.

Breath-hold task induces temporal heterogeneity in electroencephalographic regional field power in healthy subjects

We demonstrated that changes in CO₂ values cause oscillations in the cortical activity in δ-and α-bands. The analysis of the regional field power (RFP) showed evidence that different cortical areas respond with different time delays to CO₂ challenges. An opposite behavior was found for the end-tidal O₂. We suppose that the different cortical time delays likely expresse specific ascending pathways to the cortex, generated by chemoreceptor nuclei in the brain stem. Although the brain stem is in charge of the automatic control of ventilation, the cortex is involved in the voluntary control of breathing but also receives inputs from the brain stem, which influences the perception of breathing, the arousal state and sleep architecture in conditions of hypoxia/hypercapnia. We evaluated in 11 healthy subjects the effects of breath hold (BH; 30 s of apneas and 30 s of normal breathing) and BH-related CO₂/O₂ changes on electroencephalogram (EEG) global field power (GFP) and RFP in nine different areas (3 rostrocaudal sections: anterior, central, and posterior; and 3 sagittal sections: left, middle, and right) in the δ- and α-bands by cross correlation analysis. No significant differences were observed in GFP or RFP when comparing free breathing (FB) with the BH task. Within the BH task, the shift from apnea to normal ventilation was accompanied by an increase in the δ-power and a decrease in the α-power. The end-tidal pressure of CO₂ ([Formula: see text]) was positively correlated with the δ-band and negatively with the α- band with a positive time shift, whereas an opposite behavior was found for the end-tidal pressure of O₂ ([Formula: see text]). Notably, the time shift between [Formula: see text] / [Formula: see text] signals and cortical activity at RFP was heterogenous and seemed to follow a hierarchical activation, with the δ-band responding earlier than the α-band. Overall, these findings suggest that the effect of BH on the cortex may follow specific ascending pathways from the brain stem and be related to chemoreflex stimulation.NEW & NOTEWORTHY We demonstrated that the end tidal CO₂ oscillation causes oscillations of delta and alpha bands. The analysis of the regional field power showed that different cortical areas respond with different time delays to CO₂ challenges. An opposite behavior was found for the end-tidal O₂. We can suppose that the different cortical time delay response likely expresses specific ascending pathways to the cortex generated by chemoreceptor nuclei in the brainstem.

The Locus Coeruleus- Norepinephrine System in Stress and Arousal: Unraveling Historical, Current, and Future Perspectives

Arousal may be understood on a spectrum, with excessive sleepiness, cognitive dysfunction, and inattention on one side, a wakeful state in the middle, and hypervigilance, panic, and psychosis on the other side. However, historically, the concepts of arousal and stress have been challenging to define as measurable experimental variables. Divergent efforts to study these subjects have given rise to several disciplines, including neurobiology, neuroendocrinology, and cognitive neuroscience. We discuss technological advancements that chronologically led to our current understanding of the arousal system, focusing on the multifaceted nucleus locus coeruleus. We share our contemporary perspective and the hypotheses of others in the context of our current technological capabilities and future developments that will be required to move forward in this area of research.

Concurrent electrophysiological and hemodynamic measurements of evoked neural oscillations in human visual cortex using sparsely interleaved fast fMRI and EEG

Electroencephalography (EEG) concurrently collected with functional magnetic resonance imaging (fMRI) is heavily distorted by the repetitive gradient coil switching during the fMRI acquisition. The performance of the typical template-based gradient artifact suppression method can be suboptimal because the artifact changes over time. Gradient artifact residuals also impede the subsequent suppression of ballistocardiography artifacts. Here we propose recording continuous EEG with temporally sparse fast fMRI (fast fMRI-EEG) to minimize the EEG artifacts caused by MRI gradient coil switching without significantly compromising the field-of-view and spatiotemporal resolution of fMRI. Using simultaneous multi-slice inverse imaging to achieve whole-brain fMRI with isotropic 5-mm resolution in 0.1 ​s, and performing these acquisitions once every 2 ​s, we have 95% of the duty cycle available to record EEG with substantially less gradient artifact. We found that the standard deviation of EEG signals over the entire acquisition period in fast fMRI-EEG was reduced to 54% of that in conventional concurrent echo-planar imaging (EPI) and EEG recordings (EPI-EEG) across participants. When measuring 15-Hz steady-state visual evoked potentials (SSVEPs), the baseline-normalized oscillatory neural response in fast fMRI-EEG was 2.5-fold of that in EPI-EEG. The functional MRI responses associated with the SSVEP delineated by EPI and fast fMRI were similar in the spatial distribution, the elicited waveform, and detection power. Sparsely interleaved fast fMRI-EEG provides high-quality EEG without substantially compromising the quality of fMRI in evoked response measurements, and has the potential utility for applications where the onset of the target stimulus cannot be precisely determined, such as epilepsy.

Simultaneous EEG-fMRI during a neurofeedback task, a brain imaging dataset for multimodal data integration

Combining EEG and fMRI allows for integration of fine spatial and accurate temporal resolution yet presents numerous challenges, noticeably if performed in real-time to implement a Neurofeedback (NF) loop. Here we describe a multimodal dataset of EEG and fMRI acquired simultaneously during a motor imagery NF task, supplemented with MRI structural data. The study involved 30 healthy volunteers undergoing five training sessions. We showed the potential and merit of simultaneous EEG-fMRI NF in previous work. Here we illustrate the type of information that can be extracted from this dataset and show its potential use. This represents one of the first simultaneous recording of EEG and fMRI for NF and here we present the first open access bi-modal NF dataset integrating EEG and fMRI. We believe that it will be a valuable tool to (1) advance and test methodologies for multi-modal data integration, (2) improve the quality of NF provided, (3) improve methodologies for de-noising EEG acquired under MRI and (4) investigate the neuromarkers of motor-imagery using multi-modal information.

Modeling functional resting-state brain networks through neural message passing on the human connectome

In this work, we propose a natural model for information flow in the brain through a neural message-passing dynamics on a structural network of macroscopic regions, such as the human connectome (HC). In our model, each brain region is assumed to have a binary behavior (active or not), the strengths of interactions among them are encoded in the anatomical connectivity matrix defined by the HC, and the dynamics of the system is defined by the Belief Propagation (BP) algorithm, working near the critical point of the network. We show that in the absence of direct external stimuli the BP algorithm converges to a spatial map of activations that is similar to the Default Mode Network (DMN) of the brain, which has been defined from the analysis of functional MRI data. Moreover, we use Susceptibility Propagation (SP) to compute the matrix of long-range correlations between the different regions and show that the modules defined by a clustering of this matrix resemble several Resting State Networks (RSN) determined experimentally. Both results suggest that the functional DMN and RSNs can be seen as simple consequences of the anatomical structure of the brain and a neural message-passing dynamics between macroscopic regions. With the new model, we explore predictions on how functional maps change when the anatomical brain network suffers structural alterations, like in Alzheimer's disease and in lesions of the Corpus Callosum. The implications and novel interpretations suggested by the model, as well as the role of criticality, are discussed.

Method for spatial overlap estimation of electroencephalography and functional magnetic resonance imaging responses

BACKGROUND

Simultaneous functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) measurements may represent activity from partially divergent neural sources, but this factor is seldom modeled in fMRI-EEG data integration.

NEW METHOD

This paper proposes an approach to estimate the spatial overlap between sources of activity measured simultaneously with fMRI and EEG. Following the extraction of task-related activity, the key steps include, 1) distributed source reconstruction of the task-related ERP activity (ERP source model), 2) transformation of fMRI activity to the ERP spatial scale by forward modelling of the scalp potential field distribution and backward source reconstruction (fMRI source simulation), and 3) optimization of fMRI and ERP thresholds to maximize spatial overlap without a priori constraints of coupling (overlap calculation).

RESULTS

FMRI and ERP responses were recorded simultaneously in 15 subjects performing an auditory oddball task. A high degree of spatial overlap between sources of fMRI and ERP responses (in 9 or more of 15 subjects) was found specifically within temporoparietal areas associated with the task. Areas of non-overlap in fMRI and ERP sources were relatively small and inconsistent across subjects.

COMPARISON WITH EXISTING METHOD

The ERP and fMRI sources estimated with solely jICA overlapped in just 4 of 15 subjects, and strictly in the parietal cortex.

CONCLUSION

The study demonstrates that the new fMRI-ERP spatial overlap estimation method provides greater spatiotemporal detail of the cortical dynamics than solely jICA. As such, we propose that it is a superior method for the integration of fMRI and EEG to study brain function.

To integrate or not to integrate: Temporal dynamics of hierarchical Bayesian causal inference

To form a percept of the environment, the brain needs to solve the binding problem-inferring whether signals come from a common cause and are integrated or come from independent causes and are segregated. Behaviourally, humans solve this problem near-optimally as predicted by Bayesian causal inference; but the neural mechanisms remain unclear. Combining Bayesian modelling, electroencephalography (EEG), and multivariate decoding in an audiovisual spatial localisation task, we show that the brain accomplishes Bayesian causal inference by dynamically encoding multiple spatial estimates. Initially, auditory and visual signal locations are estimated independently; next, an estimate is formed that combines information from vision and audition. Yet, it is only from 200 ms onwards that the brain integrates audiovisual signals weighted by their bottom-up sensory reliabilities and top-down task relevance into spatial priority maps that guide behavioural responses. As predicted by Bayesian causal inference, these spatial priority maps take into account the brain's uncertainty about the world's causal structure and flexibly arbitrate between sensory integration and segregation. The dynamic evolution of perceptual estimates thus reflects the hierarchical nature of Bayesian causal inference, a statistical computation, which is crucial for effective interactions with the environment.

Reliability and Generalizability of Similarity-Based Fusion of MEG and fMRI Data in Human Ventral and Dorsal Visual Streams

To build a representation of what we see, the human brain recruits regions throughout the visual cortex in cascading sequence. Recently, an approach was proposed to evaluate the dynamics of visual perception in high spatiotemporal resolution at the scale of the whole brain. This method combined functional magnetic resonance imaging (fMRI) data with magnetoencephalography (MEG) data using representational similarity analysis and revealed a hierarchical progression from primary visual cortex through the dorsal and ventral streams. To assess the replicability of this method, we here present the results of a visual recognition neuro-imaging fusion experiment and compare them within and across experimental settings. We evaluated the reliability of this method by assessing the consistency of the results under similar test conditions, showing high agreement within participants. We then generalized these results to a separate group of individuals and visual input by comparing them to the fMRI-MEG fusion data of Cichy et al (2016), revealing a highly similar temporal progression recruiting both the dorsal and ventral streams. Together these results are a testament to the reproducibility of the fMRI-MEG fusion approach and allows for the interpretation of these spatiotemporal dynamic in a broader context.

Multimodal Neuroimaging: Basic Concepts and Classification of Neuropsychiatric Diseases

Neuroimaging techniques are widely used in neuroscience to visualize neural activity, to improve our understanding of brain mechanisms, and to identify biomarkers-especially for psychiatric diseases; however, each neuroimaging technique has several limitations. These limitations led to the development of multimodal neuroimaging (MN), which combines data obtained from multiple neuroimaging techniques, such as electroencephalography, functional magnetic resonance imaging, and yields more detailed information about brain dynamics. There are several types of MN, including visual inspection, data integration, and data fusion. This literature review aimed to provide a brief summary and basic information about MN techniques (data fusion approaches in particular) and classification approaches. Data fusion approaches are generally categorized as asymmetric and symmetric. The present review focused exclusively on studies based on symmetric data fusion methods (data-driven methods), such as independent component analysis and principal component analysis. Machine learning techniques have recently been introduced for use in identifying diseases and biomarkers of disease. The machine learning technique most widely used by neuroscientists is classification-especially support vector machine classification. Several studies differentiated patients with psychiatric diseases and healthy controls with using combined datasets. The common conclusion among these studies is that the prediction of diseases increases when combining data via MN techniques; however, there remain a few challenges associated with MN, such as sample size. Perhaps in the future N-way fusion can be used to combine multiple neuroimaging techniques or nonimaging predictors (eg, cognitive ability) to overcome the limitations of MN.

Artifacts in EEG of simultaneous EEG-fMRI: pulse artifact remainders in the gradient artifact template are a source of artifact residuals after average artifact subtraction

OBJECTIVE

The simultaneous application of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) opens up new ways to investigate the human brain. The EEG recordings of simultaneous EEG-fMRI, however, are overlaid to a great degree by fMRI related artifacts and an artifact reduction is mandatory before any EEG analysis. The most severe artifacts-the gradient artifact and the pulse artifact-are repetitive. Average artifact subtraction (AAS) technique exploits the repetitiveness and is presumably the most often used artifact reduction technique. In this method artifact templates are calculated by averaging over adjacent artifact epochs and subsequently the templates are subtracted to reduce the artifacts. Although the AAS technique is one of the best performing methods, artifact residuals are usually present in the resulting EEG after applying the AAS technique. This work aims at identifying sources of the artifact residuals.

APPROACH

Application of the AAS technique to artificial EEG that is contaminated with artificial fMRI related artifacts.

MAIN RESULTS

A new source of artifact residuals was identified. It was found that the AAS technique itself adds artifacts to the EEG during gradient artifact reduction, because the gradient artifact template is corrupted by pulse artifact remainders.

SIGNIFICANCE

This work shows that using a standard number of 25 epochs to calculate the gradient artifact template-as suggested by the inventors of AAS-results in substantial artifact residuals and consequently to a low EEG quality. Furthermore, the work discusses how potential solutions to this problem have serious side effects such as loss of adaptivity of the AAS technique. Hence, this problem must be considered carefully already in the design of simultaneous EEG-fMRI experiments.

Neuroscience Information Toolbox: An Open Source Toolbox for EEG-fMRI Multimodal Fusion Analysis

Recently, scalp electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) multimodal fusion has been pursued in an effort to study human brain function and dysfunction to obtain more comprehensive information on brain activity in which the spatial and temporal resolutions are both satisfactory. However, a more flexible and easy-to-use toolbox for EEG–fMRI multimodal fusion is still lacking. In this study, we therefore developed a freely available and open-source MATLAB graphical user interface toolbox, known as the Neuroscience Information Toolbox (NIT), for EEG–fMRI multimodal fusion analysis. The NIT consists of three modules: (1) the fMRI module, which has batch fMRI preprocessing, nuisance signal removal, bandpass filtering, and calculation of resting-state measures; (2) the EEG module, which includes artifact removal, extracting EEG features (event onset, power, and amplitude), and marking interesting events; and (3) the fusion module, in which fMRI-informed EEG analysis and EEG-informed fMRI analysis are included. The NIT was designed to provide a convenient and easy-to-use toolbox for researchers, especially for novice users. The NIT can be downloaded for free at http://www.neuro.uestc.edu.cn/NIT.html, and detailed information, including the introduction of NIT, user’s manual and example data sets, can also be observed on this website. We hope that the NIT is a promising toolbox for exploring brain information in various EEG and fMRI studies.

Concurrent EEG and Functional MRI Recording and Integration Analysis for Dynamic Cortical Activity Imaging

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) are two of the fundamental noninvasive methods for identifying brain activity. Multimodal methods have sought to combine the high temporal resolution of EEG with the spatial precision of fMRI, but the complexity of this approach is currently in need of improvement. The protocol presented here describes the recently developed spatiotemporal fMRI-constrained EEG source imaging method, which seeks to rectify source biases and improve EEG-fMRI source localization through the dynamic recruitment of fMRI sub-regions. The process begins with the collection of multimodal data from concurrent EEG and fMRI scans, the generation of 3D cortical models, and independent EEG and fMRI processing. The processed fMRI activation maps are then split into multiple priors, according to their location and surrounding area. These are taken as priors in a two-level hierarchical Bayesian algorithm for EEG source localization. For each window of interest (defined by the operator), specific segments of the fMRI activation map will be identified as active to optimize a parameter known as model evidence. These will be used as soft constraints on the identified cortical activity, increasing the specificity of the multimodal imaging method by reducing cross-talk and avoiding erroneous activity in other conditionally active fMRI regions. The method generates cortical maps of activity and time-courses, which may be taken as final results, or used as a basis for further analyses (analyses of correlation, causation, etc.) While the method is somewhat limited by its modalities (it will not find EEG-invisible sources), it is broadly compatible with most major processing software, and is suitable for most neuroimaging studies.

Multimodal approaches to functional connectivity in autism spectrum disorders: An integrative perspective

Atypical functional connectivity has been implicated in autism spectrum disorders (ASDs). However, the literature to date has been largely inconsistent, with mixed and conflicting reports of hypo- and hyper-connectivity. These discrepancies are partly due to differences between various neuroimaging modalities. Functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and magnetoencephalography (MEG) measure distinct indices of functional connectivity (e.g., blood-oxygenation level-dependent [BOLD] signal vs. electrical activity). Furthermore, each method has unique benefits and disadvantages with respect to spatial and temporal resolution, vulnerability to specific artifacts, and practical implementation. Thus far, functional connectivity research on ASDs has remained almost exclusively unimodal; therefore, interpreting findings across modalities remains a challenge. Multimodal integration of fMRI, EEG, and MEG data is critical in resolving discrepancies in the literature, and working toward a unifying framework for interpreting past and future findings. This review aims to provide a theoretical foundation for future multimodal research on ASDs. First, we will discuss the merits and shortcomings of several popular theories in ASD functional connectivity research, using examples from the literature to date. Next, the neurophysiological relationships between imaging modalities, including their relationship with invasive neural recordings, will be reviewed. Finally, methodological approaches to multimodal data integration will be presented, and their future application to ASDs will be discussed. Analyses relating transient patterns of neural activity ("states") are particularly promising. This strategy provides a comparable measure across modalities, captures complex spatiotemporal patterns, and is a natural extension of recent dynamic fMRI research in ASDs. © 2017 Wiley Periodicals, Inc. Develop Neurobiol 78: 456-473, 2018.

Detailed spatiotemporal brain mapping of chromatic vision combining high-resolution VEP with fMRI and retinotopy

Neuroimaging studies have identified so far, several color-sensitive visual areas in the human brain, and the temporal dynamics of these activities have been separately investigated using the visual-evoked potentials (VEPs). In the present study, we combined electrophysiological and neuroimaging methods to determine a detailed spatiotemporal profile of chromatic VEP and to localize its neural generators. The accuracy of the present co-registration study was obtained by combining standard fMRI data with retinotopic and motion mapping data at the individual level. We found a sequence of occipito activities more complex than that typically reported for chromatic VEPs, including feed-forward and reentrant feedback. Results showed that chromatic human perception arises by the combined activity of at the least five parieto-occipital areas including V1, LOC, V8/VO, and the motion-sensitive dorsal region MT+. However, the contribution of V1 and V8/VO seems dominant because the re-entrant activity in these areas was present more than once (twice in V8/VO and thrice in V1). This feedforward and feedback chromatic processing appears delayed compared with the luminance processing. Associating VEPs and neuroimaging measures, we showed for the first time a complex spatiotemporal pattern of activity, confirming that chromatic stimuli produce intricate interactions of many different brain dorsal and ventral areas.

EEG-Informed fMRI: A Review of Data Analysis Methods

The simultaneous acquisition of electroencephalography (EEG) with functional magnetic resonance imaging (fMRI) is a very promising non-invasive technique for the study of human brain function. Despite continuous improvements, it remains a challenging technique, and a standard methodology for data analysis is yet to be established. Here we review the methodologies that are currently available to address the challenges at each step of the data analysis pipeline. We start by surveying methods for pre-processing both EEG and fMRI data. On the EEG side, we focus on the correction for several MR-induced artifacts, particularly the gradient and pulse artifacts, as well as other sources of EEG artifacts. On the fMRI side, we consider image artifacts induced by the presence of EEG hardware inside the MR scanner, and the contamination of the fMRI signal by physiological noise of non-neuronal origin, including a review of several approaches to model and remove it. We then provide an overview of the approaches specifically employed for the integration of EEG and fMRI when using EEG to predict the blood oxygenation level dependent (BOLD) fMRI signal, the so-called EEG-informed fMRI integration strategy, the most commonly used strategy in EEG-fMRI research. Finally, we systematically review methods used for the extraction of EEG features reflecting neuronal phenomena of interest.

Optimizing Within-Subject Experimental Designs for jICA of Multi-Channel ERP and fMRI

Joint independent component analysis (jICA) can be applied within subject for fusion of multi-channel event-related potentials (ERP) and functional magnetic resonance imaging (fMRI), to measure brain function at high spatiotemporal resolution (Mangalathu-Arumana et al., 2012). However, the impact of experimental design choices on jICA performance has not been systematically studied. Here, the sensitivity of jICA for recovering neural sources in individual data was evaluated as a function of imaging SNR, number of independent representations of the ERP/fMRI data, relationship between instantiations of the joint ERP/fMRI activity (linear, non-linear, uncoupled), and type of sources (varying parametrically and non-parametrically across representations of the data), using computer simulations. Neural sources were simulated with spatiotemporal and noise attributes derived from experimental data. The best performance, maximizing both cross-modal data fusion and the separation of brain sources, occurred with a moderate number of representations of the ERP/fMRI data (10-30), as in a mixed block/event related experimental design. Importantly, the type of relationship between instantiations of the ERP/fMRI activity, whether linear, non-linear or uncoupled, did not in itself impact jICA performance, and was accurately recovered in the common profiles (i.e., mixing coefficients). Thus, jICA provides an unbiased way to characterize the relationship between ERP and fMRI activity across brain regions, in individual data, rendering it potentially useful for characterizing pathological conditions in which neurovascular coupling is adversely affected.

Correlation of BOLD Signal with Linear and Nonlinear Patterns of EEG in Resting State EEG-Informed fMRI

Concurrent EEG and fMRI acquisitions in resting state showed a correlation between EEG power in various bands and spontaneous BOLD fluctuations. However, there is a lack of data on how changes in the complexity of brain dynamics derived from EEG reflect variations in the BOLD signal. The purpose of our study was to correlate both spectral patterns, as linear features of EEG rhythms, and nonlinear EEG dynamic complexity with neuronal activity obtained by fMRI. We examined the relationships between EEG patterns and brain activation obtained by simultaneous EEG-fMRI during the resting state condition in 25 healthy right-handed adult volunteers. Using EEG-derived regressors, we demonstrated a substantial correlation of BOLD signal changes with linear and nonlinear features of EEG. We found the most significant positive correlation of fMRI signal with delta spectral power. Beta and alpha spectral features had no reliable effect on BOLD fluctuation. However, dynamic changes of alpha peak frequency exhibited a significant association with BOLD signal increase in right-hemisphere areas. Additionally, EEG dynamic complexity as measured by the HFD of the 2-20 Hz EEG frequency range significantly correlated with the activation of cortical and subcortical limbic system areas. Our results indicate that both spectral features of EEG frequency bands and nonlinear dynamic properties of spontaneous EEG are strongly associated with fluctuations of the BOLD signal during the resting state condition.

Concurrent measurement of cerebral hemodynamics and electroencephalography during transcranial direct current stimulation

Transcranial direct current stimulation (tDCS) is currently being used for research and treatment of some neurological and neuropsychiatric disorders, as well as for improvement of cognitive functions. In order to better understand cerebral response to the stimulation and to redefine protocols and dosage, its effects must be monitored. To this end, we have used functional diffuse correlation spectroscopy (fDCS) and time-resolved functional near-infrared spectroscopy (TR-fNIRS) together with electroencephalography (EEG) during and after stimulation of the frontal cortex. Twenty subjects participated in two sessions of stimulation with two different polarity montages and twelve also underwent a sham session. Cerebral blood flow and oxyhemoglobin concentration increased during and after active stimulation in the region under the stimulation electrode while deoxyhemoglobin concentration decreased. The EEG spectrum displayed statistically significant power changes across different stimulation sessions in delta (2 to 4 Hz), theta (4 to 8 Hz), and beta (12 to 18 Hz) bands. Results suggest that fDCS and TR-fNIRS can be employed as neuromonitors of the effects of transcranial electrical stimulation and can be used together with EEG.

The roadmap for estimation of cell-type-specific neuronal activity from non-invasive measurements

The computational properties of the human brain arise from an intricate interplay between billions of neurons connected in complex networks. However, our ability to study these networks in healthy human brain is limited by the necessity to use non-invasive technologies. This is in contrast to animal models where a rich, detailed view of cellular-level brain function with cell-type-specific molecular identity has become available due to recent advances in microscopic optical imaging and genetics. Thus, a central challenge facing neuroscience today is leveraging these mechanistic insights from animal studies to accurately draw physiological inferences from non-invasive signals in humans. On the essential path towards this goal is the development of a detailed 'bottom-up' forward model bridging neuronal activity at the level of cell-type-specific populations to non-invasive imaging signals. The general idea is that specific neuronal cell types have identifiable signatures in the way they drive changes in cerebral blood flow, cerebral metabolic rate of O2 (measurable with quantitative functional Magnetic Resonance Imaging), and electrical currents/potentials (measurable with magneto/electroencephalography). This forward model would then provide the 'ground truth' for the development of new tools for tackling the inverse problem-estimation of neuronal activity from multimodal non-invasive imaging data.This article is part of the themed issue 'Interpreting BOLD: a dialogue between cognitive and cellular neuroscience'.

Complementary contributions of concurrent EEG and fMRI connectivity for predicting structural connectivity

While averaged dynamics of brain function are known to estimate the underlying structure, the exact relationship between large-scale function and structure remains an unsolved issue in network neuroscience. These complex functional dynamics, measured by EEG and fMRI, are thought to arise from a shared underlying structural architecture, which can be measured by diffusion MRI (dMRI). While simulation and data transformation (e.g. graph theory measures) have been proposed to refine the understanding of the underlying function-structure relationship, the potential complementary and/or independent contribution of EEG and fMRI to this relationship is still poorly understood. As such, we explored this relationship by analyzing the function-structure correlation in fourteen healthy subjects with simultaneous resting-state EEG-fMRI and dMRI acquisitions. We show that the combination of EEG and fMRI connectivity better explains dMRI connectivity and that this represents a genuine model improvement over fMRI-only models for both group-averaged connectivity matrices and at the individual level. Furthermore, this model improves the prediction within each resting-state network. The best model fit to underlying structure is mediated by fMRI and EEG-δ connectivity in combination with Euclidean distance and interhemispheric connectivity with more local contributions of EEG-γ at the scale of resting-state networks. This highlights that the factors mediating the relationship between functional and structural metrics of connectivity are context and scale dependent, influenced by topological, geometric and architectural features. It also suggests that fMRI studies employing simultaneous EEG measures may characterize additional and essential parts of the underlying neuronal activity of the resting-state, which might be of special interest for both clinical studies and the investigation of resting-state dynamics.

Resolving the neural dynamics of visual and auditory scene processing in the human brain: a methodological approach

In natural environments, visual and auditory stimulation elicit responses across a large set of brain regions in a fraction of a second, yielding representations of the multimodal scene and its properties. The rapid and complex neural dynamics underlying visual and auditory information processing pose major challenges to human cognitive neuroscience. Brain signals measured non-invasively are inherently noisy, the format of neural representations is unknown, and transformations between representations are complex and often nonlinear. Further, no single non-invasive brain measurement technique provides a spatio-temporally integrated view. In this opinion piece, we argue that progress can be made by a concerted effort based on three pillars of recent methodological development: (i) sensitive analysis techniques such as decoding and cross-classification, (ii) complex computational modelling using models such as deep neural networks, and (iii) integration across imaging methods (magnetoencephalography/electroencephalography, functional magnetic resonance imaging) and models, e.g. using representational similarity analysis. We showcase two recent efforts that have been undertaken in this spirit and provide novel results about visual and auditory scene analysis. Finally, we discuss the limits of this perspective and sketch a concrete roadmap for future research.

This article is part of the themed issue ‘Auditory and visual scene analysis’.

EEG Source Imaging Guided by Spatiotemporal Specific fMRI: Toward an Understanding of Dynamic Cognitive Processes

Understanding the mechanism of neuroplasticity is the first step in treating neuromuscular system impairments with cognitive rehabilitation approaches. To characterize the dynamics of the neural networks and the underlying neuroplasticity of the central motor system, neuroimaging tools with high spatial and temporal accuracy are desirable. EEG and fMRI stand among the most popular noninvasive neuroimaging modalities with complementary features, yet achieving both high spatial and temporal accuracy remains a challenge. A novel multimodal EEG/fMRI integration method was developed in this study to achieve high spatiotemporal accuracy by employing the most probable fMRI spatial subsets to guide EEG source localization in a time-variant fashion. In comparison with the traditional fMRI constrained EEG source imaging method in a visual/motor activation task study, the proposed method demonstrated superior localization accuracy with lower variation and identified neural activity patterns that agreed well with previous studies. This spatiotemporal fMRI constrained source imaging method was then implemented in a "sequential multievent-related potential" paradigm where motor activation is evoked by emotion-related visual stimuli. Results demonstrate that the proposed method can be used as a powerful neuroimaging tool to unveil the dynamics and neural networks associated with the central motor system, providing insights into neuroplasticity modulation mechanism.

Building an EEG-fMRI Multi-Modal Brain Graph: A Concurrent EEG-fMRI Study

The topological architecture of brain connectivity has been well-characterized by graph theory based analysis. However, previous studies have primarily built brain graphs based on a single modality of brain imaging data. Here we develop a framework to construct multi-modal brain graphs using concurrent EEG-fMRI data which are simultaneously collected during eyes open (EO) and eyes closed (EC) resting states. FMRI data are decomposed into independent components with associated time courses by group independent component analysis (ICA). EEG time series are segmented, and then spectral power time courses are computed and averaged within 5 frequency bands (delta; theta; alpha; beta; low gamma). EEG-fMRI brain graphs, with EEG electrodes and fMRI brain components serving as nodes, are built by computing correlations within and between fMRI ICA time courses and EEG spectral power time courses. Dynamic EEG-fMRI graphs are built using a sliding window method, versus static ones treating the entire time course as stationary. In global level, static graph measures and properties of dynamic graph measures are different across frequency bands and are mainly showing higher values in eyes closed than eyes open. Nodal level graph measures of a few brain components are also showing higher values during eyes closed in specific frequency bands. Overall, these findings incorporate fMRI spatial localization and EEG frequency information which could not be obtained by examining only one modality. This work provides a new approach to examine EEG-fMRI associations within a graph theoretic framework with potential application to many topics.

Similarity-Based Fusion of MEG and fMRI Reveals Spatio-Temporal Dynamics in Human Cortex During Visual Object Recognition

Every human cognitive function, such as visual object recognition, is realized in a complex spatio-temporal activity pattern in the brain. Current brain imaging techniques in isolation cannot resolve the brain's spatio-temporal dynamics, because they provide either high spatial or temporal resolution but not both. To overcome this limitation, we developed an integration approach that uses representational similarities to combine measurements of magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) to yield a spatially and temporally integrated characterization of neuronal activation. Applying this approach to 2 independent MEG-fMRI data sets, we observed that neural activity first emerged in the occipital pole at 50-80 ms, before spreading rapidly and progressively in the anterior direction along the ventral and dorsal visual streams. Further region-of-interest analyses established that dorsal and ventral regions showed MEG-fMRI correspondence in representations later than early visual cortex. Together, these results provide a novel and comprehensive, spatio-temporally resolved view of the rapid neural dynamics during the first few hundred milliseconds of object vision. They further demonstrate the feasibility of spatially unbiased representational similarity-based fusion of MEG and fMRI, promising new insights into how the brain computes complex cognitive functions.

Temporally Dissociable Contributions of Human Medial Prefrontal Subregions to Reward-Guided Learning

In decision making, dorsal and ventral medial prefrontal cortex show a sensitivity to key decision variables, such as reward prediction errors. It is unclear whether these signals reflect parallel processing of a common synchronous input to both regions, for example from mesocortical dopamine, or separate and consecutive stages in reward processing. These two perspectives make distinct predictions about the relative timing of feedback-related activity in each of these regions, a question we address here. To reconstruct the unique temporal contribution of dorsomedial (dmPFC) and ventromedial prefrontal cortex (vmPFC) to simultaneously measured EEG activity in human subjects, we developed a novel trialwise fMRI-informed EEG analysis that allows dissociating correlated and overlapping sources. We show that vmPFC uniquely contributes a sustained activation profile shortly after outcome presentation, whereas dmPFC contributes a later and more peaked activation pattern. This temporal dissociation is expressed mainly in the alpha band for a vmPFC signal, which contrasts with a theta based dmPFC signal. Thus, our data show reward-related vmPFC and dmPFC responses have distinct time courses and unique spectral profiles, findings that support distinct functional roles in a reward-processing network.

SIGNIFICANCE STATEMENT

Multiple subregions of the medial prefrontal cortex are known to be involved in decision making and learning, and expose similar response patterns in fMRI. Here, we used a novel approach to analyzing simultaneous EEG-fMRI that allows to dissociate the individual time courses of brain regions. We find that vmPFC and dmPFC have distinguishable time courses and time-frequency patterns.

Carbon-wire loop based artifact correction outperforms post-processing EEG/fMRI corrections--A validation of a real-time simultaneous EEG/fMRI correction method

Simultaneous EEG-fMRI combines two powerful neuroimaging techniques, but the EEG signal suffers from severe artifacts in the MRI environment that are difficult to remove. These are the MR scanning artifact and the blood-pulsation artifact--strategies to remove them are a topic of ongoing research. Additionally large, unsystematic artifacts are produced across the full frequency spectrum by the magnet's helium pump (and ventilator) systems which are notoriously hard to remove. As a consequence, experimenters routinely deactivate the helium pump during simultaneous EEG-fMRI acquisitions which potentially risks damaging the MRI system and necessitates more frequent and expensive helium refills. We present a novel correction method addressing both helium pump and ballisto-cardiac (BCG) artifacts, consisting of carbon-wire loops (CWL) as additional sensors to accurately track unpredictable artifacts related to subtle movements in the scanner, and an EEGLAB plugin to perform artifact correction. We compare signal-to-noise metrics of EEG data, corrected with CWL and three conventional correction methods, for helium pump off and on measurements. Because the CWL setup records signals in real-time, it fits requirements of applications where immediate correction is necessary, such as neuro-feedback applications or stimulation time-locked to specific sleep oscillations. The comparison metrics in this paper relate to: (1) the EEG signal itself, (2) the "eyes open vs. eyes closed" effect, and (3) an assessment of how the artifact corrections impacts the ability to perform meaningful correlations between EEG alpha power and the BOLD signal. Results show that the CWL correction corrects for He pump artifact and also produces EEG data more comparable to EEG obtained outside the magnet than conventional post-processing methods.

Characterizing nonlinear relationships in functional imaging data using eigenspace maximal information canonical correlation analysis (emiCCA)

Many important problems in the analysis of neuroimages can be formulated as discovering the relationship between two sets of variables, a task for which linear techniques such as canonical correlation analysis (CCA) have been commonly used. However, to further explore potential nonlinear processes that might co-exist with linear ones in brain function, a more flexible method is required. Here, we propose a new unsupervised and data-driven method, termed the eigenspace maximal information canonical correlation analysis (emiCCA), which is capable of automatically capturing the linear and/or nonlinear relationships between various data sets. A simulation confirmed the superior performance of emiCCA in comparison with linear CCA and kernel CCA (a nonlinear version of CCA). An emiCCA framework for functional magnetic resonance imaging (fMRI) data processing was designed and applied to data from a real motor execution fMRI experiment. This analysis uncovered one linear (in primary motor cortex) and a few nonlinear networks (e.g., in the supplementary motor area, bilateral insula, and cerebellum). This suggests that these various task-related brain areas are part of networks that also contribute to the execution of movements of the hand. These results suggest that emiCCA is a promising technique for exploring various data.

Characterisation and reduction of the EEG artefact caused by the helium cooling pump in the MR environment: validation in epilepsy patient data

The EEG acquired simultaneously with fMRI is distorted by a number of artefacts related to the presence of strong magnetic fields, which must be reduced in order to allow for a useful interpretation and quantification of the EEG data. For the two most prominent artefacts, associated with magnetic field gradient switching and the heart beat, reduction methods have been developed and applied successfully. However, a number of artefacts related to the MR-environment can be found to distort the EEG data acquired even without ongoing fMRI acquisition. In this paper, we investigate the most prominent of those artefacts, caused by the Helium cooling pump, and propose a method for its reduction and respective validation in data collected from epilepsy patients. Since the Helium cooling pump artefact was found to be repetitive, an average template subtraction method was developed for its reduction with appropriate adjustments for minimizing the degradation of the physiological part of the signal. The new methodology was validated in a group of 15 EEG-fMRI datasets collected from six consecutive epilepsy patients, where it successfully reduced the amplitude of the artefact spectral peaks by 95 ± 2 % while the background spectral amplitude within those peaks was reduced by only -5 ± 4 %. Although the Helium cooling pump should ideally be switched off during simultaneous EEG-fMRI acquisitions, we have shown here that in cases where this is not possible the associated artefact can be effectively reduced in post processing.

Recursive approach of EEG-segment-based principal component analysis substantially reduces cryogenic pump artifacts in simultaneous EEG-fMRI data

Electroencephalography (EEG) data simultaneously acquired with functional magnetic resonance imaging (fMRI) data are preprocessed to remove gradient artifacts (GAs) and ballistocardiographic artifacts (BCAs). Nonetheless, these data, especially in the gamma frequency range, can be contaminated by residual artifacts produced by mechanical vibrations in the MRI system, in particular the cryogenic pump that compresses and transports the helium that chills the magnet (the helium-pump). However, few options are available for the removal of helium-pump artifacts. In this study, we propose a recursive approach of EEG-segment-based principal component analysis (rsPCA) that enables the removal of these helium-pump artifacts. Using the rsPCA method, feature vectors representing helium-pump artifacts were successfully extracted as eigenvectors, and the reconstructed signals of the feature vectors were subsequently removed. A test using simultaneous EEG-fMRI data acquired from left-hand (LH) and right-hand (RH) clenching tasks performed by volunteers found that the proposed rsPCA method substantially reduced helium-pump artifacts in the EEG data and significantly enhanced task-related gamma band activity levels (p=0.0038 and 0.0363 for LH and RH tasks, respectively) in EEG data that have had GAs and BCAs removed. The spatial patterns of the fMRI data were estimated using a hemodynamic response function (HRF) modeled from the estimated gamma band activity in a general linear model (GLM) framework. Active voxel clusters were identified in the post-/pre-central gyri of motor area, only from the rsPCA method (uncorrected p<0.001 for both LH/RH tasks). In addition, the superior temporal pole areas were consistently observed (uncorrected p<0.001 for the LH task and uncorrected p<0.05 for the RH task) in the spatial patterns of the HRF model for gamma band activity when the task paradigm and movement were also included in the GLM.

Application of partial directed coherence to the analysis of resting-state EEG-fMRI data

The simultaneous acquisition of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data potentially allows measurement of brain signals with both high spatial and temporal resolution. Partial directed coherence (PDC) is a Granger causality measure in the frequency domain, which is often used to infer the intensity of information flow over the brain from EEG data. In the current study, we propose a new approach to investigate functional connectivity in resting-state (RS) EEG-fMRI data by combining time-varying PDC with the analysis of blood oxygenation level-dependent (BOLD) signal fluctuations. Basically, we aim to identify brain circuits that are more active when the information flow is increased between distinct remote neuronal modules. The usefulness of the proposed method is illustrated by application to simultaneously recorded EEG-fMRI data from healthy subjects at rest. Using this approach, we decomposed the nodes of RS networks in fMRI data according to the frequency band and directed flow of information provided from EEG. This approach therefore has the potential to inform our understanding of the regional characteristics of oscillatory processes in the human brain.

Single-trial EEG-informed fMRI reveals spatial dependency of BOLD signal on early and late IC-ERP amplitudes during face recognition

Simultaneous EEG-fMRI has opened up new avenues for improving the spatio-temporal resolution of functional brain studies. However, this method usually suffers from poor EEG quality, especially for evoked potentials (ERPs), due to specific artifacts. As such, the use of EEG-informed fMRI analysis in the context of cognitive studies has particularly focused on optimizing narrow ERP time windows of interest, which ignores the rich diverse temporal information of the EEG signal. Here, we propose to use simultaneous EEG-fMRI to investigate the neural cascade occurring during face recognition in 14 healthy volunteers by using the successive ERP peaks recorded during the cognitive part of this process. N170, N400 and P600 peaks, commonly associated with face recognition, were successfully and reproducibly identified for each trial and each subject by using a group independent component analysis (ICA). For the first time we use this group ICA to extract several independent components (IC) corresponding to the sequence of activation and used single-trial peaks as modulation parameters in a general linear model (GLM) of fMRI data. We obtained an occipital-temporal-frontal stream of BOLD signal modulation, in accordance with the three successive IC-ERPs providing an unprecedented spatio-temporal characterization of the whole cognitive process as defined by BOLD signal modulation. By using this approach, the pattern of EEG-informed BOLD modulation provided improved characterization of the network involved than the fMRI-only analysis or the source reconstruction of the three ERPs; the latter techniques showing only two regions in common localized in the occipital lobe.

Simultaneous EEG-fMRI: trial level spatio-temporal fusion for hierarchically reliable information discovery

Simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) have been pursued in an effort to integrate complementary noninvasive information on brain activity. The primary goal involves better information discovery of the event-related neural activations at a spatial region of the BOLD fluctuation with the temporal resolution of the electrical signal. Many techniques and algorithms have been developed to integrate EEGs and fMRIs; however, the relative reliability of the integrated information is unclear. In this work, we propose a hierarchical framework to ensure the relative reliability of the integrated results and attempt to understand brain activation using this hierarchical ideal. First, spatial Independent Component Analysis (ICA) of fMRI and temporal ICA of EEG were performed to extract features at the trial level. Second, the maximal information coefficient (MIC) was adopted to temporally match them across the modalities for both linear and non-linear associations. Third, fMRI-constrained EEG source imaging was utilized to spatially match components across modalities. The simultaneously occurring events in the above two match steps provided EEG-fMRI spatial-temporal reliable integrated information, resulting in the most reliable components with high spatial and temporal resolution information. The other components discovered in the second or third steps provided second-level complementary information for flexible and cautious explanations. This paper contains two simulations and an example of real data, and the results indicate that the framework is a feasible approach to reveal cognitive processing in the human brain.

Brain network dysfunction in late-life depression: a literature review

As a common psychiatric disorder in the growing geriatric population, late-life depression (LLD) has a negative impact on the cognitive, affective, and somatic domains of the lives of the elderly individuals. Accumulating evidence from the structural and functional imaging studies on LLD supports a "network dysfunction model" rather than a "lesion pathology model" for understanding the underlying biological mechanism in this mental disorder. In this work, we used network dysfunction model as a conceptual framework for reviewing recent neuroimaging findings in LLD. Our focus was on 4 major neurocircuits that have been shown to be involved in LLD: default mood network, cognitive control network, affective/frontolimbic network, and corticostriatal circuits. Findings of LLD-related gray and white matter structural abnormalities and resting-state and task-based functional changes were discussed for each network separately. We extended our review by summarizing the latest works that apply graph theory-based network analysis techniques for testing alterations in whole-brain network properties associated with LLD.