Multimodal functional imaging using fMRI-informed regional EEG/MEG source estimation

A revisit of the amygdala theory of autism: Twenty years after

The human amygdala has long been implicated to play a key role in autism spectrum disorder (ASD). Yet it remains unclear to what extent the amygdala accounts for the social dysfunctions in ASD. Here, we review studies that investigate the relationship between amygdala function and ASD. We focus on studies that employ the same task and stimuli to directly compare people with ASD and patients with focal amygdala lesions, and we also discuss functional data associated with these studies. We show that the amygdala can only account for a limited number of deficits in ASD (primarily face perception tasks but not social attention tasks), a network view is, therefore, more appropriate. We next discuss atypical brain connectivity in ASD, factors that can explain such atypical brain connectivity, and novel tools to analyze brain connectivity. Lastly, we discuss new opportunities from multimodal neuroimaging with data fusion and human single-neuron recordings that can enable us to better understand the neural underpinnings of social dysfunctions in ASD. Together, the influential amygdala theory of autism should be extended with emerging data-driven scientific discoveries such as machine learning-based surrogate models to a broader framework that considers brain connectivity at the global scale.

A simultaneous EEG-fMRI study of thalamic load-dependent working memory delay period activity

Introduction

Working memory (WM) is an essential component of executive functions which depend on maintaining task-related information online for brief periods in both the presence and absence of interfering stimuli. Active maintenance occurs during the WM delay period, the time between stimulus encoding and subsequent retrieval. Previous studies have extensively documented prefrontal and posterior parietal cortex activity during the WM delay period, but the role of subcortical structures including the thalamus remains to be fully elucidated, especially in humans.

Methods

Using a simultaneous electroencephalogram (EEG)-functional magnetic resonance imaging (fMRI) approach, we investigated the role of the thalamus during the WM delay period in a modified Sternberg paradigm following low and high memory load encoding of naturalistic scenes. During the delay, participants passively viewed scrambled scenes containing similar color and spatial frequency to serve as a perceptual baseline. Individual source estimation was weighted by the location of the thalamic fMRI signal relative to the WM delay period onset.

Results

The effects memory load on maintenance were observed bilaterally in thalamus with higher EEG source amplitudes in the low compared to high load condition occurring 160-390 ms after the onset of the delay period.

Conclusion

The main finding that thalamic activation was elevated during the low compared to high condition despite similar duration of perceptual input and upcoming motor requirements suggests a capacity-limited role for sensory filtering of the thalamus during consolidation of stimuli into WM, where the highest activity occurs when fewer stimuli need to be maintained in the presence of interfering perceptual stimuli during the delay. The results are discussed in the context of theories regarding the role of the thalamus in sensory gating during working memory.

Simultaneous EEG-fMRI: What Have We Learned and What Does the Future Hold?

Simultaneous EEG-fMRI has developed into a mature measurement technique in the past 25 years. During this time considerable technical and analytical advances have been made, enabling valuable scientific contributions to a range of research fields. This review will begin with an introduction to the measurement principles involved in EEG and fMRI and the advantages of combining these methods. The challenges faced when combining the two techniques will then be considered. An overview of the leading application fields where EEG-fMRI has made a significant contribution to the scientific literature and emerging applications in EEG-fMRI research trends is then presented.

fMRI-Informed EEG for brain mapping of imagined lower limb movement: Feasibility of a brain computer interface

BACKGROUND

EEG and fMRI have contributed greatly to our understanding of brain activity and its link to behaviors by helping to identify both when and where the activity occurs. This is particularly important in the development of brain-computer interfaces (BCIs), where feed forward systems gather data from imagined brain activity and then send that information to an effector. The purpose of this study was to develop and evaluate a computational approach that enables an accurate mapping of spatial brain activity (fMRI) in relation to the temporal receptors (EEG electrodes) associated with imagined lower limb movement.

NEW METHOD

EEG and fMRI data from 16 healthy, male participants while imagining lower limb movement were used for this purpose. A combined analysis of fMRI data and EEG electrode locations was developed to identify EEG electrodes with a high likelihood of capturing imagined lower limb movement originating from various clusters of brain activity. This novel feature selection tool was used to develop an artificial neural network model to classify right and left lower limb movement.

RESULTS

Results showed that left versus right lower limb imagined movement could be classified with 66.5% accuracy using this approach. Comparison with existing methods: Adopting a purely data-driven approach for feature selection to use in the right/left classification task resulted in the same accuracy (66.6%) but with reduced interpretability.

CONCLUSIONS

The developed fMRI-informed EEG approach could pave the way towards improved brain computer interfaces for lower limb movement while also being applicable to other systems where fMRI could be helpful to inform EEG acquisition and processing.

Bayesian MEG time courses with fMRI priors

Magnetoencephalography (MEG) records brain activity with excellent temporal and good spatial resolution, while functional magnetic resonance imaging (fMRI) offers good temporal and excellent spatial resolution. The aim of this study is to implement a Bayesian framework to use fMRI data as spatial priors for MEG inverse solutions. We used simulated MEG data with both evoked and induced activity and experimental MEG data from sixteen participants to examine the effectiveness of using fMRI spatial priors in MEG source reconstruction. For simulated MEG data, incorporating the prior information from fMRI increased the spatial resolution of MEG source reconstruction by 3 mm on average. For experimental MEG data, fMRI spatial information reduced the spurious clusters for evoked activity and showed more left-lateralized activation pattern for induced activity. The use of fMRI spatial priors greatly reduced location error for induced source in MEG data. Our results provide empirical evidence that the use of fMRI spatial priors improves the accuracy of MEG source reconstruction. The combined MEG and fMRI approach can provide neuroimaging data with better spatial and temporal resolutions to add another perspective to our understanding of the neurobiology of language. The potential clinical applications include pre-surgical evaluation of language function for epilepsy patients and evaluation of language network for children with language disorders.

Inferring Macroscale Brain Dynamics via Fusion of Simultaneous EEG-fMRI

Advances in the instrumentation and signal processing for simultaneously acquired electroencephalography and functional magnetic resonance imaging (EEG-fMRI) have enabled new ways to observe the spatiotemporal neural dynamics of the human brain. Central to the utility of EEG-fMRI neuroimaging systems are the methods for fusing the two data streams, with machine learning playing a key role. These methods can be dichotomized into those that are symmetric and asymmetric in terms of how the two modalities inform the fusion. Studies using these methods have shown that fusion yields new insights into brain function that are not possible when each modality is acquired separately. As technology improves and methods for fusion become more sophisticated, the future of EEG-fMRI for noninvasive measurement of brain dynamics includes mesoscale mapping at ultrahigh magnetic resonance fields, targeted perturbation-based neuroimaging, and using deep learning to uncover nonlinear representations that link the electrophysiological and hemodynamic measurements.

Thoughts on neurophysiological signal analysis and classification

Neurophysiological signals are crucial intermediaries, through which brain activity can be quantitatively measured and brain mechanisms are able to be revealed. In particular, non‐invasive neurophysiological signals, such as electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI), are welcomed and frequently utilised in various studies since these signals can be non‐invasively recorded without harming the human brain while they convey abundant information pertaining to brain activity. The recorded neurophysiological signals are analysed to mine meaningful information for the understanding of brain mechanisms or are classified to distinguish different patterns (e.g., different cognitive states, brain diseases versus healthy controls). To date, remarkable progress has been made in both the analysis and classification of neurophysiological signals, but scholars are not feeling complacent. Consistent effort ought to be paid to advance the research of analysis and classification based on neurophysiological signals. In this paper, I express my thoughts regarding promising future directions in neurophysiological signal analysis and classification based on current developments and accomplishments. I will elucidate the thoughts after brief summaries of relevant backgrounds, accomplishments, and tendencies. According to my personal selection and preference, I mainly focus on brain connectivity, multidimensional array (tensor), multi‐modality, multiple task classification, deep learning, big data, and naturalistic experiment. Hopefully, my thoughts could give a little help to inspire new ideas and contribute to the research of the analysis and classification of neurophysiological signals in some way.

Efficient high resolution sLORETA in brain source localization

Objective. Estimation of the source location within the brain from electroencephalography (EEG) and magnetoencephalography measures is a challenging task. Among the existing techniques in the field, which are known as brain imaging methods, standardized low-resolution brain electromagnetic tomography (sLORETA) is the most popular method due to its simplicity and high accuracy. However, in this work we illustrate that sLORETA is still noisy and the additive noise is causing the blurry image. The existing pre-fixed/manual thresholding process after sLORETA can partially take care of denoising. However, this ad-hoc theresholding can either remove so much of the desired data or leave much of the noise in the process. Manual correction to avoid such extreme cases can be time-consuming. The objective of this paper is to automate the denoising process in the form of adaptive thresholding. Approach. The proposed method, denoted by efficient high-resolution sLORETA (EHR-sLORETA), is based on minimizing the error between the desired denoised source and the source estimates. Main results. The approach is evaluated using synthetic EEG and real EEG data. spatial dispersion (SD), and mean square error (MSE) are used as metrics to provide the quantitative performance of the method. In addition, qualitative analysis of the method is provided for real EEG data. This proposed model demonstrates advantages over the existing methods in sense of accuracy and robustness with SD and MSE comparison. Significance. EHR-sLORETA could have a significant impact on clinical studies with source estimation task, as it improves the accuracy of source estimation and eliminates the need for manual thresholding.

Concurrent electrophysiological and hemodynamic measurements of evoked neural oscillations in human visual cortex using sparsely interleaved fast fMRI and EEG

Electroencephalography (EEG) concurrently collected with functional magnetic resonance imaging (fMRI) is heavily distorted by the repetitive gradient coil switching during the fMRI acquisition. The performance of the typical template-based gradient artifact suppression method can be suboptimal because the artifact changes over time. Gradient artifact residuals also impede the subsequent suppression of ballistocardiography artifacts. Here we propose recording continuous EEG with temporally sparse fast fMRI (fast fMRI-EEG) to minimize the EEG artifacts caused by MRI gradient coil switching without significantly compromising the field-of-view and spatiotemporal resolution of fMRI. Using simultaneous multi-slice inverse imaging to achieve whole-brain fMRI with isotropic 5-mm resolution in 0.1 ​s, and performing these acquisitions once every 2 ​s, we have 95% of the duty cycle available to record EEG with substantially less gradient artifact. We found that the standard deviation of EEG signals over the entire acquisition period in fast fMRI-EEG was reduced to 54% of that in conventional concurrent echo-planar imaging (EPI) and EEG recordings (EPI-EEG) across participants. When measuring 15-Hz steady-state visual evoked potentials (SSVEPs), the baseline-normalized oscillatory neural response in fast fMRI-EEG was 2.5-fold of that in EPI-EEG. The functional MRI responses associated with the SSVEP delineated by EPI and fast fMRI were similar in the spatial distribution, the elicited waveform, and detection power. Sparsely interleaved fast fMRI-EEG provides high-quality EEG without substantially compromising the quality of fMRI in evoked response measurements, and has the potential utility for applications where the onset of the target stimulus cannot be precisely determined, such as epilepsy.

Multi-subject MEG/EEG source imaging with sparse multi-task regression

Magnetoencephalography and electroencephalography (M/EEG) are non-invasive modalities that measure the weak electromagnetic fields generated by neural activity. Estimating the location and magnitude of the current sources that generated these electromagnetic fields is an inverse problem. Although it can be cast as a linear regression, this problem is severely ill-posed as the number of observations, which equals the number of sensors, is small. When considering a group study, a common approach consists in carrying out the regression tasks independently for each subject using techniques such as MNE or sLORETA. An alternative is to jointly localize sources for all subjects taken together, while enforcing some similarity between them. By pooling S subjects in a single joint regression, the number of observations is S times larger, potentially making the problem better posed and offering the ability to identify more sources with greater precision. Here we show how the coupling of the different regression problems can be done through a multi-task regularization that promotes focal source estimates. To take into account intersubject variabilities, we propose the Minimum Wasserstein Estimates (MWE). Thanks to a new joint regression method based on optimal transport (OT) metrics, MWE does not enforce perfect overlap of activation foci for all subjects but rather promotes spatial proximity on the cortical mantle. Besides, by estimating the noise level of each subject, MWE copes with the subject-specific signal-to-noise ratios with only one regularization parameter. On realistic simulations, MWE decreases the localization error by up to 4 ​mm per source compared to individual solutions. Experiments on the Cam-CAN dataset show improvements in spatial specificity in population imaging compared to individual models such as dSPM as well as a state-of-the-art Bayesian group level model. Our analysis of a multimodal dataset shows how multi-subject source localization reduces the gap between MEG and fMRI for brain mapping.

Robust Empirical Bayesian Reconstruction of Distributed Sources for Electromagnetic Brain Imaging

Electromagnetic brain imaging is the reconstruction of brain activity from non-invasive recordings of the magnetic fields and electric potentials. An enduring challenge in this imaging modality is estimating the number, location, and time course of sources, especially for the reconstruction of distributed brain sources with complex spatial extent. Here, we introduce a novel robust empirical Bayesian algorithm that enables better reconstruction of distributed brain source activity with two key ideas: kernel smoothing and hyperparameter tiling. Since the proposed algorithm builds upon many of the performance features of the sparse source reconstruction algorithm - Champagne and we refer to this algorithm as Smooth Champagne. Smooth Champagne is robust to the effects of high levels of noise, interference, and highly correlated brain source activity. Simulations demonstrate excellent performance of Smooth Champagne when compared to benchmark algorithms in accurately determining the spatial extent of distributed source activity. Smooth Champagne also accurately reconstructs real MEG and EEG data.

Hierarchical multiscale Bayesian algorithm for robust MEG/EEG source reconstruction

In this paper, we present a novel hierarchical multiscale Bayesian algorithm for electromagnetic brain imaging using magnetoencephalography (MEG) and electroencephalography (EEG). In particular, we present a solution to the source reconstruction problem for sources that vary in spatial extent. We define sensor data measurements using a generative probabilistic graphical model that is hierarchical across spatial scales of brain regions and voxels. We then derive a novel Bayesian algorithm for probabilistic inference with this graphical model. This algorithm enables robust reconstruction of sources that have different spatial extent, from spatially contiguous clusters of dipoles to isolated dipolar sources. We compare the new algorithm with several representative benchmarks on both simulated and real brain activities. The source locations and the correct estimation of source time courses used for the simulated data are chosen to test the performance on challenging source configurations. In simulations, performance of the novel algorithm shows superiority to several existing benchmark algorithms. We also demonstrate that the new algorithm is more robust to correlated brain activity present in real MEG and EEG data and is able to resolve distinct and functionally relevant brain areas with real MEG and EEG datasets.

EEG/MEG source imaging using fMRI informed time-variant constraints

Multimodal functional neuroimaging by combining functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) or magnetoencephalography (MEG) is able to provide high spatiotemporal resolution mapping of brain activity. However, the accuracy of fMRI-constrained EEG/MEG source imaging may be degraded by potential spatial mismatches between the locations of fMRI activation and electrical source activities. To address this problem, we propose a novel fMRI informed time-variant constraint (FITC) method. The weights in FITC are determined by combining the fMRI activities and electrical source activities in a time-variant manner to reduce the impact of the fMRI extra sources. The fMRI weights are modified using cross-talk matrix and normalized partial area under the curve to reduce the impact of fMRI missing sources. Monte Carlo simulations were performed to compare the source estimates produced by L2-minimum norm estimation (MNE), fMRI-weighted minimum norm estimation (fMNE), FITC, and depth-weighted FITC (wFITC) algorithms with various spatial mismatch conditions. Localization error and temporal correlation were calculated to compare the four algorithms under different conditions. The simulation results indicated that the FITC and wFITC methods were more robust than the MNE and fMNE algorithms. Moreover, FITC and wFITC were significantly better than fMNE under the fMRI missing sources condition. A human visual-stimulus EEG, MEG, and fMRI test was performed, and the experimental data revealed that FITC and wFITC displayed more focal areas than fMNE and MNE. In conclusion, the proposed FITC method is able to better resolve the spatial mismatch problems encountered in fMRI-constrained EEG/MEG source imaging.

A multi-subject, multi-modal human neuroimaging dataset

We describe data acquired with multiple functional and structural neuroimaging modalities on the same nineteen healthy volunteers. The functional data include Electroencephalography (EEG), Magnetoencephalography (MEG) and functional Magnetic Resonance Imaging (fMRI) data, recorded while the volunteers performed multiple runs of hundreds of trials of a simple perceptual task on pictures of familiar, unfamiliar and scrambled faces during two visits to the laboratory. The structural data include T1-weighted MPRAGE, Multi-Echo FLASH and Diffusion-weighted MR sequences. Though only from a small sample of volunteers, these data can be used to develop methods for integrating multiple modalities from multiple runs on multiple participants, with the aim of increasing the spatial and temporal resolution above that of any one modality alone. They can also be used to integrate measures of functional and structural connectivity, and as a benchmark dataset to compare results across the many neuroimaging analysis packages. The data are freely available from https://openfmri.org/.

Bayesian Models for fMRI Data Analysis

Functional magnetic resonance imaging (fMRI), a noninvasive neuroimaging method that provides an indirect measure of neuronal activity by detecting blood flow changes, has experienced an explosive growth in the past years. Statistical methods play a crucial role in understanding and analyzing fMRI data. Bayesian approaches, in particular, have shown great promise in applications. A remarkable feature of fully Bayesian approaches is that they allow a flexible modeling of spatial and temporal correlations in the data. This paper provides a review of the most relevant models developed in recent years. We divide methods according to the objective of the analysis. We start from spatio-temporal models for fMRI data that detect task-related activation patterns. We then address the very important problem of estimating brain connectivity. We also touch upon methods that focus on making predictions of an individual's brain activity or a clinical or behavioral response. We conclude with a discussion of recent integrative models that aim at combining fMRI data with other imaging modalities, such as EEG/MEG and DTI data, measured on the same subjects. We also briefly discuss the emerging field of imaging genetics.

Mapping preictal networks preceding childhood absence seizures using magnetoencephalography

The electrographic hallmark of childhood absence seizures is 3 Hz generalized spike and wave discharges; however, there is likely a focal thalamic or cortical onset that cannot be detected using scalp electroencephalography (EEG). The purpose of this study was to study the earliest preictal changes in children with absence epilepsy. In this report, magnetoencephalography recordings of 44 absence seizures recorded from 12 children with drug-naïve childhood absence seizures were used to perform time frequency analysis and source localization prior to the onset of the seizures. Evidence of preictal magnetoencephalography frequency changes were detected a mean of 694 ms before the initial spike on the EEG. A consistent pattern of focal sources was present in the frontal cortex and thalamus during this preictal period, but source localization occurred synchronously so that independent activity between the 2 structures could not be distinguished.

How to use fMRI functional localizers to improve EEG/MEG source estimation

EEG and MEG have excellent temporal resolution, but the estimation of the neural sources that generate the signals recorded by the sensors is a difficult, ill-posed problem. The high spatial resolution of functional MRI makes it an ideal tool to improve the localization of the EEG/MEG sources using data fusion. However, the combination of the two techniques remains challenging, as the neural generators of the EEG/MEG and BOLD signals might in some cases be very different. Here we describe a data fusion approach that was developed by our team over the last decade in which fMRI is used to provide source constraints that are based on functional areas defined individually for each subject. This mini-review describes the different steps that are necessary to perform source estimation using this approach. It also provides a list of pitfalls that should be avoided when doing fMRI-informed EEG/MEG source imaging. Finally, it describes the advantages of using a ROI-based approach for group-level analysis and for the study of sensory systems.

Simultaneous EEG-fMRI: trial level spatio-temporal fusion for hierarchically reliable information discovery

Simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) have been pursued in an effort to integrate complementary noninvasive information on brain activity. The primary goal involves better information discovery of the event-related neural activations at a spatial region of the BOLD fluctuation with the temporal resolution of the electrical signal. Many techniques and algorithms have been developed to integrate EEGs and fMRIs; however, the relative reliability of the integrated information is unclear. In this work, we propose a hierarchical framework to ensure the relative reliability of the integrated results and attempt to understand brain activation using this hierarchical ideal. First, spatial Independent Component Analysis (ICA) of fMRI and temporal ICA of EEG were performed to extract features at the trial level. Second, the maximal information coefficient (MIC) was adopted to temporally match them across the modalities for both linear and non-linear associations. Third, fMRI-constrained EEG source imaging was utilized to spatially match components across modalities. The simultaneously occurring events in the above two match steps provided EEG-fMRI spatial-temporal reliable integrated information, resulting in the most reliable components with high spatial and temporal resolution information. The other components discovered in the second or third steps provided second-level complementary information for flexible and cautious explanations. This paper contains two simulations and an example of real data, and the results indicate that the framework is a feasible approach to reveal cognitive processing in the human brain.

The Quest for EEG Power Band Correlation with ICA Derived fMRI Resting State Networks

The neuronal underpinnings of blood oxygen level dependent (BOLD) functional magnetic resonance imaging (fMRI) resting state networks (RSNs) are still unclear. To investigate the underlying mechanisms, specifically the relation to the electrophysiological signal, we used simultaneous recordings of electroencephalography (EEG) and fMRI during eyes open resting state (RS). Earlier studies using the EEG signal as independent variable show inconclusive results, possibly due to variability in the temporal correlations between RSNs and power in the low EEG frequency bands, as recently reported (Goncalves et al., 2006, 2008; Meyer et al., 2013). In this study we use three different methods including one that uses RSN timelines as independent variable to explore the temporal relationship of RSNs and EEG frequency power in eyes open RS in detail. The results of these three distinct analysis approaches support the hypothesis that the correlation between low EEG frequency power and BOLD RSNs is instable over time, at least in eyes open RS.

EEG-fMRI integration for the study of human brain function

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) have proved to be extremely valuable tools for the non-invasive study of human brain function. Moreover, due to a notable degree of complementarity between the two modalities, the combination of EEG and fMRI data has been actively sought in the last two decades. Although initially focused on epilepsy, EEG-fMRI applications were rapidly extended to the study of healthy brain function, yielding new insights into its underlying mechanisms and pathways. Nevertheless, EEG and fMRI have markedly different spatial and temporal resolutions, and probe neuronal activity through distinct biophysical processes, many aspects of which are still poorly understood. The remarkable conceptual and methodological challenges associated with EEG-fMRI integration have motivated the development of a wide range of analysis approaches over the years, each relying on more or less restrictive assumptions, and aiming to shed further light on the mechanisms of brain function along with those of the EEG-fMRI coupling itself. Here, we present a review of the most relevant EEG-fMRI integration approaches yet proposed for the study of brain function, supported by a general overview of our current understanding of the biophysical mechanisms coupling the signals obtained from the two modalities.

MEG source localization of spatially extended generators of epileptic activity: comparing entropic and hierarchical bayesian approaches

Localizing the generators of epileptic activity in the brain using Electro-EncephaloGraphy (EEG) or Magneto-EncephaloGraphy (MEG) signals is of particular interest during the pre-surgical investigation of epilepsy. Epileptic discharges can be detectable from background brain activity, provided they are associated with spatially extended generators. Using realistic simulations of epileptic activity, this study evaluates the ability of distributed source localization methods to accurately estimate the location of the generators and their sensitivity to the spatial extent of such generators when using MEG data. Source localization methods based on two types of realistic models have been investigated: (i) brain activity may be modeled using cortical parcels and (ii) brain activity is assumed to be locally smooth within each parcel. A Data Driven Parcellization (DDP) method was used to segment the cortical surface into non-overlapping parcels and diffusion-based spatial priors were used to model local spatial smoothness within parcels. These models were implemented within the Maximum Entropy on the Mean (MEM) and the Hierarchical Bayesian (HB) source localization frameworks. We proposed new methods in this context and compared them with other standard ones using Monte Carlo simulations of realistic MEG data involving sources of several spatial extents and depths. Detection accuracy of each method was quantified using Receiver Operating Characteristic (ROC) analysis and localization error metrics. Our results showed that methods implemented within the MEM framework were sensitive to all spatial extents of the sources ranging from 3 cm² to 30 cm², whatever were the number and size of the parcels defining the model. To reach a similar level of accuracy within the HB framework, a model using parcels larger than the size of the sources should be considered.

Increasing the accuracy of electromagnetic inverses using functional area source correlation constraints

Estimating cortical current distributions from electroencephalographic (EEG) or magnetoencephalographic data is a difficult inverse problem whose solution can be improved by the addition of priors on the associated neural responses. In the context of visual activation studies, we propose a new approach that uses a functional area constrained estimator (FACE) to increase the accuracy of the reconstructions. It derives the source correlation matrix from a segmentation of the cortex into areas defined by retinotopic maps of the visual field or by functional localizers obtained independently by fMRI. These areas are computed once for each individual subject and the associated estimators can therefore be reused for any new study on the same participant. The resulting FACE reconstructions emphasize the activity of sources within these areas or enforce their intercorrelations. We used realistic Monte-Carlo simulations to demonstrate that this approach improved our estimates of a diverse set of source configurations. Reconstructions obtained from a real EEG dataset demonstrate that our priors improve the localization of the cortical areas involved in horizontal disparity processing.

Methods for simultaneous EEG-fMRI: an introductory review

The simultaneous recording and analysis of electroencephalography (EEG) and fMRI data in human systems, cognitive and clinical neurosciences is rapidly evolving and has received substantial attention. The significance of multimodal brain imaging is documented by a steadily increasing number of laboratories now using simultaneous EEG-fMRI aiming to achieve both high temporal and spatial resolution of human brain function. Due to recent developments in technical and algorithmic instrumentation, the rate-limiting step in multimodal studies has shifted from data acquisition to analytic aspects. Here, we introduce and compare different methods for data integration and identify the benefits that come with each approach, guiding the reader toward an understanding and informed selection of the integration approach most suitable for addressing a particular research question.

Cortical current source estimation from electroencephalography in combination with near-infrared spectroscopy as a hierarchical prior

Previous simulation and experimental studies have demonstrated that the application of Variational Bayesian Multimodal EncephaloGraphy (VBMEG) to magnetoencephalography (MEG) data can be used to estimate cortical currents with high spatio-temporal resolution, by incorporating functional magnetic resonance imaging (fMRI) activity as a hierarchical prior. However, the use of combined MEG and fMRI is restricted by the high costs involved, a lack of portability and high sensitivity to body-motion artifacts. One possible solution for overcoming these limitations is to use a combination of electroencephalography (EEG) and near-infrared spectroscopy (NIRS). This study therefore aimed to extend the possible applications of VBMEG to include EEG data with NIRS activity as a hierarchical prior. Using computer simulations and real experimental data, we evaluated the performance of VBMEG applied to EEG data under different conditions, including different numbers of EEG sensors and different prior information. The results suggest that VBMEG with NIRS prior performs well, even with as few as 19 EEG sensors. These findings indicate the potential value of clinically applying VBMEG using a combination of EEG and NIRS.

Multimodal functional network connectivity: an EEG-fMRI fusion in network space

EEG and fMRI recordings measure the functional activity of multiple coherent networks distributed in the cerebral cortex. Identifying network interaction from the complementary neuroelectric and hemodynamic signals may help to explain the complex relationships between different brain regions. In this paper, multimodal functional network connectivity (mFNC) is proposed for the fusion of EEG and fMRI in network space. First, functional networks (FNs) are extracted using spatial independent component analysis (ICA) in each modality separately. Then the interactions among FNs in each modality are explored by Granger causality analysis (GCA). Finally, fMRI FNs are matched to EEG FNs in the spatial domain using network-based source imaging (NESOI). Investigations of both synthetic and real data demonstrate that mFNC has the potential to reveal the underlying neural networks of each modality separately and in their combination. With mFNC, comprehensive relationships among FNs might be unveiled for the deep exploration of neural activities and metabolic responses in a specific task or neurological state.

A Parametric Empirical Bayesian Framework for the EEG/MEG Inverse Problem: Generative Models for Multi-Subject and Multi-Modal Integration

We review recent methodological developments within a parametric empirical Bayesian (PEB) framework for reconstructing intracranial sources of extracranial electroencephalographic (EEG) and magnetoencephalographic (MEG) data under linear Gaussian assumptions. The PEB framework offers a natural way to integrate multiple constraints (spatial priors) on this inverse problem, such as those derived from different modalities (e.g., from functional magnetic resonance imaging, fMRI) or from multiple replications (e.g., subjects). Using variations of the same basic generative model, we illustrate the application of PEB to three cases: (1) symmetric integration (fusion) of MEG and EEG; (2) asymmetric integration of MEG or EEG with fMRI, and (3) group-optimization of spatial priors across subjects. We evaluate these applications on multi-modal data acquired from 18 subjects, focusing on energy induced by face perception within a time–frequency window of 100–220 ms, 8–18 Hz. We show the benefits of multi-modal, multi-subject integration in terms of the model evidence and the reproducibility (over subjects) of cortical responses to faces.

Electrophysiological imaging of brain activity and connectivity-challenges and opportunities

Unlocking the dynamic inner workings of the brain continues to remain a grand challenge of the 21st century. To this end, functional neuroimaging modalities represent an outstanding approach to better understand the mechanisms of both normal and abnormal brain functions. The ability to image brain function with ever increasing spatial and temporal resolution has made a significant leap over the past several decades. Further delineation of functional networks could lead to improved understanding of brain function in both normal and diseased states. This paper reviews recent advancements and current challenges in dynamic functional neuroimaging techniques, including electrophysiological source imaging, multimodal neuroimaging integrating fMRI with EEG/MEG, and functional connectivity imaging.

Informatics and data mining tools and strategies for the human connectome project

The Human Connectome Project (HCP) is a major endeavor that will acquire and analyze connectivity data plus other neuroimaging, behavioral, and genetic data from 1,200 healthy adults. It will serve as a key resource for the neuroscience research community, enabling discoveries of how the brain is wired and how it functions in different individuals. To fulfill its potential, the HCP consortium is developing an informatics platform that will handle: (1) storage of primary and processed data, (2) systematic processing and analysis of the data, (3) open-access data-sharing, and (4) mining and exploration of the data. This informatics platform will include two primary components. ConnectomeDB will provide database services for storing and distributing the data, as well as data analysis pipelines. Connectome Workbench will provide visualization and exploration capabilities. The platform will be based on standard data formats and provide an open set of application programming interfaces (APIs) that will facilitate broad utilization of the data and integration of HCP services into a variety of external applications. Primary and processed data generated by the HCP will be openly shared with the scientific community, and the informatics platform will be available under an open source license. This paper describes the HCP informatics platform as currently envisioned and places it into the context of the overall HCP vision and agenda.

Region-based current-source reconstruction for the inverse EEG problem

This paper presents a new method for the reconstruction of current sources for the electroencephalography (EEG) inverse problem, which produces reconstructed sources, which are confined to a few anatomical regions. The method is based on a partition of the gray matter into a set of regions, and in the construction of a simple linear model for the potential produced by feasible source configurations inside each one of these regions. The proposed method computes the solution in two stages: in the first one, a subset of active regions is found so that the combined potentials produced by sources inside them approximate the measured potential data. In the second stage, a detailed reconstruction of the current sources inside each active region is performed. Experimental results with synthetic data are presented, which show that the proposed scheme is fast, computationally efficient and robust to noise, producing results that are competitive with other published methods, especially when the current sources are effectively distributed in few anatomical regions. The proposed method is also validated with real data from an experiment with visual evoked potentials.

Bayesian symmetrical EEG/fMRI fusion with spatially adaptive priors

In this paper, we propose a novel symmetrical EEG/fMRI fusion method which combines EEG and fMRI by means of a common generative model. We use a total variation (TV) prior to model the spatial distribution of the cortical current responses and hemodynamic response functions, and utilize spatially adaptive temporal priors to model their temporal shapes. The spatial adaptivity of the prior model allows for adaptation to the local characteristics of the estimated responses and leads to high estimation performance for the cortical current distribution and the hemodynamic response functions. We utilize a Bayesian formulation with a variational Bayesian framework and obtain a fully automatic fusion algorithm. Simulations with synthetic data and experiments with real data from a multimodal study on face perception demonstrate the performance of the proposed method.