Model driven EEG/fMRI fusion of brain oscillations

How to Build a Hybrid Neurofeedback Platform Combining EEG and fMRI

Multimodal neurofeedback estimates brain activity using information acquired with more than one neurosignal measurement technology. In this paper we describe how to set up and use a hybrid platform based on simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI), then we illustrate how to use it for conducting bimodal neurofeedback experiments. The paper is intended for those willing to build a multimodal neurofeedback system, to guide them through the different steps of the design, setup, and experimental applications, and help them choose a suitable hardware and software configuration. Furthermore, it reports practical information from bimodal neurofeedback experiments conducted in our lab. The platform presented here has a modular parallel processing architecture that promotes real-time signal processing performance and simple future addition and/or replacement of processing modules. Various unimodal and bimodal neurofeedback experiments conducted in our lab showed high performance and accuracy. Currently, the platform is able to provide neurofeedback based on electroencephalography and functional magnetic resonance imaging, but the architecture and the working principles described here are valid for any other combination of two or more real-time brain activity measurement technologies.

Visual brain activity patterns classification with simultaneous EEG-fMRI: A multimodal approach

BACKGROUND

Classification of the visual information from the brain activity data is a challenging task. Many studies reported in the literature are based on the brain activity patterns using either fMRI or EEG/MEG only. EEG and fMRI considered as two complementary neuroimaging modalities in terms of their temporal and spatial resolution to map the brain activity. For getting a high spatial and temporal resolution of the brain at the same time, simultaneous EEG-fMRI seems to be fruitful.

METHODS

In this article, we propose a new method based on simultaneous EEG-fMRI data and machine learning approach to classify the visual brain activity patterns. We acquired EEG-fMRI data simultaneously on the ten healthy human participants by showing them visual stimuli. Data fusion approach is used to merge EEG and fMRI data. Machine learning classifier is used for the classification purposes.

RESULTS

Results showed that superior classification performance has been achieved with simultaneous EEG-fMRI data as compared to the EEG and fMRI data standalone. This shows that multimodal approach improved the classification accuracy results as compared with other approaches reported in the literature.

CONCLUSIONS

The proposed simultaneous EEG-fMRI approach for classifying the brain activity patterns can be helpful to predict or fully decode the brain activity patterns.

Mathematical Frameworks for Oscillatory Network Dynamics in Neuroscience

The tools of weakly coupled phase oscillator theory have had a profound impact on the neuroscience community, providing insight into a variety of network behaviours ranging from central pattern generation to synchronisation, as well as predicting novel network states such as chimeras. However, there are many instances where this theory is expected to break down, say in the presence of strong coupling, or must be carefully interpreted, as in the presence of stochastic forcing. There are also surprises in the dynamical complexity of the attractors that can robustly appear-for example, heteroclinic network attractors. In this review we present a set of mathematical tools that are suitable for addressing the dynamics of oscillatory neural networks, broadening from a standard phase oscillator perspective to provide a practical framework for further successful applications of mathematics to understanding network dynamics in neuroscience.

Overview of MEG

Magnetoencephalography (MEG) is a method to study electrical activity in the human brain by recording the neuromagnetic field outside the head. MEG, like electroencephalography (EEG), provides an excellent, millisecond-scale time resolution, and allows the estimation of the spatial distribution of the underlying activity, in favorable cases with a localization accuracy of a few millimeters. To detect the weak neuromagnetic signals, superconducting sensors, magnetically shielded rooms, and advanced signal processing techniques are used. The analysis and interpretation of MEG data typically involves comparisons between subject groups and experimental conditions using various spatial, temporal, and spectral measures of cortical activity and connectivity. The application of MEG to cognitive neuroscience studies is illustrated with studies of spoken language processing in subjects with normal and impaired reading ability. The mapping of spatiotemporal patterns of activity within networks of cortical areas can provide useful information about the functional architecture of the brain related to sensory and cognitive processing, including language, memory, attention, and perception.

Topology, Cross-Frequency, and Same-Frequency Band Interactions Shape the Generation of Phase-Amplitude Coupling in a Neural Mass Model of a Cortical Column

Phase-amplitude coupling (PAC), a type of cross-frequency coupling (CFC) where the phase of a low-frequency rhythm modulates the amplitude of a higher frequency, is becoming an important indicator of information transmission in the brain. However, the neurobiological mechanisms underlying its generation remain undetermined. A realistic, yet tractable computational model of the phenomenon is thus needed. Here we analyze a neural mass model of a cortical column, comprising fourteen neuronal populations distributed across four layers (L2/3, L4, L5 and L6). A control analysis showed that the conditional transfer entropy (cTE) measure is able to correctly estimate the flow of information between neuronal populations. Then, we computed cTE from the phases to the amplitudes of the oscillations generated in the cortical column. This approach provides information regarding directionality by distinguishing PAC from APC (amplitude-phase coupling), i.e. the information transfer from amplitudes to phases, and can be used to estimate other types of CFC such as amplitude-amplitude coupling (AAC) and phase-phase coupling (PPC). While experiments often only focus on one or two PAC combinations (e.g., theta-gamma or alpha-gamma), we found that a cortical column can simultaneously generate almost all possible PAC combinations, depending on connectivity parameters, time constants, and external inputs. PAC interactions with and without an anatomical equivalent (direct and indirect interactions, respectively) were analyzed. We found that the strength of PAC between two populations was strongly correlated with the strength of the effective connections between the populations and, on average, did not depend on whether the PAC connection was direct or indirect. When considering a cortical column circuit as a complex network, we found that neuronal populations making indirect PAC connections had, on average, higher local clustering coefficient, efficiency, and betweenness centrality than populations making direct connections and populations not involved in PAC connections. This suggests that their interactions were more effective when transmitting information. Since approximately 60% of the obtained interactions represented indirect connections, our results highlight the importance of the topology of cortical circuits for the generation of the PAC phenomenon. Finally, our results demonstrated that indirect PAC interactions can be explained by a cascade of direct CFC and same-frequency band interactions, suggesting that PAC analysis of experimental data should be accompanied by the estimation of other types of frequency interactions for an integrative understanding of the phenomenon.

Building an EEG-fMRI Multi-Modal Brain Graph: A Concurrent EEG-fMRI Study

The topological architecture of brain connectivity has been well-characterized by graph theory based analysis. However, previous studies have primarily built brain graphs based on a single modality of brain imaging data. Here we develop a framework to construct multi-modal brain graphs using concurrent EEG-fMRI data which are simultaneously collected during eyes open (EO) and eyes closed (EC) resting states. FMRI data are decomposed into independent components with associated time courses by group independent component analysis (ICA). EEG time series are segmented, and then spectral power time courses are computed and averaged within 5 frequency bands (delta; theta; alpha; beta; low gamma). EEG-fMRI brain graphs, with EEG electrodes and fMRI brain components serving as nodes, are built by computing correlations within and between fMRI ICA time courses and EEG spectral power time courses. Dynamic EEG-fMRI graphs are built using a sliding window method, versus static ones treating the entire time course as stationary. In global level, static graph measures and properties of dynamic graph measures are different across frequency bands and are mainly showing higher values in eyes closed than eyes open. Nodal level graph measures of a few brain components are also showing higher values during eyes closed in specific frequency bands. Overall, these findings incorporate fMRI spatial localization and EEG frequency information which could not be obtained by examining only one modality. This work provides a new approach to examine EEG-fMRI associations within a graph theoretic framework with potential application to many topics.

Phase-amplitude coupling and the BOLD signal: A simultaneous intracranial EEG (icEEG) - fMRI study in humans performing a finger-tapping task

Although it has been consistently found that local blood-oxygen-level-dependent (BOLD) changes are better modelled by a combination of the power of multiple EEG frequency bands rather than by the power of a unique band alone, the local electro-haemodynamic coupling function is not yet fully characterised. Electrophysiological studies have revealed that the strength of the coupling between the phase of low- and the amplitude of high- frequency EEG activities (phase–amplitude coupling - PAC) has an important role in brain function in general, and in preparation and execution of movement in particular. Using electrocorticographic (ECoG) and functional magnetic resonance imaging (fMRI) data recorded simultaneously in humans performing a finger-tapping task, we investigated the single-trial relationship between the amplitude of the BOLD signal and the strength of PAC and the power of α, β, and γ bands, at a local level. In line with previous studies, we found a positive correlation for the γ band, and negative correlations for the PAC_βγ strength, and the α and β bands. More importantly, we found that the PAC_βγ strength explained variance of the amplitude of the BOLD signal that was not explained by a combination of the α, β, and γ band powers. Our main finding sheds further light on the distinct nature of PAC as a functionally relevant mechanism and suggests that the sensitivity of EEG-informed fMRI studies may increase by including the PAC strength in the BOLD signal model, in addition to the power of the low- and high- frequency EEG bands.

•

First study of single-trial correlations between the phase amplitude coupling strength and BOLD.

•

Intracranial EEG and fMRI data simultaneously recorded in humans during a motor task.

•

PAC_βγ strength explains variance of BOLD in addition a combination of α, β, and γ band powers.

One-Class FMRI-Inspired EEG Model for Self-Regulation Training

Recent evidence suggests that learned self-regulation of localized brain activity in deep limbic areas such as the amygdala, may alleviate symptoms of affective disturbances. Thus far self-regulation of amygdala activity could be obtained only via fMRI guided neurofeedback, an expensive and immobile procedure. EEG on the other hand is relatively inexpensive and can be easily implemented in any location. However the clinical utility of EEG neurofeedback for affective disturbances remains limited due to low spatial resolution, which hampers the targeting of deep limbic areas such as the amygdala. We introduce an EEG prediction model of amygdala activity from a single electrode. The gold standard used for training is the fMRI-BOLD signal in the amygdala during simultaneous EEG/fMRI recording. The suggested model is based on a time/frequency representation of the EEG data with varying time-delay. Previous work has shown a strong inhomogeneity among subjects as is reflected by the models created to predict the amygdala BOLD response from EEG data. In that work, different models were constructed for different subjects. In this work, we carefully analyzed the inhomogeneity among subjects and were able to construct a single model for the majority of the subjects. We introduce a method for inhomogeneity assessment. This enables us to demonstrate a choice of subjects for which a single model could be derived. We further demonstrate the ability to modulate brain-activity in a neurofeedback setting using feedback generated by the model. We tested the effect of the neurofeedback training by showing that new subjects can learn to down-regulate the signal amplitude compared to a sham group, which received a feedback obtained by a different participant. This EEG based model can overcome substantial limitations of fMRI-NF. It can enable investigation of NF training using multiple sessions and large samples in various locations.

Modeling the Generation of Phase-Amplitude Coupling in Cortical Circuits: From Detailed Networks to Neural Mass Models

Phase-amplitude coupling (PAC), the phenomenon where the amplitude of a high frequency oscillation is modulated by the phase of a lower frequency oscillation, is attracting an increasing interest in the neuroscience community due to its potential relevance for understanding healthy and pathological information processing in the brain. PAC is a diverse phenomenon, having been experimentally detected in at least ten combinations of rhythms: delta-theta, delta-alpha, delta-beta, delta-gamma, theta-alpha, theta-beta, theta-gamma, alpha-beta, alpha-gamma, and beta-gamma. However, a complete understanding of the biophysical mechanisms generating this diversity is lacking. Here we review computational models of PAC generation that range from detailed models of neuronal networks, where each cell is described by Hodgkin-Huxley-type equations, to neural mass models (NMMs) where only the average activities of neuronal populations are considered. We argue that NMMs are an appropriate mathematical framework (due to the small number of parameters and variables involved and the richness of the dynamics they can generate) to study the PAC phenomenon.

Incorporating priors for EEG source imaging and connectivity analysis

Electroencephalography source imaging (ESI) is a useful technique to localize the generators from a given scalp electric measurement and to investigate the temporal dynamics of the large-scale neural circuits. By introducing reasonable priors from other modalities, ESI reveals the most probable sources and communication structures at every moment in time. Here, we review the available priors from such techniques as magnetic resonance imaging (MRI), functional MRI (fMRI), and positron emission tomography (PET). The modality's specific contribution is analyzed from the perspective of source reconstruction. For spatial priors, EEG-correlated fMRI, temporally coherent networks (TCNs) and resting-state fMRI are systematically introduced in the ESI. Moreover, the fiber tracking (diffusion tensor imaging, DTI) and neuro-stimulation techniques (transcranial magnetic stimulation, TMS) are also introduced as the potential priors, which can help to draw inferences about the neuroelectric connectivity in the source space. We conclude that combining EEG source imaging with other complementary modalities is a promising approach toward the study of brain networks in cognitive and clinical neurosciences.

On the central role of brain connectivity in neurodegenerative disease progression

Increased brain connectivity, in all its variants, is often considered an evolutionary advantage by mediating complex sensorimotor function and higher cognitive faculties. Interaction among components at all spatial scales, including genes, proteins, neurons, local neuronal circuits and macroscopic brain regions, are indispensable for such vital functions. However, a growing body of evidence suggests that, from the microscopic to the macroscopic levels, such connections might also be a conduit for in intra-brain disease spreading. For instance, cell-to-cell misfolded proteins (MP) transmission and neuronal toxicity are prominent connectivity-mediated factors in aging and neurodegeneration. This article offers an overview of connectivity dysfunctions associated with neurodegeneration, with a specific focus on how these may be central to both normal aging and the neuropathologic degenerative progression.

Characterizing nonlinear relationships in functional imaging data using eigenspace maximal information canonical correlation analysis (emiCCA)

Many important problems in the analysis of neuroimages can be formulated as discovering the relationship between two sets of variables, a task for which linear techniques such as canonical correlation analysis (CCA) have been commonly used. However, to further explore potential nonlinear processes that might co-exist with linear ones in brain function, a more flexible method is required. Here, we propose a new unsupervised and data-driven method, termed the eigenspace maximal information canonical correlation analysis (emiCCA), which is capable of automatically capturing the linear and/or nonlinear relationships between various data sets. A simulation confirmed the superior performance of emiCCA in comparison with linear CCA and kernel CCA (a nonlinear version of CCA). An emiCCA framework for functional magnetic resonance imaging (fMRI) data processing was designed and applied to data from a real motor execution fMRI experiment. This analysis uncovered one linear (in primary motor cortex) and a few nonlinear networks (e.g., in the supplementary motor area, bilateral insula, and cerebellum). This suggests that these various task-related brain areas are part of networks that also contribute to the execution of movements of the hand. These results suggest that emiCCA is a promising technique for exploring various data.

On self-feedback connectivity in neural mass models applied to event-related potentials

Neural mass models (NMMs) applied to neuroimaging data often do not emphasise intrinsic self-feedback within a neural population. However, based on mean-field theory, any population of coupled neurons is intrinsically endowed with effective self-coupling. In this work, we examine the effectiveness of three cortical NMMs with different self-feedbacks using a dynamic causal modelling approach. Specifically, we compare the classic Jansen and Rit (1995) model (no self-feedback), a modified model by Moran et al. (2007) (only inhibitory self-feedback), and our proposed model with inhibitory and excitatory self-feedbacks. Using bifurcation analysis, we show that single-unit Jansen-Rit model is less robust in generating oscillatory behaviour than the other two models. Next, under Bayesian inversion, we simulate single-channel event-related potentials (ERPs) within a mismatch negativity auditory oddball paradigm. We found fully self-feedback model (FSM) to provide the best fit to single-channel data. By analysing the posterior covariances of model parameters, we show that self-feedback connections are less sensitive to the generated evoked responses than the other model parameters, and hence can be treated analogously to "higher-order" parameter corrections of the original Jansen-Rit model. This is further supported in the more realistic multi-area case where FSM can replicate data better than JRM and MoM in the majority of subjects by capturing the finer features of the ERP data more accurately. Our work informs how NMMs with full self-feedback connectivity are not only more consistent with the underlying neurophysiology, but can also account for more complex features in ERP data.

Electrophysiological correlates of the BOLD signal for EEG-informed fMRI

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) are important tools in cognitive and clinical neuroscience. Combined EEG-fMRI has been shown to help to characterise brain networks involved in epileptic activity, as well as in different sensory, motor and cognitive functions. A good understanding of the electrophysiological correlates of the blood oxygen level-dependent (BOLD) signal is necessary to interpret fMRI maps, particularly when obtained in combination with EEG. We review the current understanding of electrophysiological-haemodynamic correlates, during different types of brain activity. We start by describing the basic mechanisms underlying EEG and BOLD signals and proceed by reviewing EEG-informed fMRI studies using fMRI to map specific EEG phenomena over the entire brain (EEG-fMRI mapping), or exploring a range of EEG-derived quantities to determine which best explain colocalised BOLD fluctuations (local EEG-fMRI coupling). While reviewing studies of different forms of brain activity (epileptic and nonepileptic spontaneous activity; cognitive, sensory and motor functions), a significant attention is given to epilepsy because the investigation of its haemodynamic correlates is the most common application of EEG-informed fMRI. Our review is focused on EEG-informed fMRI, an asymmetric approach of data integration. We give special attention to the invasiveness of electrophysiological measurements and the simultaneity of multimodal acquisitions because these methodological aspects determine the nature of the conclusions that can be drawn from EEG-informed fMRI studies. We emphasise the advantages of, and need for, simultaneous intracranial EEG-fMRI studies in humans, which recently became available and hold great potential to improve our understanding of the electrophysiological correlates of BOLD fluctuations.

Simultaneous EEG-fMRI: trial level spatio-temporal fusion for hierarchically reliable information discovery

Simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) have been pursued in an effort to integrate complementary noninvasive information on brain activity. The primary goal involves better information discovery of the event-related neural activations at a spatial region of the BOLD fluctuation with the temporal resolution of the electrical signal. Many techniques and algorithms have been developed to integrate EEGs and fMRIs; however, the relative reliability of the integrated information is unclear. In this work, we propose a hierarchical framework to ensure the relative reliability of the integrated results and attempt to understand brain activation using this hierarchical ideal. First, spatial Independent Component Analysis (ICA) of fMRI and temporal ICA of EEG were performed to extract features at the trial level. Second, the maximal information coefficient (MIC) was adopted to temporally match them across the modalities for both linear and non-linear associations. Third, fMRI-constrained EEG source imaging was utilized to spatially match components across modalities. The simultaneously occurring events in the above two match steps provided EEG-fMRI spatial-temporal reliable integrated information, resulting in the most reliable components with high spatial and temporal resolution information. The other components discovered in the second or third steps provided second-level complementary information for flexible and cautious explanations. This paper contains two simulations and an example of real data, and the results indicate that the framework is a feasible approach to reveal cognitive processing in the human brain.

General overview on the merits of multimodal neuroimaging data fusion

Multimodal neuroimaging has become a mainstay of basic and cognitive neuroscience in humans and animals, despite challenges to consider when acquiring and combining non-redundant imaging data. Multimodal data integration can yield important insights into brain processes and structures in addition to spatiotemporal resolution complementarity, including: a comprehensive physiological view on brain processes and structures, quantification, generalization and normalization, and availability of biomarkers. In this review, we discuss data acquisition and fusion in multimodal neuroimaging in the context of each of these potential merits. However, limitations - due to differences in the neuronal and structural underpinnings of each method - have to be taken into account when modeling and interpreting multimodal data using generative models. We conclude that when these challenges are adequately met, multimodal data fusion can create substantial added value for neuroscience applications making it an indispensable approach for studying the brain.

EEG and MEG: relevance to neuroscience

To understand dynamic cognitive processes, the high time resolution of EEG/MEG is invaluable. EEG/MEG signals can play an important role in providing measures of functional and effective connectivity in the brain. After a brief description of the foundations and basic methodological aspects of EEG/MEG signals, the relevance of the signals to obtain novel insights into the neuronal mechanisms underlying cognitive processes is surveyed, with emphasis on neuronal oscillations (ultra-slow, theta, alpha, beta, gamma, and HFOs) and combinations of oscillations. Three main functional roles of brain oscillations are put in evidence: (1) coding specific information, (2) setting and modulating brain attentional states, and (3) assuring the communication between neuronal populations such that specific dynamic workspaces may be created. The latter form the material core of cognitive functions.

Fusing concurrent EEG-fMRI with dynamic causal modeling: application to effective connectivity during face perception

Despite the wealth of research on face perception, the interactions between core regions in the face-sensitive network of the visual cortex are not well understood. In particular, the link between neural activity in face-sensitive brain regions measured by fMRI and EEG markers of face-selective processing in the N170 component is not well established. In this study, we used dynamic causal modeling (DCM) as a data fusion approach to integrate concurrently acquired EEG and fMRI data during the perception of upright compared with inverted faces. Data features derived from single-trial EEG variability were used as contextual modulators on fMRI-derived estimates of effective connectivity between key regions of the face perception network. The overall construction of our model space was highly constrained by the effects of task and ERP parameters on our fMRI data. Bayesian model selection suggested that the occipital face area (OFA) acted as a central gatekeeper directing visual information to the superior temporal sulcus (STS), the fusiform face area (FFA), and to a medial region of the fusiform gyrus (mFG). The connection from the OFA to the STS was strengthened on trials in which N170 amplitudes to upright faces were large. In contrast, the connection from the OFA to the mFG, an area known to be involved in object processing, was enhanced for inverted faces particularly on trials in which N170 amplitudes were small. Our results suggest that trial-by-trial variation in neural activity at around 170 ms, reflected in the N170 component, reflects the relative engagement of the OFA to STS/FFA network over the OFA to mFG object processing network for face perception. Importantly, the DCMs predicted the observed data significantly better by including the modulators derived from the N170, highlighting the value of incorporating EEG-derived information to explain interactions between regions as a multi-modal data fusion method for combined EEG-fMRI.

The Virtual Brain: a simulator of primate brain network dynamics

We present The Virtual Brain (TVB), a neuroinformatics platform for full brain network simulations using biologically realistic connectivity. This simulation environment enables the model-based inference of neurophysiological mechanisms across different brain scales that underlie the generation of macroscopic neuroimaging signals including functional MRI (fMRI), EEG and MEG. Researchers from different backgrounds can benefit from an integrative software platform including a supporting framework for data management (generation, organization, storage, integration and sharing) and a simulation core written in Python. TVB allows the reproduction and evaluation of personalized configurations of the brain by using individual subject data. This personalization facilitates an exploration of the consequences of pathological changes in the system, permitting to investigate potential ways to counteract such unfavorable processes. The architecture of TVB supports interaction with MATLAB packages, for example, the well known Brain Connectivity Toolbox. TVB can be used in a client-server configuration, such that it can be remotely accessed through the Internet thanks to its web-based HTML5, JS, and WebGL graphical user interface. TVB is also accessible as a standalone cross-platform Python library and application, and users can interact with the scientific core through the scripting interface IDLE, enabling easy modeling, development and debugging of the scientific kernel. This second interface makes TVB extensible by combining it with other libraries and modules developed by the Python scientific community. In this article, we describe the theoretical background and foundations that led to the development of TVB, the architecture and features of its major software components as well as potential neuroscience applications.

EEG-fMRI integration for the study of human brain function

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) have proved to be extremely valuable tools for the non-invasive study of human brain function. Moreover, due to a notable degree of complementarity between the two modalities, the combination of EEG and fMRI data has been actively sought in the last two decades. Although initially focused on epilepsy, EEG-fMRI applications were rapidly extended to the study of healthy brain function, yielding new insights into its underlying mechanisms and pathways. Nevertheless, EEG and fMRI have markedly different spatial and temporal resolutions, and probe neuronal activity through distinct biophysical processes, many aspects of which are still poorly understood. The remarkable conceptual and methodological challenges associated with EEG-fMRI integration have motivated the development of a wide range of analysis approaches over the years, each relying on more or less restrictive assumptions, and aiming to shed further light on the mechanisms of brain function along with those of the EEG-fMRI coupling itself. Here, we present a review of the most relevant EEG-fMRI integration approaches yet proposed for the study of brain function, supported by a general overview of our current understanding of the biophysical mechanisms coupling the signals obtained from the two modalities.

Towards a large-scale model of patient-specific epileptic spike-wave discharges

Clinical electroencephalographic (EEG) recordings of the transition into generalised epileptic seizures show a sudden onset of spike-wave dynamics from a low-amplitude irregular background. In addition, non-trivial and variable spatio-temporal dynamics are widely reported in combined EEG/fMRI studies on the scale of the whole cortex. It is unknown whether these characteristics can be accounted for in a large-scale mathematical model with fixed heterogeneous long-range connectivities. Here, we develop a modelling framework with which to investigate such EEG features. We show that a neural field model composed of a few coupled compartments can serve as a low-dimensional prototype for the transition between irregular background dynamics and spike-wave activity. This prototype then serves as a node in a large-scale network with long-range connectivities derived from human diffusion-tensor imaging data. We examine multivariate properties in 42 clinical EEG seizure recordings from 10 patients diagnosed with typical absence epilepsy and 50 simulated seizures from the large-scale model using 10 DTI connectivity sets from humans. The model can reproduce the clinical feature of stereotypy where seizures are more similar within a patient than between patients, essentially creating a patient-specific fingerprint. We propose the approach as a feasible technique for the investigation of patient-specific large-scale epileptic features in space and time.

The virtual brain integrates computational modeling and multimodal neuroimaging

Brain function is thought to emerge from the interactions among neuronal populations. Apart from traditional efforts to reproduce brain dynamics from the micro- to macroscopic scales, complementary approaches develop phenomenological models of lower complexity. Such macroscopic models typically generate only a few selected-ideally functionally relevant-aspects of the brain dynamics. Importantly, they often allow an understanding of the underlying mechanisms beyond computational reproduction. Adding detail to these models will widen their ability to reproduce a broader range of dynamic features of the brain. For instance, such models allow for the exploration of consequences of focal and distributed pathological changes in the system, enabling us to identify and develop approaches to counteract those unfavorable processes. Toward this end, The Virtual Brain (TVB) ( www.thevirtualbrain.org ), a neuroinformatics platform with a brain simulator that incorporates a range of neuronal models and dynamics at its core, has been developed. This integrated framework allows the model-based simulation, analysis, and inference of neurophysiological mechanisms over several brain scales that underlie the generation of macroscopic neuroimaging signals. In this article, we describe how TVB works, and we present the first proof of concept.

EEG current source density and the phenomenology of the default network

In recent years, there has been an increasing line of research dedicated to the investigation of the default mode network (DMN) of the brain and resting state networks. However, the mental activity of the DMN has not been rigorously assessed to date. The specific aims of the current study were 2-fold: First, we sought to determine whether the current source density (CSD) levels in the DMN would correspond to other neuroimaging techniques. Second, we sought to understand the subjective mental activity of the DMN during baseline recordings. This study was conducted with 63 nonclinical participants, 34 female and 29 males with a mean age of 19.2 years (standard deviation = 2.0). The participants were recorded in 8 conditions. First, 4-minute eyes-closed baseline (ECB) and eyes-opened baseline (EOB) were obtained. The participants then completed 3 assessment instruments and 3 image conditions while the electroencephalography (EEG) was continuously recorded. Participants completed subjective reports for baselines and image conditions. These were rated by 3 independent raters and compared for reliability using a random effects model with an absolute agreement definition. The mean CSD between all conditions differed significantly, in many but not all regions of interest in the DMN. Interestingly, as suggested by other studies, the DMN appears preferential to self-relevant, self-specific, or self-perceptive processes. The reliability analyses show α for interrater agreement for ECB at .95 and EOB at .96. The subjective reports obtained from the participants regarding the mental activities employed during baseline recordings correspond to attentional and self-regulatory processes, which may also implicate the resting state or DMN as playing a direct role in the maintenance of a complex behavior (eg, being still, attending, and self-regulating). Thus, attention and self-regulation constitute the phenomenology of the resting state (DMN) in this study. The results also demonstrate that EEG CSD is a useful method to examine the DMN during concept-specific tasks to elucidate the neural activity associated with these concepts. Standardized low-resolution electromagnetic tomography (sLORETA) can localize to 5 mm(3), which is comparable to the findings in functional magnetic resonance imaging (fMRI). However, sLORETA can provide data about the difference in activity between groups, individuals, or populations which in many cases fMRI cannot provide.

EEG/fMRI fusion based on independent component analysis: integration of data-driven and model-driven methods

Simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) provide complementary noninvasive information of brain activity, and EEG/fMRI fusion can achieve higher spatiotemporal resolution than each modality separately. This focuses on independent component analysis (ICA)-based EEG/fMRI fusion. In order to appreciate the issues, we first describe the potential and limitations of the developed fusion approaches: fMRI-constrained EEG imaging, EEG-informed fMRI analysis, and symmetric fusion. We then outline some newly developed hybrid fusion techniques using ICA and the combination of data-/model-driven methods, with special mention of the spatiotemporal EEG/fMRI fusion (STEFF). Finally, we discuss the current trend in methodological development and the existing limitations for extrapolating neural dynamics.

Methods for simultaneous EEG-fMRI: an introductory review

The simultaneous recording and analysis of electroencephalography (EEG) and fMRI data in human systems, cognitive and clinical neurosciences is rapidly evolving and has received substantial attention. The significance of multimodal brain imaging is documented by a steadily increasing number of laboratories now using simultaneous EEG-fMRI aiming to achieve both high temporal and spatial resolution of human brain function. Due to recent developments in technical and algorithmic instrumentation, the rate-limiting step in multimodal studies has shifted from data acquisition to analytic aspects. Here, we introduce and compare different methods for data integration and identify the benefits that come with each approach, guiding the reader toward an understanding and informed selection of the integration approach most suitable for addressing a particular research question.

Cortical dynamics of human scalp EEG origins in a visually guided motor execution

The EEG mu rhythm is often used as an index of activation in the sensorimotor cortex. However, the blur caused by volume conduction makes it difficult to identify the exact origin of the EEG rhythm in the brain using only the human scalp EEG. In this study, simultaneous fMRI and EEG measurements were performed during a visually guided motor execution task in order to investigate whether the mu rhythm in the scalp EEG is an indication of the activity in the sensorimotor cortex. In addition, a new method was proposed for reconstruction of the cortical EEG activity through the fusion of fMRI and EEG data. A suppression of mu rhythm appeared around the lateral central electrode sites, just above the sensorimotor cortex, in association with the activity in that region. During a visually guided motor execution task, the alpha rhythms at the occipital electrode sites and the alpha rhythm at the central electrode sites also showed a correlation with the fMRI signal in the occipital and the supplementary motor cortices, respectively. This method allows the investigation of the scalp EEG origin with the spatial precision of fMRI, while retaining dynamic properties of the cortex with the temporal precision of EEG.

Effective Connectivity Modeling for fMRI: Six Issues and Possible Solutions Using Linear Dynamic Systems

Analysis of directionally specific or causal interactions between regions in functional magnetic resonance imaging (fMRI) data has proliferated. Here we identify six issues with existing effective connectivity methods that need to be addressed. The issues are discussed within the framework of linear dynamic systems for fMRI (LDSf). The first concerns the use of deterministic models to identify inter-regional effective connectivity. We show that deterministic dynamics are incapable of identifying the trial-to-trial variability typically investigated as the marker of connectivity while stochastic models can capture this variability. The second concerns the simplistic (constant) connectivity modeled by most methods. Connectivity parameters of the LDSf model can vary at the same timescale as the input data. Further, extending LDSf to mixtures of multiple models provides more robust connectivity variation. The third concerns the correct identification of the network itself including the number and anatomical origin of the network nodes. Augmentation of the LDSf state space can identify additional nodes of a network. The fourth concerns the locus of the signal used as a "node" in a network. A novel extension LDSf incorporating sparse canonical correlations can select most relevant voxels from an anatomically defined region based on connectivity. The fifth concerns connection interpretation. Individual parameter differences have received most attention. We present alternative network descriptors of connectivity changes which consider the whole network. The sixth concerns the temporal resolution of fMRI data relative to the timescale of the inter-regional interactions in the brain. LDSf includes an "instantaneous" connection term to capture connectivity occurring at timescales faster than the data resolution. The LDS framework can also be extended to statistically combine fMRI and EEG data. The LDSf framework is a promising foundation for effective connectivity analysis.

A short history of causal modeling of fMRI data

Twenty years ago, the discovery of the blood oxygen level dependent (BOLD) contrast and invention of functional magnetic resonance imaging (MRI) not only allowed for enhanced analyses of regional brain activity, but also laid the foundation for novel approaches to studying effective connectivity, which is essential for mechanistically interpretable accounts of neuronal systems. Dynamic causal modeling (DCM) and Granger causality (G-causality) modeling have since become the most frequently used techniques for inferring effective connectivity from fMRI data. In this paper, we provide a short historical overview of these approaches, describing milestones of their development from our subjective perspectives.

EEG-fMRI integration: a critical review of biophysical modeling and data analysis approaches

The diverse nature of cerebral activity, as measured using neuroimaging techniques, has been recognised long ago. It seems obvious that using single modality recordings can be limited when it comes to capturing its complex nature. Thus, it has been argued that moving to a multimodal approach will allow neuroscientists to better understand the dynamics and structure of this activity. This means that integrating information from different techniques, such as electroencephalography (EEG) and the blood oxygenated level dependent (BOLD) signal recorded with functional magnetic resonance imaging (fMRI), represents an important methodological challenge. In this work, we review the work that has been done thus far to derive EEG/fMRI integration approaches. This leads us to inspect the conditions under which such an integration approach could work or fail, and to disclose the types of scientific questions one could (and could not) hope to answer with it.

A review of multivariate methods for multimodal fusion of brain imaging data

The development of various neuroimaging techniques is rapidly improving the measurements of brain function/structure. However, despite improvements in individual modalities, it is becoming increasingly clear that the most effective research approaches will utilize multi-modal fusion, which takes advantage of the fact that each modality provides a limited view of the brain. The goal of multi-modal fusion is to capitalize on the strength of each modality in a joint analysis, rather than a separate analysis of each. This is a more complicated endeavor that must be approached more carefully and efficient methods should be developed to draw generalized and valid conclusions from high dimensional data with a limited number of subjects. Numerous research efforts have been reported in the field based on various statistical approaches, e.g. independent component analysis (ICA), canonical correlation analysis (CCA) and partial least squares (PLS). In this review paper, we survey a number of multivariate methods appearing in previous multimodal fusion reports, mostly fMRI with other modality, which were performed with or without prior information. A table for comparing optimization assumptions, purpose of the analysis, the need of priors, dimension reduction strategies and input data types is provided, which may serve as a valuable reference that helps readers understand the trade-offs of the 7 methods comprehensively. Finally, we evaluate 3 representative methods via simulation and give some suggestions on how to select an appropriate method based on a given research.

Towards a model-based integration of co-registered electroencephalography/functional magnetic resonance imaging data with realistic neural population meshes

Brain activity can be measured with several non-invasive neuroimaging modalities, but each modality has inherent limitations with respect to resolution, contrast and interpretability. It is hoped that multimodal integration will address these limitations by using the complementary features of already available data. However, purely statistical integration can prove problematic owing to the disparate signal sources. As an alternative, we propose here an advanced neural population model implemented on an anatomically sound cortical mesh with freely adjustable connectivity, which features proper signal expression through a realistic head model for the electroencephalogram (EEG), as well as a haemodynamic model for functional magnetic resonance imaging based on blood oxygen level dependent contrast (fMRI BOLD). It hence allows simultaneous and realistic predictions of EEG and fMRI BOLD from the same underlying model of neural activity. As proof of principle, we investigate here the influence on simulated brain activity of strengthening visual connectivity. In the future we plan to fit multimodal data with this neural population model. This promises novel, model-based insights into the brain's activity in sleep, rest and task conditions.

Critical comments on dynamic causal modelling

Dynamic causal modelling (DCM) (Friston et al., 2003) is a technique designed to investigate the influence between brain areas using time series data obtained by EEG/MEG or functional magnetic resonance imaging (fMRI). The basic idea is to fit various models to time series data, and select one of those models using Bayesian model comparison. Here, we present a critical evaluation of DCM in which we show that DCM can be challenged on several grounds. We will discuss three main points relating to combinatorial explosion, the validity of the model selection procedure, and problems with respect to model validation.

Multimodal functional network connectivity: an EEG-fMRI fusion in network space

EEG and fMRI recordings measure the functional activity of multiple coherent networks distributed in the cerebral cortex. Identifying network interaction from the complementary neuroelectric and hemodynamic signals may help to explain the complex relationships between different brain regions. In this paper, multimodal functional network connectivity (mFNC) is proposed for the fusion of EEG and fMRI in network space. First, functional networks (FNs) are extracted using spatial independent component analysis (ICA) in each modality separately. Then the interactions among FNs in each modality are explored by Granger causality analysis (GCA). Finally, fMRI FNs are matched to EEG FNs in the spatial domain using network-based source imaging (NESOI). Investigations of both synthetic and real data demonstrate that mFNC has the potential to reveal the underlying neural networks of each modality separately and in their combination. With mFNC, comprehensive relationships among FNs might be unveiled for the deep exploration of neural activities and metabolic responses in a specific task or neurological state.

Effective connectivity: influence, causality and biophysical modeling

This is the final paper in a Comments and Controversies series dedicated to "The identification of interacting networks in the brain using fMRI: Model selection, causality and deconvolution". We argue that discovering effective connectivity depends critically on state-space models with biophysically informed observation and state equations. These models have to be endowed with priors on unknown parameters and afford checks for model Identifiability. We consider the similarities and differences among Dynamic Causal Modeling, Granger Causal Modeling and other approaches. We establish links between past and current statistical causal modeling, in terms of Bayesian dependency graphs and Wiener-Akaike-Granger-Schweder influence measures. We show that some of the challenges faced in this field have promising solutions and speculate on future developments.

Dynamic causal modeling with neural fields

The aim of this paper is twofold: first, to introduce a neural field model motivated by a well-known neural mass model; second, to show how one can estimate model parameters pertaining to spatial (anatomical) properties of neuronal sources based on EEG or LFP spectra using Bayesian inference. Specifically, we consider neural field models of cortical activity as generative models in the context of dynamic causal modeling (DCM). This paper considers the simplest case of a single cortical source modeled by the spatiotemporal dynamics of hidden neuronal states on a bounded cortical surface or manifold. We build this model using multiple layers, corresponding to cortical lamina in the real cortical manifold. These layers correspond to the populations considered in classical (Jansen and Rit) neural mass models. This allows us to formulate a neural field model that can be reduced to a neural mass model using appropriate constraints on its spatial parameters. In turn, this enables one to compare and contrast the predicted responses from equivalent neural field and mass models respectively. We pursue this using empirical LFP data from a single electrode to show that the parameters controlling the spatial dynamics of cortical activity can be recovered, using DCM, even in the absence of explicit spatial information in observed data.