Improving human brain mapping via joint inversion of brain electrodynamics and the BOLD signal

Neural Activity and Decoding of Action Observation Using Combined EEG and fNIRS Measurement

In a social world, observing the actions of others is fundamental to understanding what they are doing, as well as their intentions and feelings. Studies of the neural basis and decoding of action observation are important for understanding action-related processes and have implications for cognitive, social neuroscience, and human-machine interaction (HMI). In the current study, we first investigated temporal-spatial dynamics during action observation using a combined 64-channel electroencephalography (EEG) and 48-channel functional near-infrared spectroscopy (fNIRS) system. We measured brain activation while 16 healthy participants observed three action tasks: (1) grasping a cup with the intention of drinking; (2) grasping a cup with the intention of moving it; and (3) touching a cup with an unclear intention. The EEG and fNIRS source analysis results revealed the dynamic involvement of both the mirror neuron system (MNS) and the theory of mind (ToM)/mentalizing network during action observation. The source analysis results suggested that the extent to which these two systems were engaged was determined by the clarity of the intention of the observed action. Based on the difference in neural activity observed among different action-observation tasks in the first experiment, we conducted a second experiment to classify the neural processes underlying action observation using a feature classification method. We constructed complex brain networks based on the EEG and fNIRS data. Fusing features from both EEG and fNIRS complex brain networks resulted in a classification accuracy of 72.7% for the three action observation tasks. This study provides a theoretical and empirical basis for elucidating the neural mechanisms of action observation and intention understanding, and a feasible method for decoding the underlying neural processes.

Neural Basis of Action Observation and Understanding From First- and Third-Person Perspectives: An fMRI Study

Understanding the intentions of others while observing their actions is a fundamental aspect of social behavior. However, the differences in neural and functional mechanisms between observing actions from the first-person perspective (1PP) and third-person perspective (3PP) are poorly understood. The present study had two aims: (1) to delineate the neural basis of action observation and understanding from the 1PP and 3PP; and (2) to identify whether there are different activation patterns during action observation and understanding from 1PP and 3PP. We used a blocked functional magnetic resonance imaging (fMRI) experimental design. Twenty-six right-handed participants observed interactions between the right hand and a cup from 1PP and 3PP. The results indicated that both 1PP and 3PP were associated with similar patterns of activation in key areas of the mirror neuron system underlying action observation and understanding. Importantly, besides of the core network of mirror neuron system, we also found that parts of the basal ganglia and limbic system were involved in action observation in both the 1PP and 3PP tasks, including the putamen, insula and hippocampus, providing a more complete understanding of the neural basis for action observation and understanding. Moreover, compared with the 3PP, the 1PP task caused more extensive and stronger activation. In contrast, the opposite comparison revealed that no regions exhibited significantly more activation in the 3PP compared with the 1PP condition. The current results have important implications for understanding the role of the core network underlying the mirror neuron system, as well as parts of the basal ganglia and limbic system, during action observation and understanding.

An algorithm for separation of mixed sparse and Gaussian sources

Independent component analysis (ICA) is a ubiquitous method for decomposing complex signal mixtures into a small set of statistically independent source signals. However, in cases in which the signal mixture consists of both nongaussian and Gaussian sources, the Gaussian sources will not be recoverable by ICA and will pollute estimates of the nongaussian sources. Therefore, it is desirable to have methods for mixed ICA/PCA which can separate mixtures of Gaussian and nongaussian sources. For mixtures of purely Gaussian sources, principal component analysis (PCA) can provide a basis for the Gaussian subspace. We introduce a new method for mixed ICA/PCA which we call Mixed ICA/PCA via Reproducibility Stability (MIPReSt). Our method uses a repeated estimations technique to rank sources by reproducibility, combined with decomposition of multiple subsamplings of the original data matrix. These multiple decompositions allow us to assess component stability as the size of the data matrix changes, which can be used to determinine the dimension of the nongaussian subspace in a mixture. We demonstrate the utility of MIPReSt for signal mixtures consisting of simulated sources and real-word (speech) sources, as well as mixture of unknown composition.

Reproducible paired sources from concurrent EEG-fMRI data using BICAR

We introduce BICAR, an algorithm for obtaining robust, reproducible pairs of temporal and spatial components at the individual subject level from concurrent electroencephalographic and functional magnetic resonance imaging data. BICAR assigns a task-independent measure of component quality, reproducibility, to each paired source. Under BICAR a reproducibility cutoff is derived that can be used to objectively discard spuriously paired EEG-fMRI components. BICAR is run on minimally processed data: fMRI images undergo the standard preprocessing steps (alignment, motion correction, etc.) and EEG data, after scanner artifact removal, are simply bandpass filtered. This minimal processing allows the secondary scoring of the same set of BICAR components for a variety of different endpoint analyses; in this manuscript we propose a general method for scoring components for task event synchronization (evoked response analysis), but scoring using many other criteria, for example frequency content, are possible. BICAR is applied to five subjects performing a visual search task, and among the most reproducible components we find biologically relevant paired sources involved in visual processing, motor planning, execution, and attention.

BICAR: a new algorithm for multiresolution spatiotemporal data fusion

We introduce a method for spatiotemporal data fusion and demonstrate its performance on three constructed data sets: one entirely simulated, one with temporal speech signals and simulated spatial images, and another with recorded music time series and astronomical images defining the spatial patterns. Each case study is constructed to present specific challenges to test the method and demonstrate its capabilities. Our algorithm, BICAR (Bidirectional Independent Component Averaged Representation), is based on independent component analysis (ICA) and extracts pairs of temporal and spatial sources from two data matrices with arbitrarily different spatiotemporal resolution. We pair the temporal and spatial sources using a physical transfer function that connects the dynamics of the two. BICAR produces a hierarchy of sources ranked according to reproducibility; we show that sources which are more reproducible are more similar to true (known) sources. BICAR is robust to added noise, even in a "worst case" scenario where all physical sources are equally noisy. BICAR is also relatively robust to misspecification of the transfer function. BICAR holds promise as a useful data-driven assimilation method in neuroscience, earth science, astronomy, and other signal processing domains.

Within-subject joint independent component analysis of simultaneous fMRI/ERP in an auditory oddball paradigm

The integration of event-related potential (ERP) and functional magnetic resonance imaging (fMRI) can contribute to characterizing neural networks with high temporal and spatial resolution. This research aimed to determine the sensitivity and limitations of applying joint independent component analysis (jICA) within-subjects, for ERP and fMRI data collected simultaneously in a parametric auditory frequency oddball paradigm. In a group of 20 subjects, an increase in ERP peak amplitude ranging 1-8 μV in the time window of the P300 (350-700 ms), and a correlated increase in fMRI signal in a network of regions including the right superior temporal and supramarginal gyri, was observed with the increase in deviant frequency difference. JICA of the same ERP and fMRI group data revealed activity in a similar network, albeit with stronger amplitude and larger extent. In addition, activity in the left pre- and post-central gyri, likely associated with right hand somato-motor response, was observed only with the jICA approach. Within-subject, the jICA approach revealed significantly stronger and more extensive activity in the brain regions associated with the auditory P300 than the P300 linear regression analysis. The results suggest that with the incorporation of spatial and temporal information from both imaging modalities, jICA may be a more sensitive method for extracting common sources of activity between ERP and fMRI.

Understanding actions of others: the electrodynamics of the left and right hemispheres. A high-density EEG neuroimaging study

Background

When we observe an individual performing a motor act (e.g. grasping a cup) we get two types of information on the basis of how the motor act is done and the context: what the agent is doing (i.e. grasping) and the intention underlying it (i.e. grasping for drinking). Here we examined the temporal dynamics of the brain activations that follow the observation of a motor act and underlie the observer's capacity to understand what the agent is doing and why.

Methodology/Principal Findings

Volunteers were presented with two-frame video-clips. The first frame (T0) showed an object with or without context; the second frame (T1) showed a hand interacting with the object. The volunteers were instructed to understand the intention of the observed actions while their brain activity was recorded with a high-density 128-channel EEG system. Visual event-related potentials (VEPs) were recorded time-locked with the frame showing the hand-object interaction (T1). The data were analyzed by using electrical neuroimaging, which combines a cluster analysis performed on the group-averaged VEPs with the localization of the cortical sources that give rise to different spatio-temporal states of the global electrical field. Electrical neuroimaging results revealed four major steps: 1) bilateral posterior cortical activations; 2) a strong activation of the left posterior temporal and inferior parietal cortices with almost a complete disappearance of activations in the right hemisphere; 3) a significant increase of the activations of the right temporo-parietal region with simultaneously co-active left hemispheric sources, and 4) a significant global decrease of cortical activity accompanied by the appearance of activation of the orbito-frontal cortex.

Conclusions/Significance

We conclude that the early striking left hemisphere involvement is due to the activation of a lateralized action-observation/action execution network. The activation of this lateralized network mediates the understanding of the goal of object-directed motor acts (mirror mechanism). The successive right hemisphere activation indicates that this hemisphere plays an important role in understanding the intention of others.