Multimodal approach to estimate the ocular movements during EEG recordings: A coupled tensor factorization method

Exploring Neural Mechanisms of Reward Processing Using Coupled Matrix Tensor Factorization: A Simultaneous EEG-fMRI Investigation

BACKGROUND

It is crucial to understand the neural feedback mechanisms and the cognitive decision-making of the brain during the processing of rewards. Here, we report the first attempt for a simultaneous electroencephalography (EEG)-functional magnetic resonance imaging (fMRI) study in a gambling task by utilizing tensor decomposition.

METHODS

First, the single-subject EEG data are represented as a third-order spectrogram tensor to extract frequency features. Next, the EEG and fMRI data are jointly decomposed into a superposition of multiple sources characterized by space-time-frequency profiles using coupled matrix tensor factorization (CMTF). Finally, graph-structured clustering is used to select the most appropriate model according to four quantitative indices.

RESULTS

The results clearly show that not only are the regions of interest (ROIs) found in other literature activated, but also the olfactory cortex and fusiform gyrus which are usually ignored. It is found that regions including the orbitofrontal cortex and insula are activated for both winning and losing stimuli. Meanwhile, regions such as the superior orbital frontal gyrus and anterior cingulate cortex are activated upon winning stimuli, whereas the inferior frontal gyrus, cingulate cortex, and medial superior frontal gyrus are activated upon losing stimuli.

CONCLUSION

This work sheds light on the reward-processing progress, provides a deeper understanding of brain function, and opens a new avenue in the investigation of neurovascular coupling via CMTF.

Early soft and flexible fusion of electroencephalography and functional magnetic resonance imaging via double coupled matrix tensor factorization for multisubject group analysis

Data fusion refers to the joint analysis of multiple datasets that provide different (e.g., complementary) views of the same task. In general, it can extract more information than separate analyses can. Jointly analyzing electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) measurements has been proved to be highly beneficial to the study of the brain function, mainly because these neuroimaging modalities have complementary spatiotemporal resolution: EEG offers good temporal resolution while fMRI is better in its spatial resolution. The EEG–fMRI fusion methods that have been reported so far ignore the underlying multiway nature of the data in at least one of the modalities and/or rely on very strong assumptions concerning the relation of the respective datasets. For example, in multisubject analysis, it is commonly assumed that the hemodynamic response function is a priori known for all subjects and/or the coupling across corresponding modes is assumed to be exact (hard). In this article, these two limitations are overcome by adopting tensor models for both modalities and by following soft and flexible coupling approaches to implement the multimodal fusion. The obtained results are compared against those of parallel independent component analysis and hard coupling alternatives, with both synthetic and real data (epilepsy and visual oddball paradigm). Our results demonstrate the clear advantage of using soft and flexible coupled tensor decompositions in scenarios that do not conform with the hard coupling assumption.

Identifying Oscillatory Hyperconnectivity and Hypoconnectivity Networks in Major Depression Using Coupled Tensor Decomposition

Previous researches demonstrate that major depression disorder (MDD) is associated with widespread network dysconnectivity, and the dynamics of functional connectivity networks are important to delineate the neural mechanisms of MDD. Neural oscillations exert a key role in coordinating the activity of remote brain regions, and various assemblies of oscillations can modulate different networks to support different cognitive tasks. Studies have demonstrated that the dysconnectivity of electroencephalography (EEG) oscillatory networks is related with MDD. In this study, we investigated the oscillatory hyperconnectivity and hypoconnectivity networks in MDD under a naturalistic and continuous stimuli condition of music listening. With the assumption that the healthy group and the MDD group share similar brain topology from the same stimuli and also retain individual brain topology for group differences, we applied the coupled nonnegative tensor decomposition algorithm on two adjacency tensors with the dimension of time × frequency × connectivity × subject, and imposed double-coupled constraints on spatial and spectral modes. The music-induced oscillatory networks were identified by a correlation analysis approach based on the permutation test between extracted temporal factors and musical features. We obtained three hyperconnectivity networks from the individual features of MDD and three hypoconnectivity networks from common features. The results demonstrated that the dysfunction of oscillatory networks could affect the involvement in music perception for MDD patients. Those oscillatory dysconnectivity networks may provide promising references to reveal the pathoconnectomics of MDD and potential biomarkers for the diagnosis of MDD.

Extraction of Common Task Features in EEG-fMRI Data Using Coupled Tensor-Tensor Decomposition

The fusion of simultaneously recorded EEG and fMRI data is of great value to neuroscience research due to the complementary properties of the individual modalities. Traditionally, techniques such as PCA and ICA, which rely on strong non-physiological assumptions such as orthogonality and statistical independence, have been used for this purpose. Recently, tensor decomposition techniques such as parallel factor analysis have gained more popularity in neuroimaging applications as they are able to inherently contain the multidimensionality of neuroimaging data and achieve uniqueness in decomposition without making strong assumptions. Previously, the coupled matrix-tensor decomposition (CMTD) has been applied for the fusion of the EEG and fMRI. Only recently the coupled tensor-tensor decomposition (CTTD) has been proposed. Here for the first time, we propose the use of CTTD of a 4th order EEG tensor (space, time, frequency, and participant) and 3rd order fMRI tensor (space, time, participant), coupled partially in time and participant domains, for the extraction of the task related features in both modalities. We used both the sensor-level and source-level EEG for the coupling. The phase shifted paradigm signals were incorporated as the temporal initializers of the CTTD to extract the task related features. The validation of the approach is demonstrated on simultaneous EEG-fMRI recordings from six participants performing an N-Back memory task. The EEG and fMRI tensors were coupled in 9 components out of which seven components had a high correlation (more than 0.85) with the task. The result of the fusion recapitulates the well-known attention network as being positively, and the default mode network working negatively time-locked to the memory task.

Coupled CP Decomposition of Simultaneous MEG-EEG Signals for Differentiating Oscillators During Photic Driving

Magnetoencephalography (MEG) and electroencephalography (EEG) are contemporary methods to investigate the function and organization of the brain. Simultaneously acquired MEG-EEG data are inherently multi-dimensional and exhibit coupling. This study uses a coupled tensor decomposition to extract the signal sources from MEG-EEG during intermittent photic stimulation (IPS). We employ the Coupled Semi-Algebraic framework for approximate CP decomposition via SImultaneous matrix diagonalization (C-SECSI). After comparing its performance with alternative methods using simulated benchmark data, we apply it to MEG-EEG recordings of 12 participants during IPS with fractions of the individual alpha frequency between 0.4 and 1.3. In the benchmark tests, C-SECSI is more accurate than SECSI and alternative methods, especially in ill-conditioned scenarios, e.g., involving collinear factors or noise sources with different variances. The component field-maps allow us to separate physiologically meaningful oscillations of visually evoked brain activity from background signals. The frequency signatures of the components identify either an entrainment to the respective stimulation frequency or its first harmonic, or an oscillation in the individual alpha band or theta band. In the group analysis of both, MEG and EEG data, we observe a reciprocal relationship between alpha and theta band oscillations. The coupled tensor decomposition using C-SECSI is a robust, powerful method for the extraction of physiologically meaningful sources from multidimensional biomedical data. Unsupervised signal source extraction is an essential solution for rendering advanced multi-modal signal acquisition technology accessible to clinical diagnostics, pre-surgical planning, and brain computer interface applications.

Unraveling Diagnostic Biomarkers of Schizophrenia Through Structure-Revealing Fusion of Multi-Modal Neuroimaging Data

Fusing complementary information from different modalities can lead to the discovery of more accurate diagnostic biomarkers for psychiatric disorders. However, biomarker discovery through data fusion is challenging since it requires extracting interpretable and reproducible patterns from data sets, consisting of shared/unshared patterns and of different orders. For example, multi-channel electroencephalography (EEG) signals from multiple subjects can be represented as a third-order tensor with modes: subject, time, and channel, while functional magnetic resonance imaging (fMRI) data may be in the form of subject by voxel matrices. Traditional data fusion methods rearrange higher-order tensors, such as EEG, as matrices to use matrix factorization-based approaches. In contrast, fusion methods based on coupled matrix and tensor factorizations (CMTF) exploit the potential multi-way structure of higher-order tensors. The CMTF approach has been shown to capture underlying patterns more accurately without imposing strong constraints on the latent neural patterns, i.e., biomarkers. In this paper, EEG, fMRI, and structural MRI (sMRI) data collected during an auditory oddball task (AOD) from a group of subjects consisting of patients with schizophrenia and healthy controls, are arranged as matrices and higher-order tensors coupled along the subject mode, and jointly analyzed using structure-revealing CMTF methods [also known as advanced CMTF (ACMTF)] focusing on unique identification of underlying patterns in the presence of shared/unshared patterns. We demonstrate that joint analysis of the EEG tensor and fMRI matrix using ACMTF reveals significant and biologically meaningful components in terms of differentiating between patients with schizophrenia and healthy controls while also providing spatial patterns with high resolution and improving the clustering performance compared to the analysis of only the EEG tensor. We also show that these patterns are reproducible, and study reproducibility for different model parameters. In comparison to the joint independent component analysis (jICA) data fusion approach, ACMTF provides easier interpretation of EEG data by revealing a single summary map of the topography for each component. Furthermore, fusion of sMRI data with EEG and fMRI through an ACMTF model provides structural patterns; however, we also show that when fusing data sets from multiple modalities, hence of very different nature, preprocessing plays a crucial role.