1
|
Liu Y, Zhang Y, Jiang Z, Kong W, Zou L. Exploring Neural Mechanisms of Reward Processing Using Coupled Matrix Tensor Factorization: A Simultaneous EEG-fMRI Investigation. Brain Sci 2023; 13:485. [PMID: 36979295 PMCID: PMC10046863 DOI: 10.3390/brainsci13030485] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Revised: 03/06/2023] [Accepted: 03/08/2023] [Indexed: 03/14/2023] Open
Abstract
BACKGROUND It is crucial to understand the neural feedback mechanisms and the cognitive decision-making of the brain during the processing of rewards. Here, we report the first attempt for a simultaneous electroencephalography (EEG)-functional magnetic resonance imaging (fMRI) study in a gambling task by utilizing tensor decomposition. METHODS First, the single-subject EEG data are represented as a third-order spectrogram tensor to extract frequency features. Next, the EEG and fMRI data are jointly decomposed into a superposition of multiple sources characterized by space-time-frequency profiles using coupled matrix tensor factorization (CMTF). Finally, graph-structured clustering is used to select the most appropriate model according to four quantitative indices. RESULTS The results clearly show that not only are the regions of interest (ROIs) found in other literature activated, but also the olfactory cortex and fusiform gyrus which are usually ignored. It is found that regions including the orbitofrontal cortex and insula are activated for both winning and losing stimuli. Meanwhile, regions such as the superior orbital frontal gyrus and anterior cingulate cortex are activated upon winning stimuli, whereas the inferior frontal gyrus, cingulate cortex, and medial superior frontal gyrus are activated upon losing stimuli. CONCLUSION This work sheds light on the reward-processing progress, provides a deeper understanding of brain function, and opens a new avenue in the investigation of neurovascular coupling via CMTF.
Collapse
Affiliation(s)
- Yuchao Liu
- School of Computer and Artificial Intelligence, Changzhou University, Changzhou 213164, China
| | - Yin Zhang
- School of Microelectronics and Control Engineering, Changzhou University, Changzhou 213164, China
| | - Zhongyi Jiang
- School of Computer and Artificial Intelligence, Changzhou University, Changzhou 213164, China
| | - Wanzeng Kong
- College of Computer Science, Hangzhou Dianzi University, Hangzhou 310018, China
- Key Laboratory of Brain Machine Collaborative Intelligence Foundation of Zhejiang Province, Hangzhou 310018, China
| | - Ling Zou
- School of Computer and Artificial Intelligence, Changzhou University, Changzhou 213164, China
- School of Microelectronics and Control Engineering, Changzhou University, Changzhou 213164, China
- Key Laboratory of Brain Machine Collaborative Intelligence Foundation of Zhejiang Province, Hangzhou 310018, China
| |
Collapse
|
2
|
Chatzichristos C, Kofidis E, Van Paesschen W, De Lathauwer L, Theodoridis S, Van Huffel S. Early soft and flexible fusion of electroencephalography and functional magnetic resonance imaging via double coupled matrix tensor factorization for multisubject group analysis. Hum Brain Mapp 2021; 43:1231-1255. [PMID: 34806255 PMCID: PMC8837580 DOI: 10.1002/hbm.25717] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2021] [Revised: 09/29/2021] [Accepted: 10/18/2021] [Indexed: 11/12/2022] Open
Abstract
Data fusion refers to the joint analysis of multiple datasets that provide different (e.g., complementary) views of the same task. In general, it can extract more information than separate analyses can. Jointly analyzing electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) measurements has been proved to be highly beneficial to the study of the brain function, mainly because these neuroimaging modalities have complementary spatiotemporal resolution: EEG offers good temporal resolution while fMRI is better in its spatial resolution. The EEG–fMRI fusion methods that have been reported so far ignore the underlying multiway nature of the data in at least one of the modalities and/or rely on very strong assumptions concerning the relation of the respective datasets. For example, in multisubject analysis, it is commonly assumed that the hemodynamic response function is a priori known for all subjects and/or the coupling across corresponding modes is assumed to be exact (hard). In this article, these two limitations are overcome by adopting tensor models for both modalities and by following soft and flexible coupling approaches to implement the multimodal fusion. The obtained results are compared against those of parallel independent component analysis and hard coupling alternatives, with both synthetic and real data (epilepsy and visual oddball paradigm). Our results demonstrate the clear advantage of using soft and flexible coupled tensor decompositions in scenarios that do not conform with the hard coupling assumption.
Collapse
Affiliation(s)
- Christos Chatzichristos
- Department of Electrical Engineering (ESAT), STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Leuven, Belgium
| | - Eleftherios Kofidis
- Department of Statistics and Insurance Science, University of Piraeus, Piraeus, Greece.,Computer Technology Institute and Press "Diophantus" (CTI), Patras, Greece
| | | | - Lieven De Lathauwer
- Department of Electrical Engineering (ESAT), STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Leuven, Belgium.,Engineering, Science and Technology, KU Leuven Kulak, Kortrijk, Belgium
| | - Sergios Theodoridis
- Department of Informatics and Telecommunications, National and Kapodistrian University of Athens, Athens, Greece.,Department of Electronic Systems, University of Aalborg, Aalborg, Denmark
| | - Sabine Van Huffel
- Department of Electrical Engineering (ESAT), STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Leuven, Belgium
| |
Collapse
|
3
|
Liu W, Wang X, Xu J, Chang Y, Hamalainen T, Cong F. Identifying Oscillatory Hyperconnectivity and Hypoconnectivity Networks in Major Depression Using Coupled Tensor Decomposition. IEEE Trans Neural Syst Rehabil Eng 2021; 29:1895-1904. [PMID: 34499604 DOI: 10.1109/tnsre.2021.3111564] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Previous researches demonstrate that major depression disorder (MDD) is associated with widespread network dysconnectivity, and the dynamics of functional connectivity networks are important to delineate the neural mechanisms of MDD. Neural oscillations exert a key role in coordinating the activity of remote brain regions, and various assemblies of oscillations can modulate different networks to support different cognitive tasks. Studies have demonstrated that the dysconnectivity of electroencephalography (EEG) oscillatory networks is related with MDD. In this study, we investigated the oscillatory hyperconnectivity and hypoconnectivity networks in MDD under a naturalistic and continuous stimuli condition of music listening. With the assumption that the healthy group and the MDD group share similar brain topology from the same stimuli and also retain individual brain topology for group differences, we applied the coupled nonnegative tensor decomposition algorithm on two adjacency tensors with the dimension of time × frequency × connectivity × subject, and imposed double-coupled constraints on spatial and spectral modes. The music-induced oscillatory networks were identified by a correlation analysis approach based on the permutation test between extracted temporal factors and musical features. We obtained three hyperconnectivity networks from the individual features of MDD and three hypoconnectivity networks from common features. The results demonstrated that the dysfunction of oscillatory networks could affect the involvement in music perception for MDD patients. Those oscillatory dysconnectivity networks may provide promising references to reveal the pathoconnectomics of MDD and potential biomarkers for the diagnosis of MDD.
Collapse
|
4
|
Jonmohamadi Y, Muthukumaraswamy S, Chen J, Roberts J, Crawford R, Pandey A. Extraction of Common Task Features in EEG-fMRI Data Using Coupled Tensor-Tensor Decomposition. Brain Topogr 2020; 33:636-650. [PMID: 32728794 DOI: 10.1007/s10548-020-00787-0] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2019] [Accepted: 07/23/2020] [Indexed: 01/20/2023]
Abstract
The fusion of simultaneously recorded EEG and fMRI data is of great value to neuroscience research due to the complementary properties of the individual modalities. Traditionally, techniques such as PCA and ICA, which rely on strong non-physiological assumptions such as orthogonality and statistical independence, have been used for this purpose. Recently, tensor decomposition techniques such as parallel factor analysis have gained more popularity in neuroimaging applications as they are able to inherently contain the multidimensionality of neuroimaging data and achieve uniqueness in decomposition without making strong assumptions. Previously, the coupled matrix-tensor decomposition (CMTD) has been applied for the fusion of the EEG and fMRI. Only recently the coupled tensor-tensor decomposition (CTTD) has been proposed. Here for the first time, we propose the use of CTTD of a 4th order EEG tensor (space, time, frequency, and participant) and 3rd order fMRI tensor (space, time, participant), coupled partially in time and participant domains, for the extraction of the task related features in both modalities. We used both the sensor-level and source-level EEG for the coupling. The phase shifted paradigm signals were incorporated as the temporal initializers of the CTTD to extract the task related features. The validation of the approach is demonstrated on simultaneous EEG-fMRI recordings from six participants performing an N-Back memory task. The EEG and fMRI tensors were coupled in 9 components out of which seven components had a high correlation (more than 0.85) with the task. The result of the fusion recapitulates the well-known attention network as being positively, and the default mode network working negatively time-locked to the memory task.
Collapse
Affiliation(s)
- Yaqub Jonmohamadi
- School of Electrical Engineering and Robotics, Queensland University of Technology, Brisbane, Australia.
| | | | - Joseph Chen
- School of Pharmacy, The University of Auckland, Auckland, New Zealand
| | - Jonathan Roberts
- School of Electrical Engineering and Robotics, Queensland University of Technology, Brisbane, Australia
| | - Ross Crawford
- Institute of Health and Biomedical Innovation, Queensland University of Technology, Brisbane, Australia
| | - Ajay Pandey
- School of Electrical Engineering and Robotics, Queensland University of Technology, Brisbane, Australia
| |
Collapse
|
5
|
Naskovska K, Lau S, Korobkov AA, Haueisen J, Haardt M. Coupled CP Decomposition of Simultaneous MEG-EEG Signals for Differentiating Oscillators During Photic Driving. Front Neurosci 2020; 14:261. [PMID: 32327966 PMCID: PMC7161426 DOI: 10.3389/fnins.2020.00261] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2019] [Accepted: 03/09/2020] [Indexed: 11/13/2022] Open
Abstract
Magnetoencephalography (MEG) and electroencephalography (EEG) are contemporary methods to investigate the function and organization of the brain. Simultaneously acquired MEG-EEG data are inherently multi-dimensional and exhibit coupling. This study uses a coupled tensor decomposition to extract the signal sources from MEG-EEG during intermittent photic stimulation (IPS). We employ the Coupled Semi-Algebraic framework for approximate CP decomposition via SImultaneous matrix diagonalization (C-SECSI). After comparing its performance with alternative methods using simulated benchmark data, we apply it to MEG-EEG recordings of 12 participants during IPS with fractions of the individual alpha frequency between 0.4 and 1.3. In the benchmark tests, C-SECSI is more accurate than SECSI and alternative methods, especially in ill-conditioned scenarios, e.g., involving collinear factors or noise sources with different variances. The component field-maps allow us to separate physiologically meaningful oscillations of visually evoked brain activity from background signals. The frequency signatures of the components identify either an entrainment to the respective stimulation frequency or its first harmonic, or an oscillation in the individual alpha band or theta band. In the group analysis of both, MEG and EEG data, we observe a reciprocal relationship between alpha and theta band oscillations. The coupled tensor decomposition using C-SECSI is a robust, powerful method for the extraction of physiologically meaningful sources from multidimensional biomedical data. Unsupervised signal source extraction is an essential solution for rendering advanced multi-modal signal acquisition technology accessible to clinical diagnostics, pre-surgical planning, and brain computer interface applications.
Collapse
Affiliation(s)
- Kristina Naskovska
- Communications Research Laboratory, Ilmenau University of Technology, Ilmenau, Germany
| | - Stephan Lau
- Institute of Biomedical Engineering and Informatics, Ilmenau University of Technology, Ilmenau, Germany.,School of Computer Science, Australian Institute for Machine Learning, The University of Adelaide, Adelaide, SA, Australia
| | - Alexey A Korobkov
- Communications Research Laboratory, Ilmenau University of Technology, Ilmenau, Germany.,Institute for Radio-Electronics and Telecommunications, Department for Radio-Electronic and Telecommunication Systems, Kazan National Research Technical University named after A.N Tupolev-KAI, Kazan, Russia
| | - Jens Haueisen
- Institute of Biomedical Engineering and Informatics, Ilmenau University of Technology, Ilmenau, Germany
| | - Martin Haardt
- Communications Research Laboratory, Ilmenau University of Technology, Ilmenau, Germany
| |
Collapse
|
6
|
Fabietti M, Mahmud M, Lotfi A. Effectiveness of Employing Multimodal Signals in Removing Artifacts from Neuronal Signals: An Empirical Analysis. Brain Inform 2020. [DOI: 10.1007/978-3-030-59277-6_17] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022] Open
|
7
|
Acar E, Schenker C, Levin-Schwartz Y, Calhoun VD, Adali T. Unraveling Diagnostic Biomarkers of Schizophrenia Through Structure-Revealing Fusion of Multi-Modal Neuroimaging Data. Front Neurosci 2019; 13:416. [PMID: 31130835 PMCID: PMC6509223 DOI: 10.3389/fnins.2019.00416] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2018] [Accepted: 04/11/2019] [Indexed: 11/13/2022] Open
Abstract
Fusing complementary information from different modalities can lead to the discovery of more accurate diagnostic biomarkers for psychiatric disorders. However, biomarker discovery through data fusion is challenging since it requires extracting interpretable and reproducible patterns from data sets, consisting of shared/unshared patterns and of different orders. For example, multi-channel electroencephalography (EEG) signals from multiple subjects can be represented as a third-order tensor with modes: subject, time, and channel, while functional magnetic resonance imaging (fMRI) data may be in the form of subject by voxel matrices. Traditional data fusion methods rearrange higher-order tensors, such as EEG, as matrices to use matrix factorization-based approaches. In contrast, fusion methods based on coupled matrix and tensor factorizations (CMTF) exploit the potential multi-way structure of higher-order tensors. The CMTF approach has been shown to capture underlying patterns more accurately without imposing strong constraints on the latent neural patterns, i.e., biomarkers. In this paper, EEG, fMRI, and structural MRI (sMRI) data collected during an auditory oddball task (AOD) from a group of subjects consisting of patients with schizophrenia and healthy controls, are arranged as matrices and higher-order tensors coupled along the subject mode, and jointly analyzed using structure-revealing CMTF methods [also known as advanced CMTF (ACMTF)] focusing on unique identification of underlying patterns in the presence of shared/unshared patterns. We demonstrate that joint analysis of the EEG tensor and fMRI matrix using ACMTF reveals significant and biologically meaningful components in terms of differentiating between patients with schizophrenia and healthy controls while also providing spatial patterns with high resolution and improving the clustering performance compared to the analysis of only the EEG tensor. We also show that these patterns are reproducible, and study reproducibility for different model parameters. In comparison to the joint independent component analysis (jICA) data fusion approach, ACMTF provides easier interpretation of EEG data by revealing a single summary map of the topography for each component. Furthermore, fusion of sMRI data with EEG and fMRI through an ACMTF model provides structural patterns; however, we also show that when fusing data sets from multiple modalities, hence of very different nature, preprocessing plays a crucial role.
Collapse
Affiliation(s)
- Evrim Acar
- Machine Intelligence Department, Simula Metropolitan Center for Digital Engineering, Oslo, Norway
| | - Carla Schenker
- Machine Intelligence Department, Simula Metropolitan Center for Digital Engineering, Oslo, Norway
| | - Yuri Levin-Schwartz
- Department of Environmental Medicine and Public Health, Icahn School of Medicine at Mount Sinai, New York, NY, United States
| | - Vince D. Calhoun
- The Mind Research Network, Albuquerque, NM, United States
- Department of Electrical and Computer Engineering, University of New Mexico, Albuquerque, NM, United States
| | - Tülay Adali
- Department of Computer Science and Electrical Engineering, University of Maryland Baltimore County, Baltimore, MD, United States
| |
Collapse
|