Four-way multimodal fusion of 7 T imaging data using an mCCA+jICA model in first-episode schizophrenia

The covariant structural and functional neuro-correlates of cognitive impairments in patients with end-stage renal diseases

Introduction

Cognitive impairment (CI) is a common complication of end-stage renal disease (ESRD) that is associated with structural and functional changes in the brain. However, whether a joint structural and functional alteration pattern exists that is related to CI in ESRD is unclear.

Methods

In this study, instead of looking at brain structure and function separately, we aim to investigate the covariant characteristics of both functional and structural aspects. Specifically, we took the fusion analysis approach, namely, multimodal canonical correlation analysis and joint independent component analysis (mCCA+jICA), to jointly study the discriminative features in gray matter volume (GMV) measured by T1-weighted (T1w) MRI, fractional anisotropy (FA) in white matter measured by diffusion MRI, and the amplitude of low-frequency fluctuation (ALFF) measured by blood oxygenation-level-dependent (BOLD) MRI in 78 ESRD patients versus 64 healthy controls (HCs), followed by a mediation effect analysis to explore the relationship between neuroimaging findings, cognitive impairments and uremic toxins.

Results

Two joint group-discriminative independent components (ICs) were found to show covariant abnormalities across FA, GMV, and ALFF (all p < 0.05). The most dominant joint IC revealed associative patterns of alterations of GMV (in the precentral gyrus, occipital lobe, temporal lobe, parahippocampal gyrus, and hippocampus), alterations of ALFF (in the precuneus, superior parietal gyrus, and superior occipital gyrus), and of white matter FA (in the corticospinal tract and inferior frontal occipital fasciculus). Another significant IC revealed associative alterations of GMV (in the dorsolateral prefrontal and orbitofrontal cortex) and FA (in the forceps minor). Moreover, the brain changes identified by FA and GMV in the above-mentioned brain regions were found to mediate the negative correlation between serum phosphate and mini-mental state examination (MMSE) scores (all p < 0.05).

Conclusion

The mCCA+jICA method was demonstrated to be capable of revealing covariant abnormalities across neuronal features of different types in ESRD patients as contrasted to HCs, and joint brain changes may play an important role in mediating the relationship between serum toxins and CIs in ESRD. Our results show the mCCA+jICA fusion analysis approach may provide new insights into similar neurobiological studies.

Structure of brain grey and white matter in infants with spastic cerebral palsy and periventricular white matter injury

AIM

To investigate the possible covariation of grey matter volume (GMV) and white matter fractional anisotropy in infants with spastic cerebral palsy (CP) and periventricular white matter injury.

METHOD

Thirty-nine infants with spastic CP and 25 typically developing controls underwent structural magnetic resonance imaging and diffusion tensor imaging. Multimodal canonical correlation analysis with joint independent component analysis were used to capture differences in GMV and fractional anisotropy between groups. Correlation analysis was performed between imaging findings and clinical features.

RESULTS

Infants with spastic CP showed one joint group-discriminating component (i.e. GMV-fractional anisotropy) associated with regions in the cortico-basal ganglia-thalamo-cortical loop and in the corpus callosum compared to typically developing controls and one modality-specific group-discriminating component (i.e. GMV). Significant negative correlations were found between loadings in certain regions and the motor function score in spastic CP.

INTERPRETATION

In infants with spastic CP, covarying GMV-fractional anisotropy and altered GMV in specific regions were implicated in motor dysfunction, which confirmed that simultaneous GMV and fractional anisotropy changes underly motor deficits, but might also extend to sensory, cognitive, or visual dysfunction. These findings also suggest that multimodal fusion analysis allows for a more comprehensive understanding of the relevance between grey and white matter structures and its crucial role in the neuropathological mechanisms of spastic CP.

Exploring changes in brain function in IBD patients using SPCCA: a study of simultaneous EEG-fMRI

Research on functional changes in the brain of inflammatory bowel disease (IBD) patients is emerging around the world, which brings new perspectives to medical research. In this paper, the methods of canonical correlation analysis (CCA), kernel canonical correlation analysis (KCCA), and sparsity preserving canonical correlation analysis (SPCCA) were applied to the fusion of simultaneous EEG-fMRI data from 25 IBD patients and 15 healthy individuals. The CCA, KCCA and SPCCA fusion methods were used for data processing to compare the results obtained by the three methods. The results clearly show that there is a significant difference in the activation intensity between IBD and healthy control (HC), not only in the frontal lobe (p < 0.01) and temporal lobe (p < 0.01) regions, but also in the posterior cingulate gyrus (p < 0.01), gyrus rectus (p < 0.01), and amygdala (p < 0.01) regions, which are usually neglected. The mean difference in the SPCCA activation intensity was 60.1. However, the mean difference in activation intensity was only 36.9 and 49.8 by using CCA and KCCA. In addition, the correlation of the relevant components selected during the SPCCA calculation was high, with correlation components of up to 0.955; alternatively, the correlations obtained from CCA and KCCA calculations were only 0.917 and 0.926, respectively. It can be seen that SPCCA is indeed superior to CCA and KCCA in processing high-dimensional multimodal data. This work reveals the process of analyzing the brain activation state in IBD disease, provides a further perspective for the study of brain function, and opens up a new avenue for studying the SPCCA method and the change in the intensity of brain activation in IBD disease.

Data-driven multimodal fusion: approaches and applications in psychiatric research

In the era of big data, where vast amounts of information are being generated and collected at an unprecedented rate, there is a pressing demand for innovative data-driven multi-modal fusion methods. These methods aim to integrate diverse neuroimaging perspectives to extract meaningful insights and attain a more comprehensive understanding of complex psychiatric disorders. However, analyzing each modality separately may only reveal partial insights or miss out on important correlations between different types of data. This is where data-driven multi-modal fusion techniques come into play. By combining information from multiple modalities in a synergistic manner, these methods enable us to uncover hidden patterns and relationships that would otherwise remain unnoticed. In this paper, we present an extensive overview of data-driven multimodal fusion approaches with or without prior information, with specific emphasis on canonical correlation analysis and independent component analysis. The applications of such fusion methods are wide-ranging and allow us to incorporate multiple factors such as genetics, environment, cognition, and treatment outcomes across various brain disorders. After summarizing the diverse neuropsychiatric magnetic resonance imaging fusion applications, we further discuss the emerging neuroimaging analyzing trends in big data, such as N-way multimodal fusion, deep learning approaches, and clinical translation. Overall, multimodal fusion emerges as an imperative approach providing valuable insights into the underlying neural basis of mental disorders, which can uncover subtle abnormalities or potential biomarkers that may benefit targeted treatments and personalized medical interventions.

Alterations in grey matter structure linked to frequency-specific cortico-subcortical connectivity in schizophrenia via multimodal data fusion

Schizophrenia (SZ) is a complex psychiatric disorder that is currently defined by symptomatic and behavioral, rather than biological, criteria. Neuroimaging is an appealing avenue for SZ biomarker development, as several neuroimaging-based studies comparing individuals with SZ to healthy controls (HC) have shown measurable group differences in brain structure, as well as functional brain alterations in both static and dynamic functional network connectivity (sFNC and dFNC, respectively). The recently proposed filter-banked connectivity (FBC) method extends the standard dFNC sliding-window approach to estimate FNC within an arbitrary number of distinct frequency bands. The initial implementation used a set of filters spanning the full connectivity spectral range, providing a unified approach to examine both sFNC and dFNC in a single analysis. Initial FBC results found that individuals with SZ spend more time in a less structured, more disconnected low-frequency (i.e., static) FNC state than HC, as well as preferential SZ occupancy in high-frequency connectivity states, suggesting a frequency-specific component underpinning the functional dysconnectivity observed in SZ. Building on these findings, we sought to link such frequency-specific patterns of FNC to covarying data-driven structural brain networks in the context of SZ. Specifically, we employ a multi-set canonical correlation analysis + joint independent components analysis (mCCA + jICA) data fusion framework to study the connection between grey matter volume (GMV) maps and FBC states across the full connectivity frequency spectrum. Our multimodal analysis identified two joint sources that captured co-varying patterns of frequency-specific functional connectivity and alterations in GMV with significant group differences in loading parameters between the SZ group and HC. The first joint source linked frequency-modulated connections between the subcortical and sensorimotor networks and GMV alterations in the frontal and temporal lobes, while the second joint source identified a relationship between low-frequency cerebellar-sensorimotor connectivity and structural changes in both the cerebellum and motor cortex. Together, these results show a strong connection between cortico-subcortical functional connectivity at both high and low frequencies and alterations in cortical GMV that may be relevant to the pathogenesis and pathophysiology of SZ.

Inter-individual single-trial classification of MEG data using M-CCA

Neuroscientific studies often involve some form of group analysis over multiple participants. This requires alignment of recordings across participants. A naive solution is to assume that participants' recordings can be aligned anatomically in sensor space. However, this assumption is likely violated due to anatomical and functional differences between individual brains. In magnetoencephalography (MEG) recordings the problem of inter-subject alignment is exacerbated by the susceptibility of MEG to individual cortical folding patterns as well as the inter-subject variability of sensor locations over the brain due to the use of a fixed helmet. Hence, an approach to combine MEG data over individual brains should relax the assumptions that a) brain anatomy and function are tightly linked and b) that the same sensors capture functionally comparable brain activation across individuals. Here we use multiset canonical correlation analysis (M-CCA) to find a common representation of MEG activations recorded from 15 participants performing a grasping task. The M-CCA algorithm was applied to transform the data of a set of multiple participants into a common space with maximum correlation between participants. Importantly, we derive a method to transform data from a new, previously unseen participant into this common representation. This makes it useful for applications that require transfer of models derived from a group of individuals to new individuals. We demonstrate the usefulness and superiority of the approach with respect to previously used approaches. Finally, we show that our approach requires only a small number of labeled data from the new participant. The proposed method demonstrates that functionally motivated common spaces have potential applications in reducing training time of online brain-computer interfaces, where models can be pre-trained on previous participants/sessions. Moreover, inter-subject alignment via M-CCA has the potential for combining data of different participants and could become helpful in future endeavors on large open datasets.

Resting-state brain functional alterations and their genetic mechanisms in drug-naive first-episode psychosis

Extensive research has established the presence of resting-state brain functional damage in psychosis. However, the genetic mechanisms of such disease phenotype are yet to be unveiled. We investigated resting-state brain functional alterations in patients with drug-naive first-episode psychosis (DFP) by performing a neuroimaging meta-analysis of 8 original studies comprising 500 patients and 469 controls. Combined with the Allen Human Brain Atlas, we further conducted transcriptome-neuroimaging spatial correlations to identify genes whose expression levels were linked to brain functional alterations in DFP, followed by a range of gene functional characteristic analyses. Meta-analysis revealed a mixture of increased and decreased brain function in widespread areas including the default-mode, visual, motor, striatal, and cerebellar systems in DFP. Moreover, these brain functional alterations were spatially associated with the expression of 1662 genes, which were enriched for molecular functions, cellular components, and biological processes of the cerebral cortex, as well as psychiatric disorders including schizophrenia. Specific expression analyses demonstrated that these genes were specifically expressed in the brain tissue, in cortical neurons and immune cells, and during nearly all developmental periods. Concurrently, the genes could construct a protein-protein interaction network supported by hub genes and were linked to multiple behavioral domains including emotion, attention, perception, and motor. Our findings provide empirical evidence for the notion that brain functional damage in DFP involves a complex interaction of polygenes with various functional characteristics.

Hybrid brain model accurately predict human procrastination behavior

Procrastination behavior is quite ubiquitous, and should warrant cautions to us owing to its significant influences in poor mental health, low subjective well-beings and bad academic performance. However, how to identify this behavioral problem have not yet to be fully elucidated. 1132 participants were recruited as distribution of benchmark. 81 high trait procrastinators (HP) and matched low trait procrastinators (LP) were screened. To address this issue, we have built upon the hybrid brain model by using hierarchical machine learning techniques to classify HP and LP with multi-modalities neuroimaging data (i.e., grey matter volume, fractional anisotropy, static/dynamic amplitude of low frequency fluctuation and static/dynamic degree centrality). Further, we capitalized on the multiple Canonical Correlation Analysis (mCCA) and joint Independent Component Analysis algorithm (mCCA + jICA) to clarify its fusion neural components as well. The hybrid brain model showed high accuracy to discriminate HP and LP (accuracy rate = 87.04%, sensitivity rate = 86.42%, specificity rate = 85.19%). Moreover, results of mCCA + jICA model revealed several joint-discriminative neural independent components (ICs) of this classification, showing wider co-variants of frontoparietal cortex and hippocampus networks. In addition, this study demonstrated three modal-specific discriminative ICs for classification, highlighting the temporal variants of brain local and global natures in ventromedial prefrontal cortex (vmPFC) and PHC in HP. To sum-up, this research developed a hybrid brain model to identify trait procrastination with high accuracy, and further revealed the neural hallmarks of this trait by integrating neuroimaging fusion data.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-021-09765-z.

Multimodal MRI data fusion reveals distinct structural, functional and neurochemical correlates of heavy cannabis use

Heavy cannabis use (HCU) is frequently associated with a plethora of cognitive, psychopathological and sensorimotor phenomena. Although HCU is frequent, specific patterns of abnormal brain structure and function underlying HCU in individuals presenting without cannabis-use disorder or other current and life-time major mental disorders are unclear at present. This multimodal magnetic resonance imaging (MRI) study examined resting-state functional MRI (rs-fMRI) and structural MRI (sMRI) data from 24 persons with HCU and 16 controls. Parallel independent component analysis (p-ICA) was used to examine covarying components among grey matter volume (GMV) maps computed from sMRI and intrinsic neural activity (INA), as derived from amplitude of low-frequency fluctuations (ALFF) maps computed from rs-fMRI data. Further, we used JuSpace toolbox for cross-modal correlations between MRI-based modalities with nuclear imaging derived estimates, to examine specific neurotransmitter system changes underlying HCU. We identified two transmodal components, which significantly differed between the HCU and controls (GMV: p = 0.01, ALFF p = 0.03, respectively). The GMV component comprised predominantly cerebello-temporo-thalamic regions, whereas the INA component included fronto-parietal regions. Across HCU, loading parameters of both components were significantly associated with distinct HCU behavior. Finally, significant associations between GMV and the serotonergic system as well as between INA and the serotonergic, dopaminergic and μ-opioid receptor system were detected. This study provides novel multimodal neuromechanistic insights into HCU suggesting co-altered structure/function-interactions in neural systems subserving cognitive and sensorimotor functions.

Ultra-high field neuroimaging in psychosis: A narrative review

Schizophrenia and related psychoses are complex neuropsychiatric diseases representing dysconnectivity across multiple scales, through the micro (cellular), meso (brain network), manifest (behavioral), and social (interpersonal) levels. In vivo human neuroimaging, particularly at ultra-high field (UHF), offers unprecedented opportunity to examine multiscale dysconnectivity in psychosis. In this review, we provide an overview of the literature to date on UHF in psychosis, focusing on microscale findings from magnetic resonance spectroscopy (MRS), mesoscale studies on structural and functional magnetic resonance imaging (fMRI), and multiscale studies assessing multiple neuroimaging modalities and relating UHF findings to behavior. We highlight key insights and considerations from multiscale and longitudinal studies and provide recommendations for future research on UHF neuroimaging in psychosis.

Machine learning approaches for parsing comorbidity/heterogeneity in antisociality and substance use disorders: A primer

By some accounts, as many as 93% of individuals diagnosed with antisocial personality disorder (ASPD) or psychopathy also meet criteria for some form of substance use disorder (SUD). This high level of comorbidity, combined with an overlapping biopsychosocial profile, and potentially interacting features, has made it difficult to delineate the shared/unique characteristics of each disorder. Moreover, while rarely acknowledged, both SUD and antisociality exist as highly heterogeneous disorders in need of more targeted parcellation. While emerging data-driven nosology for psychiatric disorders (e.g., Research Domain Criteria (RDoC), Hierarchical Taxonomy of Psychopathology (HiTOP)) offers the opportunity for a more systematic delineation of the externalizing spectrum, the interrogation of large, complex neuroimaging-based datasets may require data-driven approaches that are not yet widely employed in psychiatric neuroscience. With this in mind, the proposed article sets out to provide an introduction into machine learning methods for neuroimaging that can help parse comorbid, heterogeneous externalizing samples. The modest machine learning work conducted to date within the externalizing domain demonstrates the potential utility of the approach but remains highly nascent. Within the paper, we make suggestions for how future work can make use of machine learning methods, in combination with emerging psychiatric nosology systems, to further diagnostic and etiological understandings of the externalizing spectrum. Finally, we briefly consider some challenges that will need to be overcome to encourage further progress in the field.

Fusion analysis of gray matter and white matter in subjective cognitive decline and mild cognitive impairment by multimodal CCA-joint ICA

BACKGROUND

Previous multimodal neuroimaging studies analyzed each dataset independently in subjective cognitive decline (SCD) and mild cognitive impairment (MCI), missing the cross-information. Multi-modal fusion analysis can provide more integral and comprehensive information regarding the brain. There has been a paucity of research on fusion analysis of sMRI and DTI in SCD and MCI.

MATERIALS AND METHODS

In the present study, we conducted fusion analysis of structural MRI and DTI by applying multimodal canonical correlation analysis with joint independent component analysis (mCCA-jICA) to capture the cross-information of gray matter (GM) and white matter (WM) in 62 SCD patients, 99 MCI patients, and 70 healthy controls (HCs). We further analyzed correlations between the mixing coefficients of mCCA-jICA and neuropsychological scores among the three groups.

RESULTS

A set of joint-discriminative independent components of GM and fractional anisotropy (FA) exhibited significant links between SCD and HCs, as well as between MCI and HCs. The covariant abnormalities primarily involved the frontal lobe/middle temporal gyrus/calcarine sulcus-anterior thalamic radiation/superior longitudinal fasciculus in SCD, and middle temporal gyrus/ fusiform gyrus/caudate necleus-forceps minor/anterior thalamic radiation in MCI. There was no significant difference between SCD and MCI groups.

CONCLUSIONS

The covariant GM-WM abnormalities in SCD and MCI were found in specific brain regions involved in cognitive processing, which confirms the simultaneous GM and WM changes underlying cognitive decline. These findings suggest that multimodal fusion analysis allows for a more comprehensive understanding of the association among different types of brain tissues and its crucial role in the neuropathological mechanism of SCD and MCI.

A neurodevelopmental signature of parkinsonism in schizophrenia

While sensorimotor abnormalities in schizophrenia (SZ) are of increasing scientific interest, little is known about structural changes and their developmental origins that may underlie parkinsonism. This multimodal magnetic resonance imaging (MRI) study examined healthy controls (HC, n = 20) and SZ patients with (SZ-P, n = 38) and without (SZ-nonP, n = 35) parkinsonism, as defined by Simpson-Angus Scale total scores of ≥4 or ≤1, respectively. Using the Computational Anatomy Toolbox (CAT12), voxel- and surface-based morphometry were applied to investigate cortical and subcortical gray matter volume (GMV) and three cortical surface markers of distinct neurodevelopmental origin: cortical thickness (CTh), complexity of cortical folding (CCF) and sulcus depth. In a subgroup of patients (29 SZ-nonP, 25 SZ-P), resting-state fMRI data were also analyzed using a regions-of-interest approach based on fractional amplitude of low frequency fluctuations (fALFF). SZ-P patients showed increased CCF in the left supplementary motor cortex (SMC) and decreased left postcentral sulcus (PCS) depth compared to SZ-nonP patients (p < 0.05, FWE-corrected at cluster level). In SMC, CCF was associated negatively with activity, which also differed significantly between the patient groups and between patients and HC. In regression models, severity of parkinsonism was associated negatively with left middle frontal CCF and left anterior cingulate CTh. These data provide novel insights into altered trajectories of cortical development in SZ patients with parkinsonism. These cortical surface changes involve the sensorimotor system, suggesting abnormal neurodevelopmental processes tightly coupled with cortical activity and subcortical morphology that convey increased risk for sensorimotor abnormalities in SZ.

Multi-modal neuroimaging feature fusion via 3D Convolutional Neural Network architecture for schizophrenia diagnosis

Early and precise diagnosis of schizophrenia disorder (SZ) has an essential role in the quality of a patient’s life and future treatments. Structural and functional neuroimaging provides robust biomarkers for understanding the anatomical and functional changes associated with SZ. Each of the neuroimaging techniques shows only a different perspective on the functional or structural of the brain, while multi-modal fusion can reveal latent connections in the brain. In this paper, we propose an approach for the fusion of structural and functional brain data with a deep learning-based model to take advantage of data fusion and increase the accuracy of schizophrenia disorder diagnosis. The proposed method consists of an architecture of 3D convolutional neural networks (CNNs) that applied to magnetic resonance imaging (MRI), functional magnetic resonance imaging (fMRI), and diffusion tensor imaging (DTI) extracted features. We use 3D MRI patches, fMRI spatial independent component analysis (ICA) map, and DTI fractional anisotropy (FA) as model inputs. Our method is validated on the COBRE dataset, and an average accuracy of 99.35% is obtained. The proposed method demonstrates promising classification performance and can be applied to real data.

A Multimodal Fusion Analysis of Pretreatment Anatomical and Functional Cortical Abnormalities in Responsive and Non-responsive Schizophrenia

Background: Antipsychotic medications provide limited long-term benefit to ~30% of schizophrenia patients. Multimodal magnetic resonance imaging (MRI) data have been used to investigate brain features between responders and nonresponders to antipsychotic treatment; however, these analytical techniques are unable to weigh the interrelationships between modalities. Here, we used multiset canonical correlation and joint independent component analysis (mCCA + jICA) to fuse MRI data to examine the shared and specific multimodal features between the patients and healthy controls (HCs) and between the responders and non-responders. Method: Resting-state functional and structural MRI data were collected from 55 patients with drug-naïve first-episode schizophrenia (FES) and demographically matched HCs. Based on the decrease in Positive and Negative Syndrome Scale scores from baseline to the 1-year follow-up, FES patients were divided into a responder group (RG) and a non-responder group (NRG). Gray matter volume (GMV), fractional amplitude of low-frequency fluctuation (fALFF), and regional homogeneity (ReHo) maps were used as features in mCCA + jICA. Results: Between FES patients and HCs, there were three modality-specific discriminative independent components (ICs) showing the difference in mixing coefficients (GMV-IC7, GMV-IC8, and fALFF-IC5). The fusion analysis indicated one modality-shared IC (GMV-IC2 and ReHo-IC2) and three modality-specific ICs (GMV-IC1, GMV-IC3, and GMV-IC6) between the RG and NRG. The right postcentral gyrus showed a significant difference in GMV features between FES patients and HCs and modality-shared features (GMV and ReHo) between responders and nonresponders. The modality-shared component findings were highlighted by GMV, mainly in the bilateral temporal gyrus and the right cerebellum associated with ReHo in the right postcentral gyrus. Conclusions: This study suggests that joint anatomical and functional features of the cortices may reflect an early pathophysiological mechanism that is related to a 1-year treatment response.

Multimodal MRI assessment for first episode psychosis: A major change in the thalamus and an efficient stratification of a subgroup

Multi‐institutional brain imaging studies have emerged to resolve conflicting results among individual studies. However, adjusting multiple variables at the technical and cohort levels is challenging. Therefore, it is important to explore approaches that provide meaningful results from relatively small samples at institutional levels. We studied 87 first episode psychosis (FEP) patients and 62 healthy subjects by combining supervised integrated factor analysis (SIFA) with a novel pipeline for automated structure‐based analysis, an efficient and comprehensive method for dimensional data reduction that our group recently established. We integrated multiple MRI features (volume, DTI indices, resting state fMRI—rsfMRI) in the whole brain of each participant in an unbiased manner. The automated structure‐based analysis showed widespread DTI abnormalities in FEP and rs‐fMRI differences between FEP and healthy subjects mostly centered in thalamus. The combination of multiple modalities with SIFA was more efficient than the use of single modalities to stratify a subgroup of FEP (individuals with schizophrenia or schizoaffective disorder) that had more robust deficits from the overall FEP group. The information from multiple MRI modalities and analytical methods highlighted the thalamus as significantly abnormal in FEP. This study serves as a proof‐of‐concept for the potential of this methodology to reveal disease underpins and to stratify populations into more homogeneous sub‐groups.

Multimodal fusion analysis of structural connectivity and gray matter morphology in migraine

No specific migraine biomarkers have been found in single‐modality MRI studies. We aimed at establishing biomarkers for episodic and chronic migraine using diverse MRI modalities. We employed canonical correlation analysis and joint independent component analysis to find structural connectivity abnormalities that are related to gray matter morphometric alterations. The number of streamlines (trajectories of estimated fiber‐tracts from tractography) was employed as structural connectivity measure, while cortical curvature, thickness, surface area, and volume were used as gray matter parameters. These parameters were compared between 56 chronic and 54 episodic migraine patients, and 50 healthy controls. Cortical curvature alterations were associated with abnormalities in the streamline count in episodic migraine patients compared to controls, with higher curvature values in the frontal and temporal poles being related to a higher streamline count. Lower streamline count was found in migraine compared to controls in connections between cortical regions within each of the four lobes. Higher streamline count was found in migraine in connections between subcortical regions, the insula, and the cingulate and orbitofrontal cortex, and between the insula and the temporal region. The connections between the caudate nucleus and the orbitofrontal cortex presented worse connectivity in chronic compared to episodic migraine. The hippocampus was involved in connections with higher and lower number of streamlines in chronic migraine. Strengthening of structural networks involving pain processing and subcortical regions coexists in migraine with weakening of cortical networks within each lobe. The multimodal analysis offers a new insight about the association between brain structure and connectivity.

A technical review of canonical correlation analysis for neuroscience applications

Collecting comprehensive data sets of the same subject has become a standard in neuroscience research and uncovering multivariate relationships among collected data sets have gained significant attentions in recent years. Canonical correlation analysis (CCA) is one of the powerful multivariate tools to jointly investigate relationships among multiple data sets, which can uncover disease or environmental effects in various modalities simultaneously and characterize changes during development, aging, and disease progressions comprehensively. In the past 10 years, despite an increasing number of studies have utilized CCA in multivariate analysis, simple conventional CCA dominates these applications. Multiple CCA-variant techniques have been proposed to improve the model performance; however, the complicated multivariate formulations and not well-known capabilities have delayed their wide applications. Therefore, in this study, a comprehensive review of CCA and its variant techniques is provided. Detailed technical formulation with analytical and numerical solutions, current applications in neuroscience research, and advantages and limitations of each CCA-related technique are discussed. Finally, a general guideline in how to select the most appropriate CCA-related technique based on the properties of available data sets and particularly targeted neuroscience questions is provided.

Finding the needle in a high-dimensional haystack: Canonical correlation analysis for neuroscientists

The 21st century marks the emergence of "big data" with a rapid increase in the availability of datasets with multiple measurements. In neuroscience, brain-imaging datasets are more commonly accompanied by dozens or hundreds of phenotypic subject descriptors on the behavioral, neural, and genomic level. The complexity of such "big data" repositories offer new opportunities and pose new challenges for systems neuroscience. Canonical correlation analysis (CCA) is a prototypical family of methods that is useful in identifying the links between variable sets from different modalities. Importantly, CCA is well suited to describing relationships across multiple sets of data, such as in recently available big biomedical datasets. Our primer discusses the rationale, promises, and pitfalls of CCA.

Discriminating VCID subgroups: A diffusion MRI multi-model fusion approach

BACKGROUND

Vascular cognitive impairment and dementia (VCID) and Alzheimer's disease are predominant diseases among the aging population resulting in decline of various cognitive domains. Diffusion weighted MRI (DW-MRI) has been shown to be a promising aid in the diagnosis of such diseases. However, there are various models of DW-MRI and the interpretation of diffusion metrics depends on the model used in fitting data. Most previous studies are entirely based on parameters calculated from a single diffusion model.

NEW METHOD

We employ a data fusion framework wherein diffusion metrics from different models such as diffusion tensor imaging, diffusion kurtosis imaging and constrained spherical deconvolution model are fused using well known blind source separation approach to investigate white matter microstructural changes in population comprising of controls and VCID subgroups. Multiple comparisons between subject groups and prediction analysis using features from individual models and proposed fusion model are carried out to evaluate performance of proposed method.

RESULTS

Diffusion features from individual models successfully distinguished between controls and disease groups, but failed to differentiate between disease groups, whereas fusion approach showed group differences between disease groups too. WM tracts showing significant differences are superior longitudinal fasciculus, anterior thalamic radiation, arcuate fasciculus, optic radiation and corticospinal tract.

COMPARISON WITH EXISTING METHOD

ROC analysis showed increased AUC for fusion (AUC = 0.913, averaged across groups and tracts) compared to that of uni-model features (AUC = 0.77) demonstrating increased sensitivity of proposed method.

CONCLUSION

Overall our results highlight the benefits of multi-model fusion approach, providing improved sensitivity in discriminating VCID subgroups.

Multimodal Magnetic Resonance Imaging Data Fusion Reveals Distinct Patterns of Abnormal Brain Structure and Function in Catatonia

Catatonia is a nosologically unspecific syndrome, which subsumes a plethora of mostly complex affective, motor, and behavioral phenomena. Although catatonia frequently occurs in schizophrenia spectrum disorders (SSD), specific patterns of abnormal brain structure and function underlying catatonia are unclear at present. Here, we used a multivariate data fusion technique for multimodal magnetic resonance imaging (MRI) data to investigate patterns of aberrant intrinsic neural activity (INA) and gray matter volume (GMV) in SSD patients with and without catatonia. Resting-state functional MRI and structural MRI data were collected from 87 right-handed SSD patients. Catatonic symptoms were examined on the Northoff Catatonia Rating Scale (NCRS). A multivariate analysis approach was used to examine co-altered patterns of INA and GMV. Following a categorical approach, we found predominantly frontothalamic and corticostriatal abnormalities in SSD patients with catatonia (NCRS total score ≥ 3; n = 24) when compared to SSD patients without catatonia (NCRS total score = 0; n = 22) matched for age, gender, education, and medication. Corticostriatal network was associated with NCRS affective scores. Following a dimensional approach, 33 SSD patients with catatonia according to Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision were identified. NCRS behavioral scores were associated with a joint structural and functional system that predominantly included cerebellar and prefrontal/cortical motor regions. NCRS affective scores were associated with frontoparietal INA. This study provides novel neuromechanistic insights into catatonia in SSD suggesting co-altered structure/function-interactions in neural systems subserving coordinated visuospatial functions and motor behavior.

Patterns of co-altered brain structure and function underlying neurological soft signs in schizophrenia spectrum disorders

Neurological soft signs (NSS) comprise a broad range of subtle neurological deficits and are considered to represent external markers of sensorimotor dysfunction frequently found in mental disorders of presumed neurodevelopmental origin. Although NSS frequently occur in schizophrenia spectrum disorders (SSD), specific patterns of co-altered brain structure and function underlying NSS in SSD have not been investigated so far. It is unclear whether gray matter volume (GMV) alterations or aberrant brain activity or a combination of both, are associated with NSS in SSD. Here, 37 right-handed SSD patients and 37 matched healthy controls underwent motor assessment and magnetic resonance imaging (MRI) at 3 T. NSS were examined on the Heidelberg NSS scale. We used a multivariate data fusion technique for multimodal MRI data-multiset canonical correlation and joint independent component analysis (mCCA + jICA)-to investigate co-altered patterns of GMV and intrinsic neural fluctuations (INF) in SSD patients exhibiting NSS. The mCCA + jICA model indicated two joint group-discriminating components (temporoparietal/cortical sensorimotor and frontocerebellar/frontoparietal networks) and one modality-specific group-discriminating component (p < .05, FDR corrected). NSS motor score was associated with joint frontocerebellar/frontoparietal networks in SSD patients. This study highlights complex neural pathomechanisms underlying NSS in SSD suggesting aberrant structure and function, predominantly in cortical and cerebellar systems that critically subserve sensorimotor dynamics and psychomotor organization.

Constrained temporal parallel decomposition for EEG-fMRI fusion

OBJECTIVE

Multimodal neuroimaging has become a common practice in neuroscience research. Simultaneous EEG-fMRI is a popular multimodal recording approach due to the complementary spatiotemporal relationship between the two modalities. Several data fusion techniques have been proposed in the literature for EEG-fMRI fusion, including joint-ICA and parallel-ICA frameworks. Previous EEG-fMRI fusion approaches have used sensor-level EEG features. Recently, we introduced source-space ICA for EEG-MEG source reconstruction and component identification, which was shown to be a superior alternative to sensor-space ICA.

APPROACH

Here, we extend source-space ICA to the fusion of EEG-fMRI data. Additionally, we incorporate the use of a paradigm signal (constrained) and a lag-based signal decomposition approach to accommodate recent findings demonstrating the potentially variable lag structure between electrophysiological and BOLD signals. We evaluated this method on simulated concurrent EEG-fMRI during a boxcar task design, as well as real concurrent EEG-fMRI data from three participants performing an N-Back working memory task. The block diagram of the algorithm and corresponding source codes are provided.

MAIN RESULTS

Based on the results of the real working memory task, for all three subjects, one frontal theta component, and one right posterior alpha component had the highest contribution coefficients (~0.5) to the paradigm-related fused component. There were also two more alpha band components with contribution coefficients of 0.3. The highest contributing fMRI component (~0.8) was one known in the literature to be related to the attention network. The second fMRI component was related to the well-known default mode network, with a contribution coefficient of 0.3.

SIGNIFICANCE

The proposed EEG-fMRI fusion approach, is capable of estimating the brain maps of the EEG and fMRI for the fused components and account for the variable lag structure between electrophysiological and BOLD signals.

Four-way multimodal fusion of 7 T imaging data using an mCCA+jICA model in first-episode schizophrenia

Acquisition of multimodal brain imaging data for the same subject has become more common leading to a growing interest in determining the intermodal relationships between imaging modalities to further elucidate the pathophysiology of schizophrenia. Multimodal data have previously been individually analyzed and subsequently integrated; however, these analysis techniques lack the ability to examine true modality inter-relationships. The utilization of a multiset canonical correlation and joint independent component analysis (mCCA + jICA) model for data fusion allows shared or distinct abnormalities between modalities to be examined. In this study, first-episode schizophrenia patients (n_SZ =19) and matched controls (n_HC =21) completed a resting-state functional magnetic resonance imaging (fMRI) scan at 7 T. Grey matter (GM), white matter (WM), cerebrospinal fluid (CSF), and amplitude of low frequency fluctuation (ALFF) maps were used as features in a mCCA + jICA model. Results of the mCCA + jICA model indicated three joint group-discriminating components (GM-CSF, WM-ALFF, GM-ALFF) and two modality-unique group-discriminating components (GM, WM). The joint component findings are highlighted by GM basal ganglia, somatosensory, parietal lobe, and thalamus abnormalities associated with ventricular CSF volume; WM occipital and frontal lobe abnormalities associated with temporal lobe function; and GM frontal, temporal, parietal, and occipital lobe abnormalities associated with caudate function. These results support and extend major findings throughout the literature using independent single modality analyses. The multimodal fusion of 7 T data in this study provides a more comprehensive illustration of the relationships between underlying neuronal abnormalities associated with schizophrenia than examination of imaging data independently.