Multimodal functional and structural imaging investigations in psychosis research

A review of the applications of generative adversarial networks to structural and functional MRI based diagnostic classification of brain disorders

Structural and functional MRI (magnetic resonance imaging) based diagnostic classification using machine learning has long held promise, but there are many roadblocks to achieving their potential. While traditional machine learning models suffered from their inability to capture the complex non-linear mapping, deep learning models tend to overfit the model. This is because there is data scarcity and imbalanced classes in neuroimaging; it is expensive to acquire data from human subjects and even more so in clinical populations. Due to their ability to augment data by learning underlying distributions, generative adversarial networks (GAN) provide a potential solution to this problem. Here, we provide a methodological primer on GANs and review the applications of GANs to classification of mental health disorders from neuroimaging data such as functional MRI and showcase the progress made thus far. We also highlight gaps in methodology as well as interpretability that are yet to be addressed. This provides directions about how the field can move forward. We suggest that since there are a range of methodological choices available to users, it is critical for users to interact with method developers so that the latter can tailor their development according to the users' needs. The field can be enriched by such synthesis between method developers and users in neuroimaging.

Analysis of Longitudinal Change Patterns in Developing Brain Using Functional and Structural Magnetic Resonance Imaging via Multimodal Fusion

Functional and structural magnetic resonance imaging (fMRI and sMRI) are complementary approaches that can be used to study longitudinal brain changes in adolescents. Each individual modality offers distinct insights into the brain. Each individual modality may overlook crucial aspects of brain analysis. By combining them, we can uncover hidden brain connections and gain a more comprehensive understanding. In previous work, we identified multivariate patterns of change in whole-brain function during adolescence. In this work, we focus on linking functional change patterns (FCPs) to brain structure. We introduce two approaches and applied them to data from the Adolescent Brain and Cognitive Development (ABCD) dataset. First, we evaluate voxelwise sMRI-FCP coupling to identify structural patterns linked to our previously identified FCPs. Our approach revealed multiple interesting patterns in functional network connectivity (FNC) and gray matter volume (GMV) data that were linked to subject level variation. FCP components 2 and 4 exhibit extensive associations between their loadings and voxel-wise GMV data. Secondly, we leveraged a symmetric multimodal fusion technique called multiset canonical correlation analysis (mCCA) + joint independent component analysis (jICA). Using this approach, we identify structured FCPs such as one showing increased connectivity between visual and sensorimotor domains and decreased connectivity between sensorimotor and cognitive control domains, linked to structural change patterns (SCPs) including alterations in the bilateral sensorimotor cortex. Interestingly, females exhibit stronger coupling between brain functional and structural changes than males, highlighting sex-related differences. The combined results from both asymmetric and symmetric multimodal fusion methods underscore the intricate sex-specific nuances in neural dynamics. By utilizing two complementary multimodal approaches, our study enhances our understanding of the dynamic nature of brain connectivity and structure during the adolescent period, shedding light on the nuanced processes underlying adolescent brain development.

A meta-analysis and systematic review of single vs. multimodal neuroimaging techniques in the classification of psychosis

BACKGROUND

Psychotic disorders are characterized by structural and functional abnormalities in brain networks. Neuroimaging techniques map and characterize such abnormalities using unique features (e.g., structural integrity, coactivation). However, it is unclear if a specific method, or a combination of modalities, is particularly effective in identifying differences in brain networks of someone with a psychotic disorder.

METHODS

A systematic meta-analysis evaluated machine learning classification of schizophrenia spectrum disorders in comparison to healthy control participants using various neuroimaging modalities (i.e., T1-weighted imaging (T1), diffusion tensor imaging (DTI), resting state functional connectivity (rs-FC), or some combination (multimodal)). Criteria for manuscript inclusion included whole-brain analyses and cross-validation to provide a complete picture regarding the predictive ability of large-scale brain systems in psychosis. For this meta-analysis, we searched Ovid MEDLINE, PubMed, PsychInfo, Google Scholar, and Web of Science published between inception and March 13th 2023. Prediction results were averaged for studies using the same dataset, but parallel analyses were run that included studies with pooled sample across many datasets. We assessed bias through funnel plot asymmetry. A bivariate regression model determined whether differences in imaging modality, demographics, and preprocessing methods moderated classification. Separate models were run for studies with internal prediction (via cross-validation) and external prediction.

RESULTS

93 studies were identified for quantitative review (30 T1, 9 DTI, 40 rs-FC, and 14 multimodal). As a whole, all modalities reliably differentiated those with schizophrenia spectrum disorders from controls (OR = 2.64 (95%CI = 2.33 to 2.95)). However, classification was relatively similar across modalities: no differences were seen across modalities in the classification of independent internal data, and a small advantage was seen for rs-FC studies relative to T1 studies in classification in external datasets. We found large amounts of heterogeneity across results resulting in significant signs of bias in funnel plots and Egger's tests. Results remained similar, however, when studies were restricted to those with less heterogeneity, with continued small advantages for rs-FC relative to structural measures. Notably, in all cases, no significant differences were seen between multimodal and unimodal approaches, with rs-FC and unimodal studies reporting largely overlapping classification performance. Differences in demographics and analysis or denoising were not associated with changes in classification scores.

CONCLUSIONS

The results of this study suggest that neuroimaging approaches have promise in the classification of psychosis. Interestingly, at present most modalities perform similarly in the classification of psychosis, with slight advantages for rs-FC relative to structural modalities in some specific cases. Notably, results differed substantially across studies, with suggestions of biased effect sizes, particularly highlighting the need for more studies using external prediction and large sample sizes. Adopting more rigorous and systematized standards will add significant value toward understanding and treating this critical population.

The substantia nigra in the pathology of schizophrenia: A review on post-mortem and molecular imaging findings

Dysregulation of striatal dopamine is considered to be an important driver of pathophysiological processes in schizophrenia. Despite being one of the main origins of dopaminergic input to the striatum, the (dys)functioning of the substantia nigra (SN) has been relatively understudied in schizophrenia. Hence, this paper aims to review different molecular aspects of nigral functioning in patients with schizophrenia compared to healthy controls by integrating post-mortem and molecular imaging studies. We found evidence for hyperdopaminergic functioning in the SN of patients with schizophrenia (i.e. increased AADC activity in antipsychotic-free/-naïve patients and elevated neuromelanin accumulation). Reduced GABAergic inhibition (i.e. decreased density of GABAergic synapses, lower VGAT mRNA levels and lower mRNA levels for GABA_A receptor subunits), excessive glutamatergic excitation (i.e. increased NR1 and Glur5 mRNA levels and a reduced number of astrocytes), and several other disturbances implicating the SN (i.e. immune functioning and copper concentrations) could potentially underlie this nigral hyperactivity and associated striatal hyperdopaminergic functioning in schizophrenia. These results highlight the importance of the SN in schizophrenia pathology and suggest that some aspects of molecular functioning in the SN could potentially be used as treatment targets or biomarkers.

Alpha and gamma EEG coherence during on-task and mind wandering states in schizophrenia

OBJECTIVE

Electroencephalographic (EEG) coherence is one of the most relevant physiological measures used to detect abnormalities in patients with schizophrenia. The present study applies a task-related EEG coherence approach to understand cognitive processing in patients with schizophrenia and healthy controls.

METHODS

EEG coherence for alpha and gamma frequency bands was analyzed in a group of patients with schizophrenia and a group of healthy controls during the performance of an ecological task of sustained attention. We compared EEG coherence when participants presented externally directed cognitive states (On-Task) and when they presented cognitive distraction episodes (Mind-Wandering).

RESULTS

Results reflect cortical differences between groups (higher coherence for schizophrenia in the frontocentral and fronto-temporal regions, and higher coherence for healthy-controls in the postero-central regions), especially in the On-Task condition for the alpha band, compared to Mind-Wandering episodes. Few individual differences in gamma coherence were found.

CONCLUSIONS

The current study provides evidence of neurophysiological differences underlying different cognitive states in schizophrenia and healthy controls.

SIGNIFICANCE

Differences between groups may reflect inhibitory processes necessary for the successful processing of information, especially in the alpha band, given its role in cortical inhibition processes. Patients may activate compensatory inhibitory mechanisms when performing the task, reflected in increased coherence in fronto-temporal regions.

Multimodal meta-analysis of structural gray matter, neurocognitive and social cognitive fMRI findings in schizophrenia patients

Neuroimaging research has shown that patients with schizophrenia (SCZ) present brain structural and functional alterations, but the results across imaging modalities and task paradigms are difficult to reconcile. Specifically, no meta-analyses have tested whether the same brain systems that are structurally different in SCZ patients are also involved in neurocognitive and social cognitive tasks. To answer this, we conducted separate meta-analyses of voxel-based morphometry, neurocognitive functional magnetic resonance imaging (fMRI), and social cognitive fMRI studies. Next, with a multimodal approach, we identified the common alterations across meta-analyses. Further exploratory meta-analyses were performed taking into account several clinical variables (illness duration, medication status, and symptom severity). A cluster covering the dorsomedial prefrontal cortex (dmPFC) and the supplementary motor area (SMA), and the right inferior frontal gyrus (IFG), presented shared structural and neurocognitive-related activation decreases, while the right angular gyrus presented shared decreases between structural and social cognitive-related activation. The exploratory meta-analyses replicated to some extent these findings, while new regions of alterations appeared in patient subgroups with specific clinical features. In conclusion, we found task-specific correlates of brain structure and function in SCZ, which help summarize and integrate a growing literature.

A Multi-sequence MRI Study in Parkinson's Disease: Association Between Rigidity and Myelin

BACKGROUND

The pathophysiology of rigidity in Parkinson's disease (PD) is poorly understood. Multi-sequence functional and structural brain MRI may further clarify the origin of this clinical characteristic.

PURPOSE

To examine both joint and unique relationships of MRI-based functional and structural imaging modalities to rigidity and other clinical features of PD.

STUDY TYPE

Retrospective cross-sectional study.

POPULATION

31 PD subjects (aged 68.0 ± 5.9 years, 21 males) with average disease duration 9.3 ± 5.4 years.

FIELD STRENGTH/SEQUENCE

Multi-echo GRASE, diffusion-weighted echo planar imaging (EPI), and blood oxygen level dependent contrast EPI T2*-weighted sequences on a 3T scanner.

ASSESSMENT

Myelin water fraction (MWF) and fractional anisotropy (FA) of 20 white-matter regions of interest (ROIs), and functional connectivity derived from resting-state fMRI among 56 ROIs were assessed. The Unified Parkinson's Disease Rating Scale-Part III, Montreal Cognitive Assessment, Beck Depression Index, and Apathy Rating Scales were used to assess motor and non-motor symptoms.

STATISTICAL TESTS

Multiset canonical correlation analysis (MCCA) and canonical correlation analysis (CCA) were utilized to examine the joint and unique relationships of multiple imaging measures with clinical symptoms of PD. A permutation test was used to determine statistical significance (P < 0.05).

RESULTS

MCCA revealed a single significant component jointly linking MWF, FA, and functional connectivity to age, bradykinesia, and leg agility, non-motor symptoms of cognition, depression, and apathy, but not rigidity (P = 0.77), tremor (P = 0.50 and 0.67 on the left and right side), or sex (P = 0.54). After controlling for this joint component, CCA found a unique significant association between MWF and rigidity, but no other associations were detected, including with FA (P = 0.87).

DATA CONCLUSION

MWF, FA, and functional connectivity can serve as multi-sequence imaging markers to characterize many PD symptoms. However, rigidity in PD is additionally associated with widespread myelin changes.

EVIDENCE LEVEL

4 TECHNICAL EFFICACY: Stage 3.

Multimodal MRI assessment for first episode psychosis: A major change in the thalamus and an efficient stratification of a subgroup

Multi‐institutional brain imaging studies have emerged to resolve conflicting results among individual studies. However, adjusting multiple variables at the technical and cohort levels is challenging. Therefore, it is important to explore approaches that provide meaningful results from relatively small samples at institutional levels. We studied 87 first episode psychosis (FEP) patients and 62 healthy subjects by combining supervised integrated factor analysis (SIFA) with a novel pipeline for automated structure‐based analysis, an efficient and comprehensive method for dimensional data reduction that our group recently established. We integrated multiple MRI features (volume, DTI indices, resting state fMRI—rsfMRI) in the whole brain of each participant in an unbiased manner. The automated structure‐based analysis showed widespread DTI abnormalities in FEP and rs‐fMRI differences between FEP and healthy subjects mostly centered in thalamus. The combination of multiple modalities with SIFA was more efficient than the use of single modalities to stratify a subgroup of FEP (individuals with schizophrenia or schizoaffective disorder) that had more robust deficits from the overall FEP group. The information from multiple MRI modalities and analytical methods highlighted the thalamus as significantly abnormal in FEP. This study serves as a proof‐of‐concept for the potential of this methodology to reveal disease underpins and to stratify populations into more homogeneous sub‐groups.

Hallucination-Specific structure-function associations in schizophrenia

Combining structural (sMRI) and functional magnetic resonance imaging (fMRI) data in schizophrenia patients with and without auditory hallucinations (9 SZ_AVH, 12 SZ_nAVH), 18 patients with bipolar disorder, and 22 healthy controls, we examined whether cortical thinning was associated with abnormal activity in functional brain networks associated with auditory hallucinations. Language-task fMRI data were combined with mean cortical thickness values from 148 brain regions in a constrained principal component analysis (CPCA) to identify brain structure-function associations predictable from group differences. Two components emerged from the multimodal analysis. The "AVH component" highlighted an association of frontotemporal and cingulate thinning with altered brain activity characteristic of hallucinations among patients with AVH. In contrast, the "Bipolar component" distinguished bipolar patients from healthy controls and linked increased activity in the language network with cortical thinning in the left occipital-temporal lobe. Our findings add to a body of evidence of the biological underpinnings of hallucinations and illustrate a method for multimodal data analysis of structure-function associations in psychiatric illness.

Structural and functional imaging markers for susceptibility to psychosis

The introduction of clinical criteria for the operationalization of psychosis high risk provided a basis for early detection and treatment of vulnerable individuals. However, about two-thirds of people meeting clinical high-risk (CHR) criteria will never develop a psychotic disorder. In the effort to increase prognostic precision, structural and functional neuroimaging have received growing attention as a potentially useful resource in the prediction of psychotic transition in CHR patients. The present review summarizes current research on neuroimaging biomarkers in the CHR state, with a particular focus on their prognostic utility and limitations. Large, multimodal/multicenter studies are warranted to address issues important for clinical applicability such as generalizability and replicability, standardization of clinical definitions and neuroimaging methods, and consideration of contextual factors (e.g., age, comorbidity).

Predicting response to electroconvulsive therapy combined with antipsychotics in schizophrenia using multi-parametric magnetic resonance imaging

Electroconvulsive therapy (ECT) has been shown to be effective in schizophrenia, particularly when rapid symptom reduction is needed or in cases of resistance to drug treatment. However, there are no markers available to predict response to ECT. Here, we examine whether multi-parametric magnetic resonance imaging (MRI)-based radiomic features can predict response to ECT for individual patients. A total of 57 treatment-resistant schizophrenia patients, or schizophrenia patients with an acute episode or suicide attempts were randomly divided into primary (42 patients) and test (15 patients) cohorts. We collected T1-weighted structural MRI and diffusion MRI for 57 patients before receiving ECT and extracted 600 radiomic features for feature selection and prediction. To predict a continuous improvement in symptoms (ΔPANSS), the prediction process was performed with a support vector regression model based on a leave-one-out cross-validation framework in primary cohort and was tested in test cohort. The multi-parametric MRI-based radiomic model, including four structural MRI feature from left inferior frontal gyrus, right insula, left middle temporal gyrus and right superior temporal gyrus respectively and six diffusion MRI features from tracts connecting frontal or temporal gyrus possessed a low root mean square error of 15.183 in primary cohort and 14.980 in test cohort. The Pearson's correlation coefficients between predicted and actual values were 0.671 and 0.777 respectively. These results demonstrate that multi-parametric MRI-based radiomic features may predict response to ECT for individual patients. Such features could serve as prognostic neuroimaging biomarkers that provide a critical step toward individualized treatment response prediction in schizophrenia.

Cerebral glutamate and GABA levels in high-risk of psychosis states: A focused review and meta-analysis of 1H-MRS studies

Disturbances in the brain glutamate and GABA (γ-aminobutyric acid) homeostasis may be markers of transition to psychosis in individuals at high-risk (HR). Knowledge of GABA and glutamate levels in HR stages could give an insight into changes in the neurochemistry underlying psychosis. Studies on glutamate in HR have provided conflicting data, and GABA studies have only recently been initialized. In this meta-analysis, we compared cerebral levels of glutamate and GABA in HR individuals with healthy controls (HC). We searched Medline and Embase for articles published on ¹H-MRS studies on glutamate and GABA in HR states until April 9th, 2019. We identified a total of 28 eligible studies, of which eight reported GABA (243 HR, 356 HC) and 26 reported glutamate (299 HR, 279 HC) or Glx (glutamate + glutamine) (584 HR, 632 HC) levels. Sample sizes varied from 6 to 75 for HR and 10 to 184 for HC. Our meta-analysis of ¹H-MRS studies on glutamate and GABA in HR states displayed significantly lower (P = 0.0003) levels of thalamic glutamate in HR individuals than in HC and significantly higher (P = 0.001) Glx in the frontal lobe of genetic HR individuals (1st-degree relatives) than in HC. No other significant differences in glutamate and GABA levels were found. Subject numbers in the studies on glutamate as well as GABA levels were generally small and the data conflicting. Our meta-analytical findings highlight the need for larger and more homogeneous studies of glutamate and GABA in high-risk states.

Multimodal Neuroimaging: Basic Concepts and Classification of Neuropsychiatric Diseases

Neuroimaging techniques are widely used in neuroscience to visualize neural activity, to improve our understanding of brain mechanisms, and to identify biomarkers-especially for psychiatric diseases; however, each neuroimaging technique has several limitations. These limitations led to the development of multimodal neuroimaging (MN), which combines data obtained from multiple neuroimaging techniques, such as electroencephalography, functional magnetic resonance imaging, and yields more detailed information about brain dynamics. There are several types of MN, including visual inspection, data integration, and data fusion. This literature review aimed to provide a brief summary and basic information about MN techniques (data fusion approaches in particular) and classification approaches. Data fusion approaches are generally categorized as asymmetric and symmetric. The present review focused exclusively on studies based on symmetric data fusion methods (data-driven methods), such as independent component analysis and principal component analysis. Machine learning techniques have recently been introduced for use in identifying diseases and biomarkers of disease. The machine learning technique most widely used by neuroscientists is classification-especially support vector machine classification. Several studies differentiated patients with psychiatric diseases and healthy controls with using combined datasets. The common conclusion among these studies is that the prediction of diseases increases when combining data via MN techniques; however, there remain a few challenges associated with MN, such as sample size. Perhaps in the future N-way fusion can be used to combine multiple neuroimaging techniques or nonimaging predictors (eg, cognitive ability) to overcome the limitations of MN.

Longitudinal brain functional and structural connectivity changes after hemispherotomy in two pediatric patients with drug-resistant epilepsy

The main focus of the present study was to explore the longitudinal changes in the brain executive control system and default mode network after hemispherotomy. Resting-state functional magnetic resonance imaging and diffusion tensor imaging were collected in two children with drug-resistnt epilepsy underwent hemispherotomy. Two patients with different curative effects showed different trajectories of brain connectivity after surgery. The failed hemispherotomy might be due to the fact that the synchrony of epileptic neurons in both hemispheres is preserved by residual neural pathways. Loss of interhemispheric correlations with increased intrahemispheric correlations can be considered as neural marker for evaluating the success of hemispherotomy.

Reading the (functional) writing on the (structural) wall: Multimodal fusion of brain structure and function via a deep neural network based translation approach reveals novel impairments in schizophrenia

This work presents a novel approach to finding linkage/association between multimodal brain imaging data, such as structural MRI (sMRI) and functional MRI (fMRI). Motivated by the machine translation domain, we employ a deep learning model, and consider two different imaging views of the same brain like two different languages conveying some common facts. That analogy enables finding linkages between two modalities. The proposed translation-based fusion model contains a computing layer that learns "alignments" (or links) between dynamic connectivity features from fMRI data and static gray matter patterns from sMRI data. The approach is evaluated on a multi-site dataset consisting of eyes-closed resting state imaging data collected from 298 subjects (age- and gender matched 154 healthy controls and 144 patients with schizophrenia). Results are further confirmed on an independent dataset consisting of eyes-open resting state imaging data from 189 subjects (age- and gender matched 91 healthy controls and 98 patients with schizophrenia). We used dynamic functional connectivity (dFNC) states as the functional features and ICA-based sources from gray matter densities as the structural features. The dFNC states characterized by weakly correlated intrinsic connectivity networks (ICNs) were found to have stronger association with putamen and insular gray matter pattern, while the dFNC states of profuse strongly correlated ICNs exhibited stronger links with the gray matter pattern in precuneus, posterior cingulate cortex (PCC), and temporal cortex. Further investigation with the estimated link strength (or alignment score) showed significant group differences between healthy controls and patients with schizophrenia in several key regions including temporal lobe, and linked these to connectivity states showing less occupancy in healthy controls. Moreover, this novel approach revealed significant correlation between a cognitive score (attention/vigilance) and the function/structure alignment score that was not detected when data modalities were considered separately.

Neural Indicators of Fatigue in Chronic Diseases: A Systematic Review of MRI Studies

While fatigue is prevalent in chronic diseases, the neural mechanisms underlying this symptom remain unknown. Magnetic resonance imaging (MRI) has the potential to enable us to characterize this symptom. The aim of this review was to gather and appraise the current literature on MRI studies of fatigue in chronic diseases. We systematically searched the following databases: MedLine, PsycInfo, Embase and Scopus (inception to April 2016). We selected studies according to a predefined inclusion and exclusion criteria. We assessed the quality of the studies and conducted descriptive statistical analyses. We identified 26 studies of varying design and quality. Structural and functional MRI, alongside diffusion tensor imaging (DTI) and functional connectivity (FC) studies, identified significant brain indicators of fatigue. The most common regions were the frontal lobe, parietal lobe, limbic system and basal ganglia. Longitudinal studies offered more precise and reliable analysis. Brain structures found to be related to fatigue were highly heterogeneous, not only between diseases, but also for different studies of the same disease. Given the different designs, methodologies and variable results, we conclude that there are currently no well-defined brain indicators of fatigue in chronic diseases.

Structural and functional alterations in the brain during working memory in medication-naïve patients at clinical high-risk for psychosis

Several previous studies suggest that clinical high risk for psychosis (CHR) is associated with prefrontal functional abnormalities and more widespread reduced grey matter in prefrontal, temporal and parietal areas. We investigated neural correlates to CHR in medication-naïve patients. 41 CHR patients and 37 healthy controls were examined with 1.5 Tesla MRI, yielding functional scans while performing an N-back task and structural T1-weighted brain images. Functional and structural data underwent automated preprocessing steps in SPM and Freesurfer, correspondingly. The groups were compared employing mass-univariate strategy within the generalized linear modelling framework. CHR demonstrated reduced suppression of the medial temporal lobe (MTL) regions during n-back task. We also found that, consistent with previous findings, CHR subjects demonstrated thinning in prefrontal, cingulate, insular and inferior temporal areas, as well as reduced hippocampal volumes. The present findings add to the growing evidence of specific structural and functional abnormalities in the brain as potential neuroimaging markers of psychosis vulnerability.

Schizophrenia Shows Disrupted Links between Brain Volume and Dynamic Functional Connectivity

Studies featuring multimodal neuroimaging data fusion for understanding brain function and structure, or disease characterization, leverage the partial information available in each of the modalities to reveal data variations not exhibited through the independent analyses. Similar to other complex syndromes, the characteristic brain abnormalities in schizophrenia may be better understood with the help of the additional information conveyed by leveraging an advanced modeling method involving multiple modalities. In this study, we propose a novel framework to fuse feature spaces corresponding to functional magnetic resonance imaging (functional) and gray matter (structural) data from 151 schizophrenia patients and 163 healthy controls. In particular, the features for the functional and structural modalities include dynamic (i.e., time-varying) functional network connectivity (dFNC) maps and the intensities of the gray matter (GM) maps, respectively. The dFNC maps are estimated from group independent component analysis (ICA) network time-courses by first computing windowed functional correlations using a sliding window approach, and then estimating subject specific states from this windowed data using temporal ICA followed by spatio-temporal regression. For each subject, the functional data features are horizontally concatenated with the corresponding GM features to form a combined feature space that is subsequently decomposed through a symmetric multimodal fusion approach involving a combination of multiset canonical correlation analysis (mCCA) and joint ICA (jICA). Our novel combined analyses successfully linked changes in the two modalities and revealed significantly disrupted links between GM volumes and time-varying functional connectivity in schizophrenia. Consistent with prior research, we found significant group differences in GM comprising regions in the superior parietal lobule, precuneus, postcentral gyrus, medial/superior frontal gyrus, superior/middle temporal gyrus, insula and fusiform gyrus, and several significant aberrations in the inter-regional functional connectivity strength as well. Importantly, structural and dFNC measures have independently shown changes associated with schizophrenia, and in this work we begin the process of evaluating the links between the two, which could shed light on the illness beyond what we can learn from a single imaging modality. In future work, we plan to evaluate replication of the inferred structure-function relationships in independent partitions of larger multi-modal schizophrenia datasets.

Using deep learning to investigate the neuroimaging correlates of psychiatric and neurological disorders: Methods and applications

Deep learning (DL) is a family of machine learning methods that has gained considerable attention in the scientific community, breaking benchmark records in areas such as speech and visual recognition. DL differs from conventional machine learning methods by virtue of its ability to learn the optimal representation from the raw data through consecutive nonlinear transformations, achieving increasingly higher levels of abstraction and complexity. Given its ability to detect abstract and complex patterns, DL has been applied in neuroimaging studies of psychiatric and neurological disorders, which are characterised by subtle and diffuse alterations. Here we introduce the underlying concepts of DL and review studies that have used this approach to classify brain-based disorders. The results of these studies indicate that DL could be a powerful tool in the current search for biomarkers of psychiatric and neurologic disease. We conclude our review by discussing the main promises and challenges of using DL to elucidate brain-based disorders, as well as possible directions for future research.

FDG-PET scans in patients with Kraepelinian and non-Kraepelinian schizophrenia

We recruited 14 unmedicated patients with Kraepelinian schizophrenia (12 men and 2 women; mean age = 47 years old), 27 non-Kraepelinian patients (21 men and 6 women; mean age = 36.4 years old) and a group of 56 age- and sex-matched healthy volunteers. FDG positron emission tomography and MRI scans were coregistered for both voxel-by-voxel statistical mapping and stereotaxic regions of interest analysis. While both Kraepelinian and non-Kraepelinian patients showed equally lower uptake than healthy volunteers in the frontal lobe, the temporal lobes (Brodmann areas 20 and 21) showed significantly greater decreases in Kraepelinian than in non-Kraepelinian patients. Kraepelinian patients had lower FDG uptake in parietal regions 39 and 40, especially in the right hemisphere, while non-Kraepelinian patients had similar reductions in the left. Only non-Kraepelinian patients had lower caudate FDG uptake than healthy volunteers. While both patient groups had lower uptake than healthy volunteers in the medial dorsal nucleus of the thalamus, Kraepelinian patients alone had higher uptake in the ventral nuclei of the thalamus. Kraepelinian patients also showed higher metabolic rates in white matter. Our results are consistent with other studies indicating that Kraepelinian schizophrenia is a subgroup of schizophrenia, characterized by temporal and right parietal deficits and normal rather than reduced caudate uptake. It suggests that Kraepelinian schizophrenia may be more primarily characterized by FDG uptake decreased in both the frontal and temporal lobes, while non-Kraepelinian schizophrenia may have deficits more limited to the frontal lobe. This is consistent with some neuropsychological and prognosis reports of disordered sensory information processing in Kraepelinian schizophrenia in addition to deficits in frontal lobe executive functions shared with the non-Kraepelinian subtype.

Predicting individualized clinical measures by a generalized prediction framework and multimodal fusion of MRI data

Neuroimaging techniques have greatly enhanced the understanding of neurodiversity (human brain variation across individuals) in both health and disease. The ultimate goal of using brain imaging biomarkers is to perform individualized predictions. Here we proposed a generalized framework that can predict explicit values of the targeted measures by taking advantage of joint information from multiple modalities. This framework also enables whole brain voxel-wise searching by combining multivariate techniques such as ReliefF, clustering, correlation-based feature selection and multiple regression models, which is more flexible and can achieve better prediction performance than alternative atlas-based methods. For 50 healthy controls and 47 schizophrenia patients, three kinds of features derived from resting-state fMRI (fALFF), sMRI (gray matter) and DTI (fractional anisotropy) were extracted and fed into a regression model, achieving high prediction for both cognitive scores (MCCB composite r=0.7033, MCCB social cognition r=0.7084) and symptomatic scores (positive and negative syndrome scale [PANSS] positive r=0.7785, PANSS negative r=0.7804). Moreover, the brain areas likely responsible for cognitive deficits of schizophrenia, including middle temporal gyrus, dorsolateral prefrontal cortex, striatum, cuneus and cerebellum, were located with different weights, as well as regions predicting PANSS symptoms, including thalamus, striatum and inferior parietal lobule, pinpointing the potential neuromarkers. Finally, compared to a single modality, multimodal combination achieves higher prediction accuracy and enables individualized prediction on multiple clinical measures. There is more work to be done, but the current results highlight the potential utility of multimodal brain imaging biomarkers to eventually inform clinical decision-making.

Multimodal fusion of brain imaging data: A key to finding the missing link(s) in complex mental illness

It is becoming increasingly clear that combining multi-modal brain imaging data is able to provide more information for individual subjects by exploiting the rich multimodal information that exists. However, the number of studies that do true multimodal fusion (i.e. capitalizing on joint information among modalities) is still remarkably small given the known benefits. In part, this is because multi-modal studies require broader expertise in collecting, analyzing, and interpreting the results than do unimodal studies. In this paper, we start by introducing the basic reasons why multimodal data fusion is important and what it can do, and importantly how it can help us avoid wrong conclusions and help compensate for imperfect brain imaging studies. We also discuss the challenges that need to be confronted for such approaches to be more widely applied by the community. We then provide a review of the diverse studies that have used multimodal data fusion (primarily focused on psychosis) as well as provide an introduction to some of the existing analytic approaches. Finally, we discuss some up-and-coming approaches to multi-modal fusion including deep learning and multimodal classification which show considerable promise. Our conclusion is that multimodal data fusion is rapidly growing, but it is still underutilized. The complexity of the human brain coupled with the incomplete measurement provided by existing imaging technology makes multimodal fusion essential in order to mitigate against misdirection and hopefully provide a key to finding the missing link(s) in complex mental illness.

Revisiting the Basic Symptom Concept: Toward Translating Risk Symptoms for Psychosis into Neurobiological Targets

In its initial formulation, the concept of basic symptoms (BSs) integrated findings on the early symptomatic course of schizophrenia and first in vivo evidence of accompanying brain aberrations. It argued that the subtle subclinical disturbances in mental processes described as BSs were the most direct self-experienced expression of the underlying neurobiological aberrations of the disease. Other characteristic symptoms of psychosis (e.g., delusions and hallucinations) were conceptualized as secondary phenomena, resulting from dysfunctional beliefs and suboptimal coping styles with emerging BSs and/or concomitant stressors. While BSs can occur in many mental disorders, in particular affective disorders, a subset of perceptive and cognitive BSs appear to be specific to psychosis and are currently employed in two alternative risk criteria. However, despite their clinical recognition in the early detection of psychosis, neurobiological research on the aetiopathology of psychosis with neuroimaging methods has only just begun to consider the neural correlate of BSs. This perspective paper reviews the emerging evidence of an association between BSs and aberrant brain activation, connectivity patterns, and metabolism, and outlines promising routes for the use of BSs in aetiopathological research on psychosis.

Assessing brain structural associations with working-memory related brain patterns in schizophrenia and healthy controls using linked independent component analysis

Schizophrenia (SZ) is a psychotic disorder with significant cognitive dysfunction. Abnormal brain activation during cognitive processing has been reported, both in task-positive and task-negative networks. Further, structural cortical and subcortical brain abnormalities have been documented, but little is known about how task-related brain activation is associated with brain anatomy in SZ compared to healthy controls (HC). Utilizing linked independent component analysis (LICA), a data-driven multimodal analysis approach, we investigated structure-function associations in a large sample of SZ (n = 96) and HC (n = 142). We tested for associations between task-positive (fronto-parietal) and task-negative (default-mode) brain networks derived from fMRI activation during an n-back working memory task, and brain structural measures of surface area, cortical thickness, and gray matter volume, and to what extent these associations differed in SZ compared to HC. A significant association (p < .05, corrected for multiple comparisons) was found between a component reflecting the task-positive fronto-parietal network and another component reflecting cortical thickness in fronto-temporal brain regions in SZ, indicating increased activation with increased thickness. Other structure-function associations across, between and within groups were generally moderate and significant at a nominal p-level only, with more numerous and stronger associations in SZ compared to HC. These results indicate a complex pattern of moderate associations between brain activation during cognitive processing and brain morphometry, and extend previous findings of fronto-temporal brain abnormalities in SZ by suggesting a coupling between cortical thickness of these brain regions and working memory-related brain activation.

A multimodal analysis of antipsychotic effects on brain structure and function in first-episode schizophrenia

IMPORTANCE

Recent data suggest that treatment with antipsychotics is associated with reductions in cortical gray matter in patients with schizophrenia. These findings have led to concerns about the effect of antipsychotic treatment on brain structure and function; however, no studies to date have measured cortical function directly in individuals with schizophrenia and shown antipsychotic-related reductions of gray matter.

OBJECTIVE

To examine the effects of antipsychotics on brain structure and function in patients with first-episode schizophrenia, using cortical thickness measurements and administration of the AX version of the Continuous Performance Task (AX-CPT) during event-related functional magnetic resonance imaging.

DESIGN, SETTING, AND PARTICIPANTS

This case-control cross-sectional study was conducted at the Imaging Research Center of the University of California, Davis, from November 2004 through July 2012. Participants were recruited on admission into the Early Diagnosis and Preventive Treatment Clinic, an outpatient clinic specializing in first-episode psychosis. Patients with first-episode schizophrenia who received atypical antipsychotics (medicated patient group) (n = 23) and those who received no antipsychotics (unmedicated patient group) (n = 22) and healthy control participants (n = 37) underwent functional magnetic resonance imaging using a 1.5-T scanner.

MAIN OUTCOMES AND MEASURES

Behavioral performance was measured by trial accuracy, reaction time, and d'-context score. Voxelwise statistical parametric maps tested differences in functional activity during the AX-CPT, and vertexwise maps of cortical thickness tested differences in cortical thickness across the whole brain.

RESULTS

Significant cortical thinning was identified in the medicated patient group relative to the control group in prefrontal (mean reduction [MR], 0.27 mm; P < .001), temporal (MR, 0.34 mm; P = .02), parietal (MR, 0.21 mm; P = .001), and occipital (MR, 0.24 mm; P = .001) cortices. The unmedicated patient group showed no significant cortical thickness differences from the control group after clusterwise correction. The medicated patient group showed thinner cortex compared with the unmedicated patient group in the dorsolateral prefrontal cortex (DLPFC) (MR, 0.26 mm; P = .001) and temporal cortex (MR, 0.33 mm; P = .047). During the AX-CPT, both patient groups showed reduced DLPFC activity compared with the control group (P = .02 compared with the medicated group and P < .001 compared with the unmedicated group). However, the medicated patient group demonstrated higher DLPFC activation (P = .02) and better behavioral performance (P = .02) than the unmedicated patient group.

CONCLUSIONS AND RELEVANCE

These findings highlight the complex relationship between antipsychotic treatment and the structural, functional, and behavioral deficits repeatedly identified in schizophrenia. Although short-term treatment with antipsychotics was associated with prefrontal cortical thinning, treatment was also associated with better cognitive control and increased prefrontal functional activity. This study adds important context to the growing literature on the effects of antipsychotics on the brain and suggests caution in interpreting neuroanatomical changes as being related to a potentially adverse effect on brain function.

Does neuroanatomy account for superior temporal dysfunction in early psychosis? A multimodal MRI investigation

BACKGROUND

Neuroimaging studies of ultra-high risk (UHR) and first-episode psychosis (FEP) have revealed widespread alterations in brain structure and function. Recent evidence suggests there is an intrinsic relationship between these 2 types of alterations; however, there is very little research linking these 2 modalities in the early stages of psychosis.

METHODS

To test the hypothesis that functional alteration in UHR and FEP articipants would be associated with corresponding structural alteration, we examined brain function and structure in these participants as well as in a group of healthy controls using multimodal MRI. The data were analyzed using statistical parametric mapping.

RESULTS

We included 24 participants in the FEP group, 18 in the UHR group and 21 in the control group. Patients in the FEP group showed a reduction in functional activation in the left superior temporal gyrus relative to controls, and the UHR group showed intermediate values. The same region showed a corresponding reduction in grey matter volume in the FEP group relative to controls. However, while the difference in grey matter volume remained significant after including functional activation as a covariate of no interest, the reduction in functional activation was no longer evident after including grey matter volume as a covariate of no interest.

LIMITATIONS

Our sample size was relatively small. All participants in the FEP group and 2 in the UHR group had received antipsychotic medication, which may have impacted neurofunction and/or neuroanatomy.

CONCLUSION

Our results suggest that superior temporal dysfunction in early psychosis is accounted for by a corresponding alteration in grey matter volume. This finding has important implications for the interpretation of functional alteration in early psychosis.

Current status and future role of brain PET/MRI in clinical and research settings

Hybrid PET/MRI systematically offers a complementary combination of two modalities that has often proven itself superior to the single modality approach in the diagnostic work-up of many neurological and psychiatric diseases. Emerging PET tracers, technical advances in multiparametric MRI and obvious workflow advantages may lead to a significant improvement in the diagnosis of dementia disorders, neurooncological diseases, epilepsy and neurovascular diseases using PET/MRI. Moreover, simultaneous PET/MRI is well suited to complex studies of brain function in which fast fluctuations of brain signals (e.g. related to task processing or in response to pharmacological interventions) need to be monitored on multiple levels. Initial simultaneous studies have already demonstrated that these complementary measures of brain function can provide new insights into the functional and structural organization of the brain.

Left fronto-temporal dysconnectivity within the language network in schizophrenia: an fMRI and DTI study

Schizophrenia is a mental disorder characterized by language disorders. Studies reveal that both a functional dysconnectivity and a disturbance in the integrity of white matter fibers are implicated in the language process in patients with schizophrenia. Here, we investigate the relationship between functional connectivity within a language-comprehension network and anatomical connectivity using fiber tracking in schizophrenia. We hypothesized that patients would present an impaired functional connectivity in the language network due to anatomical dysconnectivity. Participants comprised 20 patients with DSM-IV schizophrenia and 20 healthy controls who were studied with functional magnetic resonance imaging and diffusion tensor imaging. The temporal correlation coefficient and diffusion values between the left frontal and temporal clusters, belonging to the language network, were individually extracted, in order to study the relationships of anatomo-functional connectivity. In patients, functional connectivity was positively correlated with fractional anisotropy, but was negatively correlated with radial diffusivity and/or mean diffusivity, in the left arcuate fasciculus and part of the inferior occipitofrontal fasciculus, determined as the fronto-temporal tracts. Our findings indicate a close relationship between functional and anatomical dysconnectivity in patients with schizophrenia. The disturbance in the integrity of the left fronto-temporal tracts might be one origin of the functional dysconnectivity in the language-comprehension network in schizophrenia.

Functional and white matter abnormalities in the language network in patients with schizophrenia: a combined study with diffusion tensor imaging and functional magnetic resonance imaging

BACKGROUND

Schizophrenia is a mental disorder characterized by functional abnormalities in the language network. Anatomical white matter (WM) abnormalities (volume and integrity) have also been reported for this pathology. Nevertheless, few studies have investigated anatomo-functional relationships in schizophrenia, and none has focused on the language comprehension network in relation to various diffusion parameters. We hypothesized that the WM abnormalities that are reflected by several diffusion parameters underlie functional deficits in the language network.

METHODS

Eighteen DSM-IV patients with schizophrenia and 18 healthy controls without any significant differences in sex, age, or level of education were included. First, functional brain activation within the language network was estimated. Then, using diffusion tensor imaging, fractional anisotropy (FA), radial diffusivity (RD), and mean diffusivity (MD) values were extracted within WM regions adjacent to this network and their anatomo-functional relationships were investigated.

RESULTS

Compared with healthy participants, both functional and diffusion deficits were observed in patients with schizophrenia. Primarily, an altered diffusion-functional relationship was observed in patients in the left middle temporal region: functional activations were positively correlated with FA, but were negatively correlated with RD.

CONCLUSIONS

Our findings indicate a close relationship between diffusion and functional deficits in patients with schizophrenia, suggesting that WM integrity disturbance might be one cause of functional alterations in the language network in patients with schizophrenia. Thus, the present multimodal study improves our understanding of the pathophysiology of schizophrenia.

Function-structure associations of the brain: evidence from multimodal connectivity and covariance studies

Despite significant advances in multimodal imaging techniques and analysis approaches, unimodal studies are still the predominant way to investigate brain changes or group differences, including structural magnetic resonance imaging (sMRI), functional MRI (fMRI), diffusion tensor imaging (DTI) and electroencephalography (EEG). Multimodal brain studies can be used to understand the complex interplay of anatomical, functional and physiological brain alterations or development, and to better comprehend the biological significance of multiple imaging measures. To examine the function-structure associations of the brain in a more comprehensive and integrated manner, we reviewed a number of multimodal studies that combined two or more functional (fMRI and/or EEG) and structural (sMRI and/or DTI) modalities. In this review paper, we specifically focused on multimodal neuroimaging studies on cognition, aging, disease and behavior. We also compared multiple analysis approaches, including univariate and multivariate methods. The possible strengths and limitations of each method are highlighted, which can guide readers when selecting a method based on a given research question. In particular, we believe that multimodal fusion approaches will shed further light on the neuronal mechanisms underlying the major structural and functional pathophysiological features of both the healthy brain (e.g. development) or the diseased brain (e.g. mental illness) and, in the latter case, may provide a more sensitive measure than unimodal imaging for disease classification, e.g. multimodal biomarkers, which potentially can be used to support clinical diagnosis based on neuroimaging techniques.