Discriminating schizophrenia using recurrent neural network applied on time courses of multi-site FMRI data

Neuroimage Analysis Methods and Artificial Intelligence Techniques for Reliable Biomarkers and Accurate Diagnosis of Schizophrenia: Achievements Made by Chinese Scholars Around the Past Decade

BACKGROUND AND HYPOTHESIS

Schizophrenia (SZ) is characterized by significant cognitive and behavioral disruptions. Neuroimaging techniques, particularly magnetic resonance imaging (MRI), have been widely utilized to investigate biomarkers of SZ, distinguish SZ from healthy conditions or other mental disorders, and explore biotypes within SZ or across SZ and other mental disorders, which aim to promote the accurate diagnosis of SZ. In China, research on SZ using MRI has grown considerably in recent years.

STUDY DESIGN

The article reviews advanced neuroimaging and artificial intelligence (AI) methods using single-modal or multimodal MRI to reveal the mechanism of SZ and promote accurate diagnosis of SZ, with a particular emphasis on the achievements made by Chinese scholars around the past decade.

STUDY RESULTS

Our article focuses on the methods for capturing subtle brain functional and structural properties from the high-dimensional MRI data, the multimodal fusion and feature selection methods for obtaining important and sparse neuroimaging features, the supervised statistical analysis and classification for distinguishing disorders, and the unsupervised clustering and semi-supervised learning methods for identifying neuroimage-based biotypes. Crucially, our article highlights the characteristics of each method and underscores the interconnections among various approaches regarding biomarker extraction and neuroimage-based diagnosis, which is beneficial not only for comprehending SZ but also for exploring other mental disorders.

CONCLUSIONS

We offer a valuable review of advanced neuroimage analysis and AI methods primarily focused on SZ research by Chinese scholars, aiming to promote the diagnosis, treatment, and prevention of SZ, as well as other mental disorders, both within China and internationally.

Connectional-style-guided contextual representation learning for brain disease diagnosis

Structural magnetic resonance imaging (sMRI) has shown great clinical value and has been widely used in deep learning (DL) based computer-aided brain disease diagnosis. Previous DL-based approaches focused on local shapes and textures in brain sMRI that may be significant only within a particular domain. The learned representations are likely to contain spurious information and have poor generalization ability in other diseases and datasets. To facilitate capturing meaningful and robust features, it is necessary to first comprehensively understand the intrinsic pattern of the brain that is not restricted within a single data/task domain. Considering that the brain is a complex connectome of interlinked neurons, the connectional properties in the brain have strong biological significance, which is shared across multiple domains and covers most pathological information. In this work, we propose a connectional style contextual representation learning model (CS-CRL) to capture the intrinsic pattern of the brain, used for multiple brain disease diagnosis. Specifically, it has a vision transformer (ViT) encoder and leverages mask reconstruction as the proxy task and Gram matrices to guide the representation of connectional information. It facilitates the capture of global context and the aggregation of features with biological plausibility. The results indicate that CS-CRL achieves superior accuracy in multiple brain disease diagnosis tasks across six datasets and three diseases and outperforms state-of-the-art models. Furthermore, we demonstrate that CS-CRL captures more brain-network-like properties, and better aggregates features, is easier to optimize, and is more robust to noise, which explains its superiority in theory.

Deep learning for MRI lesion segmentation in rectal cancer

Rectal cancer (RC) is a globally prevalent malignant tumor, presenting significant challenges in its management and treatment. Currently, magnetic resonance imaging (MRI) offers superior soft tissue contrast and radiation-free effects for RC patients, making it the most widely used and effective detection method. In early screening, radiologists rely on patients' medical radiology characteristics and their extensive clinical experience for diagnosis. However, diagnostic accuracy may be hindered by factors such as limited expertise, visual fatigue, and image clarity issues, resulting in misdiagnosis or missed diagnosis. Moreover, the distribution of surrounding organs in RC is extensive with some organs having similar shapes to the tumor but unclear boundaries; these complexities greatly impede doctors' ability to diagnose RC accurately. With recent advancements in artificial intelligence, machine learning techniques like deep learning (DL) have demonstrated immense potential and broad prospects in medical image analysis. The emergence of this approach has significantly enhanced research capabilities in medical image classification, detection, and segmentation fields with particular emphasis on medical image segmentation. This review aims to discuss the developmental process of DL segmentation algorithms along with their application progress in lesion segmentation from MRI images of RC to provide theoretical guidance and support for further advancements in this field.

Diagnosing schizophrenia using deep learning: Novel interpretation approaches and multi-site validation

Schizophrenia is a profound and enduring mental disorder that imposes significant negative impacts on individuals, their families, and society at large. The development of more accurate and objective diagnostic tools for schizophrenia can be expedited through the employment of deep learning (DL), that excels at deciphering complex hierarchical non-linear patterns. However, the limited interpretability of deep learning has eroded confidence in the model and restricted its clinical utility. At the same time, if the data source is only derived from a single center, the model's generalizability is difficult to test. To enhance the model's reliability and applicability, leave-one-center-out validation with a large and diverse sample from multiple centers is crucial. In this study, we utilized Nine different global centers to train and test the 3D Resnet model's generalizability, resulting in an 82% classification performance (area under the curve) on all datasets sourced from different countries, employing a leave-one-center-out-validation approach. Per our approximation of the feature significance of each region on the atlas, we identified marked differences in the thalamus, pallidum, and inferior frontal gyrus between individuals with schizophrenia and healthy controls, lending credence to prior research findings. At the same time, in order to translate the model's output into clinically applicable insights, the SHapley Additive exPlanations (SHAP) permutation explainer method with an anatomical atlas have been refined, thereby offering precise neuroanatomical and functional interpretations of different brain regions.

Multi modality fusion transformer with spatio-temporal feature aggregation module for psychiatric disorder diagnosis

Bipolar disorder (BD) is characterized by recurrent episodes of depression and mild mania. In this paper, to address the common issue of insufficient accuracy in existing methods and meet the requirements of clinical diagnosis, we propose a framework called Spatio-temporal Feature Fusion Transformer (STF2Former). It improves on our previous work - MFFormer by introducing a Spatio-temporal Feature Aggregation Module (STFAM) to learn the temporal and spatial features of rs-fMRI data. It promotes intra-modality attention and information fusion across different modalities. Specifically, this method decouples the temporal and spatial dimensions and designs two feature extraction modules for extracting temporal and spatial information separately. Extensive experiments demonstrate the effectiveness of our proposed STFAM in extracting features from rs-fMRI, and prove that our STF2Former can significantly outperform MFFormer and achieve much better results among other state-of-the-art methods.

Adaptive spatial-temporal neural network for ADHD identification using functional fMRI

Computer aided diagnosis methods play an important role in Attention Deficit Hyperactivity Disorder (ADHD) identification. Dynamic functional connectivity (dFC) analysis has been widely used for ADHD diagnosis based on resting-state functional magnetic resonance imaging (rs-fMRI), which can help capture abnormalities of brain activity. However, most existing dFC-based methods only focus on dependencies between two adjacent timestamps, ignoring global dynamic evolution patterns. Furthermore, the majority of these methods fail to adaptively learn dFCs. In this paper, we propose an adaptive spatial-temporal neural network (ASTNet) comprising three modules for ADHD identification based on rs-fMRI time series. Specifically, we first partition rs-fMRI time series into multiple segments using non-overlapping sliding windows. Then, adaptive functional connectivity generation (AFCG) is used to model spatial relationships among regions-of-interest (ROIs) with adaptive dFCs as input. Finally, we employ a temporal dependency mining (TDM) module which combines local and global branches to capture global temporal dependencies from the spatially-dependent pattern sequences. Experimental results on the ADHD-200 dataset demonstrate the superiority of the proposed ASTNet over competing approaches in automated ADHD classification.

Multi-kernel Learning Fusion Algorithm Based on RNN and GRU for ASD Diagnosis and Pathogenic Brain Region Extraction

Autism spectrum disorder (ASD) is a complex, severe disorder related to brain development. It impairs patient language communication and social behaviors. In recent years, ASD researches have focused on a single-modal neuroimaging data, neglecting the complementarity between multi-modal data. This omission may lead to poor classification. Therefore, it is important to study multi-modal data of ASD for revealing its pathogenesis. Furthermore, recurrent neural network (RNN) and gated recurrent unit (GRU) are effective for sequence data processing. In this paper, we introduce a novel framework for a Multi-Kernel Learning Fusion algorithm based on RNN and GRU (MKLF-RAG). The framework utilizes RNN and GRU to provide feature selection for data of different modalities. Then these features are fused by MKLF algorithm to detect the pathological mechanisms of ASD and extract the most relevant the Regions of Interest (ROIs) for the disease. The MKLF-RAG proposed in this paper has been tested in a variety of experiments with the Autism Brain Imaging Data Exchange (ABIDE) database. Experimental findings indicate that our framework notably enhances the classification accuracy for ASD. Compared with other methods, MKLF-RAG demonstrates superior efficacy across multiple evaluation metrics and could provide valuable insights into the early diagnosis of ASD.

A Brainwide Risk Score for Psychiatric Disorder Evaluated in a Large Adolescent Population Reveals Increased Divergence Among Higher-Risk Groups Relative to Control Participants

BACKGROUND

Accurate psychiatric risk assessment requires biomarkers that are both stable and adaptable to development. Functional network connectivity (FNC), which steadily reconfigures over time, potentially contains abundant information to assess psychiatric risks. However, the absence of suitable analytical methodologies has constrained this area of investigation.

METHODS

We investigated the brainwide risk score (BRS), a novel FNC-based metric that contrasts the relative distances of an individual's FNC to that of psychiatric disorders versus healthy control references. To generate group-level disorder and healthy control references, we utilized a large brain imaging dataset containing 5231 total individuals diagnosed with schizophrenia, autism spectrum disorder, major depressive disorder, and bipolar disorder and their corresponding healthy control individuals. The BRS metric was employed to assess the psychiatric risk in 2 new datasets: Adolescent Brain Cognitive Development (ABCD) Study (n = 8191) and Human Connectome Project Early Psychosis (n = 170).

RESULTS

The BRS revealed a clear, reproducible gradient of FNC patterns from low to high risk for each psychiatric disorder in unaffected adolescents. We found that low-risk ABCD Study adolescent FNC patterns for each disorder were strongly present in over 25% of the ABCD Study participants and homogeneous, whereas high-risk patterns of each psychiatric disorder were strongly present in about 1% of ABCD Study participants and heterogeneous. The BRS also showed its effectiveness in predicting psychosis scores and distinguishing individuals with early psychosis from healthy control individuals.

CONCLUSIONS

The BRS could be a new image-based tool for assessing psychiatric vulnerability over time and in unaffected individuals, and it could also serve as a potential biomarker, facilitating early screening and monitoring interventions.

Explainable fuzzy clustering framework reveals divergent default mode network connectivity dynamics in schizophrenia

Introduction

Dynamic functional network connectivity (dFNC) analysis of resting state functional magnetic resonance imaging data has yielded insights into many neurological and neuropsychiatric disorders. A common dFNC analysis approach uses hard clustering methods like k-means clustering to assign samples to states that summarize network dynamics. However, hard clustering methods obscure network dynamics by assuming (1) that all samples within a cluster are equally like their assigned centroids and (2) that samples closer to one another in the data space than to their centroids are well-represented by their centroids. In addition, it can be hard to compare subjects, as in some cases an individual may not manifest a state strongly enough to enter a hard cluster. Approaches that allow a dimensional approach to connectivity patterns (e.g., fuzzy clustering) can mitigate these issues. In this study, we present an explainable fuzzy clustering framework by combining fuzzy c-means clustering with several explainability metrics and novel summary features.

Methods

We apply our framework for schizophrenia (SZ) default mode network analysis. Namely, we extract dFNC from individuals with SZ and controls, identify 5 dFNC states, and characterize the dFNC features most crucial to those states with a new perturbation-based clustering explainability approach. We then extract several features typically used in hard clustering and further present a variety of unique features specially designed for use with fuzzy clustering to quantify state dynamics. We examine differences in those features between individuals with SZ and controls and further search for relationships between those features and SZ symptom severity.

Results

Importantly, we find that individuals with SZ spend more time in states of moderate anticorrelation between the anterior and posterior cingulate cortices and strong anticorrelation between the precuneus and anterior cingulate cortex. We further find that individuals with SZ tend to transition more rapidly than controls between low-magnitude and high-magnitude dFNC states.

Conclusion

We present a novel dFNC analysis framework and use it to identify effects of SZ upon network dynamics. Given the ease of implementing our framework and its enhanced insight into network dynamics, it has great potential for use in future dFNC studies.

Multi feature fusion network for schizophrenia classification and abnormal brain network recognition

Schizophrenia classification and abnormal brain network recognition have an important research significance. Researchers have proposed many classification methods based on machine learning and deep learning. However, fewer studies utilized the advantages of complementary information from multi feature to learn the best representation of schizophrenia. In this study, we proposed a multi-feature fusion network (MFFN) using functional network connectivity (FNC) and time courses (TC) to distinguish schizophrenia patients from healthy controls. DNN backbone was adopted to learn the feature map of functional network connectivity, C-RNNAM backbone was designed to learn the feature map of time courses, and Deep SHAP was applied to obtain the most discriminative brain networks. We proved the effectiveness of this proposed model using the combining two public datasets and evaluated this model quantitatively using the evaluation indexes. The results showed that the functional network connectivity generated by independent component analysis has advantage in schizophrenia classification by comparing static and dynamic functional connections. This method obtained the best classification accuracy (ACC=87.30%, SPE=89.28%, SEN=85.71%, F1 =88.23%, and AUC=0.9081), and it demonstrated the superiority of this proposed model by comparing state-of-the-art methods. Ablation experiment also demonstrated that multi feature fusion and attention module can improve classification accuracy. The most discriminative brain networks showed that default mode network and visual network of schizophrenia patients have aberrant connections in brain networks. In conclusion, this method can identify schizophrenia effectively and visualize the abnormal brain network, and it has important clinical application value.

Dynamic functional connectivity analysis with temporal convolutional network for attention deficit/hyperactivity disorder identification

Introduction

Dynamic functional connectivity (dFC), which can capture the abnormality of brain activity over time in resting-state functional magnetic resonance imaging (rs-fMRI) data, has a natural advantage in revealing the abnormal mechanism of brain activity in patients with Attention Deficit/Hyperactivity Disorder (ADHD). Several deep learning methods have been proposed to learn dynamic changes from rs-fMRI for FC analysis, and achieved superior performance than those using static FC. However, most existing methods only consider dependencies of two adjacent timestamps, which is limited when the change is related to the course of many timestamps.

Methods

In this paper, we propose a novel Temporal Dependence neural Network (TDNet) for FC representation learning and temporal-dependence relationship tracking from rs-fMRI time series for automated ADHD identification. Specifically, we first partition rs-fMRI time series into a sequence of consecutive and non-overlapping segments. For each segment, we design an FC generation module to learn more discriminative representations to construct dynamic FCs. Then, we employ the Temporal Convolutional Network (TCN) to efficiently capture long-range temporal patterns with dilated convolutions, followed by three fully connected layers for disease prediction.

Results

As the results, we found that considering the dynamic characteristics of rs-fMRI time series data is beneficial to obtain better diagnostic performance. In addition, dynamic FC networks generated in a data-driven manner are more informative than those constructed by Pearson correlation coefficients.

Discussion

We validate the effectiveness of the proposed approach through extensive experiments on the public ADHD-200 database, and the results demonstrate the superiority of the proposed model over state-of-the-art methods in ADHD identification.

An attention-based multi-modal MRI fusion model for major depressive disorder diagnosis

Objective.Major depressive disorder (MDD) is one of the biggest threats to human mental health. MDD is characterized by aberrant changes in both structure and function of the brain. Although recent studies have developed some deep learning models based on multi-modal magnetic resonance imaging (MRI) for MDD diagnosis, the latent associations between deep features derived from different modalities were largely unexplored by previous studies, which we hypothesized may have potential benefits in improving the diagnostic accuracy of MDD.Approach.In this study, we proposed a novel deep learning model that fused both structural MRI (sMRI) and resting-state MRI (rs-fMRI) data to enhance the diagnosis of MDD by capturing the interactions between deep features extracted from different modalities. Specifically, we first employed a brain function encoder (BFE) and a brain structure encoder (BSE) to extract the deep features from fMRI and sMRI, respectively. Then, we designed a function and structure co-attention fusion (FSCF) module that captured inter-modal interactions and adaptively fused multi-modal deep features for MDD diagnosis.Main results.This model was evaluated on a large cohort and achieved a high classification accuracy of 75.2% for MDD diagnosis. Moreover, the attention distribution of the FSCF module assigned higher attention weights to structural features than functional features for diagnosing MDD.Significance.The high classification accuracy highlights the effectiveness and potential clinical of the proposed model.

Dissecting Psychiatric Heterogeneity and Comorbidity with Core Region-Based Machine Learning

Machine learning approaches are increasingly being applied to neuroimaging data from patients with psychiatric disorders to extract brain-based features for diagnosis and prognosis. The goal of this review is to discuss recent practices for evaluating machine learning applications to obsessive-compulsive and related disorders and to advance a novel strategy of building machine learning models based on a set of core brain regions for better performance, interpretability, and generalizability. Specifically, we argue that a core set of co-altered brain regions (namely 'core regions') comprising areas central to the underlying psychopathology enables the efficient construction of a predictive model to identify distinct symptom dimensions/clusters in individual patients. Hypothesis-driven and data-driven approaches are further introduced showing how core regions are identified from the entire brain. We demonstrate a broadly applicable roadmap for leveraging this core set-based strategy to accelerate the pursuit of neuroimaging-based markers for diagnosis and prognosis in a variety of psychiatric disorders.

A meta-analysis and systematic review of single vs. multimodal neuroimaging techniques in the classification of psychosis

BACKGROUND

Psychotic disorders are characterized by structural and functional abnormalities in brain networks. Neuroimaging techniques map and characterize such abnormalities using unique features (e.g., structural integrity, coactivation). However, it is unclear if a specific method, or a combination of modalities, is particularly effective in identifying differences in brain networks of someone with a psychotic disorder.

METHODS

A systematic meta-analysis evaluated machine learning classification of schizophrenia spectrum disorders in comparison to healthy control participants using various neuroimaging modalities (i.e., T1-weighted imaging (T1), diffusion tensor imaging (DTI), resting state functional connectivity (rs-FC), or some combination (multimodal)). Criteria for manuscript inclusion included whole-brain analyses and cross-validation to provide a complete picture regarding the predictive ability of large-scale brain systems in psychosis. For this meta-analysis, we searched Ovid MEDLINE, PubMed, PsychInfo, Google Scholar, and Web of Science published between inception and March 13th 2023. Prediction results were averaged for studies using the same dataset, but parallel analyses were run that included studies with pooled sample across many datasets. We assessed bias through funnel plot asymmetry. A bivariate regression model determined whether differences in imaging modality, demographics, and preprocessing methods moderated classification. Separate models were run for studies with internal prediction (via cross-validation) and external prediction.

RESULTS

93 studies were identified for quantitative review (30 T1, 9 DTI, 40 rs-FC, and 14 multimodal). As a whole, all modalities reliably differentiated those with schizophrenia spectrum disorders from controls (OR = 2.64 (95%CI = 2.33 to 2.95)). However, classification was relatively similar across modalities: no differences were seen across modalities in the classification of independent internal data, and a small advantage was seen for rs-FC studies relative to T1 studies in classification in external datasets. We found large amounts of heterogeneity across results resulting in significant signs of bias in funnel plots and Egger's tests. Results remained similar, however, when studies were restricted to those with less heterogeneity, with continued small advantages for rs-FC relative to structural measures. Notably, in all cases, no significant differences were seen between multimodal and unimodal approaches, with rs-FC and unimodal studies reporting largely overlapping classification performance. Differences in demographics and analysis or denoising were not associated with changes in classification scores.

CONCLUSIONS

The results of this study suggest that neuroimaging approaches have promise in the classification of psychosis. Interestingly, at present most modalities perform similarly in the classification of psychosis, with slight advantages for rs-FC relative to structural modalities in some specific cases. Notably, results differed substantially across studies, with suggestions of biased effect sizes, particularly highlighting the need for more studies using external prediction and large sample sizes. Adopting more rigorous and systematized standards will add significant value toward understanding and treating this critical population.

Reinforcement learning deficits exhibited by postnatal PCP-treated rats enable deep neural network classification

The ability to appropriately update the value of a given action is a critical component of flexible decision making. Several psychiatric disorders, including schizophrenia, are associated with impairments in flexible decision making that can be evaluated using the probabilistic reversal learning (PRL) task. The PRL task has been reverse-translated for use in rodents. Disrupting glutamate neurotransmission during early postnatal neurodevelopment in rodents has induced behavioral, cognitive, and neuropathophysiological abnormalities relevant to schizophrenia. Here, we tested the hypothesis that using the NMDA receptor antagonist phencyclidine (PCP) to disrupt postnatal glutamatergic transmission in rats would lead to impaired decision making in the PRL. Consistent with this hypothesis, compared to controls the postnatal PCP-treated rats completed fewer reversals and exhibited disruptions in reward and punishment sensitivity (i.e., win-stay and lose-shift responding, respectively). Moreover, computational analysis of behavior revealed that postnatal PCP-treatment resulted in a pronounced impairment in the learning rate throughout PRL testing. Finally, a deep neural network (DNN) trained on the rodent behavior could accurately predict the treatment group of subjects. These data demonstrate that disrupting early postnatal glutamatergic neurotransmission impairs flexible decision making and provides evidence that DNNs can be trained on behavioral datasets to accurately predict the treatment group of new subjects, highlighting the potential for DNNs to aid in the diagnosis of schizophrenia.

Sampling inequalities affect generalization of neuroimaging-based diagnostic classifiers in psychiatry

BACKGROUND

The development of machine learning models for aiding in the diagnosis of mental disorder is recognized as a significant breakthrough in the field of psychiatry. However, clinical practice of such models remains a challenge, with poor generalizability being a major limitation.

METHODS

Here, we conducted a pre-registered meta-research assessment on neuroimaging-based models in the psychiatric literature, quantitatively examining global and regional sampling issues over recent decades, from a view that has been relatively underexplored. A total of 476 studies (n = 118,137) were included in the current assessment. Based on these findings, we built a comprehensive 5-star rating system to quantitatively evaluate the quality of existing machine learning models for psychiatric diagnoses.

RESULTS

A global sampling inequality in these models was revealed quantitatively (sampling Gini coefficient (G) = 0.81, p < .01), varying across different countries (regions) (e.g., China, G = 0.47; the USA, G = 0.58; Germany, G = 0.78; the UK, G = 0.87). Furthermore, the severity of this sampling inequality was significantly predicted by national economic levels (β =  - 2.75, p < .001, R²_adj = 0.40; r =  - .84, 95% CI: - .41 to - .97), and was plausibly predictable for model performance, with higher sampling inequality for reporting higher classification accuracy. Further analyses showed that lack of independent testing (84.24% of models, 95% CI: 81.0-87.5%), improper cross-validation (51.68% of models, 95% CI: 47.2-56.2%), and poor technical transparency (87.8% of models, 95% CI: 84.9-90.8%)/availability (80.88% of models, 95% CI: 77.3-84.4%) are prevailing in current diagnostic classifiers despite improvements over time. Relating to these observations, model performances were found decreased in studies with independent cross-country sampling validations (all p < .001, BF₁₀ > 15). In light of this, we proposed a purpose-built quantitative assessment checklist, which demonstrated that the overall ratings of these models increased by publication year but were negatively associated with model performance.

CONCLUSIONS

Together, improving sampling economic equality and hence the quality of machine learning models may be a crucial facet to plausibly translating neuroimaging-based diagnostic classifiers into clinical practice.

Multi-scale convolutional recurrent neural network for psychiatric disorder identification in resting-state EEG

Background

Accurate classification based on affordable objective neuroimaging biomarkers are important steps toward designing individualized treatment.

Methods

In this work, we investigated a deep learning classification model, multi-scale convolutional recurrent neural network (MCRNN), to explore psychiatric disorder-related biomarkers by leveraging the spatiotemporal information of resting-state EEG (rsEEG) using a multiple psychiatric disorder database containing 327 individuals diagnosed with schizophrenia, bipolar, major depressive disorders, and healthy controls. All subjects were mapped to a shared low-dimensional subspace for intuitively interpreting the inter-relationship and separation of psychiatric disorders.

Results

Psychiatric disorders were identified using rsEEG with high accuracy ranged from 78.6 to 91.3% in patient vs. controls two-class classification, and 68.2% in four-class classification. The control-to-schizophrenia trajectory interpretated by the model was consistent with the disease severity in clinical observation.

Conclusion

The MsRNN demonstrated a capability in extracting discriminative rsEEG biomarkers for psychiatric disorder classification, indicating its potential to facilitate our understanding of psychiatric disorders and monitoring interventions.

Multi-model order spatially constrained ICA reveals highly replicable group differences and consistent predictive results from resting data: A large N fMRI schizophrenia study

Brain functional networks identified from resting functional magnetic resonance imaging (fMRI) data have the potential to reveal biomarkers for brain disorders, but studies of complex mental illnesses such as schizophrenia (SZ) often yield mixed results across replication studies. This is likely due in part to the complexity of the disorder, the short data acquisition time, and the limited ability of the approaches for brain imaging data mining. Therefore, the use of analytic approaches which can both capture individual variability while offering comparability across analyses is highly preferred. Fully blind data-driven approaches such as independent component analysis (ICA) are hard to compare across studies, and approaches that use fixed atlas-based regions can have limited sensitivity to individual sensitivity. By contrast, spatially constrained ICA (scICA) provides a hybrid, fully automated solution that can incorporate spatial network priors while also adapting to new subjects. However, scICA has thus far only been used with a single spatial scale (ICA dimensionality, i.e., ICA model order). In this work, we present an approach using multi-objective optimization scICA with reference algorithm (MOO-ICAR) to extract subject-specific intrinsic connectivity networks (ICNs) from fMRI data at multiple spatial scales, which also enables us to study interactions across spatial scales. We evaluate this approach using a large N (N > 1,600) study of schizophrenia divided into separate validation and replication sets. A multi-scale ICN template was estimated and labeled, then used as input into scICA which was computed on an individual subject level. We then performed a subsequent analysis of multiscale functional network connectivity (msFNC) to evaluate the patient data, including group differences and classification. Results showed highly consistent group differences in msFNC in regions including cerebellum, thalamus, and motor/auditory networks. Importantly, multiple msFNC pairs linking different spatial scales were implicated. The classification model built on the msFNC features obtained up to 85% F1 score, 83% precision, and 88% recall, indicating the strength of the proposed framework in detecting group differences between schizophrenia and the control group. Finally, we evaluated the relationship of the identified patterns to positive symptoms and found consistent results across datasets. The results verified the robustness of our framework in evaluating brain functional connectivity of schizophrenia at multiple spatial scales, implicated consistent and replicable brain networks, and highlighted a promising approach for leveraging resting fMRI data for brain biomarker development.

MPS-FFA: A multiplane and multiscale feature fusion attention network for Alzheimer's disease prediction with structural MRI

Structural magnetic resonance imaging (sMRI) is a popular technique that is widely applied in Alzheimer's disease (AD) diagnosis. However, only a few structural atrophy areas in sMRI scans are highly associated with AD. The degree of atrophy in patients' brain tissues and the distribution of lesion areas differ among patients. Therefore, a key challenge in sMRI-based AD diagnosis is identifying discriminating atrophy features. Hence, we propose a multiplane and multiscale feature-level fusion attention (MPS-FFA) model. The model has three components, (1) A feature encoder uses a multiscale feature extractor with hybrid attention layers to simultaneously capture and fuse multiple pathological features in the sagittal, coronal, and axial planes. (2) A global attention classifier combines clinical scores and two global attention layers to evaluate the feature impact scores and balance the relative contributions of different feature blocks. (3) A feature similarity discriminator minimizes the feature similarities among heterogeneous labels to enhance the ability of the network to discriminate atrophy features. The MPS-FFA model provides improved interpretability for identifying discriminating features using feature visualization. The experimental results on the baseline sMRI scans from two databases confirm the effectiveness (e.g., accuracy and generalizability) of our method in locating pathological locations. The source code is available at https://github.com/LiuFei-AHU/MPSFFA.

Automated detection of schizophrenia using deep learning: a review for the last decade

Schizophrenia (SZ) is a devastating mental disorder that disrupts higher brain functions like thought, perception, etc., with a profound impact on the individual's life. Deep learning (DL) can detect SZ automatically by learning signal data characteristics hierarchically without the need for feature engineering associated with traditional machine learning. We performed a systematic review of DL models for SZ detection. Various deep models like long short-term memory, convolution neural networks, AlexNet, etc., and composite methods have been published based on electroencephalographic signals, and structural and/or functional magnetic resonance imaging acquired from SZ patients and healthy patients control subjects in diverse public and private datasets. The studies, the study datasets, and model methodologies are reported in detail. In addition, the challenges of DL models for SZ diagnosis and future works are discussed.

Evaluation of Risk of Bias in Neuroimaging-Based Artificial Intelligence Models for Psychiatric Diagnosis: A Systematic Review

IMPORTANCE

Neuroimaging-based artificial intelligence (AI) diagnostic models have proliferated in psychiatry. However, their clinical applicability and reporting quality (ie, feasibility) for clinical practice have not been systematically evaluated.

OBJECTIVE

To systematically assess the risk of bias (ROB) and reporting quality of neuroimaging-based AI models for psychiatric diagnosis.

EVIDENCE REVIEW

PubMed was searched for peer-reviewed, full-length articles published between January 1, 1990, and March 16, 2022. Studies aimed at developing or validating neuroimaging-based AI models for clinical diagnosis of psychiatric disorders were included. Reference lists were further searched for suitable original studies. Data extraction followed the CHARMS (Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies) and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines. A closed-loop cross-sequential design was used for quality control. The PROBAST (Prediction Model Risk of Bias Assessment Tool) and modified CLEAR (Checklist for Evaluation of Image-Based Artificial Intelligence Reports) benchmarks were used to systematically evaluate ROB and reporting quality.

FINDINGS

A total of 517 studies presenting 555 AI models were included and evaluated. Of these models, 461 (83.1%; 95% CI, 80.0%-86.2%) were rated as having a high overall ROB based on the PROBAST. The ROB was particular high in the analysis domain, including inadequate sample size (398 of 555 models [71.7%; 95% CI, 68.0%-75.6%]), poor model performance examination (with 100% of models lacking calibration examination), and lack of handling data complexity (550 of 555 models [99.1%; 95% CI, 98.3%-99.9%]). None of the AI models was perceived to be applicable to clinical practices. Overall reporting completeness (ie, number of reported items/number of total items) for the AI models was 61.2% (95% CI, 60.6%-61.8%), and the completeness was poorest for the technical assessment domain with 39.9% (95% CI, 38.8%-41.1%).

CONCLUSIONS AND RELEVANCE

This systematic review found that the clinical applicability and feasibility of neuroimaging-based AI models for psychiatric diagnosis were challenged by a high ROB and poor reporting quality. Particularly in the analysis domain, ROB in AI diagnostic models should be addressed before clinical application.

Computer-aided diagnosis of schizophrenia based on node2vec and Transformer

OBJECTIVE

Compared with the healthy control (HC) group, the brain structure and function of schizophrenia (SZ) patients are significantly abnormal, so brain imaging methods can be used to achieve the aided diagnosis of SZ. However, a brain network based on brain imaging data is non-Euclidean, and its intrinsic features cannot be learned effectively by general deep learning models. Furthermore, in the majority of existing studies, brain network features were manually specified as the input of machine learning models.

METHODS

In this study, brain functional network constructed from the subject's fMRI data is analyzed, and its small-world value is calculated and t-tested; the node2vec algorithm in graph embedding is introduced to transform the constructed brain network into low-dimensional dense vectors, and the brain network's non-Euclidean spatial structure characteristics are retained to the greatest extent, so that its intrinsic features can be extracted by deep learning models; GridMask is used to randomly mask part of the information in the vectors to enhance the data; and then features can be extracted using the Transformer model to identify SZ.

RESULTS

It is again shown that the small-world value of the brain network in SZ is significantly lower than that in HC by t-test (p=0.014¡0.05). 97.78% classification accuracy is achieved by the proposed methods (node2vec + GridMask + Transformer) in 30 SZ patients and 30 healthy people.

CONCLUSION

The experiment shows that the node2vec used in this paper can effectively solve the problem of brain network features being difficult to learn by general deep learning models. The high-precision computer-aided diagnosis of SZ can be obtained by combining node2vec with Transformer and GridMask.

SIGNIFICANCE

The proposed methods in the paper are expected to be used for aided diagnosis of SZ.

Explainable Fuzzy Clustering Framework Reveals Divergent Default Mode Network Connectivity Dynamics in Schizophrenia

Dynamic functional network connectivity (dFNC) analysis of resting state functional magnetic resonance imaging data has yielded insights into many neurological and neuropsychiatric disorders. A common dFNC analysis approach uses hard clustering methods like k-means clustering to assign samples to states that summarize network dynamics. However, hard clustering methods obscure network dynamics by assuming (1) that all samples within a cluster are equally like their assigned centroids and (2) that samples closer to one another in the data space than to their centroids are well-represented by their centroids. In addition, it can be hard to compare subjects, as in some cases an individual may not manifest a state strongly enough to enter a hard cluster. Approaches that allow a dimensional approach to connectivity patterns (e.g., fuzzy clustering) can mitigate these issues. In this study, we present an explainable fuzzy clustering framework by combining fuzzy c-means clustering with several explainability metrics. We apply our framework for schizophrenia (SZ) default mode network analysis, identifying 5 states and characterizing those states with a new explainability approach. While also showing that features typically used in hard clustering can be extracted in our framework, we present a variety of unique features to quantify state dynamics and identify effects of SZ upon network dynamics. We further uncover relationships between symptom severity and interactions of the precuneus with the anterior and posterior cingulate cortex. Given the ease of implementing our framework and its enhanced insight into network dynamics, it has great potential for use in future dFNC studies.

A transformer model for learning spatiotemporal contextual representation in fMRI data

Representation learning is a core component in data-driven modeling of various complex phenomena. Learning a contextually informative representation can especially benefit the analysis of fMRI data because of the complexities and dynamic dependencies present in such datasets. In this work, we propose a framework based on transformer models to learn an embedding of the fMRI data by taking the spatiotemporal contextual information in the data into account. This approach takes the multivariate BOLD time series of the regions of the brain as well as their functional connectivity network simultaneously as the input to create a set of meaningful features that can in turn be used in various downstream tasks such as classification, feature extraction, and statistical analysis. The proposed spatiotemporal framework uses the attention mechanism as well as the graph convolution neural network to jointly inject the contextual information regarding the dynamics in time series data and their connectivity into the representation. We demonstrate the benefits of this framework by applying it to two resting-state fMRI datasets, and provide further discussion on various aspects and advantages of it over a number of other commonly adopted architectures.

Towards greater neuroimaging classification transparency via the integration of explainability methods and confidence estimation approaches

The field of neuroimaging has increasingly sought to develop artificial intelligence-based models for neurological and neuropsychiatric disorder automated diagnosis and clinical decision support. However, if these models are to be implemented in a clinical setting, transparency will be vital. Two aspects of transparency are (1) confidence estimation and (2) explainability. Confidence estimation approaches indicate confidence in individual predictions. Explainability methods give insight into the importance of features to model predictions. In this study, we integrate confidence estimation and explainability approaches for the first time. We demonstrate their viability for schizophrenia diagnosis using resting state functional magnetic resonance imaging (rs-fMRI) dynamic functional network connectivity (dFNC) data. We compare two confidence estimation approaches: Monte Carlo dropout (MCD) and MC batch normalization (MCBN). We combine them with two gradient-based explainability approaches, saliency and layer-wise relevance propagation (LRP), and examine their effects upon explanations. We find that MCD often adversely affects model gradients, making it ill-suited for integration with gradient-based explainability methods. In contrast, MCBN does not affect model gradients. Additionally, we find many participant-level differences between regular explanations and the distributions of explanations for combined explainability and confidence estimation approaches. This suggests that a similar confidence estimation approach used in a clinical context with explanations only output for the regular model would likely not yield adequate explanations. We hope that our findings will provide a starting point for the integration of the two fields, provide useful guidance for future studies, and accelerate the development of transparent neuroimaging clinical decision support systems.

Leveraging Machine Learning for Gaining Neurobiological and Nosological Insights in Psychiatric Research

Much attention is currently devoted to developing diagnostic classifiers for mental disorders. Complementing these efforts, we highlight the potential of machine learning to gain biological insights into the psychopathology and nosology of mental disorders. Studies to this end have mainly used brain imaging data, which can be obtained noninvasively from large cohorts and have repeatedly been argued to reveal potentially intermediate phenotypes. This may become particularly relevant in light of recent efforts to identify magnetic resonance imaging-derived biomarkers that yield insight into pathophysiological processes as well as to refine the taxonomy of mental illness. In particular, the accuracy of machine learning models may be used as dependent variables to identify features relevant to pathophysiology. Moreover, such approaches may help disentangle the dimensional (within diagnosis) and often overlapping (across diagnoses) symptomatology of psychiatric illness. We also point out a multiview perspective that combines data from different sources, bridging molecular and system-level information. Finally, we summarize recent efforts toward a data-driven definition of subtypes or disease entities through unsupervised and semisupervised approaches. The latter, blending unsupervised and supervised concepts, may represent a particularly promising avenue toward dissecting heterogeneous categories. Finally, we raise several technical and conceptual aspects related to the reviewed approaches. In particular, we discuss common pitfalls pertaining to flawed input data or analytic procedures that would likely lead to unreliable outputs.

A deep learning based model using RNN-LSTM for the Detection of Schizophrenia from EEG data

Normal life can be ensured for schizophrenic patients if diagnosed early. Electroencephalogram (EEG) carries information about the brain network connectivity which can be used to detect brain anomalies that are indicative of schizophrenia. Since deep learning is capable of automatically extracting the significant features and make classifications, the authors proposed a deep learning based model using RNN-LSTM to analyze the EEG signal data to diagnose schizophrenia. The proposed model used three dense layers on top of a 100 dimensional LSTM. EEG signal data of 45 schizophrenic patients and 39 healthy subjects were used in the study. Dimensionality reduction algorithm was used to obtain an optimal feature set and the classifier was run with both sets of data. An accuracy of 98% and 93.67% were obtained with the complete feature set and the reduced feature set respectively. The robustness of the model was evaluated using model performance measure and combined performance measure. Outcomes were compared with the outcome obtained with traditional machine learning classifiers such as Random Forest, SVM, FURIA, and AdaBoost, and the proposed model was found to perform better with the complete dataset. When compared with the result of the researchers who worked with the same set of data using either CNN or RNN, the proposed model's accuracy was either better or comparable to theirs.

Modern views of machine learning for precision psychiatry

In light of the National Institute of Mental Health (NIMH)'s Research Domain Criteria (RDoC), the advent of functional neuroimaging, novel technologies and methods provide new opportunities to develop precise and personalized prognosis and diagnosis of mental disorders. Machine learning (ML) and artificial intelligence (AI) technologies are playing an increasingly critical role in the new era of precision psychiatry. Combining ML/AI with neuromodulation technologies can potentially provide explainable solutions in clinical practice and effective therapeutic treatment. Advanced wearable and mobile technologies also call for the new role of ML/AI for digital phenotyping in mobile mental health. In this review, we provide a comprehensive review of ML methodologies and applications by combining neuroimaging, neuromodulation, and advanced mobile technologies in psychiatry practice. We further review the role of ML in molecular phenotyping and cross-species biomarker identification in precision psychiatry. We also discuss explainable AI (XAI) and neuromodulation in a closed human-in-the-loop manner and highlight the ML potential in multi-media information extraction and multi-modal data fusion. Finally, we discuss conceptual and practical challenges in precision psychiatry and highlight ML opportunities in future research.

Deep learning in neuroimaging data analysis: Applications, challenges, and solutions

Methods for the analysis of neuroimaging data have advanced significantly since the beginning of neuroscience as a scientific discipline. Today, sophisticated statistical procedures allow us to examine complex multivariate patterns, however most of them are still constrained by assuming inherent linearity of neural processes. Here, we discuss a group of machine learning methods, called deep learning, which have drawn much attention in and outside the field of neuroscience in recent years and hold the potential to surpass the mentioned limitations. Firstly, we describe and explain the essential concepts in deep learning: the structure and the computational operations that allow deep models to learn. After that, we move to the most common applications of deep learning in neuroimaging data analysis: prediction of outcome, interpretation of internal representations, generation of synthetic data and segmentation. In the next section we present issues that deep learning poses, which concerns multidimensionality and multimodality of data, overfitting and computational cost, and propose possible solutions. Lastly, we discuss the current reach of DL usage in all the common applications in neuroimaging data analysis, where we consider the promise of multimodality, capability of processing raw data, and advanced visualization strategies. We identify research gaps, such as focusing on a limited number of criterion variables and the lack of a well-defined strategy for choosing architecture and hyperparameters. Furthermore, we talk about the possibility of conducting research with constructs that have been ignored so far or/and moving toward frameworks, such as RDoC, the potential of transfer learning and generation of synthetic data.

A hybrid learning framework for fine-grained interpretation of brain spatiotemporal patterns during naturalistic functional magnetic resonance imaging

Naturalistic stimuli, including movie, music, and speech, have been increasingly applied in the research of neuroimaging. Relative to a resting-state or single-task state, naturalistic stimuli can evoke more intense brain activities and have been proved to possess higher test–retest reliability, suggesting greater potential to study adaptive human brain function. In the current research, naturalistic functional magnetic resonance imaging (N-fMRI) has been a powerful tool to record brain states under naturalistic stimuli, and many efforts have been devoted to study the high-level semantic features from spatial or temporal representations via N-fMRI. However, integrating both spatial and temporal characteristics of brain activities for better interpreting the patterns under naturalistic stimuli is still underexplored. In this work, a novel hybrid learning framework that comprehensively investigates both the spatial (via Predictive Model) and the temporal [via convolutional neural network (CNN) model] characteristics of the brain is proposed. Specifically, to focus on certain relevant regions from the whole brain, regions of significance (ROS), which contain common spatial activation characteristics across individuals, are selected via the Predictive Model. Further, voxels of significance (VOS), whose signals contain significant temporal characteristics under naturalistic stimuli, are interpreted via one-dimensional CNN (1D-CNN) model. In this article, our proposed framework is applied onto the N-fMRI data during naturalistic classical/pop/speech audios stimuli. The promising performance is achieved via the Predictive Model to differentiate the different audio categories. Especially for distinguishing the classic and speech audios, the accuracy of classification is up to 92%. Moreover, spatial ROS and VOS are effectively obtained. Besides, temporal characteristics of the high-level semantic features are investigated on the frequency domain via convolution kernels of 1D-CNN model, and we effectively bridge the “semantic gap” between high-level semantic features of N-fMRI and low-level acoustic features of naturalistic audios in the frequency domain. Our results provide novel insights on characterizing spatiotemporal patterns of brain activities via N-fMRI and effectively explore the high-level semantic features under naturalistic stimuli, which will further benefit the understanding of the brain working mechanism and the advance of naturalistic stimuli clinical application.

Deep multimodal predictome for studying mental disorders

Characterizing neuropsychiatric disorders is challenging due to heterogeneity in the population. We propose combining structural and functional neuroimaging and genomic data in a multimodal classification framework to leverage their complementary information. Our objectives are two-fold (i) to improve the classification of disorders and (ii) to introspect the concepts learned to explore underlying neural and biological mechanisms linked to mental disorders. Previous multimodal studies have focused on naïve neural networks, mostly perceptron, to learn modality-wise features and often assume equal contribution from each modality. Our focus is on the development of neural networks for feature learning and implementing an adaptive control unit for the fusion phase. Our mid fusion with attention model includes a multilayer feed-forward network, an autoencoder, a bi-directional long short-term memory unit with attention as the features extractor, and a linear attention module for controlling modality-specific influence. The proposed model acquired 92% (p < .0001) accuracy in schizophrenia prediction, outperforming several other state-of-the-art models applied to unimodal or multimodal data. Post hoc feature analyses uncovered critical neural features and genes/biological pathways associated with schizophrenia. The proposed model effectively combines multimodal neuroimaging and genomics data for predicting mental disorders. Interpreting salient features identified by the model may advance our understanding of their underlying etiological mechanisms.

Impaired time-distance reconfiguration patterns in Alzheimer's disease: a dynamic functional connectivity study with 809 individuals from 7 sites

Background

The dynamic functional connectivity (dFC) has been used successfully to investigate the dysfunction of Alzheimer's disease (AD) patients. The reconfiguration intensity of nodal dFC, which means the degree of alteration between FCs at different time scales, could provide additional information for understanding the reconfiguration of brain connectivity.

Results

In this paper, we introduced a feature named time distance nodal connectivity diversity (tdNCD), and then evaluated the network reconfiguration intensity in every specific brain region in AD using a large multicenter dataset (N = 809 from 7 independent sites). Our results showed that the dysfunction involved in three subnetworks in AD, including the default mode network (DMN), the subcortical network (SCN), and the cerebellum network (CBN). The nodal tdNCD inside the DMN increased in AD compared to normal controls, and the nodal dynamic FC of the SCN and the CBN decreased in AD. Additionally, the classification analysis showed that the classification performance was better when combined tdNCD and FC to classify AD from normal control (ACC = 81%, SEN = 83.4%, SPE = 80.6%, and F1-score = 79.4%) than that only using FC (ACC = 78.2%, SEN = 76.2%, SPE = 76.5%, and F1-score = 77.5%) with a leave-one-site-out cross-validation. Besides, the performance of the three classes classification was improved from 50% (only using FC) to 53.3% (combined FC and tdNCD) (macro F1-score accuracy from 46.8 to 48.9%). More importantly, the classification model showed significant clinically predictive correlations (two classes classification: R = −0.38, P < 0.001; three classes classification: R = −0.404, P < 0.001). More importantly, several commonly used machine learning models confirmed that the tdNCD would provide additional information for classifying AD from normal controls.

Conclusions

The present study demonstrated dynamic reconfiguration of nodal FC abnormities in AD. The tdNCD highlights the potential for further understanding core mechanisms of brain dysfunction in AD. Evaluating the tdNCD FC provides a promising way to understand AD processes better and investigate novel diagnostic brain imaging biomarkers for AD.

Characterization of Early Stage Parkinson's Disease From Resting-State fMRI Data Using a Long Short-Term Memory Network

Parkinson's disease (PD) is a common and complex neurodegenerative disorder with five stages on the Hoehn and Yahr scaling. Characterizing brain function alterations with progression of early stage disease would support accurate disease staging, development of new therapies, and objective monitoring of disease progression or treatment response. Functional magnetic resonance imaging (fMRI) is a promising tool in revealing functional connectivity (FC) differences and developing biomarkers in PD. While fMRI and FC data have been utilized for diagnosis of PD through application of machine learning approaches such as support vector machine and logistic regression, the characterization of FC changes in early-stage PD has not been investigated. Given the complexity and non-linearity of fMRI data, we propose the use of a long short-term memory (LSTM) network to distinguish the early stages of PD and understand related functional brain changes. The study included 84 subjects (56 in stage 2 and 28 in stage 1) from the Parkinson's Progression Markers Initiative (PPMI), the largest-available public PD dataset. Under a repeated 10-fold stratified cross-validation, the LSTM model reached an accuracy of 71.63%, 13.52% higher than the best traditional machine learning method and 11.56% higher than a CNN model, indicating significantly better robustness and accuracy compared with other machine learning classifiers. Finally, we used the learned LSTM model weights to select the top brain regions that contributed to model prediction and performed FC analyses to characterize functional changes with disease stage and motor impairment to gain better insight into the brain mechanisms of PD.

SSPNet: An interpretable 3D-CNN for classification of schizophrenia using phase maps of resting-state complex-valued fMRI data

Convolutional neural networks (CNNs) have shown promising results in classifying individuals with mental disorders such as schizophrenia using resting-state fMRI data. However, complex-valued fMRI data is rarely used since additional phase data introduces high-level noise though it is potentially useful information for the context of classification. As such, we propose to use spatial source phase (SSP) maps derived from complex-valued fMRI data as the CNN input. The SSP maps are not only less noisy, but also more sensitive to spatial activation changes caused by mental disorders than magnitude maps. We build a 3D-CNN framework with two convolutional layers (named SSPNet) to fully explore the 3D structure and voxel-level relationships from the SSP maps. Two interpretability modules, consisting of saliency map generation and gradient-weighted class activation mapping (Grad-CAM), are incorporated into the well-trained SSPNet to provide additional information helpful for understanding the output. Experimental results from classifying schizophrenia patients (SZs) and healthy controls (HCs) show that the proposed SSPNet significantly improved accuracy and AUC compared to CNN using magnitude maps extracted from either magnitude-only (by 23.4 and 23.6% for DMN) or complex-valued fMRI data (by 10.6 and 5.8% for DMN). SSPNet captured more prominent HC-SZ differences in saliency maps, and Grad-CAM localized all contributing brain regions with opposite strengths for HCs and SZs within SSP maps. These results indicate the potential of SSPNet as a sensitive tool that may be useful for the development of brain-based biomarkers of mental disorders.

Prediction of Obsessive-Compulsive Disorder: Importance of Neurobiology-Aided Feature Design and Cross-Diagnosis Transfer Learning

BACKGROUND

Machine learning applications using neuroimaging provide a multidimensional, data-driven approach that captures the level of complexity necessary for objectively aiding diagnosis and prognosis in psychiatry. However, models learned from small training samples often have limited generalizability, which continues to be a problem with automated diagnosis of mental illnesses such as obsessive-compulsive disorder (OCD). Earlier studies have shown that features incorporating prior neurobiological knowledge of brain function and combining brain parcellations from various sources can potentially improve the overall prediction. However, it is unknown whether such knowledge-driven methods can provide a performance that is comparable to state-of-the-art approaches based on neural networks.

METHODS

In this study, we apply a transparent and explainable multiparcellation ensemble learning framework EMPaSchiz (Ensemble algorithm with Multiple Parcellations for Schizophrenia prediction) to the task of predicting OCD, based on a resting-state functional magnetic resonance imaging dataset of 350 subjects. Furthermore, we apply transfer learning using the features found effective for schizophrenia to OCD to leverage the commonality in brain alterations across these psychiatric diagnoses.

RESULTS

We show that our knowledge-based approach leads to a prediction performance of 80.3% accuracy for OCD diagnosis that is better than domain-agnostic and automated feature design using neural networks. Furthermore, we show that a selection of reduced feature sets can be transferred from schizophrenia to the OCD prediction model without significant loss in prediction performance.

CONCLUSIONS

This study presents a machine learning framework for OCD prediction with neurobiology-aided feature design using resting-state functional magnetic resonance imaging that is generalizable and reasonably interpretable.

A 5D approach to study spatio-temporal dynamism of resting-state brain networks in schizophrenia

Schizophrenia is a serious brain disorder that can affect all aspects of patient's life such as thinking, behaving and even feeling. The principal cause of schizophrenia is still unknown, but there is some evidence that differences in brain networks interactions along with functional dysconnectivity may play a significant role. Prior work has mostly focused on static summaries of functional data, or more recently changes in temporal coupling between fixed networks. Here, we study differences in spatio-temporal brain dynamics using resting state fMRI images in a dataset including 510 control and 708 schizophrenia patients. To do this, we utilized a deep residual network to extract 5 different spatiotemporal networks each of which captures spatial and temporal dynamics within sensory-motor, auditory, and default mode domains. Our analysis shows significant group differences in various aspects of spatio-temporal dynamics including magnitude, voxel-wise variability, and temporal functional network connectivity. Clinical relevance- Our study explores effects of spatio-temporal brain dynamism in schizophrenia, which is rarely taken into account, but could provide unique and more sensitive information about the disorder. Here we incorporate a novel 5D brain parcellation model, that enables us to encode spatio-temporal dynamics, to extract and characterize multiple resting fMRI brain networks.

Going deep into schizophrenia with artificial intelligence

Despite years of research, the mechanisms governing the onset, relapse, symptomatology, and treatment of schizophrenia (SZ) remain elusive. The lack of appropriate analytic tools to deal with the heterogeneity and complexity of SZ may be one of the reasons behind this situation. Deep learning, a subfield of artificial intelligence (AI) inspired by the nervous system, has recently provided an accessible way of modeling and analyzing complex, high-dimensional, nonlinear systems. The unprecedented accuracy of deep learning algorithms in classification and prediction tasks has revolutionized a wide range of scientific fields and is rapidly permeating SZ research. Deep learning has the potential of becoming a valuable aid for clinicians in the prediction, diagnosis, and treatment of SZ, especially in combination with principles from Bayesian statistics. Furthermore, deep learning could become a powerful tool for uncovering the mechanisms underlying SZ thanks to a growing number of techniques designed for improving model interpretability and causal reasoning. The purpose of this article is to introduce SZ researchers to the field of deep learning and review its latest applications in SZ research. In general, existing studies have yielded impressive results in classification and outcome prediction tasks. However, methodological concerns related to the assessment of model performance in several studies, the widespread use of small training datasets, and the little clinical value of some models suggest that some of these results should be taken with caution.

Mapping relationships among schizophrenia, bipolar and schizoaffective disorders: A deep classification and clustering framework using fMRI time series

BACKGROUND

Psychiatric disorders are categorized using self-report and observational information rather than biological data. There is also considerable symptomatic overlap between different types of psychiatric disorders, which makes diagnostic categorization and multi-class classification challenging.

METHODS

In this work, we propose a unified framework for supervised classification and unsupervised clustering of psychotic disorders using brain imaging data. A new multi-scale recurrent neural network (MsRNN) model was developed and applied to fMRI time courses (TCs) for multi-class classification. The high-level representations of the original TCs were then submitted to a tSNE clustering model for visualizing the group differences between disorders. A leave-one-feature-out approach was used for disorder-related biomarker identification.

RESULTS

When studying fMRI from schizophrenia, psychotic bipolar disorder, schizoaffective disorder, and healthy individuals, the accuracy of a 4-class classification reached 46%, significantly above chance. The hippocampus, supplementary motor area and paracentral lobule were discovered as the most contributing regional TCs in the multi-class classification. Beyond this, visualization of the tSNE clustering suggested that the disease severity can be captured and schizoaffective disorder (SAD) may be separated into two subtypes. SAD cluster1 has significantly higher Positive And Negative Syndrome Scale (PANSS) scores than SAD cluster2 in PANSS negative2 (emotional withdrawal), general2 (anxiety), general3 (guilt feelings), general4 (tension).

CONCLUSIONS

The proposed deep classification and clustering framework is not only able to identify psychiatric disorders with high accuracy, but also interpret the correlation between brain networks and specific psychiatric disorders, and reveal the relationship between them. This work provides a promising way to investigate a spectrum of similar disorders using neuroimaging-based measures.

DC-tCNN: A Deep Model for EEG-based Detection of Dim Targets

OBJECTIVE

Dim target detection in remote sensing images is a significant and challenging problem. In this work, we seek to explore event-related brain responses of dim target detection tasks and extend the brain-computer interface (BCI) systems to this task for efficiency enhancement.

METHODS

We develop a BCI paradigm named Asynchronous Visual Evoked Paradigm (AVEP), in which subjects are required to search the dim targets within satellite images when their scalp electroencephalography (EEG) signals are simultaneously recorded. In the paradigm, stimulus onset time and target onset time are asynchronous because subjects need enough time to confirm whether there are targets of interest in the presented serial images. We further propose a Domain adaptive and Channel-wise attention-based Time-domain Convolutional Neural Network (DC-tCNN) to solve the single-trial EEG classification problem for the AVEP task. In this model, we design a multi-scale CNN module combined with a channel-wise attention module to effectively extract event-related brain responses underlying EEG signals. Meanwhile, domain adaptation is proposed to mitigate cross-subject distribution discrepancy.

RESULTS

The results demonstrate the superior performance and better generalizability of this model in classifying the single-trial EEG data of AVEP task in contrast to typical EEG deep learning networks. Visualization analyses of spatiotemporal features also illustrate the effectiveness and interpretability of our proposed paradigm and learning model.

CONCLUSION

The proposed paradigm and model can effectively explore ambiguous event-related brain responses on EEG-based dim target detection tasks.

SIGNIFICANCE

Our work can provide a valuable reference for BCI-based image detection of dim targets.

Feature and decision-level fusion for schizophrenia detection based on resting-state fMRI data

Mental disorders, especially schizophrenia, still pose a great challenge for diagnosis in early stages. Recently, computer-aided diagnosis techniques based on resting-state functional magnetic resonance imaging (Rs-fMRI) have been developed to tackle this challenge. In this work, we investigate different decision-level and feature-level fusion schemes for discriminating between schizophrenic and normal subjects. Four types of fMRI features are investigated, namely the regional homogeneity, voxel-mirrored homotopic connectivity, fractional amplitude of low-frequency fluctuations and amplitude of low-frequency fluctuations. Data denoising and preprocessing were first applied, followed by the feature extraction module. Four different feature selection algorithms were applied, and the best discriminative features were selected using the algorithm of feature selection via concave minimization (FSV). Support vector machine classifiers were trained and tested on the COBRE dataset formed of 70 schizophrenic subjects and 70 healthy subjects. The decision-level fusion method outperformed the single-feature-type approaches and achieved a 97.85% accuracy, a 98.33% sensitivity, a 96.83% specificity. Moreover, feature-fusion scheme resulted in a 98.57% accuracy, a 99.71% sensitivity, a 97.66% specificity, and an area under the ROC curve of 0.9984. In general, decision-level and feature-level fusion schemes boosted the performance of schizophrenia detectors based on fMRI features.

An attention-based hybrid deep learning framework integrating brain connectivity and activity of resting-state functional MRI data

Functional magnetic resonance imaging (fMRI) as a promising tool to investigate psychotic disorders can be decomposed into useful imaging features such as time courses (TCs) of independent components (ICs) and functional network connectivity (FNC) calculated by TC cross-correlation. TCs reflect the temporal dynamics of brain activity and the FNC characterizes temporal coherence across intrinsic brain networks. Both features have been used as input to deep learning approaches with decent results. However, few studies have tried to leverage their complementary information to learn optimal representations at multiple facets. Motivated by this, we proposed a Hybrid Deep Learning Framework integrating brain Connectivity and Activity (HDLFCA) together by combining convolutional recurrent neural network (C-RNN) and deep neural network (DNN), aiming to improve classification accuracy and interpretability simultaneously. Specifically, C-RNNAM was proposed to extract temporal dynamic dependencies with an attention module (AM) to automatically learn discriminative knowledge from TC nodes, while DNN was applied to identify the most group-discriminative FNC patterns with layer-wise relevance propagation (LRP). Then, both prediction outputs were concatenated to build a new feature matrix, generating the final decision by logistic regression. The effectiveness of HDLFCA was validated on both multi-site schizophrenia (SZ, n ∼ 1100) and public autism datasets (ABIDE, n ∼ 1522) by outperforming 12 alternative models at 2.8-8.9% accuracy, including 8 models using either static FNC or TCs and 4 models using dynamic FNC. Appreciable classification accuracy was achieved for HC vs. SZ (85.3%) and HC vs. Autism (72.4%) respectively. More importantly, the most group-discriminative brain regions can be easily attributed and visualized, providing meaningful biological interpretability and highlighting the great potential of the proposed HDLFCA model in the identification of valid neuroimaging biomarkers.

Cross-ethnicity/race generalization failure of behavioral prediction from resting-state functional connectivity

Algorithmic biases that favor majority populations pose a key challenge to the application of machine learning for precision medicine. Here, we assessed such bias in prediction models of behavioral phenotypes from brain functional magnetic resonance imaging. We examined the prediction bias using two independent datasets (preadolescent versus adult) of mixed ethnic/racial composition. When predictive models were trained on data dominated by white Americans (WA), out-of-sample prediction errors were generally higher for African Americans (AA) than for WA. This bias toward WA corresponds to more WA-like brain-behavior association patterns learned by the models. When models were trained on AA only, compared to training only on WA or an equal number of AA and WA participants, AA prediction accuracy improved but stayed below that for WA. Overall, the results point to the need for caution and further research regarding the application of current brain-behavior prediction models in minority populations.

Deep learning model using retinal vascular images for classifying schizophrenia

Contemporary psychiatric diagnosis still relies on the subjective symptom report of the patient during a clinical interview by a psychiatrist. Given the significant variability in personal reporting and differences in the skill set of psychiatrists, it is desirable to have objective diagnostic markers that could help clinicians differentiate patients from healthy individuals. A few recent studies have reported retinal vascular abnormalities in patients with schizophrenia (SCZ) using retinal fundus images. The goal of this study was to use a trained convolution neural network (CNN) deep learning algorithm to detect SCZ using retinal fundus images. A total of 327 subjects [139 patients with Schizophrenia (SCZ) and 188 Healthy volunteers (HV)] were recruited, and retinal images were acquired using a fundus camera. The images were preprocessed and fed to a convolution neural network for the classification. The model performance was evaluated using the area under the receiver operating characteristic curve (AUC). The CNN achieved an accuracy of 95% for classifying SCZ and HV with an AUC of 0.98. Findings from the current study suggest the potential utility of deep learning to classify patients with SCZ and assist clinicians in clinical settings. Future studies need to examine the utility of the deep learning model with retinal vascular images as biomarkers in schizophrenia with larger sample sizes.

Classification of amyotrophic lateral sclerosis by brain volume, connectivity, and network dynamics

Emerging studies corroborate the importance of neuroimaging biomarkers and machine learning to improve diagnostic classification of amyotrophic lateral sclerosis (ALS). While most studies focus on structural data, recent studies assessing functional connectivity between brain regions by linear methods highlight the role of brain function. These studies have yet to be combined with brain structure and nonlinear functional features. We investigate the role of linear and nonlinear functional brain features, and the benefit of combining brain structure and function for ALS classification. ALS patients (N = 97) and healthy controls (N = 59) underwent structural and functional resting state magnetic resonance imaging. Based on key hubs of resting state networks, we defined three feature sets comprising brain volume, resting state functional connectivity (rsFC), as well as (nonlinear) resting state dynamics assessed via recurrent neural networks. Unimodal and multimodal random forest classifiers were built to classify ALS. Out-of-sample prediction errors were assessed via five-fold cross-validation. Unimodal classifiers achieved a classification accuracy of 56.35-61.66%. Multimodal classifiers outperformed unimodal classifiers achieving accuracies of 62.85-66.82%. Evaluating the ranking of individual features' importance scores across all classifiers revealed that rsFC features were most dominant in classification. While univariate analyses revealed reduced rsFC in ALS patients, functional features more generally indicated deficits in information integration across resting state brain networks in ALS. The present work undermines that combining brain structure and function provides an additional benefit to diagnostic classification, as indicated by multimodal classifiers, while emphasizing the importance of capturing both linear and nonlinear functional brain properties to identify discriminative biomarkers of ALS.

A dynamic graph convolutional neural network framework reveals new insights into connectome dysfunctions in ADHD

The pathological mechanism of attention deficit hyperactivity disorder (ADHD) is incompletely specified, which leads to difficulty in precise diagnosis. Functional magnetic resonance imaging (fMRI) has emerged as a common neuroimaging technique for studying the brain functional connectome. Most existing methods that have either ignored or simply utilized graph structure, do not fully leverage the potentially important topological information which may be useful in characterizing brain disorders. There is a crucial need for designing novel and efficient approaches which can capture such information. To this end, we propose a new dynamic graph convolutional network (dGCN), which is trained with sparse brain regional connections from dynamically calculated graph features. We also develop a novel convolutional readout layer to improve graph representation. Our extensive experimental analysis demonstrates significantly improved performance of dGCN for ADHD diagnosis compared with existing machine learning and deep learning methods. Visualizations of the salient regions of interest (ROIs) and connectivity based on informative features learned by our model show that the identified functional abnormalities mainly involve brain regions in temporal pole, gyrus rectus, and cerebellar gyri from temporal lobe, frontal lobe, and cerebellum, respectively. A positive correlation was further observed between the identified connectomic abnormalities and ADHD symptom severity. The proposed dGCN model shows great promise in providing a functional network-based precision diagnosis of ADHD and is also broadly applicable to brain connectome-based study of mental disorders.