Designing weighted correlation kernels in convolutional neural networks for functional connectivity based brain disease diagnosis

Explainability of three-dimensional convolutional neural networks for functional magnetic resonance imaging of Alzheimer's disease classification based on gradient-weighted class activation mapping

Currently, numerous studies focus on employing fMRI-based deep neural networks to diagnose neurological disorders such as Alzheimer's Disease (AD), yet only a handful have provided results regarding explainability. We address this gap by applying several prevalent explainability methods such as gradient-weighted class activation mapping (Grad-CAM) to an fMRI-based 3D-VGG16 network for AD diagnosis to improve the model's explainability. The aim is to explore the specific Region of Interest (ROI) of brain the model primarily focuses on when making predictions, as well as whether there are differences in these ROIs between AD and normal controls (NCs). First, we utilized multiple resting-state functional activity maps including ALFF, fALFF, ReHo, and VMHC to reduce the complexity of fMRI data, which differed from many studies that utilized raw fMRI data. Compared to methods utilizing raw fMRI data, this manual feature extraction approach may potentially alleviate the model's burden. Subsequently, 3D-VGG16 were employed for AD classification, where the final fully connected layers were replaced with a Global Average Pooling (GAP) layer, aimed at mitigating overfitting while preserving spatial information within the feature maps. The model achieved a maximum of 96.4% accuracy on the test set. Finally, several 3D CAM methods were employed to interpret the models. In the explainability results of the models with relatively high accuracy, the highlighted ROIs were primarily located in the precuneus and the hippocampus for AD subjects, while the models focused on the entire brain for NC. This supports current research on ROIs involved in AD. We believe that explaining deep learning models would not only provide support for existing research on brain disorders, but also offer important referential recommendations for the study of currently unknown etiologies.

Dysfunctions of multiscale dynamic brain functional networks in subjective cognitive decline

Subjective cognitive decline is potentially the earliest symptom of Alzheimer's disease, whose objective neurological basis remains elusive. To explore the potential biomarkers for subjective cognitive decline, we developed a novel deep learning method based on multiscale dynamical brain functional networks to identify subjective cognitive declines. We retrospectively constructed an internal data set (with 112 subjective cognitive decline and 64 healthy control subjects) to develop and internally validate the deep learning model. Conventional deep learning methods based on static and dynamic brain functional networks are compared. After the model is established, we prospectively collect an external data set (26 subjective cognitive decline and 12 healthy control subjects) for testing. Meanwhile, our method provides monitoring of the transitions between normal and abnormal (subjective cognitive decline-related) dynamical functional network states. The features of abnormal dynamical functional network states are quantified by network and variability metrics and associated with individual cognitions. Our method achieves an area under the receiver operating characteristic curve of 0.807 ± 0.046 in the internal validation data set and of 0.707 (P = 0.007) in the external testing data set, which shows improvements compared to conventional methods. The method further suggests that, at the local level, the abnormal dynamical functional network states are characterized by decreased connectivity strength and increased connectivity variability at different spatial scales. At the network level, the abnormal states are featured by scale-specifically altered modularity and all-scale decreased efficiency. Low tendencies to stay in abnormal states and high state transition variabilities are significantly associated with high general, language and executive functions. Overall, our work supports the deficits in multiscale brain dynamical functional networks detected by the deep learning method as reliable and meaningful neural alternation underpinning subjective cognitive decline.

Diagnosis of autism spectrum disorder based on functional brain networks and machine learning

Autism is a multifaceted neurodevelopmental condition whose accurate diagnosis may be challenging because the associated symptoms and severity vary considerably. The wrong diagnosis can affect families and the educational system, raising the risk of depression, eating disorders, and self-harm. Recently, many works have proposed new methods for the diagnosis of autism based on machine learning and brain data. However, these works focus on only one pairwise statistical metric, ignoring the brain network organization. In this paper, we propose a method for the automatic diagnosis of autism based on functional brain imaging data recorded from 500 subjects, where 242 present autism spectrum disorder considering the regions of interest throughout Bootstrap Analysis of Stable Cluster map. Our method can distinguish the control group from autism spectrum disorder patients with high accuracy. Indeed the best performance provides an AUC near 1.0, which is higher than that found in the literature. We verify that the left ventral posterior cingulate cortex region is less connected to an area in the cerebellum of patients with this neurodevelopment disorder, which agrees with previous studies. The functional brain networks of autism spectrum disorder patients show more segregation, less distribution of information across the network, and less connectivity compared to the control cases. Our workflow provides medical interpretability and can be used on other fMRI and EEG data, including small data sets.

Multicenter and Multichannel Pooling GCN for Early AD Diagnosis Based on Dual-Modality Fused Brain Network

For significant memory concern (SMC) and mild cognitive impairment (MCI), their classification performance is limited by confounding features, diverse imaging protocols, and limited sample size. To address the above limitations, we introduce a dual-modality fused brain connectivity network combining resting-state functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI), and propose three mechanisms in the current graph convolutional network (GCN) to improve classifier performance. First, we introduce a DTI-strength penalty term for constructing functional connectivity networks. Stronger structural connectivity and bigger structural strength diversity between groups provide a higher opportunity for retaining connectivity information. Second, a multi-center attention graph with each node representing a subject is proposed to consider the influence of data source, gender, acquisition equipment, and disease status of those training samples in GCN. The attention mechanism captures their different impacts on edge weights. Third, we propose a multi-channel mechanism to improve filter performance, assigning different filters to features based on feature statistics. Applying those nodes with low-quality features to perform convolution would also deteriorate filter performance. Therefore, we further propose a pooling mechanism, which introduces the disease status information of those training samples to evaluate the quality of nodes. Finally, we obtain the final classification results by inputting the multi-center attention graph into the multi-channel pooling GCN. The proposed method is tested on three datasets (i.e., an ADNI 2 dataset, an ADNI 3 dataset, and an in-house dataset). Experimental results indicate that the proposed method is effective and superior to other related algorithms, with a mean classification accuracy of 93.05% in our binary classification tasks. Our code is available at: https://github.com/Xuegang-S.

Unsupervised cross-domain functional MRI adaptation for automated major depressive disorder identification

Resting-state functional magnetic resonance imaging (rs-fMRI) data have been widely used for automated diagnosis of brain disorders such as major depressive disorder (MDD) to assist in timely intervention. Multi-site fMRI data have been increasingly employed to augment sample size and improve statistical power for investigating MDD. However, previous studies usually suffer from significant inter-site heterogeneity caused for instance by differences in scanners and/or scanning protocols. To address this issue, we develop a novel discrepancy-based unsupervised cross-domain fMRI adaptation framework (called UFA-Net) for automated MDD identification. The proposed UFA-Net is designed to model spatio-temporal fMRI patterns of labeled source and unlabeled target samples via an attention-guided graph convolution module, and also leverage a maximum mean discrepancy constrained module for unsupervised cross-site feature alignment between two domains. To the best of our knowledge, this is one of the first attempts to explore unsupervised rs-fMRI adaptation for cross-site MDD identification. Extensive evaluation on 681 subjects from two imaging sites shows that the proposed method outperforms several state-of-the-art methods. Our method helps localize disease-associated functional connectivity abnormalities and is therefore well interpretable and can facilitate fMRI-based analysis of MDD in clinical practice.

Multi-tasking deep network for tinnitus classification and severity prediction from multimodal structural MR images

Objective:Subjective tinnitus is an auditory phantom perceptual disorder without an objective biomarker. Fast and efficient diagnostic tools will advance clinical practice by detecting or confirming the condition, tracking change in severity, and monitoring treatment response. Motivated by evidence of subtle anatomical, morphological, or functional information in magnetic resonance images of the brain, we examine data-driven machine learning methods for joint tinnitus classification (tinnitus or no tinnitus) and tinnitus severity prediction.Approach:We propose a deep multi-task multimodal framework for tinnitus classification and severity prediction using structural MRI (sMRI) data. To leverage complementary information multimodal neuroimaging data, we integrate two modalities of three-dimensional sMRI-T1 weighted (T1w) and T2 weighted (T2w) images. To explore the key components in the MR images that drove task performance, we segment both T1w and T2w images into three different components-cerebrospinal fluid, grey matter and white matter, and evaluate performance of each segmented image.Main results:Results demonstrate that our multimodal framework capitalizes on the information across both modalities (T1w and T2w) for the joint task of tinnitus classification and severity prediction.Significance:Our model outperforms existing learning-based and conventional methods in terms of accuracy, sensitivity, specificity, and negative predictive value.

A cell phone app for facial acne severity assessment

Acne vulgaris, the most common skin disease, can cause substantial economic and psychological impacts to the people it affects, and its accurate grading plays a crucial role in the treatment of patients. In this paper, we firstly proposed an acne grading criterion that considers lesion classifications and a metric for producing accurate severity ratings. Due to similar appearance of acne lesions with comparable severities and difficult-to-count lesions, severity assessment is a challenging task. We cropped facial skin images of several lesion patches and then addressed the acne lesion with a lightweight acne regular network (Acne-RegNet). Acne-RegNet was built by using a median filter and histogram equalization to improve image quality, a channel attention mechanism to boost the representational power of network, a region-based focal loss to handle classification imbalances and a model pruning and feature-based knowledge distillation to reduce model size. After the application of Acne-RegNet, the severity score is calculated, and the acne grading is further optimized by the metadata of the patients. The entire acne assessment procedure was deployed to a mobile device, and a phone app was designed. Compared with state-of-the-art lightweight models, the proposed Acne-RegNet significantly improves the accuracy of lesion classifications. The acne app demonstrated promising results in severity assessments (accuracy: 94.56%) and showed a dermatologist-level diagnosis on the internal clinical dataset.The proposed acne app could be a useful adjunct to assess acne severity in clinical practice and it enables anyone with a smartphone to immediately assess acne, anywhere and anytime.

SD-CNN: A static-dynamic convolutional neural network for functional brain networks

Static functional connections (sFCs) and dynamic functional connections (dFCs) have been widely used in the resting-state functional MRI (rs-fMRI) analysis. sFCs, calculated based on entire rs-fMRI scans, can accurately describe the static topology of the brain network. dFCs, estimated by dividing rs-fMRI scans into a series of short sliding windows, are used to reveal time-varying changes in FC patterns. Currently, how to jointly use sFCs and dFCs to identify brain diseases under the framework of deep learning is still a hot issue. To this end, we propose a static-dynamic convolutional neural network for functional brain networks, which involves a static pathway and a dynamic pathway for taking full advantages of sFCs and dFCs. Specifically, the static pathway, using high-resolution convolution filters (i.e., convolution filters with a high number of channels) at a single adjacency matrix of sFCs, is performed to capture static FC patterns. The dynamic pathway, using low-resolution convolution filters at each adjacency matrix of dFCs, is performed to capture time-varying FC patterns. Two types of diffusion connections are used in this model for encouraging the transfer of information between the static pathway and the dynamic pathway, which can make the learned features more discriminative. Furthermore, a static and dynamic combination classifier is introduced to combine features from two pathways for identifying brain diseases. Experiments on two real datasets demonstrate the effectiveness and advantages of our proposed method.

Functional magnetic resonance imaging, deep learning, and Alzheimer's disease: A systematic review

Alzheimer's disease (AD) is currently diagnosed using a mixture of psychological tests and clinical observations. However, these diagnoses are not perfect, and additional diagnostic tools (e.g., MRI) can help improve our understanding of AD as well as our ability to detect the disease. Accordingly, a large amount of research has been invested into innovative diagnostic methods for AD. Functional MRI (fMRI) is a form of neuroimaging technology that has been used to diagnose AD; however, fMRI is incredibly noisy, complex, and thus lacks clinical use. Nonetheless, recent innovations in deep learning technology could enable the simplified and streamlined analysis of fMRI. Deep learning is a form of artificial intelligence that uses computer algorithms based on human neural networks to solve complex problems. For example, in fMRI research, deep learning models can automatically denoise images and classify AD by detecting patterns in participants' brain scans. In this systematic review, we investigate how fMRI (specifically resting-state fMRI) and deep learning methods are used to diagnose AD. In turn, we outline the common deep neural network, preprocessing, and classification methods used in the literature. We also discuss the accuracy, strengths, limitations, and future direction of fMRI deep learning methods. In turn, we aim to summarize the current field for new researchers, suggest specific areas for future research, and highlight the potential of fMRI to aid AD diagnoses.

Brain disorder prediction with dynamic multivariate spatio-temporal features: Application to Alzheimer’s disease and autism spectrum disorder

The dynamic functional connectivity (dFC) in functional magnetic resonance imaging (fMRI) is beneficial for the analysis and diagnosis of neurological brain diseases. The dFCs between regions of interest (ROIs) are generally delineated by a specific template and clustered into multiple different states. However, these models inevitably fell into the model-driven self-contained system which ignored the diversity at spatial level and the dynamics at time level of the data. In this study, we proposed a spatial and time domain feature extraction approach for Alzheimer’s disease (AD) and autism spectrum disorder (ASD)-assisted diagnosis which exploited the dynamic connectivity among independent functional sub networks in brain. Briefly, independent sub networks were obtained by applying spatial independent component analysis (SICA) to the preprocessed fMRI data. Then, a sliding window approach was used to segment the time series of the spatial components. After that, the functional connections within the window were obtained sequentially. Finally, a temporal signal-sensitive long short-term memory (LSTM) network was used for classification. The experimental results on Alzheimer’s Disease Neuroimaging Initiative (ADNI) and Autism Brain Imaging Data Exchange (ABIDE) datasets showed that the proposed method effectively predicted the disease at the early stage and outperformed the existing algorithms. The dFCs between the different components of the brain could be used as biomarkers for the diagnosis of diseases such as AD and ASD, providing a reliable basis for the study of brain connectomics.

Convolutional Recurrent Neural Network for Dynamic Functional MRI Analysis and Brain Disease Identification

Dynamic functional connectivity (dFC) networks derived from resting-state functional magnetic resonance imaging (rs-fMRI) help us understand fundamental dynamic characteristics of human brains, thereby providing an efficient solution for automated identification of brain diseases, such as Alzheimer's disease (AD) and its prodromal stage. Existing studies have applied deep learning methods to dFC network analysis and achieved good performance compared with traditional machine learning methods. However, they seldom take advantage of sequential information conveyed in dFC networks that could be informative to improve the diagnosis performance. In this paper, we propose a convolutional recurrent neural network (CRNN) for automated brain disease classification with rs-fMRI data. Specifically, we first construct dFC networks from rs-fMRI data using a sliding window strategy. Then, we employ three convolutional layers and long short-term memory (LSTM) layer to extract high-level features of dFC networks and also preserve the sequential information of extracted features, followed by three fully connected layers for brain disease classification. Experimental results on 174 subjects with 563 rs-fMRI scans from the Alzheimer's Disease Neuroimaging Initiative (ADNI) demonstrate the effectiveness of our proposed method in binary and multi-category classification tasks.

Convolutional Neural Networks for Classification of T2DM Cognitive Impairment Based on Whole Brain Structural Features

Purpose

Cognitive impairment is generally found in individuals with type 2 diabetes mellitus (T2DM). Although they may not have visible symptoms of cognitive impairment in the early stages of the disorder, they are considered to be at high risk. Therefore, the classification of these patients is important for preventing the progression of cognitive impairment.

Methods

In this study, a convolutional neural network was used to construct a model for classifying 107 T2DM patients with and without cognitive impairment based on T1-weighted structural MRI. The Montreal cognitive assessment score served as an index of the cognitive status of the patients.

Results

The classifier could identify T2DM-related cognitive decline with a classification accuracy of 84.85% and achieved an area under the curve of 92.65%.

Conclusions

The model can help clinicians analyze and predict cognitive impairment in patients and enable early treatment.

A deep graph neural network architecture for modelling spatio-temporal dynamics in resting-state functional MRI data

Resting-state functional magnetic resonance imaging (rs-fMRI) has been successfully employed to understand the organisation of the human brain. Typically, the brain is parcellated into regions of interest (ROIs) and modelled as a graph where each ROI represents a node and association measures between ROI-specific blood-oxygen-level-dependent (BOLD) time series are edges. Recently, graph neural networks (GNNs) have seen a surge in popularity due to their success in modelling unstructured relational data. The latest developments with GNNs, however, have not yet been fully exploited for the analysis of rs-fMRI data, particularly with regards to its spatio-temporal dynamics. In this paper, we present a novel deep neural network architecture which combines both GNNs and temporal convolutional networks (TCNs) in order to learn from both the spatial and temporal components of rs-fMRI data in an end-to-end fashion. In particular, this corresponds to intra-feature learning (i.e., learning temporal dynamics with TCNs) as well as inter-feature learning (i.e., leveraging interactions between ROI-wise dynamics with GNNs). We evaluate our model with an ablation study using 35,159 samples from the UK Biobank rs-fMRI database, as well as in the smaller Human Connectome Project (HCP) dataset, both in a unimodal and in a multimodal fashion. We also demonstrate that out architecture contains explainability-related features which easily map to realistic neurobiological insights. We suggest that this model could lay the groundwork for future deep learning architectures focused on leveraging the inherently and inextricably spatio-temporal nature of rs-fMRI data.

BrainGNN: Interpretable Brain Graph Neural Network for fMRI Analysis

Understanding which brain regions are related to a specific neurological disorder or cognitive stimuli has been an important area of neuroimaging research. We propose BrainGNN, a graph neural network (GNN) framework to analyze functional magnetic resonance images (fMRI) and discover neurological biomarkers. Considering the special property of brain graphs, we design novel ROI-aware graph convolutional (Ra-GConv) layers that leverage the topological and functional information of fMRI. Motivated by the need for transparency in medical image analysis, our BrainGNN contains ROI-selection pooling layers (R-pool) that highlight salient ROIs (nodes in the graph), so that we can infer which ROIs are important for prediction. Furthermore, we propose regularization terms-unit loss, topK pooling (TPK) loss and group-level consistency (GLC) loss-on pooling results to encourage reasonable ROI-selection and provide flexibility to encourage either fully individual- or patterns that agree with group-level data. We apply the BrainGNN framework on two independent fMRI datasets: an Autism Spectrum Disorder (ASD) fMRI dataset and data from the Human Connectome Project (HCP) 900 Subject Release. We investigate different choices of the hyper-parameters and show that BrainGNN outperforms the alternative fMRI image analysis methods in terms of four different evaluation metrics. The obtained community clustering and salient ROI detection results show a high correspondence with the previous neuroimaging-derived evidence of biomarkers for ASD and specific task states decoded for HCP. Our code is available at https://github.com/xxlya/BrainGNN_Pytorch.

Modularity-Guided Functional Brain Network Analysis for Early-Stage Dementia Identification

Function brain network (FBN) analysis has shown great potential in identifying brain diseases, such as Alzheimer's disease (AD) and its prodromal stage, namely mild cognitive impairment (MCI). It is essential to identify discriminative and interpretable features from function brain networks, so as to improve classification performance and help us understand the pathological mechanism of AD-related brain disorders. Previous studies usually extract node statistics or edge weights from FBNs to represent each subject. However, these methods generally ignore the topological structure (such as modularity) of FBNs. To address this issue, we propose a modular-LASSO feature selection (MLFS) framework that can explicitly model the modularity information to identify discriminative and interpretable features from FBNs for automated AD/MCI classification. Specifically, the proposed MLFS method first searches the modular structure of FBNs through a signed spectral clustering algorithm, and then selects discriminative features via a modularity-induced group LASSO method, followed by a support vector machine (SVM) for classification. To evaluate the effectiveness of the proposed method, extensive experiments are performed on 563 resting-state functional MRI scans from the public ADNI database to identify subjects with AD/MCI from normal controls and predict the future progress of MCI subjects. Experimental results demonstrate that our method is superior to previous methods in both tasks of AD/MCI identification and MCI conversion prediction, and also helps discover discriminative brain regions and functional connectivities associated with AD.

Towards evaluating the robustness of deep diagnostic models by adversarial attack

Deep learning models (with neural networks) have been widely used in challenging tasks such as computer-aided disease diagnosis based on medical images. Recent studies have shown deep diagnostic models may not be robust in the inference process and may pose severe security concerns in clinical practice. Among all the factors that make the model not robust, the most serious one is adversarial examples. The so-called "adversarial example" is a well-designed perturbation that is not easily perceived by humans but results in a false output of deep diagnostic models with high confidence. In this paper, we evaluate the robustness of deep diagnostic models by adversarial attack. Specifically, we have performed two types of adversarial attacks to three deep diagnostic models in both single-label and multi-label classification tasks, and found that these models are not reliable when attacked by adversarial example. We have further explored how adversarial examples attack the models, by analyzing their quantitative classification results, intermediate features, discriminability of features and correlation of estimated labels for both original/clean images and those adversarial ones. We have also designed two new defense methods to handle adversarial examples in deep diagnostic models, i.e., Multi-Perturbations Adversarial Training (MPAdvT) and Misclassification-Aware Adversarial Training (MAAdvT). The experimental results have shown that the use of defense methods can significantly improve the robustness of deep diagnostic models against adversarial attacks.