Multimodal analysis of functional and structural disconnection in Alzheimer's disease using multiple kernel SVM

Multispectral 3D DNA Machine Combined with Multimodal Machine Learning for Noninvasive Precise Diagnosis of Bladder Cancer

Extracellular vesicle (EV) molecular phenotyping offers enormous opportunities for cancer diagnostics. However, the majority of the associated studies adopted biomarker-based unimodal analysis to achieve cancer diagnosis, which has high false positives and low precision. Herein, we report a multimodal platform for the high-precision diagnosis of bladder cancer (BCa) through a multispectral 3D DNA machine in combination with a multimodal machine learning (ML) algorithm. The DNA machine was constructed using magnetic microparticles (MNPs) functionalized with aptamers that specifically identify the target of interest, i.e., five protein markers on bladder-cancer-derived urinary EVs (uEVs). The aptamers were hybridized with DNA-stabilized silver nanoclusters (DNA/AgNCs) and a G-quadruplex/hemin complex to form a sensing module. Such a DNA machine ensured multispectral detection of protein markers by fluorescence (FL), inductively coupled plasma mass spectrometry (ICP-MS), and UV-vis absorption (Abs). The obtained data sets then underwent uni- or multimodal ML for BCa diagnosis to compare the analytical performance. In this study, urine samples were obtained from our prospective cohort (n = 45). Our analytical results showed that the 3D DNA machine provided a detection limit of 9.2 × 103 particles mL-1 with a linear range of 4 × 104 to 5 × 107 particles mL-1 for uEVs. Moreover, the multimodal data fusion model exhibited an accuracy of 95.0%, a precision of 93.1%, and a recall rate of 93.2% on average, while those of the three types of unimodal models were no more than 91%. The elevated diagnosis precision by using the present fusion platform offers a perspective approach to diminishing the rate of misdiagnosis and overtreatment of BCa.

End-stage renal disease accompanied by mild cognitive impairment: A study and analysis of trimodal brain network fusion

The function and structure of brain networks (BN) may undergo changes in patients with end-stage renal disease (ESRD), particularly in those accompanied by mild cognitive impairment (ESRDaMCI). Many existing methods for fusing BN focus on extracting interaction features between pairs of network nodes from each mode and combining them. This approach overlooks the correlation between different modal features during feature extraction and the potentially valuable information that may exist between more than two brain regions. To address this issue, we propose a model using a multi-head self-attention mechanism to fuse brain functional networks, white matter structural networks, and gray matter structural networks, which results in the construction of brain fusion networks (FBN). Initially, three networks are constructed: the brain function network, the white matter structure network, and the individual-based gray matter structure network. The multi-head self-attention mechanism is then applied to fuse the three types of networks, generating attention weights that are transformed into an optimized model. The optimized model introduces hypergraph popular regular term and L1 norm regular term, leading to the formation of FBN. Finally, FBN is employed in the diagnosis and prediction of ESRDaMCI to evaluate its classification performance and investigate the correlation between discriminative brain regions and cognitive dysfunction. Experimental results demonstrate that the optimal classification accuracy achieved is 92.80%, which is at least 3.63% higher than the accuracy attained using other methods. This outcome confirms the effectiveness of our proposed method. Additionally, the identification of brain regions significantly associated with scores on the Montreal cognitive assessment scale may shed light on the underlying pathogenesis of ESRDaMCI.

Machine learning based on functional and structural connectivity in mild cognitive impairment

OBJECTIVE

Alzheimer's disease (AD) is a chronic, degenerative neurological disorder characterized by progressive cognitive decline and mental behavioral abnormalities. Mild cognitive impairment (MCI) is regarded as a transitional stage in the progression from normal elderly individuals to patients with AD. While studies have identified abnormalities in brain connectivity in patients with MCI, including functional and structural connectivity, accurately identifying patients with MCI in clinical screening remains challenging. We hypothesized that utilizing machine learning (ML) based on both functional and structural connectivity could yield meaningful results in distinguishing between patients with MCI and normal elderly individuals, so as to provide valuable information for early diagnosis and precise evaluation of patients with MCI.

METHODS

Following clinical criteria, we recruited 32 patients with MCI for the patient group, and 32 normal elderly individuals for the control group. All subjects underwent examinations for resting-state functional magnetic resonance imaging (rs-fMRI) and diffusion tensor imaging (DTI). Subsequently, significant functional and structural connectivity features were selected and combined with a support vector machine for classification of the patient and control groups.

RESULTS

We observed significantly different functional connectivity in the frontal lobe and putamen between the MCI group and normal controls. The results based on functional connectivity features demonstrated a classification accuracy of 71.88% and an area under the curve (AUC) value of 0.78. In terms of structural connectivity, we found that decreased fractional anisotropy in patients with MCI was significantly associated with Montreal Cognitive Assessment scores, specifically in regions such as the precuneus and cingulate gyrus. The classification results using the structural connectivity feature yielded an accuracy of 92.19% and an AUC value of 0.99. Lastly, combining functional and structural connectivity features resulted in a classification accuracy and AUC value of 93.75% and 0.99, respectively.

CONCLUSIONS

In this study, we demonstrated a high classification performance, underscoring the potential of both brain functional and structural connectivity in distinguishing patients with MCI from normal elderly individuals. Furthermore, the integration of functional connectivity and structural connectivity features indicated that utilizing rs-fMRI and DTI could enhance the accuracy and specificity of identifying patients with MCI compared with relying on a single neuroimaging technique.

Prediction of cognitive performance differences in older age from multimodal neuroimaging data

Differences in brain structure and functional and structural network architecture have been found to partly explain cognitive performance differences in older ages. Thus, they may serve as potential markers for these differences. Initial unimodal studies, however, have reported mixed prediction results of selective cognitive variables based on these brain features using machine learning (ML). Thus, the aim of the current study was to investigate the general validity of cognitive performance prediction from imaging data in healthy older adults. In particular, the focus was with examining whether (1) multimodal information, i.e., region-wise grey matter volume (GMV), resting-state functional connectivity (RSFC), and structural connectivity (SC) estimates, may improve predictability of cognitive targets, (2) predictability differences arise for global cognition and distinct cognitive profiles, and (3) results generalize across different ML approaches in 594 healthy older adults (age range: 55-85 years) from the 1000BRAINS study. Prediction potential was examined for each modality and all multimodal combinations, with and without confound (i.e., age, education, and sex) regression across different analytic options, i.e., variations in algorithms, feature sets, and multimodal approaches (i.e., concatenation vs. stacking). Results showed that prediction performance differed considerably between deconfounding strategies. In the absence of demographic confounder control, successful prediction of cognitive performance could be observed across analytic choices. Combination of different modalities tended to marginally improve predictability of cognitive performance compared to single modalities. Importantly, all previously described effects vanished in the strict confounder control condition. Despite a small trend for a multimodal benefit, developing a biomarker for cognitive aging remains challenging.

Gaussian Process-based prediction of memory performance and biomarker status in ageing and Alzheimer's disease-A systematic model evaluation

Neuroimaging markers based on Magnetic Resonance Imaging (MRI) combined with various other measures (such as genetic covariates, biomarkers, vascular risk factors, neuropsychological tests etc.) might provide useful predictions of clinical outcomes during the progression towards Alzheimer's disease (AD). The use of multiple features in predictive frameworks for clinical outcomes has become increasingly prevalent in AD research. However, many studies do not focus on systematically and accurately evaluating combinations of multiple input features. Hence, the aim of the present work is to explore and assess optimal combinations of various features for MR-based prediction of (1) cognitive status and (2) biomarker positivity with a multi-kernel learning Gaussian process framework. The explored features and parameters included (A) combinations of brain tissues, modulation, smoothing, and image resolution; (B) incorporating demographics & clinical covariates; (C) the impact of the size of the training data set; (D) the influence of dimensionality reduction and the choice of kernel types. The approach was tested in a large German cohort including 959 subjects from the multicentric longitudinal study of cognitive impairment and dementia (DELCODE). Our evaluation suggests the best prediction of memory performance was obtained for a combination of neuroimaging markers, demographics, genetic information (ApoE4) and CSF biomarkers explaining 57% of outcome variance in out-of-sample predictions. The highest performance for Aβ42/40 status classification was achieved for a combination of demographics, ApoE4, and a memory score while usage of structural MRI further improved the classification of individual patient's pTau status.

Reproducible Abnormalities and Diagnostic Generalizability of White Matter in Alzheimer's Disease

Alzheimer's disease (AD) is associated with the impairment of white matter (WM) tracts. The current study aimed to verify the utility of WM as the neuroimaging marker of AD with multisite diffusion tensor imaging datasets [321 patients with AD, 265 patients with mild cognitive impairment (MCI), 279 normal controls (NC)], a unified pipeline, and independent site cross-validation. Automated fiber quantification was used to extract diffusion profiles along tracts. Random-effects meta-analyses showed a reproducible degeneration pattern in which fractional anisotropy significantly decreased in the AD and MCI groups compared with NC. Machine learning models using tract-based features showed good generalizability among independent site cross-validation. The diffusion metrics of the altered regions and the AD probability predicted by the models were highly correlated with cognitive ability in the AD and MCI groups. We highlighted the reproducibility and generalizability of the degeneration pattern of WM tracts in AD.

Identifying Alzheimer's disease and mild cognitive impairment with atlas-based multi-modal metrics

Introduction

Multi-modal neuroimaging metrics in combination with advanced machine learning techniques have attracted more and more attention for an effective multi-class identification of Alzheimer's disease (AD), mild cognitive impairment (MCI) and health controls (HC) recently.

Methods

In this paper, a total of 180 subjects consisting of 44 AD, 66 MCI and 58 HC subjects were enrolled, and the multi-modalities of the resting-state functional magnetic resonance imaging (rs-fMRI) and the structural MRI (sMRI) for all participants were obtained. Then, four kinds of metrics including the Hurst exponent (HE) metric and bilateral hippocampus seed independently based connectivity metrics generated from fMRI data, and the gray matter volume (GMV) metric obtained from sMRI data, were calculated and extracted in each region of interest (ROI) based on a newly proposed automated anatomical Labeling (AAL3) atlas after data pre-processing. Next, these metrics were selected with a minimal redundancy maximal relevance (MRMR) method and a sequential feature collection (SFC) algorithm, and only a subset of optimal features were retained after this step. Finally, the support vector machine (SVM) based classification methods and artificial neural network (ANN) algorithm were utilized to identify the multi-class of AD, MCI and HC subjects in single modal and multi-modal metrics respectively, and a nested ten-fold cross-validation was utilized to estimate the final classification performance.

Results

The results of the SVM and ANN based methods indicated the best accuracies of 80.36 and 74.40%, respectively, by utilizing all the multi-modal metrics, and the optimal accuracies for AD, MCI and HC were 79.55, 78.79 and 82.76%, respectively, in the SVM based method. In contrast, when using single modal metric, the SVM based method obtained a best accuracy of 72.62% with the HE metric, and the accuracies for AD, MCI and HC subjects were just 56.82, 80.30 and 75.86%, respectively. Moreover, the overlapping abnormal brain regions detected by multi-modal metrics were mainly located at posterior cingulate gyrus, superior frontal gyrus and cuneus.

Conclusion

Taken together, the SVM based method with multi-modal metrics could provide effective diagnostic information for identifying AD, MCI and HC subjects.

Genetic algorithms based optimized hybrid deep learning model for explainable Alzheimer's prediction based on temporal multimodal cognitive data.. [DOI: 10.21203/rs.3.rs-3250006/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Alzheimer's Disease (AD) is an irreversible neurodegenerative disease. Its early detection is crucial to stop disease progression at an early stage. Most deep learning (DL) literature focused on neuroimage analysis. However, there is no noticed effect of these studies in the real environment. Model's robustness, cost, and interpretability are considered the main reasons for these limitations. The medical intuition of physicians is to evaluate the clinical biomarkers of patients then test their neuroimages. Cognitive scores provide an medically acceptable and cost-effective alternative for the neuroimages to predict AD progression. Each score is calculated from a collection of sub-scores which provide a deeper insight about patient conditions. No study in the literature have explored the role of these multimodal time series sub-scores to predict AD progression. We propose a hybrid CNN-LSTM DL model for predicting AD progression based on the fusion of four longitudinal cognitive sub-scores modalities. Bayesian optimizer has been used to select the best DL architecture. A genetic algorithms based feature selection optimization step has been added to the pipeline to select the best features from extracted deep representations of CNN-LSTM. The SoftMax classifier has been replaced by a robust and optimized random forest classifier. Extensive experiments using the ADNI dataset investigated the role of each optimization step, and the proposed model achieved the best results compared to other DL and classical machine learning models. The resulting model is robust, but it is a black box and it is difficult to understand the logic behind its decisions. Trustworthy AI models must be robust and explainable. We used SHAP and LIME to provide explainability features for the proposed model. The resulting trustworthy model has a great potential to be used to provide decision support in the real environments.

An extensible hierarchical graph convolutional network for early Alzheimer's disease identification

BACKGROUND AND OBJECTIVE

For early identification of Alzheimer's disease (AD) based on multi-modal magnetic resonance imaging (MRI) data, it is important to make comprehensive use of image features and non-image information to analyze the gray matter atrophy and the structural/functional connectivity abnormalities for different courses of AD.

METHODS

In this study, we propose an extensible hierarchical graph convolutional network (EH-GCN) for early AD identification. Based on the extracted image features from multi-modal MRI data using the presented multi-branch residual network (ResNet), the brain regions-of-interests (ROIs) based GCN is built to extract structural and functional connectivity features between different ROIs of the brain. In order to further improve the performance of AD identification, an optimized spatial GCN is proposed as convolution operator in the population-based GCN to avoid rebuilding the graph network and take advantage of relationships between subjects. Finally, the proposed EH-GCN is built by embedding the image features and internal brain connectivity features into the spatial population-based GCN, which provides an extensible way to improve early AD identification performance by adding imaging features and non-image information from multi-modal data.

RESULTS

Experiments are performed on two datasets, which illustrate the effectiveness of the extracted structural/functional connectivity features and the high computational efficiency of the proposed method. The classification accuracy of AD vs NC, AD vs MCI and MCI vs NC classification tasks reaches 88.71%, 82.71% and 79.68% respectively. The extracted connectivity features between ROIs indicate that functional abnormalities are earlier than gray matter atrophy and abnormalities of structural connections, which is consistent with the clinical manifestations. The proposed method allows for the addition of other modal image features and non-image information from multi-modal data to continuously improve the performance of clinical data analysis.

CONCLUSIONS

The proposed method can help us comprehensively analyze the role of gray matter atrophy, the damage of white matter nerve fiber tracts and the degradation of functional connectivity for different courses of AD, which could be useful for further extraction of clinical biomarkers for early AD identification.

BrainGB: A Benchmark for Brain Network Analysis With Graph Neural Networks

Mapping the connectome of the human brain using structural or functional connectivity has become one of the most pervasive paradigms for neuroimaging analysis. Recently, Graph Neural Networks (GNNs) motivated from geometric deep learning have attracted broad interest due to their established power for modeling complex networked data. Despite their superior performance in many fields, there has not yet been a systematic study of how to design effective GNNs for brain network analysis. To bridge this gap, we present BrainGB, a benchmark for brain network analysis with GNNs. BrainGB standardizes the process by (1) summarizing brain network construction pipelines for both functional and structural neuroimaging modalities and (2) modularizing the implementation of GNN designs. We conduct extensive experiments on datasets across cohorts and modalities and recommend a set of general recipes for effective GNN designs on brain networks. To support open and reproducible research on GNN-based brain network analysis, we host the BrainGB website at https://braingb.us with models, tutorials, examples, as well as an out-of-box Python package. We hope that this work will provide useful empirical evidence and offer insights for future research in this novel and promising direction.

A review on Alzheimer's disease classification from normal controls and mild cognitive impairment using structural MR images

Alzheimer's disease (AD) is an irreversible neurodegenerative brain disorder that degrades the memory and cognitive ability in elderly people. The main reason for memory loss and reduction in cognitive ability is the structural changes in the brain that occur due to neuronal loss. These structural changes are most conspicuous in the hippocampus, cortex, and grey matter and can be assessed by using neuroimaging techniques viz. Positron Emission Tomography (PET), structural Magnetic Resonance Imaging (MRI) and functional MRI (fMRI), etc. Out of these neuroimaging techniques, structural MRI has evolved as the best technique as it indicates the best soft tissue contrast and high spatial resolution which is important for AD detection. Currently, the focus of researchers is on predicting the conversion of Mild Cognitive Impairment (MCI) into AD. MCI represents the transition state between expected cognitive changes with normal aging and Alzheimer's disease. Not every MCI patient progresses into Alzheimer's disease. MCI can develop into stable MCI (sMCI, patients are called non-converters) or into progressive MCI (pMCI, patients are diagnosed as MCI converters). This paper discusses the prognosis of MCI to AD conversion and presents a review of structural MRI-based studies for AD detection. AD detection framework includes feature extraction, feature selection, and classification process. This paper reviews the studies for AD detection based on different feature extraction methods and machine learning algorithms for classification. The performance of various feature extraction methods has been compared and it has been observed that the wavelet transform-based feature extraction method would give promising results for AD classification. The present study indicates that researchers are successful in classifying AD from Normal Controls (N_rmC) but, it still requires a lot of work to be done for MCI/ N_rmC and MCI/AD, which would help in detecting AD at its early stage.

Microstructural mapping of dentate gyrus pathology in Alzheimer's disease: A 16.4 Tesla MRI study

The dentate gyrus (DG) is an integral portion of the hippocampal formation, and it is composed of three layers. Quantitative magnetic resonance (MR) imaging has the capability to map brain tissue microstructural properties which can be exploited to investigate neurodegeneration in Alzheimer's disease (AD). However, assessing subtle pathological changes within layers requires high resolution imaging and histological validation. In this study, we utilized a 16.4 Tesla scanner to acquire ex vivo multi-parameter quantitative MRI measures in human specimens across the layers of the DG. Using quantitative diffusion tensor imaging (DTI) and multi-parameter MR measurements acquired from AD (N = 4) and cognitively normal control (N = 6) tissues, we performed correlation analyses with histological measurements. Here, we found that quantitative MRI measures were significantly correlated with neurofilament and phosphorylated Tau density, suggesting sensitivity to layer-specific changes in the DG of AD tissues.

Classification of mild cognitive impairment based on handwriting dynamics and qEEG

Subtle changes in fine motor control and quantitative electroencephalography (qEEG) in patients with mild cognitive impairment (MCI) are important in screening for early dementia in primary care populations. In this study, an automated, non-invasive and rapid detection protocol for mild cognitive impairment based on handwriting kinetics and quantitative EEG analysis was proposed, and a classification model based on a dual fusion of feature and decision layers was designed for clinical decision-marking. Seventy-nine volunteers (39 healthy elderly controls and 40 patients with mild cognitive impairment) were recruited for this study, and the handwritten data and the EEG signals were performed using a tablet and MUSE under four designed handwriting tasks. Sixty-eight features were extracted from the EEG and handwriting parameters of each test. Features selected from both models were fused using a late feature fusion strategy with a weighted voting strategy for decision making, and classification accuracy was compared using three different classifiers under handwritten features, EEG features and fused features respectively. The results show that the dual fusion model can further improve the classification accuracy, with the highest classification accuracy for the combined features and the best classification result of 96.3% using SVM with RBF kernel as the base classifier. In addition, this not only supports the greater significance of multimodal data for differentiating MCI, but also tests the feasibility of using the portable EEG headband as a measure of EEG in patients with cognitive impairment.

Hypergraph representation of multimodal brain networks for patients with end-stage renal disease associated with mild cognitive impairment

The structure and function of brain networks (BN) may be altered in patients with end-stage renal disease (ESRD). However, there are relatively few attentions on ESRD associated with mild cognitive impairment (ESRDaMCI). Most studies focus on the pairwise relationships between brain regions, without taking into account the complementary information of functional connectivity (FC) and structural connectivity (SC). To address the problem, a hypergraph representation method is proposed to construct a multimodal BN for ESRDaMCI. First, the activity of nodes is determined by connection features extracted from functional magnetic resonance imaging (fMRI) (i.e., FC), and the presence of edges is determined by physical connections of nerve fibers extracted from diffusion kurtosis imaging (DKI) (i.e., SC). Then, the connection features are generated through bilinear pooling and transformed into an optimization model. Next, a hypergraph is constructed according to the generated node representation and connection features, and the node degree and edge degree of the hypergraph are calculated to obtain the hypergraph manifold regularization (HMR) term. The HMR and L₁ norm regularization terms are introduced into the optimization model to achieve the final hypergraph representation of multimodal BN (HRMBN). Experimental results show that the classification performance of HRMBN is significantly better than that of several state-of-the-art multimodal BN construction methods. Its best classification accuracy is 91.0891%, at least 4.3452% higher than that of other methods, verifying the effectiveness of our method. The HRMBN not only achieves better results in ESRDaMCI classification, but also identifies the discriminative brain regions of ESRDaMCI, which provides a reference for the auxiliary diagnosis of ESRD.

Signed graph representation learning for functional-to-structural brain network mapping

MRI-derived brain networks have been widely used to understand functional and structural interactions among brain regions, and factors that affect them, such as brain development and diseases. Graph mining on brain networks can facilitate the discovery of novel biomarkers for clinical phenotypes and neurodegenerative diseases. Since brain functional and structural networks describe the brain topology from different perspectives, exploring a representation that combines these cross-modality brain networks has significant clinical implications. Most current studies aim to extract a fused representation by projecting the structural network to the functional counterpart. Since the functional network is dynamic and the structural network is static, mapping a static object to a dynamic object may not be optimal. However, mapping in the opposite direction (i.e., from functional to structural networks) are suffered from the challenges introduced by negative links within signed graphs. Here, we propose a novel graph learning framework, named as Deep Signed Brain Graph Mining or DSBGM, with a signed graph encoder that, from an opposite perspective, learns the cross-modality representations by projecting the functional network to the structural counterpart. We validate our framework on clinical phenotype and neurodegenerative disease prediction tasks using two independent, publicly available datasets (HCP and OASIS). Our experimental results clearly demonstrate the advantages of our model compared to several state-of-the-art methods.

Classification and prediction of cognitive performance differences in older age based on brain network patterns using a machine learning approach

Age-related cognitive decline varies greatly in healthy older adults, which may partly be explained by differences in the functional architecture of brain networks. Resting-state functional connectivity (RSFC) derived network parameters as widely used markers describing this architecture have even been successfully used to support diagnosis of neurodegenerative diseases. The current study aimed at examining whether these parameters may also be useful in classifying and predicting cognitive performance differences in the normally aging brain by using machine learning (ML). Classifiability and predictability of global and domain-specific cognitive performance differences from nodal and network-level RSFC strength measures were examined in healthy older adults from the 1000BRAINS study (age range: 55-85 years). ML performance was systematically evaluated across different analytic choices in a robust cross-validation scheme. Across these analyses, classification performance did not exceed 60% accuracy for global and domain-specific cognition. Prediction performance was equally low with high mean absolute errors (MAEs ≥ 0.75) and low to none explained variance (R2 ≤ 0.07) for different cognitive targets, feature sets, and pipeline configurations. Current results highlight limited potential of functional network parameters to serve as sole biomarker for cognitive aging and emphasize that predicting cognition from functional network patterns may be challenging.

Modern views of machine learning for precision psychiatry

In light of the National Institute of Mental Health (NIMH)'s Research Domain Criteria (RDoC), the advent of functional neuroimaging, novel technologies and methods provide new opportunities to develop precise and personalized prognosis and diagnosis of mental disorders. Machine learning (ML) and artificial intelligence (AI) technologies are playing an increasingly critical role in the new era of precision psychiatry. Combining ML/AI with neuromodulation technologies can potentially provide explainable solutions in clinical practice and effective therapeutic treatment. Advanced wearable and mobile technologies also call for the new role of ML/AI for digital phenotyping in mobile mental health. In this review, we provide a comprehensive review of ML methodologies and applications by combining neuroimaging, neuromodulation, and advanced mobile technologies in psychiatry practice. We further review the role of ML in molecular phenotyping and cross-species biomarker identification in precision psychiatry. We also discuss explainable AI (XAI) and neuromodulation in a closed human-in-the-loop manner and highlight the ML potential in multi-media information extraction and multi-modal data fusion. Finally, we discuss conceptual and practical challenges in precision psychiatry and highlight ML opportunities in future research.

An enhanced multi-modal brain graph network for classifying neuropsychiatric disorders

It has been proven that neuropsychiatric disorders (NDs) can be associated with both structures and functions of brain regions. Thus, data about structures and functions could be usefully combined in a comprehensive analysis. While brain structural MRI (sMRI) images contain anatomic and morphological information about NDs, functional MRI (fMRI) images carry complementary information. However, efficient extraction and fusion of sMRI and fMRI data remains challenging. In this study, we develop an enhanced multi-modal graph convolutional network (MME-GCN) in a binary classification between patients with NDs and healthy controls, based on the fusion of the structural and functional graphs of the brain region. First, based on the same brain atlas, we construct structural and functional graphs from sMRI and fMRI data, respectively. Second, we use machine learning to extract important features from the structural graph network. Third, we use these extracted features to adjust the corresponding edge weights in the functional graph network. Finally, we train a multi-layer GCN and use it in binary classification task. MME-GCN achieved 93.71% classification accuracy on the open data set provided by the Consortium for Neuropsychiatric Phenomics. In addition, we analyzed the important features selected from the structural graph and verified them in the functional graph. Using MME-GCN, we found several specific brain connections important to NDs.

Structural and Functional Brain Connectivity Uniquely Contribute to Episodic Memory Performance in Older Adults

In this study, we examined the independent contributions of structural and functional connectivity markers to individual differences in episodic memory performance in 107 cognitively normal older adults from the BIOCARD study. Structural connectivity, defined by the diffusion tensor imaging (DTI) measure of radial diffusivity (RD), was obtained from two medial temporal lobe white matter tracts: the fornix and hippocampal cingulum, while functional connectivity markers were derived from network-based resting state functional magnetic resonance imaging (rsfMRI) of five large-scale brain networks: the control, default, limbic, dorsal attention, and salience/ventral attention networks. Hierarchical and stepwise linear regression methods were utilized to directly compare the relative contributions of the connectivity modalities to individual variability in a composite delayed episodic memory score, while also accounting for age, sex, cerebrospinal fluid (CSF) biomarkers of amyloid and tau pathology (i.e., Aβ₄₂/Aβ₄₀ and p-tau₁₈₁), and gray matter volumes of the entorhinal cortex and hippocampus. Results revealed that fornix RD, hippocampal cingulum RD, and salience network functional connectivity were each significant independent predictors of memory performance, while CSF markers and gray matter volumes were not. Moreover, in the stepwise model, the addition of sex, fornix RD, hippocampal cingulum RD, and salience network functional connectivity each significantly improved the overall predictive value of the model. These findings demonstrate that both DTI and rsfMRI connectivity measures uniquely contributed to the model and that the combination of structural and functional connectivity markers best accounted for individual variability in episodic memory function in cognitively normal older adults.

Joint Embedding of Structural and Functional Brain Networks with Graph Neural Networks for Mental Illness Diagnosis

Multimodal brain networks characterize complex connectivities among different brain regions from both structural and functional aspects and provide a new means for mental disease analysis. Recently, Graph Neural Networks (GNNs) have become a de facto model for analyzing graph-structured data. However, how to employ GNNs to extract effective representations from brain networks in multiple modalities remains rarely explored. Moreover, as brain networks provide no initial node features, how to design informative node attributes and leverage edge weights for GNNs to learn is left unsolved. To this end, we develop a novel multiview GNN for multimodal brain networks. In particular, we treat each modality as a view for brain networks and employ contrastive learning for multimodal fusion. Then, we propose a GNN model which takes advantage of the message passing scheme by propagating messages based on degree statistics and brain region connectivities. Extensive experiments on two real-world disease datasets (HIV and Bipolar) demonstrate the effectiveness of our proposed method over state-of-the-art baselines.

Multiple-Kernel Support Vector Machine for Predicting Internet Gaming Disorder Using Multimodal Fusion of PET, EEG, and Clinical Features

Internet gaming disorder (IGD) has become an important social and psychiatric issue in recent years. To prevent IGD and provide the appropriate intervention, an accurate prediction method for identifying IGD is necessary. In this study, we investigated machine learning methods of multimodal neuroimaging data including Positron Emission Tomography (PET), Electroencephalography (EEG), and clinical features to enhance prediction accuracy. Unlike the conventional methods which usually concatenate all features into one feature vector, we adopted a multiple-kernel support vector machine (MK-SVM) to classify IGD. We compared the prediction performance of standard machine learning methods such as SVM, random forest, and boosting with the proposed method in patients with IGD (N = 28) and healthy controls (N = 24). We showed that the prediction accuracy of the optimal MK-SVM using three kinds of modalities was much higher than other conventional machine learning methods, with the highest accuracy being 86.5%, the sensitivity 89.3%, and the specificity 83.3%. Furthermore, we deduced that clinical variables had the highest contribution to the optimal IGD prediction model and that the other two modalities were also indispensable. We found that more efficient integration of multimodal data through kernel combination could contribute to better performance of the prediction model. This study is a novel attempt to integrate each method from different sources and suggests that integrating each method, such as self-administrated reports, PET, and EEG, improves the prediction of IGD.

Automated Classification of Mild Cognitive Impairment by Machine Learning With Hippocampus-Related White Matter Network

Background

Detection of mild cognitive impairment (MCI) is essential to screen high risk of Alzheimer’s disease (AD). However, subtle changes during MCI make it challenging to classify in machine learning. The previous pathological analysis pointed out that the hippocampus is the critical hub for the white matter (WM) network of MCI. Damage to the white matter pathways around the hippocampus is the main cause of memory decline in MCI. Therefore, it is vital to biologically extract features from the WM network driven by hippocampus-related regions to improve classification performance.

Methods

Our study proposes a method for feature extraction of the whole-brain WM network. First, 42 MCI and 54 normal control (NC) subjects were recruited using diffusion tensor imaging (DTI), resting-state functional magnetic resonance imaging (rs-fMRI), and T1-weighted (T1w) imaging. Second, mean diffusivity (MD) and fractional anisotropy (FA) were calculated from DTI, and the whole-brain WM networks were obtained. Third, regions of interest (ROIs) with significant functional connectivity to the hippocampus were selected for feature extraction, and the hippocampus (HIP)-related WM networks were obtained. Furthermore, the rank sum test with Bonferroni correction was used to retain significantly different connectivity between MCI and NC, and significant HIP-related WM networks were obtained. Finally, the classification performances of these three WM networks were compared to select the optimal feature and classifier.

Results

(1) For the features, the whole-brain WM network, HIP-related WM network, and significant HIP-related WM network are significantly improved in turn. Also, the accuracy of MD networks as features is better than FA. (2) For the classification algorithm, the support vector machine (SVM) classifier with radial basis function, taking the significant HIP-related WM network in MD as a feature, has the optimal classification performance (accuracy = 89.4%, AUC = 0.954). (3) For the pathologic mechanism, the hippocampus and thalamus are crucial hubs of the WM network for MCI.

Conclusion

Feature extraction from the WM network driven by hippocampus-related regions provides an effective method for the early diagnosis of AD.

A Multi-Modal and Multi-Atlas Integrated Framework for Identification of Mild Cognitive Impairment

BACKGROUND

Multi-modal neuroimaging with appropriate atlas is vital for effectively differentiating mild cognitive impairment (MCI) from healthy controls (HC).

METHODS

The resting-state functional magnetic resonance imaging (rs-fMRI) and structural MRI (sMRI) of 69 MCI patients and 61 HC subjects were collected. Then, the gray matter volumes obtained from the sMRI and Hurst exponent (HE) values calculated from rs-fMRI data in the Automated Anatomical Labeling (AAL-90), Brainnetome (BN-246), Harvard-Oxford (HOA-112) and AAL3-170 atlases were extracted, respectively. Next, these characteristics were selected with a minimal redundancy maximal relevance algorithm and a sequential feature collection method in single or multi-modalities, and only the optimal features were retained after this procedure. Lastly, the retained characteristics were served as the input features for the support vector machine (SVM)-based method to classify MCI patients, and the performance was estimated with a leave-one-out cross-validation (LOOCV).

RESULTS

Our proposed method obtained the best 92.00% accuracy, 94.92% specificity and 89.39% sensitivity with the sMRI in AAL-90 and the fMRI in HOA-112 atlas, which was much better than using the single-modal or single-atlas features.

CONCLUSION

The results demonstrated that the multi-modal and multi-atlas integrated method could effectively recognize MCI patients, which could be extended into various neurological and neuropsychiatric diseases.

A resting-state fMRI study of temporal lobe epilepsy using multivariate pattern analysis and Granger causality analysis

BACKGROUND AND PURPOSE

Understanding the pathogenesis of temporal lobe epilepsy (TLE) is essential for its diagnosis and treatment. The study aimed to explore regional homogeneity (ReHo) and changes in effective connectivity (EC) between brain regions in TLE patients, hoping to discover potential abnormalities in certain brain regions in TLE patients.

METHODS

Resting-state functional magnetic resonance data were collected from 23 TLE patients and 32 normal controls (NC). ReHo was used as a feature of multivariate pattern analysis (MVPA) to explore the ability of its alterations in identifying TLE. Based on the results of the MVPA, certain brain regions were selected as seed points to further explore alterations in EC between brain regions using Granger causality analysis.

RESULTS

MVPA results showed that the classification accuracy for the TLE and NC groups was 87.27%, and the right posterior cerebellum lobe, right lingual gyrus (LING_R), right cuneus (CUN_R), and left superior temporal gyrus (STG_L) provided significant contributions. Moreover, the EC from STG_L to right fusiform gyrus (FFG_R) and LING_R and the EC from CUN_R to the right occipital superior gyrus (SOG_R) and right occipital middle gyrus (MOG_R) were altered compared to the NC group.

CONCLUSION

The MVPA results indicated that ReHo abnormalities in brain regions may be an important feature in the identification of TLE. The enhanced EC from STG_L to FFG_R and LING_R indicates a shift in language processing to the right hemisphere, and the weakened EC from SOG_R and MOG_R to CUN_R may reveal an underlying mechanism of TLE.

Multi-Modal Neuroimaging Neural Network-Based Feature Detection for Diagnosis of Alzheimer’s Disease

Alzheimer’s disease (AD) is a neurodegenerative brain disease, and it is challenging to mine features that distinguish AD and healthy control (HC) from multiple datasets. Brain network modeling technology in AD using single-modal images often lacks supplementary information regarding multi-source resolution and has poor spatiotemporal sensitivity. In this study, we proposed a novel multi-modal LassoNet framework with a neural network for AD-related feature detection and classification. Specifically, data including two modalities of resting-state functional magnetic resonance imaging (rs-fMRI) and diffusion tensor imaging (DTI) were adopted for predicting pathological brain areas related to AD. The results of 10 repeated experiments and validation experiments in three groups prove that our proposed framework outperforms well in classification performance, generalization, and reproducibility. Also, we found discriminative brain regions, such as Hippocampus, Frontal_Inf_Orb_L, Parietal_Sup_L, Putamen_L, Fusiform_R, etc. These discoveries provide a novel method for AD research, and the experimental study demonstrates that the framework will further improve our understanding of the mechanisms underlying the development of AD.

An Ensemble Hybrid Feature Selection Method for Neuropsychiatric Disorder Classification

Magnetic resonance imagings (MRIs) are providing increased access to neuropsychiatric disorders that can be made available for advanced data analysis. However, the single type of data limits the ability of psychiatrists to distinguish the subclasses of this disease. In this paper, we propose an ensemble hybrid features selection method for the neuropsychiatric disorder classification. The method consists of a 3D DenseNet and a XGBoost, which are used to select the image features from structural MRI images and the phenotypic feature from phenotypic records, respectively. The hybrid feature is composed of image features and phenotypic features. The proposed method is validated in the Consortium for Neuropsychiatric Phenomics (CNP) dataset, where samples are classified into one of the four classes (healthy controls (HC), attention deficit hyperactivity disorder (ADHD), bipolar disorder (BD), and schizophrenia (SD)). Experimental results show that the hybrid feature can improve the performance of classification methods. The best accuracy of binary and multi-class classification can reach 91.22 and 78.62 percent, respectively. We analyze the importance of phenotypic features and image features in different classification tasks. The importance of the structure MRI images is highlighted by incorporating phenotypic features with image features to generate hybrid features. We also visualize the features of three neuropsychiatric disorders and analyze their locations in the brain region.

Artificial intelligence in brain MRI analysis of Alzheimer's disease over the past 12 years: A systematic review

INTRODUCTION

Multiple structural brain changes in Alzheimer's disease (AD) and mild cognitive impairment (MCI) have been revealed on magnetic resonance imaging (MRI). There is a fast-growing effort in applying artificial intelligence (AI) to analyze these data. Here, we review and evaluate the AI studies in brain MRI analysis with synthesis.

METHODS

A systematic review of the literature, spanning the years from 2009 to 2020, was completed using the PubMed database. AI studies using MRI imaging to investigate normal aging, mild cognitive impairment, and AD-dementia were retrieved for review. Bias assessment was completed using the PROBAST criteria.

RESULTS

97 relevant studies were included in the review. The studies were typically focused on the classification of AD, MCI, and normal aging (71% of the reported studies) and the prediction of MCI conversion to AD (25%). The best performance was achieved by using the deep learning-based convolution neural network algorithms (weighted average accuracy 89%), in contrast to 76-86% using Logistic Regression, Support Vector Machines, and other AI methods.

DISCUSSION

The synthesized evidence is paramount to developing sophisticated AI approaches to reliably capture and quantify multiple subtle MRI changes in the whole brain that exemplify the complexity and heterogeneity of AD and brain aging.

Analysis of fMRI Signals from Working Memory Tasks and Resting-State of Brain: Neutrosophic-Entropy-Based Clustering Algorithm

This study applies a neutrosophic-entropy-based clustering algorithm (NEBCA) to analyze the fMRI signals. We consider the data obtained from four different working memory tasks and the brain's resting state for the experimental purpose. Three non-overlapping clusters of data related to temporal brain activity are determined and statistically analyzed. Moreover, we used the Uniform Manifold Approximation and Projection (UMAP) method to reduce system dimensionality and present the effectiveness of NEBCA. The results show that using NEBCA, we are able to distinguish between different working memory tasks and resting-state and identify subtle differences in the related activity of brain regions. By analyzing the statistical properties of the entropy inside the clusters, the various regions of interest (ROIs), according to Automated Anatomical Labeling (AAL) atlas crucial for clustering procedure, are determined. The inferior occipital gyrus is established as an important brain region in distinguishing the resting state from the tasks. Moreover, the inferior occipital gyrus and superior parietal lobule are identified as necessary to correct the data discrimination related to the different memory tasks. We verified the statistical significance of the results through the two-sample t-test and analysis of surrogates performed by randomization of the cluster elements. The presented methodology is also appropriate to determine the influence of time of day on brain activity patterns. The differences between working memory tasks and resting-state in the morning are related to a lower index of small-worldness and sleep inertia in the first hours after waking. We also compared the performance of NEBCA to two existing algorithms, KMCA and FKMCA. We showed the advantage of the NEBCA over these algorithms that could not effectively accumulate fMRI signals with higher variability.

A methodological scoping review of the integration of fMRI to guide dMRI tractography. What has been done and what can be improved: A 20-year perspective

Combining MRI modalities is a growing trend in neurosciences. It provides opportunities to investigate the brain architecture supporting cognitive functions. Integrating fMRI activation to guide dMRI tractography offers potential advantages over standard tractography methods. A quick glimpse of the literature on this topic reveals that this technique is challenging, and no consensus or "best practices" currently exist, at least not within a single document. We present the first attempt to systematically analyze and summarize the literature of 80 studies that integrated task-based fMRI results to guide tractography, over the last two decades. We report 19 findings that cover challenges related to sample size, microstructure modelling, seeding methods, multimodal space registration, false negatives/positives, specificity/validity, gray/white matter interface and more. These findings will help the scientific community (1) understand the strengths and limitations of the approaches, (2) design studies using this integrative framework, and (3) motivate researchers to fill the gaps identified. We provide references toward best practices, in order to improve the overall result's replicability, sensitivity, specificity, and validity.

Machine Learning-Derived Multimodal Neuroimaging of Presurgical Target Area to Predict Individual's Seizure Outcomes After Epilepsy Surgery

Objectives: Half of the patients who have tailored resection of the suspected epileptogenic zone for drug-resistant epilepsy have recurrent postoperative seizures. Although neuroimaging has become an indispensable part of delineating the epileptogenic zone, no validated method uses neuroimaging of presurgical target area to predict an individual's post-surgery seizure outcome. We aimed to develop and validate a machine learning-powered approach incorporating multimodal neuroimaging of a presurgical target area to predict an individual's post-surgery seizure outcome in patients with drug-resistant focal epilepsy. Materials and Methods: One hundred and forty-one patients with drug-resistant focal epilepsy were classified either as having seizure-free (Engel class I) or seizure-recurrence (Engel class II through IV) at least 1 year after surgery. The presurgical magnetic resonance imaging, positron emission tomography, computed tomography, and postsurgical magnetic resonance imaging were co-registered for surgical target volume of interest (VOI) segmentation; all VOIs were decomposed into nine fixed views, then were inputted into the deep residual network (DRN) that was pretrained on Tiny-ImageNet dataset to extract and transfer deep features. A multi-kernel support vector machine (MKSVM) was used to integrate multiple views of feature sets and to predict seizure outcomes of the targeted VOIs. Leave-one-out validation was applied to develop a model for verifying the prediction. In the end, performance using this approach was assessed by calculating accuracy, sensitivity, and specificity. Receiver operating characteristic curves were generated, and the optimal area under the receiver operating characteristic curve (AUC) was calculated as a metric for classifying outcomes. Results: Application of DRN-MKSVM model based on presurgical target area neuroimaging demonstrated good performance in predicting seizure outcomes. The AUC ranged from 0.799 to 0.952. Importantly, the classification performance DRN-MKSVM model using data from multiple neuroimaging showed an accuracy of 91.5%, a sensitivity of 96.2%, a specificity of 85.5%, and AUCs of 0.95, which were significantly better than any other single-modal neuroimaging (all p ˂ 0.05). Conclusion: DRN-MKSVM, using multimodal compared with unimodal neuroimaging from the surgical target area, accurately predicted postsurgical outcomes. The preoperative individualized prediction of seizure outcomes in patients who have been judged eligible for epilepsy surgery could be conveniently facilitated. This may aid epileptologists in presurgical evaluation by providing a tool to explore various surgical options, offering complementary information to existing clinical techniques.

Combined Structural MR and Diffusion Tensor Imaging Classify the Presence of Alzheimer's Disease With the Same Performance as MR Combined With Amyloid Positron Emission Tomography: A Data Integration Approach

Background: In recent years, classification frameworks using imaging data have shown that multimodal classification methods perform favorably over the use of a single imaging modality for the diagnosis of Alzheimer's Disease. The currently used clinical approach often emphasizes the use of qualitative MRI and/or PET data for clinical diagnosis. Based on the hypothesis that classification of isolated imaging modalities is not predictive of their respective value in combined approaches, we investigate whether the combination of T1 Weighted MRI and diffusion tensor imaging (DTI) can yield an equivalent performance as the combination of quantitative structural MRI (sMRI) with amyloid-PET. Methods: We parcellated the brain into regions of interest (ROI) following different anatomical labeling atlases. For each region of interest different metrics were extracted from the different imaging modalities (sMRI, PiB-PET, and DTI) to be used as features. Thereafter, the feature sets were reduced using an embedded-based feature selection method. The final reduced sets were then used as input in support vector machine (SVM) classifiers. Three different base classifiers were created, one for each imaging modality, and validated using internal (n = 41) and external data from the ADNI initiative (n = 330 for sMRI, n = 148 for DTI and n = 55 for PiB-PET) sources. Finally, the classifiers were ensembled using a weighted method in order to evaluate the performance of different combinations. Results: For the base classifiers the following performance levels were found: sMRI-based classifier (accuracy, 92%; specificity, 97% and sensitivity, 87%), PiB-PET (accuracy, 91%; specificity, 89%; and sensitivity, 92%) and the lowest performance was attained with DTI (accuracy, 80%; specificity, 76%; and sensitivity, 82%). From the multimodal approaches, when integrating two modalities, the following results were observed: sMRI+PiB-PET (accuracy, 98%; specificity, 98%; and sensitivity, 99%), sMRI+DTI (accuracy, 97%; specificity, 99%; and sensitivity, 94%) and PiB-PET+DTI (accuracy, 91%; specificity, 90%; and sensitivity, 93%). Finally, the combination of all imaging modalities yielded an accuracy of 98%, specificity of 97% and sensitivity of 99%. Conclusion: Although DTI in isolation shows relatively poor performance, when combined with structural MR, it showed a surprising classification performance which was comparable to MR combined with amyloid PET. These results are consistent with the notion that white matter changes are also important in Alzheimer's Disease.

Multimodal Classification of Alzheimer's Disease and Amnestic Mild Cognitive Impairment: Integrated 18F-FDG PET and DTI Study

BACKGROUND

Alzheimer's disease (AD) is a progressive neurodegenerative disease characterized by cognitive decline and memory impairment. Amnestic mild cognitive impairment (aMCI) is the intermediate stage between normal cognitive aging and early dementia caused by AD. It can be challenging to differentiate aMCI patients from healthy controls (HC) and mild AD patients.

OBJECTIVE

To validate whether the combination of 18F-fluorodeoxyglucose positron emission tomography (18F-FDG PET) and diffusion tensor imaging (DTI) will improve classification performance compared with that based on a single modality.

METHODS

A total of thirty patients with AD, sixty patients with aMCI, and fifty healthy controls were included. AD was diagnosed according to the National Institute of Neurological and Communicative Diseases and Stroke/Alzheimer's Disease and Related Disorders Association (NINCDS-ADRDA) criteria for probable. aMCI diagnosis was based on Petersen's criteria. The 18F-FDG PET and DTI measures were each used separately or in combination to evaluate sensitivity, specificity, and accuracy for differentiating HC, aMCI, and AD using receiver operating characteristic analysis together with binary logistic regression. The rate of accuracy was based on the area under the curve (AUC).

RESULTS

For classifying AD from HC, we achieve an AUC of 0.96 when combining two modalities of biomarkers and 0.93 when using 18F-FDG PET individually. For classifying aMCI from HC, we achieve an AUC of 0.79 and 0.76 using the best individual modality of biomarkers.

CONCLUSION

Our results show that the combination of two modalities improves classification performance, compared with that using any individual modality.

Multimodal Brain Network Jointly Construction and Fusion for Diagnosis of Epilepsy

Brain network analysis has been proved to be one of the most effective methods in brain disease diagnosis. In order to construct discriminative brain networks and improve the performance of disease diagnosis, many machine learning–based methods have been proposed. Recent studies show that combining functional and structural brain networks is more effective than using only single modality data. However, in the most of existing multi-modal brain network analysis methods, it is a common strategy that constructs functional and structural network separately, which is difficult to embed complementary information of different modalities of brain network. To address this issue, we propose a unified brain network construction algorithm, which jointly learns both functional and structural data and effectively face the connectivity and node features for improving classification. First, we conduct space alignment and brain network construction under a unified framework, and then build the correlation model among all brain regions with functional data by low-rank representation so that the global brain region correlation can be captured. Simultaneously, the local manifold with structural data is embedded into this model to preserve the local structural information. Second, the PageRank algorithm is adaptively used to evaluate the significance of different brain regions, in which the interaction of multiple brain regions is considered. Finally, a multi-kernel strategy is utilized to solve the data heterogeneity problem and merge the connectivity as well as node information for classification. We apply the proposed method to the diagnosis of epilepsy, and the experimental results show that our method can achieve a promising performance.

Application of Artificial Intelligence in the MRI Classification Task of Human Brain Neurological and Psychiatric Diseases: A Scoping Review

Artificial intelligence (AI) for medical imaging is a technology with great potential. An in-depth understanding of the principles and applications of magnetic resonance imaging (MRI), machine learning (ML), and deep learning (DL) is fundamental for developing AI-based algorithms that can meet the requirements of clinical diagnosis and have excellent quality and efficiency. Moreover, a more comprehensive understanding of applications and opportunities would help to implement AI-based methods in an ethical and sustainable manner. This review first summarizes recent research advances in ML and DL techniques for classifying human brain magnetic resonance images. Then, the application of ML and DL methods to six typical neurological and psychiatric diseases is summarized, including Alzheimer's disease (AD), Parkinson's disease (PD), major depressive disorder (MDD), schizophrenia (SCZ), attention-deficit/hyperactivity disorder (ADHD), and autism spectrum disorder (ASD). Finally, the limitations of the existing research are discussed, and possible future research directions are proposed.

Deep Fusion of Brain Structure-Function in Mild Cognitive Impairment

Multimodal fusion of different types of neural image data provides an irreplaceable opportunity to take advantages of complementary cross-modal information that may only partially be contained in single modality. To jointly analyze multimodal data, deep neural networks can be especially useful because many studies have suggested that deep learning strategy is very efficient to reveal complex and non-linear relations buried in the data. However, most deep models, e.g., convolutional neural network and its numerous extensions, can only operate on regular Euclidean data like voxels in 3D MRI. The interrelated and hidden structures that beyond the grid neighbors, such as brain connectivity, may be overlooked. Moreover, how to effectively incorporate neuroscience knowledge into multimodal data fusion with a single deep framework is understudied. In this work, we developed a graph-based deep neural network to simultaneously model brain structure and function in Mild Cognitive Impairment (MCI): the topology of the graph is initialized using structural network (from diffusion MRI) and iteratively updated by incorporating functional information (from functional MRI) to maximize the capability of differentiating MCI patients from elderly normal controls. This resulted in a new connectome by exploring "deep relations" between brain structure and function in MCI patients and we named it as Deep Brain Connectome. Though deep brain connectome is learned individually, it shows consistent patterns of alteration comparing to structural network at group level. With deep brain connectome, our developed deep model can achieve 92.7% classification accuracy on ADNI dataset.

Combined Support Vector Machine Classifier and Brain Structural Network Features for the Individual Classification of Amnestic Mild Cognitive Impairment and Subjective Cognitive Decline Patients

OBJECTIVE

Individuals with subjective cognitive decline (SCD) or amnestic mild cognitive impairment (aMCI) represent important targets for the early detection and intervention of Alzheimer's disease (AD). In this study, we employed a multi-kernel support vector machine (SVM) to examine whether white matter (WM) structural networks can be used for screening SCD and aMCI.

METHODS

A total of 138 right-handed participants [51 normal controls (NC), 36 SCD, 51 aMCI] underwent MRI brain scans. For each participant, three types of WM networks with different edge weights were constructed with diffusion MRI data: fiber number-weighted networks, mean fractional anisotropy-weighted networks, and mean diffusivity (MD)-weighted networks. By employing a multiple-kernel SVM, we seek to integrate information from three weighted networks to improve classification performance. The accuracy of classification between each pair of groups was evaluated via leave-one-out cross-validation.

RESULTS

For the discrimination between SCD and NC, an area under the curve (AUC) value of 0.89 was obtained, with an accuracy of 83.9%. Further analysis revealed that the methods using three types of WM networks outperformed other methods using single WM network. Moreover, we found that most of discriminative features were from MD-weighted networks, which distributed among frontal lobes. Similar classification performance was also reported in the differentiation between subjects with aMCI and NCs (accuracy = 83.3%). Between SCD and aMCI, an AUC value of 0.72 was obtained, with an accuracy of 72.4%, sensitivity of 74.5% and specificity of 69.4%. The highest accuracy was achieved with features only selected from MD-weighted networks.

CONCLUSION

White matter structural network features help machine learning algorithms accurately identify individuals with SCD and aMCI from NCs. Our findings have significant implications for the development of potential brain imaging markers for the early detection of AD.

Effects of Brain Atlases and Machine Learning Methods on the Discrimination of Schizophrenia Patients: A Multimodal MRI Study

Recently, machine learning techniques have been widely applied in discriminative studies of schizophrenia (SZ) patients with multimodal magnetic resonance imaging (MRI); however, the effects of brain atlases and machine learning methods remain largely unknown. In this study, we collected MRI data for 61 first-episode SZ patients (FESZ), 79 chronic SZ patients (CSZ) and 205 normal controls (NC) and calculated 4 MRI measurements, including regional gray matter volume (GMV), regional homogeneity (ReHo), amplitude of low-frequency fluctuation and degree centrality. We systematically analyzed the performance of two classifications (SZ vs NC; FESZ vs CSZ) based on the combinations of three brain atlases, five classifiers, two cross validation methods and 3 dimensionality reduction algorithms. Our results showed that the groupwise whole-brain atlas with 268 ROIs outperformed the other two brain atlases. In addition, the leave-one-out cross validation was the best cross validation method to select the best hyperparameter set, but the classification performances by different classifiers and dimensionality reduction algorithms were quite similar. Importantly, the contributions of input features to both classifications were higher with the GMV and ReHo features of brain regions in the prefrontal and temporal gyri. Furthermore, an ensemble learning method was performed to establish an integrated model, in which classification performance was improved. Taken together, these findings indicated the effects of these factors in constructing effective classifiers for psychiatric diseases and showed that the integrated model has the potential to improve the clinical diagnosis and treatment evaluation of SZ.

Machine learning based on the multimodal connectome can predict the preclinical stage of Alzheimer's disease: a preliminary study

OBJECTIVES

Subjective cognitive decline (SCD) may be a preclinical stage of Alzheimer's disease (AD). Neuroimaging studies suggest that abnormal brain connectivity plays an important role in the pathophysiology of SCD. However, most previous studies focused on single modalities only. Multimodal combinations can more effectively utilize various information and little is known about their diagnostic value in SCD.

METHODS

One hundred ten SCD individuals and well-matched healthy controls (HCs) were recruited in this study (the primary sample: 35 SCD and 36 HC; the validation sample: 21 SCD and 18 HC). Multimodal imaging data were used to construct functional, anatomical, and morphological networks, respectively. These networks were used in combination with a multiple kernel learning-support vector machine to predict SCD individuals. We validated our model on another independent sample. Multiple linear regression (MLR) analyses were conducted to investigate the relationships among network metrics, cognition, and pathological biomarkers.

RESULTS

We found that the characteristics identified from the multimodal network were primarily located in the default mode network (DMN) and salience network (SN), achieving an accuracy of 88.73% (an accuracy of 79.49% for an independent sample) based on the integration of the three modalities. MLR analyses showed that increased AV45 SUVRs were significantly associated with impaired memory function, the enhanced functional connectivity, and the decreased morphological connectivity.

CONCLUSION

This study suggests that abnormal multimodal connections within DMN and SN can be used as effective biomarkers to identify SCD and provide insight into understanding the pathophysiological mechanisms underlying SCD.

KEY POINTS

• Multimodal brain networks improve the detection accuracy of SCD. • Abnormal connections within DMN and SN can be used as effective biomarkers for the identification of SCD.

Diagnostic power of resting-state fMRI for detection of network connectivity in Alzheimer's disease and mild cognitive impairment: A systematic review

Resting‐state fMRI (rs‐fMRI) detects functional connectivity (FC) abnormalities that occur in the brains of patients with Alzheimer's disease (AD) and mild cognitive impairment (MCI). FC of the default mode network (DMN) is commonly impaired in AD and MCI. We conducted a systematic review aimed at determining the diagnostic power of rs‐fMRI to identify FC abnormalities in the DMN of patients with AD or MCI compared with healthy controls (HCs) using machine learning (ML) methods. Multimodal support vector machine (SVM) algorithm was the commonest form of ML method utilized. Multiple kernel approach can be utilized to aid in the classification by incorporating various discriminating features, such as FC graphs based on “nodes” and “edges” together with structural MRI‐based regional cortical thickness and gray matter volume. Other multimodal features include neuropsychiatric testing scores, DTI features, and regional cerebral blood flow. Among AD patients, the posterior cingulate cortex (PCC)/Precuneus was noted to be a highly affected hub of the DMN that demonstrated overall reduced FC. Whereas reduced DMN FC between the PCC and anterior cingulate cortex (ACC) was observed in MCI patients. Evidence indicates that the nodes of the DMN can offer moderate to high diagnostic power to distinguish AD and MCI patients. Nevertheless, various concerns over the homogeneity of data based on patient selection, scanner effects, and the variable usage of classifiers and algorithms pose a challenge for ML‐based image interpretation of rs‐fMRI datasets to become a mainstream option for diagnosing AD and predicting the conversion of HC/MCI to AD.

Prediction of unenhanced lesion evolution in multiple sclerosis using radiomics-based models: a machine learning approach

BACKGROUND

The volume change of multiple sclerosis (MS) lesion is related to its activity and can be used to assess disease progression. Therefore, the purpose of this study was to develop radiomics models for predicting the evolution of unenhanced MS lesions by using different kinds of machine learning algorithms and explore the optimal model.

METHODS

In this prospective observation, 45 follow-up MR images obtained in 36 patients with MS (mean age 32.53±10.91; 23 women, 13 men) were evaluated. The lesions will be defined as interval activity and interval inactivity, respectively, based on the percentage of enlargement or reduction of the lesion >20% in the follow-up MR images. We extracted radiomic features of lesions on FLAIR images, and used recursive feature elimination (RFE), ReliefF algorithm and least absolute shrinkage and selection operator (LASSO) for feature selection, then three classification models including logistic regression, random forest and support vector machine (SVM) were used to build predictive models. The performance of the models were evaluated based on the sensitivity, specificity, precision, negative predictive value (NPV) and receiver operating characteristic curve (ROC) curves analyses.

RESULTS

135 interval inactivity lesions and 110 interval activity lesions were registered in our study. A total of 972 radiomics features were extracted, of which 265 were robust. The consistency and effectiveness of model performance were compared and verified by different combinations of feature selection and machine learning methods in different K-fold cross-validation strategies where K ranges from 5 to 10, thus demonstrating the stability and robustness. SVM classifier with ReliefF algorithm had the best prediction performance with an average accuracy of 0.827, sensitivity of 0.809, specificity of 0.841, precision of 0.921, NPV of 0.948 and the areas under the ROC curves (AUC) of 0.857 (95% CI: 0.812-0.902) in the cohorts.

CONCLUSION

The results demonstrated that the radiomics-based machine learning model has potential in predicting the evolution of MS lesions.

Individual variation underlying brain age estimates in typical development

Typical brain development follows a protracted trajectory throughout childhood and adolescence. Deviations from typical growth trajectories have been implicated in neurodevelopmental and psychiatric disorders. Recently, the use of machine learning algorithms to model age as a function of structural or functional brain properties has been used to examine advanced or delayed brain maturation in healthy and clinical populations. Termed 'brain age', this approach often relies on complex, nonlinear models that can be difficult to interpret. In this study, we use model explanation methods to examine the cortical features that contribute to brain age modelling on an individual basis. In a large cohort of n = 768 typically-developing children (aged 3-21 years), we build models of brain development using three different machine learning approaches. We employ SHAP, a model-agnostic technique to identify sample-specific feature importance, to identify regional cortical metrics that explain errors in brain age prediction. We find that, on average, brain age prediction and the cortical features that explain model predictions are consistent across model types and reflect previously reported patterns of regions brain development. However, while several regions are found to contribute to brain age prediction error, we find little spatial correspondence between individual estimates of feature importance, even when matched for age, sex and brain age prediction error. We also find no association between brain age error and cognitive performance in this typically-developing sample. Overall, this study shows that, while brain age estimates based on cortical development are relatively robust and consistent across model types and preprocessing strategies, significant between-subject variation exists in the features that explain erroneous brain age predictions on an individual level.

Reproducible Evaluation of Diffusion MRI Features for Automatic Classification of Patients with Alzheimer's Disease

Diffusion MRI is the modality of choice to study alterations of white matter. In past years, various works have used diffusion MRI for automatic classification of Alzheimer's disease. However, classification performance obtained with different approaches is difficult to compare because of variations in components such as input data, participant selection, image preprocessing, feature extraction, feature rescaling (FR), feature selection (FS) and cross-validation (CV) procedures. Moreover, these studies are also difficult to reproduce because these different components are not readily available. In a previous work (Samper-González et al. 2018), we propose an open-source framework for the reproducible evaluation of AD classification from T1-weighted (T1w) MRI and PET data. In the present paper, we first extend this framework to diffusion MRI data. Specifically, we add: conversion of diffusion MRI ADNI data into the BIDS standard and pipelines for diffusion MRI preprocessing and feature extraction. We then apply the framework to compare different components. First, FS has a positive impact on classification results: highest balanced accuracy (BA) improved from 0.76 to 0.82 for task CN vs AD. Secondly, voxel-wise features generally gives better performance than regional features. Fractional anisotropy (FA) and mean diffusivity (MD) provided comparable results for voxel-wise features. Moreover, we observe that the poor performance obtained in tasks involving MCI were potentially caused by the small data samples, rather than by the data imbalance. Furthermore, no extensive classification difference exists for different degree of smoothing and registration methods. Besides, we demonstrate that using non-nested validation of FS leads to unreliable and over-optimistic results: 5% up to 40% relative increase in BA. Lastly, with proper FR and FS, the performance of diffusion MRI features is comparable to that of T1w MRI. All the code of the framework and the experiments are publicly available: general-purpose tools have been integrated into the Clinica software package ( www.clinica.run ) and the paper-specific code is available at: https://github.com/aramis-lab/AD-ML .

Multimodal deep learning models for early detection of Alzheimer's disease stage

Most current Alzheimer's disease (AD) and mild cognitive disorders (MCI) studies use single data modality to make predictions such as AD stages. The fusion of multiple data modalities can provide a holistic view of AD staging analysis. Thus, we use deep learning (DL) to integrally analyze imaging (magnetic resonance imaging (MRI)), genetic (single nucleotide polymorphisms (SNPs)), and clinical test data to classify patients into AD, MCI, and controls (CN). We use stacked denoising auto-encoders to extract features from clinical and genetic data, and use 3D-convolutional neural networks (CNNs) for imaging data. We also develop a novel data interpretation method to identify top-performing features learned by the deep-models with clustering and perturbation analysis. Using Alzheimer's disease neuroimaging initiative (ADNI) dataset, we demonstrate that deep models outperform shallow models, including support vector machines, decision trees, random forests, and k-nearest neighbors. In addition, we demonstrate that integrating multi-modality data outperforms single modality models in terms of accuracy, precision, recall, and meanF1 scores. Our models have identified hippocampus, amygdala brain areas, and the Rey Auditory Verbal Learning Test (RAVLT) as top distinguished features, which are consistent with the known AD literature.

Multimodal MRI assessment for first episode psychosis: A major change in the thalamus and an efficient stratification of a subgroup

Multi‐institutional brain imaging studies have emerged to resolve conflicting results among individual studies. However, adjusting multiple variables at the technical and cohort levels is challenging. Therefore, it is important to explore approaches that provide meaningful results from relatively small samples at institutional levels. We studied 87 first episode psychosis (FEP) patients and 62 healthy subjects by combining supervised integrated factor analysis (SIFA) with a novel pipeline for automated structure‐based analysis, an efficient and comprehensive method for dimensional data reduction that our group recently established. We integrated multiple MRI features (volume, DTI indices, resting state fMRI—rsfMRI) in the whole brain of each participant in an unbiased manner. The automated structure‐based analysis showed widespread DTI abnormalities in FEP and rs‐fMRI differences between FEP and healthy subjects mostly centered in thalamus. The combination of multiple modalities with SIFA was more efficient than the use of single modalities to stratify a subgroup of FEP (individuals with schizophrenia or schizoaffective disorder) that had more robust deficits from the overall FEP group. The information from multiple MRI modalities and analytical methods highlighted the thalamus as significantly abnormal in FEP. This study serves as a proof‐of‐concept for the potential of this methodology to reveal disease underpins and to stratify populations into more homogeneous sub‐groups.

An fMRI Feature Selection Method Based on a Minimum Spanning Tree for Identifying Patients with Autism

Autism spectrum disorder (ASD) is a neurodevelopmental disorder originating in infancy and childhood that may cause language barriers and social difficulties. However, in the diagnosis of ASD, the current machine learning methods still face many challenges in determining the location of biomarkers. Here, we proposed a novel feature selection method based on the minimum spanning tree (MST) to seek neuromarkers for ASD. First, we constructed an undirected graph with nodes of candidate features. At the same time, a weight calculation method considering both feature redundancy and discriminant ability was introduced. Second, we utilized the Prim algorithm to construct the MST from the initial graph structure. Third, the sum of the edge weights of all connected nodes was sorted for each node in the MST. Then, N features corresponding to the nodes with the first N smallest sum were selected as classification features. Finally, the support vector machine (SVM) algorithm was used to evaluate the discriminant performance of the aforementioned feature selection method. Comparative experiments results show that our proposed method has improved the ASD classification performance, i.e., the accuracy, sensitivity, and specificity were 86.7%, 87.5%, and 85.7%, respectively.

Supervised machine learning for diagnostic classification from large-scale neuroimaging datasets

There are growing concerns about the generalizability of machine learning classifiers in neuroimaging. In order to evaluate this aspect across relatively large heterogeneous populations, we investigated four disorders: Autism spectrum disorder (N = 988), Attention deficit hyperactivity disorder (N = 930), Post-traumatic stress disorder (N = 87) and Alzheimer's disease (N = 132). We applied 18 different machine learning classifiers (based on diverse principles) wherein the training/validation and the hold-out test data belonged to samples with the same diagnosis but differing in either the age range or the acquisition site. Our results indicate that overfitting can be a huge problem in heterogeneous datasets, especially with fewer samples, leading to inflated measures of accuracy that fail to generalize well to the general clinical population. Further, different classifiers tended to perform well on different datasets. In order to address this, we propose a consensus-classifier by combining the predictive power of all 18 classifiers. The consensus-classifier was less sensitive to unmatched training/validation and holdout test data. Finally, we combined feature importance scores obtained from all classifiers to infer the discriminative ability of connectivity features. The functional connectivity patterns thus identified were robust to the classification algorithm used, age and acquisition site differences, and had diagnostic predictive ability in addition to univariate statistically significant group differences between the groups. A MATLAB toolbox called Machine Learning in NeuroImaging (MALINI), which implements all the 18 different classifiers along with the consensus classifier is available from Lanka et al. (2019) The toolbox can also be found at the following URL: https://github.com/pradlanka/malini .