An Effective Multimodal Image Fusion Method Using MRI and PET for Alzheimer's Disease Diagnosis

Latent diffusion model-based MRI superresolution enhances mild cognitive impairment prognostication and Alzheimer's disease classification

INTRODUCTION

Timely diagnosis and prognostication of Alzheimer's disease (AD) and mild cognitive impairment (MCI) are pivotal for effective intervention. Artificial intelligence (AI) in neuroradiology may aid in such appropriate diagnosis and prognostication. This study aimed to evaluate the potential of novel diffusion model-based AI for enhancing AD and MCI diagnosis through superresolution (SR) of brain magnetic resonance (MR) images.

METHODS

1.5T brain MR scans of patients with AD or MCI and healthy controls (NC) from Alzheimer's Disease Neuroimaging Initiative 1 (ADNI1) were superresolved to 3T using a novel diffusion model-based generative AI (d3T*) and a convolutional neural network-based model (c3T*). Comparisons of image quality to actual 1.5T and 3T MRI were conducted based on signal-to-noise ratio (SNR), naturalness image quality evaluator (NIQE), and Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE). Voxel-based volumetric analysis was then conducted to study whether 3T* images offered more accurate volumetry than 1.5T images. Binary and multiclass classifications of AD, MCI, and NC were conducted to evaluate whether 3T* images offered superior AD classification performance compared to actual 1.5T MRI. Moreover, CNN-based classifiers were used to predict conversion of MCI to AD, to evaluate the prognostication performance of 3T* images. The classification performances were evaluated using accuracy, sensitivity, specificity, F1 score, Matthews correlation coefficient (MCC), and area under the receiver-operating curves (AUROC).

RESULTS

Analysis of variance (ANOVA) detected significant differences in image quality among the 1.5T, c3T*, d3T*, and 3T groups across all metrics. Both c3T* and d3T* showed superior image quality compared to 1.5T MRI in NIQE and BRISQUE with statistical significance. While the hippocampal volumes measured in 3T* and 3T images were not significantly different, the hippocampal volume measured in 1.5T images showed significant difference. 3T*-based AD classifications showed superior performance across all performance metrics compared to 1.5T-based AD classification. Classification performance between d3T* and actual 3T was not significantly different. 3T* images offered superior accuracy in predicting the conversion of MCI to AD than 1.5T images did.

CONCLUSIONS

The diffusion model-based MRI SR enhances the resolution of brain MR images, significantly improving diagnostic and prognostic accuracy for AD and MCI. Superresolved 3T* images closely matched actual 3T MRIs in quality and volumetric accuracy, and notably improved the prediction performance of conversion from MCI to AD.

A review of deep learning-based information fusion techniques for multimodal medical image classification

Multimodal medical imaging plays a pivotal role in clinical diagnosis and research, as it combines information from various imaging modalities to provide a more comprehensive understanding of the underlying pathology. Recently, deep learning-based multimodal fusion techniques have emerged as powerful tools for improving medical image classification. This review offers a thorough analysis of the developments in deep learning-based multimodal fusion for medical classification tasks. We explore the complementary relationships among prevalent clinical modalities and outline three main fusion schemes for multimodal classification networks: input fusion, intermediate fusion (encompassing single-level fusion, hierarchical fusion, and attention-based fusion), and output fusion. By evaluating the performance of these fusion techniques, we provide insight into the suitability of different network architectures for various multimodal fusion scenarios and application domains. Furthermore, we delve into challenges related to network architecture selection, handling incomplete multimodal data management, and the potential limitations of multimodal fusion. Finally, we spotlight the promising future of Transformer-based multimodal fusion techniques and give recommendations for future research in this rapidly evolving field.

ADHD diagnosis guided by functional brain networks combined with domain knowledge

Utilizing functional magnetic resonance imaging (fMRI) to model functional brain networks (FBNs) is increasingly prominent in attention-deficit/hyperactivity disorder (ADHD) research, revealing neural impact and mechanisms through the exploration of activated brain regions. However, current FBNs-based methods face two major challenges. The primary challenge stems from the limitations of existing modeling methods in accurately capturing both regional correlations and long-distance dependencies (LDDs) within the dynamic brain, thereby affecting the diagnostic accuracy of FBNs as biomarkers. Additionally, limited sample size and class imbalance also pose a challenge to the learning performance of the model. To address the issues, we propose an automated diagnostic framework, which integrates modeling, multimodal fusion, and classification into a unified process. It aims to extract representative FBNs and efficiently incorporate domain knowledge to guide ADHD classification. Our work mainly includes three-fold: 1) A multi-head attention-based region-enhancement module (MAREM) is designed to simultaneously capture regional correlations and LDDs across the entire sequence of brain activity, which facilitates the construction of representative FBNs. 2) The multimodal supplementary learning module (MSLM) is proposed to integrate domain knowledge from phenotype data with FBNs from neuroimaging data, achieving information complementarity and alleviating the problems of insufficient medical data and unbalanced sample categories. 3) An ADHD automatic diagnosis framework guided by FBNs and domain knowledge (ADF-FAD) is proposed to help doctors make more accurate decisions, which is applied to the ADHD-200 dataset to confirm its effectiveness. The results indicate that the FBNs extracted by MAREM perform well in modeling and classification. After with MSLM, the model achieves accuracy of 92.4%, 74.4%, and 80% at NYU, PU, and KKI, respectively, demonstrating its ability to effectively capture crucial information related to ADHD diagnosis. Codes are available at https://github.com/zhuimengxuebao/ADF-FAD.

MACFNet: Detection of Alzheimer's disease via multiscale attention and cross-enhancement fusion network

BACKGROUND AND OBJECTIVE

Alzheimer's disease (AD) is a dreaded degenerative disease that results in a profound decline in human cognition and memory. Due to its intricate pathogenesis and the lack of effective therapeutic interventions, early diagnosis plays a paramount role in AD. Recent research based on neuroimaging has shown that the application of deep learning methods by multimodal neural images can effectively detect AD. However, these methods only concatenate and fuse the high-level features extracted from different modalities, ignoring the fusion and interaction of low-level features across modalities. It consequently leads to unsatisfactory classification performance.

METHOD

In this paper, we propose a novel multi-scale attention and cross-enhanced fusion network, MACFNet, which enables the interaction of multi-stage low-level features between inputs to learn shared feature representations. We first construct a novel Cross-Enhanced Fusion Module (CEFM), which fuses low-level features from different modalities through a multi-stage cross-structure. In addition, an Efficient Spatial Channel Attention (ECSA) module is proposed, which is able to focus on important AD-related features in images more efficiently and achieve feature enhancement from different modalities through two-stage residual concatenation. Finally, we also propose a multiscale attention guiding block (MSAG) based on dilated convolution, which can obtain rich receptive fields without increasing model parameters and computation, and effectively improve the efficiency of multiscale feature extraction.

RESULTS

Experiments on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset demonstrate that our MACFNet has better classification performance than existing multimodal methods, with classification accuracies of 99.59 %, 98.85 %, 99.61 %, and 98.23 % for AD vs. CN, AD vs. MCI, CN vs. MCI and AD vs. CN vs. MCI, respectively, and specificity of 98.92 %, 97.07 %, 99.58 % and 99.04 %, and sensitivity of 99.91 %, 99.89 %, 99.63 % and 97.75 %, respectively.

CONCLUSIONS

The proposed MACFNet is a high-accuracy multimodal AD diagnostic framework. Through the cross mechanism and efficient attention, MACFNet can make full use of the low-level features of different modal medical images and effectively pay attention to the local and global information of the images. This work provides a valuable reference for multi-mode AD diagnosis.

Machine learning with multimodal neuroimaging data to classify stages of Alzheimer's disease: a systematic review and meta-analysis

In recent years, Alzheimer's disease (AD) has been a serious threat to human health. Researchers and clinicians alike encounter a significant obstacle when trying to accurately identify and classify AD stages. Several studies have shown that multimodal neuroimaging input can assist in providing valuable insights into the structural and functional changes in the brain related to AD. Machine learning (ML) algorithms can accurately categorize AD phases by identifying patterns and linkages in multimodal neuroimaging data using powerful computational methods. This study aims to assess the contribution of ML methods to the accurate classification of the stages of AD using multimodal neuroimaging data. A systematic search is carried out in IEEE Xplore, Science Direct/Elsevier, ACM DigitalLibrary, and PubMed databases with forward snowballing performed on Google Scholar. The quantitative analysis used 47 studies. The explainable analysis was performed on the classification algorithm and fusion methods used in the selected studies. The pooled sensitivity and specificity, including diagnostic efficiency, were evaluated by conducting a meta-analysis based on a bivariate model with the hierarchical summary receiver operating characteristics (ROC) curve of multimodal neuroimaging data and ML methods in the classification of AD stages. Wilcoxon signed-rank test is further used to statistically compare the accuracy scores of the existing models. With a 95% confidence interval of 78.87-87.71%, the combined sensitivity for separating participants with mild cognitive impairment (MCI) from healthy control (NC) participants was 83.77%; for separating participants with AD from NC, it was 94.60% (90.76%, 96.89%); for separating participants with progressive MCI (pMCI) from stable MCI (sMCI), it was 80.41% (74.73%, 85.06%). With a 95% confidence interval (78.87%, 87.71%), the Pooled sensitivity for distinguishing mild cognitive impairment (MCI) from healthy control (NC) participants was 83.77%, with a 95% confidence interval (90.76%, 96.89%), the Pooled sensitivity for distinguishing AD from NC was 94.60%, likewise (MCI) from healthy control (NC) participants was 83.77% progressive MCI (pMCI) from stable MCI (sMCI) was 80.41% (74.73%, 85.06%), and early MCI (EMCI) from NC was 86.63% (82.43%, 89.95%). Pooled specificity for differentiating MCI from NC was 79.16% (70.97%, 87.71%), AD from NC was 93.49% (91.60%, 94.90%), pMCI from sMCI was 81.44% (76.32%, 85.66%), and EMCI from NC was 85.68% (81.62%, 88.96%). The Wilcoxon signed rank test showed a low P-value across all the classification tasks. Multimodal neuroimaging data with ML is a promising future in classifying the stages of AD but more research is required to increase the validity of its application in clinical practice.

Automated detection of Alzheimer's disease: a multi-modal approach with 3D MRI and amyloid PET

Recent advances in deep learning and imaging technologies have revolutionized automated medical image analysis, especially in diagnosing Alzheimer's disease through neuroimaging. Despite the availability of various imaging modalities for the same patient, the development of multi-modal models leveraging these modalities remains underexplored. This paper addresses this gap by proposing and evaluating classification models using 2D and 3D MRI images and amyloid PET scans in uni-modal and multi-modal frameworks. Our findings demonstrate that models using volumetric data learn more effective representations than those using only 2D images. Furthermore, integrating multiple modalities enhances model performance over single-modality approaches significantly. We achieved state-of-the-art performance on the OASIS-3 cohort. Additionally, explainability analyses with Grad-CAM indicate that our model focuses on crucial AD-related regions for its predictions, underscoring its potential to aid in understanding the disease's causes.

Applications of machine learning and deep learning in SPECT and PET imaging: General overview, challenges and future prospects

The integration of positron emission tomography (PET) and single-photon emission computed tomography (SPECT) imaging techniques with machine learning (ML) algorithms, including deep learning (DL) models, is a promising approach. This integration enhances the precision and efficiency of current diagnostic and treatment strategies while offering invaluable insights into disease mechanisms. In this comprehensive review, we delve into the transformative impact of ML and DL in this domain. Firstly, a brief analysis is provided of how these algorithms have evolved and which are the most widely applied in this domain. Their different potential applications in nuclear imaging are then discussed, such as optimization of image adquisition or reconstruction, biomarkers identification, multimodal fusion and the development of diagnostic, prognostic, and disease progression evaluation systems. This is because they are able to analyse complex patterns and relationships within imaging data, as well as extracting quantitative and objective measures. Furthermore, we discuss the challenges in implementation, such as data standardization and limited sample sizes, and explore the clinical opportunities and future horizons, including data augmentation and explainable AI. Together, these factors are propelling the continuous advancement of more robust, transparent, and reliable systems.

EAMNet: an Alzheimer's disease prediction model based on representation learning

Objective. Brain18F-FDG PET images indicate brain lesions' metabolic status and offer the predictive potential for Alzheimer's disease (AD). However, the complexity of extracting relevant lesion features and dealing with extraneous information in PET images poses challenges for accurate prediction.Approach. To address these issues, we propose an innovative solution called the efficient adaptive multiscale network (EAMNet) for predicting potential patient populations using positron emission tomography (PET) image slices, enabling effective intervention and treatment. Firstly, we introduce an efficient convolutional strategy to enhance the receptive field of PET images during the feature learning process, avoiding excessive extraction of fine tissue features by deep-level networks while reducing the model's computational complexity. Secondly, we construct a channel attention module that enables the prediction model to adaptively allocate weights between different channels, compensating for the spatial noise in PET images' impact on classification. Finally, we use skip connections to merge features from different-scale lesion information. Through visual analysis, the network constructed in this article aligns with the regions of interest of clinical doctors.Main results. Through visualization analysis, our network aligns with regions of interest identified by clinical doctors. Experimental evaluations conducted on the ADNI (Alzheimer's Disease Neuroimaging Initiative) dataset demonstrate the outstanding classification performance of our proposed method. The accuracy rates for AD versus NC (Normal Controls), AD versus MCI (Mild Cognitive Impairment), MCI versus NC, and AD versus MCI versus NC classifications achieve 97.66%, 96.32%, 95.23%, and 95.68%, respectively.Significance. The proposed method surpasses advanced algorithms in the field, providing a hopeful advancement in accurately predicting and classifying Alzheimer's Disease using18F-FDG PET images. The source code has been uploaded tohttps://github.com/Haoliang-D-AHU/EAMNet/tree/master.

Optimized Convolutional Fusion for Multimodal Neuroimaging in Alzheimer's Disease Diagnosis: Enhancing Data Integration and Feature Extraction

Multimodal neuroimaging has gained traction in Alzheimer's Disease (AD) diagnosis by integrating information from multiple imaging modalities to enhance classification accuracy. However, effectively handling heterogeneous data sources and overcoming the challenges posed by multiscale transform methods remains a significant hurdle. This article proposes a novel approach to address these challenges. To harness the power of diverse neuroimaging data, we employ a strategy that leverages optimized convolution techniques. These optimizations include varying kernel sizes and the incorporation of instance normalization, both of which play crucial roles in feature extraction from magnetic resonance imaging (MRI) and positron emission tomography (PET) images. Specifically, varying kernel sizes allow us to adapt the receptive field to different image characteristics, enhancing the model's ability to capture relevant information. Furthermore, we employ transposed convolution, which increases spatial resolution of feature maps, and it is optimized with varying kernel sizes and instance normalization. This heightened resolution facilitates the alignment and integration of data from disparate MRI and PET data. The use of larger kernels and strides in transposed convolution expands the receptive field, enabling the model to capture essential cross-modal relationships. Instance normalization, applied to each modality during the fusion process, mitigates potential biases stemming from differences in intensity, contrast, or scale between modalities. This enhancement contributes to improved model performance by reducing complexity and ensuring robust fusion. The performance of the proposed fusion method is assessed on three distinct neuroimaging datasets, which include: Alzheimer's Disease Neuroimaging Initiative (ADNI), consisting of 50 participants each at various stages of AD for both MRI and PET (Cognitive Normal, AD, and Early Mild Cognitive); Open Access Series of Imaging Studies (OASIS), consisting of 50 participants each at various stages of AD for both MRI and PET (Cognitive Normal, Mild Dementia, Very Mild Dementia); and whole-brain atlas neuroimaging (AANLIB) (consisting of 50 participants each at various stages of AD for both MRI and PET (Cognitive Normal, AD). To evaluate the quality of the fused images generated via our method, we employ a comprehensive set of evaluation metrics, including Structural Similarity Index Measurement (SSIM), which assesses the structural similarity between two images; Peak Signal-to-Noise Ratio (PSNR), which measures how closely the generated image resembles the ground truth; Entropy (E), which assesses the amount of information preserved or lost during fusion; the Feature Similarity Indexing Method (FSIM), which assesses the structural and feature similarities between two images; and Edge-Based Similarity (EBS), which measures the similarity of edges between the fused and ground truth images. The obtained fused image is further evaluated using a Mobile Vision Transformer. In the classification of AD vs. Cognitive Normal, the model achieved an accuracy of 99.00%, specificity of 99.00%, and sensitivity of 98.44% on the AANLIB dataset.

Hybrid Multimodality Fusion with Cross-Domain Knowledge Transfer to Forecast Progression Trajectories in Cognitive Decline

Magnetic resonance imaging (MRI) and positron emission tomography (PET) are increasingly used to forecast progression trajectories of cognitive decline caused by preclinical and prodromal Alzheimer's disease (AD). Many existing studies have explored the potential of these two distinct modalities with diverse machine and deep learning approaches. But successfully fusing MRI and PET can be complex due to their unique characteristics and missing modalities. To this end, we develop a hybrid multimodality fusion (HMF) framework with cross-domain knowledge transfer for joint MRI and PET representation learning, feature fusion, and cognitive decline progression forecasting. Our HMF consists of three modules: 1) a module to impute missing PET images, 2) a module to extract multimodality features from MRI and PET images, and 3) a module to fuse the extracted multimodality features. To address the issue of small sample sizes, we employ a cross-domain knowledge transfer strategy from the ADNI dataset, which includes 795 subjects, to independent small-scale AD-related cohorts, in order to leverage the rich knowledge present within the ADNI. The proposed HMF is extensively evaluated in three AD-related studies with 272 subjects across multiple disease stages, such as subjective cognitive decline and mild cognitive impairment. Experimental results demonstrate the superiority of our method over several state-of-the-art approaches in forecasting progression trajectories of AD-related cognitive decline.

DAUF: A disease-related attentional UNet framework for progressive and stable mild cognitive impairment identification

Identifying progressive mild cognitive impairment (pMCI) and stable mild cognitive impairment (sMCI) plays a significant role in early Alzheimer's disease (AD) diagnosis, which can effectively boost the life quality of patients. Recently, convolutional neural network (CNN)- based methods using structural magnetic resonance imaging (sMRI) images have shown effective for AD identification. However, these CNN-based methods fail to effectively explore the feature extraction of disease-related multi-scale tissues, such as ventricles, hippocampi and cerebral cortex. To address this issue, we propose an end-to-end disease-related attentional UNet framework (DAUF) for identifying pMCI and sMCI, by embedding a devised dual disease-related attention module (D2AM) and a novel tree-structured feature fusion classifier (TFFC). Specifically, D2AM leverages the complementarity between feature maps and attention maps and the complementary features from the encoder and decoder, so as to highlight discriminative semantic and detailed features. Additionally, TFFC is a powerfully joint multi-scale feature fusion and classification head, by employing the homogeneity among multi-scale features, so that the discriminative features of the multi-scale tissues are adequately fused for enhancing classification performance. Finally, extensive experiments demonstrate the superior performance of DAUF, with the effectiveness of D2AM and TFFC on identifying pMCI and sMCI subjects.

A Mutual Knowledge Distillation-Empowered AI Framework for Early Detection of Alzheimer's Disease Using Incomplete Multi-Modal Images

Early detection of Alzheimer's Disease (AD) is crucial to ensure timely interventions and optimize treatment outcomes for patients. While integrating multi-modal neuroimages, such as MRI and PET, has shown great promise, limited research has been done to effectively handle incomplete multi-modal image datasets in the integration. To this end, we propose a deep learning-based framework that employs Mutual Knowledge Distillation (MKD) to jointly model different sub-cohorts based on their respective available image modalities. In MKD, the model with more modalities (e.g., MRI and PET) is considered a teacher while the model with fewer modalities (e.g., only MRI) is considered a student. Our proposed MKD framework includes three key components: First, we design a teacher model that is student-oriented, namely the Student-oriented Multi-modal Teacher (SMT), through multi-modal information disentanglement. Second, we train the student model by not only minimizing its classification errors but also learning from the SMT teacher. Third, we update the teacher model by transfer learning from the student's feature extractor because the student model is trained with more samples. Evaluations on Alzheimer's Disease Neuroimaging Initiative (ADNI) datasets highlight the effectiveness of our method. Our work demonstrates the potential of using AI for addressing the challenges of incomplete multi-modal neuroimage datasets, opening new avenues for advancing early AD detection and treatment strategies.

Decoupled learning for brain image registration

Image registration is one of the important parts in medical image processing and intelligent analysis. The accuracy of image registration will greatly affect the subsequent image processing and analysis. This paper focuses on the problem of brain image registration based on deep learning, and proposes the unsupervised deep learning methods based on model decoupling and regularization learning. Specifically, we first decompose the highly ill-conditioned inverse problem of brain image registration into two simpler sub-problems, to reduce the model complexity. Further, two light neural networks are constructed to approximate the solution of the two sub-problems and the training strategy of alternating iteration is used to solve the problem. The performance of algorithms utilizing model decoupling is evaluated through experiments conducted on brain MRI images from the LPBA40 dataset. The obtained experimental results demonstrate the superiority of the proposed algorithm over conventional learning methods in the context of brain image registration tasks.

Detection of β-amyloid aggregates/plaques in 5xFAD mice by labelled native PLGA nanoparticles: implication in the diagnosis of alzheimer's disease

Evidence suggests that increased level/aggregation of β-amyloid (Aβ) peptide, together with enhanced phosphorylation/aggregation of tau protein, play a critical role in the development of Alzheimer's disease (AD), the leading cause of dementia in the elderly. At present, AD diagnosis is based primarily on cognitive assessment, neuroimaging, and immunological assays to detect altered levels/deposition of Aβ peptides and tau protein. While measurement of Aβ and tau in the cerebrospinal fluid/blood can indicate disease status, neuroimaging of aggregated Aβ and tau protein in the brain using positron emission tomography (PET) enable to monitor the pathological changes in AD patients. With advancements in nanomedicine, several nanoparticles, apart from drug-delivery, have been used as a diagnostic agent to identify more accurately changes in AD patients. Recently, we reported that FDA approved native PLGA nanoparticles can interact with Aβ to inhibit its aggregation/toxicity in cellular and animal models of AD. Here, we reveal that fluorescence labelled native PLGA following acute intracerebellar injection can identify majority of the immunostained Aβ as well as Congo red labelled neuritic plaques in the cortex of 5xFAD mice. Labelling of plaques by PLGA is apparent at 1 h, peak around 3 h and then start declining by 24 h after injection. No fluorescent PLGA was detected in the cerebellum of 5xFAD mice or in any brain regions of wild-type control mice following injection. These results provide the very first evidence that native PLGA nanoparticles can be used as a novel nano-theragnostic agent in the treatment as well as diagnosis of AD pathology.

Pareto Optimized Adaptive Learning with Transposed Convolution for Image Fusion Alzheimer's Disease Classification

Alzheimer's disease (AD) is a neurological condition that gradually weakens the brain and impairs cognition and memory. Multimodal imaging techniques have become increasingly important in the diagnosis of AD because they can help monitor disease progression over time by providing a more complete picture of the changes in the brain that occur over time in AD. Medical image fusion is crucial in that it combines data from various image modalities into a single, better-understood output. The present study explores the feasibility of employing Pareto optimized deep learning methodologies to integrate Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) images through the utilization of pre-existing models, namely the Visual Geometry Group (VGG) 11, VGG16, and VGG19 architectures. Morphological operations are carried out on MRI and PET images using Analyze 14.0 software and after which PET images are manipulated for the desired angle of alignment with MRI image using GNU Image Manipulation Program (GIMP). To enhance the network's performance, transposed convolution layer is incorporated into the previously extracted feature maps before image fusion. This process generates feature maps and fusion weights that facilitate the fusion process. This investigation concerns the assessment of the efficacy of three VGG models in capturing significant features from the MRI and PET data. The hyperparameters of the models are tuned using Pareto optimization. The models' performance is evaluated on the ADNI dataset utilizing the Structure Similarity Index Method (SSIM), Peak Signal-to-Noise Ratio (PSNR), Mean-Square Error (MSE), and Entropy (E). Experimental results show that VGG19 outperforms VGG16 and VGG11 with an average of 0.668, 0.802, and 0.664 SSIM for CN, AD, and MCI stages from ADNI (MRI modality) respectively. Likewise, an average of 0.669, 0.815, and 0.660 SSIM for CN, AD, and MCI stages from ADNI (PET modality) respectively.

CsAGP: Detecting Alzheimer's disease from multimodal images via dual-transformer with cross-attention and graph pooling

Alzheimer's disease (AD) is a terrible and degenerative disease commonly occurring in the elderly. Early detection can prevent patients from further damage, which is crucial in treating AD. Over the past few decades, it has been demonstrated that neuroimaging can be a critical diagnostic tool for AD, and the feature fusion of different neuroimaging modalities can enhance diagnostic performance. Most previous studies in multimodal feature fusion have only concatenated the high-level features extracted by neural networks from various neuroimaging images simply. However, a major problem of these studies is over-looking the low-level feature interactions between modalities in the feature extraction stage, resulting in suboptimal performance in AD diagnosis. In this paper, we develop a dual-branch vision transformer with cross-attention and graph pooling, namely CsAGP, which enables multi-level feature interactions between the inputs to learn a shared feature representation. Specifically, we first construct a brand-new cross-attention fusion module (CAFM), which processes MRI and PET images by two independent branches of differing computational complexity. These features are fused merely by the cross-attention mechanism to enhance each other. After that, a concise graph pooling algorithm-based Reshape-Pooling-Reshape (RPR) framework is developed for token selection to reduce token redundancy in the proposed model. Extensive experiments on the Alzheimer's Disease Neuroimaging Initiative (ADNI) database demonstrated that the suggested method obtains 99.04%, 97.43%, 98.57%, and 98.72% accuracy for the classification of AD vs. CN, AD vs. MCI, CN vs. MCI, and AD vs. CN vs. MCI, respectively.

Automated Medical Diagnosis of Alzheimer´s Disease Using an Efficient Net Convolutional Neural Network

Alzheimer's disease (AD) poses an enormous challenge to modern healthcare. Since 2017, researchers have been using deep learning (DL) models for the early detection of AD using neuroimaging biomarkers. In this paper, we implement the EfficietNet-b0 convolutional neural network (CNN) with a novel approach-"fusion of end-to-end and transfer learning"-to classify different stages of AD. 245 T1W MRI scans of cognitively normal (CN) subjects, 229 scans of AD subjects, and 229 scans of subjects with stable mild cognitive impairment (sMCI) were employed. Each scan was preprocessed using a standard pipeline. The proposed models were trained and evaluated using preprocessed scans. For the sMCI vs. AD classification task we obtained 95.29% accuracy and 95.35% area under the curve (AUC) for model training and 93.10% accuracy and 93.00% AUC for model testing. For the multiclass AD vs. CN vs. sMCI classification task we obtained 85.66% accuracy and 86% AUC for model training and 87.38% accuracy and 88.00% AUC for model testing. Based on our experimental results, we conclude that CNN-based DL models can be used to analyze complicated MRI scan features in clinical settings.

OViTAD: Optimized Vision Transformer to Predict Various Stages of Alzheimer's Disease Using Resting-State fMRI and Structural MRI Data

Advances in applied machine learning techniques for neuroimaging have encouraged scientists to implement models to diagnose brain disorders such as Alzheimer's disease at early stages. Predicting the exact stage of Alzheimer's disease is challenging; however, complex deep learning techniques can precisely manage this. While successful, these complex architectures are difficult to interrogate and computationally expensive. Therefore, using novel, simpler architectures with more efficient pattern extraction capabilities, such as transformers, is of interest to neuroscientists. This study introduced an optimized vision transformer architecture to predict the group membership by separating healthy adults, mild cognitive impairment, and Alzheimer's brains within the same age group (>75 years) using resting-state functional (rs-fMRI) and structural magnetic resonance imaging (sMRI) data aggressively preprocessed by our pipeline. Our optimized architecture, known as OViTAD is currently the sole vision transformer-based end-to-end pipeline and outperformed the existing transformer models and most state-of-the-art solutions. Our model achieved F1-scores of 97%±0.0 and 99.55%±0.39 from the testing sets for the rs-fMRI and sMRI modalities in the triple-class prediction experiments. Furthermore, our model reached these performances using 30% fewer parameters than a vanilla transformer. Furthermore, the model was robust and repeatable, producing similar estimates across three runs with random data splits (we reported the averaged evaluation metrics). Finally, to challenge the model, we observed how it handled increasing noise levels by inserting varying numbers of healthy brains into the two dementia groups. Our findings suggest that optimized vision transformers are a promising and exciting new approach for neuroimaging applications, especially for Alzheimer's disease prediction.

A review of the application of three-dimensional convolutional neural networks for the diagnosis of Alzheimer's disease using neuroimaging

Alzheimer's disease (AD) is a degenerative disorder that leads to progressive, irreversible cognitive decline. To obtain an accurate and timely diagnosis and detect AD at an early stage, numerous approaches based on convolutional neural networks (CNNs) using neuroimaging data have been proposed. Because 3D CNNs can extract more spatial discrimination information than 2D CNNs, they have emerged as a promising research direction in the diagnosis of AD. The aim of this article is to present the current state of the art in the diagnosis of AD using 3D CNN models and neuroimaging modalities, focusing on the 3D CNN architectures and classification methods used, and to highlight potential future research topics. To give the reader a better overview of the content mentioned in this review, we briefly introduce the commonly used imaging datasets and the fundamentals of CNN architectures. Then we carefully analyzed the existing studies on AD diagnosis, which are divided into two levels according to their inputs: 3D subject-level CNNs and 3D patch-level CNNs, highlighting their contributions and significance in the field. In addition, this review discusses the key findings and challenges from the studies and highlights the lessons learned as a roadmap for future research. Finally, we summarize the paper by presenting some major findings, identifying open research challenges, and pointing out future research directions.

Multi-feature computational framework for combined signatures of dementia in underrepresented settings

Objective.The differential diagnosis of behavioral variant frontotemporal dementia (bvFTD) and Alzheimer's disease (AD) remains challenging in underrepresented, underdiagnosed groups, including Latinos, as advanced biomarkers are rarely available. Recent guidelines for the study of dementia highlight the critical role of biomarkers. Thus, novel cost-effective complementary approaches are required in clinical settings.Approach. We developed a novel framework based on a gradient boosting machine learning classifier, tuned by Bayesian optimization, on a multi-feature multimodal approach (combining demographic, neuropsychological, magnetic resonance imaging (MRI), and electroencephalography/functional MRI connectivity data) to characterize neurodegeneration using site harmonization and sequential feature selection. We assessed 54 bvFTD and 76 AD patients and 152 healthy controls (HCs) from a Latin American consortium (ReDLat).Main results. The multimodal model yielded high area under the curve classification values (bvFTD patients vs HCs: 0.93 (±0.01); AD patients vs HCs: 0.95 (±0.01); bvFTD vs AD patients: 0.92 (±0.01)). The feature selection approach successfully filtered non-informative multimodal markers (from thousands to dozens).Results. Proved robust against multimodal heterogeneity, sociodemographic variability, and missing data.Significance. The model accurately identified dementia subtypes using measures readily available in underrepresented settings, with a similar performance than advanced biomarkers. This approach, if confirmed and replicated, may potentially complement clinical assessments in developing countries.

End-to-End Deep Learning Architectures Using 3D Neuroimaging Biomarkers for Early Alzheimer’s Diagnosis

This study uses magnetic resonance imaging (MRI) data to propose end-to-end learning implementing volumetric convolutional neural network (CNN) models for two binary classification tasks: Alzheimer’s disease (AD) vs. cognitively normal (CN) and stable mild cognitive impairment (sMCI) vs. AD. The baseline MP-RAGE T1 MR images of 245 AD patients and 229 with sMCI were obtained from the ADNI dataset, whereas 245 T1 MR images of CN people were obtained from the IXI dataset. All of the images were preprocessed in four steps: N4 bias field correction, denoising, brain extraction, and registration. End-to-end-learning-based deep CNNs were used to discern between different phases of AD. Eight CNN-based architectures were implemented and assessed. The DenseNet264 excelled in both types of classification, with 82.5% accuracy and 87.63% AUC for training and 81.03% accuracy for testing relating to the sMCI vs. AD and 100% accuracy and 100% AUC for training and 99.56% accuracy for testing relating to the AD vs. CN. Deep learning approaches based on CNN and end-to-end learning offer a strong tool for examining minute but complex properties in MR images which could aid in the early detection and prediction of Alzheimer’s disease in clinical settings.

Application of artificial intelligence in brain molecular imaging

Initial development of artificial Intelligence (AI) and machine learning (ML) dates back to the mid-twentieth century. A growing awareness of the potential for AI, as well as increases in computational resources, research, and investment are rapidly advancing AI applications to medical imaging and, specifically, brain molecular imaging. AI/ML can improve imaging operations and decision making, and potentially perform tasks that are not readily possible by physicians, such as predicting disease prognosis, and identifying latent relationships from multi-modal clinical information. The number of applications of image-based AI algorithms, such as convolutional neural network (CNN), is increasing rapidly. The applications for brain molecular imaging (MI) include image denoising, PET and PET/MRI attenuation correction, image segmentation and lesion detection, parametric image formation, and the detection/diagnosis of Alzheimer's disease and other brain disorders. When effectively used, AI will likely improve the quality of patient care, instead of replacing radiologists. A regulatory framework is being developed to facilitate AI adaptation for medical imaging.