Multimodal deep learning for Alzheimer's disease dementia assessment

Alzheimer's disease diagnosis from multi-modal data via feature inductive learning and dual multilevel graph neural network

Multi-modal data can provide complementary information of Alzheimer's disease (AD) and its development from different perspectives. Such information is closely related to the diagnosis, prevention, and treatment of AD, and hence it is necessary and critical to study AD through multi-modal data. Existing learning methods, however, usually ignore the influence of feature heterogeneity and directly fuse features in the last stages. Furthermore, most of these methods only focus on local fusion features or global fusion features, neglecting the complementariness of features at different levels and thus not sufficiently leveraging information embedded in multi-modal data. To overcome these shortcomings, we propose a novel framework for AD diagnosis that fuses gene, imaging, protein, and clinical data. Our framework learns feature representations under the same feature space for different modalities through a feature induction learning (FIL) module, thereby alleviating the impact of feature heterogeneity. Furthermore, in our framework, local and global salient multi-modal feature interaction information at different levels is extracted through a novel dual multilevel graph neural network (DMGNN). We extensively validate the proposed method on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset and experimental results demonstrate our method consistently outperforms other state-of-the-art multi-modal fusion methods. The code is publicly available on the GitHub website. (https://github.com/xiankantingqianxue/MIA-code.git).

Perspectives on Advancing Multimodal Learning in Environmental Science and Engineering Studies

The environment faces increasing anthropogenic impacts, resulting in a rapid increase in environmental issues that undermine the natural capital essential for human wellbeing. These issues are complex and often influenced by various factors represented by data with different modalities. While machine learning (ML) provides data-driven tools for addressing the environmental issues, the current ML models in environmental science and engineering (ES&E) often neglect the utilization of multimodal data. With the advancement in deep learning, multimodal learning (MML) holds promise for comprehensive descriptions of the environmental issues by harnessing data from diverse modalities. This advancement has the potential to significantly elevate the accuracy and robustness of prediction models in ES&E studies, providing enhanced solutions for various environmental modeling tasks. This perspective summarizes MML methodologies and proposes potential applications of MML models in ES&E studies, including environmental quality assessment, prediction of chemical hazards, and optimization of pollution control techniques. Additionally, we discuss the challenges associated with implementing MML in ES&E and propose future research directions in this domain.

Synthetic lumbar MRI can aid in diagnosis and treatment strategies based on self-pix networks

CT and MR tools are commonly used to diagnose lumbar fractures (LF). However, numerous limitations have been found in practice. The aims of this study were to innovate and develop a spinal disease-specific neural network and to evaluate whether synthetic MRI of the LF affected clinical diagnosis and treatment strategies. A total of 675 LF patients who met the inclusion and exclusion criteria were included in the study. For each participant, two mid-sagittal CT and T2-weighted MR images were selected; 1350 pairs of LF images were also included. A new Self-pix based on Pix2pix and Self-Attention was constructed. A total of 1350 pairs of CT and MR images, which were randomly divided into a training group (1147 pairs) and a test group (203 pairs), were fed into Pix2pix and Self-pix. The quantitative evaluation included PSNR and SSIM (PSNR1 and SSIM1: real MR images and Pix2pix-generated MR images; PSNR2 and SSIM2: real MR images and Self-pix-generated MR images). The qualitative evaluation, including accurate diagnosis of acute fractures and accurate selection of treatment strategies based on Self-pix-generated MRI, was performed by three spine surgeons. In the LF group, PSNR1 and PSNR2 were 10.884 and 11.021 (p < 0.001), and SSIM1 and SSIM2 were 0.766 and 0.771 (p < 0.001), respectively. In the ROI group, PSNR1 and PSNR2 were 12.350 and 12.670 (p = 0.004), and SSIM1 and SSIM2 were 0.816 and 0.832 (p = 0.005), respectively. According to the qualitative evaluation, Self-pix-generated MRI showed no significant difference from real MRI in identifying acute fractures (p = 0.689), with a good sensitivity of 84.36% and specificity of 96.65%. No difference in treatment strategy was found between the Self-pix-generated MRI group and the real MRI group (p = 0.135). In this study, a disease-specific GAN named Self-pix was developed, which demonstrated better image generation performance compared to traditional GAN. The spine surgeon could accurately diagnose LF and select treatment strategies based on Self-pix-generated T2 MR images.

A transformer-based unified multimodal framework for Alzheimer's disease assessment

In Alzheimer's disease (AD) assessment, traditional deep learning approaches have often employed separate methodologies to handle the diverse modalities of input data. Recognizing the critical need for a cohesive and interconnected analytical framework, we propose the AD-Transformer, a novel transformer-based unified deep learning model. This innovative framework seamlessly integrates structural magnetic resonance imaging (sMRI), clinical, and genetic data from the extensive Alzheimer's Disease Neuroimaging Initiative (ADNI) database, encompassing 1651 subjects. By employing a Patch-CNN block, the AD-Transformer efficiently transforms image data into image tokens, while a linear projection layer adeptly converts non-image data into corresponding tokens. As the core, a transformer block learns comprehensive representations of the input data, capturing the intricate interplay between modalities. The AD-Transformer sets a new benchmark in AD diagnosis and Mild Cognitive Impairment (MCI) conversion prediction, achieving remarkable average area under curve (AUC) values of 0.993 and 0.845, respectively, surpassing those of traditional image-only models and non-unified multimodal models. Our experimental results confirmed the potential of the AD-Transformer as a potent tool in AD diagnosis and MCI conversion prediction. By providing a unified framework that jointly learns holistic representations of both image and non-image data, the AD-Transformer paves the way for more effective and precise clinical assessments, offering a clinically adaptable strategy for leveraging diverse data modalities in the battle against AD.

3D Multimodal Fusion Network With Disease-Induced Joint Learning for Early Alzheimer's Disease Diagnosis

Multimodal neuroimaging provides complementary information critical for accurate early diagnosis of Alzheimer's disease (AD). However, the inherent variability between multimodal neuroimages hinders the effective fusion of multimodal features. Moreover, achieving reliable and interpretable diagnoses in the field of multimodal fusion remains challenging. To address them, we propose a novel multimodal diagnosis network based on multi-fusion and disease-induced learning (MDL-Net) to enhance early AD diagnosis by efficiently fusing multimodal data. Specifically, MDL-Net proposes a multi-fusion joint learning (MJL) module, which effectively fuses multimodal features and enhances the feature representation from global, local, and latent learning perspectives. MJL consists of three modules, global-aware learning (GAL), local-aware learning (LAL), and outer latent-space learning (LSL) modules. GAL via a self-adaptive Transformer (SAT) learns the global relationships among the modalities. LAL constructs local-aware convolution to learn the local associations. LSL module introduces latent information through outer product operation to further enhance feature representation. MDL-Net integrates the disease-induced region-aware learning (DRL) module via gradient weight to enhance interpretability, which iteratively learns weight matrices to identify AD-related brain regions. We conduct the extensive experiments on public datasets and the results confirm the superiority of our proposed method. Our code will be available at: https://github.com/qzf0320/MDL-Net.

Deep learning to predict risk of lateral skull base cerebrospinal fluid leak or encephalocele

PURPOSE

Skull base features, including increased foramen ovale (FO) cross-sectional area, are associated with lateral skull base spontaneous cerebrospinal fluid (sCSF) leak and encephalocele. Manual measurement requires skill in interpreting imaging studies and is time consuming. The goal of this study was to develop a fully automated deep learning method for FO segmentation and to determine the predictive value in identifying patients with sCSF leak or encephalocele.

METHODS

A retrospective cohort study at a tertiary care academic hospital of 34 adults with lateral skull base sCSF leak or encephalocele were compared with 815 control patients from 2013-2021. A convolutional neural network (CNN) was constructed for image segmentation of axial computed tomography (CT) studies. Predicted FO segmentations were compared to manual segmentations, and receiver operating characteristic (ROC) curves were constructed.

RESULTS

295 CTs were used for training and validation of the CNN. A separate dataset of 554 control CTs was matched 5:1 on age and sex with the sCSF leak/encephalocele group. The mean Dice score was 0.81. The sCSF leak/encephalocele group had greater mean (SD) FO cross-sectional area compared to the control group, 29.0 (7.7) mm2 versus 24.3 (7.6) mm2 (P = .002, 95% confidence interval 0.02-0.08). The area under the ROC curve was 0.69.

CONCLUSION

CNNs can be used to segment the cross-sectional area of the FO accurately and efficiently. Used together with other predictors, this method could be used as part of a clinical tool to predict the risk of sCSF leak or encephalocele.

MAFT-SO: A novel multi-atlas fusion template based on spatial overlap for ASD diagnosis

Autism spectrum disorder (ASD) is a common neurological condition. Early diagnosis and treatment are essential for enhancing the life quality of individuals with ASD. However, most existing studies either focus solely on the brain networks of subjects within a single atlas or merely employ simple matrix concatenation to represent the fusion of multi-atlas. These approaches neglected the natural spatial overlap that exists between brain regions across multi-atlas and did not fully capture the comprehensive information of brain regions under different atlases. To tackle this weakness, in this paper, we propose a novel multi-atlas fusion template based on spatial overlap degree of brain regions, which aims to obtain a comprehensive representation of brain networks. Specifically, we formally define a measurement of the spatial overlap among brain regions across different atlases, named spatial overlap degree. Then, we fuse the multi-atlas to obtain brain networks of each subject based on spatial overlap. Finally, the GCN is used to perform the final classification. The experimental results on Autism Brain Imaging Data Exchange (ABIDE) demonstrate that our proposed method achieved an accuracy of 0.757. Overall, our method outperforms SOTA methods in ASD/TC classification.

Recent advances in Alzheimer's disease: Mechanisms, clinical trials and new drug development strategies

Alzheimer's disease (AD) stands as the predominant form of dementia, presenting significant and escalating global challenges. Its etiology is intricate and diverse, stemming from a combination of factors such as aging, genetics, and environment. Our current understanding of AD pathologies involves various hypotheses, such as the cholinergic, amyloid, tau protein, inflammatory, oxidative stress, metal ion, glutamate excitotoxicity, microbiota-gut-brain axis, and abnormal autophagy. Nonetheless, unraveling the interplay among these pathological aspects and pinpointing the primary initiators of AD require further elucidation and validation. In the past decades, most clinical drugs have been discontinued due to limited effectiveness or adverse effects. Presently, available drugs primarily offer symptomatic relief and often accompanied by undesirable side effects. However, recent approvals of aducanumab (1) and lecanemab (2) by the Food and Drug Administration (FDA) present the potential in disrease-modifying effects. Nevertheless, the long-term efficacy and safety of these drugs need further validation. Consequently, the quest for safer and more effective AD drugs persists as a formidable and pressing task. This review discusses the current understanding of AD pathogenesis, advances in diagnostic biomarkers, the latest updates of clinical trials, and emerging technologies for AD drug development. We highlight recent progress in the discovery of selective inhibitors, dual-target inhibitors, allosteric modulators, covalent inhibitors, proteolysis-targeting chimeras (PROTACs), and protein-protein interaction (PPI) modulators. Our goal is to provide insights into the prospective development and clinical application of novel AD drugs.

Frontiers and hotspots evolution in mild cognitive impairment: a bibliometric analysis of from 2013 to 2023

Background

Mild cognitive impairment is a heterogeneous syndrome. The heterogeneity of the syndrome and the absence of consensus limited the advancement of MCI. The purpose of our research is to create a visual framework of the last decade, highlight the hotspots of current research, and forecast the most fruitful avenues for future MCI research.

Methods

We collected all the MCI-related literature published between 1 January 2013, and 24 April 2023, on the "Web of Science." The visual graph was created by the CiteSpace and VOSviewer. The current research hotspots and future research directions are summarized through the analysis of keywords and co-cited literature.

Results

There are 6,075 articles were included in the final analysis. The number of publications shows an upward trend, especially after 2018. The United States and the University of California System are the most prolific countries and institutions, respectively. Petersen is the author who ranks first in terms of publication volume and influence. Journal of Alzheimer's Disease was the most productive journal. "neuroimaging," "fluid markers," and "predictors" are the focus of current research, and "machine learning," "electroencephalogram," "deep learning," and "blood biomarkers" are potential research directions in the future.

Conclusion

The cognition of MCI has been continuously evolved and renewed by multiple countries' joint efforts in the past decade. Hotspots for current research are on diagnostic biomarkers, such as fluid markers, neuroimaging, and so on. Future hotspots might be focused on the best prognostic and diagnostic models generated by machine learning and large-scale screening tools such as EEG and blood biomarkers.

Mapping Knowledge Landscapes and Emerging Trends in AI for Dementia Biomarkers: Bibliometric and Visualization Analysis

BACKGROUND

With the rise of artificial intelligence (AI) in the field of dementia biomarker research, exploring its current developmental trends and research focuses has become increasingly important. This study, using literature data mining, analyzes and assesses the key contributions and development scale of AI in dementia biomarker research.

OBJECTIVE

The aim of this study was to comprehensively evaluate the current state, hot topics, and future trends of AI in dementia biomarker research globally.

METHODS

This study thoroughly analyzed the literature in the application of AI to dementia biomarkers across various dimensions, such as publication volume, authors, institutions, journals, and countries, based on the Web of Science Core Collection. In addition, scales, trends, and potential connections between AI and biomarkers were extracted and deeply analyzed through multiple expert panels.

RESULTS

To date, the field includes 1070 publications across 362 journals, involving 74 countries and 1793 major research institutions, with a total of 6455 researchers. Notably, 69.41% (994/1432) of the researchers ceased their studies before 2019. The most prevalent algorithms used are support vector machines, random forests, and neural networks. Current research frequently focuses on biomarkers such as imaging biomarkers, cerebrospinal fluid biomarkers, genetic biomarkers, and blood biomarkers. Recent advances have highlighted significant discoveries in biomarkers related to imaging, genetics, and blood, with growth in studies on digital and ophthalmic biomarkers.

CONCLUSIONS

The field is currently in a phase of stable development, receiving widespread attention from numerous countries, institutions, and researchers worldwide. Despite this, stable clusters of collaborative research have yet to be established, and there is a pressing need to enhance interdisciplinary collaboration. Algorithm development has shown prominence, especially the application of support vector machines and neural networks in imaging studies. Looking forward, newly discovered biomarkers are expected to undergo further validation, and new types, such as digital biomarkers, will garner increased research interest and attention.

AI-based differential diagnosis of dementia etiologies on multimodal data

Differential diagnosis of dementia remains a challenge in neurology due to symptom overlap across etiologies, yet it is crucial for formulating early, personalized management strategies. Here, we present an artificial intelligence (AI) model that harnesses a broad array of data, including demographics, individual and family medical history, medication use, neuropsychological assessments, functional evaluations and multimodal neuroimaging, to identify the etiologies contributing to dementia in individuals. The study, drawing on 51,269 participants across 9 independent, geographically diverse datasets, facilitated the identification of 10 distinct dementia etiologies. It aligns diagnoses with similar management strategies, ensuring robust predictions even with incomplete data. Our model achieved a microaveraged area under the receiver operating characteristic curve (AUROC) of 0.94 in classifying individuals with normal cognition, mild cognitive impairment and dementia. Also, the microaveraged AUROC was 0.96 in differentiating the dementia etiologies. Our model demonstrated proficiency in addressing mixed dementia cases, with a mean AUROC of 0.78 for two co-occurring pathologies. In a randomly selected subset of 100 cases, the AUROC of neurologist assessments augmented by our AI model exceeded neurologist-only evaluations by 26.25%. Furthermore, our model predictions aligned with biomarker evidence and its associations with different proteinopathies were substantiated through postmortem findings. Our framework has the potential to be integrated as a screening tool for dementia in clinical settings and drug trials. Further prospective studies are needed to confirm its ability to improve patient care.

A review of deep learning-based information fusion techniques for multimodal medical image classification

Multimodal medical imaging plays a pivotal role in clinical diagnosis and research, as it combines information from various imaging modalities to provide a more comprehensive understanding of the underlying pathology. Recently, deep learning-based multimodal fusion techniques have emerged as powerful tools for improving medical image classification. This review offers a thorough analysis of the developments in deep learning-based multimodal fusion for medical classification tasks. We explore the complementary relationships among prevalent clinical modalities and outline three main fusion schemes for multimodal classification networks: input fusion, intermediate fusion (encompassing single-level fusion, hierarchical fusion, and attention-based fusion), and output fusion. By evaluating the performance of these fusion techniques, we provide insight into the suitability of different network architectures for various multimodal fusion scenarios and application domains. Furthermore, we delve into challenges related to network architecture selection, handling incomplete multimodal data management, and the potential limitations of multimodal fusion. Finally, we spotlight the promising future of Transformer-based multimodal fusion techniques and give recommendations for future research in this rapidly evolving field.

Deep Learning Models for the Screening of Cognitive Impairment Using Multimodal Fundus Images

OBJECTIVE

We aimed to develop a deep learning system capable of identifying subjects with cognitive impairment quickly and easily based on multimodal ocular images.

DESIGN

Cross sectional study.

SUBJECTS

Participants of Beijing Eye Study 2011 and patients attending Beijing Tongren Eye Center and Beijing Tongren Hospital Physical Examination Center.

METHODS

We trained and validated a deep learning algorithm to assess cognitive impairment using retrospectively collected data from the Beijing Eye Study 2011. Cognitive impairment was defined as a Mini-Mental State Examination score < 24. Based on fundus photographs and OCT images, we developed 5 models based on the following sets of images: macula-centered fundus photographs, optic disc-centered fundus photographs, fundus photographs of both fields, OCT images, and fundus photographs of both fields with OCT (multimodal). The performance of the models was evaluated and compared in an external validation data set, which was collected from patients attending Beijing Tongren Eye Center and Beijing Tongren Hospital Physical Examination Center.

MAIN OUTCOME MEASURES

Area under the curve (AUC).

RESULTS

A total of 9424 retinal photographs and 4712 OCT images were used to develop the model. The external validation sets from each center included 1180 fundus photographs and 590 OCT images. Model comparison revealed that the multimodal performed best, achieving an AUC of 0.820 in the internal validation set, 0.786 in external validation set 1, and 0.784 in external validation set 2. We evaluated the performance of the multi-model in different sexes and different age groups; there were no significant differences. The heatmap analysis showed that signals around the optic disc in fundus photographs and the retina and choroid around the macular and optic disc regions in OCT images were used by the multimodal to identify participants with cognitive impairment.

CONCLUSIONS

Fundus photographs and OCT can provide valuable information on cognitive function. Multimodal models provide richer information compared with single-mode models. Deep learning algorithms based on multimodal retinal images may be capable of screening cognitive impairment. This technique has potential value for broader implementation in community-based screening or clinic settings.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Multispectral 3D DNA Machine Combined with Multimodal Machine Learning for Noninvasive Precise Diagnosis of Bladder Cancer

Extracellular vesicle (EV) molecular phenotyping offers enormous opportunities for cancer diagnostics. However, the majority of the associated studies adopted biomarker-based unimodal analysis to achieve cancer diagnosis, which has high false positives and low precision. Herein, we report a multimodal platform for the high-precision diagnosis of bladder cancer (BCa) through a multispectral 3D DNA machine in combination with a multimodal machine learning (ML) algorithm. The DNA machine was constructed using magnetic microparticles (MNPs) functionalized with aptamers that specifically identify the target of interest, i.e., five protein markers on bladder-cancer-derived urinary EVs (uEVs). The aptamers were hybridized with DNA-stabilized silver nanoclusters (DNA/AgNCs) and a G-quadruplex/hemin complex to form a sensing module. Such a DNA machine ensured multispectral detection of protein markers by fluorescence (FL), inductively coupled plasma mass spectrometry (ICP-MS), and UV-vis absorption (Abs). The obtained data sets then underwent uni- or multimodal ML for BCa diagnosis to compare the analytical performance. In this study, urine samples were obtained from our prospective cohort (n = 45). Our analytical results showed that the 3D DNA machine provided a detection limit of 9.2 × 103 particles mL-1 with a linear range of 4 × 104 to 5 × 107 particles mL-1 for uEVs. Moreover, the multimodal data fusion model exhibited an accuracy of 95.0%, a precision of 93.1%, and a recall rate of 93.2% on average, while those of the three types of unimodal models were no more than 91%. The elevated diagnosis precision by using the present fusion platform offers a perspective approach to diminishing the rate of misdiagnosis and overtreatment of BCa.

Disease-driven domain generalization for neuroimaging-based assessment of Alzheimer's disease

Development of deep learning models to evaluate structural brain changes caused by cognitive impairment in MRI scans holds significant translational value. The efficacy of these models often encounters challenges due to variabilities arising from different data generation protocols, imaging equipment, radiological artifacts, and shifts in demographic distributions. Domain generalization (DG) techniques show promise in addressing these challenges by enabling the model to learn from one or more source domains and apply this knowledge to new, unseen target domains. Here we present a framework that utilizes model interpretability to enhance the generalizability of classification models across various cohorts. We used MRI scans and clinical diagnoses from four independent cohorts: Alzheimer's Disease Neuroimaging Initiative (ADNI, n = 1821), the Framingham Heart Study (FHS, n = 304), the Australian Imaging Biomarkers & Lifestyle Study of Ageing (AIBL, n = 661), and the National Alzheimer's Coordinating Center (NACC, n = 4647). With this data, we trained a deep neural network to focus on areas of the brain identified as relevant to the disease for model training. Our approach involved training a classifier to differentiate between structural neurodegeneration in individuals with normal cognition (NC), mild cognitive impairment (MCI), and dementia due to Alzheimer's disease (AD). This was achieved by aligning class-wise attention with a unified visual saliency prior, which was computed offline for each class using all the training data. Our method not only competes with state-of-the-art approaches but also shows improved correlation with postmortem histology. This alignment with the gold standard evidence is a significant step towards validating the effectiveness of DG frameworks, paving the way for their broader application in the field.

Enhancing psychiatric rehabilitation outcomes through a multimodal multitask learning model based on BERT and TabNet: An approach for personalized treatment and improved decision-making

Evaluating the rehabilitation status of individuals with serious mental illnesses (SMI) necessitates a comprehensive analysis of multimodal data, including unstructured text records and structured diagnostic data. However, progress in the effective assessment of rehabilitation status remains limited. Our study develops a deep learning model integrating Bidirectional Encoder Representations from Transformers (BERT) and TabNet through a late fusion strategy to enhance rehabilitation prediction, including referral risk, dangerous behaviors, self-awareness, and medication adherence, in patients with SMI. BERT processes unstructured textual data, such as doctor's notes, whereas TabNet manages structured diagnostic information. The model's interpretability function serves to assist healthcare professionals in understanding the model's predictive decisions, improving patient care. Our model exhibited excellent predictive performance for all four tasks, with an accuracy exceeding 0.78 and an area under the curve of 0.70. In addition, a series of tests proved the model's robustness, fairness, and interpretability. This study combines multimodal and multitask learning strategies into a model and applies it to rehabilitation assessment tasks, offering a promising new tool that can be seamlessly integrated with the clinical workflow to support the provision of optimized patient care.

Application of Multiomics Approach to Investigate the Therapeutic Potentials of Stem Cell-derived Extracellular Vesicle Subpopulations for Alzheimer's Disease

Alzheimer's disease (AD) presents a complex interplay of molecular alterations, yet understanding its pathogenesis remains a challenge. In this study, we delved into the intricate landscape of proteome and transcriptome changes in AD brains compared to healthy controls, examining 788 brain samples revealing common alterations at both protein and mRNA levels. Moreover, our analysis revealed distinct protein-level changes in aberrant energy metabolism pathways in AD brains that were not evident at the mRNA level. This suggests that the changes in protein expression could provide a deeper molecular representation of AD pathogenesis. Subsequently, using a comparative proteomic approach, we explored the therapeutic potential of mesenchymal stem cell-derived extracellular vehicles (EVs), isolated through various methods, in mitigating AD-associated changes at the protein level. Our analysis revealed a particular EV-subtype that can be utilized for compensating dysregulated mitochondrial proteostasis in the AD brain. By using network biology approaches, we further revealed the potential regulators of key therapeutic proteins. Overall, our study illuminates the significance of proteome alterations in AD pathogenesis and identifies the therapeutic promise of a specific EV subpopulation with reduced pro-inflammatory protein cargo and enriched proteins to target mitochondrial proteostasis.

Introduction to Large Language Models (LLMs) for dementia care and research

Introduction

Dementia is a progressive neurodegenerative disorder that affects cognitive abilities including memory, reasoning, and communication skills, leading to gradual decline in daily activities and social engagement. In light of the recent advent of Large Language Models (LLMs) such as ChatGPT, this paper aims to thoroughly analyse their potential applications and usefulness in dementia care and research.

Method

To this end, we offer an introduction into LLMs, outlining the key features, capabilities, limitations, potential risks, and practical considerations for deployment as easy-to-use software (e.g., smartphone apps). We then explore various domains related to dementia, identifying opportunities for LLMs to enhance understanding, diagnostics, and treatment, with a broader emphasis on improving patient care. For each domain, the specific contributions of LLMs are examined, such as their ability to engage users in meaningful conversations, deliver personalized support, and offer cognitive enrichment. Potential benefits encompass improved social interaction, enhanced cognitive functioning, increased emotional well-being, and reduced caregiver burden. The deployment of LLMs in caregiving frameworks also raises a number of concerns and considerations. These include privacy and safety concerns, the need for empirical validation, user-centered design, adaptation to the user's unique needs, and the integration of multimodal inputs to create more immersive and personalized experiences. Additionally, ethical guidelines and privacy protocols must be established to ensure responsible and ethical deployment of LLMs.

Results

We report the results on a questionnaire filled in by people with dementia (PwD) and their supporters wherein we surveyed the usefulness of different application scenarios of LLMs as well as the features that LLM-powered apps should have. Both PwD and supporters were largely positive regarding the prospect of LLMs in care, although concerns were raised regarding bias, data privacy and transparency.

Discussion

Overall, this review corroborates the promising utilization of LLMs to positively impact dementia care by boosting cognitive abilities, enriching social interaction, and supporting caregivers. The findings underscore the importance of further research and development in this field to fully harness the benefits of LLMs and maximize their potential for improving the lives of individuals living with dementia.

Identifying autism spectrum disorder from multi-modal data with privacy-preserving

The application of deep learning models to precision medical diagnosis often requires the aggregation of large amounts of medical data to effectively train high-quality models. However, data privacy protection mechanisms make it difficult to perform medical data collection from different medical institutions. In autism spectrum disorder (ASD) diagnosis, automatic diagnosis using multimodal information from heterogeneous data has not yet achieved satisfactory performance. To address the privacy preservation issue as well as to improve ASD diagnosis, we propose a deep learning framework using multimodal feature fusion and hypergraph neural networks for disease prediction in federated learning (FedHNN). By introducing the federated learning strategy, each local model is trained and computed independently in a distributed manner without data sharing, allowing rapid scaling of medical datasets to achieve robust and scalable deep learning predictive models. To further improve the performance with privacy preservation, we improve the hypergraph model for multimodal fusion to make it suitable for autism spectrum disorder (ASD) diagnosis tasks by capturing the complementarity and correlation between modalities through a hypergraph fusion strategy. The results demonstrate that our proposed federated learning-based prediction model is superior to all local models and outperforms other deep learning models. Overall, our proposed FedHNN has good results in the work of using multi-site data to improve the performance of ASD identification.

PED: a novel predictor-encoder-decoder model for Alzheimer drug molecular generation

Alzheimer's disease (AD) is a gradually advancing neurodegenerative disorder characterized by a concealed onset. Acetylcholinesterase (AChE) is an efficient hydrolase that catalyzes the hydrolysis of acetylcholine (ACh), which regulates the concentration of ACh at synapses and then terminates ACh-mediated neurotransmission. There are inhibitors to inhibit the activity of AChE currently, but its side effects are inevitable. In various application fields where Al have gained prominence, neural network-based models for molecular design have recently emerged and demonstrate encouraging outcomes. However, in the conditional molecular generation task, most of the current generation models need additional optimization algorithms to generate molecules with intended properties which make molecular generation inefficient. Consequently, we introduce a cognitive-conditional molecular design model, termed PED, which leverages the variational auto-encoder. Its primary function is to adeptly produce a molecular library tailored for specific properties. From this library, we can then identify molecules that inhibit AChE activity without adverse effects. These molecules serve as lead compounds, hastening AD treatment and concurrently enhancing the AI's cognitive abilities. In this study, we aim to fine-tune a VAE model pre-trained on the ZINC database using active compounds of AChE collected from Binding DB. Different from other molecular generation models, the PED can simultaneously perform both property prediction and molecule generation, consequently, it can generate molecules with intended properties without additional optimization process. Experiments of evaluation show that proposed model performs better than other methods benchmarked on the same data sets. The results indicated that the model learns a good representation of potential chemical space, it can well generate molecules with intended properties. Extensive experiments on benchmark datasets confirmed PED's efficiency and efficacy. Furthermore, we also verified the binding ability of molecules to AChE through molecular docking. The results showed that our molecular generation system for AD shows excellent cognitive capacities, the molecules within the molecular library could bind well to AChE and inhibit its activity, thus preventing the hydrolysis of ACh.

Prioritization of genes for translation: a computational approach

INTRODUCTION

Gene identification for genetic diseases is critical for the development of new diagnostic approaches and personalized treatment options. Prioritization of gene translation is an important consideration in the molecular biology field, allowing researchers to focus on the most promising candidates for further investigation.

AREAS COVERED

In this paper, we discussed different approaches to prioritize genes for translation, including the use of computational tools and machine learning algorithms, as well as experimental techniques such as knockdown and overexpression studies. We also explored the potential biases and limitations of these approaches and proposed strategies to improve the accuracy and reliability of gene prioritization methods. Although numerous computational methods have been developed for this purpose, there is a need for computational methods that incorporate tissue-specific information to enable more accurate prioritization of candidate genes. Such methods should provide tissue-specific predictions, insights into underlying disease mechanisms, and more accurate prioritization of genes.

EXPERT OPINION

Using advanced computational tools and machine learning algorithms to prioritize genes, we can identify potential targets for therapeutic intervention of complex diseases. This represents an up-and-coming method for drug development and personalized medicine.

AI-based differential diagnosis of dementia etiologies on multimodal data

Differential diagnosis of dementia remains a challenge in neurology due to symptom overlap across etiologies, yet it is crucial for formulating early, personalized management strategies. Here, we present an AI model that harnesses a broad array of data, including demographics, individual and family medical history, medication use, neuropsychological assessments, functional evaluations, and multimodal neuroimaging, to identify the etiologies contributing to dementia in individuals. The study, drawing on 51,269 participants across 9 independent, geographically diverse datasets, facilitated the identification of 10 distinct dementia etiologies. It aligns diagnoses with similar management strategies, ensuring robust predictions even with incomplete data. Our model achieved a micro-averaged area under the receiver operating characteristic curve (AUROC) of 0.94 in classifying individuals with normal cognition, mild cognitive impairment and dementia. Also, the micro-averaged AUROC was 0.96 in differentiating the dementia etiologies. Our model demonstrated proficiency in addressing mixed dementia cases, with a mean AUROC of 0.78 for two co-occurring pathologies. In a randomly selected subset of 100 cases, the AUROC of neurologist assessments augmented by our AI model exceeded neurologist-only evaluations by 26.25%. Furthermore, our model predictions aligned with biomarker evidence and its associations with different proteinopathies were substantiated through postmortem findings. Our framework has the potential to be integrated as a screening tool for dementia in various clinical settings and drug trials, with promising implications for person-level management.

Comparing a pre-defined versus deep learning approach for extracting brain atrophy patterns to predict cognitive decline due to Alzheimer's disease in patients with mild cognitive symptoms

BACKGROUND

Predicting future Alzheimer's disease (AD)-related cognitive decline among individuals with subjective cognitive decline (SCD) or mild cognitive impairment (MCI) is an important task for healthcare. Structural brain imaging as measured by magnetic resonance imaging (MRI) could potentially contribute when making such predictions. It is unclear if the predictive performance of MRI can be improved using entire brain images in deep learning (DL) models compared to using pre-defined brain regions.

METHODS

A cohort of 332 individuals with SCD/MCI were included from the Swedish BioFINDER-1 study. The goal was to predict longitudinal SCD/MCI-to-AD dementia progression and change in Mini-Mental State Examination (MMSE) over four years. Four models were evaluated using different predictors: (1) clinical data only, including demographics, cognitive tests and APOE ε4 status, (2) clinical data plus hippocampal volume, (3) clinical data plus all regional MRI gray matter volumes (N = 68) extracted using FreeSurfer software, (4) a DL model trained using multi-task learning with MRI images, Jacobian determinant images and baseline cognition as input. A double cross-validation scheme, with five test folds and for each of those ten validation folds, was used. External evaluation was performed on part of the ADNI dataset, including 108 patients. Mann-Whitney U-test was used to determine statistically significant differences in performance, with p-values less than 0.05 considered significant.

RESULTS

In the BioFINDER cohort, 109 patients (33%) progressed to AD dementia. The performance of the clinical data model for prediction of progression to AD dementia was area under the curve (AUC) = 0.85 and four-year cognitive decline was R2 = 0.14. The performance was improved for both outcomes when adding hippocampal volume (AUC = 0.86, R2 = 0.16). Adding FreeSurfer brain regions improved prediction of four-year cognitive decline but not progression to AD (AUC = 0.83, R2 = 0.17), while the DL model worsened the performance for both outcomes (AUC = 0.84, R2 = 0.08). A sensitivity analysis showed that the Jacobian determinant image was more informative than the MRI image, but that performance was maximized when both were included. In the external evaluation cohort from ADNI, 23 patients (21%) progressed to AD dementia. The results for predicted progression to AD dementia were similar to the results for the BioFINDER test data, while the performance for the cognitive decline was deteriorated.

CONCLUSIONS

The DL model did not significantly improve the prediction of clinical disease progression in AD, compared to regression models with a single pre-defined brain region.

Automated detection of Alzheimer's disease: a multi-modal approach with 3D MRI and amyloid PET

Recent advances in deep learning and imaging technologies have revolutionized automated medical image analysis, especially in diagnosing Alzheimer's disease through neuroimaging. Despite the availability of various imaging modalities for the same patient, the development of multi-modal models leveraging these modalities remains underexplored. This paper addresses this gap by proposing and evaluating classification models using 2D and 3D MRI images and amyloid PET scans in uni-modal and multi-modal frameworks. Our findings demonstrate that models using volumetric data learn more effective representations than those using only 2D images. Furthermore, integrating multiple modalities enhances model performance over single-modality approaches significantly. We achieved state-of-the-art performance on the OASIS-3 cohort. Additionally, explainability analyses with Grad-CAM indicate that our model focuses on crucial AD-related regions for its predictions, underscoring its potential to aid in understanding the disease's causes.

Alzheimer's disease prediction algorithm based on de-correlation constraint and multi-modal feature interaction

Alzheimer's disease (AD) is a neurodegenerative disease characterized by various pathological changes. Utilizing multimodal data from Fluorodeoxyglucose positron emission tomography(FDG-PET) and Magnetic Resonance Imaging(MRI) of the brain can offer comprehensive information about the lesions from different perspectives and improve the accuracy of prediction. However, there are significant differences in the feature space of multimodal data. Commonly, the simple concatenation of multimodal features can cause the model to struggle in distinguishing and utilizing the complementary information between different modalities, thus affecting the accuracy of predictions. Therefore, we propose an AD prediction model based on de-correlation constraint and multi-modal feature interaction. This model consists of the following three parts: (1) The feature extractor employs residual connections and attention mechanisms to capture distinctive lesion features from FDG-PET and MRI data within their respective modalities. (2) The de-correlation constraint function enhances the model's capacity to extract complementary information from different modalities by reducing the feature similarity between them. (3) The mutual attention feature fusion module interacts with the features within and between modalities to enhance the modal-specific features and adaptively adjust the weights of these features based on information from other modalities. The experimental results on ADNI database demonstrate that the proposed model achieves a prediction accuracy of 86.79% for AD, MCI and NC, which is higher than the existing multi-modal AD prediction models.

Leveraging electronic health records and knowledge networks for Alzheimer's disease prediction and sex-specific biological insights

Identification of Alzheimer's disease (AD) onset risk can facilitate interventions before irreversible disease progression. We demonstrate that electronic health records from the University of California, San Francisco, followed by knowledge networks (for example, SPOKE) allow for (1) prediction of AD onset and (2) prioritization of biological hypotheses, and (3) contextualization of sex dimorphism. We trained random forest models and predicted AD onset on a cohort of 749 individuals with AD and 250,545 controls with a mean area under the receiver operating characteristic of 0.72 (7 years prior) to 0.81 (1 day prior). We further harnessed matched cohort models to identify conditions with predictive power before AD onset. Knowledge networks highlight shared genes between multiple top predictors and AD (for example, APOE, ACTB, IL6 and INS). Genetic colocalization analysis supports AD association with hyperlipidemia at the APOE locus, as well as a stronger female AD association with osteoporosis at a locus near MS4A6A. We therefore show how clinical data can be utilized for early AD prediction and identification of personalized biological hypotheses.

Deep learning applications in vascular dementia using neuroimaging

PURPOSE OF REVIEW

Vascular dementia (VaD) is the second common cause of dementia after Alzheimer's disease, and deep learning has emerged as a critical tool in dementia research. The aim of this article is to highlight the current deep learning applications in VaD-related imaging biomarkers and diagnosis.

RECENT FINDINGS

The main deep learning technology applied in VaD using neuroimaging data is convolutional neural networks (CNN). CNN models have been widely used for lesion detection and segmentation, such as white matter hyperintensities (WMH), cerebral microbleeds (CMBs), perivascular spaces (PVS), lacunes, cortical superficial siderosis, and brain atrophy. Applications in VaD subtypes classification also showed excellent results. CNN-based deep learning models have potential for further diagnosis and prognosis of VaD.

SUMMARY

Deep learning neural networks with neuroimaging data in VaD research represent significant promise for advancing early diagnosis and treatment strategies. Ongoing research and collaboration between clinicians, data scientists, and neuroimaging experts are essential to address challenges and unlock the full potential of deep learning in VaD diagnosis and management.

Alzheimer's disease detection using data fusion with a deep supervised encoder

Alzheimer's disease (AD) is affecting a growing number of individuals. As a result, there is a pressing need for accurate and early diagnosis methods. This study aims to achieve this goal by developing an optimal data analysis strategy to enhance computational diagnosis. Although various modalities of AD diagnostic data are collected, past research on computational methods of AD diagnosis has mainly focused on using single-modal inputs. We hypothesize that integrating, or "fusing," various data modalities as inputs to prediction models could enhance diagnostic accuracy by offering a more comprehensive view of an individual's health profile. However, a potential challenge arises as this fusion of multiple modalities may result in significantly higher dimensional data. We hypothesize that employing suitable dimensionality reduction methods across heterogeneous modalities would not only help diagnosis models extract latent information but also enhance accuracy. Therefore, it is imperative to identify optimal strategies for both data fusion and dimensionality reduction. In this paper, we have conducted a comprehensive comparison of over 80 statistical machine learning methods, considering various classifiers, dimensionality reduction techniques, and data fusion strategies to assess our hypotheses. Specifically, we have explored three primary strategies: (1) Simple data fusion, which involves straightforward concatenation (fusion) of datasets before inputting them into a classifier; (2) Early data fusion, in which datasets are concatenated first, and then a dimensionality reduction technique is applied before feeding the resulting data into a classifier; and (3) Intermediate data fusion, in which dimensionality reduction methods are applied individually to each dataset before concatenating them to construct a classifier. For dimensionality reduction, we have explored several commonly-used techniques such as principal component analysis (PCA), autoencoder (AE), and LASSO. Additionally, we have implemented a new dimensionality-reduction method called the supervised encoder (SE), which involves slight modifications to standard deep neural networks. Our results show that SE substantially improves prediction accuracy compared to PCA, AE, and LASSO, especially in combination with intermediate fusion for multiclass diagnosis prediction.

Immunological Profile and Markers of Endothelial Dysfunction in Elderly Patients with Cognitive Impairments

The process of aging is accompanied by a dynamic restructuring of the immune response, a phenomenon known as immunosenescence. Further, damage to the endothelium can be both a cause and a consequence of many diseases, especially in elderly people. The purpose of this study was to carry out immunological and biochemical profiling of elderly people with acute ischemic stroke (AIS), chronic cerebral circulation insufficiency (CCCI), prediabetes or newly diagnosed type II diabetes mellitus (DM), and subcortical ischemic vascular dementia (SIVD). Socio-demographic, lifestyle, and cognitive data were obtained. Biochemical, hematological, and immunological analyses were carried out, and extracellular vesicles (EVs) with endothelial CD markers were assessed. The greatest number of significant deviations from conditionally healthy donors (HDs) of the same age were registered in the SIVD group, a total of 20, of which 12 were specific and six were non-specific but with maximal differences (as compared to the other three groups) from the HDs group. The non-specific deviations were for the MOCA (Montreal Cognitive Impairment Scale), the MMSE (Mini Mental State Examination) and life satisfaction self-assessment scores, a decrease of albumin levels, and ADAMTS13 (a Disintegrin and Metalloproteinase with a Thrombospondin Type 1 motif, member 13) activity, and an increase of the VWF (von Willebrand factor) level. Considering the significant changes in immunological parameters (mostly Th17-like cells) and endothelial CD markers (CD144 and CD34), vascular repair was impaired to the greatest extent in the DM group. The AIS patients showed 12 significant deviations from the HD controls, including three specific to this group. These were high NEFAs (non-esterified fatty acids) and CD31 and CD147 markers of EVs. The lowest number of deviations were registered in the CCCI group, nine in total. There were significant changes from the HD controls with no specifics to this group, and just one non-specific with a maximal difference from the control parameters, which was α1-AGP (alpha 1 acid glycoprotein, orosomucoid). Besides the DM patients, impairments of vascular repair were also registered in the CCCI and AIS patients, with a complete absence of such in patients with dementia (SIVD group). On the other hand, microvascular damage seemed to be maximal in the latter group, considering the biochemical indicators VWF and ADAMTS13. In the DM patients, a maximum immune response was registered, mainly with Th17-like cells. In the CCCI group, the reaction was not as pronounced compared to other groups of patients, which may indicate the initial stages and/or compensatory nature of organic changes (remodeling). At the same time, immunological and biochemical deviations in SIVD patients indicated a persistent remodeling in microvessels, chronic inflammation, and a significant decrease in the anabolic function of the liver and other tissues. The data obtained support two interrelated assumptions. Taking into account the primary biochemical factors that trigger the pathological processes associated with vascular pathology and related diseases, the first assumption is that purine degradation in skeletal muscle may be a major factor in the production of uric acid, followed by its production by non-muscle cells, the main of which are endothelial cells. Another assumption is that therapeutic factors that increase the levels of endothelial progenitor cells may have a therapeutic effect in reducing the risk of cerebrovascular disease and related neurodegenerative diseases.

Alzheimer's Disease Evaluation Through Visual Explainability by Means of Convolutional Neural Networks

Background and Objective: Alzheimer's disease is nowadays the most common cause of dementia. It is a degenerative neurological pathology affecting the brain, progressively leading the patient to a state of total dependence, thus creating a very complex and difficult situation for the family that has to assist him/her. Early diagnosis is a primary objective and constitutes the hope of being able to intervene in the development phase of the disease. Methods: In this paper, a method to automatically detect the presence of Alzheimer's disease, by exploiting deep learning, is proposed. Five different convolutional neural networks are considered: ALEX_NET, VGG16, FAB_CONVNET, STANDARD_CNN and FCNN. The first two networks are state-of-the-art models, while the last three are designed by authors. We classify brain images into one of the following classes: non-demented, very mild demented and mild demented. Moreover, we highlight on the image the areas symptomatic of Alzheimer presence, thus providing a visual explanation behind the model diagnosis. Results: The experimental analysis, conducted on more than 6000 magnetic resonance images, demonstrated the effectiveness of the proposed neural networks in the comparison with the state-of-the-art models in Alzheimer's disease diagnosis and localization. The best results in terms of metrics are the best with STANDARD_CNN and FCNN with accuracy, precision and recall between 98% and 95%. Excellent results also from a qualitative point of view are obtained with the Grad-CAM for localization and visual explainability. Conclusions: The analysis of the heatmaps produced by the Grad-CAM algorithm shows that in almost all cases the heatmaps highlight regions such as ventricles and cerebral cortex. Future work will focus on the realization of a network capable of analyzing the three anatomical views simultaneously.

An explainable machine learning approach for Alzheimer's disease classification

The early diagnosis of Alzheimer's disease (AD) presents a significant challenge due to the subtle biomarker changes often overlooked. Machine learning (ML) models offer a promising tool for identifying individuals at risk of AD. However, current research tends to prioritize ML accuracy while neglecting the crucial aspect of model explainability. The diverse nature of AD data and the limited dataset size introduce additional challenges, primarily related to high dimensionality. In this study, we leveraged a dataset obtained from the National Alzheimer's Coordinating Center, comprising 169,408 records and 1024 features. After applying various steps to reduce the feature space. Notably, support vector machine (SVM) models trained on the selected features exhibited high performance when tested on an external dataset. SVM achieved a high F1 score of 98.9% for binary classification (distinguishing between NC and AD) and 90.7% for multiclass classification. Furthermore, SVM was able to predict AD progression over a 4-year period, with F1 scores reached 88% for binary task and 72.8% for multiclass task. To enhance model explainability, we employed two rule-extraction approaches: class rule mining and stable and interpretable rule set for classification model. These approaches generated human-understandable rules to assist domain experts in comprehending the key factors involved in AD development. We further validated these rules using SHAP and LIME models, underscoring the significance of factors such as MEMORY, JUDGMENT, COMMUN, and ORIENT in determining AD risk. Our experimental outcomes also shed light on the crucial role of the Clinical Dementia Rating tool in predicting AD.

Multimodal diagnosis model of Alzheimer's disease based on improved Transformer

PURPOSE

Recent technological advancements in data acquisition tools allowed neuroscientists to acquire different modality data to diagnosis Alzheimer's disease (AD). However, how to fuse these enormous amount different modality data to improve recognizing rate and find significance brain regions is still challenging.

METHODS

The algorithm used multimodal medical images [structural magnetic resonance imaging (sMRI) and positron emission tomography (PET)] as experimental data. Deep feature representations of sMRI and PET images are extracted by 3D convolution neural network (3DCNN). An improved Transformer is then used to progressively learn global correlation information among features. Finally, the information from different modalities is fused for identification. A model-based visualization method is used to explain the decisions of the model and identify brain regions related to AD.

RESULTS

The model attained a noteworthy classification accuracy of 98.1% for Alzheimer's disease (AD) using the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. Upon examining the visualization results, distinct brain regions associated with AD diagnosis were observed across different image modalities. Notably, the left parahippocampal region emerged consistently as a prominent and significant brain area.

CONCLUSIONS

A large number of comparative experiments have been carried out for the model, and the experimental results verify the reliability of the model. In addition, the model adopts a visualization analysis method based on the characteristics of the model, which improves the interpretability of the model. Some disease-related brain regions were found in the visualization results, which provides reliable information for AD clinical research.

Applications of Artificial Intelligence in the Neuropsychological Assessment of Dementia: A Systematic Review

In the context of advancing healthcare, the diagnosis and treatment of cognitive disorders, particularly Mild Cognitive Impairment (MCI) and Alzheimer's Disease (AD), pose significant challenges. This review explores Artificial Intelligence (AI) and Machine Learning (ML) in neuropsychological assessment for the early detection and personalized treatment of MCI and AD. The review includes 37 articles that demonstrate that AI could be an useful instrument for optimizing diagnostic procedures, predicting cognitive decline, and outperforming traditional tests. Three main categories of applications are identified: (1) combining neuropsychological assessment with clinical data, (2) optimizing existing test batteries using ML techniques, and (3) employing virtual reality and games to overcome the limitations of traditional tests. Despite advancements, the review highlights a gap in developing tools that simplify the clinician's workflow and underscores the need for explainable AI in healthcare decision making. Future studies should bridge the gap between technical performance measures and practical clinical utility to yield accurate results and facilitate clinicians' roles. The successful integration of AI/ML in predicting dementia onset could reduce global healthcare costs and benefit aging societies.

A Convolutional Neural Network Model for Distinguishing Hemangioblastomas From Other Cerebellar-and-Brainstem Tumors Using Contrast-Enhanced MRI

BACKGROUND

Hemangioblastoma (HB) is a highly vascularized tumor most commonly occurring in the posterior cranial fossa, requiring accurate preoperative diagnosis to avoid accidental intraoperative hemorrhage and even death.

PURPOSE

To accurately distinguish HBs from other cerebellar-and-brainstem tumors using a convolutional neural network model based on a contrast-enhanced brain MRI dataset.

STUDY TYPE

Retrospective.

POPULATION

Four hundred five patients (182 = HBs; 223 = other cerebellar-and brainstem tumors): 305 cases for model training, and 100 for evaluation.

FIELD STRENGTH/SEQUENCE

3 T/contrast-enhanced T1-weighted imaging (T1WI + C).

ASSESSMENT

A CNN-based 2D classification network was trained by using sliced data along the z-axis. To improve the performance of the network, we introduced demographic information, various data-augmentation methods and an auxiliary task to segment tumor region. Then, this method was compared with the evaluations performed by experienced and intermediate-level neuroradiologists, and the heatmap of deep feature, which indicates the contribution of each pixel to model prediction, was visualized by Grad-CAM for analyzing the misclassified cases.

STATISTICAL TESTS

The Pearson chi-square test and an independent t-test were used to test for distribution difference in age and sex. And the independent t-test was exploited to evaluate the performance between experts and our proposed method. P value <0.05 was considered significant.

RESULTS

The trained network showed a higher accuracy for identifying HBs (accuracy = 0.902 ± 0.031, F1 = 0.891 ± 0.035, AUC = 0.926 ± 0.040) than experienced (accuracy = 0.887 ± 0.013, F1 = 0.868 ± 0.011, AUC = 0.881 ± 0.008) and intermediate-level (accuracy = 0.827 ± 0.037, F1 = 0.768 ± 0.068, AUC = 0.810 ± 0.047) neuroradiologists. The recall values were 0.910 ± 0.050, 0.659 ± 0.084, and 0.828 ± 0.019 for the trained network, intermediate and experienced neuroradiologists, respectively. Additional ablation experiments verified the utility of the introduced demographic information, data augmentation, and the auxiliary-segmentation task.

DATA CONCLUSION

Our proposed method can successfully distinguish HBs from other cerebellar-and-brainstem tumors and showed diagnostic efficiency comparable to that of experienced neuroradiologists.

EVIDENCE LEVEL

3 TECHNICAL EFFICACY: Stage 2.

Intelligent digital tools for screening of brain connectivity and dementia risk estimation in people affected by mild cognitive impairment: the AI-Mind clinical study protocol

More than 10 million Europeans show signs of mild cognitive impairment (MCI), a transitional stage between normal brain aging and dementia stage memory disorder. The path MCI takes can be divergent; while some maintain stability or even revert to cognitive norms, alarmingly, up to half of the cases progress to dementia within 5 years. Current diagnostic practice lacks the necessary screening tools to identify those at risk of progression. The European patient experience often involves a long journey from the initial signs of MCI to the eventual diagnosis of dementia. The trajectory is far from ideal. Here, we introduce the AI-Mind project, a pioneering initiative with an innovative approach to early risk assessment through the implementation of advanced artificial intelligence (AI) on multimodal data. The cutting-edge AI-based tools developed in the project aim not only to accelerate the diagnostic process but also to deliver highly accurate predictions regarding an individual's risk of developing dementia when prevention and intervention may still be possible. AI-Mind is a European Research and Innovation Action (RIA H2020-SC1-BHC-06-2020, No. 964220) financed between 2021 and 2026. First, the AI-Mind Connector identifies dysfunctional brain networks based on high-density magneto- and electroencephalography (M/EEG) recordings. Second, the AI-Mind Predictor predicts dementia risk using data from the Connector, enriched with computerized cognitive tests, genetic and protein biomarkers, as well as sociodemographic and clinical variables. AI-Mind is integrated within a network of major European initiatives, including The Virtual Brain, The Virtual Epileptic Patient, and EBRAINS AISBL service for sensitive data, HealthDataCloud, where big patient data are generated for advancing digital and virtual twin technology development. AI-Mind's innovation lies not only in its early prediction of dementia risk, but it also enables a virtual laboratory scenario for hypothesis-driven personalized intervention research. This article introduces the background of the AI-Mind project and its clinical study protocol, setting the stage for future scientific contributions.

Screening Targets and Therapeutic Drugs for Alzheimer's Disease Based on Deep Learning Model and Molecular Docking

Background

Alzheimer's disease (AD) is a neurodegenerative disorder caused by a complex interplay of various factors. However, a satisfactory cure for AD remains elusive. Pharmacological interventions based on drug targets are considered the most cost-effective therapeutic strategy. Therefore, it is paramount to search potential drug targets and drugs for AD.

Objective

We aimed to provide novel targets and drugs for the treatment of AD employing transcriptomic data of AD and normal control brain tissues from a new perspective.

Methods

Our study combined the use of a multi-layer perceptron (MLP) with differential expression analysis, variance assessment and molecular docking to screen targets and drugs for AD.

Results

We identified the seven differentially expressed genes (DEGs) with the most significant variation (ANKRD39, CPLX1, FABP3, GABBR2, GNG3, PPM1E, and WDR49) in transcriptomic data from AD brain. A newly built MLP was used to confirm the association between the seven DEGs and AD, establishing these DEGs as potential drug targets. Drug databases and molecular docking results indicated that arbaclofen, baclofen, clozapine, arbaclofen placarbil, BML-259, BRD-K72883421, and YC-1 had high affinity for GABBR2, and FABP3 bound with oleic, palmitic, and stearic acids. Arbaclofen and YC-1 activated GABAB receptor through PI3K/AKT and PKA/CREB pathways, respectively, thereby promoting neuronal anti-apoptotic effect and inhibiting p-tau and Aβ formation.

Conclusions

This study provided a new strategy for the identification of targets and drugs for the treatment of AD using deep learning. Seven therapeutic targets and ten drugs were selected by using this method, providing new insight for AD treatment.

Early Diagnosing and Transformation Prediction of Alzheimer's Disease Using Multi-Scaled Self-Attention Network on Structural MRI Images with Occlusion Sensitivity Analysis

BACKGROUND

Structural magnetic resonance imaging (sMRI) is vital for early Alzheimer's disease (AD) diagnosis, though confirming specific biomarkers remains challenging. Our proposed Multi-Scale Self-Attention Network (MUSAN) enhances classification of cognitively normal (CN) and AD individuals, distinguishing stable (sMCI) from progressive mild cognitive impairment (pMCI).

OBJECTIVE

This study leverages AD structural atrophy properties to achieve precise AD classification, combining different scales of brain region features. The ultimate goal is an interpretable algorithm for this method.

METHODS

The MUSAN takes whole-brain sMRI as input, enabling automatic extraction of brain region features and modeling of correlations between different scales of brain regions, and achieves personalized disease interpretation of brain regions. Furthermore, we also employed an occlusion sensitivity algorithm to localize and visualize brain regions sensitive to disease.

RESULTS

Our method is applied to ADNI-1, ADNI-2, and ADNI-3, and achieves high performance on the classification of CN from AD with accuracy (0.93), specificity (0.82), sensitivity (0.96), and area under curve (AUC) (0.95), as well as notable performance on the distinguish of sMCI from pMCI with accuracy (0.85), specificity (0.84), sensitivity (0.74), and AUC (0.86). Our sensitivity masking algorithm identified key regions in distinguishing CN from AD: hippocampus, amygdala, and vermis. Moreover, cingulum, pallidum, and inferior frontal gyrus are crucial for sMCI and pMCI discrimination. These discoveries align with existing literature, confirming the dependability of our model in AD research.

CONCLUSION

Our method provides an effective AD diagnostic and conversion prediction method. The occlusion sensitivity algorithm enhances deep learning interpretability, bolstering AD research reliability.

A scoping review of interpretability and explainability concerning artificial intelligence methods in medical imaging

PURPOSE

To review eXplainable Artificial Intelligence/(XAI) methods available for medical imaging/(MI).

METHOD

A scoping review was conducted following the Joanna Briggs Institute's methodology. The search was performed on Pubmed, Embase, Cinhal, Web of Science, BioRxiv, MedRxiv, and Google Scholar. Studies published in French and English after 2017 were included. Keyword combinations and descriptors related to explainability, and MI modalities were employed. Two independent reviewers screened abstracts, titles and full text, resolving differences through discussion.

RESULTS

228 studies met the criteria. XAI publications are increasing, targeting MRI (n = 73), radiography (n = 47), CT (n = 46). Lung (n = 82) and brain (n = 74) pathologies, Covid-19 (n = 48), Alzheimer's disease (n = 25), brain tumors (n = 15) are the main pathologies explained. Explanations are presented visually (n = 186), numerically (n = 67), rule-based (n = 11), textually (n = 11), and example-based (n = 6). Commonly explained tasks include classification (n = 89), prediction (n = 47), diagnosis (n = 39), detection (n = 29), segmentation (n = 13), and image quality improvement (n = 6). The most frequently provided explanations were local (78.1 %), 5.7 % were global, and 16.2 % combined both local and global approaches. Post-hoc approaches were predominantly employed. The used terminology varied, sometimes indistinctively using explainable (n = 207), interpretable (n = 187), understandable (n = 112), transparent (n = 61), reliable (n = 31), and intelligible (n = 3).

CONCLUSION

The number of XAI publications in medical imaging is increasing, primarily focusing on applying XAI techniques to MRI, CT, and radiography for classifying and predicting lung and brain pathologies. Visual and numerical output formats are predominantly used. Terminology standardisation remains a challenge, as terms like "explainable" and "interpretable" are sometimes being used indistinctively. Future XAI development should consider user needs and perspectives.

The promise of explainable deep learning for omics data analysis: Adding new discovery tools to AI

Deep learning has already revolutionised the way a wide range of data is processed in many areas of daily life. The ability to learn abstractions and relationships from heterogeneous data has provided impressively accurate prediction and classification tools to handle increasingly big datasets. This has a significant impact on the growing wealth of omics datasets, with the unprecedented opportunity for a better understanding of the complexity of living organisms. While this revolution is transforming the way these data are analyzed, explainable deep learning is emerging as an additional tool with the potential to change the way biological data is interpreted. Explainability addresses critical issues such as transparency, so important when computational tools are introduced especially in clinical environments. Moreover, it empowers artificial intelligence with the capability to provide new insights into the input data, thus adding an element of discovery to these already powerful resources. In this review, we provide an overview of the transformative effects explainable deep learning is having on multiple sectors, ranging from genome engineering and genomics, from radiomics to drug design and clinical trials. We offer a perspective to life scientists, to better understand the potential of these tools, and a motivation to implement them in their research, by suggesting learning resources they can use to move their first steps in this field.

Comparing a pre-defined versus deep learning approach for extracting brain atrophy patterns to predict cognitive decline due to Alzheimer's disease in patients with mild cognitive symptoms

Background

Predicting future Alzheimer's disease (AD)-related cognitive decline among individuals with subjective cognitive decline (SCD) or mild cognitive impairment (MCI) is an important task for healthcare. Structural brain imaging as measured by magnetic resonance imaging (MRI) could potentially contribute when making such predictions. It is unclear if the predictive performance of MRI can be improved using entire brain images in deep learning (DL) models compared to using pre-defined brain regions.

Methods

A cohort of 332 individuals with SCD/MCI were included from the Swedish BioFINDER-1 study. The goal was to predict longitudinal SCD/MCI-to-AD dementia progression and change in Mini-Mental State Examination (MMSE) over four years. Four models were evaluated using different predictors: 1) clinical data only, including demographics, cognitive tests and APOE e4 status, 2) clinical data plus hippocampal volume, 3) clinical data plus all regional MRI gray matter volumes (N=68) extracted using FreeSurfer software, 4) a DL model trained using multi-task learning with MRI images, Jacobian determinant images and baseline cognition as input. Models were developed on 80% of subjects (N=267) and tested on the remaining 20% (N=65). Mann-Whitney U-test was used to determine statistically significant differences in performance, with p-values less than 0.05 considered significant.

Results

In the test set, 21 patients (32.3%) progressed to AD dementia. The performance of the clinical data model for prediction of progression to AD dementia was area under the curve (AUC)=0.87 and four-year cognitive decline was R2=0.17. The performance was significantly improved for both outcomes when adding hippocampal volume (AUC=0.91, R2=0.26, p-values <0.05) or FreeSurfer brain regions (AUC=0.90, R2=0.27, p-values <0.05). Conversely, the DL model did not show any significant difference from the clinical data model (AUC=0.86, R2=0.13). A sensitivity analysis showed that the Jacobian determinant image was more informative than the MRI image, but that performance was maximized when both were included.

Conclusions

The DL model did not significantly improve the prediction of clinical disease progression in AD, compared to regression models with a single pre-defined brain region.

Improving Alzheimer Diagnoses With An Interpretable Deep Learning Framework: Including Neuropsychiatric Symptoms

Alzheimer's disease (AD) is a prevalent neurodegenerative disorder characterized by the progressive cognitive decline. Among the various clinical symptoms, neuropsychiatric symptoms (NPS) commonly occur during the course of AD. Previous researches have demonstrated a strong association between NPS and severity of AD, while the research methods are not sufficiently intuitive. Here, we report a hybrid deep learning framework for AD diagnosis using multimodal inputs such as structural MRI, behavioral scores, age, and gender information. The framework uses a 3D convolutional neural network to automatically extract features from MRI. The imaging features are passed to the Principal Component Analysis for dimensionality reduction, which fuse with non-imaging information to improve the diagnosis of AD. According to the experimental results, our model achieves an accuracy of 0.91 and an area under the curve of 0.97 in the task of classifying AD and cognitively normal individuals. SHapley Additive exPlanations are used to visually exhibit the contribution of specific NPS in the proposed model. Among all behavioral symptoms, apathy plays a particularly important role in the diagnosis of AD, which can be considered a valuable factor in further studies, as well as clinical trials.

A neuroimaging biomarker for Individual Brain-Related Abnormalities In Neurodegeneration (IBRAIN): a cross-sectional study

Background

Alzheimer's disease (AD) is a prevalent neurodegenerative disorder that poses a worldwide public health challenge. A neuroimaging biomarker would significantly improve early diagnosis and intervention, ultimately enhancing the quality of life for affected individuals and reducing the burden on healthcare systems.

Methods

Cross-sectional and longitudinal data (10,099 participants with 13,380 scans) from 12 independent datasets were used in the present study (this study was performed between September 1, 2021 and February 15, 2023). The Individual Brain-Related Abnormalities In Neurodegeneration (IBRAIN) score was developed via integrated regional- and network-based measures under an ensemble machine learning model based on structural MRI data. We systematically assessed whether IBRAIN could be a neuroimaging biomarker for AD.

Findings

IBRAIN accurately differentiated individuals with AD from NCs (AUC = 0.92) and other neurodegenerative diseases, including Frontotemporal dementia (FTD), Parkinson's disease (PD), Vascular dementia (VaD) and Amyotrophic Lateral Sclerosis (ALS) (AUC = 0.92). IBRAIN was significantly correlated to clinical measures and gene expression, enriched in immune process and protein metabolism. The IBRAIN score exhibited a significant ability to reveal the distinct progression of prodromal AD (i.e., Mild cognitive impairment, MCI) (Hazard Ratio (HR) = 6.52 [95% CI: 4.42∼9.62], p < 1 × 10-16), which offers similar powerful performance with Cerebrospinal Fluid (CSF) Aβ (HR = 3.78 [95% CI: 2.63∼5.43], p = 2.13 × 10-14) and CSF Tau (HR = 3.77 [95% CI: 2.64∼5.39], p = 9.53 × 10-15) based on the COX and Log-rank test. Notably, the IBRAIN shows comparable sensitivity (beta = -0.70, p < 1 × 10-16) in capturing longitudinal changes in individuals with conversion to AD than CSF Aβ (beta = -0.26, p = 4.40 × 10-9) and CSF Tau (beta = 0.12, p = 1.02 × 10-5).

Interpretation

Our findings suggested that IBRAIN is a biologically relevant, specific, and sensitive neuroimaging biomarker that can serve as a clinical measure to uncover prodromal AD progression. It has strong potential for application in future clinical practice and treatment trials.

Funding

Science and Technology Innovation 2030 Major Projects, the National Natural Science Foundation of China, Beijing Natural Science Funds, the Fundamental Research Funds for the CentralUniversity, and the Startup Funds for Talents at Beijing Normal University.

A multimodal deep learning model for predicting severe hemorrhage in placenta previa

Placenta previa causes life-threatening bleeding and accurate prediction of severe hemorrhage leads to risk stratification and optimum allocation of interventions. We aimed to use a multimodal deep learning model to predict severe hemorrhage. Using MRI T2-weighted image of the placenta and tabular data consisting of patient demographics and preoperative blood examination data, a multimodal deep learning model was constructed to predict cases of intraoperative blood loss > 2000 ml. We evaluated the prediction performance of the model by comparing it with that of two machine learning methods using only tabular data and MRI images, as well as with that of two human expert obstetricians. Among the enrolled 48 patients, 26 (54.2%) lost > 2000 ml of blood and 22 (45.8%) lost < 2000 ml of blood. Multimodal deep learning model showed the best accuracy of 0.68 and AUC of 0.74, whereas the machine learning model using tabular data and MRI images had a class accuracy of 0.61 and 0.53, respectively. The human experts had median accuracies of 0.61. Multimodal deep learning models could integrate the two types of information and predict severe hemorrhage cases. The model might assist human expert in the prediction of intraoperative hemorrhage in the case of placenta previa.

Potential Ocular Biomarkers for Early Detection of Alzheimer's Disease and Their Roles in Artificial Intelligence Studies

Alzheimer's disease (AD) is the leading cause of dementia worldwide. Early detection is believed to be essential to disease management because it enables physicians to initiate treatment in patients with early-stage AD (early AD), with the possibility of stopping the disease or slowing disease progression, preserving function and ultimately reducing disease burden. The purpose of this study was to review prior research on the use of eye biomarkers and artificial intelligence (AI) for detecting AD and early AD. The PubMed database was searched to identify studies for review. Ocular biomarkers in AD research and AI research on AD were reviewed and summarized. According to numerous studies, there is a high likelihood that ocular biomarkers can be used to detect early AD: tears, corneal nerves, retina, visual function and, in particular, eye movement tracking have been identified as ocular biomarkers with the potential to detect early AD. However, there is currently no ocular biomarker that can be used to definitely detect early AD. A few studies that used AI with ocular biomarkers to detect AD reported promising results, demonstrating that using AI with ocular biomarkers through multimodal imaging could improve the accuracy of identifying AD patients. This strategy may become a screening tool for detecting early AD in older patients prior to the onset of AD symptoms.

The use of neuroimaging techniques in the early and differential diagnosis of dementia

Dementia is a leading cause of disability and death worldwide. At present there is no disease modifying treatment for any of the most common types of dementia such as Alzheimer's disease (AD), Vascular dementia, Lewy Body Dementia (LBD) and Frontotemporal dementia (FTD). Early and accurate diagnosis of dementia subtype is critical to improving clinical care and developing better treatments. Structural and molecular imaging has contributed to a better understanding of the pathophysiology of neurodegenerative dementias and is increasingly being adopted into clinical practice for early and accurate diagnosis. In this review we summarise the contribution imaging has made with particular focus on multimodal magnetic resonance imaging (MRI) and positron emission tomography imaging (PET). Structural MRI is widely used in clinical practice and can help exclude reversible causes of memory problems but has relatively low sensitivity for the early and differential diagnosis of dementia subtypes. 18F-fluorodeoxyglucose PET has high sensitivity and specificity for AD and FTD, while PET with ligands for amyloid and tau can improve the differential diagnosis of AD and non-AD dementias, including recognition at prodromal stages. Dopaminergic imaging can assist with the diagnosis of LBD. The lack of a validated tracer for α-synuclein or TAR DNA-binding protein 43 (TDP-43) imaging remain notable gaps, though work is ongoing. Emerging PET tracers such as 11C-UCB-J for synaptic imaging may be sensitive early markers but overall larger longitudinal multi-centre cross diagnostic imaging studies are needed.

Brain Status Transferring Generative Adversarial Network for Decoding Individualized Atrophy in Alzheimer's Disease

Deep learning has been widely investigated in brain image computational analysis for diagnosing brain diseases such as Alzheimer's disease (AD). Most of the existing methods built end-to-end models to learn discriminative features by group-wise analysis. However, these methods cannot detect pathological changes in each subject, which is essential for the individualized interpretation of disease variances and precision medicine. In this article, we propose a brain status transferring generative adversarial network (BrainStatTrans-GAN) to generate corresponding healthy images of patients, which are further used to decode individualized brain atrophy. The BrainStatTrans-GAN consists of generator, discriminator, and status discriminator. First, a normative GAN is built to generate healthy brain images from normal controls. However, it cannot generate healthy images from diseased ones due to the lack of paired healthy and diseased images. To address this problem, a status discriminator with adversarial learning is designed in the training process to produce healthy brain images for patients. Then, the residual between the generated and input images can be computed to quantify pathological brain changes. Finally, a residual-based multi-level fusion network (RMFN) is built for more accurate disease diagnosis. Compared to the existing methods, our method can model individualized brain atrophy for facilitating disease diagnosis and interpretation. Experimental results on T1-weighted magnetic resonance imaging (MRI) data of 1,739 subjects from three datasets demonstrate the effectiveness of our method.

Disease-driven domain generalization for neuroimaging-based assessment of Alzheimer's disease

Development of deep learning models to assess the degree of cognitive impairment on magnetic resonance imaging (MRI) scans has high translational significance. Performance of such models is often affected by potential variabilities stemming from independent protocols for data generation, imaging equipment, radiology artifacts, and demographic distributional shifts. Domain generalization (DG) frameworks have the potential to overcome these issues by learning signal from one or more source domains that can be transferable to unseen target domains. We developed an approach that leverages model interpretability as a means to improve generalizability of classification models across multiple cohorts. Using MRI scans and clinical diagnosis obtained from four independent cohorts (Alzheimer's Disease Neuroimaging Initiative (ADNI, n = 1,821), the Framingham Heart Study (FHS, n = 304), the Australian Imaging Biomarkers and Lifestyle Study of Ageing (AIBL, n = 661), and the National Alzheimer's Coordinating Center (NACC, n = 4,647)), we trained a deep neural network that used model-identified regions of disease relevance to inform model training. We trained a classifier to distinguish persons with normal cognition (NC) from those with mild cognitive impairment (MCI) and Alzheimer's disease (AD) by aligning class-wise attention with a unified visual saliency prior computed offline per class over all training data. Our proposed method competes with state-of-the-art methods with improved correlation with postmortem histology, thus grounding our findings with gold standard evidence and paving a way towards validating DG frameworks.

Uncovering the Achilles heel of genetic heterogeneity: machine learning-based classification and immunological properties of necroptosis clusters in Alzheimer's disease

Background

Alzheimer's disease (AD) is an age-associated neurodegenerative disease, and the currently available diagnostic modalities and therapeutic agents are unsatisfactory due to its high clinical heterogeneity. Necroptosis is a common type of programmed cell death that has been shown to be activated in AD.

Methods

In this study, we first investigated the expression profiles of necroptosis-related genes (NRGs) and the immune landscape of AD based on GSE33000 dataset. Next, the AD samples in the GSE33000 dataset were extracted and subjected to consensus clustering based upon the differentially expressed NRGs. Key genes associated with necroptosis clusters were identified using Weighted Gene Co-Expression Network Analysis (WGCNA) algorithm, and then intersected with the key gene related to AD. Finally, we developed a diagnostic model for AD by comparing four different machine learning approaches. The discrimination performance and clinical relevance of the diagnostic model were assessed using various evaluation metrics, including the nomogram, calibration plot, decision curve analysis (DCA), and independent validation datasets.

Results

Aberrant expression patterns of NRGs and specific immune landscape were identified in the AD samples. Consensus clustering revealed that patients in the GSE33000 dataset could be classified into two necroptosis clusters, each with distinct immune landscapes and enriched pathways. The Extreme Gradient Boosting (XGB) was found to be the most optimal diagnostic model for the AD based on the predictive ability and reliability of the models constructed by four machine learning approaches. The five most important variables, including ACAA2, BHLHB4, CACNA2D3, NRN1, and TAC1, were used to construct a five-gene diagnostic model. The constructed nomogram, calibration plot, DCA, and external independent validation datasets exhibited outstanding diagnostic performance for AD and were closely related with the pathologic hallmarks of AD.

Conclusion

This work presents a novel diagnostic model that may serve as a framework to study disease heterogeneity and provide a plausible mechanism underlying neuronal loss in AD.

Deep learning for risk-based stratification of cognitively impaired individuals

Quantifying the risk of progression to Alzheimer's disease (AD) could help identify persons who could benefit from early interventions. We used data from the Alzheimer's Disease Neuroimaging Initiative (ADNI, n = 544, discovery cohort) and the National Alzheimer's Coordinating Center (NACC, n = 508, validation cohort), subdividing individuals with mild cognitive impairment (MCI) into risk groups based on cerebrospinal fluid amyloid-β levels and identifying differential gray matter patterns. We then created models that fused neural networks with survival analysis, trained using non-parcellated T1-weighted brain MRIs from ADNI data, to predict the trajectories of MCI to AD conversion within the NACC cohort (integrated Brier score: 0.192 [discovery], and 0.108 [validation]). Using modern interpretability techniques, we verified that regions important for model prediction are classically associated with AD. We confirmed AD diagnosis labels using postmortem data. We conclude that our framework provides a strategy for risk-based stratification of individuals with MCI and for identifying regions key for disease prognosis.