Robust hybrid deep learning models for Alzheimer’s progression detection

TMS: Ensemble Deep Learning Model for Accurate Classification of Monkeypox Lesions Based on Transformer Models with SVM

BACKGROUND/OBJECTIVES

The emergence of monkeypox outside its endemic region in Africa has raised significant concerns within the public health community due to its rapid global dissemination. Early clinical differentiation of monkeypox from similar diseases, such as chickenpox and measles, presents a challenge. The Monkeypox Skin Lesion Dataset (MSLD) used in this study comprises monkeypox skin lesions, which were collected primarily from publicly accessible sources. The dataset contains 770 original images captured from 162 unique patients. The MSLD includes four distinct class labels: monkeypox, measles, chickenpox, and normal.

METHODS

This paper presents an ensemble model for classifying the monkeypox dataset, which includes transformer models and support vector machine (SVM). The model development process begins with an evaluation of seven convolutional neural network (CNN) architectures. The proposed model is developed by selecting the top four models based on evaluation metrics for performance. The top four CNN architectures, namely EfficientNetB0, ResNet50, MobileNet, and Xception, are used for feature extraction. The high-dimensional feature vectors extracted from each network are then concatenated and optimized before being inputted into the SVM classifier.

RESULTS

The proposed ensemble model, in conjunction with the SVM classifier, achieves an accuracy of 95.45b%. Furthermore, the model demonstrates high precision (95.51%), recall (95.45%), and F1 score (95.46%), indicating its effectiveness in identifying monkeypox lesions.

CONCLUSIONS

The results of the study show that the proposed hybrid framework achieves robust diagnostic performance in monkeypox detection, offering potential utility for enhanced disease monitoring and outbreak management. The model's high diagnostic accuracy and computational efficiency indicate that it can be used as an additional tool for clinical decision support.

ERABiLNet: enhanced residual attention with bidirectional long short-term memory

Alzheimer's Disease (AD) causes slow death in brain cells due to shrinkage of brain cells which is more prevalent in older people. In most cases, the symptoms of AD are mistaken as age-related stresses. The most widely utilized method to detect AD is Magnetic Resonance Imaging (MRI). Along with Artificial Intelligence (AI) techniques, the efficacy of identifying diseases related to the brain has become easier. But, the identical phenotype makes it challenging to identify the disease from the neuro-images. Hence, a deep learning method to detect AD at the beginning stage is suggested in this work. The newly implemented "Enhanced Residual Attention with Bi-directional Long Short-Term Memory (Bi-LSTM) (ERABi-LNet)" is used in the detection phase to identify the AD from the MRI images. This model is used for enhancing the performance of the Alzheimer's detection in scale of 2-5%, minimizing the error rates, increasing the balance of the model, so that the multi-class problems are supported. At first, MRI images are given to "Residual Attention Network (RAN)", which is specially developed with three convolutional layers, namely atrous, dilated and Depth-Wise Separable (DWS), to obtain the relevant attributes. The most appropriate attributes are determined by these layers, and subjected to target-based fusion. Then the fused attributes are fed into the "Attention-based Bi-LSTM". The final outcome is obtained from this unit. The detection efficiency based on median is 26.37% and accuracy is 97.367% obtained by tuning the parameters in the ERABi-LNet with the help of Modified Search and Rescue Operations (MCDMR-SRO). The obtained results are compared with ROA-ERABi-LNet, EOO-ERABi-LNet, GTBO-ERABi-LNet and SRO-ERABi-LNet respectively. The ERABi_LNet thus provides enhanced accuracy and other performance metrics compared to such deep learning models. The proposed method has the better sensitivity, specificity, F1-Score and False Positive Rate compared with all the above mentioned competing models with values such as 97.49%.97.84%,97.74% and 2.616 respective;y. This ensures that the model has better learning capabilities and provides lesser false positives with balanced prediction.

MAPPING ALZHEIMER'S DISEASE PSEUDO-PROGRESSION WITH MULTIMODAL BIOMARKER TRAJECTORY EMBEDDINGS

Alzheimer's Disease (AD) is a neurodegenerative disorder characterized by progressive cognitive degeneration and motor impairment, affecting millions worldwide. Mapping the progression of AD is crucial for early detection of loss of brain function, timely intervention, and development of effective treatments. However, accurate measurements of disease progression are still challenging at present. This study presents a novel approach to understanding the heterogeneous pathways of AD through longitudinal biomarker data from medical imaging and other modalities. We propose an analytical pipeline adopting two popular machine learning methods from the single-cell transcriptomics domain, PHATE and Slingshot, to project multimodal biomarker trajectories to a low-dimensional space. These embeddings serve as our pseudotime estimates. We applied this pipeline to the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset to align longitudinal data across individuals at various disease stages. Our approach mirrors the technique used to cluster single-cell data into cell types based on developmental timelines. Our pseudotime estimates revealed distinct patterns of disease evolution and biomarker changes over time, providing a deeper understanding of the temporal dynamics of AD. The results show the potential of the approach in the clinical domain of neurodegenerative diseases, enabling more precise disease modeling and early diagnosis.

An Alzheimer's Disease classification network based on MRI utilizing diffusion maps for multi-scale feature fusion in graph convolution

Graph convolutional networks (GCN) have been widely utilized in Alzheimer's disease (AD) classification research due to its ability to automatically learn robust and powerful feature representations. Inter-patient relationships are effectively captured by constructing patients magnetic resonance imaging (MRI) data as graph data, where nodes represent individuals and edges denote the relationships between them. However, the performance of GCNs might be constrained by the construction of the graph adjacency matrix, thereby leading to learned features potentially overlooking intrinsic correlations among patients, which ultimately causes inaccurate disease classifications. To address this issue, we propose an Alzheimer's disease Classification network based on MRI utilizing diffusion maps for multi-scale feature fusion in graph convolution. This method aims to tackle the problem of features neglecting intrinsic relationships among patients while integrating features from diffusion mapping with different neighbor counts to better represent patients and achieve an accurate AD classification. Initially, the diffusion maps method conducts diffusion information in the feature space, thus breaking free from the constraints of diffusion based on the adjacency matrix. Subsequently, the diffusion features with different neighbor counts are merged, and a self-attention mechanism is employed to adaptively adjust the weights of diffusion features at different scales, thereby comprehensively and accurately capturing patient characteristics. Finally, metric learning techniques enhance the similarity of node features within the same category in the graph structure and bring node features of different categories more distant from each other. This study aims to enhance the classification accuracy of AD, by providing an effective tool for early diagnosis and intervention. It offers valuable information for clinical decisions and personalized treatment. Experimentation on the publicly accessible Alzheimer's disease neuroimaging initiative (ADNI) dataset validated our method's competitive performance across various AD-related classification tasks. Compared to existing methodologies, our approach captures patient characteristics more effectively and demonstrates superior generalization capabilities.

Employing Deep-Learning Approach for the Early Detection of Mild Cognitive Impairment Transitions through the Analysis of Digital Biomarkers

Mild cognitive impairment (MCI) is the precursor to the advanced stage of Alzheimer's disease (AD), and it is important to detect the transition to the MCI condition as early as possible. Trends in daily routines/activities provide a measurement of cognitive/functional status, particularly in older adults. In this study, activity data from longitudinal monitoring through in-home ambient sensors are leveraged in predicting the transition to the MCI stage at a future time point. The activity dataset from the Oregon Center for Aging and Technology (ORCATECH) includes measures representing various domains such as walk, sleep, etc. Each sensor-captured activity measure is constructed as a time series, and a variety of summary statistics is computed. The similarity between one individual's activity time series and that of the remaining individuals is also computed as distance measures. The long short-term memory (LSTM) recurrent neural network is trained with time series statistics and distance measures for the prediction modeling, and performance is evaluated by classification accuracy. The model outcomes are explained using the SHapley Additive exPlanations (SHAP) framework. LSTM model trained using the time series statistics and distance measures outperforms other modeling scenarios, including baseline classifiers, with an overall prediction accuracy of 83.84%. SHAP values reveal that sleep-related features contribute the most to the prediction of the cognitive stage at the future time point, and this aligns with the findings in the literature. Findings from this study not only demonstrate that a practical, less expensive, longitudinal monitoring of older adults' activity routines can benefit immensely in modeling AD progression but also unveil the most contributing features that are medically applicable and meaningful.

Computer aided progression detection model based on optimized deep LSTM ensemble model and the fusion of multivariate time series data

Alzheimer's disease (AD) is the most common form of dementia. Early and accurate detection of AD is crucial to plan for disease modifying therapies that could prevent or delay the conversion to sever stages of the disease. As a chronic disease, patient's multivariate time series data including neuroimaging, genetics, cognitive scores, and neuropsychological battery provides a complete profile about patient's status. This data has been used to build machine learning and deep learning (DL) models for the early detection of the disease. However, these models still have limited performance and are not stable enough to be trusted in real medical settings. Literature shows that DL models outperform classical machine learning models, but ensemble learning has proven to achieve better results than standalone models. This study proposes a novel deep stacking framework which combines multiple DL models to accurately predict AD at an early stage. The study uses long short-term memory (LSTM) models as base models over patient's multivariate time series data to learn the deep longitudinal features. Each base LSTM classifier has been optimized using the Bayesian optimizer using different feature sets. As a result, the final optimized ensembled model employed heterogeneous base models that are trained on heterogeneous data. The performance of the resulting ensemble model has been explored using a cohort of 685 patients from the University of Washington's National Alzheimer's Coordinating Center dataset. Compared to the classical machine learning models and base LSTM classifiers, the proposed ensemble model achieves the highest testing results (i.e., 82.02, 82.25, 82.02, and 82.12 for accuracy, precision, recall, and F1-score, respectively). The resulting model enhances the performance of the state-of-the-art literature, and it could be used to build an accurate clinical decision support tool that can assist domain experts for AD progression detection.

Disease progression modelling of Alzheimer's disease using probabilistic principal components analysis

The recent biological redefinition of Alzheimer's Disease (AD) has spurred the development of statistical models that relate changes in biomarkers with neurodegeneration and worsening condition linked to AD. The ability to measure such changes may facilitate earlier diagnoses for affected individuals and help in monitoring the evolution of their condition. Amongst such statistical tools, disease progression models (DPMs) are quantitative, data-driven methods that specifically attempt to describe the temporal dynamics of biomarkers relevant to AD. Due to the heterogeneous nature of this disease, with patients of similar age experiencing different AD-related changes, a challenge facing longitudinal mixed-effects-based DPMs is the estimation of patient-realigning time-shifts. These time-shifts are indispensable for meaningful biomarker modelling, but may impact fitting time or vary with missing data in jointly estimated models. In this work, we estimate an individual's progression through Alzheimer's disease by combining multiple biomarkers into a single value using a probabilistic formulation of principal components analysis. Our results show that this variable, which summarises AD through observable biomarkers, is remarkably similar to jointly estimated time-shifts when we compute our scores for the baseline visit, on cross-sectional data from the Alzheimer's Disease Neuroimaging Initiative (ADNI). Reproducing the expected properties of clinical datasets, we confirm that estimated scores are robust to missing data or unavailable biomarkers. In addition to cross-sectional insights, we can model the latent variable as an individual progression score by repeating estimations at follow-up examinations and refining long-term estimates as more data is gathered, which would be ideal in a clinical setting. Finally, we verify that our score can be used as a pseudo-temporal scale instead of age to ignore some patient heterogeneity in cohort data and highlight the general trend in expected biomarker evolution in affected individuals.

Genetic algorithms based optimized hybrid deep learning model for explainable Alzheimer's prediction based on temporal multimodal cognitive data.. [DOI: 10.21203/rs.3.rs-3250006/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Alzheimer's Disease (AD) is an irreversible neurodegenerative disease. Its early detection is crucial to stop disease progression at an early stage. Most deep learning (DL) literature focused on neuroimage analysis. However, there is no noticed effect of these studies in the real environment. Model's robustness, cost, and interpretability are considered the main reasons for these limitations. The medical intuition of physicians is to evaluate the clinical biomarkers of patients then test their neuroimages. Cognitive scores provide an medically acceptable and cost-effective alternative for the neuroimages to predict AD progression. Each score is calculated from a collection of sub-scores which provide a deeper insight about patient conditions. No study in the literature have explored the role of these multimodal time series sub-scores to predict AD progression. We propose a hybrid CNN-LSTM DL model for predicting AD progression based on the fusion of four longitudinal cognitive sub-scores modalities. Bayesian optimizer has been used to select the best DL architecture. A genetic algorithms based feature selection optimization step has been added to the pipeline to select the best features from extracted deep representations of CNN-LSTM. The SoftMax classifier has been replaced by a robust and optimized random forest classifier. Extensive experiments using the ADNI dataset investigated the role of each optimization step, and the proposed model achieved the best results compared to other DL and classical machine learning models. The resulting model is robust, but it is a black box and it is difficult to understand the logic behind its decisions. Trustworthy AI models must be robust and explainable. We used SHAP and LIME to provide explainability features for the proposed model. The resulting trustworthy model has a great potential to be used to provide decision support in the real environments.

Improving Alzheimer's Disease Diagnosis With Multi-Modal PET Embedding Features by a 3D Multi-Task MLP-Mixer Neural Network

Positron emission tomography (PET) with fluorodeoxyglucose (FDG) or florbetapir (AV45) has been proved effective in the diagnosis of Alzheimer's disease. However, the expensive and radioactive nature of PET has limited its application. Here, employing multi-layer perceptron mixer architecture, we present a deep learning model, namely 3-dimensional multi-task multi-layer perceptron mixer, for simultaneously predicting the standardized uptake value ratios (SUVRs) for FDG-PET and AV45-PET from the cheap and widely used structural magnetic resonance imaging data, and the model can be further used for Alzheimer's disease diagnosis based on embedding features derived from SUVR prediction. Experiment results demonstrate the high prediction accuracy of the proposed method for FDG/AV45-PET SUVRs, where we achieved Pearson's correlation coefficients of 0.66 and 0.61 respectively between the estimated and actual SUVR and the estimated SUVRs also show high sensitivity and distinct longitudinal patterns for different disease status. By taking into account PET embedding features, the proposed method outperforms other competing methods on five independent datasets in the diagnosis of Alzheimer's disease and discriminating between stable and progressive mild cognitive impairments, achieving the area under receiver operating characteristic curves of 0.968 and 0.776 respectively on ADNI dataset, and generalizes better to other external datasets. Moreover, the top-weighted patches extracted from the trained model involve important brain regions related to Alzheimer's disease, suggesting good biological interpretability of our proposed method."

Explainable machine learning models based on multimodal time-series data for the early detection of Parkinson's disease

BACKGROUND AND OBJECTIVES

Parkinson's Disease (PD) is a devastating chronic neurological condition. Machine learning (ML) techniques have been used in the early prediction of PD progression. Fusion of heterogeneous data modalities proved its capability to improve the performance of ML models. Time series data fusion supports the tracking of the disease over time. In addition, the trustworthiness of the resulting models is improved by adding model explainability features. The literature on PD has not sufficiently explored these three points.

METHODS

In this work, we proposed an ML pipeline for predicting the progression of PD that is both accurate and explainable. We explore the fusion of different combinations of five time series modalities from the Parkinson's Progression Markers Initiative (PPMI) real-world dataset, including patient characteristics, biosamples, medication history, motor, and non-motor function data. Each patient has six visits. The problem has been formulated in two ways: ❶ a three-class based progression prediction with 953 patients in each time series modality, and ❷ a four-class based progression prediction with 1,060 patients in each time series modality. The statistical features of these six visits were calculated from each modality and diverse feature selection methods were applied to select the most informative feature sets. The extracted features were used to train a set of well-known ML models including Support vector machines (SVM), random forests (RF), extra tree classifier (ETC), light gradient boosting machines (LGBM), and stochastic gradient descent (SGD). We examined a number of data-balancing strategies in the pipeline with different combinations of modalities. ML models have been optimized using the Bayesian optimizer. A comprehensive evaluation of various ML methods has been conducted, and the best models have been extended to provide different explainability features.

RESULTS

We compare the performance of ML models before and after optimization and using and without using feature selection. In the three-class experiment and with various modality fusions, the LGBM model produced the most accurate results with a 10-fold cross-validation (10-CV) accuracy of 90.73% using non-motor function modality. RF produced the best results in the four-class experiment with various modality fusions with a 10-CV accuracy of 94.57% using non-motor modality. With the fused dataset of non-motor and motor function modalities, the LGBM model outperformed the other ML models in both the 3-class and 4-class experiments (i.e., 10-CV accuracy of 94.89% and 93.73%, respectively). Using the Shapely Additive Explanations (SHAP) framework, we employed global and instance-based explanations to explain the behavior of each ML classifier. Moreover, we extended the explainability by implementing the LIME and SHAPASH local explainers. The consistency of these explainers has been explored. The resultant classifiers were accurate, explainable, and thus medically more relevant and applicable.

CONCLUSIONS

The select modalities and feature sets were confirmed by the literature and medical experts. The various explainers suggest that the bradykinesia (NP3BRADY) feature was the most dominant and consistent. By providing thorough insights into the influence of multiple modalities on the disease risk, the suggested approach is expected to help improve the clinical knowledge of PD progression processes.

A scoping review of neurodegenerative manifestations in explainable digital phenotyping

Neurologists nowadays no longer view neurodegenerative diseases, like Parkinson's and Alzheimer's disease, as single entities, but rather as a spectrum of multifaceted symptoms with heterogeneous progression courses and treatment responses. The definition of the naturalistic behavioral repertoire of early neurodegenerative manifestations is still elusive, impeding early diagnosis and intervention. Central to this view is the role of artificial intelligence (AI) in reinforcing the depth of phenotypic information, thereby supporting the paradigm shift to precision medicine and personalized healthcare. This suggestion advocates the definition of disease subtypes in a new biomarker-supported nosology framework, yet without empirical consensus on standardization, reliability and interpretability. Although the well-defined neurodegenerative processes, linked to a triad of motor and non-motor preclinical symptoms, are detected by clinical intuition, we undertake an unbiased data-driven approach to identify different patterns of neuropathology distribution based on the naturalistic behavior data inherent to populations in-the-wild. We appraise the role of remote technologies in the definition of digital phenotyping specific to brain-, body- and social-level neurodegenerative subtle symptoms, emphasizing inter- and intra-patient variability powered by deep learning. As such, the present review endeavors to exploit digital technologies and AI to create disease-specific phenotypic explanations, facilitating the understanding of neurodegenerative diseases as "bio-psycho-social" conditions. Not only does this translational effort within explainable digital phenotyping foster the understanding of disease-induced traits, but it also enhances diagnostic and, eventually, treatment personalization.

Automatic Detection of Alzheimer's Disease using Deep Learning Models and Neuro-Imaging: Current Trends and Future Perspectives

Deep learning algorithms have a huge influence on tackling research issues in the field of medical image processing. It acts as a vital aid for the radiologists in producing accurate results toward effective disease diagnosis. The objective of this research is to highlight the importance of deep learning models in the detection of Alzheimer's Disease (AD). The main objective of this research is to analyze different deep learning methods used for detecting AD. This study examines 103 research articles published in various research databases. These articles have been selected based on specific criteria to find the most relevant findings in the field of AD detection. The review was carried out based on deep learning techniques such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Transfer Learning (TL). To propose accurate methods for the detection, segmentation, and severity grading of AD, the radiological features need to be examined in greater depth. This review attempts to analyze different deep learning methods applied for AD detection using neuroimaging modalities like Positron Emission Tomography (PET), Magnetic Resonance Imaging (MRI), etc. The focus of this review is restricted to deep learning works based on radiological imaging data for AD detection. There are a few works that have utilized other biomarkers to understand the effect of AD. Also, articles published in English were alone considered for analysis. This work concludes by highlighting the key research issues towards effective AD detection. Though several methods have yielded promising results in AD detection, the progression from Mild Cognitive Impairment (MCI) to AD need to be analyzed in greater depth using DL models.

A Novel Proposal for Deep Learning-Based Diabetes Prediction: Converting Clinical Data to Image Data

Diabetes, one of the most common diseases worldwide, has become an increasingly global threat to humans in recent years. However, early detection of diabetes greatly inhibits the progression of the disease. This study proposes a new method based on deep learning for the early detection of diabetes. Like many other medical data, the PIMA dataset used in the study contains only numerical values. In this sense, the application of popular convolutional neural network (CNN) models to such data are limited. This study converts numerical data into images based on the feature importance to use the robust representation of CNN models in early diabetes diagnosis. Three different classification strategies are then applied to the resulting diabetes image data. In the first, diabetes images are fed into the ResNet18 and ResNet50 CNN models. In the second, deep features of the ResNet models are fused and classified with support vector machines (SVM). In the last approach, the selected fusion features are classified by SVM. The results demonstrate the robustness of diabetes images in the early diagnosis of diabetes.

Visual question answering model for fruit tree disease decision-making based on multimodal deep learning

Visual Question Answering (VQA) about diseases is an essential feature of intelligent management in smart agriculture. Currently, research on fruit tree diseases using deep learning mainly uses single-source data information, such as visible images or spectral data, yielding classification and identification results that cannot be directly used in practical agricultural decision-making. In this study, a VQA model for fruit tree diseases based on multimodal feature fusion was designed. Fusing images and Q&A knowledge of disease management, the model obtains the decision-making answer by querying questions about fruit tree disease images to find relevant disease image regions. The main contributions of this study were as follows: (1) a multimodal bilinear factorized pooling model using Tucker decomposition was proposed to fuse the image features with question features: (2) a deep modular co-attention architecture was explored to simultaneously learn the image and question attention to obtain richer graphical features and interactivity. The experiments showed that the proposed unified model combining the bilinear model and co-attentive learning in a new network architecture obtained 86.36% accuracy in decision-making under the condition of limited data (8,450 images and 4,560k Q&A pairs of data), outperforming existing multimodal methods. The data augmentation is adopted on the training set to avoid overfitting. Ten runs of 10-fold cross-validation are used to report the unbiased performance. The proposed multimodal fusion model achieved friendly interaction and fine-grained identification and decision-making performance. Thus, the model can be widely deployed in intelligent agriculture.

CAD-ALZ: A Blockwise Fine-Tuning Strategy on Convolutional Model and Random Forest Classifier for Recognition of Multistage Alzheimer's Disease

Mental deterioration or Alzheimer's (ALZ) disease is progressive and causes both physical and mental dependency. There is a need for a computer-aided diagnosis (CAD) system that can help doctors make an immediate decision. (1) Background: Currently, CAD systems are developed based on hand-crafted features, machine learning (ML), and deep learning (DL) techniques. Those CAD systems frequently require domain-expert knowledge and massive datasets to extract deep features or model training, which causes problems with class imbalance and overfitting. Additionally, there are still manual approaches used by radiologists due to the lack of dataset availability and to train the model with cost-effective computation. Existing works rely on performance improvement by neglecting the problems of the limited dataset, high computational complexity, and unavailability of lightweight and efficient feature descriptors. (2) Methods: To address these issues, a new approach, CAD-ALZ, is developed by extracting deep features through a ConvMixer layer with a blockwise fine-tuning strategy on a very small original dataset. At first, we apply the data augmentation method to images to increase the size of datasets. In this study, a blockwise fine-tuning strategy is employed on the ConvMixer model to detect robust features. Afterwards, a random forest (RF) is used to classify ALZ disease stages. (3) Results: The proposed CAD-ALZ model obtained significant results by using six evaluation metrics such as the F1-score, Kappa, accuracy, precision, sensitivity, and specificity. The CAD-ALZ model performed with a sensitivity of 99.69% and an F1-score of 99.61%. (4) Conclusions: The suggested CAD-ALZ approach is a potential technique for clinical use and computational efficiency compared to state-of-the-art approaches. The CAD-ALZ model code is freely available on GitHub for the scientific community.

Multimodal attention-based deep learning for Alzheimer's disease diagnosis

OBJECTIVE

Alzheimer's disease (AD) is the most common neurodegenerative disorder with one of the most complex pathogeneses, making effective and clinically actionable decision support difficult. The objective of this study was to develop a novel multimodal deep learning framework to aid medical professionals in AD diagnosis.

MATERIALS AND METHODS

We present a Multimodal Alzheimer's Disease Diagnosis framework (MADDi) to accurately detect the presence of AD and mild cognitive impairment (MCI) from imaging, genetic, and clinical data. MADDi is novel in that we use cross-modal attention, which captures interactions between modalities-a method not previously explored in this domain. We perform multi-class classification, a challenging task considering the strong similarities between MCI and AD. We compare with previous state-of-the-art models, evaluate the importance of attention, and examine the contribution of each modality to the model's performance.

RESULTS

MADDi classifies MCI, AD, and controls with 96.88% accuracy on a held-out test set. When examining the contribution of different attention schemes, we found that the combination of cross-modal attention with self-attention performed the best, and no attention layers in the model performed the worst, with a 7.9% difference in F1-scores.

DISCUSSION

Our experiments underlined the importance of structured clinical data to help machine learning models contextualize and interpret the remaining modalities. Extensive ablation studies showed that any multimodal mixture of input features without access to structured clinical information suffered marked performance losses.

CONCLUSION

This study demonstrates the merit of combining multiple input modalities via cross-modal attention to deliver highly accurate AD diagnostic decision support.

Multimodal machine learning in precision health: A scoping review

Machine learning is frequently being leveraged to tackle problems in the health sector including utilization for clinical decision-support. Its use has historically been focused on single modal data. Attempts to improve prediction and mimic the multimodal nature of clinical expert decision-making has been met in the biomedical field of machine learning by fusing disparate data. This review was conducted to summarize the current studies in this field and identify topics ripe for future research. We conducted this review in accordance with the PRISMA extension for Scoping Reviews to characterize multi-modal data fusion in health. Search strings were established and used in databases: PubMed, Google Scholar, and IEEEXplore from 2011 to 2021. A final set of 128 articles were included in the analysis. The most common health areas utilizing multi-modal methods were neurology and oncology. Early fusion was the most common data merging strategy. Notably, there was an improvement in predictive performance when using data fusion. Lacking from the papers were clear clinical deployment strategies, FDA-approval, and analysis of how using multimodal approaches from diverse sub-populations may improve biases and healthcare disparities. These findings provide a summary on multimodal data fusion as applied to health diagnosis/prognosis problems. Few papers compared the outputs of a multimodal approach with a unimodal prediction. However, those that did achieved an average increase of 6.4% in predictive accuracy. Multi-modal machine learning, while more robust in its estimations over unimodal methods, has drawbacks in its scalability and the time-consuming nature of information concatenation.

Automatic classification of brain magnetic resonance images with hypercolumn deep features and machine learning

Brain tumours are life-threatening and their early detection is very important in a patient's life. At the present time, magnetic resonance imaging is one of the methods used for detecting brain tumours. Expert decision support systems serve specialist physicians to make more accurate diagnoses by minimizing the errors arising from their subjective opinions in real clinical settings. The model proposed in this study detects important keypoints and then extracts hypercolumn deep features of these keypoints from some convolutional layers of VGG16. Finally, Random Forest and Logistic Regression classifiers are fed with a set of these features. Random Forest classifier offered the best performance with 94.51% accuracy, 91.61% sensitivity, 8.39% false-negative rate, 97.42% specificity, and 97.29% precision using fivefold cross-validation in this study. Consequently, it is thought that the proposed model could contribute to field experts by integrating it into computer-aided brain magnetic resonance imaging diagnosis systems.

A3C-TL-GTO: Alzheimer Automatic Accurate Classification Using Transfer Learning and Artificial Gorilla Troops Optimizer

Alzheimer's disease (AD) is a chronic disease that affects the elderly. There are many different types of dementia, but Alzheimer's disease is one of the leading causes of death. AD is a chronic brain disorder that leads to problems with language, disorientation, mood swings, bodily functions, memory loss, cognitive decline, mood or personality changes, and ultimately death due to dementia. Unfortunately, no cure has yet been developed for it, and it has no known causes. Clinically, imaging tools can aid in the diagnosis, and deep learning has recently emerged as an important component of these tools. Deep learning requires little or no image preprocessing and can infer an optimal data representation from raw images without prior feature selection. As a result, they produce a more objective and less biased process. The performance of a convolutional neural network (CNN) is primarily affected by the hyperparameters chosen and the dataset used. A deep learning model for classifying Alzheimer's patients has been developed using transfer learning and optimized by Gorilla Troops for early diagnosis. This study proposes the A3C-TL-GTO framework for MRI image classification and AD detection. The A3C-TL-GTO is an empirical quantitative framework for accurate and automatic AD classification, developed and evaluated with the Alzheimer's Dataset (four classes of images) and the Alzheimer's Disease Neuroimaging Initiative (ADNI). The proposed framework reduces the bias and variability of preprocessing steps and hyperparameters optimization to the classifier model and dataset used. Our strategy, evaluated on MRIs, is easily adaptable to other imaging methods. According to our findings, the proposed framework was an excellent instrument for this task, with a significant potential advantage for patient care. The ADNI dataset, an online dataset on Alzheimer's disease, was used to obtain magnetic resonance imaging (MR) brain images. The experimental results demonstrate that the proposed framework achieves 96.65% accuracy for the Alzheimer's Dataset and 96.25% accuracy for the ADNI dataset. Moreover, a better performance in terms of accuracy is demonstrated over other state-of-the-art approaches.

Sentiment Analysis of Users’ Reactions on Social Media During the Pandemic

During the outbreak of the COVID-19 pandemic, social networks became the preeminent medium for communication, social discussion, and entertainment. Social network users are regularly expressing their opinions about the impacts of the coronavirus pandemic. Therefore, social networks serve as a reliable source for studying the topics, emotions, and attitudes of users that have been discussed during the pandemic. In this paper, we investigate the reactions and attitudes of people towards topics raised on social media platforms. We collected data of two large-scale COVID-19 datasets from Twitter and Instagram for six and three months, respectively. This paper analyzes the reaction of social network users in terms of different aspects including sentiment analysis, topic detection, emotions, and the geo-temporal characteristics of our dataset. We show that the dominant sentiment reactions on social media are neutral, while the most discussed topics by social network users are about health issues. This paper examines the countries that attracted a higher number of posts and reactions from people, as well as the distribution of health-related topics discussed in the most mentioned countries. We shed light on the temporal shift of topics over countries. Our results show that posts from the top-mentioned countries influence and attract more reactions worldwide than posts from other parts of the world.

Enhanced Fuzzy Elephant Herding Optimization-Based OTSU Segmentation and Deep Learning for Alzheimer’s Disease Diagnosis

Several neurological illnesses and diseased sites have been studied, along with the anatomical framework of the brain, using structural MRI (sMRI). It is critical to diagnose Alzheimer’s disease (AD) patients in a timely manner to implement preventative treatments. The segmentation of brain anatomy and categorization of AD have received increased attention since they can deliver good findings spanning a vast range of information. The first research gap considered in this work is the real-time efficiency of OTSU segmentation, which is not high, despite its simplicity and good accuracy. A second issue is that feature extraction could be automated by implementing deep learning techniques. To improve picture segmentation’s real-timeliness, enhanced fuzzy elephant herding optimization (EFEHO) was used for OTSU segmentation, and named EFEHO-OTSU. The main contribution of this work is twofold. One is utilizing EFEHO in the recommended technique to seek the optimal segmentation threshold for the OTSU method. Second, dual attention multi-instance deep learning network (DA-MIDL) is recommended for the timely diagnosis of AD and its prodromal phase, mild cognitive impairment (MCI). Tests show that this technique converges faster and takes less time than the classic OTSU approach without reducing segmentation performance. This study develops a valuable tool for quick picture segmentation with good real-time efficiency. Compared to numerous conventional techniques, the suggested study attains improved categorization performance regarding accuracy and transferability.

Multi-view prediction of Alzheimer's disease progression with end-to-end integrated framework

Alzheimer's disease is a common neurodegenerative brain disease that affects the elderly population worldwide. Its early automatic detection is vital for early intervention and treatment. A common solution is to perform future cognitive score prediction based on the baseline brain structural magnetic resonance image (MRI), which can directly infer the potential severity of disease. Recently, several studies have modelled disease progression by predicting the future brain MRI that can provide visual information of brain changes over time. Nevertheless, no studies explore the intra correlation of these two solutions, and it is unknown whether the predicted MRI can assist the prediction of cognitive score. Here, instead of independent prediction, we aim to predict disease progression in multi-view, i.e., predicting subject-specific changes of cognitive score and MRI volume concurrently. To achieve this, we propose an end-to-end integrated framework, where a regression model and a generative adversarial network are integrated together and then jointly optimized. Three integration strategies are exploited to unify these two models. Moreover, considering that some brain regions, such as hippocampus and middle temporal gyrus, could change significantly during the disease progression, a region-of-interest (ROI) mask and a ROI loss are introduced into the integrated framework to leverage this anatomical prior knowledge. Experimental results on the longitudinal Alzheimer's Disease Neuroimaging Initiative dataset demonstrated that the integrated framework outperformed the independent regression model for cognitive score prediction. And its performance can be further improved with the ROI loss for both cognitive score and MRI prediction.

A hybrid deep learning paradigm for carotid plaque tissue characterization and its validation in multicenter cohorts using a supercomputer framework

BACKGROUND

Early and automated detection of carotid plaques prevents strokes, which are the second leading cause of death worldwide according to the World Health Organization. Artificial intelligence (AI) offers automated solutions for plaque tissue characterization. Recently, solo deep learning (SDL) models have been used, but they do not take advantage of the tandem connectivity offered by AI's hybrid nature. Therefore, this study explores the use of hybrid deep learning (HDL) models in a multicenter framework, making this study the first of its kind.

METHODS

We hypothesize that HDL techniques perform better than SDL and transfer learning (TL) techniques. We propose two kinds of HDL frameworks: (i) the fusion of two SDLs (Inception with ResNet) or (ii) 10 other kinds of tandem models that fuse SDL with ML. The system Atheromatic™ 2.0_HDL (AtheroPoint, CA, USA) was designed on an augmentation framework and three kinds of loss functions (cross-entropy, hinge, and mean-square-error) during training to determine the best optimization paradigm. These 11 combined HDL models were then benchmarked against one SDL model and five types of TL models; thus, this study considers a total of 17 AI models.

RESULTS

Among the 17 AI models, the best performing HDL system was that comprising CNN and decision tree (DT), as its accuracy and area-under-the-curve were 99.78 ± 1.05% and 0.99 (p<0.0001), respectively. These values are 6.4% and 3.2% better than those recorded for the SDL and TL models, respectively. We validated the performance of the HDL models with diagnostics odds ratio (DOR) and Cohen and Kappa statistics; here, HDL outperformed DL and TL by 23% and 7%, respectively. The online system ran in <2 s.

CONCLUSION

HDL is a fast, reliable, and effective tool for characterizing the carotid plaque for early stroke risk stratification.

The impact of recency and adequacy of historical information on sepsis predictions using machine learning

Sepsis is a major public and global health concern. Every hour of delay in detecting sepsis significantly increases the risk of death, highlighting the importance of accurately predicting sepsis in a timely manner. A growing body of literature has examined developing new or improving the existing machine learning (ML) approaches for timely and accurate predictions of sepsis. This study contributes to this literature by providing clear insights regarding the role of the recency and adequacy of historical information in predicting sepsis using ML. To this end, we implemented a deep learning model using a bidirectional long short-term memory (BiLSTM) algorithm and compared it with six other ML algorithms based on numerous combinations of the prediction horizons (to capture information recency) and observation windows (to capture information adequacy) using different measures of predictive performance. Our results indicated that the BiLSTM algorithm outperforms all other ML algorithms and provides a great separability of the predicted risk of sepsis among septic versus non-septic patients. Moreover, decreasing the prediction horizon (in favor of information recency) always boosts the predictive performance; however, the impact of expanding the observation window (in favor of information adequacy) depends on the prediction horizon and the purpose of prediction. More specifically, when the prediction is responsive to the positive label (i.e., Sepsis), increasing historical data improves the predictive performance when the prediction horizon is short-moderate.

Automatic Detection of Covid-19 with Bidirectional LSTM Network Using Deep Features Extracted from Chest X-ray Images

Coronavirus disease, which comes up in China at the end of 2019 and showed different symptoms in people infected, affected millions of people. Computer-aided expert systems are needed due to the inadequacy of the reverse transcription-polymerase chain reaction kit, which is widely used in the diagnosis of this disease. Undoubtedly, expert systems that provide effective solutions to many problems will be very useful in the detection of Covid-19 disease, especially when unskilled personnel and financial deficiencies in underdeveloped countries are taken into consideration. In the literature, there are numerous machine learning approaches built with different classifiers in the detection of this disease. This paper proposes an approach based on deep learning which detects Covid-19 and no-finding cases using chest X-ray images. Here, the classification performance of the Bi-LSTM network on the deep features was compared with the Deep Neural Network within the frame of the fivefold cross-validation technique. Accuracy, sensitivity, specificity and precision metrics were used to evaluate the classification performance of the trained models. Bi-LSTM network presented better performance compare to DNN with 97.6% value of high accuracy despite the few numbers of Covid-19 images in the dataset. In addition, it is understood that concatenated deep features more meaningful than deep features obtained with pre-trained networks by one by, as well. Consequently, it is thought that the proposed study based on the Bi-LSTM network and concatenated deep features will be noteworthy in the design of highly sensitive automated Covid-19 monitoring systems.

ADVIAN: Alzheimer's Disease VGG-Inspired Attention Network Based on Convolutional Block Attention Module and Multiple Way Data Augmentation

Aim: Alzheimer's disease is a neurodegenerative disease that causes 60-70% of all cases of dementia. This study is to provide a novel method that can identify AD more accurately. Methods: We first propose a VGG-inspired network (VIN) as the backbone network and investigate the use of attention mechanisms. We proposed an Alzheimer's Disease VGG-Inspired Attention Network (ADVIAN), where we integrate convolutional block attention modules on a VIN backbone. Also, 18-way data augmentation is proposed to avoid overfitting. Ten runs of 10-fold cross-validation are carried out to report the unbiased performance. Results: The sensitivity and specificity reach 97.65 ± 1.36 and 97.86 ± 1.55, respectively. Its precision and accuracy are 97.87 ± 1.53 and 97.76 ± 1.13, respectively. The F1 score, MCC, and FMI are obtained as 97.75 ± 1.13, 95.53 ± 2.27, and 97.76 ± 1.13, respectively. The AUC is 0.9852. Conclusion: The proposed ADVIAN gives better results than 11 state-of-the-art methods. Besides, experimental results demonstrate the effectiveness of 18-way data augmentation.

Analysis of Features of Alzheimer's Disease: Detection of Early Stage from Functional Brain Changes in Magnetic Resonance Images Using a Finetuned ResNet18 Network

One of the first signs of Alzheimer's disease (AD) is mild cognitive impairment (MCI), in which there are small variants of brain changes among the intermediate stages. Although there has been an increase in research into the diagnosis of AD in its early levels of developments lately, brain changes, and their complexity for functional magnetic resonance imaging (fMRI), makes early detection of AD difficult. This paper proposes a deep learning-based method that can predict MCI, early MCI (EMCI), late MCI (LMCI), and AD. The Alzheimer's Disease Neuroimaging Initiative (ADNI) fMRI dataset consisting of 138 subjects was used for evaluation. The finetuned ResNet18 network achieved a classification accuracy of 99.99%, 99.95%, and 99.95% on EMCI vs. AD, LMCI vs. AD, and MCI vs. EMCI classification scenarios, respectively. The proposed model performed better than other known models in terms of accuracy, sensitivity, and specificity.

Graph Models of Pathology Spread in Alzheimer's Disease: An Alternative to Conventional Graph Theoretic Analysis

Background: Graph theory and connectomics are new techniques for uncovering disease-induced changes in the brain's structural network. Most prior studied have focused on network statistics as biomarkers of disease. However, an emerging body of work involves exploring how the network serves as a conduit for the propagation of disease factors in the brain and has successfully mapped the functional and pathological consequences of disease propagation. In Alzheimer's disease (AD), progressive deposition of misfolded proteins amyloid and tau is well-known to follow fiber projections, under a "prion-like" trans-neuronal transmission mechanism, through which misfolded proteins cascade along neuronal pathways, giving rise to network spread. Methods: In this review, we survey the state of the art in mathematical modeling of connectome-mediated pathology spread in AD. Then we address several open questions that are amenable to mathematically precise parsimonious modeling of pathophysiological processes, extrapolated to the whole brain. We specifically identify current formal models of how misfolded proteins are produced, aggregate, and disseminate in brain circuits, and attempt to understand how this process leads to stereotyped progression in Alzheimer's and other related diseases. Conclusion: This review serves to unify current efforts in modeling of AD progression that together have the potential to explain observed phenomena and serve as a test-bed for future hypothesis generation and testing in silico. Impact statement Graph theory is a powerful new approach that is transforming the study of brain processes. There do not exist many focused reviews of the subfield of graph modeling of how Alzheimer's and other dementias propagate within the brain network, and how these processes can be mapped mathematically. By providing timely and topical review of this subfield, we fill a critical gap in the community and present a unified view that can serve as an in silico test-bed for future hypothesis generation and testing. We also point to several open and unaddressed questions and controversies that future practitioners can tackle.