Global Transformer and Dual Local Attention Network via Deep-Shallow Hierarchical Feature Fusion for Retinal Vessel Segmentation

Clinically, retinal vessel segmentation is a significant step in the diagnosis of fundus diseases. However, recent methods generally neglect the difference of semantic information between deep and shallow features, which fail to capture the global and local characterizations in fundus images simultaneously, resulting in the limited segmentation performance for fine vessels. In this article, a global transformer (GT) and dual local attention (DLA) network via deep-shallow hierarchical feature fusion (GT-DLA-dsHFF) are investigated to solve the above limitations. First, the GT is developed to integrate the global information in the retinal image, which effectively captures the long-distance dependence between pixels, alleviating the discontinuity of blood vessels in the segmentation results. Second, DLA, which is constructed using dilated convolutions with varied dilation rates, unsupervised edge detection, and squeeze-excitation block, is proposed to extract local vessel information, consolidating the edge details in the segmentation result. Finally, a novel deep-shallow hierarchical feature fusion (dsHFF) algorithm is studied to fuse the features in different scales in the deep learning framework, respectively, which can mitigate the attenuation of valid information in the process of feature fusion. We verified the GT-DLA-dsHFF on four typical fundus image datasets. The experimental results demonstrate our GT-DLA-dsHFF achieves superior performance against the current methods and detailed discussions verify the efficacy of the proposed three modules. Segmentation results of diseased images show the robustness of our proposed GT-DLA-dsHFF. Implementation codes will be available on https://github.com/YangLibuaa/GT-DLA-dsHFF.

Artificial intelligence for diagnostic and prognostic neuroimaging in dementia: A systematic review

INTRODUCTION

Artificial intelligence (AI) and neuroimaging offer new opportunities for diagnosis and prognosis of dementia.

METHODS

We systematically reviewed studies reporting AI for neuroimaging in diagnosis and/or prognosis of cognitive neurodegenerative diseases.

RESULTS

A total of 255 studies were identified. Most studies relied on the Alzheimer's Disease Neuroimaging Initiative dataset. Algorithmic classifiers were the most commonly used AI method (48%) and discriminative models performed best for differentiating Alzheimer's disease from controls. The accuracy of algorithms varied with the patient cohort, imaging modalities, and stratifiers used. Few studies performed validation in an independent cohort.

DISCUSSION

The literature has several methodological limitations including lack of sufficient algorithm development descriptions and standard definitions. We make recommendations to improve model validation including addressing key clinical questions, providing sufficient description of AI methods and validating findings in independent datasets. Collaborative approaches between experts in AI and medicine will help achieve the promising potential of AI tools in practice.

HIGHLIGHTS

There has been a rapid expansion in the use of machine learning for diagnosis and prognosis in neurodegenerative disease Most studies (71%) relied on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset with no other individual dataset used more than five times There has been a recent rise in the use of more complex discriminative models (e.g., neural networks) that performed better than other classifiers for classification of AD vs healthy controls We make recommendations to address methodological considerations, addressing key clinical questions, and validation We also make recommendations for the field more broadly to standardize outcome measures, address gaps in the literature, and monitor sources of bias.

Correlation between cerebral hemodynamic functional near-infrared spectroscopy and positron emission tomography for assessing mild cognitive impairment and Alzheimer's disease: An exploratory study

This study was performed to investigate the usefulness of functional near-infrared spectroscopy (fNIRS) by conducting a comparative analysis of hemodynamic activation detected by fNIRS and positron emission tomography (PET) and magnetic resonance imaging (MRI) in patients with mild cognitive impairment (MCI) and Alzheimer's disease (AD). Participants were divided into four groups: the subjective memory impairment (SMI), amnestic MCI (aMCI), non-amnestic MCI (naMCI), and AD groups. We recorded the hemodynamic response during the semantic verbal fluency task (SVFT) using a commercial wireless continuous-wave NIRS system. The correlation between the parameters of the neuroimaging assessments among the groups was analyzed. Region of interest-based comparisons showed that the four groups had significantly different hemodynamic responses during SVFT in the bilateral dorsolateral prefrontal cortex (DLPFC). The linear mixed effect model result indicates that the mean ΔHbO2 from the bilateral DLPFC regions showed a significant positive correlation to the overall FDG-PET after controlling for age and group differences in the fNIRS signals. Amyloid PET signals tended to better differentiate the AD group from other groups, and fNIRS signals tended to better differentiate the SMI group from other groups. In addition, a comparison between the group pairs revealed a mirrored pattern between the hippocampal volume and hemodynamic response in the DLPFC. The hemodynamic response detected by fNIRS showed a significant correlation with metabolic and anatomical changes associated with disease progression. Therefore, fNIRS may be considered as a screening tool to predict the hemodynamic and metabolic statuses of the brain in patients with MCI and AD.

Autism spectrum disorder diagnosis based on deep unrolling-based spatial constraint representation

Accurate diagnosis of autism spectrum disorder (ASD) is crucial for effective treatment and prognosis. Functional brain networks (FBNs) constructed from functional magnetic resonance imaging (fMRI) have become a popular tool for ASD diagnosis. However, existing model-driven approaches used to construct FBNs lack the ability to capture potential non-linear relationships between data and labels. Moreover, most existing studies treat the FBNs construction and disease classification as separate steps, leading to large inter-subject variability in the estimated FBNs and reducing the statistical power of subsequent group comparison. To address these limitations, we propose a new approach to FBNs construction called the deep unrolling-based spatial constraint representation (DUSCR) model and integrate it with a convolutional classifier to create an end-to-end framework for ASD recognition. Specifically, the model spatial constraint representation (SCR) is solved using a proximal gradient descent algorithm, and we unroll it into deep networks using the deep unrolling algorithm. Classification is then performed using a convolutional prototype learning model. We evaluated the effectiveness of the proposed method on the ABIDE I dataset and observed a significant improvement in model performance and classification accuracy. The resting state fMRI images are preprocessed into time series data and 3D coordinates of each region of interest. The data are fed into the DUSCR model, a model for building functional brain networks using deep learning instead of traditional models, that we propose, and then the outputs are fed into the convolutional classifier with prototype learning to determine whether the patient has ASD disease.

Learning to Fuse Multiple Brain Functional Networks for Automated Autism Identification

Functional connectivity network (FCN) has become a popular tool to identify potential biomarkers for brain dysfunction, such as autism spectrum disorder (ASD). Due to its importance, researchers have proposed many methods to estimate FCNs from resting-state functional MRI (rs-fMRI) data. However, the existing FCN estimation methods usually only capture a single relationship between brain regions of interest (ROIs), e.g., linear correlation, nonlinear correlation, or higher-order correlation, thus failing to model the complex interaction among ROIs in the brain. Additionally, such traditional methods estimate FCNs in an unsupervised way, and the estimation process is independent of the downstream tasks, which makes it difficult to guarantee the optimal performance for ASD identification. To address these issues, in this paper, we propose a multi-FCN fusion framework for rs-fMRI-based ASD classification. Specifically, for each subject, we first estimate multiple FCNs using different methods to encode rich interactions among ROIs from different perspectives. Then, we use the label information (ASD vs. healthy control (HC)) to learn a set of fusion weights for measuring the importance/discrimination of those estimated FCNs. Finally, we apply the adaptively weighted fused FCN on the ABIDE dataset to identify subjects with ASD from HCs. The proposed FCN fusion framework is straightforward to implement and can significantly improve diagnostic accuracy compared to traditional and state-of-the-art methods.

BolT: Fused window transformers for fMRI time series analysis

Deep-learning models have enabled performance leaps in analysis of high-dimensional functional MRI (fMRI) data. Yet, many previous methods are suboptimally sensitive for contextual representations across diverse time scales. Here, we present BolT, a blood-oxygen-level-dependent transformer model, for analyzing multi-variate fMRI time series. BolT leverages a cascade of transformer encoders equipped with a novel fused window attention mechanism. Encoding is performed on temporally-overlapped windows within the time series to capture local representations. To integrate information temporally, cross-window attention is computed between base tokens in each window and fringe tokens from neighboring windows. To gradually transition from local to global representations, the extent of window overlap and thereby number of fringe tokens are progressively increased across the cascade. Finally, a novel cross-window regularization is employed to align high-level classification features across the time series. Comprehensive experiments on large-scale public datasets demonstrate the superior performance of BolT against state-of-the-art methods. Furthermore, explanatory analyses to identify landmark time points and regions that contribute most significantly to model decisions corroborate prominent neuroscientific findings in the literature.

MCFSA-Net: A multi-scale channel fusion and spatial activation network for retinal vessel segmentation

As the only vascular tissue that can be directly viewed in vivo, retinal vessels are medically important in assisting the diagnosis of ocular and cardiovascular diseases. They generally appear as different morphologies and uneven thickness in fundus images. Therefore, the single-scale segmentation method may fail to capture abundant morphological features, suffering from the deterioration in vessel segmentation, especially for tiny vessels. To alleviate this issue, we propose a multi-scale channel fusion and spatial activation network (MCFSA-Net) for retinal vessel segmentation with emphasis on tiny ones. Specifically, the Hybrid Convolution-DropBlock (HC-Drop) is first used to extract deep features of vessels and construct multi-scale feature maps by progressive down-sampling. Then, the Channel Cooperative Attention Fusion (CCAF) module is designed to handle different morphological vessels in a multi-scale manner. Finally, the Global Spatial Activation (GSA) module is introduced to aggregate global feature information for improving the attention on tiny vessels in the spatial domain and realizing effective segmentation for them. Experiments are carried out on three datasets including DRIVE, CHASE_DB1, and STARE. Our retinal vessel segmentation method achieves Accuracy of 96.95%, 97.57%, and 97.83%, and F1 score of 82.67%, 81.82%, and 82.95% in the above datasets, respectively. Qualitative and quantitative analysis show that the proposed method outperforms current advanced vessel segmentation methods, especially for tiny vessels.

Dynamic multi-site graph convolutional network for autism spectrum disorder identification

Multi-site learning has attracted increasing interests in autism spectrum disorder (ASD) identification tasks by its efficacy on capturing data heterogeneity of neuroimaging taken from different medical sites. However, existing multi-site graph convolutional network (MSGCN) often ignores the correlations between different sites, and may obtain suboptimal identification results. Moreover, current feature extraction methods characterizing temporal variations of functional magnetic resonance imaging (fMRI) signals require the time series to be of the same length and cannot be directly applied to multi-site fMRI datasets. To address these problems, we propose a dual graph based dynamic multi-site graph convolutional network (DG-DMSGCN) for multi-site ASD identification. First, a sliding-window dual-graph convolutional network (SW-DGCN) is introduced for feature extraction, simultaneously capturing temporal and spatial features of fMRI data with different series lengths. Then we aggregate the features extracted from multiple medical sites through a novel dynamic multi-site graph convolutional network (DMSGCN), which effectively considers the correlations between different sites and is beneficial to improve identification performance. We evaluate the proposed DG-DMSGCN on public ABIDE I dataset containing data from 17 medical sites. The promising results obtained by our framework outperforms the state-of-the-art methods with increase in identification accuracy, indicating that it has a potential clinical prospect for practical ASD diagnosis. Our codes are available on https://github.com/Junling-Du/DG-DMSGCN.

Multicenter and Multichannel Pooling GCN for Early AD Diagnosis Based on Dual-Modality Fused Brain Network

For significant memory concern (SMC) and mild cognitive impairment (MCI), their classification performance is limited by confounding features, diverse imaging protocols, and limited sample size. To address the above limitations, we introduce a dual-modality fused brain connectivity network combining resting-state functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI), and propose three mechanisms in the current graph convolutional network (GCN) to improve classifier performance. First, we introduce a DTI-strength penalty term for constructing functional connectivity networks. Stronger structural connectivity and bigger structural strength diversity between groups provide a higher opportunity for retaining connectivity information. Second, a multi-center attention graph with each node representing a subject is proposed to consider the influence of data source, gender, acquisition equipment, and disease status of those training samples in GCN. The attention mechanism captures their different impacts on edge weights. Third, we propose a multi-channel mechanism to improve filter performance, assigning different filters to features based on feature statistics. Applying those nodes with low-quality features to perform convolution would also deteriorate filter performance. Therefore, we further propose a pooling mechanism, which introduces the disease status information of those training samples to evaluate the quality of nodes. Finally, we obtain the final classification results by inputting the multi-center attention graph into the multi-channel pooling GCN. The proposed method is tested on three datasets (i.e., an ADNI 2 dataset, an ADNI 3 dataset, and an in-house dataset). Experimental results indicate that the proposed method is effective and superior to other related algorithms, with a mean classification accuracy of 93.05% in our binary classification tasks. Our code is available at: https://github.com/Xuegang-S.

A ViT-AMC Network With Adaptive Model Fusion and Multiobjective Optimization for Interpretable Laryngeal Tumor Grading From Histopathological Images

The tumor grading of laryngeal cancer pathological images needs to be accurate and interpretable. The deep learning model based on the attention mechanism-integrated convolution (AMC) block has good inductive bias capability but poor interpretability, whereas the deep learning model based on the vision transformer (ViT) block has good interpretability but weak inductive bias ability. Therefore, we propose an end-to-end ViT-AMC network (ViT-AMCNet) with adaptive model fusion and multiobjective optimization that integrates and fuses the ViT and AMC blocks. However, existing model fusion methods often have negative fusion: 1). There is no guarantee that the ViT and AMC blocks will simultaneously have good feature representation capability. 2). The difference in feature representations learning between the ViT and AMC blocks is not obvious, so there is much redundant information in the two feature representations. Accordingly, we first prove the feasibility of fusing the ViT and AMC blocks based on Hoeffding's inequality. Then, we propose a multiobjective optimization method to solve the problem that ViT and AMC blocks cannot simultaneously have good feature representation. Finally, an adaptive model fusion method integrating the metrics block and the fusion block is proposed to increase the differences between feature representations and improve the deredundancy capability. Our methods improve the fusion ability of ViT-AMCNet, and experimental results demonstrate that ViT-AMCNet significantly outperforms state-of-the-art methods. Importantly, the visualized interpretive maps are closer to the region of interest of concern by pathologists, and the generalization ability is also excellent. Our code is publicly available at https://github.com/Baron-Huang/ViT-AMCNet.

Exploring high-order correlations with deep-broad learning for autism spectrum disorder diagnosis

Recently, a lot of research has been conducted on diagnosing neurological disorders, such as autism spectrum disorder (ASD). Functional magnetic resonance imaging (fMRI) is the commonly used technique to assist in the diagnosis of ASD. In the past years, some conventional methods have been proposed to extract the low-order functional connectivity network features for ASD diagnosis, which ignore the complexity and global features of the brain network. Most deep learning-based methods generally have a large number of parameters that need to be adjusted during the learning process. To overcome the limitations mentioned above, we propose a novel deep-broad learning method for learning the higher-order brain functional connectivity network features to assist in ASD diagnosis. Specifically, we first construct the high-order functional connectivity network that describes global correlations of the brain regions based on hypergraph, and then we use the deep-broad learning method to extract the high-dimensional feature representations for brain networks sequentially. The evaluation of the proposed method is conducted on Autism Brain Imaging Data Exchange (ABIDE) dataset. The results show that our proposed method can achieve 71.8% accuracy on the multi-center dataset and 70.6% average accuracy on 17 single-center datasets, which are the best results compared with the state-of-the-art methods. Experimental results demonstrate that our method can describe the global features of the brain regions and get rich discriminative information for the classification task.

Inferring Effective Connectivity Networks From fMRI Time Series With a Temporal Entropy-Score

Inferring brain-effective connectivity networks from neuroimaging data has become a very hot topic in neuroinformatics and bioinformatics. In recent years, the search methods based on Bayesian network score have been greatly developed and become an emerging method for inferring effective connectivity. However, the previous score functions ignore the temporal information from functional magnetic resonance imaging (fMRI) series data and may not be able to determine all orientations in some cases. In this article, we propose a novel score function for inferring effective connectivity from fMRI data based on the conditional entropy and transfer entropy (TE) between brain regions. The new score employs the TE to capture the temporal information and can effectively infer connection directions between brain regions. Experimental results on both simulated and real-world data demonstrate the efficacy of our proposed score function.

A spatio-temporal decomposition framework for dynamic functional connectivity in the human brain

Much recent attention has been directed toward investigating the spatial and temporal organization of brain dynamics, but the rules which constrain the variation of spatio-temporal organization in functional connectivity under different brain states remain unclear. Here, we developed a novel computational approach based on tensor decomposition and regularization to represent dynamic functional connectivity as a linear combination of dynamic modules and time-varying weights. In this approach, dynamic modules represent co-activating functional connectivity patterns, and time-varying weights represent the temporal expression of dynamic modules. We applied this dynamic decomposition model (DDM) on a resting-state fMRI dataset and found that whole-brain dynamic functional connectivity can be decomposed as a linear combination of eight dynamic modules which we summarize as 'high order modules' and 'primary-high order modules', according to their spatial attributes and correspondence with existing intrinsic functional brain networks. By clustering the time-varying weights, we identified five brain states including three major states and two minor states. We found that state transitions mainly occurred between the three major states, and that temporal variation of dynamic modules may contribute to brain state transitions. We then conceptualized the variability of weights as the flexibility of the corresponding dynamic modules and found that different dynamic modules exhibit different amounts of flexibility and contribute to different cognitive measures. Finally, we applied DDM to a schizophrenia resting-state fMRI dataset and found that atypical flexibility of dynamic modules correlates with impaired cognitive flexibility in schizophrenia. Overall, this work provides a quantitative framework that characterizes temporal variation in the topology of dynamic functional connectivity.

Deep Spatio-Temporal Attention-based Recurrent Network from Dynamic Adaptive Functional Connectivity for MCI Identification

Most existing methods of constructing dynamic functional connectivity (dFC) network obtain the connectivity strength via the sliding window correlation (SWC) method, which estimates the connectivity strength at each time segment, rather than at each time point, and thus is difficult to produce accurate dFC network due to the influence of the window type and window width. Furthermore, the deep learning methods may not capture the discriminative spatio-temporal information that is closely related to disease, thus impacting the performance of (mild cognitive impairment) MCI identification. In this paper, a novel spatio-temporal attention-based bidirectional gated recurrent unit (STA-BiGRU) network is proposed to extract inherent spatio-temporal information from a dynamic adaptive functional connectivity (dAFC) network for MCI diagnosis. Specifically, we adopt a group lasso-based Kalman filter algorithm to obtain the dAFC network with more accurate connectivity strength at each time step. Then a spatial attention module with self-attention and a temporal attention module with multiple temporal attention vectors are incorporated into the BiGRU network to extract more discriminative disease-related spatio-temporal information. Finally, the spatio-temporal regularizations are employed to better guide the attention learning of STA-BiGRU network to enhance the robustness of the deep network. Experimental results show that the proposed framework achieves mean accuracies of 90.2%, 90.0%, and 81.5%, respectively, for three MCI classification tasks. This study provides a more effective deep spatio-temporal attention-based recurrent network and obtains good performance and interpretability of deep learning for psychiatry diagnosis research.

Dual Encoder-Based Dynamic-Channel Graph Convolutional Network With Edge Enhancement for Retinal Vessel Segmentation

Retinal vessel segmentation with deep learning technology is a crucial auxiliary method for clinicians to diagnose fundus diseases. However, the deep learning approaches inevitably lose the edge information, which contains spatial features of vessels while performing down-sampling, leading to the limited segmentation performance of fine blood vessels. Furthermore, the existing methods ignore the dynamic topological correlations among feature maps in the deep learning framework, resulting in the inefficient capture of the channel characterization. To address these limitations, we propose a novel dual encoder-based dynamic-channel graph convolutional network with edge enhancement (DE-DCGCN-EE) for retinal vessel segmentation. Specifically, we first design an edge detection-based dual encoder to preserve the edge of vessels in down-sampling. Secondly, we investigate a dynamic-channel graph convolutional network to map the image channels to the topological space and synthesize the features of each channel on the topological map, which solves the limitation of insufficient channel information utilization. Finally, we study an edge enhancement block, aiming to fuse the edge and spatial features in the dual encoder, which is beneficial to improve the accuracy of fine blood vessel segmentation. Competitive experimental results on five retinal image datasets validate the efficacy of the proposed DE-DCGCN-EE, which achieves more remarkable segmentation results against the other state-of-the-art methods, indicating its potential clinical application.

Parkinson's Disease Classification and Clinical Score Regression via United Embedding and Sparse Learning From Longitudinal Data

Parkinson's disease (PD) is known as an irreversible neurodegenerative disease that mainly affects the patient's motor system. Early classification and regression of PD are essential to slow down this degenerative process from its onset. In this article, a novel adaptive unsupervised feature selection approach is proposed by exploiting manifold learning from longitudinal multimodal data. Classification and clinical score prediction are performed jointly to facilitate early PD diagnosis. Specifically, the proposed approach performs united embedding and sparse regression, which can determine the similarity matrices and discriminative features adaptively. Meanwhile, we constrain the similarity matrix among subjects and exploit the l_2,p norm to conduct sparse adaptive control for obtaining the intrinsic information of the multimodal data structure. An effective iterative optimization algorithm is proposed to solve this problem. We perform abundant experiments on the Parkinson's Progression Markers Initiative (PPMI) data set to verify the validity of the proposed approach. The results show that our approach boosts the performance on the classification and clinical score regression of longitudinal data and surpasses the state-of-the-art approaches.

A Multi-Modal and Multi-Atlas Integrated Framework for Identification of Mild Cognitive Impairment

Background: Multi-modal neuroimaging with appropriate atlas is vital for effectively differentiating mild cognitive impairment (MCI) from healthy controls (HC). Methods: The resting-state functional magnetic resonance imaging (rs-fMRI) and structural MRI (sMRI) of 69 MCI patients and 61 HC subjects were collected. Then, the gray matter volumes obtained from the sMRI and Hurst exponent (HE) values calculated from rs-fMRI data in the Automated Anatomical Labeling (AAL-90), Brainnetome (BN-246), Harvard–Oxford (HOA-112) and AAL3-170 atlases were extracted, respectively. Next, these characteristics were selected with a minimal redundancy maximal relevance algorithm and a sequential feature collection method in single or multi-modalities, and only the optimal features were retained after this procedure. Lastly, the retained characteristics were served as the input features for the support vector machine (SVM)-based method to classify MCI patients, and the performance was estimated with a leave-one-out cross-validation (LOOCV). Results: Our proposed method obtained the best 92.00% accuracy, 94.92% specificity and 89.39% sensitivity with the sMRI in AAL-90 and the fMRI in HOA-112 atlas, which was much better than using the single-modal or single-atlas features. Conclusion: The results demonstrated that the multi-modal and multi-atlas integrated method could effectively recognize MCI patients, which could be extended into various neurological and neuropsychiatric diseases.

Diagnosis of Amnesic Mild Cognitive Impairment Using MGS-WBC and VGBN-LM Algorithms

Computer-aided diagnosis (CAD) has undergone rapid development with the advent of advanced neuroimaging and machine learning methods. Nevertheless, how to extract discriminative features from the limited and high-dimensional data is not ideal, especially for amnesic mild cognitive impairment (aMCI) data based on resting-state functional magnetic resonance imaging (rs-fMRI). Furthermore, a robust and reliable system for aMCI detection is conducive to timely detecting and screening subjects at a high risk of Alzheimer’s disease (AD). In this scenario, we first develop the mask generation strategy based on within-class and between-class criterion (MGS-WBC), which primarily aims at reducing data redundancy and excavating multiscale features of the brain. Concurrently, vector generation for brain networks based on Laplacian matrix (VGBN-LM) is presented to obtain the global features of the functional network. Finally, all multiscale features are fused to further improve the diagnostic performance of aMCI. Typical classifiers for small data learning, such as naive Bayesian (NB), linear discriminant analysis (LDA), logistic regression (LR), and support vector machines (SVMs), are adopted to evaluate the diagnostic performance of aMCI. This study helps to reveal discriminative neuroimaging features, and outperforms the state-of-the-art methods, providing new insights for the intelligent construction of CAD system of aMCI.

Feature aggregation graph convolutional network based on imaging genetic data for diagnosis and pathogeny identification of Alzheimer's disease

The roles of brain regions activities and gene expressions in the development of Alzheimer's disease (AD) remain unclear. Existing imaging genetic studies usually has the problem of inefficiency and inadequate fusion of data. This study proposes a novel deep learning method to efficiently capture the development pattern of AD. First, we model the interaction between brain regions and genes as node-to-node feature aggregation in a brain region-gene network. Second, we propose a feature aggregation graph convolutional network (FAGCN) to transmit and update the node feature. Compared with the trivial graph convolutional procedure, we replace the input from the adjacency matrix with a weight matrix based on correlation analysis and consider common neighbor similarity to discover broader associations of nodes. Finally, we use a full-gradient saliency graph mechanism to score and extract the pathogenetic brain regions and risk genes. According to the results, FAGCN achieved the best performance among both traditional and cutting-edge methods and extracted AD-related brain regions and genes, providing theoretical and methodological support for the research of related diseases.

Improved Multiple Vector Representations of Images and Robust Dictionary Learning

Each sparse representation classifier has different classification accuracy for different samples. It is difficult to achieve good performance with a single feature classification model. In order to balance the large-scale information and global features of images, a robust dictionary learning method based on image multi-vector representation is proposed in this paper. First, this proposed method generates a reasonable virtual image for the original image and obtains the multi-vector representation of all images. Second, the same dictionary learning algorithm is used for each vector representation to obtain multiple sets of image features. The proposed multi-vector representation can provide a good global understanding of the whole image contour and increase the content of dictionary learning. Last, the weighted fusion algorithm is used to classify the test samples. The introduction of influencing factors and the automatic adjustment of the weights of each classifier in the final decision results have a significant indigenous effect on better extracting image features. The study conducted experiments on the proposed algorithm on a number of widely used image databases. A large number of experimental results show that it effectively improves the accuracy of image classification. At the same time, to fully dig and exploit possible representation diversity might be a better way to lead to potential various appearances and high classification accuracy concerning the image.

Multi-Scale Graph Representation Learning for Autism Identification With Functional MRI

Resting-state functional MRI (rs-fMRI) has been widely used for the early diagnosis of autism spectrum disorder (ASD). With rs-fMRI, the functional connectivity networks (FCNs) are usually constructed for representing each subject, with each element representing the pairwise relationship between brain region-of-interests (ROIs). Previous studies often first extract handcrafted network features (such as node degree and clustering coefficient) from FCNs and then construct a prediction model for ASD diagnosis, which largely requires expert knowledge. Graph convolutional networks (GCNs) have recently been employed to jointly perform FCNs feature extraction and ASD identification in a data-driven manner. However, existing studies tend to focus on the single-scale topology of FCNs by using one single atlas for ROI partition, thus ignoring potential complementary topology information of FCNs at different spatial scales. In this paper, we develop a multi-scale graph representation learning (MGRL) framework for rs-fMRI based ASD diagnosis. The MGRL consists of three major components: (1) multi-scale FCNs construction using multiple brain atlases for ROI partition, (2) FCNs representation learning via multi-scale GCNs, and (3) multi-scale feature fusion and classification for ASD diagnosis. The proposed MGRL is evaluated on 184 subjects from the public Autism Brain Imaging Data Exchange (ABIDE) database with rs-fMRI scans. Experimental results suggest the efficacy of our MGRL in FCN feature extraction and ASD identification, compared with several state-of-the-art methods.

Virtual Adversarial Training-Based Deep Feature Aggregation Network From Dynamic Effective Connectivity for MCI Identification

Dynamic functional connectivity (dFC) network inferred from resting-state fMRI reveals macroscopic dynamic neural activity patterns for brain disease identification. However, dFC methods ignore the causal influence between the brain regions. Furthermore, due to the complex non-Euclidean structure of brain networks, advanced deep neural networks are difficult to be applied for learning high-dimensional representations from brain networks. In this paper, a group constrained Kalman filter (gKF) algorithm is proposed to construct dynamic effective connectivity (dEC), where the gKF provides a more comprehensive understanding of the directional interaction within the dynamic brain networks than the dFC methods. Then, a novel virtual adversarial training convolutional neural network (VAT-CNN) is employed to extract the local features of dEC. The VAT strategy improves the robustness of the model to adversarial perturbations, and therefore avoids the overfitting problem effectively. Finally, we propose the high-order connectivity weight-guided graph attention networks (cwGAT) to aggregate features of dEC. By injecting the weight information of high-order connectivity into the attention mechanism, the cwGAT provides more effective high-level feature representations than the conventional GAT. The high-level features generated from the cwGAT are applied for binary classification and multiclass classification tasks of mild cognitive impairment (MCI). Experimental results indicate that the proposed framework achieves the classification accuracy of 90.9%, 89.8%, and 82.7% for normal control (NC) vs. early MCI (EMCI), EMCI vs. late MCI (LMCI), and NC vs. EMCI vs. LMCI classification respectively, outperforming the state-of-the-art methods significantly.

Harris Hawks Optimization with Multi-Strategy Search and Application

The probability of the basic HHO algorithm in choosing different search methods is symmetric: about 0.5 in the interval from 0 to 1. The optimal solution from the previous iteration of the algorithm affects the current solution, the search for prey in a linear way led to a single search result, and the overall number of updates of the optimal position was low. These factors limit Harris Hawks optimization algorithm. For example, an ease of falling into a local optimum and the efficiency of convergence is low. Inspired by the prey hunting behavior of Harris’s hawk, a multi-strategy search Harris Hawks optimization algorithm is proposed, and the least squares support vector machine (LSSVM) optimized by the proposed algorithm was used to model the reactive power output of the synchronous condenser. Firstly, we select the best Gauss chaotic mapping method from seven commonly used chaotic mapping population initialization methods to improve the accuracy. Secondly, the optimal neighborhood perturbation mechanism is introduced to avoid premature maturity of the algorithm. Simultaneously, the adaptive weight and variable spiral search strategy are designed to simulate the prey hunting behavior of Harris hawk to improve the convergence speed of the improved algorithm and enhance the global search ability of the improved algorithm. A numerical experiment is tested with the classical 23 test functions and the CEC2017 test function set. The results show that the proposed algorithm outperforms the Harris Hawks optimization algorithm and other intelligent optimization algorithms in terms of convergence speed, solution accuracy and robustness, and the model of synchronous condenser reactive power output established by the improved algorithm optimized LSSVM has good accuracy and generalization ability.

MRI-based Alzheimer's disease prediction via distilling the knowledge in multi-modal data

Mild cognitive impairment (MCI) conversion prediction, i.e., identifying MCI patients of high risks converting to Alzheimer's disease (AD), is essential for preventing or slowing the progression of AD. Although previous studies have shown that the fusion of multi-modal data can effectively improve the prediction accuracy, their applications are largely restricted by the limited availability or high cost of multi-modal data. Building an effective prediction model using only magnetic resonance imaging (MRI) remains a challenging research topic. In this work, we propose a multi-modal multi-instance distillation scheme, which aims to distill the knowledge learned from multi-modal data to an MRI-based network for MCI conversion prediction. In contrast to existing distillation algorithms, the proposed multi-instance probabilities demonstrate a superior capability of representing the complicated atrophy distributions, and can guide the MRI-based network to better explore the input MRI. To our best knowledge, this is the first study that attempts to improve an MRI-based prediction model by leveraging extra supervision distilled from multi-modal information. Experiments demonstrate the advantage of our framework, suggesting its potentials in the data-limited clinical settings.

Estimating Effective Connectivity by Recurrent Generative Adversarial Networks

Estimating effective connectivity from functional magnetic resonance imaging (fMRI) time series data has become a very hot topic in neuroinformatics and brain informatics. However, it is hard for the current methods to accurately estimate the effective connectivity due to the high noise and small sample size of fMRI data. In this paper, we propose a novel framework for estimating effective connectivity based on recurrent generative adversarial networks, called EC-RGAN. The proposed framework employs the generator that consists of a set of effective connectivity generators based on recurrent neural networks to generate the fMRI time series of each brain region, and uses the discriminator to distinguish between the joint distributions of the real and generated fMRI time series. When the model is well-trained and generated fMRI data is similar to real fMRI data, EC-RGAN outputs the effective connectivity by means of the causal parameters of the effective connectivity generators. Experimental results on both simulated and real-world fMRI time series data demonstrate the efficacy of our proposed framework.

A dynamic graph convolutional neural network framework reveals new insights into connectome dysfunctions in ADHD

The pathological mechanism of attention deficit hyperactivity disorder (ADHD) is incompletely specified, which leads to difficulty in precise diagnosis. Functional magnetic resonance imaging (fMRI) has emerged as a common neuroimaging technique for studying the brain functional connectome. Most existing methods that have either ignored or simply utilized graph structure, do not fully leverage the potentially important topological information which may be useful in characterizing brain disorders. There is a crucial need for designing novel and efficient approaches which can capture such information. To this end, we propose a new dynamic graph convolutional network (dGCN), which is trained with sparse brain regional connections from dynamically calculated graph features. We also develop a novel convolutional readout layer to improve graph representation. Our extensive experimental analysis demonstrates significantly improved performance of dGCN for ADHD diagnosis compared with existing machine learning and deep learning methods. Visualizations of the salient regions of interest (ROIs) and connectivity based on informative features learned by our model show that the identified functional abnormalities mainly involve brain regions in temporal pole, gyrus rectus, and cerebellar gyri from temporal lobe, frontal lobe, and cerebellum, respectively. A positive correlation was further observed between the identified connectomic abnormalities and ADHD symptom severity. The proposed dGCN model shows great promise in providing a functional network-based precision diagnosis of ADHD and is also broadly applicable to brain connectome-based study of mental disorders.

A parallel attention-augmented bilinear network for early magnetic resonance imaging-based diagnosis of Alzheimer's disease

Structural magnetic resonance imaging (sMRI) can capture the spatial patterns of brain atrophy in Alzheimer's disease (AD) and incipient dementia. Recently, many sMRI‐based deep learning methods have been developed for AD diagnosis. Some of these methods utilize neural networks to extract high‐level representations on the basis of handcrafted features, while others attempt to learn useful features from brain regions proposed by a separate module. However, these methods require considerable manual engineering. Their stepwise training procedures would introduce cascading errors. Here, we propose the parallel attention‐augmented bilinear network, a novel deep learning framework for AD diagnosis. Based on a 3D convolutional neural network, the framework directly learns both global and local features from sMRI scans without any prior knowledge. The framework is lightweight and suitable for end‐to‐end training. We evaluate the framework on two public datasets (ADNI‐1 and ADNI‐2) containing 1,340 subjects. On both the AD classification and mild cognitive impairment conversion prediction tasks, our framework achieves competitive results. Furthermore, we generate heat maps that highlight discriminative areas for visual interpretation. Experiments demonstrate the effectiveness of the proposed framework when medical priors are unavailable or the computing resources are limited. The proposed framework is general for 3D medical image analysis with both efficiency and interpretability.

A Temporal-Spectral-Based Squeeze-and- Excitation Feature Fusion Network for Motor Imagery EEG Decoding

Motor imagery (MI) electroencephalography (EEG) decoding plays an important role in brain-computer interface (BCI), which enables motor-disabled patients to communicate with the outside world via external devices. Recent deep learning methods, which fail to fully explore both deep-temporal characterizations in EEGs itself and multi-spectral information in different rhythms, generally ignore the temporal or spectral dependencies in MI-EEG. Also, the lack of effective feature fusion probably leads to redundant or irrelative information and thus fails to achieve the most discriminative features, resulting in the limited MI-EEG decoding performance. To address these issues, in this paper, a MI-EEG decoding framework is proposed, which uses a novel temporal-spectral-based squeeze-and-excitation feature fusion network (TS-SEFFNet). First, the deep-temporal convolution block (DT-Conv block) implements convolutions in a cascade architecture, which extracts high-dimension temporal representations from raw EEG signals. Second, the multi-spectral convolution block (MS-Conv block) is then conducted in parallel using multi-level wavelet convolutions to capture discriminative spectral features from corresponding clinical subbands. Finally, the proposed squeeze-and-excitation feature fusion block (SE-Feature-Fusion block) maps the deep-temporal and multi-spectral features into comprehensive fused feature maps, which highlights channel-wise feature responses by constructing interdependencies among different domain features. Competitive experimental results on two public datasets demonstrate that our method is able to achieve promising decoding performance compared with the state-of-the-art methods.

Data-driven approaches to neuroimaging biomarkers for neurological and psychiatric disorders: emerging approaches and examples

PURPOSE OF REVIEW

The 'holy grail' of clinical applications of neuroimaging to neurological and psychiatric disorders via personalized biomarkers has remained mostly elusive, despite considerable effort. However, there are many reasons to continue to be hopeful, as the field has made remarkable advances over the past few years, fueled by a variety of converging technical and data developments.

RECENT FINDINGS

We discuss a number of advances that are accelerating the push for neuroimaging biomarkers including the advent of the 'neuroscience big data' era, biomarker data competitions, the development of more sophisticated algorithms including 'guided' data-driven approaches that facilitate automation of network-based analyses, dynamic connectivity, and deep learning. Another key advance includes multimodal data fusion approaches which can provide convergent and complementary evidence pointing to possible mechanisms as well as increase predictive accuracy.

SUMMARY

The search for clinically relevant neuroimaging biomarkers for neurological and psychiatric disorders is rapidly accelerating. Here, we highlight some of these aspects, provide recent examples from studies in our group, and link to other ongoing work in the field. It is critical that access and use of these advanced approaches becomes mainstream, this will help propel the community forward and facilitate the production of robust and replicable neuroimaging biomarkers.

Classification of COVID-19 by Compressed Chest CT Image through Deep Learning on a Large Patients Cohort

Abstract

Corona Virus Disease (COVID-19) has spread globally quickly, and has resulted in a large number of causalities and medical resources insufficiency in many countries. Reverse-transcriptase polymerase chain reaction (RT-PCR) testing is adopted as biopsy tool for confirmation of virus infection. However, its accuracy is as low as 60-70%, which is inefficient to uncover the infected. In comparison, the chest CT has been considered as the prior choice in diagnosis and monitoring progress of COVID-19 infection. Although the COVID-19 diagnostic systems based on artificial intelligence have been developed for assisting doctors in diagnosis, the small sample size and the excessive time consumption limit their applications. To this end, this paper proposed a diagnosis prototype system for COVID-19 infection testing. The proposed deep learning model is trained and is tested on 2267 CT sequences from 1357 patients clinically confirmed with COVID-19 and 1235 CT sequences from non-infected people. The main highlights of the prototype system are: (1) no data augmentation is needed to accurately discriminate the COVID-19 from normal controls with the specificity of 0.92 and sensitivity of 0.93; (2) the raw DICOM image is not necessary in testing. Highly compressed image like Jpeg can be used to allow a quick diagnosis; and (3) it discriminates the virus infection within 6 seconds and thus allows an online test with light cost. We also applied our model on 48 asymptomatic patients diagnosed with COVID-19. We found that: (1) the positive rate of RT-PCR assay is 63.5% (687/1082). (2) 45.8% (22/48) of the RT-PCR assay is negative for asymptomatic patients, yet the accuracy of CT scans is 95.8%. The online detection system is available: http://212.64.70.65/covid.

Graphic Abstract

Diagnosing Coronavirus Disease 2019 (COVID-19): Efficient Harris Hawks-Inspired Fuzzy K-Nearest Neighbor Prediction Methods

This study is devoted to proposing a useful intelligent prediction model to distinguish the severity of COVID-19, to provide a more fair and reasonable reference for assisting clinical diagnostic decision-making. Based on patients' necessary information, pre-existing diseases, symptoms, immune indexes, and complications, this article proposes a prediction model using the Harris hawks optimization (HHO) to optimize the Fuzzy K-nearest neighbor (FKNN), which is called HHO-FKNN. This model is utilized to distinguish the severity of COVID-19. In HHO-FKNN, the purpose of introducing HHO is to optimize the FKNN's optimal parameters and feature subsets simultaneously. Also, based on actual COVID-19 data, we conducted a comparative experiment between HHO-FKNN and several well-known machine learning algorithms, which result shows that not only the proposed HHO-FKNN can obtain better classification performance and higher stability on the four indexes but also screen out the key features that distinguish severe COVID-19 from mild COVID-19. Therefore, we can conclude that the proposed HHO-FKNN model is expected to become a useful tool for COVID-19 prediction.

BBPpred: Sequence-Based Prediction of Blood-Brain Barrier Peptides with Feature Representation Learning and Logistic Regression

Blood-brain barrier peptides (BBPs) have a large range of biomedical applications since they can cross the blood-brain barrier based on different mechanisms. As experimental methods for the identification of BBPs are laborious and expensive, computational approaches are necessary to be developed for predicting BBPs. In this work, we describe a computational method, BBPpred (blood-brain barrier peptides prediction), that can efficiently identify BBPs using logistic regression. We investigate a wide variety of features from amino acid sequence information, and then a feature learning method is adopted to represent the informative features. To improve the prediction performance, seven informative features are selected for classification by eliminating redundant and irrelevant features. In addition, we specifically create two benchmark data sets (training and independent test), which contain a total of 119 BBPs from public databases and the literature. On the training data set, BBPpred shows promising performances with an AUC score of 0.8764 and an AUPR score of 0.8757 using the 10-fold cross-validation. We also test our new method on the independent test data set and obtain a favorable performance. We envision that BBPpred will be a useful tool for identifying, annotating, and characterizing BBPs. BBPpred is freely available at http://BBPpred.xialab.info.

FedDPGAN: Federated Differentially Private Generative Adversarial Networks Framework for the Detection of COVID-19 Pneumonia

Existing deep learning technologies generally learn the features of chest X-ray data generated by Generative Adversarial Networks (GAN) to diagnose COVID-19 pneumonia. However, the above methods have a critical challenge: data privacy. GAN will leak the semantic information of the training data which can be used to reconstruct the training samples by attackers, thereby this method will leak the privacy of the patient. Furthermore, for this reason, that is the limitation of the training data sample, different hospitals jointly train the model through data sharing, which will also cause privacy leakage. To solve this problem, we adopt the Federated Learning (FL) framework, a new technique being used to protect data privacy. Under the FL framework and Differentially Private thinking, we propose a Federated Differentially Private Generative Adversarial Network (FedDPGAN) to detect COVID-19 pneumonia for sustainable smart cities. Specifically, we use DP-GAN to privately generate diverse patient data in which differential privacy technology is introduced to make sure the privacy protection of the semantic information of the training dataset. Furthermore, we leverage FL to allow hospitals to collaboratively train COVID-19 models without sharing the original data. Under Independent and Identically Distributed (IID) and non-IID settings, the evaluation of the proposed model is on three types of chest X-ray (CXR)images dataset (COVID-19, normal, and normal pneumonia). A large number of truthful reports make the verification of our model can effectively diagnose COVID-19 without compromising privacy.

Graph convolution network with similarity awareness and adaptive calibration for disease-induced deterioration prediction

Graph convolution networks (GCN) have been successfully applied in disease prediction tasks as they capture interactions (i.e., edges and edge weights on the graph) between individual elements. The interactions in existing works are constructed by fusing similarity between imaging information and distance between non-imaging information, whereas disregarding the disease status of those individuals in the training set. Besides, the similarity is being evaluated by computing the correlation distance between feature vectors, which limits prediction performance, especially for predicting significant memory concern (SMC) and mild cognitive impairment (MCI). In this paper, we propose three mechanisms to improve GCN, namely similarity-aware adaptive calibrated GCN (SAC-GCN), for predicting SMC and MCI. First, we design a similarity-aware graph using different receptive fields to consider disease status. The labelled subjects on the graph are only connected with those labelled subjects with the same status. Second, we propose an adaptive mechanism to evaluate similarity. Specifically, we construct initial GCN with evaluating similarity by using traditional correlation distance, then pre-train the initial GCN by using training samples and use it to score all subjects. Then, the difference between these scores replaces correlation distance to update similarity. Last, we devise a calibration mechanism to fuse functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI) information into edges. The proposed method is tested on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. Experimental results demonstrate that our proposed method is useful to predict disease-induced deterioration and superior to other related algorithms, with a mean classification accuracy of 86.83% in our prediction tasks.

Effect of APOE ε4 on multimodal brain connectomic traits: a persistent homology study

Background

Although genetic risk factors and network-level neuroimaging abnormalities have shown effects on cognitive performance and brain atrophy in Alzheimer’s disease (AD), little is understood about how apolipoprotein E (APOE) ε4 allele, the best-known genetic risk for AD, affect brain connectivity before the onset of symptomatic AD. This study aims to investigate APOE ε4 effects on brain connectivity from the perspective of multimodal connectome.

Results

Here, we propose a novel multimodal brain network modeling framework and a network quantification method based on persistent homology for identifying APOE ε4-related network differences. Specifically, we employ sparse representation to integrate multimodal brain network information derived from both the resting state functional magnetic resonance imaging (rs-fMRI) data and the diffusion-weighted magnetic resonance imaging (dw-MRI) data. Moreover, persistent homology is proposed to avoid the ad hoc selection of a specific regularization parameter and to capture valuable brain connectivity patterns from the topological perspective. The experimental results demonstrate that our method outperforms the competing methods, and reasonably yields connectomic patterns specific to APOE ε4 carriers and non-carriers.

Conclusions

We have proposed a multimodal framework that integrates structural and functional connectivity information for constructing a fused brain network with greater discriminative power. Using persistent homology to extract topological features from the fused brain network, our method can effectively identify APOE ε4-related brain connectomic biomarkers.

Fusion of ULS Group Constrained High- and Low-Order Sparse Functional Connectivity Networks for MCI Classification

Functional connectivity networks, derived from resting-state fMRI data, have been found as effective biomarkers for identifying mild cognitive impairment (MCI) from healthy elderly. However, the traditional functional connectivity network is essentially a low-order network with the assumption that the brain activity is static over the entire scanning period, ignoring temporal variations among the correlations derived from brain region pairs. To overcome this limitation, we proposed a new type of sparse functional connectivity network to precisely describe the relationship of temporal correlations among brain regions. Specifically, instead of using the simple pairwise Pearson's correlation coefficient as connectivity, we first estimate the temporal low-order functional connectivity for each region pair based on an ULS Group constrained-UOLS regression algorithm, where a combination of ultra-least squares (ULS) criterion with a Group constrained topology structure detection algorithm is applied to detect the topology of functional connectivity networks, aided by an Ultra-Orthogonal Least Squares (UOLS) algorithm to estimate connectivity strength. Compared to the classical least squares criterion which only measures the discrepancy between the observed signals and the model prediction function, the ULS criterion takes into consideration the discrepancy between the weak derivatives of the observed signals and the model prediction function and thus avoids the overfitting problem. By using a similar approach, we then estimate the high-order functional connectivity from the low-order connectivity to characterize signal flows among the brain regions. We finally fuse the low-order and the high-order networks using two decision trees for MCI classification. Experimental results demonstrate the effectiveness of the proposed method on MCI classification.

Artificial intelligence for brain diseases: A systematic review

Artificial intelligence (AI) is a major branch of computer science that is fruitfully used for analyzing complex medical data and extracting meaningful relationships in datasets, for several clinical aims. Specifically, in the brain care domain, several innovative approaches have achieved remarkable results and open new perspectives in terms of diagnosis, planning, and outcome prediction. In this work, we present an overview of different artificial intelligent techniques used in the brain care domain, along with a review of important clinical applications. A systematic and careful literature search in major databases such as Pubmed, Scopus, and Web of Science was carried out using "artificial intelligence" and "brain" as main keywords. Further references were integrated by cross-referencing from key articles. 155 studies out of 2696 were identified, which actually made use of AI algorithms for different purposes (diagnosis, surgical treatment, intra-operative assistance, and postoperative assessment). Artificial neural networks have risen to prominent positions among the most widely used analytical tools. Classic machine learning approaches such as support vector machine and random forest are still widely used. Task-specific algorithms are designed for solving specific problems. Brain images are one of the most used data types. AI has the possibility to improve clinicians' decision-making ability in neuroscience applications. However, major issues still need to be addressed for a better practical use of AI in the brain. To this aim, it is important to both gather comprehensive data and build explainable AI algorithms.

Predicting Coronary Atherosclerotic Heart Disease: An Extreme Learning Machine with Improved Salp Swarm Algorithm

To provide an available diagnostic model for diagnosing coronary atherosclerotic heart disease to provide an auxiliary function for doctors, we proposed a new evolutionary classification model in this paper. The core of the prediction model is a kernel extreme learning machine (KELM) optimized by an improved salp swarm algorithm (SSA). To get a better subset of parameters and features, the space transformation mechanism is introduced in the optimization core to improve SSA for obtaining an optimal KELM model. The KELM model for the diagnosis of coronary atherosclerotic heart disease (STSSA-KELM) is developed based on the optimal parameters and a subset of features. In the experiment, STSSA-KELM is compared with some widely adopted machine learning methods (MLM) in coronary atherosclerotic heart disease prediction. The experimental results show that STSSA-KELM can realize excellent classification performance and more robust stability under four indications. We also compare the convergence of STSSA-KELM with other MLM; the STSSA-KELM model has demonstrated a higher classification performance. Therefore, the STSSA-KELM model can effectively help doctors to diagnose coronary heart disease.